We have reached the end of the fourth week of the Year of our Lord 2026. Or the third working week of Q4, if that’s how you choose to mark time. Week one passed in the fug of a mince-pies-comedown, our crack team of librarians wiping the sleep from their eyes to deal with the inevitable post-recess deluge of tabled written questions. Week two whipped by in a blur of days long planning sessions and conference attendances. By week three, our librarians were fully back at their wheels, purring along like a fleet of Italian V12s, whilst our ‘expert’ - if slightly aged - computational crew dusted off their crank handles, engines reluctantly producing notes more reminiscent of a misfiring Massey Ferguson. Normal working life resumed them.
We should kick things off with a cheery wave to our dear reader. We’ve missed you and we dare say you’ve missed us. Rest assured, normal service is now resumed; we’re already missing our weekly target of writing to you weekly. Please strap yourself in for another rollercoaster ride of all things parliamentary procedure and semantic web related.
Whilst we’re delighted to welcome back our dear reader, our very warmest welcome back goes to Developer Jon, who, following a nine month absence, has rejoined the team. This time as an actual employee. Brave lad. Our dear reader will undoubtedly remember that our Jon was a contractor, churning out both tests and code for our new Parliamentary Search application. The end of that contract brought our development efforts to a premature pause. Now Jon’s back in the fold, Search work has resumed. So let’s start there.
With Jon’s feet back under the desk, there were three important tasks to get stuck into: get a working laptop, refamiliarise himself with the code he last set eyes on nine months ago, and upgrade all the dependencies for the application. Excellent progress has been made on the last two line items, Jon taking the opportunity to get back up to speed on the codebase by writing some more tests. And what better way? Always about the value add is our Jon. Progress on the first item has been rather less rewarding, the fan on Jon’s new laptop sounding a little like Vietnam, when Vietnam sounded like helicopters. It did have administrator rights, but for about an hour. A matter we continue to chase. Happily, Jon had already popped down to the Apple store, so it’s not as if he’s without what Young Robert would probably call ‘functioning compute’.
Apple Macintosh in hand, Jon has already squished one rather large bug whereby clicking on a link to a thing defined in our taxonomy failed. We can’t show you that quite yet; firstly, because it’s not yet deployed and secondly, because we’re in the midst of changing where it’s deployed to. Stay tuned.
As a result of our planning efforts, we now have an actual plan. By the last working day of March we aim to go live with what Young Robert might call a ‘soft-launch pre-beta’. In order to pull that off, Product Owner - and crack Librarian - Anya has decreed that it must:
be resplendent in the Parliamentary Design System
support query expansion (making use of our thesuarus to expand search queries)
have error logging in place
be served from the parliamentary domain, behind Cloudflare
be taking its content and taxonomy data from a fully populated and up to date Open Data Platform
Of all of these, the last decree is the highest hurdle, our Open Data Platform being currently bereft of data beyond the occasional dump and a testing spike. But fear not. Our Jianhan has saddled up his donkey and come riding to the rescue. Or so we hope. So let’s go there next.
Initial efforts to populate the Open Data Platform concentrated on replicating the data from a fully-stocked triplestore on the inside of the parliamentary firewall, to an empty one on the open internet. But that didn’t quite abide by our ‘no triple left behind’ mantra. Jianhan’s new plan is to do away completely with the internal version and instead coax the four components that talk to the internal triplestore into a meaningful dialogue with the external one instead. Which means we now have a new copy of our Poller application taking data from the ODP triplestore and popping it into Solr for the purposes of Search. It also means updates to the three applications which currently POST data internally, to POST externally instead. Those three applications being:
Thresher - an application which takes data from line of business systems (Hansard, written questions, written statements, laid papers and so on) and reshapes it with a touch of business logic and then writes it to the triplestore. Think the T and the L of an ETL and you wouldn’t be far wrong.
The Indexing application - used by our crack team of librarians to subject index, procedurally index and interlink the aforementioned business data.
ORefIS - an application charged with keeping the taxonomy in the triplestore in sync with the taxonomy in our taxonomy management service.
In order to prove all of this works, we have three next steps: checking that all the data is propagating to all the right places, load testing the bits under stress, and acceptance testing by our crack team of librarians. To that end, SQL Neurotic Rachel is hard at work putting together plans for both data flow testing and load testing. The Thresher part of the setup has already survived the load testing part, running happily from mid-November and even surviving the post-Christmas written question deluge. The ORefIS part runs overnight, or when team:Thesaurus presses a button and team:Thesaurus press that button rarely. So again, no real load testing required.
As for the Indexing application - last Thursday, Rachel brought together a hand-assembled team of computational experts to give it a damned good kicking. A kicking that we’re happily informed it survived with no visible bruising. So that’s good. We’re also told that performance was particularly pleasing, much improved since our Jianhan moved the test indexing application to the same Azure region as the subscription it was attempting to talk to. Physical proximity makes quite a difference, even on the moderne internet. Next Monday, our crack team of librarians will step on stage for a spot of user acceptance testing of the Indexing application. Should that not end in tears, we may well be ready to go live in the not too distant. Phew.
Way back when we set out on our procedure mapping adventures - 2017 in fact - we had a notion of procedural clocks, but nothing in the procedure model to describe them. Which meant, when it came to adding applicable clocks to work packages, our crack team of librarians had no model to populate. And librarians with no model to populate make for very sad librarians. As I think we all know.
At some point between then and now - we forget quite when - procedural clocks finally made their way from whiteboard to procedure model and from the procedure model to software. Since then our busy bee ‘brarians have been beavering away, populating clocks for all subsequent work packages where clocks prove pertinent. Which left us with an unfortunate gap for older instruments. Never being one to keep a corner undusted, Librarian Emma set to work through our extensive back catalogue adding clock data to work packages for all flavours of instrument. That work is now considered complete, clocks being in place for every work package since 2017 where time is a factor. Sterling effort, Librarian Emma.
As a side effect of that work, Emma also found a number of older instruments laid under the Sanctions and Anti-Money Laundering Act 2018, all of which follow the made affirmative procedure with a slight wiggle from normality. If any made affirmative can ever be described as ‘normal’.
As any fule kno, the clock for an instrument subject to the made affirmative procedure usually starts ticking when the minister takes out their pen and signs the instrument into law. In the case of instruments laid under the Sanctions and Anti-Money Laundering Act, however, making of the instrument may happen and a clock does not start ticking. Instead a 60 day clock starts only when the Minister signs the commencement order into law. What a palaver.
This left Librarian Jayne with something of a dilemma. A dilemma she solved by adding a new specialised making step for certain instruments laid under the 2018 Act. A step you can now feast your eyes on in our shiny new Procedure Browsable Space™. She also added a new specialised step for the signing of a commencement order. A step you can also see in our shiny new Procedure Browsable Space™. And then hooked up the latter step to a brand new approval period clock. Which, as you’d probably expect by this point, you can also see in our shiny new Procedure Browsable Space™. Finally actualising business items across an assortment of work packages with the newly added steps, where appropriate. Please never say we do not go the extra mile with our procedural pedantry.
Sticking with our shiny new Procedure Browsable Space™, Michael’s also been making tiny changes behind the scenes. Should someone mistype an identifier - or if some magic sand lad’s web crawler requests a thing with an identifier that doesn’t exist; which is a thing that happens fairly frequently these days - the website now returns a 404, and doesn’t crash with a 500. We hope the crawlers interpret this to mean, “please don’t visit again, at least for a wee while”. Although, as the crawlers appear to hallucinate entire URL structures, gobbling up text much as Pac-Man chomped through pellets, we suspect this hope may be in vain.
A new feature for our shiny new Procedure Browsable Space™ came in the form of a legislation lookup. This was enabled by Librarian Jayne’s herculean efforts some months back, to tidy our acts of Parliament data. She ensured that every act with a legislation.gov.uk identifier URI has that URI stored in our store. Which means, one SPARQL query later, we can look for an act by its legislation.gov.uk identifier URI and, if we find such an act, respond with a redirect to a list of instruments currently before Parliament that are enabled by that act. If any. A feature applauded by both Alice and John. Let’s be honest, in our line of work, praise does not come much higher.
For the non-information resource URI initiated, the lookup pattern looks like this:
https://api.parliament.uk/procedure-browser/enabling-legislation/lookup?legislation-gov-uk-uri=https://www.legislation.gov.uk/id/ukpga/2011/20
That as a link to click here.
For those less comfortable tinkering about at that level, there’s a handy bookmarklet. Just drag it to your browser toolbar, browse to any page under any act on legislation.gov.uk, click the bookmarklet, and you’ll arrive posthaste at a list of instruments currently before Parliament enabled by that act.
In procedure adjacent news, Shedcode James has been head down in our beloved Egg Timer™ code. He’s added a full suite of tests for both scrutiny end date calculations and scrutiny start date calculations. So the next time we tinker with it, Librarian Jayne won’t need to waste half an afternoon on testing duties. A fairly comprehensive refactor of our beloved Egg Timer™’s calculation code lurks just over the horizon, so the presence of tests is a relief to all of us. But mostly to Jayne.
Team Anna, Susannah, Ned, Phil and Silver continue to make excellent progress on our Library Knowledge Base™ application. This, as our dear reader will know, is the first application to make use of the hierarchical organisation of our taxonomy. All other applications treating it more like a bag of words. A bag of beautifully planted, well-weeded words, but a bag of words nonetheless. Now that it’s finally found the opportunity to stretch its gazelle-like legs, Librarian Ned has been out with his hoe and hand-trowel, ripping out more plants in wrong places. In this case removing what have come to be known as ‘scaffolding terms’ - these are ‘concepts’ in the taxonomic hierarchy intended more for librarian organisation than for subject indexing. For fairly obvious reasons, simply removing the scaffolding would have brought about disaster. Prematurely removing scaffolding does tend to do that. So Ned’s been more than busy, digging up concepts which were under scaffolding terms and replanting them in the shade of more appropriate conceptual homes, reindexing content as appropriate. Proper bit of librarianship that is.
Our regular reader will recall that the Library’s senior management upped expectations for the publishing of result data come the next general election. Last time out, nothing was published until all the results had been verified. About two weeks after polling closed and most of the world had lost interest. Come the next general election, we’re aiming for a more staged approach to publishing. In order to do that, we first need to ensure that loading the data is faster than it was. And in order to do that we need to strip out some of the denormalised data. The denormalised data made election result pages easier to build, but also made the database harder to populate. To that end, SQL Neurotic Rachel has been stripping out columns and whole tables, replacing them with SQL queries far beyond the abilities of Michael.
All was going well until, shortly before Christmas, Rachel slammed shut her laptop, packed her suitcase and took a well-deserved trip to Japan. It had been agreed with Michael that he wouldn’t attempt to do the work in her absence. But then Michael got bored, as Michael so often does. And he made a ham-fisted attempt to remove cumulative counts from the general elections table. Pleased with his work, he confidently informed both team and product manager that there was no real need to test. At which point a not unimportant section of the website exploded for about 30 minutes.
Michael has since wiped most of the egg off his face and Rachel has returned. Happily, she’s now done the same work, except this time properly. Thanks Rachel. Never have a whiteboard monkey do a data engineer’s work might be one lesson to learn here. And learned Michael has. Not wanting to repeat the experience of applying a hot fix to a website at a quarter to five on a Friday afternoon whilst bravely battling a crippling bout of man-flu, he spoke to James about how best to add automated tests to our election results website. James spent a wee while pondering before settling on the best option and adding a tiny smattering of tests to the repository. Tests that our Rachel is currently expanding to cover every ‘page’ type across the application. We can only assume she’s planning another holiday.
The only visible change to election results - and indeed Member pages - that we have to report resulted from a Liberal Democrat rebrand, their particular flavour of orange taking a slight turn for the darker. Statistician Carl provided the Library’s interpretation of their new colour, tweaked slightly so it’s distinguishable in a list of all the other parties. Librarian Phil has updated the members website and computational ‘expert’ Michael has updated election results. Not the easiest task for a colour-blind lad, but one he pulled off with aplomb.
In the nine month gap between Developer Jon departing and Developer Jon returning, Anya and Michael found themselves with no Search meetings to attend and time on their hands. Being generous souls, they offered to lend their whiteboarding skills to the Commons Library team charged with looking at how research is published now and how we might want to publish it in the future. The result of all those chats being a straw man model and nothing more than a straw man model. Now, we all know that no straw man model survives contact with real data and real workflows without picking up the odd bump and bruise. For that reason, an exercise was proposed. We’ll take the model, tweak the model, add some data, make some web pages, add more data, tweak the model, adjust the web pages and so on and so forth.
For model making, model tweaking and instance data management, we’ve made a Data Graphs project. For turning that data into clickable pixels, we’ve turned to Shedcode James. As of this week, the Data Graphs project has been created, accounts have been set up, a Slack instance established, and an introduction to Data Graphs session held with the project team. Meanwhile, up in Sheffield, James has been wrapping his head around the Data Graphs API and popping the results into HTML. There’s nothing to show quite yet, but do stay tuned.
A single model shunted its way out of the respray shed, as our depositing ontology expanded to include applications for orders under the Transport and Works Act 1992 together with associated applicants and agents. If nothing else, providing an opportunity for Michael to sneak in a reference to the ‘Transpennine Line between Stalybridge and Diggle (Saddleworth)’.
Over on data tidying duties, Librarian Emma has trawled through four sessions worth of committee reports correcting any attributed to the mis-titled Public Accounts Committee to the more appropriate Committee of Public Accounts. Emma’s also been spring cleaning some elderly bill data, correcting a discrepancy between the House mentioned in the reference field and the House to which the bill had been assigned. That’s another nine sessions’ worth of data now considered both spick and indeed span. Thankless tasks, thanked here.
We would not wish our dear reader to depart thinking it’s all work and no play. Christmas trees chopped and composted, Anya and Michael made their yearly trek to Oxford town for the Study of Parliament Group annual weekend. Librarian Jayne had also intended to attend, but then it snowed. So that put paid to that. Well, it would be unrealistic to expect to travel 59 miles when there’s half an inch of snow on the platform. Anyway, the conference sessions were enjoyed, old faces acknowledged and intelligent sounding conversations shyly avoided by the judicious taking of cigarette breaks. Then Edward turned up. Drink was taken. We’ve never been ones to sneer at tradition.
The other trip to report happily coincided with Shedcode James being down from Sheffield for the day. Following a morning of yet more planning, we boarded the Circle Line to the History of Parliament Trust’s new gaff by the Barbican. Over tea and biscuits we chatted through all the usual topics from election results to constituency data to the Rush database to the Beamish database to standing orders to parliamentary time and historic sitting day data. Thanks for hosting Jennie and Martin. Pleasant as ever.