We start, for once, with what Young Robert would inevitably call “the visible end of our procedure value chain”, and a feature request from a rather unusual source. Following a conversation on important procedural matters - what else but the definition of a sitting day - Lord Pack asked if it would be possible to add links from statutory instruments on legislation.gov.uk to the same instrument going through Parliament. We’re fairly certain we could provide information to our colleagues at The National Archives to enable such links, in whatever serialisation they’re happiest to consume. But we’re aware that our John probably has enough on his plate right now. What with the one billion hits a month he’s trying to satisfy the magic sand lads with.
Luckily, we have our Procedure Browsable Space™ bookmarklet code, which we’d set up to allow for navigation between a public Act and any legislation enabled by that Act which happens to be currently before Parliament. That code has now been expanded to cope with statutory instruments. Should our dear reader be armed with said bookmarklet, they can now navigate to a page for a statutory instrument on legislation.gov.uk, click the magical bookmarklet, and get taken posthaste to the same instrument in its parliamentary procedural context. This has been tested for made statutory instruments, draft statutory instruments, and Northern Ireland Statutory Rules. Generally speaking, the bookmarklet works for instruments which have been laid before Parliament and which are subject to parliamentary procedure. There are a couple of gnarly caveats which are described on the bookmarklet bookmarklet page.
Given this is the first feature that we’ve ever implemented following a request from a Peer of the Realm, Librarian Jayne and her hypertext-obsessed-comrade-in-arms Michael are keen to see it tested and working. Why not grab your own copy of the bookmarklet, take a trip around legislation.gov.uk, and let us know if it breaks in any way. Also, let us know if there are other tricks we’ve missed. You do not need to be a Peer of the Realm to raise feature requests.
Thanks are due to James and Jon - our resident Sons of Zebedee - who kindly stepped in to give Michael a helping hand with the JavaScript bits and bobs. And for advising, quite firmly, that he really should rewrite most of it in Ruby. Thanks also to Dan, who, in the absence of Jayne, fired up his Claude code, fed it a skill or two, and came back with the SPARQL query needed to make the lookup work.
Additional thanks are due to Dan for his welcome intervention at the less visible end of the old value chain. He’s been called in by Librarian Jayne to help out with a long-standing issue pertaining to a preponderance of blasted “smart quotes”. The Microsoft productivity suite cannot see a pair of quotation marks without making them curly. When papers are laid with the Journal Offices, their details often arrive with Microsoft productivity suite enhancements, which then get copied and pasted into the Papers Laid application, from where they continue their journey, polluting every system they meet. Librarian Jayne has explained this problem to each incumbent in the Clerk of the Papers role, but a) this role rotates regularly, and b) it’s a silly thing to have to remember.
This is a problem as our systems have been taught to expect standards-compliant UTF-8, and “smart quotes” do not fit that definition. Happily, we had Dan to call on, who once more fired up Claude, once more gave it a skill or two, and once more churned out a SPARQL query to help. This one reporting all instrument titles on which “smart quotes” remain present. Regular tidying to commence shortly.
Having broken the back of query expansion - weeknotes passim - Developer Jon has been busy issuing tidies large and small to our new, old Parliamentary Search. Some of those tidies relate to query expansion, others not.
On the query expansion side of things, should a search result in a single word being sent off to our search service, it is now no longer wrapped in quotation marks. It should be noted at this point that single word searches don’t yet always return the same results as production search. Which may be related to which properties are being queried. There is, of course, a card for that. Secondly, should one find oneself on an object page and click on a link from either a subject or an item of legislation, the resulting search query will now be an exact string match on the appropriate field, rather than being chopped up into its constituent parts.
Over on our object pages, the presentation of early day motions has undergone a bit of a redesign necessitated by the slightly weird way in which our Solr search service stores amendments. Unfortunately, we’ve had to remove some information from display. Should an EDM find itself in receipt of more than one amendment, the details of those amendments - their numbers, ordinality, date and tabling Member - are each stored in a single bag of data. Which makes it impossible to associate which amendment has which member on what date. For that reason, Jon has stripped back the display of EDMs with more than one amendment to show the amendment text only.
Elsewhere, our ‘meta’ pages now feature a snazzy little navigation widget and our coverage page has been fettled by both Jon and Young Robert, finally matching Librarian Anya’s exacting requirements.
Stepping out of the drawing room for a second, we find ourselves treading down the service stairs, and entering the scullery. There to chance upon our Jianhan, soap suds up to his elbows as he attempts to bring a shine to our aged thesaurus software. Without which none of Developer Jon’s query expansion work would, well, work.
For reasons we’ve already gone into, this particular upgrade involves a fundamental change to the underlying data model, which was Zthes and will, in future, be SKOS-based. Because of the number of dependent downstream systems, a direct swap of what was to what will be proved too stingy a nettle to grasp. Sidestepping a mass rebuild of everything, Jianhan has instead created an emulator to map old-style requests to new-style requests and respond appropriately.
As of this week, Jianhan has put in place a regular migration to take thesaurus data from its old Zthes home to the upgraded SKOS-based application - thus helping the team with the small business change mountain they have to climb. He’s also set up a test environment which so far contains copies of the thesaurus management software, the emulator, old / live Parliamentary Search, Search Material and the indexing service. All of which await the keen eyeballs of Librarian Anna for testing purposes.
Moving on to matters psephological, we had not one, not two, but three by-elections to cope with. Happily all on the same day. Following some trial, and indeed some error, all three are now live on our election results websites. And also in our Datasette instance, for those more comfortable with querying than pixel clicking.
Louie and Michael have also agreed a first draft of the specification for the CSV files we hope to see our new Elections Manager - née Candidates Database Tool - churn out. One job for the next quarter being a rewrite of our general election import scripts to cope with this slightly new shape and the not so new requirement for staged publishing - more weeknotes passim.
Over in Member Management corner, Librarians Deanne, Harry and Phil have had quite a lot on their plate. The three by-elections, obviously, but also yet another government reshuffle threw a fair number of cards up in the air. All now neatly caught and shuffled. Until next time.
In other news, we’re delighted to announce that the countries table in the Members’ Names Information System is now considered both neat and tidy. Back before Librarians Anna and Emily crossed the aisle, a sizeable amount of work had taken place to make our country list a lot less crazy. As part of that work, any and all Members having the United Kingdom recorded as a birthplace were shunted into England, Northern Ireland, Scotland or Wales, as appropriate. Unfortunately, that work became blocked when we stumbled into a number of ghost profiles in the system, some marked as having the UK as country of birth. Ghost profiles being partial records of Members who have never, in fact, been Members. Colleagues over in the Parliamentary Computational Section have now detached the ghosts in the system from the countries they were never born in, and the United Kingdom is finally scrubbed off our list.
Our regular reader will be well aware that our data piping is about as reliable as Thames Water, often springing leaks at the most inopportune of moments. Of all the things that can and do go wrong, parsing of the Hansard XML is probably our biggest problem. We’re not about to list all the XML parsing mishaps, that would take far too long. But amongst the burst pipe problems, we sometimes see Member contributions go missing, occasionally even whole proceedings. On other occasions, both proceeding and contributions make their way into our triplestore, but, for some reason, get detached. Meaning we have orphan contributions that everyone knows formed part of a proceeding, except the poor computers.
Firing up her instance of Workbench, Librarian Jayne cobbled together a SPARQL query identifying most, if not all, of our orphaned contributions across the 12 most recent sessions. Which Librarian Martin has now worked his way through, amending descriptive data where necessary and raising calls with the Computational Section to reunite orphan contributions with their parent proceedings. In turn, the Computational Section are currently working their way through that list. Godspeed Martin, godspeed Computational Section.