Here we go then. One last push as we dot the Ts, cross the Is and pop a decorative bow on the year. And what a year it has been. By our count, we’ve managed to churn out 49,808 words on the subjects of parliamentary procedure and computers. Which is quite some word count. It’s not as though this material is easy to work with.
Before we fill the hot water bottle, draw the curtains and tuck 2023 up in bed, a canter through what happened in the last week of our working year. See you on the other side, guys.
It will not have escaped our dear reader’s notice that 2023 has been largely dedicated to updating and upgrading our antiquated search system. All in the hope it might some day match some definition of modernity. Our Jianhan has been particularly busy, upgrading Solr, cloning Poller and writing new redaction code. He also popped the old code atop the new instance of Solr resulting in performance improvements that have left our crack team of librarians quite agog. In our final week of 2023 he picked up yet another loose end, when boss brarian Anya spotted that the barebones CMS which accompanies our search service had not been ported to the new old code. ‘CMS’ is overstating things here, however the thing at least allows our crack team of librarians to control some of the text appearing on the home page, about page and help pages and it was felt we could not go live without it. That problem is now largely fixed, our Jianhan learning a great deal about Azure authentication on route. Top work, as ever, Jianhan.
Over in frontend world, developer Jon is also making magnificent progress. Cards representing object pages for statutory instruments, European deposited documents and Transport and Works Act order applications having made their way through the review column and over into weeknotes. Given that any object may exist in a whole variety of states, a full review of all object pages will be undertaken when our attentions fully return in the new year. Which usually means February.
With a helping hand from Jianhan, Jon has finally solved our BST-based problems. The problems arose because, despite the fact that we only ever really deal in dates, for some unknown reason many of our dates are stored as datetimes. Many of those datetimes displaying a strange and unhelpful affinity for 11pm. Calmly, clearly and with a great deal of patience, Jon has managed to explain to the machines that they are in fact based in London and should therefore be familiar with our strange relationship with time and summertime. We doubt the machines are based in London, but - should you bump into one - please don’t let this slip.
Jon has also solved our pluralisation problem. It’s not usually a problem one encounters in Rails, that language’s understanding of pluralisation and singularisation being really rather good. Even if it does spell them incorrectly. But if there’s one thing Parliament is good at, it’s unexpected pluralisations. Orders in Council, points of order and observations on petitions not being plurals taught in primary schools. We thank Jon for fixing.
Most of the core backend search work now being done, Jianhan’s attention has turned toward our taxonomy management software, improving what can be improved and not getting dragged down into dreams of idealised end states. We are pleased to report that we now have an install of the latest version running and populated. Not only this, we also have two instances of the accompanying API, one reflecting everything in the taxonomy, one a redacted version.
Testing of the new taxonomy management software is also coming on apace. Librarian Susannah convened a meeting of our crack team of taxonomists to sketch out a testing plan. Our Jianhan once more stepped into the breach, enabling concurrent users - and therefore concurrent testers - in the new system. The benefits started to prove themselves the very next day, when a Lords Library researcher got in touch, quite baffled by the results returned in a search for ‘British Museum’. Said search resulted in a great many results, many of which were on the wrong side of random. It turned out the term ‘British Museum’ has a synonym of ‘BM’ in our thesaurus, which might seem fair enough. But BM was being automagically rolled into the wider query, which is less than helpful. The BM synonym has now been expunged from the taxonomy, leading to cleaner and more pertinent search results. Our crack librarians have known for some time that there’s a number of such acronyms knocking about in our 44 year old taxonomy, none of which are really in general usage and none of which help with retrieval. These have hitherto been fixed on find - the problem has been finding them all. A problem that no longer exists, given our new taxonomy software comes complete with querying functionality. Armed with his trusty SPARQL spanner, Librarian Ned fashioned a new query returning all two character synonyms and found 77 of the blighters lurking. All of which have now been cleansed.
PM Lydia and Young Robert - or YB as he now likes to be known - returned from their encounter with the TWG shellshocked, shaking shrapnel from their shoes, but largely ungrazed. We’re informed that their TD search MVP HLD also escaped ungrazed. Quite the feat. Stage TWG behind them, they’re limbering up for the boss level: TDA. Which appears to be largely the same set of people but in a different Teams meeting. This gives us cause to be hopeful that there should not be too many unpleasant surprises. If our two heroes can make it through this, they can make it through anything.
Whilst quietly going about his daily business, perusing Votes and Proceedings, our Librarian Martin chanced upon a number of interesting new treaty things. Four things in fact.
Firstly there was correspondence about treaties, from the Public Accounts Committee, the Home Affairs Committee and the Women and Equalities Committee. Correspondence from the Women and Equalities Committee finding its way to the timeline page for the UK-Gulf Cooperation Council Free Trade Agreement. The other two can be found on the timeline for our treaty of the week. Or, as we like to call it, the ‘Agreement, done at Kigali on 5 December 2023, between the Government of the United Kingdom of Great Britain and Northern Ireland and the Government of the Republic of Rwanda for the Provision of an Asylum Partnership Agreement to Strengthen Shared International Commitments on the Protection of Refugees and Migrants’.
Martin also spotted that the Public Accounts Committee had not only opened an inquiry, but also taken oral evidence on the subject of refugees and Rwanda.
These new steps have been added to our CRaG treaty map and the Rwanda treaty timeline grows by the day. This means we now have two CRaG-related Commons committee maps: one covering the Public Accounts Committee, one covering everything else. The latter is minimal for now, and we expect to see it expand over time. Never one to down tools before the job is fully done, Librarian Jayne has also expanded our deservedly-famed SPARQL query library covering both Commons committee consideration and correspondence published by committees. Would you expect less?
Still on Votes and Proceedings duties and still on the matter of the Rwanda treaty, Martin also spotted our first ever private notice question on the subject of a treaty. Again the step has been added to our CRaG treaty map and can be found on the ever-expanding timeline page for the Rwanda treaty. Again, Librarian Jayne has cobbled together a query. Top work Librarian Martin. Top work Librarian Jayne.
Martin also spotted an oversight in our pre-laying scrutiny map. Back in April 2023, we saw a machinery of government change with the Department for International Trade morphing into the Department for Business and Trade. Had we been on the ball, we would have created a number of new steps to cover the Trade and Agriculture Commission laying - and possibly withdrawing - advice to the Secretary of State for Business and Trade. Unfortunately, we were not on the ball. Indeed it took us a fair few months to catch up with the ball. But, thanks to Martin, catch up we finally did, and that snafu is now fixed. The timeline page for the ‘Protocol on the Accession of the United Kingdom of Great Britain and Northern Ireland to the Comprehensive and Progressive Agreement for Trans-Pacific Partnership’ providing all the proof you’ll ever need. And, yes, it would be nice if one could link to a fragment of such a page but, unfortunately, the website in general and fragment identifiers in particular are outside the control of your regular correspondents. If HTML is too much bother - and we find it often is - once again, there is a SPARQL query at your disposal.
A pleasing degree of progress has been made over in psephologyland. Statistician Carl has published our much improved spreadsheets for the 2015, 2017 and 2019 general elections, as supporting documents to the Commons Library briefings covering those elections. Spreadsheets lovingly polished by Librarians Emily and Anna and spreadsheets that now come complete with consistent column definitions, MNIS identifiers for Members and Electoral Commission identifiers for political parties. A virtuous circle indeed. Please do remember, dear reader, everything published by the House of Commons Library is made available under an Open Parliament Licence, so you are free to take those spreadsheets and make your own psephology website, a nod of acknowledgment the only thing expected in return. Should you be short of a new year project, we can heartily recommend it. It’s all quite fun.
In the course of polishing the spreadsheets, Librarian Emily came across a number of examples of people who had unsuccesfully contested elections who later did eventually find success and became Members of Parliament. Our MNIS database is set up to hold not only successful election attempts by our Members, but also any unsuccessful attempts. Those early unsuccessful attempts are often difficult to track down, and Emily found that a number of these were missing from MNIS. The missing data has now been backfilled and another virtuous circle squared. Top librarianship Emily.
Emily has also been looking into how we manage political party names in our beloved thesaurus, that policy now being brought into line with how we manage party names in MNIS. Which, again thanks to the diligent efforts of Librarians Anna and Emily, is also in line with how the Electoral Commission capture party names. Which in turn is in line with their Democracy Club equivalents. Proof, as if proof were needed, that quality librarianship is always greater than the sum of its parts.
If you tuned in last time out, you’ll know that statistician Carl had combined maps, code and maths to calculate overlaps between current constituency areas and proposed constituency areas. The data generated covers overlaps in geographic areas, residential property and population. We’re not quite sure how he managed to do this but it’s pretty impressive stuff, plugging a huge semantic hole in our election-related offerings. Michael imported Carl’s data and splashed it over our constituency area HTML. If you click the ‘constituency overlap details’ link down the bottom, you’ll see Carl’s workings for that constituency area. And if you click the ‘overlap details for all constituencies’ link down the bottom of that page, you’ll find all Carl’s workings. Wow. Impressive math.
Part of the aim of the psephology exercise is the kicking of tyres of our election-related models, and so Librarian Ned and his computational helpmates, Young Robert and Michael, have once again thrown themselves in the direction of matters ontological. There is now a new and hopefully improved geographic area model and a nascent geographic area overlap model, the latter to capture some of the results of Carl’s sums. A new and improved election model should join the pile in the new year.
In terms of pixels, both Member and party pages have been tinkered with, our general election party performance tables now include vote shares, and anything that can have previous / next navigation, now has previous / next navigation. The latter at the request of boss-boss-brarian Bryn and appears on pages including Parliament periods, general elections, boundary sets and elections in a constituency area in a boundary set. Splendid stuff.
Thanks to the kindness of strangers and much pixel-punching by Librarian Anna, we’re delighted to say our Bluesky written answer bot accounts have this week been joined by:
With 12 new bots aboard, we are - we believe - finally answering body complete across both Mastodon and Bluesky. Twitter being a rather more difficult beast to deal with. At least under its current ownership. If you’re registered on Bluesky or Mastodon, and would like to keep an eye on Parliament keeping an eye on government, the full list of our beloved bots can be found here.
… to say goodbye to you.
So long, farewell, Auf Wiedersehen, good night.
Until next year, dear reader, when we hope to wave again.