Given it’s been over a month since general election night and the numbers have not only been counted but also verified, you’d think we’d be over it by now. After all, the rest of the world appears to have moved on. Unfortunately, for our crack team of librarians, moving on just isn’t an option. There is work to be done, more data to be entered and yet more data to be tidied.
In tidying terms much of our effort has been firmly focussed on former Members, Librarian Anna taking up the reins and removing any and all vestigial contact details from MNIS. More complicated has been the wrangling of data for the newly arrived. Should you ever find yourself successfully standing for a seat in Parliament, within the first couple of weeks you’ll find yourself on the receiving end of a Microsoft form - they get everywhere - requesting the submission of your contact details - both physical addresses and online - and, optionally, your date and place of birth. Librarians Anna and Emily report 196 responses had been received by Thursday of week 28, all of which have been entered into MNIS. Elsewhere, Librarian Phil has been liaising with the House of Commons Journal Office on the matter of preferred names for new Members. The names appearing on ballot papers not always providing a reliable guide to how they wish to be addressed once elected.
Still on the subject of people, we’re also delighted to announce that all incumbencies in government positions have also been filled. Computationally speaking. It not being within our gift to affect anything that’s not ones and zeros. We know our limits. Somewhat closer to home, a number of parliamentary positions now also have digital bottoms in digital seats, Monday of week 28 seeing the election of three new Deputy Speakers, all of whom are now ‘in the system’. As it were.
Over in the world of parties - sadly, not those kind of parties - Librarian Anna has been busy clearing out the Electoral Commission identifiers from the MNIS political party table. These were added when we thought we’d be sourcing all our data from Democracy Club and figured we’d need some way to map from their Electoral Commission-based party identifiers to our own MNIS identifiers. Sadly, for various reasons we won’t go into here, that route was not taken, though the party mapping spreadsheet compiled by Anna and Emily still came in handy for Data Scientist Louie’s triangulation operation. It turned out that adding Electoral Commission identifiers directly to our parties table was a bad idea, some parties having more than one Commission identifier, with different registrations over time and different registrations in Great Britain and Northern Ireland. This part of the data model is now considered dangerous and deprecated, the ever-present manual being updated to reflect this.
Still with parties, and still inside the computers, the Green Party of England and Wales has now parted company from the Green Party of Scotland, the computer bits now reorganised to better reflect reality. Top work Librarian Phil.
On the subject of excellent work by Phil, he’s also made a start compiling MNIS-shaped queries that we suspect will help to answer some fairly common questions in the not too distant. Looking at the list, your regular correspondents are half wondering why this list has been left in noSharepoint, rather than made public. Once Phil returns from his well-earned vacation, we shall endeavour to make that so.
If you’d nursed the hope that we’d reached the end of our general election-themed ramblings, you will be sadly disappointed to hear that developments to our election results website continue. Despite making every effort to verify the 2024 results, a mistake or two crept through, as mistakes so often do. Computational crackerjack Michael has cobbled together yet another script, this one keying off Democracy Club person identifers, to target a given candidate and update assorted fields for that candidate and the election they stood in. So far, the script has been run twice, once to correct electorate figures in both Bexleyheath and Crayford and Old Bexley and Sidcup, and once to reverse a couple of candidates who’d unexpectedly found themselves upside down, once again in Old Bexley and Sidcup. On both occasions, Michael’s script ran successfully on first attempt, which must count as something of a first. If you’re interested in keeping track of changes, Statistician Carl has been compiling a record of dataset changes, available as an Excel document from the 2024 general election briefing page.
Michael’s script-writing adventures did not stop there, however. Librarian Emily went back over all unsuccessful candidates across the 2010, 2015, 2017 and 2019 general elections, identifying any who’d found subsequent success in 2024. For obvious reasons, those people did not have MNIS Member identifiers when those spreadsheets were first compiled. Emily has seen to it that they do now. Once again, Michael’s MNIS identifier updating script ran first time and with no errors, marking what we hope will be something of a sea change in his computational fortunes.
As our psephology database continues to grow, we now have two general elections fought - in whole or in part - on new boundaries; 2010 seeing changes in England, Wales and Northern Ireland, 2024 seeing changes across all four nations. Wherever an election is fought on new boundaries, vote change figures and hold / gain messaging are based on notional general election results. For this reason, yet more code was written to flag such calculations to our interested reader and scattered liberally over any and all pertinent pages. Hexmap pages being just one example.
As part of their general election planning, the House of Commons Library issued updated guidance on the use of colours to indicate parties. New colours were added to both the psephology website and to MNIS. Now MNIS is - pretty much, give or take - our ‘single-source of truth’ on all matters party political in nature across both Houses. A revised colour scheme for the DUP and the inevitable data flow from MNIS into the House of Lords attendance system caused some confusion for our red carpet colleagues. To deal with this, a new revised version of the revised colour has now been issued. In psephology terms, both CSS and all six hexmaps have been updated accordingly.
Updates have also been made to both templates and CSS, which mean our previous and next links now render better on mobile telephones and other small screens, and our table heads finally render correctly. That said, since the introduction of horizontal scrolling of tables, we appear to have lost our sticky headers. Young Robert has investigated but, at the time of typing, it does not appear possible to have both horizontal scrolling and sticky headers. If you know differently, please do get in touch.
Getting the psephology website over the redesign line in time for the general election was something of a rush, Young Robert and Michael finding themselves mired in template changes for a good fortnight. When the clock struck for go live, the website looked, well, not perhaps swanlike, but doing a reasonable impression of a well-dressed mallard. Under the waterline, things were less good. The hastily assembled crumb-trail, for example, being composed of a series of concatenations resulting in one MASSIVE STRING. An inelegant solution and a bit of a bugger to maintain. That has happily now been rewritten - or refactored, as Young Robert would no doubt say - the new implementation instead constructing the crumb as an array of hashes. Much nicer. Website wise, you should see very little difference, though, if you’re a screen reader user, we hope the addition of aria labels makes some difference to your experience. The change to an array-based implementation also means we’re able to follow schema.org best practice and encode the crumb as inline JSON-LD. Will Google et al do anything with this? Dear reader, as in all such search engine matters, your guess is as good as ours. We assume someone at Google knows the truth, but even that may be an assumption too far.
If, like us, you find pointing and clicking on websites a primitive practice and prefer your interactions more SQL-based, we also have good news for you. Following a request from Democracy Club-related jonathanf, we’ve upped the maximum row count returned by our Datasette offering to 5000. Which we hope will be more than enough. At the request of our Tony, we’ve also added an explicit link to a Datasette Lite endpoint from our psephology homepage, for those who prefer to query whilst offline. Lovely.
Plenty of progress has been made over in new, old search land, Developer Jon reporting that he’s now implemented everything set out in our result item specification. Obviously, we trust Jon implicitly, but if work is done, then work must be tested. To that end, Librarian Jayne, is currently ploughing through result item listings, checking everything is shipshape. In a break with tradition, we’ve decided not to do this on an object type by object type basis, but instead on an attribute by attribute basis. Given the templating is common, this feels altogether more sensible. So far, Jayne has been able to check and confirm that the display of answers, answering bodies, who an early day motion amendment was tabled by and when it was tabled, all pass muster. Splendid.
Jon has also made a couple of changes to our item pages, ensuring that related items are always accompanied by their type and that deposited papers available only in hardcopy now say as much.
In search facet news, Jon has also responded to librarian feedback, reinstating the legislative stage box, which had gone strangely missing. Following on from what can only be described as a ‘facilitated workshop’ - as ever, to the tune of Industrial Estate - we think we’ve finally nailed down the expected search facet behaviour and its interplay with other page elements. As least we hope so. It all means poor Jon has more work on his hands. Happily, much like us, he appears to also believe that ‘design’ is emergent from implementation and never gets too cross when we look at something and change our minds. Sensible lad.
Should we ever get past the result item and facet stage - and we might be kidding ourselves, but think we’re getting close - the next item on the agenda is a thing our crack team of librarians call ‘power search’. A way of building search queries that provides aliases for grouping behaviour in Solr. As of last week, we did not know where those aliases lived. Now, thanks as ever to our Jianhan, we do. Librarian Anya’s next job is to read through 632 lines of XML and try to work out what’s still useful and what could be usefully left out.
Back behind the scenes, Librarian Ned has been combing through the indexer notes in our triplestore, weeding out unnecessary noise and moving anything useful to people performing searches across to the scope notes field. Where searchers can see them. Notes to self being useful, notes to searchers being more useful. Much more useful.
Librarians Anya, Claire and Silver continue to meet on a weekly basis with computational second grip Michael in some attempt to liberate our taxonomy from its self-imposed software constraints. Progress was definitely being made, whole limbs of the taxonomy being visible to librarian eyes in our Data Graphs implementation of OaSIS. [Weeknotes passim.] It was at this point that we came to the conclusion that ‘whole limbs’ was a little much, there being some fields where we’d prefer to see something a little more branch-like. Or even twig-like to be honest. Some time was spent determining how we could use term attributes to further split limbs into branches and twigs and a plan was hatched. The, erm, ‘target model’ has now been redrawn and we remain hopeful that, with slightly more intelligent code, we can start to pipe through the taxonomy in a more granular and more useful fashion.
Our beloved egg timer has seen a couple of recent improvements, though, both being oversights from previous work, not really something to brag about. First off the default day count for the approval of a super-affirmative legislative reform order has been updated. It had been set - incorrectly - at 40 days. It should, of course, have been 60 days. As any fule kno. It now is.
Second off, when we added calculations for enhanced affirmatives laid under the Investigatory Powers Act 2016, we quite forgot to add details to our code comments. Most sloppy and most unlike us. Comments on the appropriate calculation have now been added. In markdown, of course. Which means our comment parser code has picked them up and spat them out in a format suitable for even the most code-shy eyes amongst us.
Finally, we continue to get out and about, Anya and Michael journeying as far as Portcullis House for the Study of Parliament Group’s Annual Michael Ryle Memorial Lecture. This year in the form of a valedictory, from Legislation Office Liam. And delightful it was too. We had no idea our Liam could speak that many languages. My word. He’s been all over the place.
The evening was made more delightful still when Michael got his first opportunity to thank ex-Commons Clerk of the Journals Mark for coming to his rescue some time back over a small misunderstanding that was deeply upsetting at the time, but that we will not go into here. Thanks Mark. Smiley face.
