Too much work and some particularly severe hangovers - weekends only, never on a working day - once more lead to weeknotes taking on a more monthnotes cadence. Apologies as ever.
Our Constitutional Reform and Governance Act 2010 treaty procedure map has recently expanded to include steps taken in Parliament prior to the laying of a treaty under Part 2 of CRaG. This in order to better to reflect procedural innovations such as statements announcing agreements in principle, statements on the progress of negotiations, statements upon signature and deposits in the Libraries of both Houses of the government’s negotiating objectives. Unfortunately - although the maps and data were easy enough to change - the website code wasn’t, relying as it does on the actualisation of a laying step and leading to ‘laid dates’ for as yet unlaid treaties popping out in pixels as 1/1/0001. Librarian Jayne and computational whizkid Jianhan, pulled out their SPARQL spanners and amended queries appropriately. Those new queries have now been deployed by colleagues in Software Engineering and the wonky date issue on not yet laid treaties is now fixed. At least in staging.
Unfortunately, we’re dealing with computers here. Which is a little like playing whack-a-mole armed only with a miniscule mallet. No sooner does one mole get whacked than another pops up, only to prove equally annoying. For reasons we do not yet pretend to understand, something in the list code appears to have caused panic in the SPARQL query which proceeded to time out. Or not time out, depending on the mood it was in. Investigations continue.
Several weeks after the fact, we continue to play catchup with changes to the machinery of government being reflected in Parliament’s committee structures. In this case a new committee, in the shape of the Business and Trade Committee. That work has been completed to include concerns raised by that committee in the “recommended not to ratify” website filter. Albeit hindered by the resulting list of treaties sometimes turning up. And sometimes not.
Additional work has also happened on the filter “motion to not ratify tabled”, extending it to allow filtering for the tabling of motions of any kind. Which should prove useful. Should we ever get our list of treaties back.
Away from the code and back in the more manageable land of map-making, Librarian Jayne decided the addition of pre-laying business steps to the top bit of the main treaty map was making things a little busy to the point of illegibility. That problem is now solved, with all pre-laying scrutiny steps decanted into their own component map. Said map also includes logic which precludes steps that can only happen before a treaty is laid from happening post-laying. Much neater.
If you tuned in a wee while back, you’ll know that Librarian Jayne was the lucky recipient of one of our rare and much coveted Librarian of the Week Awards. This time for her sterling efforts wrangling Act data. The original data was kindly donated by The National Archives, covering all Acts of Parliament back to 1801. Though sadly it lacked Royal Assent dates. For a country lacking any single source of a constitution, you’d think recording these things would be quite important. But apparently not. Following much fine librarianship and a lot of hard graft and research, the Royal Assent date field is now filled in, the much improved data winging its way back to Kew, to be incorporated into their corpus. And not just that. Jayne’s fine work has also been dispatched to Apostle Andrew who’s kindly taken on the mantle of spreading her words and deeds to the Wikidata community worldwide. Thanks as ever, Andrew.
Unfortunately we are, once again, dealing with computer things here. And, as we all know, computer things have a terrible tendency to go wrong. What was supposed to happen was the new and improved Act data would be loaded into data platform mark 1, from where it would flow without further incident to data platform mark 3. What actually happened was the data was loaded into data platform mark 3 and data platform mark 1. It then proceeded to drip feed from data platform mark 1 to data platform mark 3, spraying duplicates hither and thither. Luckily, Librarian Andrew is not just a very fine apostle, but also a very fine librarian. He’s kindly donated an additional dollop of discretionary labour and is currently helping Librarian Jayne to pick through the resulting mess. A plan to tidy begins to emerge. Thanks as ever Andrew. Why you don’t come and work with us, we’ll never know.
In slightly better news, our Jianhan has reconfigured data platform mark 1 to also search over the shiny new long titles kindly supplied by TNA. Which should improve things immeasurably. At least once we’ve managed to sweep all the duplicates under the carpet.
In our continuing quest to answer Tony Benn’s second question, Librarian Ned has expanded his horizons from Lords Temporal to Lords Spiritual. Building upon work done to reshape and publish the Beamish / Sainty data, Ned’s now compiled a dataset describing established churches, Lord Bishop dioceses and the incumbents of said dioceses. Young Robert and Michael wrapping his efforts in their usual shonky Ruby code. Unfortunately, despite every research effort being made, many of the dates - of diocese name changes, births, deaths and incumbencies are only approximate. So it’s perfectly possible we’ve misattributed a title or two. We’d hoped to repeat our egg timer success and wrap the data in code to calculate parliamentary seniority of diocese incumbents at any one point in time. Even building in a ‘is advanced under the Lords Spiritual (Women) Act 2015’ flag to help with said calculations. Unfortunately, without precise incumbency dates, that currently looks impossible. Work nevertheless continues.
Back over in the elected House, we continue to make preparations for an upcoming general election. Much complicated by the expectation of a boundary change between now and then. We know there are two fields in our constituency table that appear to be designed to decorate boundary changes. Unfortunately, the semantics of said fields are undocumented. At least to the best of our knowledge. ‘Previous constituency ID’ gives every impression of linking from a current constituency to its predecessor, but on the question of whether that’s supposed to point to a preceding constituency with the same name and same geography, or a preceding constituency with the same name but a different geography, or a preceding constituency with a different name but the same geography, we remain clueless. Judging only by the name of the field, ‘created from constituency ID’ would appear to be intended to point from a current constituency to its geographic predecessors. Given it has never been populated and the implied semantics leading one to the natural conclusion of a many-to-many, we can’t see how it would ever have worked. So we’re choosing to ignore that one.
That said, we have flagged the impact on constituency pages of the expected boundary change to our colleagues in the Parliamentary Computational Section. The Maidenhead constituency page being a great example of a current constituency that makes no mention of its preceding incarnation and no mention of Theresa May being the Member for some shape of Maidenhead since 1997.
Still on the subject of boundary changes, we have the additional challenge of if and how we generate vote changes and party held / gain fields, given there will be no previous election to directly compare to. We have had confirmation that a crack team of academics will be compiling notional results for what might have happened if the 2019 general election had been run on the proposed new boundaries. And further confirmation that the Library is not expecting those notional results to be loaded into MNIS. Which comes as some relief. But if anyone’s expecting vote change and gain / hold data to appear on the website, that’s more side-loading work for data scientist Louie. Poor lad.
Elsewhere in election planning land, Librarians Anna, Emily and Phil have been pulling out all the stops tidying our political party data. Parties are now de-duplicated and aligned to Democracy Club by means of an Electoral Commission ID. In the course of that work, our crack team of librarians came across a number of parties that had been entered with a truncated name. Working with historic election data from Democracy Club, Wikipedia and the BBC, those abbreviations have now been expanded.
Meanwhile, Librarian Emily has been turning brain cells to the matter of end reasons. There is a single table listing such things in MNIS, used for both the leaving of Parliament and the leaving of a constituency representation. At some point in the last couple of weeks, Librarians Anya, Emily, Anna and Phil were joined in pixels by Parliament and Constitution experts Richard and Neil. The list of end reasons was drawn out and a new sketch made, all present coming to the conclusion that many of our ‘end reasons’ were not in fact end reasons. One example being ‘standing down’ which is a thing we want to capture but not actually the reason anyone leaves the Commons. That being largely dissolution. We now have a much smaller list to work with and a much better idea of what we need to tidy.
Hot on the heels of their unexpected breakthrough Bill Papers EP, Young Robert and Michael are delighted to announce they’ve been signed to a major label and released their first long player: the Committee Papers LP. Both services were commissioned by Librarian Anya to plug gaps where systems that were supposed to come fully equipped with feeds ran out of time, money and patience before any feeds were provided. Which means our crack team of librarians must rely on frequent checks of Votes and Proceedings and repeat visits to the website to check if anything has been published. At least for the House of Commons. For the House of Lords, they had even less to go on.
Now they have RSS and ICS feeds, we hope their work will become easier. Working on the usual basis that making something useful to one set of people will probably mean it’s of use to some other set of people, if you’re following progress on a particular bill or the oral evidence given to a particular committee, you can also now plug the RSS into your favourite email client and the ICS into your favourite calendar app and keep track of both published documents and upcoming events. You could - should you wish - also subscribe to feeds of all oral evidence, all committee events and all bill papers of a given type. Though - unless you’re a duty librarian - we wouldn’t necessarily recommend this. If there are people in Parliament - or indeed elsewhere - with a need for feeds by some other cut of the data, please do get in touch and we’ll do our best.
Meanwhile, team:Claire have been looking to backfill House of Lords’ oral evidence sessions, a good number of which had been missing from our search service for want of a simple feed. Librarian Emma has already populated all Lords’ oral evidence sessions from 2022 and 2023 and continues to work backwards. Top work Librarian Emma.
Over in more ontological lands, we’re pleased to announce two more models have been shunted onto our done pile. Librarians Anya, Jayne and Claire, together with the computational cabin lads Young Robert and Michael have finishing commenting both the depositing and the delegation model. The latter emerging when our legislation model got a little too large and we decided to split it into an enabling / enabled model and a complementary, FRBR-informed delegated legislation model.
On the subject of FRBR, Jayne, Young Robert and Michael also met with usual clerkly suspects Matt and Mike to chat about swapping out the word ‘expression’ in favour of the word ‘copy’. A conversation we will not trouble you with here. It’s rare we have a whole meeting dedicated to a single word and we do feel we may have may have excelled ourselves. Apologies Matt. Apologies Mike. Useful as ever though.
Given we now have a fair number of models and given we hope to use those models in data platform mark 4 in the not too distant future, boss computational expert Ian suggested Michael might want to stretch their legs a little. To that end, Michael has once more partnered up with Librarians Jayne and Ned to write up some example data for a bill. Librarian Ned took to Workbench and churned out some triples for an example bill, unfortunately in the medium of a csv. That apparently being a known limitation of our version of Workbench. Since then, Jayne and Michael have taken those triples and mapped them to our new models. Or at least all the bits they could manage. They even made a picture. There remain bits of data describing librarian workflow that we have - as yet - not remodelled. And an odd collection of bNodes and a self-referential skos:prefLabel that even our crack team found baffling. More work probably needed.
Thursday last saw a mass outing as Librarians Anya, Jayne and Phil, boss-boss ‘brarian Bryn, data scientist Louie and computational back-markers Young Robert and Michael headed over to sunny Vauxhall. There to be hosted by Full Fact Andy 1, Full Fact Andy 2 and Full Fact Will. Thanks lads.
They were joined in Vauxhall by friends of the family Silver and Wikidata Apostle Andrew; the BBC’s Jeremy, Tom and Duncan; Googler Dan; and the Department of Levelling Up’s very own Paul. The question before the House being, ‘Wikidata: should we map everything we know?’ And a very lovely conversation it was too. Sometime around 4pm, the question was put and the House resolved in favour without division. And promptly headed off to the pub. Again.
Another meeting is half planned for a couple of month’s time, this one with the House of Commons Library hosting. Having established that establishing conceptual equivalence to Wikidata leads to good things, we’d like to move the conversation on to what - if anything - could be built at the intersection of our interests and our data. If you’re in the public sector and like to work with Wikidata, maybe say hello and we’ll see if we can expand the invite list a little.
