A gentle reminder that by the time this issue drops into your Twitter hole or your inbox, the old Heroku version of our beloved egg timer will have been turned off. Michael’s badly battered purse being close on twenty pounds a month heavier as a result. Thanks to our Jianhan, the new home of the egg timer is:
Please update links and bookmarks accordingly.
If you’re a fan of standing orders - and, let’s face it, if you’re reading this, you might well be - boss ‘brarian Anya has been busy setting up a show and tell covering our work with the ParlRules people and Shedcode James. If you’re around Westminster on the 16th November - or just want to join in pixels - please do get in touch.
Once again, we’ve skipped a week. Jumping straight into the action - or in medias res as Young Robert might say - Friday of week 42 saw yet another Tothill Street whiteboard filled with scribblings, as we attempted to map our election model to the data model under the hood of Democracy Club. Librarian Anya, computational ‘expert’ Michael and actual computational expert Louie were joined by Sym, who we haven’t bumped into outside of pixels since pre-lockdown days. And very pleasant it all was too. Having exhausted yet another pack of dry wipe markers, a tour of our Palace was taken, culminating in a slap up school dinner down the parliamentary caf. Sadly, Young Robert was not able to join being, once more, on vacation.
In terms of model matching, our dear reader will be delighted to learn that although our labels don’t always match, the shape of the things seemed to fit perfectly. Quite some relief. Models mapped, talk turned to identifiers. Part of the Democracy Club contract with the Library already covered the addition of MNIS Member IDs to their candidates where applicable - that is, candidates who are current Members and candidates who once were Members. A problem hopefully made easier by the presence of Wikidata people IDs in Democracy Club data and of Rush, MNIS and Democracy Club IDs in Wikidata. Thanks, as ever, Andrew. Truly a one man digital infrastructure hero.
That said, more identifier alignment is required. The first problem is political parties, both systems having a slew of identifiers but, sadly, none in common. Democracy Club - somewhat sensibly you might think - chose not to mint new IDs but instead use those supplied by the Electoral Commission. The casual reader might wonder why we don’t just add Electoral Commission IDs to the MNIS schema. But then the casual reader might not realise quite how difficult it can be to get changes made in MNIS. So, somewhere, some mapping has to happen between MNIS party IDs and Electoral Commission IDs. Given the state of party information in MNIS is not what one might term ideal, there’s a fair bit of tidying by team:Phil required before that mapping work makes much sense.
The third and final identifier mapping is from our MNIS constituency IDs to Democracy Club constituency IDs. A problem made much easier to solve by the presence of ONS codes in both systems. And a problem made much harder to solve by the expectation that we’ll see boundary changes between now and the next general election. Whilst we believe new ONS constituency IDs are minted well in advance of an election, sadly they’re not published in a particularly timely fashion. A chat between the House of Commons Library, Democracy Club, the Electoral Commission, the Ordnance Survey and the ONS may well be useful here.
By way of background, back in the olden days - when Oli still resided in the parish of Westminster - he built and maintained a general election database. For little more than the price of a pizza, every time a general election came round, assorted librarians and Hansard reporters would fire up Oli’s database and add information in three stages:
the candidates and their parties
the results in terms of winners and losers
the results in terms of actual numbers
That last bit pushing up the Library’s telephone bill as assorted returning officers were contacted, sometimes repeatedly. Upon Oli’s departure, Young Robert attempted to ‘stand up’ - as we say in these parts - Oli’s database with little to no success. So now we have a new plan. Sym has agreed to add a new grouping of Democracy Club users dedicated to Commons Library employees, with existing Democracy Club functionality allowing said users to ‘bless’ information about results as they see fit. Henceforth, Library researchers will no longer query an internal database to analyse election results, but instead plug directly into Democracy Club. There’s a Wardley Map to be drawn here. Excuse our laziness for not drawing it.
Away from matters psephological, Oli’s old database also used to feed an assortment of downstream users from the facilities folk (who needs an office) to the computational section (who needs an email address) to security (who needs a pass). Which is a different problem someone needs to solve on a different day.
No weeknotes would be complete without a mention of more changes to our procedure model. This week, Librarian Jayne and her computational handmaiden Michael took another look at the step collection bit, screwed up their noses and decided they didn’t like it. That has now been rewritten. With our Jianhan also taking a well deserved vacation, neither Jayne nor Michael wanted him bored on his return, so they’ve also planned out two whole columns of step collection related work for his return. We’re sure he’ll be delighted.
One long anticipated use of step collections would give us the ability to control which steps we want to appear on the statutory instrument and treaty tracking websites, and which ones we don’t. Michael has once more buried his head in our procedure parsing code, expanded a SQL query or two and plugged them into a little more of his endlessly shonky code. With the new step collection model in place, toggling step visibility would appear to be a solved problem. Which is nice.
Still on the subject of Michael’s shonky code, our regular reader will be well aware that changes to sitting days have been having an adverse effect on the sanity of our librarians, over 100 scrutiny end dates needing updating whenever there’s a late breaking change. Last time out, we’d amended the procedure model to include calculation styles, clocks and two different flavours for the number of days to be counted. This time round we have a new Ruby script that looks for dates that might need updating, asks our beloved egg timer what the date should be and updates accordingly. Jayne has done a line by line check of Michael’s work and - questions around his variable naming aside - declared herself happy. One more thing to add to the Jianhan pile.
We know a lot of people have asked, what would a logically and arithmetically mapped legislative reform order procedure look like if a machine tried to draw it. At last we can answer: roughly like this. Do not try to open on your telephone. With that, we are pleased - nae delighted - to announce that all LRO steps and routes are now safely nestled inside the machines. Roughly speaking.
There’s still work to do to patch things up, which we had thought was just fixing mistakes. Turns out it wasn’t. A good couple of hours with Librarian Jayne and Librarian Ayesha turned tidies into something of a refactoring session. The House of Commons procedure determination map is now logically tighter, less verbose, easier to read and one hell of a lot DRYer. It does occasionally feel like we’ve accidentally invented a programming language.
In other LRO news, Librarian Jayne has now added step depths to both maps and data. Which means - once we have the procedure polished and a work package or two to test with - all the steps displayed on the website should be ordered in a fashion which matches both procedural and common sense. Top librarianing Jayne.
A second meeting with colleagues from the Northern Ireland Assembly saw the addition of routes to both petitions of concern and scrutiny by a standing committee added to our NIA legislative consent motion map. It could probably use one last check but we think it’s heading in the right direction and we think it’s close to done. Which means we should hopefully have signed-off LCM maps for all three devolved legislatures at some point in the not too distant future.
Week 41 saw registrars James and Thomas and computational ‘experts’ Young Robert and Michael apply finishing touches to their ruleset-compliant data model for the Register of Members’ Financial Interests. Or at least for the Commons half. Over the past couple of weeks, attentions have turned to how many competency questions that model can answer. And how many it can’t.
In Anya’s absence, we managed to purloin a handful of librarians to index both oral evidence sessions and written evidence submitted to the Standards Committee as part of their Code of Conduct consultation. The same team of crack librarians also indexed the additional questions crowdsourced by Steve. Thanks Librarians Ayesha, Anna and Deanne. User researcher Venyah also leant a hand, indexing questions in the research performed as part of the discovery phase.
We ended up with 218 competency questions, 58 taken from oral evidence, 79 from written evidence, 48 from the crowdsourced material and 41 from the user research. Of the 218, James, Thomas and Michael have marked 73 as duplicates. Of the remaining 145, they’ve compared against the model and marked:
45 the model can answer
26 the model can partially answer
59 the model cannot answer
The remaining 15 being unclear and impossible to categorise. It should be pointed out that some of the questions stray beyond the bounds of the purpose of the register so 100% coverage would never be feasible. Closing the gap between the noes and yeses or - possibly more realistically - the gap between the partials and the yeses will rely on any “gold plating” we can do in the short term. And any ruleset tightening that might happen in the longer term.
As long term publishers of Twitter bots, we’ve always been slightly frustrated by the lack of any mapping between between our government department IDs and their Twitter handles. Librarian Deanna has begun patching that gap with a spreadsheet mapping their names and acronyms to our IDs, Twitter handles, Facebook URLs and even Wikidata IDs. A spreadsheet that may well be useful for other people. Thanks Deanne.
And finally, one of the many and varied duties of team:Phil is managing contact details for Commons Members in MNIS, a mix of fields containing data postal, telephonal and hypertext in nature. For contact details requiring the use of HTTP, this task is made much easier by Phil’s script, which runs over all the links and checks the status of the response, raising amber flags for anything that’s not a 200. Which all works very well, at least for things that aren’t Twitter. In the Twitter case, it returns a 200 for active accounts, accounts yet to exist, suspended accounts and abandoned accounts. Not ideal. Maybe the new owner could arrange HTTP lessons for whoever stays on the payroll. Anyway, we add this only to note that Phil has now trained both Librarian Anna and Librarian Emily in the use of said script. Which means, should Phil fall under a bus - or, more happily, win the lottery - our link break checking won’t disappear with him.
