Hot on the heels of last week’s announcement that the House of Commons Library are on the lookout for a Data Science Lead, this week we’re pleased to announce a new role in the Parliamentary Computational Section. If you’re an acknowledged expert in all things ETL with a good grounding in graph databases, then Open Data Technical Lead may just be the job for you.
The House of Commons Library have long maintained a database dedicated to collecting General Election candidacies and collating candidacy results. For about two weeks every four(ish) years, this is probably the most important database in the building, feeding not only into the procedural systems that are often the focus of our weeknotes but also into our corporate systems. It being important to know who needs an office, who needs an email account, who needs a computer. And who doesn’t. Following Oli’s departure, this database has now made its way to team Data and Search and thoughts have turned to which bits we should build and which bits we should buy. To that end, Tuesday saw librarians Anya, Bryn and Grant and computational experts Ian and Michael joined by Democracy Club’s very own Sym Roe. A rough plan was sketched whereby Democracy Club would provide their platfrom and access to their community, the Library would collaborate with said community to enter and validate candidacies and results, and Data and Search would build any pipelines into parliamentary systems. It would not, after all, make much sense to build a platform that gets used once every four years, when there’s a reliable service that does the job and gets used every day of every week of every year. Please stay tuned for more news.
It’s been quite a while since your regular correspondents last hit the conference circuit. They look back with fondess at memories of Berlin, Oslo and Stockholm, wondering if those days will ever return. A couple of weeks back, things started to look up when Edward approached Anya and Michael with the unmissable offer of a speaking slot at the snappily titled IFLA Library and Research Services for Parliaments Section and IPU Joint Virtual Conference “Parliamentary library & research services – towards an agenda for the next decade”. A title that takes longer to parse than the talks. Delighted to once again be speaking to a global audience, Anya and Michael leapt at the chance. Unfortunately, the opportunity to talk was not accompanied by tickets to fly. So Thursday saw them roll up at our Tothill Street headquarters, practice, panic, practice some more, panic some more, log in an hour too early because time zones, panic, talk, dart in the direction of Guinness and promptly collapse. The slides are here should our dear reader wish to peruse. And if anyone out there knows any conference organisers with a bent toward the procedural pendantic web, we’d be more than delighted to reprise. Especially if the conference is not in Tothill Street.
Over summer recess, librarians Anna and Ayesha did some late spring cleaning of the Rush database. The database holds assorted bits of reference data that were stored as strings and the data management had tended to rely on Prof. Rush corralling a PhD student and hoping for the best. Which led to people having titles like ‘Hon’ and ‘Honourable’ and other variations. Data tided, we wanted to keep it tidy and change strings into things. Luckily money was found to pay James to take the newly tidied data and normalise it into new tables. Which means the History of Parliament Trust are now in possession of one of the very few websites with a page dedicated to Admiral Honourable Sirs. Much more tidying and normalisation to follow.
A week or two back, Table Office Matt was kind enough to pour a small part of his brains over one of our whiteboards. The result being a process flow diagram for House of Lords written questions and answers. Unfortunately, the right hand side of the whiteboard got lost behind a chair and didn’t quite make it to our photograph. The return to Tothill Street afforded Michael the opportunity to finally decant the last bit of ink off the board and into pixels. So our swimlane-style diagram has gained a new ‘late for answer column’ and a handful of boxes and arrows terminating in the departmental naughty step. Thanks again Matt.
Our loyal reader will know by now that the prodder is our attempt to make a low code wrapper for the public MNIS API. In the hope that some HTML and a CSV will be of more use to Library researchers than a screen full of XML. This week it has gained the ability for users to specify the inclusion of any House of Lords memberships and oaths, any House of Commons constituencies represented and any “party” affiliations over time. Again, more to follow.
We have now remapped all our procedures to our new logical and arithmetic model (weeknotes passim and then some). The intention behind this undertaking was to tighten up the logic to an extent that allowed machines to parse the resulting graph and, given what has happened, determine what may, must and can’t happen next. In order to do this, young Robert and Michael had cobbled together some Ruby code to grab all the routes in a procedure, parse them according to the actualisation of business steps and calculate the light cone of procedural possibilities. This week, our Jianhan took over the reins of this work and has been busy rewriting in C#. There is still work to do here. The made affirmative procedure contains 1223, only 404 of which are currently being parsed. More news next week.