Dear reader, we have once again let you down, let our crack team of librarians down and let ourselves down. It has been four weeks without notes. For once, we’re not blaming our own indolence. We’ve actually been quite busy. Partly with play, mostly with work. Let’s start with the work.
The long, hot summer months passed as long, hot summer months tend to do. Rest, relaxation and recuperation punctuated with moments of boredom and the occasional, impatient desire to re-enter the fray. Eventually, the final Sunday evening came around, and with Songs of Praise, Antiques Roadshow and That’s Life disposed of, packed lunches packed and schoolbags dragged out from under the stairs, the existential dread of the last weekend before a new term set in for all concerned.
As our dear reader knows all too well, our crack team of librarians aren’t ones to allow existential dread to stand in their way for long. By the time the Erskine Express had pulled into Westminster Station and our Members had disembarked, Slack channels and Teams calls were already filling up with the chatter of fresh-faced librarians keen to organise some information. If you please.
The first day of term is always a busy time in these parts. Members submit questions for approval by the Table Offices over recess. Should those questions meet Table Office standards they are not considered tabled until the new term starts. Which means our crack team faced a daunting pile of somewhere around 1500 written questions to subject index by tea time. And this is also when our triplestore caught fire.
For reasons we still do not pretend to understand - we’re working on it, we’re working on it - at some point on or before Monday morning, our indexing application - the tool used by our crack team of librarians to subject index, procedurally index and interlink parliamentary material - slowed down. Right down. All the way down. We should note that it’s never been what one might call quick. Or ‘performant’ as the experts might put it. There’s been many occasions on which librarians press save and sit around for five seconds or more, waiting for the computer to get back to them. Five seconds of twelve librarians finger drumming is but a trifle compared to five minutes, 15 minutes or - on occasion - the 45 minutes it was taking to save a record on Monday. Given ~1500 written questions to process and a response time closing in on an hour, we’ll leave the maths to our dear reader.
No librarian is an island, and our librarians have a service agreement with the Table Offices to add, check and publish headings for written questions. And there’s always more work tomorrow. With the deadlines of today and tomorrow and tomorrow in mind, our team of indexers were joined by the back up team, and the back up team to the back up team.
Reader, we did not make it. And if memory serves, for perhaps the first time in around 15 years. Tuesday dawned with Monday’s questions still on our to do list. On Monday we were stuck between the rock and the hard place created by a burning indexing application, a mountain of questions, and deadlines. More users meant sharing the workload, distributing the pain and frustration, but also fanned the flames in our triple store. The more people using the system, the slower the system got. On Tuesday with the back of Monday’s questions - and no doubt of one or two of our indexers - broken, the cry turned from “all hands on deck!” to “just some hands on deck!” Half of our indexing team were forcibly retired from indexing duties, whilst the remainder battled on, with smoke in their eyes and noses.
On the computational side of the aisle, all the usual fixes were tried. Things were turned off. And back on again. The web server was upgraded. The memory allocation was upped. The latter course of action being the only thing tried so far that has made any, albeit minimal, impact. By Thursday lunchtime, over the roar of the flames, Anya declared “We can’t go on like this”, and convened an emergency meeting of technical experts. Our Jianhan, Nishanth, Sadia and Ian pulled themselves clear of the triplestore wreckage to join us in pixels. Faces only slightly sooty, hair only slightly singed. We have a list of actions and owners of actions. But - as we scribble these notes - we’re still seeing response times measured in minutes, not seconds.
Even as confirmed Cynefinists, the collapse from complex to chaotic can be teeth-jarringly terrifying. For now, we’re following what can only be described as ‘novel practice’. Or act, sense, respond as the good Mr Snowden might say. After all, when one’s house catches fire, one does not stop to make a plan.
Which brings us neatly on to …
A boss of steely heart, it takes quite a lot for Anya to hand out one of our much coveted Librarian of the Week awards. Not this week. This week, all the librarians get a Librarian of the Week award - they’re in the post, honest. A list then: Librarian Anna, Librarian Anya, Librarian Ayesha, Librarian Claire, Librarian Deanne, Librarian Emily, Librarian Emma, Librarian Jayne, Librarian Jayshree, Librarian Kirsty, Librarian Jason, Librarian Martin, Librarian Ned, Librarian Phil, Librarian Sarah, Librarian Steve and last - but only alphabetically - Librarian Tim. As hot chocolates were sipped and Monday night bedtime closed in, a number of our crack team were still hard at work. Which is going way beyond the call of duty.
Not only this. Given Anya’s in an extra generous mood, she’s also distributed Librarian of the Week awards to boss computational expert Ian, single-point of success Jianhan - his second such award in as many weeks, to computational bodyline bowler Michael, our Sadia and Nishanth, and to project management mage Yomi.
I’m Librarian of the Week, you’re Librarian of the Week, we’re all Librarian of the Week. And, by heck, do we deserve it. There’s only really Young Robert missing from the list. Mostly because he chose this week, of all weeks, to take a vacation. You picked a fine time to leave us, sunshine.
Before our dear reader forms the impression that Library life is passing slopping great buckets of water along the human chain for our Jianhan to throw over blazing bin fires, actual non-firefighting work has also happened. Though it’s a bloody miracle we remember any of it. Here, then, goes.
Whilst the good ship parliamentary search shows every sign of sinking, that’s no reason to not give the deckchairs a lick of paint. Work therefore continues on new old parliamentary search, albeit somewhat slower with Anya’s attentions understandably entirely elsewhere. Data analyst Raafay has now, well, analysed the attributes present for 28 of our 34 content types. Top work Raafay. There’s more work we’d like to do to, well, analyse the data, particularly in terms of slicing it by time periods and distribution. A meeting has been pencilled in for next week when Raafay, data scientist Louie and computational dibbly-dobbler Michael hope to spec out what that model might look like.
Elsewhere, designer Graeme continues to churn through object views under the expert tutelage of Librarians Anya and Jayne. At this point, he’s probably learned more about parliamentary materials than he ever hoped to know, but he appears to stay cheerful. Frontend developer Jon has also been hard at work turning Graeme’s pixels into functioning code. A process that would be made much easier if Anya had the time to follow up on Raafay’s work and fill out our three-dimensional data dictionary. As things stand, fat chance of that.
In more backend news, our Jianhan has made magnificent progress upgrading our antediluvian Solr instance to something more modern. We’re also delighted to finally have access to a development environment, so remain hopeful that at some point in the not too distant future we may be able to plug Jon’s Rails efforts into Jianhan’s new Solr instance and the whole thing will form like Voltron. The sooner we can couple frontend to backend, the sooner we’ll be confident we’re on the right track. Or aren’t. To state the blindingly obvious though, our backend - Solr - is the frontend for the wider search service. So extinguishing the fire below remains the priority.
Luckily, data platform mk 3 has not yet caught fire so we’re still able to update statutory instrument and treaty tracker data in near real time. Librarian Jayne’s hair happily showing no evidence of singeing, a testament to a job well done. Thanks big guy.
We are pleased, nae delighted, to announce that Librarians Ayesha, Claire and Jayne have now completed redrawing all 41 of our procedure maps in Visio. 41! Count ‘em. This neatly removes our dependency on Omnigraffle and - by extension - on Macs. The necessity of doing this work provided an opportunity for Jayne to instruct Ayesha and Claire in the fine art of logical and arithmetic procedure mapping. Such fine work would, in a normal week, merit a Librarian of the Week award. But Ayesha, Claire and Jayne have already received their awards for chipping in to pour water over our computational conflagration. Even in her most generous of moods, Anya is not the type to hand out double Librarian of the Week awards. That would clearly be an extravagance too far on the public purse.
As part of the redrawing work, Librarian Claire has patched over something of an omission. Whilst we’ve had a procedural map and accompanying data for Queens’ Consent Motions since some point in 2021 - possibly earlier - we hadn’t quite got round to doing the same for the King. An omission that Ben pointed out some months back. And an omission that Librarian Claire has now fixed. We know we’re occasionally slow, but we get there eventually.
At some point over the summer, Librarian Emma was perusing Votes and Proceedings when she came across a job lot of committee correspondence on the subject of treaties. “Should we include such letters on our treaty timelines?” is a good next question. And so Emma asked it. The question of what may and may not form part of a procedure or a work package is a vexed one for us. Where a procedure begins and ends being a little too existential for our limited brains. In such cases, we tend to fall back on what we think might be useful. And in this case, we came down on the side of useful.
Procedure maps being data, not code - again, the distinction is an existential one - they are easy to change. So Librarians Ayesha, Claire and Jayne have added new steps to a bunch of treaty procedure component maps, describing committee correspondence, with Jayne sweeping along behind and actualising said steps in all applicable work packages. A quick glance down our timeline for the Australia Free Trade Agreement will rapidly reveal the results. Quite a lot of correspondence. The International Trade Committee must get through Speaker Stamps like our librarians get through tea bags. As ever we show the the domain and trust the user will find utility. Which, in this case, is more than likely to be Arabella. Other opinions more than welcome.
Stage left, general election preparations continue. We think we’re mainly on top of things, what had been something of a scrabble is at last taking on a pleasingly Kanban cadence. Following tip offs from statistician Carl, our constituencies spreadsheet now comes complete with ONS codes for the proposed new constituencies in England, Wales and Scotland. We eagerly await the release of new identifiers for constituencies in Northern Ireland. Thanks Carl. Top work Librarian Anna.
For reasons that escape us, whilst new identifiers have been minted for the complete set of new constituencies in England and Wales - whether or not their names or boundaries have changed - the same is not true in Scotland. For new Scottish constituencies where boundaries remain unchanged from their predecessor, the old ONS code is reused. Not particularly neat. Or indeed tidy. This slightly unexpected behaviour has been documented in one of our many manuals. I mean, of course it has. We’re librarians. We live for manuals. There’s still work to do to check that MNIS doesn’t choke when presented with two constituencies with the same ONS code. But we don’t think there’s any uniqueness constraint on that field, so we should be safe.
Finally, again for reasons we don’t fully comprehend, several hundred ex-Members of the Commons had no end date on their party affiliation. Party affiliations, in our case, pretty much corresponding to whippings. Edge cases aside. What one does not want, when analysing whipping data, is several hundred people whipped to parties when those people haven’t been in Parliament for the best part of a decade. Slowly, patiently, methodically, Librarian Anna has spent a large part of the summer working through them and all records are now fixed. Splendid stuff, Anna.
In more green carpet / Librarian Anna related news, she’s also spent a good part of her summer squinting at Professor Rush’s index cards, attempting to decipher his handwriting and adding the results to our Rush database. Immediately after a general election, we create skeleton records - name, constituency, party - for every new Member. Adding the prof’s special sauce of educational background, employment background, trade union associations and whatnot taking considerably longer. Thanks to Librarian Anna, that work for the 2019 intake is at last now done. At least until the next general election. Splendid.
Librarian Emily has spent some of her summer wrapping her head around “end reasons”. Or why a Member might leave the House of Commons. Following much to-ing and a deal of fro-ing, that work has been signed off by researcher Neil. And a fine bit of work it is too. Now we have a definitive list, it only remains to tidy the Commons data in MNIS and let the red carpet contingent know our conclusions. Thanks for your patience, Neil. Top work Emily.
Whilst team:Anya may be based in the House of Commons Library, we are nothing if not bicameral. The Rush database is about Members of the House of Commons, but it also contains records for said Members receiving peerages. Since Young Robert kindly introduced us all to querying Postgres directly on Heroku, Librarian Anna has been able to put together a query returning all Rush Members with a peerage record alongside the type, degree and title of that peerage. Librarian Ned has taken that query and added a whole bunch of additional ‘subsequent’ holders of peerages to our Beamish database, which means that every person in Rush with a peerage holding in the Peerages field is now also in Beamish. A librarian’s job never being done until the identifiers are aligned, Ned has also added the newly created Beamish IDs to both Rush and Wikidata. Wonderful librarianship.
Work also continues on refactoring, reformatting and generally improving our Parliament: Facts and Figures series, this week seeing a new edition of Sittings of the House of Commons over 24 hours since 1901 revised and published by Librarian Jayne. Lovely stuff.
Thanks to the diligent efforts of Librarian Ned, our beloved egg timer came equipped with a comprehensive set of dates for parliamentary periods back to 1801. Our session date data however extended back only as far as the birth of the statutory instrument service, in 2017. Contact was made with our friends at The National Archives, who kindly sent through a spreadsheet of session dates, again back to 1801. Librarian Jayne added those dates to our own spreadsheet and computational renaissance man Michael uploaded them to the egg timer.
The story did not end there, however. In the course of compiling and revising yet another briefing in the Facts and Figures series - this one on the number of sitting days per session - Jayne discovered an error or two in TNA data, namely swearing in days not always being counted. The same errors appear to also crop up over in Wikidataland. Again, our spreadsheet was updated, the egg timer data refreshed and both TNA and Wikidata folks have been notified. Or will have been by the time you read this.
Way back in week 33, it was as lovely as ever to see James who popped round our gaff for a standing orders related chinwag. Having heard whispers about a possible new set of standing orders to cover hybrid business, we wondered how one might take orders from a revision set in one business extent and splice them into a revision set in another business extent. Taking to a whiteboard, two possible approaches were outlined. And promptly dismissed as all too complicated. Given a hybrid only publication may well never happen and given that, if it did, such “splicing” would be a once and once only job, writing a query or two seems quicker and cheaper than building a management interface. We think there’s still a little work to do to tidy bits and bobs of the workflow, but it feels like we’re getting closer every week. More news soon, we hope.
Boss librarian Anya and her tame computational “experts” Young Robert and Michael took a trip up north to call in at the Wuthering Bytes festival. And splendid it was too, talks ranging from the history of telephone exchange buildings to rescuing the work of Daphne Oram from defunct Archimedes disks to designing your own computer chips. On the subject of which, Michael was forced to enquire of Matt whether our procedure maps could be encoded on chips, to which the answer was - excitingly - “possibly!” Imagine that. Parliamentary procedure but at chip level. Wow. We’d be the fastest democracy in the West.
Our intrepid explorers were supposed to be accommodated at a hotel just outside Hebden Bridge, which - being notorious hipsters - would have suited them perfectly. Unfortunately, the hotel went bankrupt shortly before the conference, so they ended up in Bradford. Which is a very different place. To help them settle in, Tom was kind enough to brave the West Yorkshire Cycle Superhighway and join them for a pint or two. Lovely to see you as ever, Tom. On the subject of people it was lovely to see, waves also to Bill, James, Paul and Giles. We hope to be back next year.
Those of us who have met Michael would probably not consider him a “reunion” type of guy. Not one for nostalgia. He’s been described - mostly by himself admittedly - as an “avatar of aheadness”. It may surprise people to learn then, that over a pint or two with Anya and Jake, he came to the conclusion that what the Parliamentary Computational Section Class of 2019 needed was a reunion. And so a reunion was organised.
Not having two ha’pennies to rub together, the decision was taken to hire the salubrious Garden Room at the Crown and Cushion public house, who’ve never been known to evict a paying clientele. Sadly, Usman couldn’t make it, finding himself on the receiving end of a nasty bout of ‘rona a couple of days before curtains up. We hope you’re well on the road to recovery now, Usman. Along for the ride were Librarians Anya and Jayne, residential computational “experts” Young Robert, Jianhan and Michael, House of Lords Ben, and our much-missed former colleagues Bex, Jake, James, Jamie, Matt, Matthieu, Mike, Samu and Wojciech. Enough to bring a tear to the hardest of hearts. Sighs.