This week starts, not with our usual apology, but with a confession. There has been many a Friday afternoon when, looking back over our many and various Trello boards, Librarian Anya and computational midshipman Michael have come to the comforting conclusion that the material is a little too thin to warrant a weeknote. No writing this weekend, they agree. But not all divisions can be deferred, and procrastination catches up with us all. Occasionally not just catching up, but having the temerity to lap one on the home straight. Such was the case last week, when our regular correspondents lost a sizeable part of their weekend turning pixel based post-it notes into prose. Not a mistake we’ll make again, they said, to the sound of early autumnal leaves turning. From here on in, they fully intend to up the tempo of their notes. Or the cadence, as we say in these parts. Never being ones to use normal words, when a fancy one will do.
And so, this edition makes two notes in two weeks. Quite the treat for our dear reader. Please rest assured, all other terms and conditions remain unchanged. We continue to drop no cookies on our reader, holding firmly to the belief that tracking people is not only wrong, but an offence to the scientific method. You’ll continue to find weeknotes a KPI and ‘metric’ free zone, as we endeavour to exploit the lucrative - and largely untapped - market for people who dig Erskine May and grok HTTPRange-14. Onwards!
No sooner had we pressed git push on last week’s offering - complete with an exciting glimpse of our Jianhan’s next bit of procedural tinkering - than Jianhan got back in touch to declare it done. The lad really is our single point of success and we’d be completely lost without him. Librarian Jayne has tested and signed off on the latest changes. Which means the same changes are now being made in live, even as we type. And will probably be live in live before we finish typing. Our crack team of librarians will have control over both the ordering of procedures and the ability to distinguish between those that are current and those that are now defunct. At least in theory. In practice, Librarian Jayne needs to supply our colleagues in Software Engineering with a new SPARQL query or two. At which point, we’ll be taxiing towards the runway, waiting for a takeoff slot from our colleagues’ air traffic control. Once the new queries are deployed, our statutory instrument website will no longer rely on contorted procedure names to control display ordering. A small change perhaps, but then the best changes are. We are aware that in her formative years, JO Jane was a member of the teaching profession. We sometimes wonder if she is still in possession of her little book of gold stars. And stare hopefully in her direction.
Not that our Jianhan ever gets a rest. Whilst the poor lad was chipping away at the coalface Librarian Jayne and her computational show ponies, Young Robert and Michael, were planning more work. If you tuned in last week, you’ll know our efforts to integrate our beloved egg timer almost fell at the final hurdle when we stumbled over a British Summer Time bug. Not for the first time. This might best be classed as a ‘legacy technology’ problem. Or a ‘technical debt’ problem, as we say nowadays. Database fields which were only ever intended to capture dates - Parliament not being big on times - were instead created as datetimes. Work to rectify this minor mistake has now been planned.
We fear the other bit of planning work requires some explanation. Back in 2018, paragraph 3 of schedule 7 of the European Union (Withdrawal) Act introduced a brand new procedure for proposed negative statutory instruments. Or PNSIs as we like to say. Not a procedure you want to pronounce when plastered.
The procedure would be invoked whenever the government intended to lay a statutory instrument enabled by that Act and subject to the negative procedure. Since then, we’ve seen the exact same procedure also pop up in paragraph 8 of schedule 5 of the European Union (Future Relationship) Act 2020 and in paragraph 6 of schedule 5 of the Retained EU Law (Revocation and Reform) Act 2023. Why our legislation drafting colleagues don’t just use hypertext, we cannot say. Transclusion is your friend, my friends.
Being librarians - and crack librarians at that - our team likes to cite sources. Which means our procedure maps come decorated with pixels describing where steps and routes are set out in legislation and standing orders. This became problematic when faced with a single procedure map covering what are, in reality, three different procedures. Albeit the same. Jayne, Robert and Michael scratched heads and stroked chins and came to the conclusion we’d need three maps for three procedures and a new way to collect procedures into groups. Although, whether the procedures are the ‘procedures’ or the groups are the ‘procedures’, we’re less than sure. Five years we’ve been noodling around with procedure data and still we’re not completely convinced we know what a procedure is. That said, we now have a plan, and no plans to execute that plan until we’ve chatted to both JO Jane and JO Eve. Just to check we’re not going mad.
If you’ve wondered why it’s been a little quiet on the procedure mapping front of late, we can give two excuses. Firstly, we’ve run out of statutory instrument and treaty procedures to map. We really need to restart the public bill work in general and the legislative consent motion work in particular, should we ever get a spare minute. Which at this point seems unlikely.
Our second excuse is more prosaic. Our procedure maps were generated in a program called Omnigraffle. Unfortunately, Omnigraffle is only available on a Mac. Our business continuity budget does not stretch to two computers per librarian, and it especially does not when one of the machines is an Apple. All of which led to Librarians Ayesha, Claire and Jayne planning to spend most of summer recess redrawing 40 maps in Visio. Librarian woman-hours being harder to calculate. So far 10 of the 40 procedures have been redrawn. Only 30 more to go. That said, they’re also taking the opportunity to convert a fair few component procedures from the purely logical to the more arithmetic. So it’s not all wasted effort. Swings and roundabouts we suppose.
Way back on Monday, we were delighted to be joined by Hansard Charlie who was kind enough to cast a critical eye over our proceeding and oral contribution models. No horses appeared to be frightened. Michael still has a job to contact Mr Evans in the hope that he be might be inclined to cast a more procedural eye over matters. In the unlikely event, you’re reading this, Mr Evans, please expect an email shortly.
Work continues apace over in general election land. Much of our data wrangling taking place in spreadsheets and, those spreadsheets being somewhat critical to success, we decided to take a belt and braces approach to preservation. Which means we now have backups on GitHub and a documented routine task to keep said backups up to date. Splendid work Librarian Anna.
Meanwhile, Librarian Emily has been plugging away, trying to wrap her head around the many and varied ways a Member of the House of Commons might lose their membership. She had thought she was on the home straight, when researcher Neil threw a curveball in the shape of disqualification for bankruptcy under section 426A of the Insolvency Act 1986. Emily and Michael scrambled to adjust both list and picture, only for Richard to point out bankruptcy is an England, Wales and Northern Ireland only thing. Sequestration being the Scottish equivalent. Once again, list and picture have been updated as we witness section 427 of the Insolvency Act 1986 entering the fray. We continue to hold on to the hope we’re getting closer and not actually making things worse.
If you tuned in last week, you’ll know we were joined by developer Jon. This week, our small but perfectly formed expeditionary force expanded once more, as we welcomed data analyst Raafay to the fold. We’re hopeful he’ll be able to help us describe what we’ve taken to referring to as the ‘lumpiness and bumpiness’ of the data in our search system. A technical term. Lumpiness and bumpiness that can be put down to some the data stretching back over a 30 year period, some having originated in assorted systems now obsolete, having been subject to different information management practices over time and having been migrated through anywhere up to five preceding systems. Lumpiness and bumpiness that can also be put down to some of our pipes being a little on the rusty side and the central system being a triple store. Not that there’s anything wrong with triple stores. When you’re trying to describe an open world, an open world system is all for the best. But the lack of constraints not only leads to some things that should be associated with, say, a session not being, but also to some things that should be associated with a session being associated with two. Or more.
All of which leads to designer Graeme and developer Jon’s lives being a little harder than would be ideal. Librarian Anya is doing her best to explain what’s in Solr and where it should appear on a page but it’s pretty much impossible to capture all the edge cases as anecdotes. What we really need are the numbers. So last week, Anya asked for a data analyst. And this week Lydia and Yomi delivered. Thanks Lydia. Thanks Yomi.
The first job for Raafy was a quick chat - or a ‘3-dimensional data dictionary workshop’ as we like to say in these parts - to sketch out a first pass data model for the kinds of numbers we’d like him to capture. That sketch has since been turned into a spreadsheet - or a set of spreadsheets - which we hope to begin populating shortly. At which point, we hope Anya, Graeme and Jon will have a better idea of which attributes are likely to turn up, which ones aren’t and which ones might turn up more than one might perhaps expect. Important information if you wish to migrate and tidy the data, let alone make web pages from it.
Stating the obvious perhaps, but Anya’s patient explanations, Raafay’s analytics, Graeme’s design efforts and Jon’s code are all to nought if we have nothing to search over. Ever the enabler, our Jianhan and his computational helpmate Young Robert have been beavering away, attempting to migrate the data in our antiquated Solr instance to something more modern. Late last night Jianhan reported back on his attempts to migrate from Solr 3.5 to Solr 4. Efforts that were not, it would seem, entirely successful. There are so many changes to the schema, that, without major configuration changes, the search returns nothing. A lesson to us all in patching early and often, perhaps?
On the more positive side, Jianhan did come across some Java code which promises a pain-free Solr upgrade path. Pain-free if you can manage to install it that is. “Building stuff in Java - not done this for a decade or so …”, reports Young Robert. “It’s as baffling as ever”, he adds, seconds later. Silence reigns for a moment or two. Then, “Jesus Christ. Things haven’t really changed much in the last ten years, it seems.” Accompanying his update with an error log the length of War and Peace. Single point of computational success Jianhan appears to be having a little more success, having got the thing almost working on Windows. Stay tuned for more exciting upgrade news next week!
We’re frequently complimented on our dedication to the procedural pedantic web cause. “Do you people never rest?” we’re asked. And yes. Yes we do. All filing and no fun makes for very dull librarians. Wednesday last was once such occasion, when thimblettes were placed into thimblette holders and our crack team of librarians and their computational compadres headed over to Hackney for a day in the country. Or a “team-building workshop with ice-breakers” as we like to say in these parts.
Librarians Anna, Anya, Ayesha, Claire, Emily, Jayne and Susannah together with computational “experts” Jianhan, Young Robert and Michael assembled at Fabrique for a pastry-based breakfast before heading over the road for a class photograph at the Museum of the Home. Gift shop perused, pencil sharpeners purchased, the class headed down to Hackney City Farm to ooh and aah at the babby goats and the babby pigs. But mostly at the babby donkey. Who was sporting the most marvellous pair of pants. Well, two pairs really. With attempts to adopt the wee babby donkey being met with flat out refusal, our party proceeded to Broadway Market for pizza and beer, staying out long past picking-up time. Librarian Phil - who’s still partly on parental leave and sadly could not join the fun and frolics - probably summed things up best. A fine day out.