Way back in June 2023, the House of Commons Procedure Committee published a report off the back of their Correcting the Record inquiry. Amongst a host of other suggestions, they proposed that non-ministerial Members of the House should be able to make written corrections. A privilege previously afforded only to Ministers. Since then, the report has been put to the House and its recommendations accepted.
All of which means we have a new type of content to deal with. Whilst Hansard bills itself as “[t]he official report of all Parliamentary debates”, for reasons lost to your correspondent it also contains other material. It is, for example, the home of written ministerial corrections. It will also be home to written non-ministerial corrections - or ‘other corrections’ as we appear to have ended up calling them. There is then work to do, identifying all corrections in the Hansard XML and placing them into ministerial and non-ministerial buckets. A job not made easy by Hansard XML definitely being a markup language whilst not really fitting the definition of extended, the available elements lending very little in the way of semantics to proceedings. As it were.
Further complications are added by another flavour of written ministerial corrections flowing down the pipes from the Questions and Answers system. These in the shape of written corrections to earlier answers. Though at least all those are ministerial in nature, so placing buckets to catch them is slightly easier.
The work to test existing systems with the new content type was work we had imagined had been done. Reader, it had not been done. Once more, we have had to call upon the talents of our Jianhan to separate the wheat from the other wheat. Which means he’s been unable to make much progress with the search backend, or replacing our taxonomy service or any of the other things the poor lad gets asked to do. We’re told he spent the weekend setting up an end-to-end test version of our processing pipelines and is now in a position to both prod and poke. Top work Jianhan and best of luck.
Progress on all things backend may have ground to an emergency halt, but developer Jon and Librarians Jayne and Ned continue to churn through the quality assurance work for new old search object views. Or ‘item pages’, as we’ve been tasked with calling them now. Joining the signed off pile this week are early day motions, written questions, oral questions, oral answers to questions, oral question time interventions, observations on petitions, papers laid, papers reported, proceeding contributions, Transport and Work Act order applications, European scrutiny memoranda, European material produced by EU institutions, the oddly named European scrutiny recommendations (House of Commons) and last, but by no means least, Ministerial corrections. Though the latter may need more work when other types of written corrections start to flow though the pipes, for reasons outlined above.
Whilst we still have a fair few object views - sorry, item pages - to go, Head Gardener Anya and computational bothy boy Michael continue to do the groundwork for results pages. Since our last outing, they have further reviewed our results item spreadsheet to validate the ‘can be many’ column and make the labels consistent across content types.
If you’re an old fogey like Michael and still using Twitter, you may also be aware that we’ve taken to decorating our office walls with search result-themed post-it notes, holding three of what Young Robert likes to call ‘faciliated workshops’. To the tune of Industrial Estate. Excellent work all round from Librarians Anya, Ayeseha, Claire, Jayne and Phil.
If all goes to plan and the post-its don’t lose their stickiness, a couple more sessions and we may well be done. Possibly. Given past experience, the post-it notes will almost certainly lose their stickiness and end up scattered over the floor. This is, in our experience, why most public sector ‘digital’ projects tend to fail.
Young Robert and Michael tell us they think they’ve finished their first pass Cucumber / RSpec test framework for our Solr upgrade. Librarian Ned has asked for the addition of a new specification to test the differences between text snippets returned for result items. So that’s more code that needs writing. Other than that, it only remains for our BDD heroes to somehow persuade a GitHub action to run the Cucumber and spit out some HTML. Ideally somewhere our librarians can see. It’s a project they’ve made a start on, but not a project on which they can report any degree of success to date, if the cries of “BLASTED THING” emanating from room 13 are anything to go by. Still, once that little snaggle is cleared up, we believe we have everything we need to enable our crack team of librarians to enter any Solr compatible query, push it to GitHub and easily check the results.
On the subject of checking the results, if you tuned in last time you’ll know Michael was griping - he gripes, he never whines - about Cucumber output not being compatible with what he likes to describe as his “colour-divergent” eyes. A call went out for a Cucumber output format that is actually accessible to the 8% of males - and 0.5% of females - who share Michael’s difficulties. The answer came from close by; Young Robert went home and wrote a Cucumber parser that prints the words ‘passed’ and ‘failed’. In reassuring black and white. Thank you Young Robert. If, like us, you’re a fan of development that’s behaviour-driven and, like Michael, find yourself confused by the red-green spectrum, please help yourself to Odin. We hope it improves your life as much as it’s improved Michael’s. We did ask Young Robert why he chose the name Odin and, apparently, it’s because he didn’t go to comprehensive school and knows all about Germanic pagan people with wonky eyes. Pounds well spent by his parents.
In correspondence with the Clerk and Policy Analyst from the International Agreements Committee, it was brought to our attention that not all treaty like things are signed. Some are in fact adopted. This went against earlier assurances from different folks that the word ‘signed’ could be substituted any time we saw the word ‘done’ in a treaty title. ‘Done’ never did feel like a well-chosen word. We are, as ever, pleased to be corrected. Librarian Jayne has taken time out from quality assuring our search object - sorry, item - pages and amended our treaty procedure map to allow for some agreements being signed and some being adopted. An example of the latter being the Convention, adopted at Paris on 17 October 2003, for the Safeguarding of the Intangible Cultural Heritage. You have to love treaty titles.
Staying with the subject of international agreement semantics, a wee while back we added pre-treaty negotiation steps to our terrific little treaty tracker. It has since been pointed out that a treaty under negotiation is not, in fact, a treaty, and may never become a treaty and should not be called a treaty. For that reason, Librarian Jayne has also removed the word treaty from all negotiation period steps. Unfortunately, the word treaty still appears on the treaty tracker for things which are not yet - and may never be - treaties. Negotiations for a potential agreement between the UK and Nigeria featuring the words ‘Treaty brought to the attention of Parliament’ being just one example. It would appear that someone has hard-coded this into an HTML template. Never an ideal place to manage information. A ticket has been raised with Computational Section colleagues to remedy this problem.
In our final piece of procedure mapping news, way back in the mists of time we added steps describing scrutiny clocks, as set out in legislation. Until this week, those steps were present in data, but never quite made it to pixels. Thanks to the diligence of Librarian Jayne, that gap is now closed, and you don’t have to be a computer to clock our clocks. Splendid finishing Librarian Jayne. Excellent completing.
A couple of weeks back, Michael was left home alone with some paper, some string and some sticky tape. He proceeded to make a right mess of everything. His efforts were not entirely unrewarded when a string shaped gap popped up in what he likes to call his ‘meta-model’. Whilst it would be hard to deny that an order to print results in some form of printed paper - at least eventually - it’s less clear what the subject of a presentation might be. Our clerkly colleagues have been known to refer to the concept of a ‘dummy bill’, but we suspect that’s because they’re not librarians and not FRBR compliant. For that reason, our order to print model now makes a printed paper a type of paper expression and our presentation model now makes a presented paper a type of paper work. All very neat. At least on paper.
Our only other change has been the addition of yet another model specialising making available, this one describing papers being reported to a House by a committee. Or a formal body, as Michael likes to call them.
Efforts to load notional results data into our psephology explorations came to a painful and grinding halt when Michael (re)realised that - in an Electoral Commission registration identifier sense - the Brexit Party and Reform UK are the same thing. In some quantum superposition sense. It’s more than probable that several people had already pointed this out to the lad, but his memory is not what it once was.
This proved problematic because our code - such as it is - was keying off Electoral Commission IDs. Meaning we had a choice between showing the Brexit Party as standing in the next general election, or Reform UK standing in previous general elections, or actually sorting the problem out. Where sorting the problem out means populating our general election spreadsheets with MNIS party IDs, populating the notional results spreadsheet with the same and reimporting to key off those IDs in the application. Librarians Anna and Emily are, as ever, on the case.
In news of work that actually did happen, we’ve identified a reliable source of by-election data - should we ever get the time to deal with that - and generated mappings between the soon to be decommissioned election result website URLs and our own. Just in case some htaccess redirection file type thing should become possible. Michael has also turned what had been four different queries - for general elections by majority, vote share, turnout and declaration time - into a single query. Which should make things more maintainable. He hopes.
A wee while back, Statistician Carl asked for general election result listings by countries, English regions, and some superset of Great Britain. We can’t actually say we’ve done that yet, but the preparatory work is ticked off. As in, we now at least acknowledge the concept of Great Britain. Not as easy a task as one might expect and one that led to some pretty horrendous SQL contortions. For a short while, Michael managed to put Northern Ireland in Great Britain. Not an outcome that would have pleased anyone, least of all Anya. That situation is now rescued and it only remains to write yet more SQL. Onwards.
Following up on Librarian Emily’s magnificent efforts late last year to document Commons end reasons - or why a Member might leave the House of Commons - we attempted to make a start on the same work for the red carpet contingent. Then promptly forgot we’d started. A gentle poke from the Journal Office rod of Mr Korris got us back on the case and we now have a rough and ready first pass of a picture. More fine work from House of Lords Matt and Kirsty, and Librarians Emily and Anna. We remain slightly unsure around some of the more ecclesiastical matters, particularly around the retirement or defrocking of bishops. We remain hopeful that all will become clear - or at least clearer - once it’s been subject to the marking pen of Ms Murphy.
Back in the land of Commons end reasons, Librarian Emily is close to completion on her data tidying mission. Almost all departed Members have now been assigned their appropriate end reasons although we still have a slight problem turning ‘general election’ into ‘dissolution’. This, once more, down to a hard coding snafu in the Member management system. A snafu we hope will be fixed soon.
Sometime late last year we put live our committee feeds application. Should you be in the market for oral evidence transcripts and committee publications direct to your inbox, we still recommend it. Except. Well. Except, shortly after Christmas Librarian Jayne reported an error when a handful of publications from the International Agreements Committee failed to turn up. Investigations commenced, uncovering Heroku flagging some pretty horrendous errors. Errors which we believe are now fixed. Whilst HTML views still list everything that can be listed, RSS views are capped to the 20 latest items. We’re not quite sure how Roy Fielding might feel about this, but we do our best with the tools we have at our disposal. Michael would like to add that he continues to “tail and grep logs”, we suspect in some effort to sound vaguely computational.
Back on Thursday - we forget quite which Thursday - Michael had the good fortune to hear a talk on the subject of ‘agile comms’. A subject we have some interest in. But, being a shy lad, he found himself unable to dive into chat and note - if anyone needed tips on such matters - that these very weeknotes have been described as a “work of genius” by the bloke that wrote the book. In the more rarefied atmosphere of these notes - with but a single reader reading - it would appear he’s found a way through that particular struggle. We must apologise on his behalf.
