In one of our occasional breaks with tradition, our much coveted Librarian of the Week trophy this week goes to not one Librarian, not two Librarians, but All The Librarians. Librarian Anya took the bit between her teeth and mailed the entirety of her crack team with a request to apply testing eyes to new, old parliamentary search. And test they did. Feedback has flooded in. Happily, many of the issues spotted will inform work we know we’ve not yet done. Namely the application of a design layer to the thing. Up til this point, work has been concentrated on getting the bone structure of the markup in place, but - at some point soonish - we know it requires a fresh coat of CSS to make it acceptable to librarians and non-librarians alike.
That said, a bug was spotted - thanks Librarians Ayesha, Emily and Phil - and we know we need to spend a bit of time testing more fully with assistive technology. A coat of lipstick being no remedy for those using screen readers.
Still on the subject of new, old search, we remain in something of a consolidation phase. Seeing our new-ish search result pages alongside our slightly older object pages revealed a number of inconsistencies, many a user journey taking a slight stumble. Fixes to one aspect revealing the necessity of fixes to other aspects, in what could easily have turned into an ever expanding circle of template tweaks. And, for a short while, did. Luckily, our circles are once again converging, Librarian Jayne and Developer Jon continuing to bat Trello cards across the net at a frankly remarkable pace. This week saw nine more cards make their way to the happy pile. Said cards are so granular that we don’t intend to list all of them here. Let’s just say POST is now back to being a publisher, date ordering is fixed, and iOS no longer attempts to render citations as telephone numbers. That should give our dear reader some flavour of work ongoing. For those wanting a full picture, please check our done column.
Stage left, our Jianhan is preparing to wave more of his magic over our Solr configuration in the hope we can make Developer Jon’s life a little easier and iron out a performance problem or two. Librarian Anya has compiled a list of tweaks to be made and Jianhan is already hard at work, ensuring our answering department field is no longer polluted with asked to reply authors. More next week.
Stage right, poor Robert continues to wrap his brain cells around deploying Jon’s code to Azure, Parliament’s commodity cloud platform of choice. We can confirm he has finally managed to track down the internal addresses for both Solr and SES and passed this information back to Jon. The rest of Robert’s Trello board looks frankly terrifying and your regular correspondents confess they do not look forward to attempting to type about it.
Map making news has been thin on the ground of late, Librarian Jayne being otherwise occupied by her ping-pong game with Jon. This week, however, has seen one change to maps necessitated by the lapsing of a temporary standing order. Those familiar with the nitty gritty of the European Union (Withdrawal) Act 2018 will be aware that it set out a new procedure in the form of proposed negative statutory instruments. Where a minister intends to lay certain instruments subject to the negative procedure under powers delegated by this Act, a committee in each House should be tasked with either agreeing to the negative procedure or with recommending an upgrade to the affirmative procedure. The House of Commons discharged their half of this duty by establishing the European Statutory Instruments Committee. Unfortunately, for our purposes, said committee was set up under a temporary standing order which lapsed at the end of the last Parliament. It being subsequently decided that ESIC would not be resurrected, such instruments will instead be referred to an appropriate departmental select committee. All of which necessitated mapping changes. Routes into ESIC consideration have now been marked as closed and we await future referrals. At which point, new routes to new committees will be added. More excellent work from Librarian Jayne there.
In possibly more interesting news - and again thanks to Librarian Jayne - our longstanding efforts to map legislative consent motion procedure in the devolved legislatures have at last borne fruit. Taking our mapping efforts, Jayne first attempted to capture LCM related events in Belfast, Cardiff and Edinburgh, for the Retained EU Law (Revocation and Reform) Bill 2022-23, before quickly realising that the Northern Ireland Assembly had not been operational at that point in time. Undeterred, she set about tackling the Tobacco and Vapes Bill 2024-25. Because we now have our lovely little procedure browsable space™, her efforts exist in actual pixels - Retained EU Law (Revocation and Reform) Bill 2022-23 and the Tobacco and Vapes Bill 2024-25. If this were any other week, Librarian Jayne would be sipping champagne from her well-deserved Librarian of the Week trophy. Still, it’s the competition that keeps them on their toes.
Do we think we’ve accurately captured all the possible LCM-related steps in all three legislatures? Probably not. Do we think we’ve correctly captured all the possible routes and logic between those steps? Almost definitely not. Are we under the impression that we’ve created business items for everything that’s happened? Absolutely and utterly not. There are some rather important steps that we know do happen but which Jayne could find no evidence of across the three websites. Nevertheless, it’s all progress.
The next step is a meeting with Researcher Graeme for planning. Making the maps more accurate and devolving information management to where the domain knowledge is were both high on the agenda. For the past couple of years, we’ve been on a promise to Legislation Office Liam to lend a hand tracking LCMs before he retires. With that moment closing in, we’re duty bound to put a shift in. Let’s hope we get some trips out of it.
Back on the subject of our lovely little procedure browsable space™. Progress was inevitably slowed by Michael’s trip to Benidorm which in turn lent a Pedro Ximénez induced fog to his subsequent slog through election result templates - of which, more later. Still, we’ve seen some progress. Shedcode James has very kindly reunited HTML with CSS.
The addition of bill-based work packages off the back of Jayne’s LCM efforts revealed a small bug in a few our SPARQL queries. These have now been fixed. Work package and clock listings under a procedure have been added and we now have pages describing legislatures, Houses and clocks. If you tuned in last time, you’ll know that Michael was feeling quite pleased with himself, having added QR codes to our work package pages. Though he did worry a little that these would not survive contact with Librarians Anya and Jayne. Reader, they did not survive contact with Librarians Anya and Jayne and have now been placed, quite firmly in the GitHib bin.
Behind the scenes, progress has also been made. Librarians Jayne and Anna, Young Robert, Michael and friend of the family Silver continue to investigate the shift from our somewhat bespoke procedure editor application to the rather more commodity offering of Data Graphs. If we’re about anything, it’s the value chain and if a thing’s worth doing, it’s worth Wardley mapping, as Young Robert might say. We’re now in a position where the core procedure model has been imported to Data Graphs, cardinality constraints have been applied and the bits around the making available of papers to Parliament have been tacked on the side. It only remains to pop on our FRBR hats to add the bits around paper works, and we should be model complete. Next step data.
Whilst we await the verdict of our famous five on our putative ‘shift to commodity’, the world does not stop still. For now at least, procedure editor remains our production system, so when that developed a bug, we were lucky to have our Jianhan step in and fix it. Thanks Jianhan.
In procedure mapping adjacent news, we are now in proud possession of a test instance of our beloved egg-timer. A thing that has long been on our backlog, current deployments being something of a shoot and pray operation. Librarian Jayne has set up a selection of test calendars and Shedcode James has done everything necessary to deploy the application to the test instance. Being a test website, it’s not something we can point you to, but, rest assured, James has tested from one end to the other, and everything is working as expected.
Whilst the application code has never experienced any major problems, the data integration to the Google calendars has. That integration relies on us storing a sync token, which is passed to the Google API with a request for everything that has changed since said API issued said token. The API response contains a set of change events and a new token to use when we next ask for changes. All well. All good. Except, for reasons we’ve never quite understood, on occasions the API gets back to tell us the sync token it last gave us has expired. Leaving our beloved egg timer up the creek with only stale data to paddle with. James’ next job is to upgrade the interrogation to Google’s more moderne calendar API, adding a dash of observability so next time the sync token expires, we’ll at least get alerted. Super.
In previous exploits, Shedcode James also delivered our lovely little standing orders application. The data in there was kindly donated by the good folks at ParlRules, but their efforts stopped on the 30th November 2022. Since when, there have, of course, been a number of changes. Librarian Claire has now pulled together a list of all such changes and the dates on which they occurred. That covering ten changes across nine dates. The next job then, is cloning the latest revision set and making the first set of necessary changes. Then doing that eight more times. Godspeed, Librarian Claire.
As our dear reader may well remember, last time out we reported that Librarians Anna and Emily were pretty damned chuffed, having applied the last drop of spit and a fair amount of polish to a couple of spreadsheets covering by-elections in Parliament 58. We also left with the vague promise that, should you tune in ‘next week’, Michael may well have popped them into our psephology database and exposed them on our election results website. At which point, Michael announced he was buggering off to Benidorm. So that was another promise broken. We’re happy to report that Michael is now back at work, though still with a lingering whiff of Pedro Ximénez.
In fairness, it turned out that importing the data was the work of minutes, the pre-prepared script working perfectly. Exposing the data proved more problematic, all kinds of templates needing tweaking. The details of which are too boring for even these notes. On Wednesday of this week, we finally went live. Thanks are due to both Anna and Emily, and to Data Scientist Louie for data wrangling and last minute cache clearance duties.
Equally pleasing, Shedcode James has now put in place piping from the Postgres database on Heroku to a SQLite dump on GitHub. Which means both our Datasette and our Datasette Lite instances updated automatically, without Young Robert having to take out his Python pen. Thanks, as ever, James.
Parliament 58 by-elections considered done, we only have another three Parliament’s worth to go - plus the unknown unknowns of the current one - before we reach general election and by-election parity. So not much work then. Once those are in place, intentions remain committed to reaching back past 2010, with some dream of exploring all of - recent - history. The year 1950 has been mentioned. Which should take us nicely up to retirement. Unfortunately, before we travel back much further in time, we hit something of a blocker, the ONS not publishing geographic codes beyond a certain point. Statistician Carl has contacted the ONS in the hope they might be able to resolve this slight snaggle. If that doesn’t work, we may need to take matters into our own hands.
Librarians Ned and Phil have been hard at work massaging our taxonomy to cope with the transliteration of names. Users of Parliamentary Search are thought likely to search for people in a given name, family name order. They are also thought likely to switch non-English characters - be that characters with acutes, breves, carons, cedillas, circumflexes, commas, diaresis, dots, graves, hooks, horns, macrons, ogoneks, rings or tildes - for their English equivalents. A quick investigation revealed that transliterated forms had not been consistently applied to names with non-English characters. Henceforth, we have a new rule in place - where a name contains non-English characters, we use the non-English characters in the lead term, and add non-preferred forms of the name using transliterated English characters. So, in the case of Pedro Ximénez, the lead term would ‘Ximénez, Pedro’, with alterative labels listed as ‘Pedro Ximénez’, ‘Ximenez, Pedro’ and ‘Pedro Ximenez’.
With this much thought and this much effort going into taxonomy management, it would be a damned shame if only Parliamentary Search reaped the rewards. Which is where our taxonomy liberation project comes in. Silver and his colleagues back at Data Language HQ have been working on a software component with the working title ‘Mirage’. This takes updates from the taxonomy and propagates them by means of a message queue to any number of downstream systems. The first of these being our early attempts to assemble a new and improved Subject Specialist Finder, matching policy areas to Commons Library research experts. This as part of wider efforts toward a Single Subject View of the Library™ - of which, more later. This week, Silver reports his colleagues have repointed Mirage from a test version of our taxonomy API to the live one. No issues being reported en route.
This week also marked the publication of yet another briefing paper in our Parliamentary Facts and Figures series, this one being dedicated to Members of Parliament holding dual mandates. A dual mandate being the simultaneous holding of a seat in more than one legislature. More top work from Librarian Phil.
A couple of minor resprays to report over in ontology-land, where our agency model is now stripped back and fit for purpose, and our time period model no longer panics when faced with the assignment of George III’s regnal years. Edge cases, you may ask? Mate, we’re filthy for them.
Back with Shedcode James, he’s also been on duty for our friends at the History of Parliament Trust, where our Rush database now comes complete with its own Datasette instance. Unfortunately, access is restricted to academics, so we can’t link to that here. But, trust us, it’s lovely. James has also made changes to authentication on the Rush website, which now relies on tokens supplied via email, and no longer requires the issuing or the storage of passwords. So, one less thing to go wrong.
Continuing their long and winding normalisation journey, Librarian Anna and Shedcode James set to work on post-1974 counties, which have now joined pre-1974 counties on the reference data pile. They’ve also been beavering away at the new Member spreadsheet importer which they report is now fully tested and working. So, next time these weeknotes arrive in your inbox, it’s perfectly possible the Rush database will be complete with the 2024 intake. Splendid.
Outreach and engagement efforts are mostly concentrated on our partnership with Cabinet Office Kelcey and our joint mission to model government organisations specifically and other organisations accountable to Parliament more widely. Or at least that’s our intention should we ever break free of template changes. Mr Sheridan has kindly invited us all over to Kew for a whiteboard session sometime in March. We’re also due to meet Philip from the Institute for Government to chew the fat over their Ministerial Database and what they might do differently if they had their time again. If you have an interest in - or a source of data for - machinery of government changes and government appointments, please do get in touch via the usual channels.
Also in March, Anya, Silver and Michael have been invited to talk at World IA Day. Though, sadly, only at the London event. If you’d like us to do a Phil Collins and fly to Philadelphia during the interval, again, please do get in touch. This represents an excellent opportunity for anyone interested in exploring what the House of Commons Library does via the medium of Wardley maps. You’ll also find yourself in a prime position to hear all about how our Single Subject View plans will bring all of that together. Why not sign up and come along?