Wednesday was data day. The first half was a bit tough but we burnt through the agenda in the afternoon. This appears to be a theme. Maybe skip mornings?
Anyway, lots of variety and steady progress on most fronts. It was agreed that our top development priorities at the moment are:
Getting committee data from the Members’ Names Information Service (MNIS) into the data service.
Getting question and answer data into the data service.
Migrating data.parliament.uk to the cloud.
And some other notes:
Pages for government departments / organisations / answering bodies are still necessary and a precursor to lots of other potential linkages.
Data and Search work for opposition roles has been done.
Early day motions look like a possible candidate for modelling next.
Our investigative work to see how deposited papers might appear in web search yielded slightly odd results. Further investigation is needed.
It was the night before Christmas and nothing did stir, except for the bloke who was trapped in Yammer.
Ben did some (very belated (sorry)) video editing from the last Newspeak event. He’s planning to start editing the vids from our trip to ODI Leeds shortly. They should all be ready to go live about the same time. Sadly.
…once again go to Mike who we’re fairly sure got another thank you from another PhD student. It would be a disappointing week if he didn’t.
Anya, Silver and Michael met with David Beamish to idiot / sanity check some of their modelling work around house membership, elections, political blocs, formal bodies, tabling, and questions and answers. No major quibbles were raised and the sanity box was largely ticked. The tabling model still needs more work. Supplementary oral questions also need modelling. But not before Christmas.
The Lords are now mostly inflated (still really don’t ask). Ben has some work to do around giving non-“great see” bishops individual seats linked to incumbencies rather than bishoprics. A note to clarify that this will never include the Bishop of Man and Sodor. Unless you’re playing widdershins. I did say don’t ask.
In practical terms, it means we should shortly see lists of Peers present in previous Parliaments. We still need to work on ‘incumbency interruptions’ to fix the ‘wrong number of Lords’ problem. And finally do the people names thing properly.
On the subject of Peers, Ben spent most of Monday to Wednesday as the official Peer Wrangler for their photo sessions. Photos should be flowing into the platform and out to the web fairly soon. Creative Commons licensed as ever. Fill your boots.
Jianhan continued to work on the data platform OData API service. This has now been deployed and can be used to explore our data via an assortment of tools from Excel to web browser. Like much else, the code is now on GitHub.
Samu has been working with Jenna and Jamie from the website team to integrate content publishing into the data platform. There’s now code deployed on staging to get blobs of content from a headless CMS and make them available through standard data platform APIs. Chris helped out by getting the CMS vendor to fix a bug in their API. ‘Content’ should start to appear on beta.parliament shortly.
Matthieu finished performance testing the database cluster. We’re told it’s now performing like lightning. A report in the form of a post is on our blog. Not this one; our other one.
We made decent progress on migrating data.parliament from its current hosting environment to the new one. As soon as some network changes get made, we can go live.
After two weeks of testing and trying different things (with help from Samu, Wojciech and Mike) Alex now has a cool system in place to automate Solr queries and notify users based on results. These queries are usually used to search for inconsistencies and errors in data that can then be fixed, but they’re currently being carried out manually by Phil on a daily / weekly basis. Which is time consuming. There are now 7 automated tasks set up which run the queries daily then send out emails or create calendar events to alert users depending on what results are returned. A lot may be tweaked in the future including the methods of notification. But it’s a good starting point that can be further built on and improved.
We’ve got pretty bored with measuring things. Instrumentation is dead to us. Just running on pure instinct here. And raw talent. And hypertext.
Dan wrote a paper for David (our Deputy Director) and Emma about a new approach to ‘corporate’ data in 2018.
Aidan met with Samu, Michael and Anya to put together job descriptions for the new positions due to be advertised early in the New Year. Do check back later if you fancy being a developer or a data architect or just a person who helps make web things. Preferably the latter.
We’re slowly coming to the conclusion that they may have a different learning style to that provided by more traditional education. Or they’re dumb as rocks. One or t’other.
Alex fooled almost everyone by saying cryptocurrency but definitely not saying blockchain. Well played Alex.
Dan and Robert took a stroll and had lunch. Ben and Robert also went strolling and drank coffee. Ben smoked fags. To no-one’s surprise, Robert didn’t.
Robert and Michael did less strolling and more eating, stopping off at Coleman Coffee Roasters for plates of oatcakes and cheese. Other purveyors of oatcakes may be available and just as good. Seems unlikely.
Kunal celebrated the arrival of a baby girl. Next stops: school and work. It doesn’t get better, kid.
In which our Noel got named as Hillingdon’s Special Constable of the year. Result. \o/