weeknotes.data-search

2018 Week 40

Dator day

Wednesday was dator day. Or half of it was. Our Liz, Anya, Alison, Robert, Paul, Samu, Jamie, Dan and Michael spent several hours locked inside the Archbishops’ room which is really big considering there’s only two of them.

There was a conversation around plans for taxonomy management tools which everyone seemed to think was constructive. Then Jamie outlined some short term plans for the beta website in general and the statutory instrument tracker in particular. Immediate priorities being differentiating between SIs (or at least work packages) that are currently before Parliament and those that have been and gone, and listing SIs by laying body. We have a laying model, so most of the domain model bit of this is done. But currentness (or currentcy as Robert might say) is hard. Keep reading for more details. Or stop now.

Your correspondent struggles to cover the rest of proceedings, since he threw a mard, (accidentally) knocked over the water dispenser and flounced from the room like a low-rent Laurence Llewelyn-Bowen. If you’re craving more detail, there’s a photo with some ticks here.

Community

Dan headed off on another trip to Bristol where he gave a talk at Delib’s practical democracy project meetup. Where, by pure chance, the theme of the night was fixing websites for Parliaments and legislatures around the world. Because no Parliament is a snowflake. You might say. Dan says he met lots of great folks and had many useful conversations. There were some tough questions about democracy and citizens and, perhaps unsurprisingly, user needs. Which we’re sure Dan answered with aplomb.

On Thursday Anya and Michael were joined by librarians Jayne and Liz, developer James and History of Parliament Trust Paul. They chatted about progress on the Rush data project from both a technical angle and a data entry point of view. Development work is almost done and Jayne and Liz reckon they’ve added about 50% of the Members from the last three intakes. They’ve agreed to switch from adding data to a set of spreadsheets, to using James’ application. Let’s see if anything breaks. If anything goes seriously wrong we’ll use the remaining time to fix bugs. If not, we’ll use the time to add data downloads and handle identifier interlinking with Wikidata. The last bit seems easy enough so Michael has put James in touch with Andrew and a plan has emerged.

The delivery is the strategy

With Dan down in Bristol, Alison took over on data strategy open session duties. Which attracted its usual hardcore audience and a smattering of new faces. Notable by his presence was Bill Boshell, an Information Manager from Strategic Estates. Focusing on one of the principles in the data strategy - ‘protect our data and consider privacy’ - those gathered discussed practicalities in the context of Bill’s work on Information Asset Registers, Information Owners and user training. We’re told Bill left feeling like he’d had a thorough grilling. But we hope that won’t put him off attending future events.

One world, one web, one team

Sara met with librarians Liz, Martin and Steve to discuss the results of her work to correlate search terms with the controlled vocabulary. They talked through the method in some detail and came up with a plan for improvement. The next step is to look at the list of search queries that did not match any terms from the controlled vocabulary to better understand if there are any topics / themes / words that users search for that are not in the vocabulary and could potentially be included as synonyms for what’s already there.

Our Liz has been watching stuff on reproducible analysis to inform our thinking on workflow and documentation. She thinks having clear reasons for each step taken to produce a result, and making those reasons visible, is key. Liz thinks this will help make analysis more relevant and convincing. She also notes that this isn’t limited to analysis and has been discussed in relation to other work. Michael is reminded of Tony’s thinking on reproducible research here.

Domain modelling

The week started with a poorly Anya. Her nose and chest exploded almost simultaneously. We occasionally worry that she may not be smoking enough. Nevertheless, despite being a librarian down, team:whiteboard ploughed bravely on.

Michael spent a chunk of Monday morning with Martyn, clerk bloke for the House of Commons Procedure Committee. They talked through the legislation model and feedback they’ve received from David and John in the area of powers and duties. Some words got nudged around.

Robert and Michael spent much of Tuesday diving back into the legislation model, tweaking some of the shape, some of the labels and some of the comments. They spotted Nerys across Tothill Street cafe and managed to grab her to idiot check some words. It feels like the legislation model is getting to a stage where they’re almost happy with it. At least for now. Thanks Martyn. And David. And John. And Nerys.

Luckily Anya picked up her smoking and returned to work on Wednesday. Still spluttering slightly. But able to attend a meeting with librarian Jayne and Computational Section representatives Bex and Chris. They began to crack apart what it means for a statutory instrument to be ‘current’, in a currently before Parliament sense. And thought they’d cracked it. They hadn’t.

In a second stroke of luck, Robert’s brain was available on Thursday, so Anya and Michael sat down with him and rewrote the rules, checked them three times and started to add ‘no longer before the House’ steps to the procedural flowcharts. So far they’ve covered proposed negative statutory instruments and both of the negative procedures. Affirmatives to follow.

At some point they really need to bite the bullet and think about adding logic gates to the procedure model. But they’re comforted by the sound of Robert’s brain grinding into gear on this.

Data platform

Matthieu worked on automating our deployment of VocBench. He also wrote a post about generating bash scripts using our public SPARQL endpoint to help a BBC person grab a bunch of our Member portraits more easily. And for anyone else wanting similar tips.

Dashboarding all the measurements

Mike’s been hard at work creating a dashboard view of our data platform API. It’s fed by data from our analytics software and lists each endpoint by the percentage of failed requests, a breakdown of response codes per environment and the request URL of all responses tagged as failed over the last hour. It also shows the total number of API requests for all endpoints or per environment or per endpoint. Again with a breakdown by response. Top dashboarding Mike.

On search. And indeed indexing

Following the recent deployment of new analytics software to the website search service, our Liz took a look at the referrer info. For any view of a search result page we can now see the page the user was on when the search was performed. Perhaps unsurprisingly, the top locations for users starting searches are the Member biography pages. Information about MPs and Peers accounts for a large proportion of our traffic. Which means we probably need to consider referrer page counts in the context of views of those pages.

Corporate data

Bridget spent most of the week working on the upgrade of Dynamics CRM 2011 to Dynamics 364. There are plans to merge the three instances used by POST, Participation and the House of Lords Enquiry Service into a single instance on what we are told is an ‘evergreen platform’. She had a productive meeting with our third party supplier and is putting finishing touches to a data migration framework for the project. She’s hoping the migration framework may be of use to other projects, and to tap into our Liz’s skills for data profiling.

Paul’s truly terrible week (slight return)

Paul did even more strategy-wrangling because there is no strategy for interlinking our assorted strategies. He also attended his first Data Day and, I believe, stayed to the end. And he went along to a workshop run by Alison and Bridget with Janis McAnallen and Andy Woods from Strategic Estates. Luckily he’d had the foresight to book Friday off work: his head is now wrecked. Still, none of his teeth fell out. So that’s an uptick in fortunes.

Customer service of the week award…

No award this week. Nobody has been good enough to justify the cost of printing the badges.

Strolls

Robert and Dan took a stroll to the finest coffee stall in SW1. Robert and Michael did similar. There was a shortage of any real adventure and a lack of real spice in our lives.

Bridal news

Email spam from The Bridal File is still arriving thick and fast to the data@parliament.uk inbox. This has something to do with the website being blocked and the service desk not having unblocked it. We’re still hopeful this might happen soon.

Nobody got married.

Things that caught our eye