weeknotes.data-search

Week 5: 25-08-2017

Community

A reminder that our 4th Parliament, data and democracy meetup is on Thursday 14th September at Newspeak House.

There will be interesting talks, nice people and free beer. You should probably come.

Capability

Raphael went to GDS for the mid-point event in the data accelerator programme. He heard about progress of data science projects from other attendees and got feedback on a diagram he drew of implementing our machine learning and retraining pipeline.

Domain modelling

Anya and Michael went back to visit Ganesh at GDS. They chatted to a policy person about the flow of a policy from party manifesto through (possibly) to Parliament and (possibly) legislation. Came to the conclusion that what looks like a simple step flow on a diagram is probably some kind of insane Sankey diagram. So went to the pub.

The URI is the thing

Michael spent time with Colin, Angela and some post-its to design a first draft of the URIs we’ll need to support committee membership. Michael forgot to document this. Michael will document this. Soon.

Data service

Chris implemented a solid if very basic select committee / formal body membership model in the physical ontology and created a mock API for the website team to develop against. He also attempted to progress the select committee / formal body membership model to allow for chairs etc and ran into a long-fomenting name clash on “Incumbency” that needs some real work so will have to wait.

Samu was encouraged to publish his triple visualiser as a chrome extension for the purposes of fun and profit.

Mike had Alex, our new operations person, in and we talked to him about our work.

Jianhan continued with experiments to pipe data from internal databases to our data store on the cloud. He set up a pipeline for House of Commons Library enquiry data. This allows for direct database access unlike most of our other data ingest processes that rely on the presence of an OData API [warning: clip art].

Jianhan and Angela worked with colleagues from the House of Lords to further clean up committee data in the Members’ Names Information Service (MNIS).

And Jianhan and Michael had a chat about how we link up Select Committees to the Government departments they scrutinise. This was found to be hard.

Raphael troubleshooted an issue with the triple store workbench, the tool used by the website team to query the triple store.

Robert had an action-packed, if slightly short week. He met with Dia and Rachel from the communications team to discuss comms for the search roll out. And again with Dia on updates to the search release timetable.

There was a good hour spent on planning the next search sprint and a meeting between Robert and Samu on future search development.

Robert worked with user research colleagues on search research questions.

Matthieu coordinated with the website team for the search move to live.

Measuring things

Sara created a dashboard / report to monitor API performance on search requests. She also started working with Saffiyah on the Library enquiries model. They’re linking tables and creating measures at the moment.

Sara added data.parliament.uk to the analytics store which will allow us to analyse the raw data and track individual user journeys.

Lewis completed the spreadsheet to XML pipeline allowing us to process spreadsheet data directly in the service bus. This is about to be used to process information from our service incident database allowing the performance analysts to get more information on cases.

Project (and indeed programme) management office dashboards have been created allowing navigation from a general overview, to details on a programme, to details on projects within a programme.

Sara, Saffiyah and Matthieu had a good chat about the current dashboard software.

Strolls

Robert and Michael went for a long walking meeting round Kew Gardens. Because they felt they deserved it. Much chat was had on data models and hypertext and search and browse and the making of websites. Being grumpy old men they agreed that everything was better in the old days when it was all more punk and before we got cursed with professionalisation.

Things that caught our eye