weeknotes.data-search

Week 10: 29-09-2017

Community

Wednesday saw a visit from Benjy Stanton of the ONS. There was talk of potential collaboration on shared tools and lots of thoughts on a clubbing together of public sector organisations who publish things.

Dator day

Wednesday was dator day. For the first half it took too long to agree website work and there were a few unnecessary disagreements. That said the outcome was constructive and we agreed some stuff:

Digital service quarterly event

Dan spoke twice at the digital service quarterly event about the new data and search services and their relationship to the website and the web in general. Edward (who is our new user engagement person) helped out by driving the tabs. He did great. Dan did ok. Fewer laughs in the afternoon, though the ‘dangirus doggs’ search is a really good illustration of one way in which new search beats every other Parliamentary search out there.

Parliament in the area, we’re international, we’re continental. But we’re north of Glasgow

Anya, Silver and Michael travelled to Stockholm for the Euro IA conference. Anya and Silver gave a talk on Domain Driven Design at Parliament. Michael attempted to answer some questions. It seemed to go well though they were glad when it was over. Beer made nerves unfrondle.

One world, one web, one team

Dan managed to make 3 website stand ups this week.

Domain modelling

Anya, Silver and Michael continued work on modelling government departments, positions and incumbencies. The model is now published. Feedback is, as ever, welcome.

Michael met with Chris to pass on work so far around the government model. He also met with Colin to run through much the same stuff.

Data quality

Samu designed and Jianhan implemented a space on our internal network where people can easily make changes to data before it moves into the data service. This is accessible both to non-techical users through a somewhat friendly browser interface and to technical users doing batch operations. We’re already making use of this new facility to make data quality improvements to data about Members’ website links, photos, and committees. This interface will also be used for the “Lords Inflation” work. Don’t ask, really don’t ask.

Work continues on adding website and social media links to members. Again using the new interface. The data is now ready, so we’re expecting much collaborative work between the teams over the next two weeks. Thanks to Callum for help with importing the data, Jianhan for setting up interim storage and editing facilities, Chris for creating mock data in our API so website development can proceed without waiting for data, and Wojciech for writing import code at record speed.

Aidan is chasing the Ordnance Survey about constituencies going missing from regions, postcode grey zones, and Northern Ireland postcodes and constituencies.

Data platform

Wojciech (working with Matt Rayner and Sarah Allett from the website team) released a new version of the data platform infrastructure. A casual visitor to beta.parliament.uk would not notice that anything’s changed, which means we’ve done a good job. Behind the scenes, however, this release:

Samu spent time with Sean Brazier to conduct a Service Assessment for the Integrated Workspace Management System (IWMS) project. This was the first session of a new format aiming to improve technology governance.

A quote from one of the project documents made him stop for a second:

“The current parliamentary estate measures approx. 222,000 square metres and consists of 34 buildings, properties and an underground car park. There is a total of 227 floors and 9651 rooms, when consolidated, across all of the buildings.”

Michael and Robert wrote something about what we’ve done with the new search and why and assorted advantages and disadvantages. Anya added grammar. Silver removed a bit that was unintentionally rude to librarians.

The search team have been looking at how we better label search results according to document type. From what was a vague idea we now have a working prototype. The work will be considered for production soon.

To enable this Raphael picked Ed McCarthy’s brain to understand rules and exceptions around URL structures of publications which varies for current and archived materials.

Robert helped Dia with preparing a presentation for the Digital Strategy Board.

The team met with Fred to talk about library Research Briefings, particularly with regard to search. They came to the conclusion that it will take some time before we see most of what he described, but the broad aims were in line with those of the data service.

Robert is also catching up with Archives and Library folks on plans for search in the medium term.

Dia is also working on a feature list for search based on the feedback so far.

Tony wrote an excellent post on options for search result display.

On Friday, Dia presented search work to the digital strategy board. It went well.

Machines that do learning

Raphael went along to the GDS data accelerator course and learned about how hyperparameter tuning can help with the toolkit for the Indexing and Data Management Section. He also worked with Angela on her application to join the next cohort.

Measuring things

Liz met with Steve and Trine to discuss collaboration around research and analysis. She also met with Nik on measurement and analysis for product teams. These were effectively the same, positive, conversations on embedding good practice and principles.

Liz requested a session with the search product team to plan A/B testing for resurrecting the display of URLs in search results. This went well. We now have defined measures and decision criteria for assessment of whether they are a bad thing. Or not.

Corporate data

Matias met with Charlotte (head of the Enterprise Portfolio Management Office) to review the dashboard / report work he’s been doing with Sam. The main concerns were how to be sure the sources could be trusted, how we could reduce text displayed and how to display trends especially regarding dates. Matias suggested they provide 3 KPIs as examples so we can try to produce them with the current data set.

Documents from data

Tony and Michael met with Gordon Clarke and Luanne Middleton from committee land to talk them through some of the work Tony’s been doing around reproducible research using Jupyter Notebooks. This seemed to match at least some of their thinking. Tony is planning to build a demo of a committee report and a library briefing in this fashion.

Did anybody say blockchain?

To the best of my knowledge nobody said blockchain \o/

Did Samu recommend any music?

Samu recommended the Prophets of Rage. Michael looked a bit baffled.

Strolls

Anya, Silver and Michael took a number of strolls through Stockholm. This was nice.

Things that caught our eye