weeknotes.data-search

Week 15: 03-11-2017

Driller killers

This week’s weeknotes are brought to you by the letter D for drilling and the number 3. Some people are trying to “fix” our building. This seems to involve a lot of drilling. And hammering. The relentless drilling started on Monday morning and thankfully stopped by about 10.30am. Then it started again. And didn’t stop. If you’re planning on visiting us maybe bring ibuprofen.

In unrelated news someone in the office managed to catch a mouse in a plastic cup. So maybe also bring a cat.

Health and Safety wise, we are golden.

In foot shooting news we also managed to accidentally trigger a denial of service attack on ourselves. So that’s an achievement unlocked.

Data Day

Wednesday was data day. Which was kinda nice. A bit like the old days.

Jamie came along and we discussed:

Community

Tuesday saw a visit by Susan Halford and Mark Weal from the Web Science Institute. They’re doing some research around “The Everyday Practices of Semantic Linked Data” using Parliament as a case study.

Jamie was interviewed by them and enjoyed reflecting on our work with external people knowing they would understand many of the tech and social / ethical points being made. Lots of dotted lines were drawn between BBC and Royal Opera House experiences.

Chris also spoke to Susan and Mark on Tuesday and found it fun and interesting. Particularly the questions that he didn’t know the answer to or the positions he couldn’t meaningfully justify.

Raphael gave feedback on some external data visualisation training materials. He also went along to both StackTech3 and an event about use cases of open data standards organised by the Open Data Services Co-operative.

One world, one web, one team

Dan had a bit of a lax week on the ceremony front. He sat in on a bit of the user research review which he liked.

He also met with one of the project managers to learn about plans for new business systems around legislation and read Fred’s report on ‘discovery’ for Library Research Briefings. Which was apparently both punchy and clear. There was a lot about search and some interesting proposals for solutions.

Sticking with Dan, he went along to the website product roadmap meeting. There is a lot on and it’s good to have some search work coming from the product team. He’s asked Ed from the user engagement team to help with the planning for future events.

Finally we need to buy something a bit complicated. Jaie the finance sorcerer was unbelievably helpful in under an hour. Progress then. More generally buying stuff is looking up. And we should get more self sufficient, thanks to Liz.

Domain modelling

Anya and Michael tried to add comments to their work around types of committees but it did not quite meet the Ed McCarthy test. They’ll try again next week.

Raphael returned from holiday and started work on integrating search hints into the API.

Robert did a lot of standing this week. And some forgetting. The search backlog review and sprint planning attracted a select group and went fine.

Liz educated Robert on various matters concerning search and showed him the insights application, which he found very useful.

As ever Robert spoke to colleagues within PDS and in the business about stuff like computers and Parliament. Friday was meetings with Hansard, the House of Commons Library, Data and Search, and a review of search feedback led by Liz. Mail and calendar were less well attended to.

Mike has been working on the migration of the Search and Indexing triple store to the cloud. The good news is that all the systems we’d not previously tested worked with the new set-up. The bad news is that the network port, which needed to be open for new content to reach the search index, was closed. He’s raised a high priority issue with the network team for this to be opened. In the meantime, changes have been rolled back, but once the port is opened we’ll be able to change over in a matter of minutes. As soon as that happens we’ll be making a 92% saving on hosting costs. Which is good news.

Measuring things

Sara has defined the baselines for the measures we are going to track during the A / B testing of search. She used power analysis to estimate the sample size and duration of the experiment.

Liz showed some initial work on House of Commons Library workloads to the heads of sections. Making the model available to them to work with will allow greater flexibility over reporting, hopefully reducing time taken to report standard measures and ad-hoc requests, as well as providing greater opportunity for analysis. The reporting demonstrated information which they can currently run and export from the built in fixed reporting structure, as well as examples of alternative views and measures. The group seemed OK with this approach to reporting and the flexibility it provides. The model will be expanded to include members information and Research Briefings data.

Liz also spoke with Fred about supporting the next phase of Research Briefings work, and will keep in touch with him about tying together the data on briefings with the existing data on workloads.

Samu whizzed up a neat solution to not having regex support in dashboard queries. He’s built a way of processing search terms and grouping them according to criteria we specify. Liz is very happy about this and would like to say thank you again Samu.

The House of Commons board were positive about a paper Matias and Dan contributed to and now we have to make some dashboards.

Corporate data

Matias finally went live with a data integration that ties the ticketing system for Parliamentary visits to the finance system. The integration itself was developed in 2015 and the code was live but various hiccups meant we’d never turned it on. He made minor changes and deployed the new code. It is now working.

Machines that do learning

Ignorant or lazy. We’re not sure.

Capability

Robert joined Julie for a capability planning session.

Strolls

Robert and Michael took a couple of strolls around Westminster. None of the world was put to rights.

Things that caught our eye