Dan did a skype thing with the Portuguese Parliament as part of ongoing efforts to help out. They’re looking to build their data visualisation capability. Henry (ex of this parish) sent on some great links.
Dan seemed to spend most of the week talking to product teams. And a fair chunk talking to Jamie. He went along to a session with Fred and the Research Briefings team which was actually rather good. He also went to a group chat that Sharan set up to gather ideas for the parlidigital Twitter account. Which was something different.
Dan went along to the website roadmap meeting. And lots of people came to our show and tell where Chris talked about integrating content storage into the data platform. So that’s good.
Anya left the library to travel to computer land and talk to assorted product people about the Indexing and Data Management Section and the work they do to subject index parliamentary material. Michael chipped in with strong SKOS reckons. A fun time was had by all.
Anya and Michael met with Steve Wise, Anya’s boss man and the Director of Information in the House of Commons Library. They had a good chat about new search and its reliance on well planned information architecture. The trick here is still hypertext. Everything in its right place, at persistent URIs, subsidiary resources exposed and linked to. They also chatted about super specialist queries and the planned SPARQL endpoint.
Anya, Liz Marley and Michael met to chat about the TIPS meeting they’re hosting next week.
Our Liz talked to the Guide to Procedure team about how they plan to monitor the thing they’ve built. They have a problem with some of their measures where they don’t have consistent comparisons. Or comparisons at all in some cases. They talked through prioritising what to measure and looked in detail at three of the ideas. Whilst brain storming measures was a good initial approach, it’s important to understand the practical details of collecting data and assessing change. The three initial ideas were reduced down to two. They also talked about measuring failure below a certain point, rather than success above a certain point, where previous data isn’t available to judge where to set the bar.
Anya and Michael met with the Principal Clerk of the House of Commons Table Office to talk about smushing down round robin questions into a single question going to multiple answering bodies. For assorted reasons this is not possible. At least for now. They’ve updated the question and answer model to reflect that decision.
Anya, Silver, Ben and Michael spent Tuesday doing some more wrestling with statutory instruments. They’re coming to the conclusion that the basic model is pretty simple: just some routes over stepping stones. Hopscotch got mentioned. The problem is those steps touch on pretty much everything in Parliament from motions and amendments to debates and committees to divisions and reports and etc. So it’s basically easy. But almost impossible.
On Thursday Anya and Michael took a kind of break from Parliament and decided to do government for a change. Ganesh and Tim from GDS / the Cabinet Office came over to talk about objectives and targets and policies and interventions. A picture was drawn. They think it is OK.
Jianhan worked with Jeanette to get Lords’ photos into the data platform. They’ll be available on the beta website shortly and, like MPs’ photos, they’ll be openly licensed. Prepare to fill more boots.
Jianhan also released a new version of the OData service to reflect recent changes in the ontology. And he’s been working with Sarah to get more committee data into the platform. This new data is also available in the OData service.
Jianhan and Samu uncovered what they think might be a bug in the library we use for creating the OData service. They’ve written some code to reproduce and might file a report once they’re a little more confident.
Raphael created an automated import process for Parliamentary period data, which hitherto had been hard-coded. This includes the Wikidata IDs kindly supplied by Andrew Gray. Which Michael is still extremely excited about.
Matthieu, Mike and Samu gained access to the current hosting infrastructure for Historic Hansard. They’re working on plans to migrate it to a new home on our data platform, in a similar way to data.parliament and Indexing & Search.
Samu and Wojciech spent some time (re)naming things. We periodically review our API endpoints, code namespaces and package names to align the naming guidelines with our evolving understanding of the service we’re building. Here’s what they came up with:
api.parliament.uk/query - previously called the fixed-query API, these are the endpoints that supply the new website with data
api.parliament.uk/sparql - not yet public, this provides a raw query endpoint for those in the need and the know
api.parliament.uk/search - as the name suggest, this makes the new search service work
api.parliament.uk/photo - Members’ and Peers’ portraits
api.parliament.uk/odata - a standard interface that allows all our data to be pulled into spreadsheets for example
Wojciech and Liz looked at a new package for exporting our analytics data to a new blob store. Wojciech set up a test query showing page data and sent it straight to our visualisation software as a dataset. Other options are being explored.
Matthieu and Samu managed to install some new SKOS management software on our cloud infrastructure and import the current controlled vocabulary. They also came up with a longer term plan for migrating the vocabulary whilst not interrupting the authoring and indexing work that relies on it.
Samu, Wojciech and Mike migrated data.parliament to its new home in the cloud. Seven virtual machines were replaced with two service plans. Which means it’s about 85% cheaper to run. They also replaced some of the parts that were acting as bottlenecks. Content now reaches our external database 27 times faster. We’re using the latest versions of all the software including the same version of the triple store as the new data platform - but with many more statements. So we’ve learned a bunch of stuff about big repositories and running a triplestore in an app service that will help us in the future . We no longer have to use a custom cache to cope with the current load on the services and integration with analytics gives us better monitoring, better logging, and etc. Because the new instance is in our own subscription, we have full control over everything there, no more asking (and paying) a 3rd party to do things like: scale up disk sizes, VM specs, open firewall ports, handle user access, etc.
Matt continued work on the House of Commons HR system. The new integration is currently being tested with live data. A data comparison is being done to verify the amount of records being amended by the feed.
He also worked on the asset management system and tweaked the cost centre feed. All the other feeds (people details, asset usage) have now been successfully tested.
Noel got his hands on job information from the House of Commons HR system. He’s now analysing this against data in the central pot to identify differences and understand why they exist. He’s also provided a list of active users and their departments to the people responsible for our subscriptions to mailing and newspaper platforms.
David has been implementing the new facilities person interface and is currently working on the organisation interface.
Lewis continued work on a new finance and stock integration by developing and starting to test the new invoice data flow.
…this week, like every week, goes to Mike who helped a commercial third party wanting to use data around Research Briefings. Maybe they’ll switch to the data.parliament API instead of screen scraping the website….
Hopefully, not so much.
Whilst we did not witness this, we’re fairly sure it must have happened. Anya and Michael did manage to increase the population count of people who now know Parliament != government by four. Which is a small achievement. But not unrewarding.
Yes, Michael did get headhunted. Unfortunately it was for the job he does in the organisation he works for, but, given no-one has ever expressed an interest in the past, he’s still rather thrilled.
No strolls were reported, though Robert did go round to Dan’s for dinner. Dan managed not to poison anybody. Anya, Ben and Michael are all looking forward to lunch with Robert tomorrow.