open data consultation event

A couple of weeks ago, I spent a Monday afternoon at a stakeholder event hosted by the Cabinet Office’s Transparency team on their Making Open Data Real consultation, with an interesting crowd of developers and users of public data products. We spent three hours pondering our thoughts on how greater use of open data might benefit accountability, business, and the public sector itself.

Francis Maude’s preface sets the tone of the consultation:

Fundamentally, we want to be open about what we do. Open about what we spend. Open about how public services work. Open about making them better. And so we propose reform of the whole of the public sector along open, transparent and accountable lines. What we are doing is not just a first for Britain; these proposals represent our determination to be the most ambitious Open Data agenda of any government in the world.

To me, transparency doesn’t get me as excited as open data (and still less than open government in the US sense), so it’s a shame to see transparency and open data used pretty much interchangeably. Of course scrutiny matters – and there are plenty of reasons why it’s good to know how much government spends, and with whom – though I think transparency of those government processes matters less than politicians think it does.

As an illustration of this, take the open data set of civil service jobs. A couple of years ago, the Cabinet Office created an excellent API to their searchable listings, which I used to build a simple proof of concept app, enabling anyone to sign up to an RSS feed or email alert of vacancies at a specific organisation. I built it into a commercial service I’ve run for a few years now – Government Jobs Direct – and built up a database of several thousand jobhunters signed up to alerts.

Then one day a few weeks ago, it all stopped working. The Civil Service site revamp hived off the jobs site to an external portal¬†provided by WCN Recruitment, commissioned by MOD/HMRC. Sans API. Sans, indeed, even the ability to link to a list of jobs from any given organisation, which hasn’t gone unnoticed.

As an API key holder for the Government Jobs API (and the lesser known Public Appointments API, which also seems to have vanished), I might expect to receive an email giving me some notice that the service was being closed down, right? The way that Google announces changes to its data services? But the first I heard was when the service stopped working, and thousands of people stopped getting their email alerts.

To be fair, I’m pretty sure this is an oversight, rather than a conspiracy. I also hear rumours that a new like-for-like API may be on its way. Certainly, there’s no sign government is watering down its enthusiasm for open data or APIs. But I’d argue that running small-scale but commercially useful open data services like job listings maybe needs a higher priority than it’s getting now. I suspect there’s a real difference in the likelihood of commercial reuse when the data issued comes in the kind of well-packaged, REST-based API with solid documentation and quality hallmarks (like an API key), compared to a CSV file dumped on a datastore. I made a commercial decision to invest effort in consuming and integrating the API, created a public service from it which also generated some commercial benefit for me, and on which I paid tax back to HMRC. It’s a great and virtuous circle, and one that Government needs to give a higher priority to than the ‘transparency’ data which doesn’t have the same commercial applications.

Today is the last day to respond to the consultation Рyou can nip in and leave a comment on any or all of the questions, or you can submit responses in the traditional way.

Get notified of new blog posts by email


More and more I’m thinking that Open Data is nothing to do with Data, and everything to do with how we encourage innovation – how we actually build a future based on it all.

This feels like a great example of how we perceive “success” – i.e. uptake in an attention-based economy. When many eyes are on a service, or many scripts are using it, the need for communication is not just desirable, it’s a must-have, because large-scale moaning when something fails is *public* failure.

But innovation doesn’t work like this – small numbers of small-scale dabblers always have to make the first steps, and these are the people that need the most support. Things don’t just launch, and then get used widely a day later. There has to be an attitude and a culture to support the early-adopter.

Services which don’t consider themselves to have a “mainstream” audience seem naturally better at understanding this – things like the NeSS Data Exchange may not always get it right (which is fine), but are very open about communicating both changes and problems that crop up. That’s nothing to do with the data, or the API, and everything to do with user-engagement.

Personally, I’d like to see the same attitude from government across the board – from Open Data industries, to emerging technology, green tech, social enterprises, etc etc etc. Merely applauding but otherwise not supporting (somehow) initial pathfinding efforts is pretty dire.

Gaaah – I can’t believe they pulled the API without telling anyone. I assume as a key-holder they have your email address, so it can’t even be that they couldn’t find you (and others) to contact about it.

I’m more suspicious – and would suspect that someone noticed the success of your portal and decided to do their own – and I can’t see WCN recruitment being wild about exposing an API to their (seemingly less good) offering…