The field of big data has been a triumph and also a disappointment. It's a triumph in that thousands of data sets have been made open to the public; that new industries have grown up around reuse of that data (particularly in fields like transport); that some forms of abuse and corruption have been revealed and contained; and that it's quickly become conventional wisdom that (anonymised) public data should be open by default.

So why the disappointment? The most immediate reason is that relatively little use is being made of all the free data - this is a triumph of supply more than demand, at least so far. It's the latest in a long line of projects driven by clever engineers and technologists who became understandably captivated by what was technologically possible, but were not so good at understanding what problems would actually be solved, or how the technologies would interact with wants and needs.

The deeper reason for the disappointment is that almost no public services and no significant parts of governance are being seriously reshaped using digital tools. Instead they are add-ons around the edges, whether for health services and local government, or parliaments. In short, the world has discovered that open data is nice to have, but not the transformative game-changer some thought it might be. That no one is feeling very threatened is the give-away: everyone can be quite safely in favour of more open data because there's little threat to their interests.

So what's to be done? I've previously argued that funders and orchestrators need to pay much more attention to the use being made of data, apps, web tools and the like, and to shaping tools so that they're useful. I also argued that we need stronger intermediaries and better structured markets to link supply and demand (indeed all the things that turn out to be crucial in other fields of technology but are usually less visible than the glitzy inventions).

The more fundamental, and much harder, challenge is to transform whole systems. At Nesta we've been looking a lot at the lessons of systemic change and innovation. To accelerate systemic change that would make the most of the radical potential of digital technologies we need four things to happen in tandem:

1. Imagination: clearer articulation of how whole systems could be different - for example:

• health systems enriched by knowledge, orchestrated and organised in what we call 'knowledge commons', with finance shifted to outcomes, and more power in the hands of patients and primary care, more networked solutions.
• transport systems incentivised for flow or speed, with integrated information and payment systems, and fluid interchanges.
• budgeting processes opened up to systematic deliberation, argument, and simulation of alternatives - all feeding into legislatures and governments.

2. Experiment: agencies taking responsibility for trialling elements of the new systems, including places acting as laboratories of the future (something we are working on in relation to ageing); commissioning of technologies (eg. for co-production in healthcare); trialling of business models; and formal experiments.

3. Entrepreneurialism: innovators and entrepreneurs trying out new ideas, aligned to the bigger systems change they want to be part of (for example, in fields like food or edutech).

4. Leadership: political will to connect the top down systems designs to bottom up creativity and entrepreneurialism.

None of this is particularly complex, and it has happened in other fields (such as waste). But it's not being done using digital technologies - anywhere. If anything the energy around open data may have distracted some of the best minds from the bigger challenge. Nor have the craft skills needed for effective systemic innovation been properly cultivated. Yet this is probably the most exciting field to be in today - and with the fiscal crisis worsening, this is the only way to discover radically higher productivity models for achieving public value.