Which new technologies will deliver the highest future growth? Which skills and capabilities should businesses invest in if they want to innovate? Which R&D projects will generate the greatest knowledge spillovers? And which innovation policies work and which don't?

The reality is that policymakers don't know the answers to these questions (and if they did, they would most likely be starting up new businesses not devising policies to help them). The uncertainties mean that even the smartest policymakers working on innovation usually do so on the basis of near ignorance, and yet governments everywhere invariably act as if they have all the answers. What approach should they in fact follow?

In a 2011 Nesta pamphlet, State of Uncertainty, Alan Freeman, Jason Potts and I proposed that the government should develop an approach to innovation policy that embraces, not ignores, these extreme uncertainties. The very aim of innovation policy should be to find answers to some of these fundamental questions. "The goals of innovation policy need to be led by research and learning priorities... Government should stand ready to scale up or down the experimental inquiry, perhaps significantly, depending on emerging findings."

We coined the term the 'experimental state' to describe this conception of innovation policy, and pointed to policies like challenges, testbeds, networks, charter cities and experimental funds as the preferred means of intervention by this state. This contrasts with alternative accounts such as Nesta Fellow Mariana Mazzucato's 'entrepreneurial state' which emphasise government-led investments in new technology.

Turning the idea of evidence-based policy on its head, we suggested that data from policy experiments should be one of the main outputs of, not just inputs to, state intervention. A key function of innovation policy then is to disseminate these data for the wider good, as well as to ensure that they are fed back into policy design. This dual emphasis on open and iterative learning, we suggested in State of Uncertainty, is a hallmark of some of the world's most dynamic innovation systems.

Such arguments might appear abstract, or too theoretical to be of use to ministers. However, today we publish an example - an industrial policy measure to boost innovation in SMEs - which illustrates in concrete terms just the sort of thing we have in mind. Creative Credits: A randomized controlled industrial policy experiment uses what my co-authors and I term a 'randomized controlled trial plus (RCT+)' methodology to establish the effectiveness of 'Creative Credits': a business-to-business voucher mechanism designed to encourage SMEs to innovate in collaboration with creative service businesses.

In the Manchester pilot, 150 SMEs received Creative Credits worth £4,000, which they could use to purchase a variety of creative services from local creative businesses of their choice. The demand-led nature of the voucher mechanism gave Manchester's policymakers a timely snapshot of innovation needs as identified by businesses; the randomized allocation of the Creative Credits allowed the scheme's additional impacts to be rigorously evaluated (by comparing the performance of SMEs receiving the vouchers with those that didn't); the longitudinal data collection allowed the longer term as well as short-term performance of businesses to be tracked, and the use of qualitative as well as quantitative research methods gave rich insights into why some projects failed and others succeeded.

We found that Creative Credits created genuinely new relationships between SMEs and creative businesses, with the award of a Creative Credit increasing the likelihood that firms would undertake their project with a creative business by over 80 per cent. It turned out that the clear majority of innovation projects chosen by SMEs were website-related in nature.

The experiment also provided striking evidence for how working with creative businesses can generate near-term commercial benefits in terms of innovation and sales growth, but how in the longer term the benefits appear to dissipate when partners view their project as one-off or 'transactional' in nature or when there is lack of agreement on the project objectives.

Following the logic of State of Uncertainty we are publishing the detailed results from this policy experiment in the hope that they can be used to improve business understanding of the opportunities and challenges of working with creative businesses, and are also making available the programme materials to help design future innovation voucher schemes such as those currently being developed by the UK government and in the rest of Europe.

Hasan Bakhshi is Director, Creative Economy in Nesta's Policy & Research Unit and Research Fellow, ARC Centre of Excellence for Creative Industries and Innovation at the Queensland University of Technology.