This week the government launched its new 'What Works' centres at an event hosted by Nesta.

As one of the ministers there commented, the remarkable thing is not that it's happening but that it hasn't happened before: you might have thought that government would want to know what works. I'm a strong advocate of greater visibility and authority for evidence. It's a necessary condition for more competent government, and there is still much more to be done to encourage intelligent production and use of evidence, not just by policy-makers, but also by the professions involved in delivering public services.

But evidence is only part of a system of more intelligent government. More rigorous analysis of what works has to be complemented by more experiment and creativity to find out what could work better. That's why we're going to follow up our work on the Alliance for Useful Evidence with a strand of work looking at the meaning of experimental government.

In essence this is the idea that, wherever possible, governments should test new ideas - preferably on a relatively small scale - before spreading them. Not all the methods used in medical science (like double blinds) are directly transferable to public policy. But a surprising amount of the experimental method can be used in everything from the design of forms or websites to how police catch burglars.

Experimentation isn't new

Experimentalism sounds quite novel, but there is a very long history of explicit experimentation around the state. Chinese emperors talked about it openly (and Deng Xiaoping's commitment to facts, what works, and crossing the river by walking on the stones deliberately echoed that tradition). Four centuries ago in England, Francis Bacon advocated experiment in New Atlantis.

In the 19th century, there was much discussion of applying scientific methods to social issues. In Saint-Simon's utopia, a "Chamber of Invention" would develop ideas which were then to be overseen by a second "Chamber of Execution". The Fabians were explicitly experimentalist as was President FD Roosevelt ("The country needs, and unless I mistake its temper, the country demands, bold, persistent experimentation. It is common sense to take a method and try it; if it fails, admit it frankly and try another. But above all, try something"). Karl Popper gave experimentalism a theoretical foundation, presenting it as the mark of an open society that was willing to disprove its favoured ideas, and there have been many more recent advocates, including Brazilian theorist, and sometime minister, Roberto Mangabeira Unger, and social scientists such as Charles Sabel.

Less has been written systematically about the modern forms of experimentalism, but it's not hard to describe them. They include pilots of all kinds such as formal randomised control trials (RCTs), whether the very quick ones of the UK's behavioural insights team, or the longer lasting ones associated with medical treatments, welfare to work programmes and schooling. Pilots can create their own problems. For example, if they're given too much political prominence (so that they can't be allowed to fail), or if they take too long (so that their results come long after the political appetite has moved on). But the principle that ideas should be tested is a sound one.

The tools of experimentalism include many types of structure, such as the many design labs in and around government. These have their own network for sharing experiences on everything from the design of tax collection to redesigning hospitals, or consciously innovating more radical ways of organising services. We're currently doing a survey of these teams, in both national and city governments, to better understand how they handle the connections between structures, processes and cultures.

There's a related fashion for markets for outcomes as a sort of outsourced experimentalism. Payment by results, social impact bonds and the like are justified in part because they may promote more creative approaches than bureaucracies can, and with the advantage that they will quickly reveal which ones actually work.

Sometimes it doesn't work

However, not all fields are suitable for experimentalism. You can't easily experiment with one-off choices. Experimentation isn't possible with secession for example. I doubt that Scotland or Catalonia could ask to return to the UK and Spain if they became independent, and decided it wasn't working.

Values can stand in the way of experiment too. Some laws are seen to embody fundamental principles and might be desirable (or not) regardless of the outcomes they achieve (think of gay marriage, or restrictions on stem cell research). Some issues may just be too politically contentious for experimentalism to be practical.

These exclusions still leave a very large swathe of government activity that could be more overtly experimental. Of course experiments bring with them plenty of practical challenges - in particular how to handle downside risk. Their design needs to reflect the appropriate handling of risk, taking into account, for example, whether participants have been able to choose; the gravity of the risks of failure; or degrees of reversibility. As a result experimentation should take very different forms whether it's dealing with children in care, traffic light systems or the provision of public libraries.

There are also interesting challenges in the politics of experimentation. Some politicians worry that they can't easily justify failed experiments: but Roosevelt's formulation, quoted above, confirms that serious politicians don't find it too hard. The key is to be honest, in advance, about the fact that you're doing experiments and that some of them are bound to fail. There's also a fascinating set of issues around the relationship between experiments and ideology (how much can experiments be just about the implementation of ideological principles or stances?).

This feels like an idea whose time is coming, or at least returning. If governments want to know what works, they have to be willing to invest in finding out. That will require them to experiment. Over the next few months we hope to start a discussion with others interested in the field - academics as well as practitioners. Comments, ideas, references and ideas welcome.