We’re happy to announce that we will be supporting Taftie’s new task force on experiments. We will help the network of European innovation agencies through a two-year long programme of workshops on experimentation. The programme will be a blend of capacity building, peer exchanges of ideas, and a platform for participants to launch experiments in their own home countries.

This week three economists won the Nobel prize for their work using experiments to tackle poverty. In a 2017 essay, one of them (Duflo) argued that when economists help design and deliver new policies, they should be like plumbers: bringing their knowledge into the real world whilst also ‘mindful that tinkering and adjusting will be necessary’. Her argument applies just as well to innovation policymakers: by focusing on the details, they can ensure new policy ideas work not just in theory but also in practice. At the Innovation Growth Lab (IGL), we’ve long argued that experimentation – as a method but also a mindset – is the key to better innovation policies, and through the IGL partnership we have been working with government agencies to provide them with hands-on support in becoming experimental.

That’s why we’re thrilled for a new opportunity to work with a group of innovation agencies to put this approach into practice. Last November, James and I ran a workshop on experimental policymaking at Taftie, Europe’s network of innovation agencies. There was a lot of interest in the room to apply the ideas discussed to policies back home.

A year later, we’re excited to announce that Taftie has decided to launch a two-year long task force on experiments - ‘Experiment!’ Participating agencies will come together in a series of workshops to learn about experimentation and will have the opportunity to apply the new approach to policies in their home countries. IGL will lead on the development and delivery of the workshops.

Over the course of two years, innovation agencies from across the Europe (as well as some from further afield) will come together for a series of workshops. The task force is structured as a blended programme including:

1. Capacity building covering the wide range of skills needed to experiment – building on IGL’s years-long experience of helping government agencies experiment;
2. Practical sessions in ideation, experiment design, and exploitation of results – all focusing on real life challenges faced by participating agencies;
3. Opportunities for peer learning – allowing policymakers to exchange ideas and concerns and learn from each other’s experiences;
4. Support disseminating the results – ensuring that the findings from the experiments are useful beyond the limits of the task force.

At the beginning of this month we held the kick off workshop, hosted by Luxinnovation in their beautiful Esch-sur-Alzette offices in reconverted steelworks. The workshop brought together a group of 29 from 17 innovation agencies over the course of two days. It was an opportunity to get to know all the participants and their expectations, and to begin discussing how they can become more experimental.

Not every agency in the task force is at the stage where they’ll be able to experiment on their interventions and processes, but the programme will ensure there is space for different levels of engagement – with some participants applying the lessons from the workshops to run experiments in their home countries, and others taking part as observers and learning from the process.

One thing we know for certain is that there is no simple step-by-step guide to becoming an experimental agency – it is a journey. That’s why we’ve designed our approach to the task force to follow the entire policy cycle – from understanding a problem to scaling solutions. This tracks Duflo’s point in her 2017 essay that policy ‘plumbers’ should be ‘more concerned about “how” to do things than about “what” to do’.

The first step (which will be the focus of the first two workshops) is to spend some time really understanding the problem. Often new policies are devised without spending much time figuring out what they’re meant to be solving – and what the real drivers are behind the issue.

An experimental approach requires using techniques such as design thinking to engage with those affected by the problem, uncover hidden dynamics and get a better sense of what’s happening (or not happening) on the ground. It also requires making the most of any information – expertise from practitioners on the ground, programme data, etc – to shed light on the crux of the issue.

That’s why we’ve devoted the first section of the task force to pick a real-life problem, and before even beginning to think about solutions, collect the necessary information to confidently diagnose the issue.

The next steps will focus on mapping solutions, and identifying possible experiments to test them. For the latter it is key to turn the initial ideas into hypotheses.1 This means not just trying out a new approach, but framing it as a scientist would: if I do x, then y will happen.

In later workshops, we will unpack how to design an experiment. The key is to ensure the experiments are robust – so that agencies can learn something with the confidence that the results are valid. At IGL we’ve traditionally focused on RCTs, and they will certainly play an important role in this programme; but for the task force we will also open up to a wider variety of experiments, including novel ways to use data to predict the impact of changes to programme structures. We will also spend some time thinking through how to measure the right thing – as this is a key challenge in the field of innovation.

The latter part of the task force will focus on what to make of the results from experiments. We will concentrate on how to make the most of the results – providing tips to ensure agencies learn from experiments even when the results don’t go as planned. We will also discuss how to think about scalability, and how to design experiments to learn about how a programme can be implemented at a larger scale.

Finally, the task force will delve into what it means for an innovation agency to become experimental. Running an experiment can be a great experience, but the journey doesn’t end there. Our aspiration is that participating agencies can find new ways to incorporate some of these lessons into their every-day approach to policy, so that experimenting becomes not the exception, but an integral part of policy design and delivery.

We’ve been asked to support Taftie agencies with our expertise in experimentation, but what makes this programme really exciting is that we will all have a chance to learn from each other – building on the work of participants to define a new way to do innovation policy.

The workshops will run into 2021, and we are keen to continue disseminating our progress and what we learn from the task force. We hope that in the coming months we will be able to share more news as experiments take off and new experiences teach us more about how innovation agencies can become experimental.

We will be running a series of blog posts to keep you updated on the progress of the task force. Subscribe to our newsletter or follow us on Twitter to get the latest updates!