How do you measure a programme of government reform? What counts as evidence that it's working or not? I've been asked this question many times, so this very brief note suggests some simple answers - mainly prompted by seeing a few writings on this question which I thought confused some basic points.

Any type of reform programme will combine elements at very different levels. These may include:

• A new device - for example, adjusting the wording in an official letter or a call centre script to see what impact this has on such things as tax compliance.
• A new kind of action - for example a new way of teaching maths in schools, treating patients with diabetes, handling prison leavers.
• A new kind of policy - for example opening up planning processes to more local input; making welfare payments more conditional.
• A new strategy - for example a scheme to cut carbon in cities, combining retrofitting of housing with promoting bicycle use; or a strategy for public health.
• A new approach to strategy - for example making more use of foresight, scenarios or big data.
• A new approach to governance - for example bringing hitherto excluded groups into political debate and decision-making.

This rough list hopefully shows just how different these levels are in their nature. Generally as we go down the list the following things rise:

• The number of variables and the complexity of the processes involved
• The timescales over which any judgements can be made
• The difficultness involved in making judgements about causation
• The importance of qualitative relative to quantitative assessment

In judging a programme of reform it's therefore vital not to mistake what is being assessed. So in the earlier examples - new devices or actions - evaluation can be very rigorously quantitative and often fast. There is no excuse for not measuring impacts; not using plenty of control groups; not feeding results fast into decision-making. 

For the later examples - that are more strategic - all of these tools for assessment risk becoming vices rather than virtues. Then you need complex judgement; a sense of history and context; and above all, the time and space to interrogate those judgements. The conversations prompted are likely to be as useful as any written outputs.