About Nesta

Nesta is an innovation foundation. For us, innovation means turning bold ideas into reality and changing lives for the better. We use our expertise, skills and funding in areas where there are big challenges facing society.

Standards of Evidence in UK social policy – what are they and how are they used?

When we started this research, I assumed that standards of evidence would all look relatively similar. I could not have been more wrong. Although there are common principles underpinning them, particularly the shared goal of improving decision making, they often ask different questions, are engaging different audiences, generate different content, and have varying uses.

Our analysis segmented the standards of evidence into three groups based on how they are used by their host organisation:

  • The first group is standards of evidence that can be used to inform the host organisation’s strategy and funding decisions. An example, is the National Institute of Health and Care Excellence’s (NICE) GRADE framework which is used to help develop the clinical guidelines used in the NHS.
  • The second group is standards of evidence that to make recommendations to the wider field. An example in this category is the What Works Centre for Crime Reduction’s EMMIE Framework. The EMMIE framework summarises the best available research on what works to reduce crime. The What Works Centre for Crime Reduction uses EMMIE with the primary goal of making recommendations to the wider field on how effective different interventions are at reducing crime. It does this by presenting findings in an easy to access and understandable format.
  • The third group are those standards of evidence which provide a resource for organisations developing interventions. An example is Project Oracle’s Standards of Evidence. Project Oracle supports organisations to understand and assess the evidence behind their interventions. This review, or “validation” as Project Oracle term it, doesn’t endorse the organisation or their work, but instead uses the standards of evidence to assess the plans and evidence in place and to understand whether the evidence submitted is of a sufficiently high quality.

As these categories demonstrate, the standards of evidence used in UK social policy are very different. It is understandable that differences may reflect the practical goal of the host organisation. However, there is a need to consider more philosophical and theoretical tensions about what constitutes good evidence particularly as differences in opinion can foster confusion. In our research, we came across examples of different organisations reaching different conclusions about the same intervention; one thought it worked well, and the other was less confident. Who is right? Does the intervention work, or not?

One suggested response to minimise confusion is to develop a single set of standards of evidence. Although this sounds inherently sensible, our research has identified several major challenges which would need to be overcome to achieve this.

We have mapped the landscape of standards of evidence after in-depth discussions with providers - the ‘supply side’. Our next steps are to facilitate knowledge sharing and to explore the feasibility of creating a single set of standards. As part of this, we are keen to understand the ‘demand side’, engaging the wider field in a conversation about how to make standards of evidence as useable and useful as possible. We are forging partnerships internationally, such as with the OECD, to share best practice more widely.

We are very keen to involve organisations who have used standards of evidence in their work. If you would like to share your experiences of using standards of evidence, or would like to know more about this work, please contact us at [email protected]

This blog was originally published on www.alliance4usefulevidence.org

Author

Ruth Puttick

Ruth Puttick

Ruth Puttick

Principal Researcher - Public and Social Innovation

Ruth was Principal Researcher for public and social innovation. Ruth joined the Nesta policy and research team in 2009, working on a range of projects across innovation, investment and…

View profile
Jonathan Breckon

Jonathan Breckon

Jonathan Breckon

Director, Alliance for Useful Evidence

Jonathan was the Director of the Alliance for Useful Evidence from 2012 to 2021.

View profile