Have you ever looked at an impact report from a charity, funder, or impact investor and thought that it was just a bit too good to be true? Sixty thousand jobs created! £100 of public money saved for every £1 spent! You are not alone. At our roundtable on impact in impact investing at the end of last year, the credibility of impact reporting was raised as a major issue. If closely scrutinised, would claims of impact by fund managers stand up? We'd like to believe that most would, but if any are found to be exaggerated, that could undermine the reputation of the whole impact investment industry. Over-claimed impact could be the next financial reporting scandal.

One reason that these large numbers can arouse scepticism, even when they are genuine, is that the effect of impact investment is yet to be observed on a national or even regional scale. If so many jobs are being created and so many affordable houses being built, why do we still have youth unemployment and homelessness? Our common sense tells us that lots of small scale successes should, at some point, add up to large scale, observable change.

So if there is so much impact out there, why can’t we see it? Two big reasons come to mind (though of course there may be many others):

1. The problems we are trying to solve are actually really really hard. In the UK alone, there are over 20 million people suffering from depression or anxiety, 281,000 people are living in temporary, insecure accommodation and over half a million young people are unemployed. And it is not simply a question of scale; these problems are deeply entrenched, often resisting years of government intervention and sometimes exacerbated by it. So even though local successes are likely to be real, they may simply not add up to a big number relative to the problem.
2. Big numbers hide a multitude of sins. A headline number may mix together the achievements of a high intensity employment programme helping the most marginalised with a light touch advice service helping many people who may have found jobs anyway. Headline numbers also communicate what is most easily aggregated, it is easier to count jobs than it is to assess their quality or suitability.

Both of these reasons call for humility on the part of impact investors. If we want people to believe our numbers, we need to put them in the context of the problems we are trying to solve, setting what we have achieved against what we hope to achieve. And we must be willing to open up our impact data beyond the big numbers, so that we can learn from the detail. What failures are we masking? Where is it that we have added the most value?

At Nesta Impact Investments we are experimenting with an ‘impact audit’ approach to increase the transparency and scrutiny of our impact reporting. And with transparency and scrutiny comes, we hope, credibility. At its best, this could be a cost-effective way of holding ourselves to account and pushing us to improve. We are in the middle of that process now and will share our experience once it is complete.

Big numbers can generate scepticism, but they can also inspire and energise, recruiting people to the cause of social change. We should not be shy of ambition. The shift in capital flows towards impact investing is just beginning, and there is significant potential for that capital to help deliver measurable change on a larger scale. However, we must hold ourselves to a higher standard in our messaging around impact. If we want people to use our impact data to make decisions about how they allocate their capital, then we need our evidence to be robust and meaningful, not just headline-grabbing.