What do ‘improving older people’s fitness’, ‘making pre-surgical risk-assessment smarter’, ‘helping children learn mathematics’ and ‘matching young people to apprenticeships’ have in common? The answer: they are all outcomes that businesses in our portfolio are aiming to achieve.

The challenge of adding up impact across a diverse portfolio was one of the big themes to come out of our impact round table in November. Simple headline figures were recognised as being very compelling and good for engaging people in the potential for impact investing to improve lives; however, there was a legitimate concern that these aggregated numbers tend to oversimplify and are not necessarily a good basis for decision making.

At Nesta Impact Investments, we believe in taking a tailored approach to understanding impact. For each business we invest in, we understand its unique theory of change or strategy for impact, and select indicators to track based on that theory of change. While we always seek measures that are robust and meaningful, we do not prescribe a predetermined list of indicators. We take this approach because small, growing businesses will not measure and learn from their impact unless they are measuring what is useful for them and their business.

At the same time, we have investors who want to understand the overall impact of our portfolio. How has their investment changed the lives of people for the better?

That leaves us with 14 different businesses with 14 different sets of KPIs and not one common measure that we can aggregate in a meaningful way. How do we square this circle? How can we be responsive to investees but also accountable to our investors?

Trying a new approach

We are currently piloting an approach, set out in our recently published impact strategy, that seeks to give a portfolio view of impact, without forcing businesses to take on measures that are not relevant to their core operations. We give each business an ‘impact return score’ based on:

• the number of people they reach
• their level of need
• the amount of positive change they achieve for their target outcome

The chart below shows where our portfolio is now, and where we would like it to be at exit in terms of this impact return score and potential risks to impact. This is an approach which tries to bring consistency and rigour to our understanding of the impact of our portfolio and of potential new investments. However, it is also highly qualitative and there is plenty of room for judgement. So, is it meaningful? We believe so. Of course, this return score is not a literal reflection of the truth, but it does help us to make decisions.

By comparing a potential new investment to our current portfolio, we can ask whether the expected impact is in line with our expectations for the fund. If an investment is very risky on the impact side, because of low levels of evidence or a complex theory of change, we can ask whether the expected returns justify these risks. It also allows us to set ourselves a baseline against which we can set targets and judge our performance.

One early insight from using this framework came from decomposing the return score into ‘depth’, which combines the level of need in the population reached with the amount of positive change achieved, and ‘scale’, which is the number of people reached. By plotting depth vs scale for our portfolio, it became clear to us just how quickly depth drops off with scale. This analysis clarified that an exciting impact proposition for us is not necessarily about huge scale or very high depth - it is about finding something that can reach tens of thousands of people that really need help, and showing some impact on their outcomes.

Is there an aggregation model that works for everyone?

There was consensus at the round table that aggregation of some sort would be useful. The reality is that we need to make decisions between very different types of investment, and we know that some investments will change lives more than others - although the factors affecting the level of impact are often subjective and may resist quantification. As Michele Giddens asked the room, is impact actually more like risk than return, composed, as it is, of several qualitative components rather than one number? There may be no holy grail of a single impact metric but it is still possible to analyse and manage impact in a systematic way.

The approach we have outlined above is tailored to our particular needs as venture capital investors in aligned for-profit businesses. We do not think that it would work for everyone and, indeed, we are still piloting it for ourselves. It is based on just 14 investments and is sure to be challenged by situations we encounter in the future.

We are keen to work with other investors in the industry both to refine and improve our model but also to experiment with different approaches to aggregation for different purposes. We recognise the importance of confidentiality when sharing data around impact measurement and suggest the use of confidentiality agreements that will protect our cultures of experimentation, iteration, and learning from failures. Only by working with concrete examples will we start to see what works best for what purpose. Please get in touch if you are interested in collaborating.