How green are Boris bikes? Are time-banks the key to urban regeneration? Does Uber really lower crime rates?

If we want impact to become a central consideration of the collaborative economy, it’s important to understand what kind of impact we’re talking about and where it’s coming from. While organisations in the collaborative economy contend with different aims and contexts, comparing impact shouldn’t be impossible – so long as we share a common language and approach.

Despite the collaborative economy’s much vaunted potential for disruption, little is known about the effect it is actually having on economies, communities and the environment. We think there are two ways of distinguishing impact in the collaborative economy.

The triple-bottom line: our starting point

Many collaborative economy platforms claim to be using assets ‘more efficiently’ than other business models, and to be ‘more finely tuned’ to users’ real needs – whether that’s access to the right-sized car on demand, the freedom to run successful micro-businesses from home, rescuing useful objects from landfills, or providing jaded modern-day folks with less alienated modes of interaction. Given the diversity of these aims, we believe the triple-bottom line approach (simultaneously measuring economic, environmental and social impact) is a particularly useful way of distinguishing impact in the collaborative economy.

Measuring effects beyond the purely economic is notoriously difficult, but doing so can help policy-makers, citizens, funders, and projects themselves make better strategic decisions about the kind of future we are collectively – if not always collaboratively – devising. After all, if something can be measured, it provides a clear and compelling incentive for strategically addressing it.

Three levels of impact

Along with the triple bottom line, there are three levels where organisations in the collaborative economy can demonstrate their impact.

1. Internal impact

Firstly, there is the impact that collaborative projects can have simply by existing in the world, such as creating jobs than didn’t exist before. There are also things that organisations chose to do which can have a positive impact but isn’t related to the ‘collaborative’ nature of their operational model. Such ‘internal’ impacts range from simple practices like providing office recycling facilities to ambitious sustainable life-cycle service design, such as Velib’s multi-level ‘zero-carbon’ approach. Depending on the size of the organisation, these internal impacts can be more or less significant, though they generally have little to do with companies being ‘collaborative’ per se.

For instance, running CSR programmes can have social and/or environmental benefits according to which initiatives are pursued. The integration of a donation at checkout button on eBay raised $557,199 from 203,411 transactions for disaster relief, following the devastating 2013 typhoon Haiyan in the Philippines. However, eBay’s clever partial re-purposing of their platform nonetheless remains marginal to their operational model.

2. Impact of operating model

A company or initiative can also design their operational model to achieve specific and deliberate kinds of impact. This seems to be where organisations really try and harness collaborative models, to ensure assets are used more efficiently, and systems map a broader range of users’ needs more closely.

In Zipcar’s model, for example, users have on-demand access to cars whenever they need them. Being able to rent a vehicle for short periods of time and in convenient locations has an economic impact, as the system turns out cheaper than the average cost of full car ownership. Zipcar’s environmental impact – whether the system has actually reduced the number of cars on the road – is harder to capture and doesn’t ‘inevitably’ flow from its operating mode. It is equally possible that the overall number of trips made by car increases thanks to lower economic entry barriers to driving.

3. The elusive ‘ripple effect’

Finally, collaborative organisations can also have a ‘ripple effect’ on economies, people, and environments that lie beyond their core targets. For instance, Spice TimeBank’s study showed – beyond its primary social benefits – that participants’ increased self-confidence and pride had resulted in a village-wide decrease in littering. This unexpected environmental impact was highlighted in qualitative participant interviews.

Measuring the ripple effect of the collaborative economy can help us imagine possible futures. It may also help local politicians make decisions on which activities to support, and law-makers to devise appropriate regulation.

While undoubtedly important – therein lays the promise of the collaborative economy – these ‘ripple effects’ are also much more difficult to measure. The majority of collaborative initiatives and companies are still too small, or too young, to be having a discernible impact at that scale. Demonstrating causality can therefore be difficult. Since Spice’s focus was social, the project did not build in adequate ways of measuring environmental changes, and cannot robustly prove this particular point. Consequently, while Spice was able to capture people’s impressions, it could not prove that environmental improvements had empirically occurred – or that such change was caused by the TimeBank.

Moreover, the high cost of carrying out such complex measurements often lay beyond what these projects can reasonably afford. The costs involved and the difficulty in constructing believable impact data puts this rigorous measurement of the ‘ripple-effect’ out of reach of many collaborative economy initiatives. Looking for evidence at such a high level, rather than focusing on finding evidence for the effectiveness of operational models is less directly useful in enabling projects to reach their immediate potential.

Interrogating evidence

While we can start to distinguish between different kinds of impact, it is important to acknowledge that not all evidence is equal. Findings and claims need to be interrogated before they are accepted. Otherwise, we risk falling into the trap of leaning on false numbers and unbacked claims.

Uber has taught us a few lessons about questionable evidence. Their recent claim to reduce crime is derived from a quantitative correlation between company data (a sudden increase in Uber-powered taxi rides) and publicly available crime rate data (decreasing for the period). While it may look convincing at first glance, the analysis assumes that the overall number of taxi-rides has increased, rather than the same number of rides occurring through a different provider (Uber). Without this information, the correlation is meaningless, and even if absolute ride numbers was taken into account, the causation can’t be proven.

A model for distinguishing impact

By combining the triple bottom line with our three levels of organisational impact, we have the beginnings of a holistic model, specific enough to be meaningful, but flexible enough to encompass the full diversity of collaborative economy initiatives. And if some aspects of a project spill-over into adjacent squares, that is only to be commended. More details on the examples below in our next post.

Economic

Social

Environmental

Internal

eBay’s data-centre efficiencies

Lyft’s CSR programme

Velib’s maintenance fleet, powered by renewable energy

Operational model

Zopa peer-to-peer finance platform broadens access to capital and provides good rates of return

Spice TimeBank regenerates depressed communities in Wales

Airbnb stays have a lower environmental impact than hotels

Ripple effect

Uber claims to have a $2.8 billion impact US 2013, including the extra spending users spend in the areas they visit and drivers’ extra disposable income

Airbnb participants feel part of a worldwide human community

Zipcar claims that car-sharing reduces the number of cars produced

In the next post, we’ll be using this framework to identify which kinds of impact collaborative economy organisations are already measuring, and which have been ignored. We’ll also review the most popular methods used, and whether they produce convincing impact data.