Measuring impact is not always easy, but it shouldn’t be treated as an after-thought, a last line of defence, or an unaffordable luxury. Collecting and analysing sound data can help organisations make strategic decisions and realise their potential. Sharing your findings more widely can help others, making the collaborative economy more effective and successful in its endeavours.

That’s the theory at least...

Organisations undoubtedly have different, multifaceted reasons for wanting to understand their impact – or not. However, these motivations can influence the type of impact an organisation focuses on, the methods they use, and the way they report their findings.

So what drives some collaborative economy organisations not only to measure, but also to publish evidence of their impact?

While reviewing the evidence and claims concerning the collaborative economy’s impact, we were struck by the range of motivations guiding impact studies and research. We can learn a lot from the ways organisations communicate their intentions to evaluate, and use their results.

Below are a few trends we encountered.

Measurement to make headlines

A perceived opportunity for growth, or a public challenge appeared to frequently preface impact measurement exercises. This is when companies begin to hire policy officers, commission consultancy reports, and generally step-up their PR game.

Slightly more cynically, we occasionally found suggestions that certain platforms use headline-grabbing impact claims to acquire new participants. For instance, Uber has stated that its New York drivers make a median $90,000 annually, based on an undisclosed private impact report. Commentators have argued this figure was used as a tool to attract more drivers to its books (and away from traditional taxi service companies and rivals, like Lyft).

Findings to inform and influence

Good, credible analysis can also be a powerful tool for informing governments, policymakers and regulators. Larger organisations have tended to be better at this, as they have the capacity to commission research (or undertake it themselves) and then communicate and apply their findings.

For instance, Airbnb has undertaken considerable research to evaluate its impact at local and international scales. Along simple outputs like infographics, they have commissioned a preliminary impact study which helped to convince the City of Portland that the platform had beneficial effects on the city. Meanwhile, eBay has published research at various points which appeals to regulators at both national and international levels. They also have an in-house policy team to further instrumentalise the economic impact demonstrated by its user data. E-bay’s Main Street advocacy network is set up to convene popular support for E-bay’s lobbying goals, who are then encouraged to act independently to lobby politicians directly.

Understanding impact as a strategy for growth

Impact measurement can be used by organisations to make better decisions about how to reach their goals, or check they are actually meeting them in the first place. While this often occurs privately, some organisations are embracing impact evaluation and openness as key features of their operations.

Open-media platform Wikimedia has recently launched a newsletter to share learning, innovation, and evaluation good practice across its community. The platform depends heavily upon its users’ drive to contribute content, check accuracy, and lead knowledge-sharing projects. Impact data can thus act as a motivator, while sharing evaluation methods openly may also encourage more people to volunteer to further build up Wikimedia’s evidence base.

We’ve noticed that many organisations which publicly share their impact finding and methods also tend to embrace a different model of growth – scaling by replication.

Local initiatives like Spice time bank and the Bristol Pound are sharing the outcomes of their evaluations openly online, and are transparent about the evidence used to devise further growth plans. Unlike Wikimedia’s global participatory ambitions, these two projects are rooted in the small communities in which they operate. Being open about what works, what doesn’t work and what could work sets them up as effective models which others can take up and run elsewhere. The social cohesion aims of these two initiatives are best served by localised replication than blanket-style growth.

Intentions are not a measure of quality

Asking why collaborative economy organisations evaluate their impact can help us understand what types of impact they focus on, and how they measure it.

However, motivation is not a measure of quality. Organisation with noble intentions can still use poor data quality and questionable methods, while a highly strategic piece of research can be incredibly rigorous. When done poorly, impact evaluation efforts can actually be counter-productive, potentially misinforming the public (and organisations themselves) or raising suspicions of over-blown hype amongst critics.

A small number of collaborative economy companies are already taking steps to carefully measure their impact, and publish their findings. For some, this is a strategic tool to draw in potential users, get buy-in from local authorities, or influence regulators. Others believe that being transparent about the effects of their activities and sharing what they learn can benefit others, and result in better outcomes for projects, people, and the planet.

Whatever an organisation’s motivation, any headline or study on impact in the collaborative economy should be carefully assessed. Equally, praising organisations that rigorously evaluate their impact (even if the findings aren’t always stellar) might motivate others to follow suit.