On the face of it, the EDSII and its results seem simple: 60 cities across the EU, each with a score for six themes, combined to create an overall score. London storms off ahead of all others; Madrid just about sees off rival Barcelona; Copenhagen nudges ahead of its Scandinavian cousin, Stockholm.

Unsurprisingly, the findings are not as simple as that. The development of the EDSII has been a journey of iteration, challenges and subjective decision-making. In this blog we discuss some of this journey and how it has shaped the EDSII.

Sadly, this is a phrase which can never be applied to something like an Index. The maths, numbers, equations and code behind an Index like the EDSII mask the fact that it is far from objective truth. The framework for the EDSII is based on a rigorous and lengthy process of interviews, roundtables, literature review and a survey, but ultimately it has been shaped by a series of subjective decisions.

Some of these are somewhat intentional: for example, one could argue that Diversity and inclusion are not really necessary for DSI to grow, but we decided to include it as a theme as we consider it essential for effective DSI which empowers people and valorises lived experience.

Some are simply inevitable: there can be no truly objective way of ranking which indicator or theme is more important than another, particularly in a field as complex and diverse as DSI.

Some may even be cultural: we had particular difficulty, for example, with the civil society theme, given the different role civil society plays in different cultures across the EU, not to mention the different conceptions of what civil society itself constitutes.

Throughout the development of the EDSII, we had to make decisions using our best judgement and the best evidence available - but others building the same index may have made different decisions. Indeed, we might have made different decisions in different circumstances. That’s why it’s so important to us to be transparent in our processes, and to make our data and methodology open for all to see.

The spectrum of DSI is so broad that two initiatives under the same umbrella often seem worlds apart. DSI uses a huge range of technologies. It addresses a huge range of social and environmental challenges. Tiny charities and user-innovators can do DSI, but so can profitable start-ups and national governments. Some DSI addresses challenges outside our front door, while other initiatives aim to deliver impact on the other side of the world. How can one Index possibly cover this breadth, diversity and complexity?

This is a challenge we’ve come up against time and time again over the past eighteen months. Availability and quality of open data is the foundation of a fair amount of DSI, but others couldn’t give a damn what their governments are publishing and how. Impact investment is the lifeblood of some DSI, while others wouldn’t dream of going near it. Access to public sector contracts is make or break for DSI in some sectors like healthcare, but wholly irrelevant in others like anti-corruption.

Again, there’s no single solution to this challenge. We’ve tried to make balanced decisions which take into account the complexity of the field, and our decisions and results won’t please everyone. That’s why we think it’s so important to publish our data and methods transparently - so that if you are someone who cares about public sector contracts and impact investment, but don’t care much for open data or grant funding, you can rework the data accordingly. And we hope you do, as there are infinite remixes of the data which will be useful for different people!

As we continued to develop the EDSII, a fundamental paradox came to light: DSI, and social innovation more broadly, is understood as a response to a challenge or problem, and often one which is close to home. Some of the most inspiring DSI initiatives we see are the ones developed in response to real lived experience - from Mirrorable, developed by parents whose child had suffered a stroke, to Monitorizare Vot, developed by Code for Romania to improve the transparency of their own elections. This is fundamentally different from, say, commercial digital innovation, where an opportunity in the market, the profit motive and consumer desire and convenience fuel innovation.

So, if DSI is a response to real challenges, does that mean “worse” places would be more conducive to DSI growing? Or, equally, why would places where everything is just fine need any DSI?

This argument sounds flippant. But as we’ve worked on the EDSII over the past year and a half, this paradox has been a real challenge for us. It made us question our initial hypothesis - which we initially took for granted - that DSI activity would correlate strongly with supportive ecosystems for DSI. As the final results show us, this is not the case - rather, there is only a weak-to-moderate correlation.

This paradox is one we haven’t quite yet solved, and which links closely to the previous point about how complex and diverse the field of DSI is, both in what it looks like and how it comes about.

The EDSII measures things we can measure: things we can put a number on, or which others have already put a number on. It measures ecosystem factors, many of which are the result of years or decades (or more) of policy, culture, economics, politics and much more. We can consider this the ‘nature’ of a place.

What the EDSII doesn’t measure is specific attempts made to support DSI, such as dedicated funding streams for DSI, supportive regulatory frameworks or commitments by government to ethical technology - what we might consider efforts to ‘nurture’ DSI.

This is why some people might be surprised not to see cities like Barcelona, Madrid or Berlin in the top 5 - cities which have made headlines in recent years for their determined efforts to advocate for and support responsible technology. We hypothesise that both nature and nurture are essential for DSI to thrive: the right ecosystem must be there, but governments must also take proactive steps to support DSI. This is partly why we developed the accompanying Ideas Bank, to draw attention to some of the amazing initiatives going on across and beyond Europe which simply can’t be measured by an Index.

Finding accurate, recent, comprehensive, granular data for the EDSII has been one of our biggest challenges. On indicators related to strength of civil society, diversity and inclusion and grant funding in particular, we’ve struggled to find high-quality data, to the point where we had to remove two EU capitals (Valletta and Malta) because of the number of missing data points.

This is a real issue. Researchers, civil society and policymakers should be able to access timely, localised and accurate data on things like who’s founding companies, what people think of non-governmental organisations or where public and philanthropic money is going. We’ve found that isn’t the case, at least at a Europe-wide level. This needs to change, with city and national governments committing to collect better and relevant data in these areas and collaborating to develop standards and criteria. As the old saying goes, what’s not measured isn’t managed, and, without data on issues crucial to the success of DSI, it simply won’t be able to reach its potential.

At its most basic, the EDSII is a ranking of cities on how their ecosystems support the creation, growth and sustainability of DSI. But it is also much more than that: it has opened up new research questions about how we conceptualise DSI, how we deal with the complexity and diversity of DSI, and how we collect data to understand what supports DSI to grow and scale. By making our methods and data open, we hope to equip others with the foundation they need to explore these questions further, for their own ends and for those of the DSI community at large, and ultimately to make European cities places where DSI can reach the potential we believe it has.