As the earth moves and water erodes layers of rock, sinkholes appear, fissures open up in the earth and volcanoes erupt. Something similar happens in societies and economies. As currents of change erode old structures, and as tectonic plates shift, new holes and gaps appear.

These are the places where new rules, and new kinds of organisation, are needed but are missing. Over the last two centuries the process has played itself out repeatedly and remorselessly. Technological change usually prompts a responsive social change – because without it so many people lose out.

In the 19th and 20th centuries, rapid urbanisation and industrialisation initially had disastrous results for millions as they moved to cities blighted by pollution, high crime and disease. In response, societies created a plethora of new institutions – from police to public health, from universal education to inspectorates, from trade unions to microcredit, and from welfare states to regulators. Their job was to fill the holes which otherwise too many people fell into. The ‘big stink’ of 1858, which led London to create institutions for public health and a network of public sewers – still in place 160 years later – is a good example.

The fairy-tale stories of technological change in which technologies simply rain down onto a grateful population miss this out. So do the dominant stories in economics. These at least recognise that institutions play a role in letting markets work, but they typically ignore everything else. Equally misleading is the simplistic view that less regulation is always better than more (if you’re in any doubt, just do a thought experiment of a world where there were no rules for cars: no driving tests, vehicle safety tests, road markings, speed limits, drink-drive rules, emission standards).

Instead the lesson of history is that technological innovation has to be matched by social and institutional innovation - otherwise we risk getting too many harms and too few benefits.

The fourth industrial revolution is not so different from the previous ones in this respect. As technologies are deployed, they create and reveal gaping holes in the landscape – sinkholes which people and places fall into. These become visible in the form of crises and deserts: crises of privacy, lost data, corporate misbehaviour or ethical infraction; and provision deserts where acute needs are not being met.

It’s not too hard to get a map of the fissures and gaps – and to see where governance is needed but is missing. There are all too many of these now.

Here are a few examples. One is long-term care, currently missing adequate financing, regulation, information and navigation tools, despite its huge and growing significance. The obvious contrast is with acute healthcare, which, for all its problems, is rich in institutions and governance.

A second example is lifelong learning and training. Again, there is a striking absence of effective institutions to provide funding, navigation, policy and problem solving, and again, the contrast with the institution-rich fields of primary, secondary and tertiary education is striking. The position on welfare is not so different, as is the absence of institutions fit for purpose in supporting people in precarious work.

I’m particularly interested in another kind of sinkhole: the absence of the right institutions to handle data and knowledge - at global, national and local levels - now that these dominate the economy, and much of daily life. In field after field, there are huge potential benefits to linking data sets and connecting artificial and human intelligence to spot patterns or prevent problems. But we lack any institutions with either the skills or the authority to do this well, and in particular to think through the trade-offs between the potential benefits and the potential risks.

Healthcare is a glaring example. There are obvious benefits to linking data sets to better target action on diabetes or cancer, and it’s even better if these can pull together people’s own experiences and insights, as well as those that the system generates. But who should do this? Who is credible? How should they be accountable? The NHS starts with a good base of trust, but it lacks any institutions to play this role, and so bumps into repeated problems.

Very similar issues arise in policing. We’re beginning to see the emergence of new platforms that link together policing data and citizen-generated data, observing crime, jointly acting to deal with it or prevent it. But again, who should take responsibility for this? Whom can we trust to handle the often complex judgements and trade-offs?

Likewise jobs. Great gains could be achieved from linking up multiple data sets on jobs and skills, particularly at a local level, and achieving what I’ve called a ‘Googlemaps’ for jobs and careers that can help anyone navigate from where they are to where they want to be. But nowhere in the world yet has the institutions to do this both competently and legitimately.

In each case the task is one of orchestration, curation and value generation – with sensitivity to ethics and legitimacy. Various proposals have been made for data trusts and data collaboratives. Private companies have tried their own stop-gaps – from Google DeepMind and Babylon Healthcare in the UK, to Sidewalk Labs in North America – but have usually discovered the limits of their legitimacy. There are many options for creating data trusts to fill this gap – some focused on research, some on transport, some public-private partnerships. The crucial consideration is that the way they are designed should help them to create trust.

At a global level the sinkholes are even more evident. We will soon need a raft of new institutions to handle everything from governance of the internet to transborder data flows and cybersecurity (I wrote about one of these for the Financial Times). The EU is setting the pace, partly because it is just about the only transnational institution with the cognitive capacity to think these things through. One of the many tragedies of the current reversion to nationalism is that it makes it much harder to put these institutions in place.

Even more important is the global governance of extreme risks – pandemics are partially covered by existing institutions, but we lack any credible governance to protect the world from new generations of destructive weapons, whether out-of-control biological weapons or killer drones.

While the current wave of populism is delaying the necessary task of reinvention, this is likely to be a crucial space for public and social innovation over the next decade. For now at least the sinkholes are getting worse not better. In a way this is the ultimate test of leadership and governance - the ability to reshape institutions to fit the most important challenges of the times. But who is really up to the job?