Can government remember? Is it condemned to repeat mistakes? Or does it remember too much and so see too many reasons why anything new is bound to fail?

The case for a permanent civil service is that it organises its memory well and is certainly better than governing systems which periodically purge their top tiers. So when there is a natural disaster, a recurring policy task like reforming the curriculum or local government finance or building hundreds of thousands of new homes, comprehensive bodies of knowledge can be tapped into to guide today’s decision-makers and save them the trouble of reinventing wheels.

Such is the theory. Unfortunately anyone with close experience of government realises that the truth is rather different. When I worked in government I regularly found myself the only person in meetings who had been at the equivalent meetings five or so years before or could recall the previous discussions. I had become the corporate memory.

Some of the reasons for this poor memory include very high turnover; rigid silos which make it hard to access information across departmental boundaries; and the lack of even rudimentary knowledge management systems. These are made worse by traditions like the prohibition on giving ministers access to the advice given to predecessors from a different party.

The net result is that government forgets as much as it remembers

In an age of search engines this might look like a problem that has gone away. Surely you can just Google anything you need?

Unfortunately things aren’t so easy. Indeed, if anything, things have got worse. Government websites used to carry extensive previous analyses and policies; these have largely disappeared now. A recent study of government’s own commissioned research showed that it mainly had been lost or hidden.

There’s no shortage of examples of forgetfulness. A current example is rough sleeping. This was a problem that was largely solved in the 2000s through a subtle, energetic and comprehensive set of policies, but has now re-appeared with a vengeance.

There is very little sign that either government or opposition is even dimly aware of what was done. None of this has been helped by the reinforcement of silos in recent years as the various attempts at joined up government were quietly dismantled in favour of more traditional structures and processes.

In his new paper, Gavin Starks suggests that, although this is an old problem, there may be new solutions. He shows the technology-based solutions — new ways of organising data, documents and works in progress. He also acknowledges they need to sit alongside complementary measures.

These are powerful ideas that could transform the everyday work of governments

At Nesta we have been closely involved in some of these — like setting up What Works Centres that effectively organise the collective memory of a field such as policing or teaching, and the broader movement to open up public data. Others concern shifting cultures to reinforce sharing of information and a shared commitment to solving problems rather than defending the departmental walls.

Other countries are showing new approaches which are promising. Canada — where Nesta is now working extensively — has pioneered ways of mapping talent and experience right across the public service and many are grappling with how to organise knowledge more effectively. In my book Big Mind, I show that memory is a crucial part of collective intelligence and that there are many ways to organise both storage and retrieval in new ways.

The challenge isn’t so much about storage and creating vast new electronic archives. The bigger challenge is to make that memory easy to find and then access. And there are innumerable complexities involved in government memory; some of it has to be extremely secret and some has to be highly sensitive to personal privacy. But the defaults are now in the wrong place and we all suffer from the poorer policies and programmes that result.

At Nesta we’ve said in the past that elected politicians have every right to ignore the facts and evidence but no right to be ignorant of them. In the same way they have every right to ignore what’s remembered. But they’ve no right to be ignorant of it. Here’s a way to bring the best of what we now take for granted on the internet into the everyday work of the state.

Geoff Mulgan and Gavin Starks will be speaking at the launch of the report 'Can government stop losing its mind?' on Wednesday 18 April. Read the report here.