But I’m repeatedly struck by what’s missing in their methods, their cultures and their approach. Here I highlight three missing habits that, for now, are almost completely absent from the theory and practice that guides the spending of billions of dollars, Euros and yuan.

The first missing habit is listening, and in particular listening to the people whose money is being spent - about their lives, their hopes and fears, and where new knowledge could achieve most for them. The planners of R&D are often good at listening to researchers themselves, and some of the time they’re good at listening to big business. But remarkably little work is done to listen to the people who pay the bills, and that’s reflected in the many distortions of R&D spending – towards hardware, prestige projects, and the interests of the well-connected.

Even the most basic survey methods are barely used to find out in which fields the public most wants new innovation. Nesta’s study in 2014 with Comres was unusual, and ignored by the then ministers, even though (or perhaps because) it showed a big gulf between where the public thought public money should be spent and where funds were actually deployed.

The second missing habit is, surprisingly, experiment. In many walks of life, improvement comes from vigorous and continuous experimentation. Wherever possible the best scientists and business innovators try, iterate, learn and improve. But this is very much the exception when it comes to scientific and technology funding which tends to follow well-worn paths, with little experiment let alone gathering of evidence to discover what might work better.

Our Innovation Growth Lab - now supported by a dozen governments, running experiments and pooling the lessons - is very much the exception, and works on business support rather than research. We’re not aware of anything similar for R&D. This links to the broader oddity of science funding - the relatively small role played by evidence or data compared to connections and influence (a point brought home vividly in our recent report on the Biomedical Bubble).

The third missing habit is open harvesting - the habit of always trying to tap the widest range of minds, organisations and data. Open innovation methods are now mainstream in many fields. They’re used by public organisations like NASA and private ones like Facebook. Large communities of inventors are now curated by organisations like TopCoder or Innocentive. These have repeatedly shown that seeking out a much wider source of ideas leads to more creative solutions.

Our own Challenge Prize Centre is expert at finding a far wider community of people with relevant expertise for solving problems. But these methods are still largely absent from R&D funding which defaults to using traditional means to finance traditional recipients and, as a result, is often a lot less efficient than it could be.

Why are these three habits so rare? Why are systems packed with very clever people so resistant to change?

I suspect that at least part of the answer can be summed up in one word: status. All three of the missing habits either explicitly or implicitly challenge the status of key groups. The first challenges the presumption that they know which problems matter most; the second challenges the presumption that their initial plans are the best possible ones; and the third challenges the presumption that they, and their colleagues, have a monopoly of wisdom.

Of course, if they were really wise they would appreciate that cultivating these missing habits could enhance their work. Those of us involved in the field shouldn’t give up however. I suspect that these three habits will slowly gain ground. They are certainly spreading in many other fields, including within governments. In the manner of scientific revolutions, they may at some point simply become common sense. I hope we don’t have to wait too long.