We've heard that as much as 90% of the data that exists today has been produced in the last two years. The Internet has a lot to do with that. Innovative businesses are exploiting this new resource to improve their operational efficiency, productivity and consumer satisfaction.

Are UK businesses making the most of this opportunity?

Rise of the Datavores, a report we are launching today, addresses this question with data from a survey of 500 UK companies with more than 50 employees and are active online. We examine how these companies are collecting, analysing and using online customer data to make better decisions, and some of the benefits from doing this. 

We identify a 'data elite', the Datavores: companies that, when making decisions about how to grow their sales, rely on data and analysis over experience and intuition. They are the 'Nate Silvers' of our sample, collecting data comprehensively, analysing it with advanced techniques such as data mining and controlled experiments, and using it to shape business strategy and product development. We also find that they are more innovative than the average, and twice as likely to report substantial benefits from their online customer data. 

We find Datavores in all sectors and company sizes, which is consistent with the idea that the innovation opportunities from data are not confined to any particular sectors - they are for everyone. 

More worryingly, it seems that despite all the 'buzz' around 'Big Data', 'Analytics' and 'Data-Driven Decision Making', Datavores are still in a minority, even in the Internet economy where our respondents trade. Only 18% of our respondents fall in the Datavore category. By comparison, as many as 43% of respondents report that, when making decisions about how to grow their sales, they rely on experience and intuition over data and analysis. 

We believe that this is a missed opportunity for innovation, and it is risky too. Running a business purely on the basis of intuition in the era of Big Internet data is the equivalent of steering a ship without a compass; there is a good chance that at some point you'll get lost.

There is an apparent paradox in our findings - if the benefits from collecting, analysing and using online data are so clear, why don't we see many more businesses doing it?

One potential explanation for this is that 'getting serious about data' is no simple matter. According to our survey, it requires substantial investments in skills, and changes in organisation and culture - for example, we find that Datavores tend to empower employees and delegate decision-making more than other firms. This makes sense - where decisions are made on the basis of hard data, there is less reliance on the 'Highest Paid Person Opinion' (the 'HIPPO'), but it doesn't diminish the challenges of re-organising the business and, to some extent, rethinking the job of managers. We have find that concerns about data security and privacy may be creating barriers to innovation in this area.

Rise of the Datavores is the first milestone in our programme of research looking at the potential of online data and innovation for the UK. In coming months, we will be matching our survey responses to financial data to measure the relationship between data-driven decision-making and economic measures of performance in our sample. We also intend to study the supply of 'data-savvy' talent in and exploring the possibility of a controlled experiment to measure the impact of an Online Analytics intervention on a randomly selected group of start-ups. Last, but not least, we will be looking for Datavores in the public and third sectors, and work to encourage the transfer of good data practices across different parts of the UK economy and society.

We are very excited about this programme of work, and not without reason - after all, and as Erik Brynjolfsson and Andrew MacAfee at the Massachusetts Institute for Technology recently pointed out, this "Big data boom is the innovation story of our time".  

We look forward to your comments and ideas.