These are questions that cannot be easily answered by official government data, if at all. Expensive one-off studies by consultants can sometimes produce answers, but cannot be easily reproduced on a local level.

We are engaged in a project for Nesta that shows how the judicious analysis of online wanted ads can address these questions and other similar ones at a relatively low cost. This work builds on research that we have done for the United States, including tracking App Economy jobs.

The key is that online want ads are a unique example of easily available ‘big data’. Recall that an online want ad contains at a minimum a description of the set of skills required by the job being advertised and the location of the job. In addition, it may contain salary information, and information about the company placing the ad (though these two are often not present).

The universe of online want ads is aggregated on a real-time basis by a variety of companies and nonprofits, including The Conference Board in the United States and Indeed.co.uk in the United Kingdom. Such aggregators make it easier to get counts on want ads, conditional on key words in the ad and location of the job.

For example, as part of the Nesta project, we are tracking the distribution of ‘big data’ jobs across the UK. This is a multistep process. First, we constructed a list of relevant keywords to identify jobs that require ‘big data’ skills. This includes 14 words or phrases from program names such as Hadoop and MapReduce to job titles such as “data scientist” and “data analyst.”

This gives us an aggregate number of want ads for jobs requiring big data skills in the UK. Based on other research we have done, we can link to the number of ads to the number of jobs. Equally important, we can look at the distribution of ads across the country to uncover clusters of big data jobs that might go unrecognized outside of London.

Once we have the basic results, we can show how data from help-wanted ads can be used to build a real-time job dashboard for any local government that wants to know about emerging industries and occupations in their area.