There is a great deal of excitement around the impact of ‘big data’ on innovation and growth: more data and improved analysis methods can help organisations create new products and services, and optimise their decision-making capacity. In this post, I look at the implications of the data deluge for the arts and cultural sector, which does not generally create vast amounts of data, and in which turning a profit is often a secondary consideration. I show how data can help the sector to create value by helping it to understand its audiences better, to measure the (public) value that it generates, to develop new business models and to learn from its creative experiments.

The data deluge

We are currently producing and collecting data at an explosive pace. Just consider the ‘data trail’ each of us leaves in our wake every day simply by carrying and using a mobile phone, participating in a social network, or shopping online. The web is like a vast mirror that reflects our actions, and it can provide insights into our behaviour, and even our desires.

We are also becoming better at extracting knowledge from this data using techniques such as machine learning (algorithms that learn from data), network science (which maps our position in relational networks) and text mining (which identifies patterns in unstructured content), to name a few. The results are products, services and platforms that we use every day – email spam filters, collaborative recommendation systems on websites like Amazon, matchmaking features on dating websites, fraud detection in the banking sector, and so on. Aside from providing the basis for the development of new products, data can also be a source of insights that allow organisations to make better decisions – about what TV series to commission, for example, what version of a website to develop, or how to reward employees.

All of these elements (a greater volume and variety of data, which is created more quickly, and can be used to create value) come together in the idea of ‘big data’, which has taken the business world by storm – Google Trends (itself a big data service that uses search data to measure user interest in different topics) reveals a dramatic increase in the popularity of this term over the last two years. Consultants and tech providers agree that big data has massive potential, and a recent report shows that 91% of Fortune 100 companies have at least one big data initiative in progress. (For a good introduction to the phenomenon and its wide-ranging implications, see the book Big Data, by Kenneth Cukier and Viktor Mayer-Schönberger).

The implications for the arts and cultural sector

But how relevant is all of this for the arts and cultural sector – which includes arts and cultural institutions as well as cultural industries like music, publishing, film and broadcasting, which rarely generates massive datasets (although there are exceptions, particularly museums and libraries with digitised collections), and in which commercial value and profit are in general less important than cultural and public value?

I believe that big data is, potentially, highly relevant for the sector. We just need to be careful about definitions.

It is clear that most arts and cultural organisations don’t generate petabytes (millions of gigabytes) of data, and don’t need to use parallel processing frameworks like Hadoop to deal with them. In this sense, they don’t operate in a big data world. But it is also true that many of these organisations can now access much more data about their users than ever before. This data is also increasingly diverse –it includes the usual tabular data (e.g. visits to their websites over time), network data (from social media platforms), text (user feedback and comments) and even video, audio and sensor data. When we take all of these together and add open data (for example, government data, or the big cultural datasets in Culture Hack), things start to get bigger, and more complex. Professor Ralph Schroeder, from the Oxford Internet Institute, defines big data as ‘data unprecedented in its scale and scope in relation to a given phenomenon’. In this respect, the arts and cultural sector is in the midst of a big data moment.

And what about the uses of this data? Can it help create the types of value that matter to arts and cultural organisations? I will shed light on this question by considering the role of big data in the types of arts and cultural innovation identified by my colleague Hasan Bakhshi and Australian Economist David Throsby in the Nesta report ‘A Culture of Innovation.’

1. Broadening and deepening relationships with audiences

Art and culture involve communication and community: in order to thrive, arts and cultural organisations need to know who their audience is, and find ways to broaden it and connect with it in innovative ways. Data plays a crucial role here, because analysing it can help organisations to understand current users, and to identify gaps that can be addressed through new initiatives and programmes. An example of an organisation using data in this way is UK public broadcaster Channel 4, whose CEO David Abraham famously declared that ‘data is the new oil’. Channel 4 is collecting vast amounts of data about its audiences, and building up the skills to analyse it, with the goal of increasing viewer loyalty, generating insights for its negotiation with advertisers, and developing new programmes and formats.

2. Creating and measuring value

Cultural and public value is clearly at the heart of arts and cultural organisations. We could even argue that these organisations create value when they ‘give it away’ (for example by inspiring their audiences, connecting them, and offering them unforgettable experiences) instead of keeping it for themselves. It has often proved difficult to measure these non-monetary types of value, and to communicate their importance to funding bodies, sponsors and the public. But this is changing as new types of data become available, writes Anthony Lilley in the Nesta report ‘Counting What Counts’. Today, arts and cultural organisations can use social media data to measure how their activities contribute to building social capital in local communities, and they can trawl user-generated sites for evidence of their popularity and influence. My colleague John Davies discusses some of these issues in this blog post for the Department for Culture, Media and Sports

3. Developing new business models

Digital distribution creates new opportunities to diversify revenue streams in the arts and cultural sector, thus boosting its resilience. At the same time, there is great uncertainty about which business models will work and which won’t. For example: How much will audiences be willing to pay for an innovative product or service, like the live-stream of a theatre performance? How much will they pay to download it? The best way to find out what works is by running ‘business model experiments’ to collect data and learn. This is what filmmaker Ben Wheatley did with A Field in England , a film supported by the British Film Institute that premiered simultaneously in theatres, TV and video on demand services across the UK. Data was collected from all these channels, as well as social media sites, to assess the impact of this strategy on the visibility of the film. The results of this experiment have been published for the benefit of independent filmmakers across the UK.

4. Encouraging creative experimentation

Too often, there is a tendency to see a dichotomy between creative experimentation and data, as if analytical organisations were less creative by definition. Although it is true that a data obsession can lead organisations to focus excessively on what can be measured rather than on what matters, if it is used judiciously, data can support creativity by making it easier to assess project risks, to identify problems and to resolve them. Moreover, the opportunity to learn means that failures become valuable, and makes the idea of trying new things more attractive. One example of this is the Alice in Wonderland exhibition organised by Tate Liverpool in late 2011, which included the specifically commissioned video game Wondermind, targeted at visitors to the website. The data that Tate collected as part of a comprehensive analytics strategy (discussed by Elena Villaespesa and Tijana Tasich in this paper) revealed high levels of user engagement with the game, consistent with the idea that this creative pilot had been successful, and encouraging future experiments along similar lines.

Conclusions

It is still early days in the adoption of data-driven innovation in the arts and cultural sector, but the examples discussed above suggest that it has great potential. If arts and cultural organisations are going to fulfil this potential, they will need to access new data skills, develop an analytical culture, adapt their processes and think innovatively about how to measure the value that they generate. They will also need to learn how to address data pitfalls and biases, and maintain a critical attitude.

Funding bodies should also explore ways to encourage experimentation with data in the arts and cultural sector – by means of targeted interventions like the Digital R&D fund for the Arts set up by the Arts Council England, Arts and Humanities Research Council and Nesta, skill-building programmes and enhanced receptiveness towards new value metrics. This is not just important for the sector itself, which stands to gain much from transforming its data into an asset, but also for the sake of the rest of society, which surely has much to learn from innovative uses of data in the arts and cultural sector, and from a critical engagement with data – one of the defining aspects of our time.

This post was originally published in the CCCLab blog. The artwork 'Dada Visualization' is by Mario Klingemnann