MEASURING CULTURAL VALUE
Hasan Bakhshi
Keynote speech delivered at Culture Count: Measuring Cultural Value Forum, Customs House, Sydney, Australia, Tuesday 20th March 2012
There is perhaps no other relationship that is more fraught with tension, more hotly contested, in cultural policy than economic and cultural value. But if there is one thing everyone agrees on, it is the need for more sophisticated public funding decisions for culture – ones that are better understood by cultural institutions and by the public.
This calls for a much more rigorous attempt to value culture than has been the case to date. The Swiss economist, Bruno Frey, makes a distinction between the 'economics of culture' and the 'economic approach to culture'. Economic impact is a creature of the economics of culture. It refers to the measurement of the employment, output and productivity consequences of cultural activities. Properly executed, economic impact studies are essential for economic development agencies that see culture as a locus for, or as an instrument of, economic development. Valuation is the subject of the economic approach to culture. It should be important both to cultural institutions which want to evaluate their performance against their core missions, and to funders who want to assess their return on 'investment'.
Of course, these debates are not new. In 2003, Demos's 'Valuing Culture' seminar in London initiated a number of conversations in the UK about the rationale for public support for cultural institutions. Many of these were framed in terms of the intrinsic versus instrumental value of culture, and cultural versus economic and public value. There was some optimism that these debates would, to quote John Holden, give rise to a new “language capable of reflecting, recognising and capturing the full range of values expressed through culture”.
Perversely, however, almost all of the economic studies we've seen in the cultural sphere in the past ten years have been of the economic impact variety. Next to none have looked at valuation, using the empirical tools endorsed, for example, by the Treasury Green Book – the British government's official guide to cost-benefit analysis – that public economists have deployed so successfully in other controversial areas like the environment and health where, like culture, a good deal of public value is not mediated through markets, and is therefore not captured through market prices and transactions.
I've previously described this curious state of affairs as having two origins:
- First, in the lack of engagement by economists in the cultural and creative area. The American Economic Association uses a system of codes to classify different scholarly contributions in the area of economics. 'Cultural economics' appears in category Z: 'Other Special Topics'. The UK's Department for Culture, Media and Sport currently has no specialist economist positions at all at Senior Civil Service grade. I know of only a handful of economists working on culture in universities in the whole of the United Kingdom.
- The second reason for the dearth of valuation studies is in the unwillingness of cultural institutions to engage with the tools of economics as a theory of value. No doubt this is in part due to the distaste at using the language of consumer surplus and willingness to pay when the decisions of most artists, and the audiences they engage with, are made on emotional, spiritual or intellectual (what most people associate with 'intrinsic'), not utilitarian, grounds. It is one of the great ironies in recent years that cultural institutions have found it easier to engage with the economics of impact, in terms of jobs created and value added, than with economic tools that can shed light on intrinsic value created as part of their core missions.
In my view this is a failing of cultural leadership. That is, cultural leadership insofar as it relates to how cultural institutions are led in a way that serves a public interest that lies beyond individual institutions or initiatives.
What we see in the UK's cultural and wider creative sector is an ever increasing number of economic impact studies, using no consistent methodology and of varying quality of execution. The poorest quality examples drag down how the better-executed ones are perceived. What's more: everyone seems to recognise this! There seems to be a variant of the Prisoners' Dilemma at play. Organisations feel forced to commission economic impact studies because others are doing the same. When funding is constrained, no one wants to be disadvantaged by not having produced their own impact estimates. But in aggregate the numbers just don't stack up: the outcome is that the intended audiences – most obviously the public funders – do not believe any of the results.
Prisoners' Dilemmas call for leadership to resolve them. In this context I'll note again that there has been a dearth of rigorous empirical valuation studies in the cultural area. There has been too little recognition that our theoretical understandings can be usefully challenged and refined through robust and imaginative empirical work; or in other words, by learning by doing. It is shocking that in 2012 there are still just two comprehensive valuation studies in the UK that tend to be cited time and time again: the British Library's study of its public value in 2003 and the study by the now defunct Museum, Libraries and Archives Association of its footprint in Bolton in 2005. With so few examples of cultural sector applications of economic valuation techniques, is it any surprise that we have not developed more fit-for-purpose tools that can be better used to value the work of cultural institutions? And in particular tools that embrace the concept of cultural value which, unlike economic value, derives from a cultural discourse that cannot be expressed in monetary terms.
The really alarming thing is that no one, yet, has had this objective in their sights: no one has been working towards the long-term aim of systematically building a rigorous body of evidence which can be used to understand the value of cultural activity in its various forms.
The experience from fields like the environment and health suggests that government can play a critical leadership role. The publication last year of the UK's National Ecosystems Assessment is a good example of this. This groundbreaking exercise was an independent and peer-reviewed quantitative assessment of the value of the UK's natural environment and the services it provides. It generated both an evidence base and policy options that policymakers could use to implement more sustainable environmental policy. It used the full panoply of techniques that economists use to value public goods, including contingent valuation, willingness-to-pay and subjective wellbeing. The assessment was the outcome of a wide-ranging partnership of government departments and agencies, local government, NGOs and research councils. Yet, it was the House of Commons Environmental Audit Committee that, by calling for such an assessment in 2007, made it happen.
The academic community has not risen to the challenge of measuring cultural value, because there has been insufficient, genuinely multidisciplinary dialogue. Far too much has been written that is critical of other disciplinary perspectives and far too little on how different intellectual disciplines can work together constructively to deepen our understanding of the value of culture.
I have long argued that in the UK the Research Councils are best placed to address the gap in leadership. They should lead an ambitious programme of multi-disciplinary applied research. This programme should be inclusive and involve economists, anthropologists, psychologists, philosophers, sociologists, political scientists and art form specialists. The dialogue must be open and eclectic in its methodological approach, and must not be hijacked by disciplinary prejudices. It should combine world-class academic rigour with grounding in the actual practice of cultural institutions, which calls for a different, much closer, relationship between cultural institutions and empirical researchers than has been the case to date.
I am delighted to have learned just last week from the Arts and Humanities Research Council (the AHRC), that it has responded to this challenge and is announcing a £2 million multi-disciplinary cultural value programme to pursue just this empirical agenda with cultural institutions. I have been particularly keen to stress to the AHRC the importance of building in international collaborations into this programme – between both researchers and cultural institutions – and I hope this is an area where we can see significant UK-Australian collaborations.
I should at this point say a few words specifically about measurement. Even though they often seem to defy logic, public policies are usually built on someone's implicit judgment about the perceived costs and benefits of intervention. This is a version of Keynes's maxim that "practical men, who believe themselves to be quite exempt from any intellectual influences, are usually the slaves of some defunct economist.” A growing emphasis on public accountability in recent decades has meant that any use of public money must be justified on explicit grounds, giving rise to the concept of evidence-based policymaking.
Like it or not, this places measurement at the heart of public policy. Some forms of value, such as jobs created or exports generated, are easier to measure than others. But there is a danger that more difficult-to-measure benefits, such as aesthetic, spiritual or social value – all of which the public associates with culture – are under-emphasised in the cost-benefit calculus.
Of course, the risk of neglect from non-measurement is heightened in an environment of financial cuts that we are experiencing in the UK and that we will experience for many years to come. Policies that create measurable value are favoured. Those which enhance social welfare in ways that are less straightforward to measure are often seen as lower priority and fall victim to the spending axe.
In my view, to shy away from measurement in these circumstances, and rely instead on the goodwill of individual politicians who view the arts favourably, is a gross error of judgment. Culture simply matters too much to be left to the whims of individual politicians. I have friends in the cultural sector who envy places like Berlin, where the local authorities are investing hundreds of millions of Euros in the city's cultural quarter, with seemingly no concern for explicit valuation. I tell them to be careful what they wish for. As well as risking poorly informed allocation of public resources, policymaking based on implicit judgments also makes it difficult to challenge established conceptions of what is, and is not, of cultural value.
In the remaining time I have available I'd like to illustrate with a practical example of what I mean by a multi-disciplinary approach which embraces both economic and cultural concepts of value. Which I hope shows that the application of hard economics and understanding of cultural value need not be incongruous. As befits a presentation given by a Brit in Sydney, this example is based on an Anglo-Australian collaboration between myself and David Throsby. I want to argue, through this example, that the theories of economic value and cultural value, when used together, can lead to an approach to cultural policymaking that is more closely aligned with what people actually want and value.
In June 2009, the Royal National Theatre in London became the world's first theatre to broadcast by satellite to cinemas in HD a live theatrical production. Sir Nicholas Hytner's production of Ted Hughes' translation of Racine's play Phèdre was seen live on 73 digital cinema screens at 70 unique sites in the UK, and was relayed to 210 further sites in the rest of the world. A total of 14,000 across the UK saw that evening's production (excluding those who experienced it at the National itself). A further 14,000 people saw it live across Europe or on the same day in North America (allowing for time zone delays). Including those cinema audiences in other countries (including Australia) who saw the production at a later date, it is estimated that more than 50,000 people saw Phèdre as it was performed on 25th June 2009.
Who were the new audiences attracted to the live screenings? How did they view their experience compared with that of audiences at the production at the NT itself? To answer these questions, David and I worked with the National Theatre and 35 of the participating UK cinemas to collect detailed survey data from those who saw Phèdre in the cinema and those who saw it at the National. The data allowed us to compare the NT Live cinema audience and the traditional audience for the National's productions at London's South Bank.
The surveys included questions addressing how respondents first found out about the production, their reasons for attending, their prior expectations about the performance and how they felt afterwards, their willingness to pay for theatre and live screening events, and the impact of their experiences on the likelihood of attending theatre and live screenings in the future. Key socio-demographic information, including age, gender, employment status, educational background and income was also collected.
Self- evidently, the National's NT Live broadcasts had allowed the theatre to expand its 'virtual capacity'. The audience for Phèdre over its whole run was doubled through the screening of the single performance on 25th June 2009. According to our audience surveys, almost half of the cinema-goers on that night said the main reason why they had not seen Phèdre at the theatre itself was because the National's location was too far away or because they had been unable to get tickets. While the great majority of the theatre audience had been to the NT in the previous 12 months, this was the case for only 41 per cent of audiences who saw Phèdre at the cinema.
The NT had reached only a relatively small number of 'theatre novices' through the broadcast however: less than 10 per cent had not been to the theatre in the previous year.
Perhaps NT Live's most striking extension in audience reach was a broadening of low-income audiences: one-third of cinema audiences had incomes of lower than £20,000 per year, compared with just over one-fifth in the case of the theatre. The potential of digital technology to help overcome the traditionally observed concentration of theatre audiences amongst relatively affluent consumers is clear, echoing a similar finding in separate research we conducted with the Tate Gallery.
One of the most significant findings was how NT Live appeared to complement, not substitute for the theatre in the eyes of the public. Over one-third of cinema audiences said that, having seen the NT Live screening of Phèdre, it was now more likely that they would attend a live performance at the National, and almost the same number again said they would do so at another theatre.
But survey findings can give notoriously imperfect, and possibly biased, indicators of audiences' future intentions. Which is why it was so interesting that the field experiment component of the research was consistent with the survey findings. By including with Nesta's agreement enough participating cinemas in the commuter belt of the National's South Bank location, and having access to detailed information on the postcodes of theatre bookers for Phèdre and similar productions, we were able to test whether districts in the catchment of participating cinemas were over- or under- represented at the NT box office compared with districts with no cinemas.
We found that there were, if anything, greater numbers of theatre goers (on average over 50 per cent more) from the catchment areas of participating cinemas. There was no evidence of cannibalisation. It seems as if the NT Live broadcast of Phèdre on 25th June, as well as growing the audience through cinema audiences, also grew audiences at the National Theatre itself.
As well as allowing us to establish these complementarities, the surveys allowed us to conduct an in-depth study of whether the live broadcast had created new or unexpected experiences for audiences. Were there new forms of cultural value created, different from those generated by theatre in its traditional mode of production? And, if so, could these be measured in ways that the NT could use to evaluate its work and funders use to evaluate their decisions?
What we found was counterintuitive.
Quite a lot is known about the emotional and aesthetic value that audiences derive from live theatre. So we were not surprised that the experiential aspects of attending the performance were valued by the theatre and cinema audiences alike. In both cases, the chance to see the actors 'up close', as well as the ambience and comfort of the venue were reported as important factors when deciding whether or not to attend.
Both groups also anticipated enjoying the 'buzz' of a live experience, an expectation that was clearly more than validated. So, for example, almost 85 per cent of cinema audiences reported feeling 'real excitement' because they knew the performance they were watching was taking place 'live' at the National Theatre. This finding suggested that there are limits to the 'anywhere, anytime' attitude towards the consumption of cultural content. It would seem that there does exist a 'right time' (live, as it happens) and a 'right place' (a cultural venue, whether a theatre or a cinema) to enjoy some cultural experiences. Responses to an additional question we included in the survey of audiences for another NT Live production, All's Well That Ends Well, provided additional support for this claim: less than half of audiences reported that they would have watched the production if it had been streamed live online, and of these less than half again said they would have been willing to pay for that experience.
This all seemed quite intuitive. What we were surprised to find was that the experience was significantly more emotionally engaging and aesthetically pleasing for cinema audiences than theatre audiences – a result that generated so much debate that we gave it its own name: 'Beyond Live'.
We took the different aspects of the production's cultural value for audiences, as measured by our survey questions, and used econometric models to explore their relationship with the economic valuations, as measured by audiences' willingness to pay. Although willingness to pay is notoriously difficult to elicit through surveys, we found, intuitively, that it was greater amongst older audiences (at least at the cinema) and higher income people. There was some evidence for a higher willingness to pay amongst less frequent cultural consumers, again confirming the potential of live screenings to help expand the traditional theatre audience. Similar tendencies were apparent amongst those cinema audiences who had not been to the National in the previous year, with a relatively higher willingness to pay for cinema tickets amongst the less well-educated members of this group.
We found that the elements of cultural value most clearly associated with consumers' economic valuation of their experiences were the aesthetic or symbolic value indicated by their emotional response, and the social value of the group experience.
Although only a tentative foray into understanding the complex relationship between audiences' economic and cultural valuations, our study of NT Live nonetheless pointed to ways in which a marriage between economic and cultural value might be achieved.
Where economic measures of value can be shown to capture and summarise the many-sided aspects of cultural valuations, we can be more comfortable that when choices are made between spending on culture and spending on other goals, the value placed on culture is more inclusive. But importantly, the approach also helps to show us where cultural value is not adequately reflected in economic valuations. As such, it raises the tantalising prospect of policymakers making holistic, yet integrated, assessments of costs and benefits when allocating public resources, using the tools of cultural economics.
The quid pro quo of all of this is that our cultural leaders must jettison the notion that the value of culture is wholly mysterious, something that can never be measured. They must accept that if they want state funding then they must demonstrate value on the state's terms, which naturally includes the economic. In the same way that they would expect to demonstrate commercial value if they were to seek funding from private investment sources, or social value if they were to attract funding from a philanthropist.
Thank you for your attention.
Hasan Bakhshi is Director, Creative Industries in Nesta's Policy and Research Unit and Research Fellow at the ARC Centre of Excellence for Creative Industries and Innovation at Queensland University of Technology.
Stay in touch on Twitter at www.twitter.com/nesta_uk on Facebook at www.facebook.com/nesta.uk and on our website at www.nesta.org.uk
Nesta 1 Plough Place London EC4A 1DE www.nesta.org.uk
