Looking out into a cathedral’s interior, learning about societies past and present in a museum, walking down a historic street. Engaging with culture and heritage brings enjoyment and meaning to many. Even if we do not directly visit historic and cultural sites, we may place a value on their existence. Perhaps we like their appearance, having the option of visiting or consider it is important that they are preserved for future generations.

Many of us intuitively feel that these values are somehow transcendent, but in practice economic resources are required to support the experiences that create them. This raises the question of whether we can estimate how much we value them in financial terms. However, as many of these experiences are free to access, or not commercial activities, they lack direct financial measures of the value that people place on them. Economic techniques have been developed that can estimate people’s valuations in this context and have been shown to produce credible results. These techniques can be complex and resource intensive though, restricting their application.

If one could demonstrate that the estimated valuations produced by these techniques are realistic and similar for comparable historic and cultural sites, then pre-existing research findings on valuation could be applied to other sites (so called Benefits Transfer). This would reduce the costs of estimating the value that culture and heritage creates. Two recently published studies that we have undertaken with Simetrica help address this issue. The studies, which are available via the links below, looked at estimating the value that both users and non-users obtain from:

1. Museums
2. Historic cities and their cathedrals

These studies also looked at the extent to which value estimates can be transferred between sites of the same type, i.e. between museums, between cathedrals and between historic cities, opening up the possibility for the values to be used in similar contexts.

The first study was funded by the Department for Digital, Culture, Media and Sport (DCMS) and the second study was funded by the Arts and Humanities Research Council (AHRC).

The valuations were estimated by analysing:

• Four museums: The Great North Museum, Newcastle; The National Railway Museum, York; The Ashmolean Museum, Oxford and The World Museum, Liverpool.
• Four historic cities: York, Lincoln, Canterbury and Winchester (and their cathedrals).

These sites being chosen on the grounds that they were considered broadly comparable in terms of size. The estimated valuations were obtained using a technique known as contingent valuation in which, to assess how they value the sites, survey respondents were asked a question about their willingness to pay (WTP) to protect these sites and institutions in a hypothetical, but realistic sounding, scenario. The data was collected by means of online surveys, which were piloted before being fully deployed. For each valuation category, e.g. cathedral visitors for a given city or museum non-visitors for a given museum, a minimum sample size of 250 respondents was achieved.

The approach followed was consistent with the UK Treasury’s Green Book which provides guidance on how appraisal and evaluation should be undertaken in central government. This means that the findings should be usable in the context of business cases in government and where public benefit is a consideration more generally.

Estimating value is subject to a number of potential sources of bias and several measures were taken to help mitigate the risk of this in the research. For example, the surveys included information on the historic city, cathedral or museum being valued so that, in so far as possible, those surveyed were making an informed decision. Checks were also implemented within the surveys to avoid people giving unrealistic answers, for example by preventing respondents from supplying open-ended valuations, or going through the survey too quickly to have given considered responses. Respondents were asked to attest to the accuracy of their valuation. Data on the demographics of survey participants was collected to check the representativeness of those surveyed and, where possible, the data weighted to ensure it reflects the underlying population whose valuations we were trying to estimate.

The World Museum Liverpool. Photograph by Jonathan Oldenbuck (Creative Commons)

In the museums study, valuation was estimated through survey respondents’ willingness to pay to prevent a hypothetical scenario of museum closure due to funding cuts. For museum visitors this was measured by means of willingness to pay an entry fee that would be imposed due to financial circumstances (all the museums studied have free entry) and for non-visitors the payment (as they would not be subject to the entry fee) was in terms of an annual donation to help preserve the collections. Those surveyed were counted as visitors if they had visited in the past three years.

The value as measured by individuals' willingness to pay averaged across all four museum sites was found to be:

• Museum visitors (use value): Mean entry fee WTP = £6.42
• Non-visitors (non-use value): Mean annual donation WTP = £3.48

The estimated use and non-use values are given credibility by the fact that they are comparable with previous studies such as the 2015 AHRC study undertaken by Nesta and Simetrica, which looked at the Natural History Museum and Tate Liverpool. The sample sizes are above 1,000 respondents each for both museum visitors and non-visitors. Here and elsewhere averages are inclusive of respondents who submitted a £0 valuation.

The extent to which it is possible to transfer values between sites was assessed by examining whether it was possible to do this within the sample. In the simplest form of this each museum's values were selected in turn and predicted using the average of the estimated values from the remaining museums. This was done for all the sites, for both users and non-users, and the percentage prediction errors averaged (the mean transfer error) to provide a measure of how accurate transferring values between sites was. The table below shows the results of this and also the largest error from all of the predictions (the max transfer error). The mean transfer errors are between 9 and 17%, which is consistent with what would be regarded as low enough to undertake benefits transfer in the academic research literature.

In addition, the analysis was also done while adjusting the prediction for differences in survey respondents' incomes between the sites and also for other demographic characteristics such as age and gender. In the case of museum visitors, this did not make much difference to the prediction errors, and in the case of museum non-visitors the effect was to make the predictions worse, as a result these are not shown. The poor performance of the adjusted predictions relates to it not being possible to explain most of the variation in individuals' use and non-use values using the sample’s demographic characteristics.

The similarity of values across the different museums, as shown by the relatively low mean transfer errors, does though support the proposition that the study's estimated average values can be used as robust estimates of the use and non-use values for museums comparable to those in the study.

Lincoln and its Cathedral. Photograph by John Davies (Creative Commons)

The valuations for the historic cities and cathedrals study were estimated in terms of willingness to make a one-off donation for measures to reduce the risk of closure and damage to historic buildings in the city (including its cathedral) due to a hypothetical scenario of the effects of future climate change and financial circumstances. Historic city users were defined to include both city residents and tourists from the past three years. All visitors to the cathedral were also surveyed as city visitors/users. This information was obtained through an online survey of people who had visited the city and cathedral and a survey of the general population that had not visited.

Where a historic city value was given, the cathedral valuation was obtained using the share that those surveyed indicated were prepared to allocate to the cathedral from their overall city-wide donation, otherwise they were asked a specific question about valuing the cathedral.

The value as measured by individuals' willingness to pay averaged across the cities and cathedrals was:

• Historic cities Resident/Visitor (use value): Mean WTP = £9.63
• Historic cities non-user (non-use value): Mean WTP = £6.14
• Cathedrals visitor: Mean WTP = £7.42
• Cathedrals non-visitor: Mean WTP = £3.75

As with the regional museums study, the research also assessed the extent to which it would be possible to apply the valuations obtained to other sites, by comparing the valuations within the sample. The table below shows the results of this assessment of benefits transfer where for each site (cathedral and historic city in turn) the valuation was estimated using the average of the three other cathedrals or historic city sites. This turned out to be a relatively good predictor of the values of the remaining sites reflecting that they had similar values. As can be seen, the mean transfer error was fairly low for both historic cities, cathedrals (and users and non-users) within the levels that would be typically considered acceptable in the academic literature for benefits transfer.

It was also attempted to do benefits transfer controlling estimates for differences in incomes and population, however this did not produce consistently better results. This is because, as in the museums study, it was not possible to explain most of the variation in terms of use and non-use values across individuals using the demographic characteristics of the survey sample, such as age, gender and income.

The similarity of values across the sites as shown by the low mean transfer errors supports the proposition that the study's estimated average values can be used as robust estimates for the use and non-use values of historic cities and cathedrals comparable to those in the study.

The research has shown that it is possible to use economic valuation techniques to assess the values that both users and non-users of cathedrals, historic cities and museums place on them. The values obtained are credible in amount and are comparable between institutions and places considered to be broadly similar. This supports these values being applied to other comparable cultural assets and institutions to provide estimates of the values that they are generating for users and non-users in assessments of their public value. There should therefore be a greater emphasis on using such economic valuation techniques to measure the benefits that heritage and culture provide and this should be more closely integrated into policy.

The willingness to pay scenarios used to assess the values were different, reflecting the differing contexts. As a result the values of the museums and historic cities/cathedrals should not be considered directly comparable. Among the sites studied, all the museums have free entry and receive public funding, while the cathedrals have entry fees for non-religious visiting and are not funded by the state. Most historic buildings are in private ownership and receive no public funding. Survey respondents' decisions about valuation are therefore not solely valuations about the cultural assets or institutions, but also reflect the financial settings within which they operate.

As it was not possible to explain most of the variation in people’s valuations in terms of their demographic characteristics, such as age, gender, income, this means it is difficult to adjust for characteristics that affect people’s valuation. The issue of what determines the variation in valuations beyond naturally occurring variation among individuals remains to be answered. In addition to this, in an analysis of cost benefit there would also need to be an assessment of the costs of providing the services that lead to the benefits and this was not directly analysed in this project.

The cathedral and museums sites studied are comparatively large and so are not representative of smaller museums, historic and religious buildings. A different approach would therefore be needed to value such smaller sites. Given the significant number of smaller arts, museums and historic sites and institutions, and their collective importance, there is a risk of inconsistency and substantive costs in widespread and uncoordinated valuation analysis. Assessing the values that these sites create would therefore benefit from a strategic approach.

References:

Bakhshi, H. (2013), 'Five principles of measuring cultural value', Nesta.

Bakhshi, H., Fujiwara, D., Lawton, R. N., Mourato, S., & Dolan, P. (2015). 'Measuring Economic Value in Cultural Institutions', Arts and Humanities Research Council.

Fujiwara, D., Bakhshi, H., Mourato, S., Hotopp, U., Lawton, R., Lagarde, A. Davies, J. (2018), ‘The Economic Value of Culture: A Benefit Transfer Study’, Nesta and Simetica a report for DCMS.

Maddison, D., & Mourato, S. (2001). Valuing different road options for Stonehenge. Conservation and Management of Archaeological Sites.

Mourato, S., Fimereli, E., Contu, D., Gaskell, C., & Boniatti-Pavese, C. (2014), 'The Economic Benefits of Cultural Built Heritage Interiors Conservation from Climate Change Damages in Europe', LSE.

Lawton, R., Fujiwara, D., Mourato, S., Bakhshi, H., Lagarde, A., Davies, J. (2018), ‘The Economic Value of Heritage: A Benefit Transfer Study’, Arts and Humanities Research Council.

HM Treasury (2018), ‘The Green Book Central Government Guidance On Appraisal And Evaluation’.

Photo credits

Lead photo credit: Canterbury Cathedral Choir, Photograph by David Iliff (CC BY-NC-SA 2.0 )

Text photo credits: The World Museum Liverpool. Photograph by Jonathan Oldenbuck (Creative Commons)

Lincoln and its Cathedral. Photograph by John Davies (Creative Commons)