Five principles for measuring the value of culture

Having seven years ago moved from the City where the why’s – even if not the how to’s – of valuation seemed straightforward, to an environment where it is so contested, I think we both over-complicate and over-simplify the issues in culture.

We over-complicate them in our endless debates about the validity of the exercise – the argument that intrinsic value cannot be measured – when it is obvious that key stakeholders need metrics. But we over-simplify them in not acknowledging that there are multiple stakeholders – funders, audiences, organisations, talent, taxpayers – each with their own values, and who therefore have different needs from a valuation system. In our understandable desire to make valuation more tractable, we've neglected the fact that all these needs are valid and that the challenges are, as a result and however inconveniently, complex.

Happily, as progress in other difficult-to-value-areas like the environment and health shows, these challenges are not insurmountable. A good deal more can be done to develop evaluation methodologies, building on the work of public economists, public service broadcasters, charities and, increasingly, private sector businesses who for a number of reasons are seeking a broader account of the value they generate.

More and more institutions are waking up to the need for more holistic attempts to measure value in the cultural sector – in the UK, we have initiatives like Geoff Crossick's Cultural Value Project for the AHRC, for example, and Vikki Heywood's new Warwick Commission on the Future of Cultural Value, both of which I am privileged to be involved with. In Europe, the European Commission is promoting the measurement of cultural value using wellbeing approaches. And in countries like the US and Australia, governments are introducing satellite national accounts for cultural statistics for the first time.

Here I set out a handful of principles which I think might usefully be born in mind when building a measurement system for cultural value.

PRINCIPLE 1. We must get more comfortable working with economists

Of course, no one discipline has a monopoly on valuing culture. But what sets apart economic measures from others is that they are commensurable – they have a common unit of account, money – which enables the value of culture to be compared with other social goods. This is of obvious use to funders.

A basic task therefore is to use the tools of microeconomics to measure the welfare the public derives from major cultural investments. This is easier when culture is mediated through markets, as then we observe what consumers are willing to pay.  

But we can use the techniques of public economics to also try and capture dimensions of value that are not reflected in market prices: like existence value, when people regard the existence of a cultural artefact or institution to be of value to themselves or their community, even if they do not enjoy use benefits themselves; or option value, where people wish to have the option that someday they, or someone else, may want to engage with it. A recent – but rare – example of a contingent valuation study in the cultural sector, by the British Library, estimates that the public places a value of over £410 million on its continued existence, over and above any use value that the British Library's readers, researchers and other users derive.

PRINCIPLE 2. The valuation agenda is multi-disciplinary

A fundamental task is to establish when measures of economic value do not capture cultural value. In such cases it becomes especially important to develop effective ways of measuring the value of culture in its own terms. How can we articulate, and if possible measure, cultural value when there are no standardised units of account? How do these measures square up against economic valuations based on willingness-to-pay, and can the relationship between the two be formally understood? Can we make informed judgements about when economic measures may serve indirectly to capture cultural value and when they do not?

In 2010, David Throsby and I took this approach in a study with the National Theatre. We found that the elements of cultural value most clearly associated with audiences’ willingness to pay included the social value of the group experience and their emotional response, whereas the link between willingness to pay and the aesthetic value indicated, for example, by their reported absorption in the show was less clear.

PRINCIPLE 3. Because valuation in culture is beset by uncertainties, the inquiry should be iterative

Who are the stakeholders for culture? What are their values? Can these be measured? Can they be aggregated? What methodologies should be used? These are complex questions to answer. Naturally, the uncertainties are magnified in the case of new cultural experiences, services and products. Cultural institutions and funders must, as a consequence, embark on valuation with a spirit of experimentation and curiosity. Testing with the relevant stakeholders what metrics are fit for purpose, and refining them where they are not.

In the New Year, Arts Council England will be working with a group of cultural organisations in Manchester to publish the results of a fascinating project to develop quality of artistic experience indicators. The indicators have been refined and patiently trialled with stakeholders, including with the public in Manchester and Cumbria. The plan now is to test how well the methodology works for organisations in other cities. As this is at an early stage of development, no conclusions have been reached about how the metrics will be used in funding decisions, nevertheless the potential is clear. This seems, to me, to be a great example of the open-ended research we need in this area.

PRINCIPLE 4. The valuation agenda should make use of new sources of observational data

It is well known that businesses in all sectors are making use of new sources of data to better understand their markets and grow their organisations. In a new study using firm-level data by myself, Albert Bravo-Biosca and Juan Mateos-Garcia which we will shortly publish, we find that UK businesses who make greater use of their online customer data are, controlling for other determinants of performance, significantly more productive and more profitable than their competitors. In the cultural sector too, more of us should follow the example of organisations like Channel 4 in how we can use new sources of data for analysis, including for valuation purposes.
 
In the past, we have tended to rely on large-scale surveys and time-intensive qualitative work to collect insights. The use of controlled experimental methods – the gold standard, whereby exogenous variation is introduced into the experience audiences face to identify the additional value created – have always been rare in the cultural sector, at least in published research (our study with the National Theatre is an exception).
 
But large-scale surveys, qualitative work and experiments have one thing in common: they are expensive. The fact that so much culture these days is mediated online, however, opens up new opportunities for the sort of “action evaluation” I advocated earlier.
 
An example of publicly-funded research is Nesta's collaboration with the novelist, David Mitchell and the Beijing-based social network, Douban. Social media platforms are allowing artists to reach audiences, tap into the wealth of information being shared and involve fans in ways that they value. This is particularly so in China, where social media sites occupy a special place in the lives of the public – not only in terms of the time they spend on them, but as a gateway to obtaining content more generally.

We are, through Douban, making available for the first time Chinese language translations of two of David Mitchell's stories. As well as analysing the qualitative content of the fan discussions we will over the next year be collecting data on how many copies of the stories are sold on Douban’s ebook platform, and what other things do the buyers like, discuss and purchase. The eventual aim of such experiments is to generate databases of what – in this case, Chinese – audiences value and how this translates into economic value. The potential for this sort of research – Douban has a movie and music, as well as a books, platform – is quite simply huge.

PRINCIPLE 5. We should consider positive ‘spillovers’ from cultural activity that are not covered by standard valuation methodologies

There are a number of different mechanisms by which such spillovers might happen. Through skills development within the cultural sector, for example, which – when combined with labour mobility – benefits employers in the wider creative industries. As we found in our study of the careers of fine arts graduates with Central St Martins College of art and design, for example, or of careers in subsidised theatre, published this year with Arts Council England and Creative & Cultural Skills. Or there may be 'knowledge' spillovers from cultural work to the wider creative industries at a local level.
 
To conclude, although the challenges are great, there are strong grounds for optimism about the possibilities for greater sophistication in the way we measure the value of culture. However, whether we actually see progress is as much a question of political will as it is technical opportunity. Valuation research has a way of raising questions which we might prefer not to think about. And it can be abused and used in undesirable ways that leaves many in the cultural sector nervous about producing it. I look forward to working with Vikki Heywood and my fellow commissioners on the Warwick Commission to address this challenge.