Five principles for measuring the value of culture
Having 7 years ago moved from the financial markets where the whys - if not necessarily the how tos - of valuation were straightforward, to an environment where valuation is so contested, I think we both over-complicate and over-simplify the issues in culture.
We over-complicate them in our endless debates about the validity of the exercise – the argument that intrinsic value cannot be measured – when it is obvious that key stakeholders need metrics. But we over-simplify them in not acknowledging that there are multiple stakeholders – funders, audiences, organisations, creative talent, taxpayers – each with their own values, and who therefore have different needs from a valuation system. In our understandable desire to make valuation more tractable, we've neglected the fact that all these needs are valid and that the challenges are, as a result and however inconveniently, complex.
Happily, as progress in other difficult-to-value-areas like the environment and health shows, these challenges are not insurmountable. A great deal more can be done to develop evaluation methodologies, building on the work of public and cultural economists, public service broadcasters, qualitative researchers and, increasingly, private sector businesses who, for a number of reasons, are seeking a broader account of the value they generate. And as in the economy more generally, digital technologies are also opening up new sources of data on preferences for culture where once we relied just on focus groups and surveys.
More and more people are waking up to the need for more holistic attempts to measure value in the cultural sector – in the UK, we have initiatives like Geoff Crossick's Cultural Value Project for the AHRC, for example, and Vicki Heywood's new Warwick Commission on the Future of Cultural Value. In Europe, the European Commission is promoting the measurement of cultural value using wellbeing approaches. And in countries like the US and Australia, governments are introducing satellite national accounts for cultural statistics.
In the rest of my comments I thought I'd set out a handful of principles which I think could usefully be born in mind when building a measurement system for cultural value, illustrating with some examples.
PRINCIPLE 1. We must get more comfortable working with economists.
Of course, no one discipline has a monopoly on valuing culture. But what sets apart economic measures from others is that they are commensurable – that is, they have a common unit of account - money - which enables the value of culture to be compared with other social goods. This is of obvious use to funders. A basic task therefore is to use the tools of microeconomics to measure the welfare consumers derive from major cultural investments.
This is of course easier when cultural consumption is mediated through markets as then we can observe what consumers are willing to pay. But more cultural institutions should follow the example of those including the BBC in using the techniques of public economics to capture dimensions of economic value that are not reflected in market prices: like existence value, when people regard the existence of a cultural artefact or institution to be of value to themselves or their community, even if they do not enjoy use benefits themselves; or option value, where people wish to have the option that someday they, or someone else, may want to engage with it.
A recent - but rare - example of a contingent valuation study in the cultural sector, by the British Library, estimates that the public places a value of over £410 million on its continued existence, over and above any use value that the British Library's readers, researchers and other users derive.
PRINCIPLE 2. The valuation agenda should be multi-disciplinary.
A fundamental empirical task is to establish when measures of economic value do NOT capture cultural value. In such cases it becomes especially important to develop effective ways of measuring the value of culture in its own terms. How can we articulate, and if possible measure, cultural value when there are no standardised units of account? How do these measures square up against economic valuations based on WTP, and can the relationship between the two be formally understood? Can we make informed judgements about when economic measures may serve indirectly to capture cultural value and when they do not?
In 2010, the Australian cultural economist, David Throsby and I took this approach in a study with the National Theatre. We found that the elements of cultural value most clearly associated with audiences’ WTP included the social value of the group experience and their emotional response, whereas the link between WTP and the aesthetic value as indicated, for example, by their reported absorption in the show was less clear. The questions we used to survey audience experiences were of the same mould as those used in Channel 4's Public Value Trackers.
PRINCIPLE 3. Valuation in culture is beset by extreme uncertainties, so the inquiry should be iterative.
Who are the stakeholders for culture? What are their values? Can these be measured? Can they be aggregated? What methodologies should be used? These are complex questions to answer. Naturally, the uncertainties are magnified in the case of new cultural experiences, services and products. Cultural institutions and funders must as a consequence embark on valuation with a spirit of experimentation and curiosity. Testing with the relevant stakeholders what metrics are fit for purpose, and refining them where they are not.
In the New year, Arts Council England will be working with a group of cultural organisations in Manchester to publish the results of a fascinating project to develop quality of artistic experience indicators. The indicators have been refined and patiently trialled with stakeholders, including with the public in Manchester and Cumbria. The plan now is to test how well the methodology works for organisations in other cities. As this is at an early stage of development, no conclusions have been reached about how the metrics will be used in funding decisions, nevertheless the potential is clear. This seems, to me, to be a great example of the open-ended research we need in this area.
PRINCIPLE 4. The valuation agenda should make use of new sources of observational data to supplement traditional survey and focus group methods.
It is well known that businesses in all sectors are making use of new sources of online data and analytic techniques to better understand their audiences and grow their organisations.
In a new study using firm-level data that Nesta will shortly publish, we find that UK businesses who make greater use of their online customer data are, controlling for other determinants of performance, significantly more productive and more profitable than their competitors. In the cultural sector too we should explore how we can use new sources of data for analysis, including for valuation purposes.
In the past, we have tended to rely on large-scale surveys and time-intensive qualitative work to collect insights. The use of controlled experimental methods – the gold standard, whereby exogenous variation is introduced into the experience audiences face to identify the additional value created – have always been rare in the cultural sector, at least in published research (the National Theatre study I mentioned earlier being an exception).
But large-scale surveys, qualitative work and experiments have one thing in common: they are expensive. The fact that so much culture these days is mediated online and through other digital channels, however, opens up new opportunities for the sort of “action evaluation” I mentioned earlier.
An example of publicly-funded research is Nesta's AHRC-funded collaboration with the novelist, David Mitchell and the Beijing-based social network, Douban. Social media platforms allow artists to reach audiences, tap into the wealth of information being shared on these platforms and involve them in ways that can deliver mutual benefit. This is particularly so in China, where social media sites occupy a special place in the lives of the public – not only in terms of the time they spend on them, but as a gateway to obtain content more generally.
We are, through Douban, making available for the first time Chinese language translations of two of David’s short stories. As well as analysing the qualitative content of the fan discussions we will over the next year be collecting data on how many copies of the stories are sold on Douban’s ebook platform, and what other things do the buyers like, discuss and purchase, as captured by their activity on Douban.
The eventual aim of such experiments is to generate databases of what – in this case Chinese – audiences value and how this translates into economic value. The potential for this sort of research – Douban has a movie and music, as well as a books, platform – is quite simply huge.
PRINCIPLE 5. The valuation agenda must also consider positive ‘spillovers’ from cultural activity that are not covered by standard valuation methodologies.
There are a number of different mechanisms by which these spillovers – which are usually unintended – might happen.
Through skills development within the cultural sector, for example, which – when combined with labour mobility – benefits employers in the wider creative industries.
A 2008 Nesta study with Central St Martins College of art and design, for example, demonstrated a striking tendency for fine arts graduates to identify skills like problem-solving and the ability to work independently as their main strengths – precisely the skills that employers say are their priorities in surveys of skills needs. In another study, of careers in subsidised and commercial theatre, undertaken with Arts Council England and Creative & Cultural Skills, we found that talent was much more likely to identify subsidised than commercial theatre as the place where they can experiment and develop new work.
And in another paper published this year, with the National Endowment for the Arts and the Brookings Institution in Washington, we reported evidence consistent with knowledge spillovers from cultural work to the wider creative industries in English cities. And, in particular, that average wages in creative industries jobs (economists' proxy for productivity) tend to be significantly higher, controlling for other variables, in cities with significant clusters of cultural sector jobs.
So, to conclude, although the challenges are great, I think there are reasonable grounds for optimism about the possibilities for greater sophistication in the way we value culture, at least on technical and analytical grounds. Whether or not we actually see progress is, however, as much a question of political will as it is technical opportunity. Valuation data and research has a way of raising questions which we might prefer not to think about. And it can be abused and used in undesirable ways that leaves, with good reason, many in the cultural sector nervous about producing it.