About Nesta

Nesta is an innovation foundation. For us, innovation means turning bold ideas into reality and changing lives for the better.

We use our expertise, skills and funding in areas where there are big challenges facing society.

Nesta is based in the UK and supported by a financial endowment. We work with partners around the globe to bring bold ideas to life to change the world for good. www.nesta.org.uk

If you'd like this publication in an alternative format such as Braille, large print or audio, please contact us at: [email protected]

Design: Green Doe Graphic Design Ltd

Summary

Recent decades have brought periodic waves of interest in finding better ways of mapping and measuring public and social value - outcomes, institutions and services that are valued by the public but not easy to count in the way that the monetary value of cars or computers can be. Governments and parliaments have sometimes tried to encourage this. In the UK for example, the Social Value Act explicitly encouraged broader notions of value, as have recent revisions of the Treasury Green Book used to assess public spending choices.

But these generally remain underdeveloped and lack influence compared to more traditional market valuation methods. Within government there are still far more developed means for measuring investment in physical things - railways, roads, airports - than in intangibles like reducing social isolation, improving health or the arts. This almost certainly continues to skew public spending.

It's obviously wrong to claim that you can't manage what you can't measure. But the lack of well-established metrics for public value in all its forms makes it harder to compare alternatives and continues to bias public spending against prevention rather than cure, against indirect benefits relative to direct ones, and against intangibles relative to physical objects.

This paper sets out alternatives. It brings together views from Nesta on better ways of mapping and measuring public value. It builds on work Nesta has done in many fields from health and culture to public services - to find more rounded and realistic ways of capturing the many dimensions of value created by public action. It is relevant to our work influencing governments and charities as well as to our own work as a funder, since our status as a charity commits us to creating public benefit.

Our aim in this work is to make value more transparent and more open to interrogation, whether that concerns libraries, bicycle lanes, museums, primary health services or training programmes for the unemployed. We recognise that value may come from government action; it can also be created by others, in civil society and business. And we recognise that value can often be complex, whether in terms of who benefits, or how it relates to values, as well as more technical issues such as what discount rates to apply.

But unless value is attended to explicitly, we risk ending up with unhappy results. This is why we have focused so much on how to both understand as well as create value, through setting up What Works Centres and the Alliance for Useful Evidence; through impact investment funds that use more precise metrics for social impact; through projects in health that recognise what is important to people themselves; through projects to measure the value of culture; and through programmes like 'Innovate to Save' in Wales that aims to achieve cashable and other savings from innovations.

This paper documents work in progress but we hope its concepts, and applications, will be useful. It shows that:

1. Despite the tensions and dilemmas, public value - in all its forms can be mapped and often measured, in ways that are useful for shaping decisions.
2. The best valuation methods avoid the twin errors of, on the one hand, believing that a single measure can adequately capture complex values, and on the other, believing that decisions should just be guided by hunch or intuition.
3. Public value, like value in markets - is not a static fact but arises from the interaction of changing options for supply and changing demands and public priorities.
4. The greatest use of measures of public value comes not just from more detailed analysis but from the conversations and negotiations they prompt. The key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions.
5. Public value methods should help governments decide on actions now that can deliver benefits in the long term, whether on climate change, early childhood or healthcare. This requires new ways of managing public investment, but also a much more sophisticated approach to discount rates.
6. The debate about public value needs to be tied into much more energetic reform of public finance methods, which need to become more transparent, more focused on outcomes, better suited to experiment and innovation and better shaped for public engagement. If this doesn't happen, anachronistic methods for managing finance will crush imaginative attempts to map and measure public value.
7. Valuation has limits: in a democracy it should be possible for citizens, or politicians, to decide that they don't want the state involved either in measuring or seeking to influence important areas of life, including families and communities.

This field is more of a craft than a science. But it's one with a growing range of tools that can in time evolve to become more reliable and rigorous. We advocate more work to seek out value and where possible to measure it and use it to guide spending decisions, and we believe that this will increasingly involve new data sources, including public administrative data. We hope that governments will commit themselves to using public value to guide their decisions and future budgets and to catalyse long-overdue reform to the processes of public finance.

Background: Searching for value

By Geoff Mulgan

Until the 1990s, international accounting conventions assumed that public sector productivity never improved: the value of public provision was assumed to be equal to its cost. Yet in a period of greater public pressure for results and value for money that position became untenable.

For a time, the main focus was on critique - showing the limitations of mainstream economics, and methods like cost-benefit analysis, and how inadequately they captured externalities and social benefits.

But over the last 20 years serious attention has turned to finding better alternatives (though critiques can still pull in a big audience). Public bodies like the BBC commissioned detailed studies of the public value they were creating back in the 2000s and for a time the Cabinet Office promoted new methods of assessing public value. These methods aim to measure not just what public interventions cost, but also provide more rounded accounts of what value they create, and, just as important, what value civil society can provide for public commissioners and purchasers. There have been advances both in economic approaches - like Cost Benefit Analysis (CBA) and the UK Treasury Green Book – and non-economic ones, that are mirrored in many countries around the world. Ambitious ideas are also being developed for more precise measures of social and other goods.¹

There's been a parallel push in civil society, mainly driven by funders wanting better measures of social impact and social returns; better measures for the impact in 'impact investment'; and better ways to compare the results of spending in different fields. A plethora of organisations have been set up to help map and measure value and impact, and - like Global Impact Investment Network – to establish some common standards for fields like clean energy or healthcare. Others offer to measure social value.

At least four distinct communities have been having very similar discussions about value, though with surprisingly little cross-pollination: statisticians attempting to measure wellbeing and outcomes of all kinds (for example brought together in the OECD Better Life Index); economists applying measurement tools to new fields, such as carbon and natural capital; philanthropists and NGOs interested in impact investment; and activists using new measures as a tool for social change (for example measuring or revealing carbon footprints or social neglect).

Not too far from these there has also been an explosion of methods for measuring intangibles - the value of ideas, knowledge, creativity – many of which Nesta has been closely involved in. Some measure innovation spending and the value it creates; others the inputs and outputs of the creative economy. Stian Westlake's book with Jonathan Haskel, Capitalism without Capital, provides an excellent overview.

Better metrics do not of themselves deliver better outcomes. You can't fatten a pig by weighing it. But if you don't have some means of weighing it you may find yourself unable to persuade others that it's as fat as you believe.

Many methods try to put a price on value. The idea that the value to be gained from a new training programme can be directly compared with the value from a health screening programme or water conservation is immediately appealing to busy bureaucrats and ministers who have to justify spending money on one area over another, as it is to foundations.

So how can this be done? Traditional economic valuation methods (based on utilitarian principles) which try to monetise public value usually draw either on what people say they would pay for a service or outcome ('stated preference') or on the choices people have made in related fields ('revealed preference'). These methods also try to adjust the estimated benefits of public services for quality of outcomes – for example comparing school exam results, or the success of operations. As they deliver commensurable estimates, economic methods imply that the benefits of disparate outcomes can be compared and added up. Other economic methods, such as subjective wellbeing,² compare public policy actions by estimating the extra income people would need to achieve an equivalent gain in life satisfaction.

Decision-making becomes a lot easier if any option can be turned into a comparable number, and there have been many attempts to create such tools. Governments would love to be able to use a single metric to compare spending on a scheme for housing the homeless, primary education, bridges and defence, though none have come even remotely close to having such a tool. Other attempts include CBA and, in the social sector, social return on investment (SROI), and more recently Y Analytics and the Impact Multiple of Money, a single number now being promoted to adjust company valuations to take account of all their impacts.

These are all well-intentioned. But paying too much attention to monetary equivalence can lead to bad decisions and I've become sceptical of moves to pretend that a single number can compare very different activities and outcomes. These fall apart intellectually and practically for many reasons (I set out the arguments in more detail in this piece).

Some are reasons of principle (the plurality of approaches to valuation in different fields or spheres); some reasons fall on the boundaries of theory and practice (such as what discount rates to apply to different activities); and some are reasons of practicality (the experience that different methods used to assess value can generate wildly different numbers, and often miss out what people turn out to value most (because people may not be willing to countenance trading off the things they care about most).³ As one recent book put it, value 'is neither an intrinsic quality of an object, nor a reflection of a subjects preferences, but rather something that is organized and brought into existence through mechanisms, technologies and practices of valuation.'⁴ So our attention needs to focus on what these mechanisms are, and whether they are fit for purpose, and how they can be made useful rather than misleading.

A quick survey of the landscape of methods

There are many methods for trying to make sense of value in the public sector and civil society, providing a bridge between the complex patterns of public demands and needs, as expressed through political and other processes, and the changing production systems that keep people healthy, educated or safe.

Cost-benefit analysis (CBA) has been the most widely used, mainly in transport (where in recent years they have been integrated with environmental appraisals) and for big projects (where they are notorious for underestimating costs).⁵ CBA has been subject to criticism for decades but continues to be used, and President Macron recently introduced a requirement that any project costing more than E20 million needed a CBA.

In philanthropy and civil society many methods are used, some linked into standards of evidence of the kind Nesta has promoted. These aim to capture data on impacts; where possible to draw on longitudinal data sets; and ideally the use of counterfactuals or control groups (New Philanthropy Capital recently mapped some 1,000 different methods in use). Often, these attempt to provide versions of 'balanced scorecards' or outcome stars to map value in multiple dimensions.

There are also the many applications of public sector welfare economics, such as environmental economics which has tried to put values on everything from the natural capital of rainforests to carbon emissions. Detailed methods have been in use for several decades to estimate the direct costs of an action (e.g. a drug treatment programme), the probability of it working, and the likely impact on future crime rates, hospital admissions or welfare payments.

Analysis of this kind can be very powerful. In the USA, for example, researchers identified what they called 'million dollar blocks' where the costs associated with criminals topped the million-dollar mark. In principle, good preventive actions targeted at the people living in these blocks might save far more than they cost if they diverted some people from a life of crime.

These analyses can also be made highly sophisticated. They can, for example, be adapted to reflect income distribution, on the principle that an extra dollar, or an extra unit of utility, is worth more to a poor person than a rich one (since 2003 the UK Government's Green Book has required all appraisals to include distributional effects), or time effects (on the principle that if growth continues, the marginal value of an extra £ will be worth less in 50 years' time simply because average incomes will be higher).

But all measurements of complex effects are inherently hard, and they can be costly. The benefits tend to become more dispersed over time, affecting ever more agencies in ever more uncertain ways. Social science is rarely robust enough to make hard predictions about what causes will lead to what effects - there are usually far too many variables involved. Meanwhile standard cost benefit models apply discount rates to these gains, usually based on prevailing commercial interest rates, which renders a benefit in a generation's time virtually worthless.

An even more fundamental problem is that these analytic methods presume that everyone agrees on what counts as valuable. But in many of the most important fields for government action or philanthropy – like childcare, crime prevention or schooling - the public are divided over values as well as value. For most people, for example, there is an intrinsic virtue in punishing criminals regardless of the costs and benefits of alternatives to prison.⁶ This is why the economic models for thinking about public goods and externalities, though informative, are often inadequate to the real choices faced by policymakers and out of sync with public attitudes and politics.

Many valuation methods also fail to take account of what matters to beneficiaries. This is particularly the case for children's services and care. Too often children are powerless and voiceless, and their concerns are excluded from the clever metrics designed to capture impact.

As a result, the key for any measure is whether it can help to enhance rather than replace conversations about priorities and decisions.

Services, outcomes and justice

Many things that governments and civil society do are valuable to the public, but they roughly fall into three main categories:

1. The first is the value provided by services – like well-maintained roads, or hospitals. Services can be relatively easy to analyse in terms of value. In some cases there are private sector benchmarks and in others people can be asked how much they would be willing to pay for different levels of service – for example, the evening opening of a library, or retaining a very local Post Office.
2. The second category of value is outcomes. – like lower crime, or security from invasion. Again, through democratic argument, or through surveys, people can place a rough value on these. One of the outcomes they say they often value highly is equity. Whether people use privately funded alternatives to public provision has surprisingly little impact on their propensity to support higher state spending. Often outcomes and services align; but in many cases they point in different directions (e.g. more police on the street experienced as an improvement in services even when it doesn't result in lower crime).
3. A third category of value is trust in its widest sense, which includes whether the work of government is seen as just and fair, or more broadly whether institutions are trustworthy (a topic with obvious relevance to charities). This is influenced partly by people's satisfaction with public services, alongside perceived integrity, reliability and responsiveness of the government. Its flow-on effects matter too, with 'perceived quality of institutions' in turn influencing people's trust in others. Our approach has been to encourage an active search for value - using the best available tools for mapping and measurement - while remaining sensitive to the tensions and dilemmas which mean that there will very rarely be a single number that can capture value in the round. In what follows we set out a series of case studies that show how different fields are grappling with these issues, primarily focusing on work we are directly involved in.

Value in the real world: Case studies from the frontlines of public value

In this section we set out a variety of different ways of thinking about public value which show how these complexities can be grappled with in ways that help decision-makers.

1. The What Works movement: Bringing more rigorous views of impact and value into public services.

By Jonathan Breckon and Mariola Tarrega

The UK's What Works Centres aim to translate value in a way that is useful for frontline professionals and policymakers. Launched in 2013 by ministers from the Treasury and Cabinet Office at Nesta, they seek to marshall the best available evidence to make smarter policy, practice and spending decisions.

The current crop of What Works Centres range from big organisations like the £125 million Education Endowment Foundation, to diverse consortiums like the What Works Centre for Children's Social Care, hosted by Nesta with partners from academia, business, and the third sector. Although most Centres receive public funding, all are independent of government: either established as charities, or based in universities, or arm's-length bodies.

By using easy-to-understand quantifiable values, they aid decision-making by pointing out what has worked before - according to international evaluations and research. For instance, the Education Endowment Foundation compares the months saved in the classroom by using particular teaching methods (see Box 4 below). Other centres compare measures in areas like wellbeing, local economic growth, quality of life, reduced crime, and children's social care. The hope is that by bundling up measures of value – particularly on effectiveness - these centres will save government money. According to the UK Government's 2012 Civil Service Reform Plan, the What Works Centres:

What Works Centres timeline

...test and trial approaches and assess what works in major social policy areas, so that commissioners in central or local government do not waste time and money on programmes that are unlikely to offer value for money.

HM Government 2012, p17

What Works Centres point to how to save money – by cutting things that don't work.⁷ But they also show how you can save money in other ways: spending taxpayers' money on successful things that are also cheaper than other options. For instance, reading comprehension strategies are both cheap and effective in schools, according to extensive evidence. Other successful approaches are just too expensive: one-to-one mentoring, for instance, is reasonably effective, but uses up too much time and money. School budgets just can't stretch that far.

More controversially, organisations like the National Institute for Health and Care Excellence (NICE) may recommend that the NHS avoids funding life-extending drugs and treatments, because their value fails to justify their costs. NICE - created in 1999 under the Blair government to end the 'postcode lottery' of availability and quality of NHS treatments - decides whether a clinical intervention or technology is merited. Their evidence-based recommendations can, however, be hard for the public to stomach. For instance, guidance issued by NICE in 2005 caused a 'firestorm of controversy' by banning drugs for Alzheimer's, known as cholinesterase inhibitors.⁸ While NICE acknowledged that there were some promising clinical trials showing cognitive improvements for Alzheimer's patients, it doubted whether the size of the benefit was worth the hefty cost of the drugs. However, there were no other drugs available to patients at the time and NICE received over 8,000 complaints.⁹ Such controversies do not go unnoticed in other countries. In the US, Obama's Affordable Care Act (ACA) explicitly banned the use of quality adjusted life years (QALYs) in health care.¹⁰

No common currency of value

The NICE model is by far the most sophisticated model of value amongst the What Works Centres. For the others, there is a more mixed record of how they assess public value. Only half of them have any clear way of comparing values – such as through cost-benefit analysis, cost-effectiveness, or cost-utility ratios (basically a medical variation of cost-effectiveness). Some, like the What Works Centre for Crime Reduction just recognise costs, but don't compare it to value. Yet others, like the Centre for Ageing Better, have no approach to value at all, although they do have plans to use more economic measures in the future.

Generally, the Centres have not picked one common currency of cost and value. Instead, they have grown their measurements organically, some would say haphazardly, with no rigid fixed model (see Annex comparing the ten What Works Centres). This is understandable, as the sectors they cover are so diverse, covering clinics, foster homes, classrooms and more. Shoehorning everything into one uber-measure would not win friends for the What Works Centres. What matters to a nurse may be very different from a teacher. But, as will be argued below, it can be confusing having ten different What Works Centres with ten different ways of thinking about value.

Valuing children in care

In some sectors, there is almost no consensus on how to measure what counts. How, for instance, can you understand whether the care system is helping children in foster care or children's homes? It's not enough to look at these children's outcomes in isolation – such as through their GCSE results, or whether they go to university. That approach doesn't take into account the legacy of trauma, abuse or neglect they may have suffered long before they came into care. According to Louise Bazalgette, Evidence Lead for the What Works Centre for Children's Social Care, incubated at Nesta:

When traumatised children get to a place of safety you might start seeing some very challenging behaviour because they're working through their trauma as well as separation from their families (e.g. going missing, getting angry with carers or damaging property). That could look alarming and lead to a judgement that a care home is performing badly on some simplistic snapshots of value and performance, making you want to move that child somewhere else. But what if this is what that child always does when they arrive in a new placement, to test the commitment of the carers and see if they can really trust them? That's how some children end up having 25 placements in seven years. In some children's minds if they break down another placement, maybe they'll be allowed to go home. We must avoid these sorts of reductive judgements when thinking of children in care.

If we want to know whether foster care or children's homes are helping, then a good start is to ask the children themselves, as Coram Voice's Bright Spots project is doing – still a rare approach to measuring the quality of children's care. Where children's feedback is sought in most cases it still isn't being gathered and analysed systematically.

We also need to listen to the children's carers and social workers and provide them with the support they need to do what is sometimes a very demanding job. The What Works Centre for Children's Social Care has to find meaningful ways of understanding what makes for great care for children. That means measuring the things that really make a difference - the quality and stability of children's relationships, their resilience and wellbeing, getting a job so that we can see which interventions support these kinds of outcomes – rather than just measuring the things that are easy to count.

How to collect trustworthy data?

The example above on Children's Social Care alludes to the difficulty of getting good data from people when subjective views are important. Sometimes it's easier to collect independent figures, such as exam results in schools, or numbers of medical diagnoses. But when quality of life comes into the equation, then people need to be asked how they feel.

For NICE, their preferred tool is the EQ-5D questionnaire. It asks patients about pain, discomfort, self-care, anxiety, and other factors that created 243 unique combinations of health.¹² For the What Works Centre for Wellbeing, subjective views of life quality is their core business. They have opted for questions on Life Satisfaction, such as answering: 'overall, how satisfied are you with your life nowadays?' Life Satisfaction is a 'common currency' for subjective wellbeing.¹³

But can you trust what people tell you? Perceptions may not match reality. For instance, results from the British Crime Survey consistently show that people's perceived likelihood of being a victim of crime is considerably higher than their risk of actually being a victim. This is important when thinking about subjective wellbeing. What is more important: the fear of crime or the actual act of crime? It would be misleading to wholly dismiss people's fears: perceptions matter.

One approach used by social scientists is to use 'mixed methods', merging qualitative and quantitative evidence. In other words, splice the subjective views from, say, interviews, with insights grounded in hard numbers. It's no methodological panacea. If qualitative evidence directly contradicts the quantitative - as in the crime example above - then there is not much you can do. But using mixed methods is a decent compromise. And there are worthy examples of attempts to bring the two together. Nesta's pioneering Manchester Creative Credits experiment used a Randomised Control Trial to evaluate a business vouchers scheme for small and medium businesses. To probe the numbers, they also interviewed people to check 'inside the bonnet' of what was going on. The researchers called this an 'RCT+' approach, mixing quantitative causal evaluation, with qualitative, and longitudinal evidence (the researchers checked if the positive results of the voucher scheme lasted over 12 months. They did not).

Nesta's standards of evidence¹

Nesta provides a framework for thinking about how much you can trust evidence. It lays out five levels of evidence of impact, ranging from early-stage Theory of Change (Level 1) to robust, replicable evidence (Level 5). The framework supports cost-benefit analysis (at Level 4). As well as being used to assess Nesta's social investment funds since then, the Standards of Evidence have been used in other parts of the organisation, such as Centre for Social Action Innovation Fund, a partnership between Nesta and the Cabinet Office. Recipients of Nesta funding such as the Action Tutoring charity have interpreted the standards according to their needs. Action Tutoring used the standards (see below) to demonstrate the quality of their impact evaluation and according to their report their evidence went up from Level 1 to 3. The comparison group approach used did not indicate direct correlation between their intervention and their initial aims. However, there was enough evidence to demonstrate the long-term potential of continuing their programme.

Citizens measuring, creating and evaluating evidence

When measuring value, it's a wise move to use credible evidence, tried-and-tested methods, mix quantitative and qualitative, and use frameworks, such as the Nesta standards of evidence. But the views of the public at large – the voters and taxpayers – may differ from those close to the project. There are different methods and strategies to incorporate a wider range of voices in evidence-making and evidence evaluation processes. To help get round this, we mentioned before that NICE runs a survey of 3,000 members of the public to check their views on Quality of Life. Essentially it's a survey of the gut instincts of people, although made in the artificial environment of filling in a questionnaire. A more sophisticated technique is to use deliberative 'mini-publics' such as Citizen Juries or Consensus Conferences.² For example, the NICE Citizen Council invites a representative group of people to get to grips with the ethical and scientific complexities of health (see Box 3). Like a legal jury, they need to weigh up the evidence, spend time really understanding the issue, discuss trade-offs, and come to some final conclusions. The Citizen Council's recommendations are incorporated into 'Social Value Judgements' and, where appropriate, into NICE's methodology on guidance for healthcare. Citizen or user participation is a core part of the Scottish public sector ethos.³ For instance, for What Works Scotland, involving the public in its work is essential. Co-production is the guiding principle for much of its work, such as the Scottish Participatory Budgetary Evaluation Toolkit. The views and contributions of citizens and others are just as important as research evidence. These initiatives require organisations to transfer part of their decision-making power to users and/or citizens. They also need sustained investment in capacity building to ensure citizens have the skills and motivation to participate in the process.⁴ Finally, organisations need to be open to enact changes promoted by these initiatives i.e. adopting new evaluation measures.

Differing time horizons

Another important way to think about value is over different time scales. The What Works Centres all handle timescales differently.

The Education Endowment Foundation (EEF) toolkit shows the amount of months gained or lost in the classroom from teaching techniques. The EEF values do not reflect benefits in the future, such as employment and welfare - they only focus on educational attainment now, in the school. The danger with this approach is that it ignores a whole raft of other longer-term social, emotional and societal benefits. For instance, pre-school support for kids such as Sure Start programmes can benefit children and families for years – and even decades later. But for the EEF, the immediate cost is prohibitively high – particularly when the evidence shows that the educational benefits are marginal.⁶ However, these costs may be worth it: if it means a disadvantaged pupil goes on to have a better life – and avoid poverty, joblessness, crime, misery and dysfunctional family life, all of which are costly to the state, let alone the individual. With these potential longer-term benefits, it is unwise to only look at short-term costs (even if that is the reality for the holders of school budgets: they have to worry about where to find the cash for next term, not worry about future generations).

Other What Works Centres do take into account the longer-term costs and benefits. The Early Intervention Foundation puts lots of attention on future value, not just immediate benefits. Other What Works Centres such as NICE and What Works Wellbeing recognise the bias towards valuing present benefits today and putting less value on benefits accrued in the future - and quantify this via the use of discount rates. For wellbeing, the proposed discount rate is 1.5 per cent per annum, in line with Department of Health and Treasury Green Book guidelines.¹⁴

Conclusion

Nesta's Alliance for Useful Evidence was set up in 2012 to champion the smarter supply and use of evidence – in whatever form evidence may take as long as it is used by the people that need it. Evidence of value is necessary and needed. But the whole point is to make measures of value actionable. Otherwise we just create new gleaming ivory towers of value metrics that are ignored - except by the back-office bean counters.

The great advantage of the What Works Centres is how they foreground the user. Any teacher, governor or parent can, for instance, grasp the Education Endowment Foundation toolkit. If you want more of the research nuance, then you can dig down a few clicks for detail. It also uses a measure of value that is plausible in the eyes of the public: the months gained or lost in the classroom by using certain teaching techniques. This is not an abstract monetisation of value, but something that matters to schools, budget holders and parents.

But there is no consistency between the different types of value they use. When the Centres were first launched in 2013 there were only six of them. Now there are ten (although the one in Scotland is closing). And there are at least four other prospective 'what works wannabees' who came together at an event in Nesta in 2017. None of these organisations is working towards using a common currency of costs, benefits and public value.

We need to address this. It's confusing if one social programme is given one measure of value by one organisation, and another measure by another organisation. The point is to make decisions easier, not harder, and to avoid cherry-picking value estimates to justify decisions about the utility of certain policies and programmes. We also need to learn from behavioural research about the best way to communicate numbers and value – in a way that lands with stakeholders. Using good user-focused design, tailored-and-targeted messaging, systems that hard-wire evidence and value into daily decision-making, will all help. Various 'nudges' may help citizens and professionals act on value too. The Behavioural Insights Team, a partnership with Nesta and the Cabinet Office, has created nudges to help citizens do things they value but that they have not got around to doing, such as enrolling in workplace pensions, or signing up for organ donation.

Other pointers to help act on value metrics can be found in a systematic review of all the evidence on the best ways to communicate research to professionals, a project Nesta ran with What Works Wellbeing, UCL and the Wellcome Trust. Anybody presenting data and research can learn from this research.

We should continue to experiment and try out different ways of communicating value, evaluate our mistakes, and celebrate success like the brave, if flawed, NICE QALYS. There will always be plenty of critics of any attempt to count value, but we must go on trying out new approaches and new ways of communicating value.

2. Measuring value in culture and heritage

By Hasan Bakhshi and John Davies

Cultural activities and cultural objects are the subject of several policy interventions which raise questions about their public value. Most obviously, many cultural institutions in the UK, such as museums and performing arts organisations, at national and local levels, receive funding – directly or indirectly from the state. Cultural objects can also be subject to specific legal protections such as licensing rules on their export or, in the case of physical buildings and structures, listing and conservation status in the planning system.

At the same time, there are a number of challenges in measuring the value of culture:

• The frequent absence of a price as a measure of the value to users. A lot of cultural activity does not have a clear market price that can be used as a proxy measure for value to the public, for example when museums have free entry. And even if there is a price, as this is often not set on a commercial basis it may not necessarily reflect the cost of the service or people's valuation of it. In the case of cultural objects many may have been acquired without involving financial transaction, and if they were sold it may have been a long time ago.
• The value extending beyond direct users. The value of culture can extend beyond people who make direct use of it (so called non-use values). For example, we may enjoy the view of a historic cathedral even if we do not visit it. We may value that a museum exists to protect cultural assets for future generations even if we have never been. There is often not a ready measure of these non-use values, and of how they vary over the population.
• Uniqueness. Some cultural objects are unique, so it is not clear how they can be valued using economic techniques based on the principle that they can be traded off with other valued objects. For example, there is only one Westminster Abbey and one Rosetta Stone.
• The value requiring expertise/experience to appreciate. Some cultural activities arguably require high levels of knowledge to fully understand and appreciate their quality, for example the technical excellence of an orchestra or the extent to which a building should be valued for its architectural importance.
• The value arising from activities which are hard to clearly define. There are also certain cultural activities, such as historic/social traditions, which do not always correspond to a tangible object or institution, so it may not be clear to the public what the thing being valued actually is.
• Social creation of value. Some forms of value in culture arise from group participation e.g. people attending a music concert get a personal value from the experience, but this value is affected by the value other attendees obtain, making the two hard to disentangle and/or aggregate. Arguably, such forms of social value cannot be assessed using traditional economic approaches based on individual's utilities.¹⁵

More fundamentally, there is a widely held view in the cultural sector, which is sometimes but not always linked to the issues just described, namely that the sector transcends valuation and that there is a danger in reducing its complexity to a single economic measure. This is despite the fact that implicit financial measures of the sector's value are made regularly, both in terms of the general allocation of public funding and in individual business cases made to funders for specific projects. There has also historically been a lack of rigorous evidence on people's valuation of culture.

To explore these issues, Nesta has undertaken three studies with research consultants Simetrica to assess the value of culture using economic techniques. These studies have focused on cultural institutions and the built environment. The main valuation challenges that they have addressed being the absence of a use value (due to free entry), the existence of non-use values and uniqueness. The studies attempt to establish a rigorous evidence base on the applicability of economic valuation techniques to heritage and culture, while also generating values that can, in principle, be applied in similar contexts.

This research has looked at the value that users and non-users obtain from the following institutions:

• Tate Liverpool and the National History Museum in London.⁷
• Four regional museums: the Great North Museum, Newcastle; the National Railway Museum, York; the Ashmolean Museum, Oxford and the World Museum, Liverpool.⁸ The findings from this are summarised in Box 5 below.
• Four historic cities (York, Lincoln, Canterbury and Winchester) and their associated cathedrals.⁹

These studies estimate both use and non-use value for these sites (although it can be difficult to separate the two, as non-users' valuation may potentially include their valuation of a future visit). The regional museums and historic cities/cathedrals studies also investigate the validity of transferring values estimated for one group of sites to others, a process known as 'benefit transfer'.

Consistency with best practice for valuation in other sectors

The valuation studies produce estimates of value using economic techniques - hence measure only those benefits that can be captured by traditional economic theory. These estimates are obtained through survey questions using scenarios designed to elicit truthful valuations of the cultural activity from survey participants. All of the studies use a technique known as contingent valuation where, to assess how much they value the sites, survey respondents are asked a question about their willingness to pay (WTP) to protect these sites in a hypothetical but realistic sounding scenario, such as damage from climate change (and in the case of the National History Museum and Tate Liverpool study separately a willingness to accept compensation for the damage). The studies use a number of procedures to maximise the robustness of the findings. For example, the survey design includes information on the cultural asset being valued so that it is, in so far as possible, an informed decision. There are also checks to avoid people giving unrealistic answers, for example by avoiding people being able to supply open-ended valuations and survey respondents being asked to attest to the accuracy of their valuations. Data on the demographics of survey participants is also collected to check the representativeness of the sample and, where possible, the data weighted to ensure the estimates reflect the appropriate target population.

The approach followed is consistent with the best practice set out in the UK Treasury's Green Book, the manual which provides guidance in UK central government on how appraisal and evaluation should be undertaken. Complying with the Treasury's guidelines allows the results to be used in business cases in the public sector.¹⁰

Using wellbeing analysis

Another approach to assess valuation is to use measures derived from estimates of the subjective wellbeing of the public, which are based on individuals' self-assessment of their wellness in the moment or in terms of life satisfaction. The approach looks at how experiencing certain activities, such as visiting a cultural institution affects wellbeing, by comparing the stated wellbeing of users and non-users of the institution. In principle, the impact of subjective wellbeing of culture on non-users can also be assessed using vignettes (whereby survey respondents are presented with a hypothetical scenario, in which they are asked to report their levels of subjective wellbeing if they were able to visit the institution more often).

The wellbeing impacts can then be converted to economic valuation estimates by assessing the change in income that is equivalent to the change in wellbeing impact using econometric models relating wellbeing impact to monetary income.

The Tate Liverpool and Natural History Museum study proposes and tests a new hybrid approach combining willingness to pay with contingent valuation and wellbeing valuation by asking people who had said that their life satisfaction would decline if they were not able to visit the museum how much money they would require for their willingness to pay to be unaffected.

Policy recommendations: Using existing valuation frameworks more effectively and involving the public in valuation in new ways

Public policy affects the cultural sector in many different ways. There are overall decisions on how much is allocated to different forms of cultural activity and decisions that relate to individual institutions, for example an investment in a cultural facility which expands its capacity, such as a museum extension. There are also legislative decisions which affect the sector.

In principle, all such decisions should be informed by economic evidence on the value of these activities/projects for society, but in practice the extent to which this happens is limited. This is partly because of a widespread belief that economic valuation estimates which are, at least in principle, straightforward to implement necessarily omit essential dimensions of value and for key stakeholders. While we share this belief, we also believe there is under-recognition of the wide scope of benefits that economic valuation techniques can in principle capture. We have long called for the research councils to lead interdisciplinary research in this area, to produce more rounded accounts of the value of culture, and sharpening our understanding of when economic approaches and other approaches align and collide,¹¹ and this gives rise to our first recommendation.

1. Improved use of evidence within the existing Government valuation framework

There should be a commitment to improving an interdisciplinary evidence base on valuing culture and heritage and to integrate it within public decision-making. In the UK, the forthcoming Centre for Cultural Value from the AHRC, Arts Council England and Paul Hamlyn Foundation, obviously has an important role to play here, although the extent to which it will be resourced to fund the new methodological research needed is unclear. Rigorously assessing the value of cultural activity can be costly. Given the many parties involved in the cultural sphere it can also lead to inefficient duplication of effort and inconsistent approaches to valuation. Even if good evidence is generated it may not be used consistently in policymaking.

There should therefore be a commitment from Government to make a greater investment in establishing the value related to cultural and heritage activity. This will involve estimating values on a consistent basis for similar cultural and heritage assets/activities, as done in our regional museums and historic cathedrals/cities studies, to provide benchmark values that can be applied in business cases in the public sector. Government should also play a greater coordinating role to ensure that the Green Book approach is more widely adopted in the evaluation and policy making framework.

There should be a requirement that all large cultural and heritage projects have an assessment of non-market valuation impact. Related to this, there should be a requirement in the business cases for all large-scale cultural projects that there is an economic assessment of the value that they generate which may not be captured in market transactions. For example, as many cultural amenities, in addition to having value for visitors are widely considered to contribute to local wellbeing, it is reasonable that consideration should be given to this in the project business cases for all large-scale capital investments.¹²

2. Greater consideration given to new approaches to involving the public in assessing cultural value using digital technology

There has never been greater potential to understand people's valuations of culture due to the advances in technology which have increased our capability for measurement and to involve the public in decision-making processes. Policy initiatives to improve understanding of the public's values about cultural activity using digital tools represent an exciting, new approach to valuation in the sector which should be explored. In other areas, for example, work has already been done in crowdsourcing information to understand perceptions of an area's safety, the extent to which people's surroundings make them happy and even the public benefits of digital online services based on much larger sample sizes than usually feasible with valuation surveys.¹³ Computer game environments have also been used to engage people in discussions about local planning decisions.¹⁶

3. Greater use of crowdsourcing and crowdfunding to involve the public in valuation

• We can allow the public a greater say in cultural policy through more sophisticated uses of crowdsourcing to ensure that their valuations are incorporated into decision- making. Initiatives could range from:
  • Using information from the public to decide programming at a cultural venue.
  • Making decisions between projects that have already gone through an expert selection process.
• It is, however, important to acknowledge the challenges involved in ensuring such activity is inclusive, as the ability to engage in crowdfunding will not be consistent across the population.

A concern relating to the use of valuation techniques in the cultural sector that, in making assessments of value, expertise is often needed to make judgements about a cultural activity or object's technical excellence and originality/importance. The concern being that the general population is not sufficiently knowledgeable to make these decisions and the result of a valuation determined by such choices would be less innovative and of lower technical quality.¹⁷ An assessment of how much this is indeed the case could be built into the trialing of such initiatives.

4. Using augmented and virtual reality to provide the public with better information on cultural and heritage facilities/objects as a way to improve accuracy of valuation assessments

Improvements in virtual and augmented reality, in principle, provides the public with much richer information on cultural and heritage facilities and objects than the more traditional text descriptions and pictures used in contingent valuation surveys. There is also a wider set of issues in the built environment as to how to involve the public in the planning process, of which that relating to new cultural and heritage facilities is just one part. There should therefore be a review of how tools such as augmented and virtual reality allow the public to be involved in valuation and the decision-making processes in government. This is consistent with the current desire by government to promote these technologies more widely and has applications in urban policy and planning too.¹⁸

3. Realising the value in health

By Halima Khan and Annie Finnis

The Realising the Value programme

In 2015-2016 Nesta Health Lab worked with a consortium of partners, including NHS England and The Health Foundation, to look at how and why health and care services can work alongside people and communities to create value. This section brings together some of what we found.

www.realisingthevalue.org.uk

When it comes to health, people value more than just clinical outcomes

In health, perceptions of value have been dominated by a mix of clinical outcomes, system targets, competition mechanisms and encouragement for single organisational units to act autonomously and be judged as single services. Despite being critiqued, these approaches continue to have a significant impact on the behaviours of individuals and organisations. The current NHS payment system is a highly complex mix of methods, prices, incentives and penalties.

QALYs (quality-adjusted life years) have been viewed with interest by other sectors, and for many sectors getting to an equivalent generic measure would be seen as a step forward. There are clear strengths to QALYs but they have also faced criticism, not least for insufficiently considering the perspective of citizens. Alternative ways of understanding value in health, such as Patient Recorded Outcome Measures and International Consortium for Health Outcomes Measurement are being explored by health systems across the UK and internationally. The Realising the Value programme set out to explore what 'value' means for people and communities, and how this can be understood most appropriately by the NHS. By doing so, the programme sought to orientate the health debate to more actively respond to what people using health services value. Our work aligns to the wider movement in health to 'measure what matters'.¹⁹

People make decisions every day that affect their health and wellbeing

It is people, not health and care services, that primarily drive the creation of value in health. Although medicine and hospitals make a vital contribution, the roots of health and wellbeing actually lie in being connected into a thriving community. For example, in relation to exercise, managing stress, taking part in social activities or developing skills to successfully look after a health condition. And people have family, friends, communities and peer networks that can work alongside healthcare professionals to support them to live well. For example, the value of voluntary activity in the UK has been estimated to be £23 billion.²⁰ The economic value of the contribution made by carers in the UK has been estimated at £132 billion a year.²¹

There is, therefore, increasing interest in how measures of value in health and care can focus on what is important to people, what skills and attributes they have, the role of their family, friends and communities and, given all this, what they need to enable them to live as well as possible. In Realising the Value we called this taking a 'person- and community-centred' approach.

A great deal of work is already being done at both national and local level to embed person- and community-centred approaches in national programmes and local services. Adult social care has been a long journey towards personalisation and a values-based approach which prioritises improved wellbeing, independence, social connectedness, choice and control.²² The NHS across the UK has followed. For example, NHS England's Five Year Forward View included a vision to develop new relationships with people and communities in which patients' own life goals are what count. Realising the Value was commissioned by NHS England in the context of the Five Year Forward View to look at a range of ways that people and communities can be involved in health (see Box 7). It deliberately included both approaches that take place in formal health and care settings between individuals and health professionals, as well as those that happen in people's own homes and in the wider community.

Through Realising the Value we aimed to make more visible the difference that such approaches can make to people's own health and wellbeing and to the wider system. We found that there is growing – and increasingly convincing – evidence that when people are actively involved in their own health and wellbeing, or support others to stay well, it can create value for them and the health and care system across a range of dimensions, outlined below.

Value for people

Realising the Value demonstrated that when people are more actively involved in health, their own mental and physical health and wellbeing can be improved, including improved clinical outcomes and improvements in people's confidence to self-manage as well as their wider wellbeing and quality of life. We think this value for people strengthens the investment case for person- and community-centred approaches.

For example, person- and community-centred approaches have been shown to increase people's self-efficacy and confidence to manage their health and care, improve health outcomes and experience, and reduce social isolation and loneliness. UK-based studies on the benefit of group activities among those with dementia and those with mental health issues have reported impressive outcomes in areas such as quality of life, anxiety, depression, communication, and feeling in control of life and able to cope with its challenges.²³

The case study below demonstrates the value that can be added through people being actively involved in health.

Footnotes:

Value for the system

In addition to being valuable for people, person- and community-centred approaches have the potential to impact how people use health and care services, bringing value for the system. Realising the Value demonstrated how involving people and communities in health can support the financial sustainability of the health and care system, including reducing demand on formal, particularly acute, health services, as well as achieving savings for local authority care budgets.

The cost-effectiveness of interventions is less reported in the formal research evidence than outcomes relating to improvements in physical and mental wellbeing, and there are merits (and limits) to this type of analysis. A number of organisations have previously sought to quantify the potential global impact. For example, Reform estimates stronger patient engagement, including more self-care, improved public health and greater patient contribution to their care, could save £2 billion.²⁴ Monitor has estimated that teaching people with long-term conditions to manage their own care could save between £0.2 billion and £0.4 billion.²⁵ And through previous programmes of work, Nesta has estimated over £4.4 billion a year could be saved if the NHS adopted innovations that involve people, their families and communities more directly in the management of long-term health conditions.²⁶

The Realising the Value programme undertook economic modelling, led by PPL Consulting, to see if it were possible to show how implementing person- and community-centred approaches at scale can contribute to efforts to slow the demand pressures on the system and yield efficiency savings – at the same time as bringing value to people and communities. We saw this modelling as a contribution to the debate, recognising that the evidence base is not yet developed enough to enable us to calculate how the potential long-term benefits of a shift towards person- and community-centred approaches will play out.

Based on a scoping review undertaken by the Institute of Health and Society at Newcastle University, which collated and analysed systematic reviews of person- and community-centred approaches, we found the evidence is relatively stronger for peer support and self-management education approaches. In terms of financial benefits, our modelling suggests that for people with a subset of particular long-term conditions, we could anticipate net savings of around £2,000 per person reached per year. There is considerable uncertainty about how these potential savings might scale up at a population level but scaling these approaches up to the whole target population in an average-sized Clinical Commissioning Group (CCG) area could lead to savings of over £5 million per year. Extrapolating the model further suggests that if implemented well and at scale across England, there may be potential for savings of up to £950 million per year from targeted peer support and self-management education with people with these particular conditions, who are expected to see the most benefit.

The full impact of investing in person- and community-centred approaches could be significantly higher than this, when wider social impacts, such as improved employment outcomes and reduced social isolation, are taken into account.

Estimated annual net savings from implementing targeted peer support and self-management education

Wider social savings are based on offering: Peer support to individuals with HIV, and self-management education interventions to people with cancer. Savings to the health system are based on providing: Peer support to people with mental health issues and coronary heart disease; and self management education to people with cardiovascular disease and asthma.

Source: Realising the value (2016)²⁷

The estimates focus on specific health conditions and interventions because they have the most robust financial evidence to date (e.g. evidence that uses control group comparison). The model aims to produce estimates within a clearly defined scope. These estimates do not, for example, extrapolate evidence from condition-specific research on other health conditions, or put financial values on outcomes where there is not sufficient evidence to do so. While the model uses some of the most robust evidence available, the data still has limitations, for example studies conducted in one part of the UK may not produce the same results in another part with different demographics, or with different levels of existing provision.

Box 9: Assumptions in the economic model

In today's financial environment there is significant pressure for approaches to demonstrate return on investment. We see this modelling as the beginning of a process of understanding how we can measure what people value in a way that is useful for the health and care system. And it could potentially be seen as representing the 'tip of the iceberg'. It is plausible to suggest that there may be unaccounted-for benefits, because we currently have quantitative, financial evidence for a very small number of conditions and approaches. If person- and community-centred approaches were offered for more conditions, to a wider population, and in different forms, the savings could be greater. Although more longitudinal evidence is needed to understand when and where the financial benefit will be felt.

Realising the Value called for further development of the evidence base for person- and community-centred approaches, across all three dimensions of value, to establish not just 'what works', but 'what works for who, when, where and with whom' using rapid experimental methods combined with long-term research.

Wider social value

Wider social value can include more resilient communities and greater social connections as well as wider societal benefits from supporting people to return to work and reducing demand on other public services. Person- and community-centred approaches can lead to a wide range of social outcomes, from improving employment prospects and school attendance to increasing volunteering. They may also contribute to reducing health inequalities for individuals and communities.

Although it is plausible to assume that person- and community-centred approaches bring wider social value, we found little authoritative work to evidence this impact. We know that poor mental and physical health can be a significant barrier to all kinds of participation in public life that creates value for society, but our ability to link investments in health and care to these outcomes is limited. We feel this is a missing part of the 'business case' for person- and community-centred approaches, and that more research and investment in this area would be useful.

A broader view of value

Value frameworks - and techniques for evaluation and measurement - that are capable of capturing all of the valued outcomes and impacts are required. These are likely to consist of financial information such as cost-benefit analyses, other forms of quantifiable data, stakeholder experience and person-reported outcomes, and also qualitative information, bound together by narratives that make sense of all the various outcomes as a whole.

Realising the Value called for a future articulation of value aligned across health, social care and community organisations, that takes a broad view in the following ways:

Source: Realising the value (2016)²⁹

Many organisations have already called for a new, simplified outcomes framework. These include the NHS Confederation, the Local Government Association, the association of directors of adult social services in England (ADASS), the Academy of Medical Royal Colleges and patient organisations. NHS England and others should respond to this cross-sector, multi-organisational demand for a change and take action. This work will need to engage with the tension between centralisation and standardisation, and the diverse perspectives of people and communities. This can be addressed by focusing on a small set of outcomes, such as: independence; empowerment; social connection; the ability to have a family and community life; health-related quality of life; feeling supported.

How people and communities achieve core outcomes like these will differ, and some people will want to achieve outcomes beyond those outlined. A clear and consistent set of core outcomes will help develop both practice and evidence by creating tangible ways for the system to measure and understand its performance, and by building commitment, focus and skills for improving it.

For areas looking to begin this process, we proposed a set of value statements and accompanying pledges, that could be adopted immediately by local areas seeking to add value to people's lives and mobilise the value that people and communities themselves can create for health and wellbeing:

• We value the creation of health and wellbeing.
• We value people feeling supported, in control, socially connected and independent.
• We value people's contributions (their strengths, time, effort, and skills).
• We value sustainable outcomes over time, achieved through working together, as services and in partnership with people.
• We value equality, and the gains to be made by targeting and tailoring our approaches to people with greater need for our partnership.

Source: Realising the value (2016)³⁰

Adopting the value statements implies local areas working to understand the outcomes that matter most to people and their communities and develop systematic ways to work with them to help identify their goals for wellbeing, and the outcomes that are important to them.

There is an opportunity for this work to be developed further through NHS England's commitment to Personalised Care, which has been influenced by the Realising the Value programme. The recent NHS England Long Term Plan includes a Comprehensive Model of Personalised Care which includes social prescribing, community-based support including peer support, personal health budgets and, importantly, a commitment to a different relationship with people based on choice, sharing control and understanding what matters to them. As part of this, there is a commitment to develop metrics and frameworks to capture progress, including drawing on citizens' own experiences and perspectives.

Conclusion

Through the Realising the Value programme, we were able to make the case for how and why putting people at the heart of health is valuable for people, as well as valuable for the wider community and the system. The programme strengthened the case for change and identified evidence-based approaches. It also sought to develop tools to support implementation across the NHS and local communities.

Based on our learning and insights from the Realising the Value programme, we developed ten major recommendations to help make the vision a reality – with a focus on both what needs to happen and how the work should be implemented.

What needs to happen

1. Implement person- and community-centred ways of working across the system, using the best available tools and evidence.
2. Develop a simplified outcomes framework, focused on what matters to people.
3. Continue to learn by doing, alongside further research.
4. Make better use of existing levers such as legislation, regulation and accountability.
5. Trial new outcomes-based payment mechanisms and implement them as part of wider national payment reform.

How people need to work differently

1. Enable health and care professional and the wider workforce to understand and work in person- and community-centred ways.
2. Develop strong and sustained networks as an integral part of implementation.
3. Value the role of people and communities in their health and wellbeing, including through co-production, volunteering and social movements for health.
4. Make greater use of behavioural insights to increase effectiveness and uptake.
5. Support a thriving and sustainable voluntary, community and social enterprise sector working alongside people, families, communities and the health and care system.

The current momentum around Personalised Care in the NHS provides an important opportunity to build on Realising the Value and other relevant work to develop a simplified outcomes framework that reflects what matters to people. For more information on the Realising the Value programme and our recommendations please see: www.realisingthevalue.org.uk

Some general lessons

By Geoff Mulgan

Some explicit measurement or assessment of value is essential for public investment decisions. Otherwise the risk is that funders and governments rely either on hunches or prejudice, or crude monetary measures. That is why we favour more transparent and explicit explorations of value.

But methods of this kind are useful only to the extent that they help inform the negotiations between providers of services and their funders, or between public agencies and the citizens who benefit from them and pay for them. These conversations and negotiations are bound to involve qualities as well as quantities, values as well as value. As many commentators have pointed out, there is an analogy with the electromagnetic spectrum. Although radiation can take many forms, only a narrow range of frequencies is visible to the naked eye in the form of light. Similarly, money focuses attention on only some of the features of the world around us and obscures others. Since many of the features it obscures matter a great deal to the voting public, it's not surprising that the simpler attempts to monetise public value have failed.

More sophisticated thinking about value tries to mitigate this common optical distortion by analysing what really matters to the public.

Some useful principles were developed in one attempt to use public value in UK government which I was involved in.³¹ This work started with a few simple principles.

One was that something should only be considered valuable if citizens – either individually or collectively - are willing to give something up in return for it. Sacrifices can be monetary (e.g. paying taxes or charges); they can involve granting coercive powers to the state (e.g. in return for security), disclosing private information (e.g. in return for more personalised services), giving time (e.g. as a school governor) or other personal resources (e.g. donating blood).

Some idea of 'opportunity cost' is essential for public value: if it is claimed that citizens would like government or charities to produce something, but they are not willing to give anything up in return, then it is doubtful that the activity really is seen as valuable. For example, an opinion poll that suggests that citizens would like government to spend more money on services but fails to indicate public willingness to pay for this course of action, does not constitute evidence that higher spending will increase public value.

Another principle was that the different dimensions of value should not automatically be treated as commensurable. Economics has traditionally treated all values in this way: anything can be traded off against anything else and turned into a monetary value. But many of the currencies governments and societies deal with are not like this. Laws ensure that such things as votes, body parts and freedoms can't be sold, and the public turn out to have very clear views about which kinds of exchange or trade-off are legitimate and which ones are not. This is one reason why the many attempts to coerce different types of value into single numbers (as has happened with standard economic measures, as well as many environmental measures, CBA, and some methods used for estimating Social Returns on Investment) have often destroyed relevant information rather than helping decision-makers. Useful methods of valuation need to cope with varied types of value, with differing degrees of certainty.

A more challenging principle is that any measures of value should be comprehensible and plausible to the public. It's not enough for a measure to make sense to specialists. If it doesn't help to educate the public about choices, and to enrich the democratic process, then it's likely at some point to be rendered irrelevant by raw politics.

Social and public value: effective demand and effective supply

A critical conclusion is that value is never an objective fact. Believing that it is has been the most consistent mistake of many of the methods used to assess social or public value. Instead these types of value can only be understood as arising from the interplay of supply and demand, and through processes of negotiation and argument. In consumer markets, value is determined by the shifting decisions of individual consumers and the interaction of supply and demand. In the public sector it's refracted through political argument, and more specifically from the interplay of what I call 'effective demand' and 'effective supply' (these are essentially what economists call supply and demand – I add the adjective effective because in the social and public fields it's important to be clear on the difference between demand, and demand that is effective). Effective demand means that someone is willing to pay for a service, an outcome or a change in trust. That someone may be a public agency, a philanthropic foundation or it may be individual citizens. Effective supply means that there is a capacity to provide that service, outcome or trust: a public agency, NGO or business.

In some fields there are mature links between supply and demand: for example, public willingness to pay through taxes for policing, or primary schools, connects to governments' ability to supply in familiar ways. In other fields the links are missing. There may be available supply but insufficient demand - because the public or politicians don't see the need as sufficiently pressing (in some countries drug treatment or sex education would fall into this category). In other cases, there may be demand but inadequate supply at a reasonable cost (for example of methods for cities to cut carbon emissions).

Both sides of the equation may be complex or fragmented. In many areas of social policy, demand for the results that come from more holistic approaches is split across many different public agencies, from welfare to prisons. Equally the supply may be equally fragmented, depending on the contribution of many different agencies, for example providing therapy, alcohol treatment, skills, and housing.

In these cases, value has to be discovered, and a critical role will be played by a 'social market maker' who brings together supply and demand. This could be a local council or a national government and in rare cases a foundation or impact investor. To do this well they need to help both sides of the market clarify what they want and what they can provide. On the demand side public agencies need to work out what they are willing to pay for lower crime in five years' time or lower welfare payments. On the supply side, analysis needs to show what can plausibly be expected from different mixes of actions. For this, some of the existing Social Impact Assessment or Social Return on Investment methods can be used.

Then a series of negotiations can take place to commission activities and services, with the 'social market maker' working to ensure that no agencies try to free ride on others. These processes are likely to work best when they can be disaggregated as well as aggregated, e.g. when commissioning agencies can directly specify what they want from provider agencies.

The recent Social Impact Bond on loneliness – supported by Nesta and managed by Social Finance - is a good example of this. Ten years ago, there was no effective demand for solutions to loneliness, but growing awareness of the direct and indirect costs of isolation has changed this, to the extent that the UK now has a minister for loneliness.

This demand is made manifest through the willingness of local authorities and the NHS to pay for demonstrable reductions in loneliness, primarily because they believe that this will in turn lead to reductions in pressure on residential care services and the accident and emergency services of the NHS. Research work has developed a reasonable robust metric for loneliness which helps to connect supply and demand and support a process of experiment and discovery to find out what really works, and what is really cost effective. Bringing these together has made it possible to use the device of the 'Social Impact Bond' to make public value not only visible, but also measurable and monetisable.

Currencies

One of the interesting dimensions of this debate is when public value can be turned into something more like a currency or asset. Carbon credits are one version of this and have become a tradable asset. Some governments have experimented with currencies in the hands of individuals – such as credits for education or learning.

We at Nesta have funded various kinds of currency such as Spice, Local Exchange Trading Systems and local currencies. These aim to make explicit the hidden value in potential local activities and exchanges – for example for childcare, everyday jobs – which are not adequately valued by the monetary economy.

Complementary currencies have been with us for decades, even centuries. They are reminders that there is no God-given law which states that only governments or central banks can create monies (even though they do reserve themselves unique powers over what counts as a money). Throughout history different kinds of money have been created, sometimes by communities, sometimes by companies and sometimes by local governments. Their aim has been to connect different kinds of supply - of time, work or things - to otherwise unmet demands or needs.

There are now thousands of examples globally. Many have thrived when the mainstream economy contracted, as happened in Argentina in the early 2000s or Greece in this decade. But there are also many examples in countries like Germany where they have become an expression of community spirit.

Some of these currencies are, like Spice (now named Tempo), explicitly about time. They allow us to exchange hours of our time with others. At their best they work by formally valuing things that the mainstream economy finds hard to understand. That might be the time and skill to care for someone; to cook; or to fix things.

Models like Spice matter because they address head on what is perhaps the worst feature of the market economy: that it treats as valueless people who obviously have social contributions to make. At a macro scale that failure translates into millions left unemployed or underemployed. At a micro scale it means that many communities simultaneously have unmet needs and underused capacities. Many of our social ills have their roots in this kind of unnecessary uselessness, because people come to internalise the message the system's implicit message that they have nothing valuable to offer.

So, one route for further research public value is to think through how variants of public value can be turned into currencies or assets. The loneliness Social Impact Bond mentioned above is an attempt to turn public value into an asset. Models like Spice turn public value into something more like a currency. With a proliferation of new tools for designing and managing currencies – some using blockchain – this is likely to be a fertile avenue to pursue.

Structures of value

A key finding of the detailed analysis of public and social value in different fields is that it has a structure. In other words, value will mean different things to different groups some of which have direct power over its definition, purchasing or creation, and others of whom don't. Again, this highlights the need to move beyond single measures of value and to see the exploration of value more as a process of uncovering that can then support better conversations and negotiations.

This became apparent to me a few years ago working on a project to define public value in the built environment. The field was full of sophisticated tools. But we soon realised that there were multiple ways of thinking about value - economic, social, design - and multiple stakeholders. These diagrams attempted to capture the key points. This first diagram provides a basic overview of the kinds of value involved:

Next, we broke these down into more detail, both in terms of elements and stakeholders:

Then we suggested how these might be used to inform negotiations, for example over a new urban development:

The key point is that this structure will vary in different fields. What works for buildings won't work so well for schools, or health. Valuation methods are in this sense mapping tools – and the better they map the better they show the distinctive characteristics of different landscapes rather than trying to make them all one-dimensional.

Horizons in time

Another of the many challenges of public value is time: how to value something today relative to something in ten, or 50 years' time. This matters particularly for any kind of preventive action.

Governments use discount rates to judge when to make investments, since a sum of money in five years' time is less useful, and less certain, than an equivalent sum of money now. Typically, annual discount rates are around 5 per cent. These try to reflect both time preferences and also, in the more sophisticated versions, to take account of the fact that extra income in the future will be worth less than income today because future populations will be richer (the UK Treasury currently applies a 1.5 per cent rate to reflect time preferences, and 2 per cent to reflect these income effects).

A strict application of discount rates from the private sector radically reduces the attraction of investments in the future. A five per cent discount rate values $100 after 30 years at $35.85 today, and after 50 years at $7.69. Even the lower discount rates sometimes applied to health (like the three per cent usually applied to Disability Adjusted Life Years in the US) still render years saved in the distant future much less valuable than years saved now. But these discount rates are at odds with how people think and act.

Much of our behaviour reflects an implicit 'hyperbolic' discount rate that starts off very high, meaning that we value things in the present much more than the same thing in five, or ten years' time. But in our own lives we generally regret past decisions that applied a high discount rate to future gains, and there is some evidence that people apply different 'mental accounts' to their choices (for example with a different way of accounting for investment in their own education, housing, pensions or their own children). Most governments also apply quite different discount rates to different phenomena, which is why they are willing to invest in future defence, education or infrastructures. Their behaviour is closer to that of a guardian or steward who is charged with sustaining or growing capital, rather than the strictly rational consumers of economic theory who always value present consumption more than future consumption.

Environmental economics has been riven by arguments over the appropriate discount rate to apply to issues like climate change. Nicholas Stern, author of an influential UK government review, argued that the 'inherent discounting' of economists such as William Nordhaus (who advocated a 3 per cent discount rate as a measure of future uncertainty in the costs and benefits of action on climate change) was ethically questionable because it devalued the future. His analysis applied a zero pure time preference and compared benefits today and in the future by comparing percentages of income (rather than cash), weighting income for the poor more than the rich, and for today's citizens more than future ones, since whereas current average global income is around $7,000 his forecasts projected average world income in 2100 at around $100,000. But in all of these analyses economics has reached its limits. It struggles to explain some basic contradictions like the very wide gap between equity returns and returns for bonds (which calls into question the idea of a single market discount rate). And it cannot explain much of what's been observed in human behaviour in relation to the future.

Linking public value to public finance

These different approaches to time take us to a fundamental challenge for public value. In order to be impactful new ideas about public value have to intersect with new approaches to public finance. Here there is, to put it mildly, a problem. The dominant methods of public finance have scarcely advanced in recent decades. They often miss out key aspects of value; discourage investment or cross-cutting action; and remain opaque not just to the public but also often to the politicians responsible for making mistakes.

Here I summarise what that problem is and how it could be addressed. Some of this material reshares ideas floated by Nesta in 2015 when we tried to create a coalition interested in transforming public finance to be more long-term, preventive and evidence-based, but found few if any collaborators.

What happened to public finance?

In handling money, governments bear some similarities to other organisations, but there are also many differences. They should want efficiency, value for money and transparency. But they usually have many more goals; more accountability; and a different environment of risk.

The standard methods used by leading governments around the world were shaped by successive waves of reform. These have resulted in models that are essentially input-based, top down controls designed to ensure money goes to agreed and approved purposes. They are rightly - focused on the primary goals of preventing overspending and corruption. Against a backdrop of chronic public debt in many countries, these traditional aspects of public finance have not lost their relevance. It's just that they address only one part of what governments need to do.

A generation ago, a group of reformers became interested in how to use finance reform to drive efficiency and reduce waste. A wave of reforms associated with the ideas of 'New Public Management' had great influence from the 1970s-2010s. These were substantially influenced by business and encouraged new ways of managing public services and policies: privatisation, outsourcing, co-payments, fees and vouchers.

These made very little reference to public value. They fed into the already substantial body of theory and practice focused on issues such as optimal tax collection, or how to structure private finance and partnerships. These are still the topics dominating public finance courses, and still being promoted by consultants. What counts as the leading edge hasn't changed much in two decades.

Falling behind - what's missing

By contrast, much less progress has been made on many other topics that are now of great concern to many governments, in cities and nations alike. The first two issues follow on from the concerns of the last wave of reforms, and are particularly relevant to governments facing serious fiscal pressures:

1Understanding costs and assets: there has been surprisingly little progress in achieving a detailed understanding of the nature of costs in the public sector, including the relationship between average, marginal and variable costs, despite the pressures of austerity. It's a rare public service which can see in real time how its finances are being managed (which is one reason why so many lurch into occasional crisis); or how cuts to one service might raise costs for another; or what the true pattern of economies of scale or scope might be in different services. A related point is the very limited use of balance sheets, or analysis of rates of return on assets held by governments (which require national registers of assets, preferably as open as possible). Even less use is made of intangible measures in the public sector.

2Making budgets transparent. The move towards making public finance more transparent to the public has also made only faltering progress, despite some brave attempts (particularly in France). In principle it is becoming easier to tag many items of spending, by geography, by beneficiary group or topic area. It's not hard to imagine a much more standardised set of data headers for finance, making it easier to automate analyses of how funds are being used, and to connect management information systems that make it feasible to dig down through layers of accounting information and make it visible. This would allow a much more granular understanding of public value.

The next set of issues point in different directions, to how government can be more agile, entrepreneurial and effective (rather than just efficient). They include:

3Funding innovation. Very little progress has been made on the finance of innovation within the public sector. Much more has been done on financing innovation in science and business and, for example, the World Bank has sophisticated tools for assessing public spending on innovation in the economy. But as far as I am aware, no government can articulate a coherent approach to funding innovation in its own operations, and answering basic questions: what optimal levels of spending might be; how these might be divided between different activities and successive stages, such as incubation and scaling; how risk might be managed; what target rates of return should be etc. Nesta's recent survey on 'Finance for Innovation' provides a good starting point in describing the methods that are now available, all of which require some view of what is valuable.

4Human investment: as far as I'm aware (and again, I would like to be proven wrong), no governments use investment methods to assess spending on education, health and training. Highly sophisticated tools are used for assessing investment in physical things, roads, airports and rail. Yet spending on people is, in many cases, investment designed to achieve returns or results many years into the future - and social science has become steadily more competent at analysing the links between actions now and long-term effects. Instead, spending on fields like health is organised year to year, with no accounting distinction between spending on a vaccination programme versus emergency care.

5Finance tied to outcomes: despite several decades of experiment, there are still few established methods for linking budgetary allocations to outcomes (though some governments have made important progress in setting strategic goals, transparently assessing when these are met or not, and establishing budgetary implications). The spread of new tools for funding outcomes is providing a useful prompt for progress, including social impact and development impact bonds, payment for outcomes, and the growing field interested in analysing the long-term returns achieved by different interventions.

6Evidence: Although some progress has been made in recent years in orchestrating evidence in public services, and making more use of data, these methods have not permeated the day to day work of many officials responsible for public finance. It's interesting to observe that the Alliance for Useful Evidence and the various What Works Centres have struggled to engage public finance professionals. They're not particularly hostile, and some are enthusiastic. But they don't see this as core to their jobs. Yet in the future, finance ministries could be responsible for overseeing not only the inputs but also the impacts achieved, helped by the key budgetary and financial committees: in other words, acting as guardians of public value.

7Public engagement: We now have the benefit of several decades of experience of participatory budgeting (PB), allowing the public to directly assert what they believe to be valuable. Famous pioneers like Porto Alegre in Brazil showed that thousands of citizens could take part in quite complex discussions. At one point, the UK alone had well over a hundred PB pilots underway. In Paris, the Mayor opened up 100m Euro each year to PB and other big cities have experimented too. The challenge faced by all of these experiments is how to make budgets comprehensible: how to show the current balance of spending; how to help deliberation on the impact of shifts of budget from one category to another; and how to get the right mix of numbers, visualisations and prose. Exactly the same challenge, of course, applies to helping committees of politicians. The PB wave came slightly ahead of the wave of open data and rapid advances in visualisation. Yet together these offer the prospect of making budgets significantly more comprehensible.

8Funding as a platform: Governments have for many years discussed how to make funding flows to individuals more flexible and useable. Some like Denmark and Singapore created personal accounts, India has used its ID scheme to create what they call the India Stack and Bangladesh is now going further with its own social security system. The lack of progress in countries like the UK is another symptom of the relative lack of innovation in public finance.

The failure to tie finance into evolving ideas about public value, and the related debates on innovation and evidence, risks becoming a major barrier to progress. There is a need for better theory, and better tools that can be applied and shared by the leading practitioners, and put to use in governments at every level, whether national, regional or city and district. This is the necessary complement to further work on public value.

Sociology may offer additional insights. If we dig deeper we find that attitudes to time generally reflect the intensity of social bonds and commitments. Very strong commitments eliminate the difference between the present and the future, even though there is still the same uncertainty as in commercial markets. Parents may commit all to their children's future; aristocratic landlords commit to passing on richer estate than they inherit; and for a committed NGO or social movement it is simply inappropriate to devalue future rewards - the cause is everything. Similarly, where fundamental rights are involved it is inappropriate to devalue the future. Assessments of QALYs in a rights-based healthcare system might be expected to treat a year of life in 2050 as equal in value to an extra year of life in 2020: if they didn't, they would build in a profound bias in favour of the old and against the young. The application of standard discount rates turns out to reflect the values of highly individualised market economies and sectors. It is, in fact, quite culturally specific.

These apparently arcane issues are very relevant to questions as diverse as climate change and childcare, both of which involve profound moral commitments. They suggest that assessments of social or public value need to explicitly take account of how public attitudes and morality affect time preferences. These attitudes are likely to show a very different view of time in those parts of the public realm which are most like private consumption (for example, air travel) as opposed to those which are touched with moral obligations of stewardship or mutuality.

Lessons for social and public value

These examples bring out some important general lessons which are also apparent in the case studies. They remind us that value is not abstract but concrete - it exists in real places, at particular moments in time, and in a social context. There is no such thing as intrinsic value. It is a consequence not a cause.

Our valuations derive from processes of valuing and to that extent is often better thought of as a verb than a noun, and we value in many ways, whether through how we allocate our time, our money, use political influence or make commitments.

As such, value has only a loose relation to price. Many things are very valuable but not priced - like air, or a relationship to God. Price and the tools of economics become relevant in particular conditions of scarcity and organisation. So, for example, we might value seeing the Aurora Borealis - but it's rare and not organised, and therefore not easy to value in economic terms.

Yet, for all that, formal valuation can help us to make choices, and can key issues more visible. Our case studies are deliberately varied, rather than reflecting a single approach. But they point to some common conclusions:

First, that measures of value are only useful to the extent that they support negotiations and arguments about what needs to be done. They are useful if they bring choices and trade-offs to the surface, useless if they disguise them.

Second, more creative ways of handling value often depend on a guardian or social market maker who takes responsibility for bringing demand and supply together. In a democracy, that is likely to involve democratically elected politicians whose constitutional role it is to distil and represent public preferences, even if sometimes they also have to challenge the public's beliefs.

Third, value is an aspect of the relationship between states and citizens rather than an objective fact. It is shaped by what each considers desirable and important and then becomes more precise through processes of conversation and negotiation.

In any real-world situation, trade-offs have to be struck between the costs and time involved in more detailed assessments of value and the need for urgency.

But more systematic methods of mapping public and social value make assumptions more explicit and allow a more honest discussion between stakeholders about what they want and about what they can realistically get, helping technocrats avoid what Oscar Wilde described as the vice of knowing the price of everything and the value of nothing.

Hopefully this collection will give new confidence that we can do more than bemoan the deficiencies of current methods of valuation: alternatives are available, in use and steadily becoming more compelling.³²

Annex: Cost and value in the UK What Works Centres

National Institute for Health and Care Excellence (established 1999 and rebranded as a What Works Centre)

Policy area: Health and social care.

Strong approach

Value for money = value of increased effectiveness of the new intervention vs increased cost of the new intervention (Social Value Judgements p.18).

Value for money attributed to health gain can be obtained from their guidance and quality standards offer cost-effectiveness analysis:

• Incremental Cost Effectiveness Ratio (ICER) = is the ratio of the difference in the mean costs of an intervention compared with the next best alternative (which could be no action or treatment) to the differences in the mean health outcomes.
• Number of QALYs obtained in evaluation: interventions costing the NHS less than £20,000 per QALY gained are cost effective and those costing up to £30,000 per QALY gained might, with certain conditions satisfied, be considered cost effective.

They are also interested in value attributed by publics and social value judgements made by experts - judgements that take account of society's expectations, preferences, culture and ethical principles:

• Publics' perspective: currently being used in the New approach to cost-effectiveness required for end-of-life drugs.
• Social value judgements: they developed a guide on how to use social value judgements.

Critiques to NICE's approach to value for money:

• From NICE: Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone (Social Value Judgements).
• They do not take into consideration potential costs falling on other public sector budgets.

Education Endowment Foundation (established 2011 and rebranded as a What Works Centre)

Policy area: Educational attainment.

Strong approach

Value for money: critical to schools' ability to secure improvements in pupil outcomes. Cost and cost effectiveness is important for schools (Cost Guidance).

Cost-effectiveness: strong element in their approach to policy interventions.

Concept:

• It is important that practitioners are able to consider the effectiveness and cost of the approach together in a combined estimate of cost effectiveness.
• Practitioners need to know which approaches will secure the biggest improvements in their pupils' learning for the lowest cost.
• It is part of their evaluation toolkit: cost rating five-point scale to show cost-effectiveness of different approaches.

Actions:

They developed a guide on cost evaluation for schools and practitioner.

It is part of their strategy for the next five years: Their strategy for the next five years is to identifying the most cost-effective ways to scale evidence for impact (Five Years Plan).

Early Intervention Foundation (established 2013)

Policy area: Early intervention.

Soft approach

Value for money: cost of the intervention.

Concept of cost:

• Their cost evaluation is presented as not pure cost-effectiveness analysis (EIF evidence standards).
• They assess cost in terms of i.e. mapping local spending.
• Cost is not part of evidence: We know that national and local decision-makers need to take other factors into account, such as cost and local context (EIF strategy).

What Works Centre for Crime Reduction (established 2013)

Policy area: Crime.

Strong approach

Value for money: it can be achieved using their guidance on how to use evidence. It is a key element to make choices about where to spend and disinvest (What Works Centre Centre for Crime Reduction).

Cost benefit: is one of the dimensions included in their EMMIE model 'what direct or indirect costs are associated with the intervention and is there evidence of cost benefits?'

Including cost benefits evidence = high quality of evidence.

Cost effectiveness: is a recommended measure for all policy interventions (Reporting guidance).

What Works Centre for Local Economic Growth (established 2013)

Policy area: Local economic growth.

Strong approach

Value for money: use evidence to decide the broad policy areas on which to spend limited resources (How to Use Evidence Reviews).

Cost effectiveness: is a key element to understand the causal effects of policy interventions.

They only consider impact evaluations.

What Works Centre for Wellbeing (established 2014)

Policy area: Wellbeing.

Mixed approach

Value for money: better allocation of budgets and resources but money cannot be the sole guide to decision-making.

Other values to take into consideration: respectful of rights, culture and dignity, protection to vulnerable groups, social and community cohesion, long-term sustainability. (Wellbeing and cost effectiveness).

Cost effectiveness: they understand cost effectiveness needs to be applied in a different way when it comes to wellbeing interventions.

They developed a guide on how to do cost effectiveness in wellbeing.

It cannot be the sole guide in decision-making but it is important to compare feasible alternatives and it should include the above mentioned alternative values.

Centre for Ageing Better (established 2015)

Policy area: Ageing.

Soft approach

Cost benefit: they will make the economic case for healthy ageing modelling the costs and benefits of a preventative approach. Work to be delivered over 2018-10. (Strategy).

Cost effectiveness: does not appear in their approach to evidence or their approach to evaluation.

What Works Centre for Children's Social Care (established 2017)

Policy area: Children's Social Care.

Strong approach

Value for money: costs and benefits of interventions. It is an expectation from the public.

Cost benefit: economic implications of using an intervention. (Outcomes framework).

One of their six principles of their outcomes framework.

Key for research and service delivery.

What Works Scotland (established 2014)

Policy area: Scottish local services.

Alternative approach

Value for money: no clear definition.

Cost effectiveness: there are some cost-effectiveness considerations in some of their reports but it is not one of their key principles.

i.e. It is clear from this evaluation that participatory processes require time and resources, and do not necessarily improve efficiency and reduce costs, at least not in the short term. (Placed-based evaluation).

i.e. Examples of What Works Scotland process-based work would include extended collaborative action research in four Scottish CPPs, and developing evaluability approaches to embedding cost-effective tailored evaluation into policy interventions. (Outcomes based approach).

Cost consequence economic evaluation mentioned in one of their reports but it is not developed in their outcomes framework.

Wales Centre for Public Policy (re-launched June 2017)

Policy area: Welsh public services

Approach not defined

Cost analysis used in one of their projects.

Endnotes

1Fisher, W. P. (2010) 'Meaningfulness, Measurement, Value Seeking and the Corporate Objective Function: An Introduction to New Possibilities'. Berkeley CA: University of California. https://ssrn.com/abstract=1713467

2Fujiwara, D. and Campbell, D. (2011) 'Valuation Techniques for Cost Benefit Analysis: Stated Preference, Revealed Preference and Subjective Well-Being Approaches.' London: HM Treasury. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/209107/greenbook_valuationtechniques.pdf

3For example, one study of an urban regeneration project was found to have achieved an improvement equivalent to £19,000 per person of working age, yet 'willingness to pay' studies came up with a figure of only £230 per year:- Dolan, P. and Metcalfe, R. (2008) 'Comparing willingness-to-pay and subjective well-being in the context of non-market goods'. LSE CEP Discussion Paper No. 0890, London School of Economics. http://eprints.lse.ac.uk/28504/1/dp0890.pdf For a further study comparing willingness to pay estimates see also Bakhshi, H., Fujiwara, D., Lawton, R., Mourato, S. and Dolan, P. (2015) 'Measuring Economic Value in Cultural Institution'. Swindon: AHRC. https://ahrc.ukri.org/documents/project-reports-and-reviews/measuringeconomicvalue/

4Kornberger, M., Justesen, L., Madsen, A. K. and Mouritsen, J. (2015) 'Making Things Valuable.' Oxford: Oxford University Press.

5Flyvbjerg, B., Bruzelius, N. and Rothengatter, W. (2003) 'Megaprojects and risk: An anatomy of ambition'.- Cambridge: Cambridge University Press.

6Gintis, H., Bowles, S., Boyd, R.-and Fehr, E. (Eds.) (2005) 'Economic learning and social evolution. Moral sentiments and material interests: The foundations of cooperation in economic life'. Cambridge MA: MIT Press. https://www.umass.edu/preferen/gintis/moral%20sentiments.pdf

7A study in Michigan, for example, found a significant link (a correlation of 0.26) between perceptions of procedural justice in government services, as distinct from outcomes, and trust in politicians: Miller, A. and Listhaug, O. (1999) 'Political Performance and Institutional Trust'. In Norris, P. (Eds.) 'Critical citizens: Global support for democratic government'. Oxford: Oxford University Press.-There are many examples of reforms which apparently improved efficiency but ruptured this relationship and therefore damaged the legitimacy of government: Demery, L., Ferroni, M. and Grootaert, C. (Eds.) (1993) 'The Social Effects of Policy Reform. A World Bank Study'. Washington D.C: The World Bank.

8Further, experimental evidence from the OECD's Trustlab shows 'an increase in all significant determinants of trust in government by one standard deviation may be conducive to an increase in trust by 30 to 60 per cent':- Murtin, F., et al. (2018) 'Trust and its determinants: Evidence from the Trustlab experiment'. OECD Statistics Working Papers, No. 2018/02.-Paris: OECD Publishing. https://doi.org/10.1787/869ef2ec-en

9The Centres provide a variety of products to communicate their evidence. The Crime Reduction Toolkit, the EIF Guidebook, EEF's Teaching and Learning Toolkit and the Early Years Toolkit, the Growth Toolkits, and the Wellbeing Evidence Comparison Tool all provide the summaries of findings across a range of topic of concern to the Centre. All of these provide information on impact, cost and strength of evidence. The Crime Reduction Toolkit in addition provides evidence on the process by which the intervention has its effect which can assist with interpreting the evidence in relation to different use contexts.

10Casey, D. A., Antimisiaris, D.-and O'Brien, J. (2010) Drugs for Alzheimer's disease: are they effective? 'P & T : a peer-reviewed journal for formulary management'. 35(4), 208-11. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2873716/

11This was subsequently toned down in later guidance, although the drug is only offered to those with moderate or severe Alzheimer's: https://www.nice.org.uk/guidance/ta217/chapter/1-Guidance

12Neumann, P. J. and Weinstein, M. C. (2010), 'Legislating against use of cost-effectiveness information', New England Journal of Medicine, 363. p. 1496.

13Neumann, P. J. and Weinstein, M. C. (2010), 'Legislating against use of cost-effectiveness information', New England Journal of Medicine, 363. р. 1496.

14This can be triangulated with surveys of the wider public to find out about the QALY of a health intervention: 3,000 members of the public help score the different dimensions of quality of life from EQ-5D. See Ogden (2017) for description.

15This doesn't mean the What Works Centre for Wellbeing only examines Life Satisfaction when reviewing research they do allow people to convert evidence from other wellbeing measures into equivalent 'units' of life satisfaction to make it easier to compare wellbeing outcomes measured in different ways.

16Source: Puttick, P. and Ludlow, J. (2012) 'Standards of evidence for impact investing'. Nesta

17'Evidence vs Democracy: How 'mini-publics' can traverse the gap between citizens, experts, and evidence'. Alliance for Useful Evidence/ANSA, 2019.

18Coutts, P. and Brotchie, J. (2017) 'The Scottish Approach to Evidence'. Nesta-and Carnegie Trust.

19Kirk, P. and Shutte, A.M. (2004) Community leadership development. 'Community Development Journal'. Vol. 39 No. 3, pp. 234-251.

20Halsam, D. (2016) 'The NICE Citizens Council and the role of social value judgements'. Presentation available at http://www.idsihealth.org/wp-content/uploads/2016/05/Citizens-Council-and-Board.pdf

21Adult/child ratios in pre-school provision tend to be higher than in school classes - so harder to justify. Family interventions have similarly high costs. The average cost per child of a Sure Start Local Programme was £1,300 in 2009-2010.

22Smith, D. and Gravelle, H. (2000) 'The Practice of Discounting Economic Evaluation of Health Care Interventions'. CHE Technical Paper Series 19. The University of York.

23Throsby, D. (2001) Economics and Culture, Cambridge University Press.

24Bakhshi, H., Fujiwara, D., Lawton, R., Mourato, S. and Dolan, P. (2015) 'Measuring Economic Value in Cultural Institution'. Swindon: AHRC.

25Fujiwara, D., Bakhshi, H., Mourato, S., Hotopp, U., Lawton, R., Lagarde, A. and-Davies, J. (2018) 'The Economic Value of Culture: A Benefit Transfer Study'. London: DCMS.

26Lawton, R., Fujiwara, D., Mourato, S., Bakhshi, H., Lagarde, A., and-Davies, J. (2018) 'The Economic Value of Heritage: A Benefit Transfer Study'. London: Nesta.

27HM Treasury (2018) 'The Green Book Central Government Guidance On Appraisal And Evaluation'. London: HM Treasury.

28Bakhshi, H (2012), 'Measuring Cultural Value', London: Nesta.

29Davies, J. (2017) 'Optimal designs: What are some of the lessons of grand landmark buildings projects.' London: Nesta.

30Naik, N., Philipoom, J., Raskar, R. and Hidalgo, C. (2014) 'Streetscore-Predicting the Perceived Safety of One Million Streetscapes'. CVPR Workshop on Web-scale Vision and Social Media. MacKerron, G. (2012) 'Happiness and environmental quality'. PhD thesis, LSE. Brynjolfsson, E., Eggers, F. and Gannamaneni, A. (2018), 'Using massive online choice experiments to measure changes in well-being'. NBER Working Paper 24514. Cambridge MA: NBER.

31Mather, L. and-Robinson, P. (2016) Civic Crafting in Urban Planning Public Consultation: Exploring Minecraft's Potential. 'International Journal of E-Planning Research'.

32Baeck, P., Bone, J. and Mitchell, S. (2017) 'Matching the crowd: Combining crowdfunding and institutional funding to get great ideas off the ground'. London: Nesta.

33HM Government (2018) 'Industrial Strategy - Creative Industries Sector Deal'. London: HM Government.

34For example, the work of the Institute for Healthcare Improvement, and the Realistic Medicine movement in Scotland.

35Payne, C. (2017) 'Changes in the value and division of unpaid volunteering in the UK: 2000 to 2015'. Newport: ONS.

36Yeandle, S. and Buckner, L. (2015) Valuing carers 2015- the rising value of carers' support'. London: Carers UK.

37For example, captured through the Care Act 2014 focus on wellbeing, prevention, personalisation and co-production.

38For further examples see https://www.nesta.org.uk/report/impact-and-cost-economic-modelling-tool-for-commissioners/

39Corrie, C. and Finch, A. (2015) 'Expert Patients'. London: Reform.

40Monitor (2015) 'Closing the NHS funding gap: how to get better value health care for patients'. London: Monitor.

41Nesta (2013) 'The business case for People Powered Health'. London: Nesta, Innovation Unit, PPL.

42Realising the value (2016). 'Ten key actions to put people and communities at the heart of health and wellbeing' London: Nesta.

43Realising the value (2016) 'Realising the value Ten key actions to put people and communities at the heart of health and wellbeing' London: Nesta.

44Realising the value (2016). 'New approaches to value in health and care.' London: Nesta.

45Realising the value (2016). 'New approaches to value in health and care.' London: Nesta.

46Published as 'Creating Public Value' by the Cabinet Office Strategy Unit in 2002.

47[i] This piece draws on several main sources including: 'Creating Public Value', by Geoff Mulgan, Gavin Kelly and Stephen Muers, published by the UK Cabinet Office in 2002; a report on value for the Commission on Architecture and the Built Environment by Geoff Mulgan, Gareth Potts, Matthew Carmona, Claudio de Magalhaes and Louie Sieh published in 2006 (available on The Young Foundation website). The website www.socialinnovator.info provides details of many measurement methods, alongside several hundred methods in use around the world for every aspect of social innovation. A forthcoming book from the US National Academy of Science provides an overview of the role of standardised indicators in different fields. [ii] 'The impact of subjective wellbeing on local authority interventions' Dolan and Metcalfe, February 2008. [iii] The following books provide a good overview: Barrow, C. J. (2000) 'Social Impact Assessment: An Introduction'. London: Arnold. Becker, H. and Vanclay, F. (2003) 'The international handbook of SIA'. Cheltenham: E Elgar. Becker, H. A. (1997) 'Social impact assessment: method and experience in Europe, North America and the developing world'. London: UCL Press. Scholten, Nicholls, Olsen, Galimidi (2006) 'SROI A Guide to Social Return on Investment'. Lenthe Publishers. A good overview of the rigorous end of assessment and evaluation is 'Identifying the environmental causes of disease: how should we decide what to believe and when to take action' (Academy of Medical Science, 2008). A recent study of tools used by foundations is 'Learning for Social Impact', from McKinsey, 2010, one of a number of recent moves to advocate separate tools for NGOs and foundations, an approach which I question. For philosophical perspectives John Dewey's 'A theory of valuation' and Luc Boltanski and Laurent Thevenot's 'On Justification: economies of worth' are highly recommended.

nesta

58 Victoria Embankment London EC4Y ODS

+44 (0)20 7438 2500 [email protected] @nesta_uk www.facebook.com/nesta.uk www.nesta.org.uk

Nesta is a registered charity in England and Wales with company number 7706036 and charity number 1144091. Registered as a charity in Scotland number SCO42833. Registered office: 58 Victoria Embankment, London, EC4Y ODS.

Creative Commons License