What do we need to measure? The case for subjectivity and disaggregation

The world is enjoying a flood of data and statistics. But the public sometimes seem as likely as ever to ignore numbers they don’t like, or to see them as irrelevant.

This can be exaggerated. Polls show very strong support for objective and independent statistics. But we shouldn’t be complacent. Here I suggest two directions for statistics that could help.

20th century statistics

The great triumph of 20th century statistics was to generate a series of aggregate numbers that helped countries understand themselves, and helped guide their policies.

The most famous were the measures of GDP that were devised in the 1930s. These came to sit alongside aggregate statistics on population, age groups, money in all its forms, education levels and mortality levels. They shaped the look and feel of modern government and many became the currency of public debate, which came to be organised around trade deficits, unemployment rates, public spending borrowing requirements, waiting lists in health, and maths results in schools, and so on.

These had many virtues and all were an advance on what preceded them. But they shared two major deficiencies. The first was that they were not very human. They were concerned with quantities and facts, but not with experiences, emotions or perceptions. Their implicit worldview was that these were somehow less important, or less real, than apparently material facts.   

Their second deficiency was that the numbers were aggregates.

It can be useful to know what a country’s GDP or Human Development Index is. But often it’s the differences that matter more

These differences include inequalities between groups, regions and generations, or the complex patterns that make one city so different to another.

So, in the 21st century, moves have been made to tackle these deficiencies head on, and my guess is that these moves will gain new momentum from the various political revolts underway.

Measuring feelings

The first shift will put a higher premium on subjective measures. This has already happened to some extent through measures of subjective wellbeing, like the ones introduced by the UK’s Office of National Statistics at the beginning of this decade - measuring happiness - following work at the OECD which resulted in their Better Life Index.   

Some fields have already taken this shift seriously: tracking fear of crime as well as crime, patient satisfaction as well as health outcomes.

Looking to the future, we could aim for many more objective measures to join with subjective ones, so as to bring humanity back into policy and politics.

Economic statistics would be a good starting point.

There are already plenty of subjective measures in economics - surveys of business and consumer confidence

But imagine if there were also more regular measures of how powerful people felt in their workplace, in the labour market, as consumers, or as actors in the housing market?   

Some surveys look at issues such as perceptions of autonomy at work, often with very interesting results. But much more could be done to track how waves of economic change are being experienced.

There’s a lively debate within statistics about the relationship between objective and subjective measures – and whether these terms are misleading (for example, see this from Andrew Gelman and Christian Hennig). 

But the main point stands  - we badly need to become better at understanding the human dimension of change.

Disaggregation rather than aggregation

To be useful, however, many of these numbers need to be disaggregated. The interesting patterns can easily be lost if numbers are brought together at too high a level. This is a more general point, and crucial to 21st century statistics.

We’re in an era when differences matter as much as averages. This is why Nesta’s recent work mapping economies aims to explore the details not the aggregates.  

The Technation studies of the digital economy, using Nesta’s methodology, show the emerging clusters of firms and jobs in new sectors.  Although there are headline numbers, these are a lot less interesting than the detail.

The study of complexity in local economies shows a similar point – and suggests a possible predictor of future growth. The studies on the dynamics of the creative industries have been designed as interactive tools so that anyone can explore the specific patterns. All focus on detail and complexity rather than averages, and are therefore more useful as guides to action for citizens, businesses and others.

Many institutions – like the UN, Eurostat, OECD, IMF and World Bank - continue to churn out vast amounts of high quality statistical data

This has its uses, and makes it possible to compare national performance.  But it would better to evolve and open up in the ways described above.

An example of a different frame for thinking about the state of a nation is summarised in the diagram below (taken from a report I co-authored in the late 2000s looking at Britain’s needs).  

The Y axis charts the familiar territory of material prosperity – how much people earn. The X axis charts the less familiar territory of psychological prosperity – how happy people are.  

Put together they give us a picture of the state of society – from the rich and happy to the poor and miserable, flanked by the poor but contented and the richly depressed.

I’m convinced that this is one of the best snapshots of how a society is doing. It combines material and subjective measures into a single composite.

But the greatest interest lies in looking at the detail of particular groups and places. The average levels of income, or well-being, tell rather little.

There’s no reason for people not to believe, and use, statistics. But to be useful they need to be tied into stories, and some of the most compelling stories will involve feelings, and detail. A one sentence summary of War and Peace has little interest because it loses all that’s interesting.

The 17th century French mathmatician and philosopher, Blaise Pascal, once wrote "pictures seen from too far or too near; there is but one exact point which is the true place from which to look at them: the rest are too near, too far, too high, or too low.’ So what is the right vantage point for numbers? That’s the question given new urgency now.

Author

Geoff Mulgan

Geoff Mulgan

Geoff Mulgan

Chief Executive Officer

Geoff Mulgan was Chief Executive of Nesta from 2011-2019.

View profile