About Nesta

Nesta is an innovation foundation. For us, innovation means turning bold ideas into reality and changing lives for the better. We use our expertise, skills and funding in areas where there are big challenges facing society.

Why systems thinkers and data scientists should work together to solve social challenges

The most interesting innovations emerge when two unlikely disciplines intersect.

A great example is the winner of Nesta's Tipping Point prize, who is working in Swarm Robotics - a field which applies insights and patterns from natural biology to robotic design.

When two unexpected fields of work come together, each can offer the other:

  • new ways of framing problems and understanding the world
  • different skills and practices
  • new methods and toolkits
  • alternative theoretical frameworks.

Cross-disciplinary collaboration can unearth unexpected synergies which create opportunities for radical innovation.

Breaking down the silos

One of the wonderful things about Nesta’s Government Innovation team is the breadth of projects and programmes we are involved in. We work on Democratic Innovation, Digital Social Innovation, new operating models for local authorities, and Offices for Data Analytics (to name just a few).

However, what I’ve observed about these projects - and the fantastic networks of passionate champions who drive them - is that they operate in quite siloed ways. People go to conferences which focus on their field of work; they read, write, and share articles about their field of expertise, and often end up operating in some version of an echo-chamber.

We need to break down these echo-chambers, and open ourselves to learning from people working in different disciplines. Cross-sectoral (or cross-field) collaboration is never easy, as I’ve written about previously; but it can produce some stunning innovations.

This is true as a general statement. But there are two fields in particular, where I believe that opportunities for greater collaboration feel particularly exciting - these are the fields of systems thinking and big data.

What is systems thinking?

Systems thinking is characterised by an approach to problem-solving which invests in understanding the system within which a problem or challenge is situated, rather than targeting a specific element of the problem.

It is best explained by an example. Imagine you’re a farmer, and your crops are being damaged. What do you do? You might identify that grasshoppers appear to be causing a lot of damage, and focus on killing all of the grasshoppers.

Linear thinking

However, while this may work in the short-term, it often doesn’t work as a long-term solution. This is because the grasshoppers are (unbeknownst to you) controlling a population of other insects. As such, killing all of the grasshoppers results in a proliferation of other species, meaning that your solution ultimately causes more crop damage. This is the danger of viewing challenges as linear and discrete.

Viewing the challenge of insects causing crop destruction as being part of a broader ecosystem would likely result in a different solution. For example, you might adopt a permaculture approach - a great example of a systems thinking model used to support crops to thrive.


Permaculture principles. Image: The Seedling at Sagada

Systems thinking is emerging as a powerful way to address social challenges. This approach rejects targeted, siloed and time-limited solutions to social challenges, in favour of a “systems change” approach.

What is big data?

Big data refers to data sets that are so large, commonly used software tools are unable to capture, curate, manage, and process that data within a reasonable period of time.

Big data encompasses both unstructured and structured data. A 2016 definition states that: "Big data represents the information assets characterised by such a high volume, velocity and variety to require specific technology and analytical methods for its transformation into value."

Big data has emerged as a field because technological advances have resulted in:

  1. an exponential growth of data (see infographic below)
  2. tools - such as AI - which allow that data to be transformed into something valuable.
A day in data

Big data can be used to reveal patterns, predict risks and opportunities, and offers new insights into systems.

An example of big data in action is the Netflix algorithm, which draws on your personal data and combines that with the data of more-than-75-million users worldwide. The data is fed into an AI algorithm, which predicts patterns in what you are likely to want to watch next, offering you recommendations which are, generally speaking, pretty spot on!

The stories we hear about big data are often framed through a negative lens. For example, Shoshana Zuboff speaks about how big data has supported the emergence of “Surveillance Capitalism.” However, while a lot of big data is being put to questionable use, it doesn’t have to be that way.

We need to be thinking more about how to reclaim the value that big data offers and apply it for social good.

Systems thinking, meet big data

Systems thinking and big data as independent fields have real strengths, but also some key weaknesses.

Big data is great for observing broad patterns and trends, but can miss nuances that would be obvious to the human eye, and which form an important part of the stories of individuals and communities.

On the other hand, systems thinking as a methodology can unearth powerful and complex insights, but this happens very slowly, and is hugely resource intensive.

The complementarity that each field offers the other is clear. When looking to understand communities and solve social challenges, systems thinking injects depth and nuance, while big data provides insights into patterns and risks that are only possible to unearth with the assistance of some computing power. As danah boyd and Kate Crawford wrote in 2011 - “Big data has... the power to inform how we understand human networks and community.”

Open data portals, for example, are extremely useful resources for those wanting to gain deeper insights into communities. The datasets published can be used to develop a sophisticated understanding about social wellbeing within a particular community; can reveal trends and spending patterns in the field of health and social care; and can even support a deeper understanding about how people travel within a city. These are all critical aspects of a high-functioning community, which systems-change practitioners need to understand.

Big data has...the power to inform how we understand human networks and community.

Unstructured data can also generate useful insights to support systems-change work. For example, job advertisements (which Nesta has been analysing as part of our Open Jobs project) can help build a richer picture of skills needs in particular places.

Another example of unstructured data is social media, which can be scraped and used for sentiment analysis in order to reveal the issues that community-members are most passionate about. For example, Facebook conversations might unearth a common desire for investment in a particular community asset - maybe a better playground. This kind of information can offer people working to solve social challenges important new insights to work with.

This is not to say that big data should replace the qualitative data that systems thinkers have tended to rely on to inform their practice. Absolutely not. What is needed is both big data and thick data, defined by ethnographer Tricia Wang as ‘data brought to light using qualitative, ethnographic research methods that uncover people’s emotions, stories, and models of their world’. When combined, big data and thick data build a rich understanding of communities and the challenges they’re facing.

Thick data vs Big Data

Making the connection

How can we make clear the value of big data to human-centred practitioners who tend to use ethnographic methods and qualitative data to support their work? And how might social change organisations access the tools, skills and resources needed to make the most of big data?

On the other side of the equation, how might we encourage data scientists and practitioners in the field of data analytics to think more deeply about how their work might support social innovation?

There is already some great thinking happening in this space - for example Data&Society is a research institute dedicated to advancing the public understanding of the social implications of data-centric technologies, and the Oxford Internet Institute is a multidisciplinary research and teaching department of the University of Oxford, dedicated to the social science of the internet.

But how do we shift this from the field of thinking to the field of doing? What needs to happen for the theory to start translating into people’s practice?

I think the first step is probably bringing people together from both fields to discuss some of the opportunities and challenges. This might be through workshops, a conference, or some sort of regular meet-up. More generally, we need to see more events which are cross-sectoral. The anthropology + technology conference which will take place in Bristol on 3 October is a great example of this. Perhaps we need a systems change + big data conference?

But how do we shift this from the field of thinking to the field of doing?

In addition, there need to be practical tools and institutions to support people in these fields to work more effectively together. The fantastic organisation DataKind is working to connect top data scientists with leading social change organisations to collaborate on cutting-edge analytics and advanced algorithms to maximise social impact. Across the other side of the world in Australia, Our Community House is offering data science tutorials to community sector organisations. We need more of this! And we need to see it beginning to happen organically and independently of the support of organisations like DataKind and Our Community (who can only do so much!).

Nesta’s work on Collective Intelligence also feels like an important part of this conversation. Through the Centre for Collective Intelligence Design, Nesta has set up a team which is dedicated to exploring the potential for innovation when human and machine intelligence is combined.

This blog sets out some nascent thinking. My intention is to begin a conversation, which I hope people are interested to join! I’d welcome any feedback and reflections either here, or via twitter @theasnow.


Thea Snow

Thea Snow

Thea Snow

Senior Programme Manager, Government Innovation

Thea was a Senior Programme Manager in the Government Innovation team.

View profile