Mission Measurable

Governments across the world are putting in place mission-oriented Research and Innovation policies to tackle some of the biggest social and environmental challenges of our time. From producing clean energy to improving health outcomes, coping with an ageing society or deploying powerful new technologies like AI in a way that is safe and fair. But what indicators do we use to design, implement and evaluate these ambitious policies?

In Innovation Mapping Now, we argued that traditional Science, Technology and Innovation indicators that measure R&D expenditures and count intellectual property outputs such as publications and patents are not up to the task. This is because they do not reflect key dimensions of mission-oriented innovation policies such as:

  • Their goals: mission-oriented policies do not just seek to increase levels of innovation and boost its economic impacts, but also to steer it towards achieving particular purposes. This requires measuring how innovation emerges and is diffused to address a mission's goals.
  • Their theories of change: mission oriented policies seek to deliver their goals by creating new communities that bridge disciplines to tackle societal challenges also incorporating knowledge from the social sciences and humanities. Indicators about missions need to capture this crossover.
  • Their risks: mission oriented policies need to avoid picking winners, which involves encouraging bottom up experimentation instead of top-down solutionism, and capture by innovation elites and vested interests, which requires public participation in innovation policy processes and the involvement of new actors in the innovation system.

We need new relevant, inclusive and trusted open indicators that capture these goals, theories of changes and risks and are thus able to inform research and innovation missions which, as we are arguing in our Fuelling the Future campaign, need to become a bigger component of the innovation policy mix. In EURITO, a European Commission-funded Horizon 2020 project we are leading in collaboration with researchers in DTU (Denmark), Fraunhofer FOKUS (Germany) and Cotec (Spain) we are exploring the potential to create those indicators with novel big and open data sources and data analytics.

At an earlier stage of the project we scoped policy needs and identified a set of pilot areas where there are big opportunities to address burning policy questions with new indicators. Innovation missions was one of them.

Today we are publishing a working paper that reports emerging findings from this pilot where we have developed prototype indicators related to the UK Grand Challenge Mission to 'Use data, Artificial Intelligence and innovation to transform the prevention, early diagnosis and treatment of chronic diseases by 2030‘. This involves an analysis of open data from the Gateway to Research, a repository of information about projects funded by UK Research Councils (now under the umbrella of UKRI) going back to 2007. We use natural language processing and machine learning methods to measure key dimensions of the ‘active mission field’ in the intersection between AI and chronic diseases including:

  • Its size and evolution. We find that AI is underrepresented in chronic disease research, providing a rationale for the mission. Having said this, we find rapid growth in the levels of activity in this area even before the UK Government announced the mission.
  • Its disciplinary composition. We find disciplinary crossover in the active mission field, and individual projects. This is consistent with the idea that tackling this complex health challenge requires new combinations of knowledge.
  • Its propensity to generate applied outputs. We find that research in the active mission field is good at generating applied outputs such as clinical products, patents and spin-outs, suggesting that the challenge that it seeks to tackle is technologically feasible.
  • The identity of the organisations participating in the active mission field. Here, we find some evidence that the organisations participating in the active mission field are newer to the UK Research and Innovation system, in line with the idea that innovation missions can bring new participants into Research and Innovation.
  • The composition of the active mission field. Although the active mission field was initially dominated by computer scientists, recent years have seen more active participation from researchers in the medical and biological sciences.

The kind of indicators that we present in the paper could inform decisions across the mission policy cycle. During mission selection, they could help identify research areas related to a societal challenge where there is less activity than might be desirable. They could also be used to find key actors in an active mission field to engage with during policy design and implementation, identify gaps in the range of solutions being explored inside the mission and monitor participation to ensure that the mission is bringing new participants in the innovation system. At the evaluation stage, it would be possible to compare the evolution of indicators with baselines, and to assess whether changes in activity and network structure have advanced the mission.

These examples illustrate the potential usefulness of novel indicators for mapping innovation missions and informing mission-oriented Research and Innovation policies. We believe that policymakers should make the most of these opportunities, thus ensuring that the latest wave of Research and Innovation policies are effectively designed, targeted and evaluated, maximising the probability that they achieve their ambitious goals.

We are currently scaling up the indicators we developed for the UK AI chronic disease mission to a wider set of countries and missions. Get in touch with us if you want to find out more and collaborate with us in our mission: to transform research and innovation policy with new data and indicators.

Author

Juan Mateos-Garcia

Juan Mateos-Garcia

Juan Mateos-Garcia

Director of Data Analytics Practice

Juan Mateos-Garcia was the Director of Data Analytics at Nesta.

View profile