Evidence is an integral part of the Nesta Impact Investment Fund (NIIF). We think that measuring the impact of the products and services we fund, is just as important as increasing the capacity of an organisation to actually deliver the product or service. Here we define our key terms and outline what we mean by evidence and impact.
Why is evidence important?
We need evidence to establish whether a product or service is benefitting those it sets out to serve, and then to focus investment on products and services that can make the most difference.
Investing in things that are shown to be working will increase the chance of achieving positive outcomes. In other words, by ensuring that products and services are safe and efficient means they will be more beneficial to the individuals and communities that they set out to serve. To do this we need high quality evidence of impact.
What do we mean by evidence?
We don't associate 'evidence' with particular types of data or specific research methods. Instead we are interested in high quality, robust evidence which helps identify the most promising investments, and then to continue to generate relevant types of evidence to ensure they are working.
We are interested in evidence that shows how a product or service can positively impact one of our outcome areas, spanning youth attainment and education, resource efficiency and sustainable communities, and supporting older people to live more independently.
We will work with investees to produce well conducted, high quality impact evaluations. We don't expect all investees to be at the same stage of development, nor do we envisage that they will all have the same types of evidence, therefore we have developed Standards of Evidence to help differentiate between them and to provide a scale for investments to move up.
Why have we developed standards of evidence?
As products and services move up the standards of evidence so will our certainty that they are having a positive impact on our target outcomes.
Rather than demanding one particular type of evaluation approach we have developed Standards of Evidence that enable products and services at different stages to be considered. This is in recognition that products and services at different stages of development will need to be evaluated in different ways, and also that not all of our investees will be at the same stage of development or evolve at the same pace. This is why our standards of evidence consider a wide range of methods, approaches and different types of evidence appropriate to different stages of the innovation process.
How did we develop the NIIF standards of evidence?
There are numerous "levels" and "standards" of evidence that have been developed to help structure how evidence is gathered, interpreted and assessed. Our starting point was the standards of evidence that underpin the Greater London Authority's Project Oracle[i], an innovative, city-wide programme that seeks to build the evidence behind youth programmes in London. We started with these standards of evidence because they effectively manage to retain academic standards of rigour whilst ensuring that the evidence requirements are appropriate to the development of services and products. This is fundamentally important for helping to ensure that the collection of evidence does not hinder innovation. We have amended and adapted the Oracle Standards to fit with the requirements of a venture fund.
What are the Nesta Standards of Evidence?
The objective of the Nesta Standards of Evidence is to help us know how confident we can be that a product or service is having a positive impact.
The Nesta Impact Investment Fund Standards of Evidence are on a 1 to 5 scale with level 1 being the first starting point that we require all investees to have, moving up to level 5 where a product or service has a very strong evidence of effectiveness. As the levels are progressed, it will be expected that data is collected to isolate the impact to the product or service, that findings are validated externally, and then at Level 5, demonstrable evidence that the product or the service can be delivered at multiple locations and still deliver a strong, positive impact.
Further details about the Standards of Evidence are available in a paper, but in summary these are:
For further information on the standards of evidence, including details on the types of methodologies that could be considered at different stages, please see our recent paper, Standards of Evidence for Impact Investing.
What level of evidence do you need before making an investment?
All applicants to the Fund will need to be at least at Level 1.
Level 1 requires a well thought account of impact. As we noted above, in essence this means understanding what a product or service can do, who it will be aimed at, and a rationale for why it could impact upon one of our outcome areas.
For all levels we expect to see high quality data and evidence. The fund team will verify the data to assess the stage met.
What do the standards of evidence mean once investment is made?
An explicit requirement of funding will be for all investees to work up through the standards of evidence and to continue to build the evidence behind their claims. At the outset of an investment we will provide support for all investees to develop their evaluation plan, structuring how they can collect the data needed. This will be structured and managed at a pace appropriate to each individual product and service. We will also help identify external support when appropriate, particularly at the higher levels.
Who decides what level a product or service has reached?
The NIIF team are highly experienced at assessing the claims behind impact evaluations and deciding where to position them on the scale. Each application will be assessed on an individual basis to assess the quality of the evidence they use to establish the strength of their claims.
How will we use the standards of evidence?
There will be four overlapping applications of the standards of evidence:
How do we define our key terms?
Output: a measurable unit of a product or a defined episode of service delivery directly produced by an investee's activities. For example, in our education strand of the Fund this could mean the length of a time a number of young people receive an anti-bullying programme.
Outcome: an observable, and measurable, change for an individual or organisation. We have already defined the outcomes we would like our investments to achieve across the three areas of ageing, self-sustaining communities, and children and young people (link to page which has info on outcomes). One example of a positive outcome from our 'ageing' strand of the Fund could be the reduction in avoidable injury and premature deaths of older people.
Impact: the effect on an outcome that can be attributed to the output. In other words, what affect is the product or service having on its target population? Across all areas of the Fund, having an impact requires the investee to have an idea of how their product or service can affect this change and then keep track of how well this impact is being achieved. Impact may be positive or negative, and through high quality evaluation at the higher levels of the standards of evidence, this impact will be quantified.
Impact risk: is a concept we have developed to give an indication of the certainty that an output will lead to the stated impact. Our Standards of Evidence are used to assess impact risk. As we noted above, as we move up through the Standards of Evidence our confidence in the product or service achieving a stated outcome will increase, reducing our impact risk.
How can I find out more?
We have published a research paper that outlines our approach to impact and evidence which is available here
If you have any other comments or questions please contact us on information@nesta.org.uk
[i] The Project Oracle Standards of Evidence are available here
![Standards of Evidence for Impact Investing thumb [original]](http://admin.nesta.org.uk/library/images/StandardsOfEvidenceThumb.jpg "Standards of Evidence for Impact Investing thumb [original]")
This paper introduces Nesta Impact Investments fund and the Standards of Evidence for impact investing.
Download the paper
T +44 (0)20 7438 2500
F +44 (0)20 7438 2501
Nesta Operating Company, a company limited by guarantee registered in England and Wales with company number 7706036 and charity number 1144091. Registered as a charity in Scotland number SC042833.
