About Nesta

Nesta is an innovation foundation. For us, innovation means turning bold ideas into reality and changing lives for the better. We use our expertise, skills and funding in areas where there are big challenges facing society.

The final step in the process of running a trial is working out what to do with the results.

An effect size (for example, a two percentage point increase in take-up) isn’t always the easiest thing to interpret or communicate to others. What we really want to know is the impact in meaningful, real-world terms.

Translate your results into meaningful impacts

Let’s imagine that you’ve tested a new two-year-old offer communication and observed a three percentage point increase in take-up. What does this mean in practice?

Go through these three steps to translate your results into a meaningful long-term estimate of impact:

  1. Develop some assumptions
  2. Estimate the overall impact
  3. Calculate the costs and benefits

The very final stage of the process is deciding whether or not to implement your intervention in the future or not. Whether or not you have a statistically significant result will play a big role here, but it is not the only consideration.

Some other questions to consider are:

  • What was the effect size? Is it meaningful in practice?
  • Are the benefits of the intervention worth the costs?
  • What feedback did we get from parents and staff about the new process?
  • Can we make any further tweaks to make it more effective?
  • Do we need to test it further to make a decision?

The very final stage of the process is deciding whether or not to implement your intervention in the future or not. Whether or not you have a statistically significant result will play a big role here, but it is not the only consideration.

Some other questions to consider are:

  • What was the effect size? Is it meaningful in practice?
  • Are the benefits of the intervention worth the costs?
  • What feedback did we get from parents and staff about the new process?
  • Can we make any further tweaks to make it more effective?
  • Do we need to test it further to make a decision?

Authors

Louise Bazalgette

Louise Bazalgette

Louise Bazalgette

Deputy Director, fairer start mission

Louise works as part of a multi-disciplinary innovation team focused on narrowing the outcome gap for disadvantaged children.

View profile
Dave Wilson

Dave Wilson

Dave Wilson

Advisor

Dave is an Advisor in the Education team at the Behavioural Insights Team (BIT) with a focus on early years projects.

View profile
Fionnuala O’Reilly

Fionnuala O’Reilly

Fionnuala O’Reilly

Lead Behavioural Scientist, fairer start mission

Fionnuala was the lead behavioural scientist in the fairer start mission, seconded from the Behavioural Insights Team (BIT).

View profile