Evidence transparency framework
DIAGNOSIS
This concerns why something is proposed, i.e. what the issue is that will be addressed.
The document should explain:
* what policy makers know about the issue, its causes, effects, and scale
* how policy makers have assessed the strengths and weaknesses of that evidence.
This section assesses the clarity of evidence used and its role in understanding the problem (SO, CAN YOU SEE WHAT EVIDENCE HAS BEEN USED AND THE ROLE IT HAS PLAYED?).
| Level 0 |
Level 1 |
Level 2 |
Level 3 |
WORKED EXAMPLE |
| Not clearly enough for level 1. |
Evidence is mentioned, with some explanation of how it has been used. |
As in level 1 but the supporting evidence is linked to the relevant parts of the policy, properly cited and you could find the source. |
As in level 2 but the evidence base is also assessed and uncertainties and contradictory information are acknowledged. |
The Government has assessed the extent of problem drinking in the UK: the economic and human cost. |
PROPOSAL
What is the Government's chosen intervention?
The document should explain:
* why the Government has chosen this intervention
* what evidence, if any, that choice is based on
* how policy makers have assessed the strengths and weaknesses of the evidence base, including what has been tried before and whether that worked or not
* whether there are other options and why they have not been chosen
* what the Government plans to do about any part of the intervention that has not yet been decided upon.
This section assesses the clarity of evidence used and its role in justifying the chosen intervention (SO, CAN YOU SEE WHAT EVIDENCE HAS BEEN USED AND THE ROLE IT HAS PLAYED?).
| Level 0 |
Level 1 |
Level 2 |
Level 3 |
WORKED EXAMPLE |
| Not clearly enough for level 1. |
Evidence is mentioned, with some explanation of how it has been used. |
As in level 1 but the supporting evidence is linked to relevant parts of the policy, properly cited and you could find the source. |
As in level 2 but the evidence base is also assessed and uncertainties and contradictory information are acknowledged. |
The Government has chosen to implement minimum unit pricing for alcohol, instead of, for example, increasing alcohol taxes or starting a new educational campaign. |
IMPLEMENTATION
How will the chosen intervention be rolled out?
The document should explain:
* why this method for delivering the intervention has been chosen
* what evidence, if any, that decision is based on
* whether there are other methods and if so the reasons for not choosing them
* if the way to deliver the intervention is still being decided, what the method is for deciding.
This section assesses the clarity of evidence used and its role in determining the implementation method (SO, CAN YOU SEE WHAT EVIDENCE HAS BEEN USED AND THE ROLE IT HAS PLAYED?).
| Level 0 |
Level 1 |
Level 2 |
Level 3 |
WORKED EXAMPLE |
| Not clearly enough for level 1. |
Evidence is mentioned, with some explanation of how it has been used. |
As in level 1 but the evidence is linked to relevant parts of the policy, properly cited and you could find the source. |
As in level 2 but the evidence base is also assessed and uncertainties and contradictory information are acknowledged. |
The Government has decided to implement minimum unit pricing through a voluntary agreement with major retailers rather than through legislation. |
VALUE FOR MONEY
This considers the costs and benefits of the policy to show why the Government thinks it is worth doing.
The document should explain:
* what the costs and benefits are estimated to be
* the assumptions behind those calculations
* what evidence is being used to make those assumptions
* the uncertainties about the costs and benefits and how likely the figures are to change.
This section assesses the clarity of evidence used and its role in the cost-benefit analysis (SO, CAN YOU SEE WHAT EVIDENCE HAS BEEN USED AND THE ROLE IT HAS PLAYED?).
| Level 0 |
Level 1 |
Level 2 |
Level 3 |
WORKED EXAMPLE |
| Not clearly enough for level 1. |
The assumptions (the basis) for conclusions about risks and benefits are described. |
As in level 1 but supporting evidence is also properly cited and you could find the source. |
As in level 2 but it is also clear how the uncertainties in these assumptions have been considered. |
The assessment shows the potential of the proposal to reduce problem drinking, but also the impacts on business (e.g. supermarkets, pubs), the public sector (e.g. police, NHS), and the public of raising prices. |
TESTING AND EVALUATION
How will we know if the policy has worked?
The document should explain:
* plans to measure the impact of the policy and the outcomes that will be measured
* plans to test the policy first, or reasons why not
* plans to evaluate the effects of the policy, including a timetable.
This section assesses the clarity of evidence used and its role in the planning for testing and evaluation (SO, CAN YOU SEE WHAT EVIDENCE HAS BEEN USED AND THE ROLE IT HAS PLAYED?).
| Level 0 |
Level 1 |
Level 2 |
Level 3 |
WORKED EXAMPLE |
| Not clearly enough for level 1. |
Some indication of success measures but no plans for testing/evaluation (or explanation of why inappropriate). |
More comprehensive success measures (or process for developing them outlined). Also provides details about use of testing and plans for evaluation or explains why testing or evaluation would not be appropriate. |
As in level 2 but explains the reasons for the use of testing and plans for evaluation. It is also clear what will happen to the results of testing and evaluation, including timing and plans for publication. |
The Government sets out how it plans to measure the results of the policy. The Government sets out plans for piloting, initial evaluation of those results and timetable for publication and then describes decision process around roll-out if the evaluation is satisfactory. |
The Institute for Government is here to act as a catalyst for better government.
The Institute for Government is an independent charity founded in 2008 to help make government more effective.
* We carry out research, look into the big governance challenges of the day and find ways to help government improve, rethink and sometimes see things differently.
* We offer unique insights and advice from experienced people who know what's like to be inside government both in the UK and overseas.
* We provide inspirational learning and development for very senior policy makers.
We do this through seminars, workshops, talks or interesting connections that invigorate and provide fresh ideas.
We are placed where senior members of all parties and the Civil Service can discuss the challenges of making government work, and where they can seek and exchange practical insights from the leading thinker practitioners, public servants, academics and opinion formers.
This framework is presented alongside further analysis in the report Show Your Workings: Assessing how government uses evidence to make policy by Jill Rutter and Jen Gold, available at:
www.instituteforgovernment.org.uk
October 2015
© Institute for Government 2015
2 Carlton Gardens
London
SW1Y 5AA
Tel: +44 (0) 20 7747 0400
Fax: +44 (0) 20 7766 0700
Email: [email protected]
The Institute is a company limited by guarantee registered in England
No. 6480524
Registered Charity No. 1123926
