The case for an international clearing house for what works
Many governments – and people – around the world are wrestling with similar problems. How to improve education; how to boost growth; how to cut crime; how to make people healthier and happier.
To help them, at least some of the world’s evidence is now organised in a truly global way. In health, the Cochrane Collaboration collates what’s been learned from trials and experiments for use by anyone anywhere. The recently created What Works centres and evidence ‘clearing houses’ review studies from across the world. Their toolkits, and the intervention studies on which they are built, are a classic public good: they impose costs on those who produce them, but of are of benefit to all.
Over the next few years we will certainly need more of these – and in particular more attention to getting evidence used as well as created, whether by policy makers or practitioners.
It would be more sensible and cost effective, if countries, cities, and professional bodies, clubbed together to commission experiments and systematic reviews
But we also need to address how new knowledge is funded. The ways in which evidence is funded haven’t kept up. The great majority of reviews, research and evaluations are commissioned at a national or even a local level by a single client, alongside some funded by global institutions like the World Bank.
This is clearly inefficient.
We think a new approach is needed, both to save money and duplication, and to improve the design of evaluations. Put simply, it would be more sensible and cost effective, if countries, cities, and professional bodies, clubbed together to commission experiments and systematic reviews.
To some extent this already happens. The OECD, the World Bank and European Joint Research Centre could be thought of as cross-national What Works institutes, building and collating evidence. Cities have in the past collaborated (privately) on things like transport planning. And Nesta’s Innovation Growth Lab brings together a dozen governments and foundations to jointly sponsor formal experiments on how best to support growing businesses and innovation, using control groups and rigorous evaluation.
But these are the exceptions not the rule, and there is remarkably little collaboration between governments facing similar problems.
A simple solution
We suggest a simple solution – a platform that would make it easier for governments, municipalities and others to find potential partners in funding and research. The model would work like Kickstarter and similar crowdfunding platforms.
Let’s say the government of a medium sized country wants to find out about what really works in reducing isolation amongst older people. It would search the platform for existing reviews, but where the government was disappointed with what it found, it would post its interest – along with rough details of how much it is willing to spend, and the desired timescale.
Members of the platform would have already signed up indicating broad areas of interest – and so at this point emails might be sent to the several dozen agencies or departments with a related interest, for example in ageing. Unless partners opted out, the platform would also record ‘fails’ – searches and subjects where no result was found. This would enable governments, foundations, and reviewers to identify key evidence gaps – and who might come together to plug them.
Discussion would often follow: for example pointing out similar studies that have already been carried out, or refining how the question is to be framed. A useful step at this stage would be to map how existing interventions fit on the standards of evidence framework.
Having done so, and usually found just how much evidence is missing, a handful of willing parties could more easily pool their resources. One would generally take the lead in the commissioning and quality assurance – or perhaps an intermediary such as an expanded Campbell Collaboration - but would ensure that the research was done in a way that met the needs of the other partners.
The platform wouldn’t need all the formality of a Kickstarter – though it might benefit from it (in the sense that money which is committed is only drawn down if other parties come on board within a specified period). It could be more informal, or like Nesta’s Innovation Growth Lab which pools together money on a rolling basis.
The platform could also allow academic groups – or What Works centres – to signal projects or reviews that they plan to or would like to do, thereby inviting interest and funding from a wider source of government and foundation funders.
The online version wouldn’t replace human relationships. On their own, websites can’t solve the problems of collaboration. People still need to do that. So sitting alongside the platform there would need to be effective brokers and fixers, nurturing partnerships of funders.
But the promise would be to significantly enhance the scale and quality of evidence generation on issues that are pressing and important for governments in many countries, while at the same time lowering the cost of individual reviews and studies to commissioners.
Of course there will be complexities in how evidence is generated and interpreted. It’s rarely possible to prove that something will work all the time and everywhere. Evidence has to be built on – to understand not just what works, but also when, where, for who and with whose support it really works.
But an international platform of this kind would represent a major step forward towards a more sane, rational and efficient way of understanding how government can succeed more and fail less.