Alliance for Useful Evidence

Does evaluation have any impact on policy?

31.08.2012

A new book reviews the value of evaluation to policy and practice.  It comes at a time that the National Audit Office is currently reviewing the value for money of the vast Government evaluation industry.  Is evaluation actually doing any good? 

The following is a guest blog by Professor Ceri J. Phillips, Swansea Centre for Health Economics:

The formal evaluation of public services has a history of little more than 50 years. Discovering what impact various social policies, programmes and projects have had on the intended beneficiaries makes political and economic sense. Why, one might ask, has evaluation had such a relatively limited pedigree?

Part of the explanation could perhaps be explained by the response from several medical practitioners in the 1970s and 1980s who considered the movement towards evidence-based medicine as an unwarranted assault on their professional wisdom and integrity. 

Nevertheless, in spite of initial opposition from some quarters, evidence-based medicine, with its emphasis on the randomised controlled trial as the primary, if not the sole method of producing cogent evidence, became widely accepted as the 'gold standard' on which to base professional practice.

Although academic articles and books began appearing in some numbers in the USA during the 1960s, there was little academic or political interest in formal evaluation in the UK until two decades later. It would appear that in the UK, for example, the formulation of a policy, particularly when enshrined in legislation, was deemed sufficient to ensure its full implementation and once implemented to have the intended effect.

However, it is highly probable that the movement towards evidence-based medicine impinged on the world of civil servants and politicians. Certainly with the Thatcher government in the 1980s questioning the value of the public sector in terms of its efficiency, major projects and initiatives - notably the National Health Service - came under close scrutiny. Government spending on public sector services now had to prove its cost-effectiveness.

In the UK this concern with efficiency and cost-effectiveness spawned a number of government documents directed at policy advisers. Politicians now need to know 'what works' and at what cost. This emerging culture of evidence-based policy prompts the question of how evaluation research commissioned by governments influenced or even shaped central policy.

It is on this question that our book focuses. Given the plethora of learned articles and books on the subject of evaluation over the past 50 years or so, what evidence is there that evaluation research in its many manifestations - commissioned project evaluation, policy evaluation, theory-driven evaluation - has had an impact on public policy at central and more local levels. In short, how cost-effective has evaluation research been?

The book looks at the possible reasons why academics, in particular, appear somewhat sceptical, if not despondent about the outcome of their research-based findings. Those who make decisions about allocating taxpayers' money to a range of policies and their embodiment in programmes and projects, are not bound by any contractual arrangements to act on the results of evaluation research  - whether this has been designed and delivered by academics or by research-oriented private companies. 

We contend that the exploration of the impact of evaluation research on public policy is long overdue.

(The views in this blog are the author's own and not necessarily those of the Alliance for Useful Evidence)