Nesta report calls for randomized control trials to be used more widely when evaluating policies to support business growth.

A Nesta report out today argues that standard government evaluations of business support schemes aren't rigorous enough. The report, Creative Credits: A Randomized Controlled Industrial Policy Experiment, calls for better assessment of schemes to save public money at a time when departmental budgets are under great pressure.

 

The study used a method of evaluation called a randomized control trial (RCT) to see whether a popular business support scheme, in this case a voucher scheme to encourage businesses to collaborate with creative service businesses, worked effectively.

 

This pilot study was run in Manchester and began in 2009. It was funded by Nesta, Manchester City Council, the Economic and Social Research Council, the Arts and Humanities Research Council, and the North West Development Agency. The experiment was structured so that vouchers, 'Creative Credits', would be randomly allocated to Manchester SMEs applying to invest in creative projects such as developing websites, video production and creative marketing campaigns, to see if they had a real effect on innovation.

 

The research found that the firms who were awarded Creative Credits enjoyed a short-term boost in their innovation and sales growth in the six months following completion of their creative projects. However, the positive effects were not sustained and twelve months after the completion of these projects there was no longer a statistically significant difference between the groups that received the credits and those that didn't.

 

The report argues that these results would have remained hidden using the normal evaluation methods [1] used by government. The findings suggest that RCTs were used more often, it would be easier to see which types of policy really worked and less public money would be wasted.

 

Recently, there has been some move in government towards using RCTs in industrial policy, for instance the Department of Business, Innovation and Skill's (BIS) recently announced £30 million growth vouchers programme. The programme will use RCTs to test a variety of innovative approaches to helping SMEs overcome barriers to achieving growth and will run for two years.

 

Hasan Bakhshi, director of creative economy at Nesta and one of the report's co-authors, said, "We know that vast amounts of public money are spent supporting businesses. For example, the Technology Strategy Board will spend approximately £440 million in 2013/14, and the Regional Growth Fund to promote the private sector is worth over £2.7 billion over 2011-12 to 2015-16. Yet, the impact this spending is having is still unclear. BIS's recent announcement to use RCTs in evaluating its growth vouchers programme is welcome but this is still only the exception rather than the rule." 

 

Creative Credits:A Randomized Controlled Industrial Policy Experiment was funded by Nesta, ESRC, AHRC and Manchester City Council.

 

-Ends-

 

Notes to editors:

 

For media enquiries please contact Natalie Hodgson on 020 7438 2614 or [email protected]

 

About the Partners:

 

About Nesta : Nesta is the UK's innovation foundation. 

 

About AHRC: The Arts and Humanities Research Council (AHRC) funds world-class, independent researchers in a wide range of subjects: ancient history, modern dance, archaeology, digital content, philosophy, English literature, design, the creative and performing arts, and much more. www.ahrc.ac.uk 

 

About ESRC: The Economic and Social Research Council is the UK's largest organisation for funding research on economic and social issues. 

 

North West Development Agency: The North West Development Agency was the regionaldevelopment agency for the North West England region and was a non-departmental public body. It was abolished on 31 March 2012.

 

Manchester City Council

 

[1] Traditional evaluation methods consist of asking businesses that have received support whether they have found it to be of use, but the report argues that businesses may have incentives to say yes even if this is not the case if they perceive this as increasing their chances of further support.