Innovation in Public Sector Organisations A pilot survey for measuring innovation across the public sector Alastair Hughes, Kyla Moore and Nimesh Kataria

Foreword

The need for us to deliver public services in new, better and cheaper ways will come as no surprise. The combination of straightened public finances with major social challenges mean that public services need to become more productive and develop new ways of working. Innovation in the public sector is therefore a pressing task, hindered, not least by the lack of available data on activity and performance.

Measurement has an important role to play in this task. Measuring innovation has played an important role in encouraging innovation in the wider economy, the Lisbon R&D target being a prime example. This research stems from a desire to create a tool that will play the same role for the UK's public services.

It should be stressed that this work is preliminary. There is scope to develop the measurement of public sector innovation in a number of ways, including by building on the ground-breaking efforts of the Office for National Statistics to measure public-sector productivity and to draw on the complementary work of the Organisation for Economic Cooperation and Development.

Although a pilot project, we believe that the findings of this pilot provide useful insights into both how innovation is happening in parts of the public sector and the factors that enable it.

As always, we welcome your comments.

Stian Westlake Executive Director of Policy and Research, NESTA

March, 2011

NESTA is the UK's foremost independent expert on how innovation can solve some of the country's major economic and social challenges. Its work is enabled by an endowment, funded by the National Lottery, and it operates at no cost to the government or taxpayer.

NESTA is a world leader in its field and carries out its work through a blend of experimental programmes, analytical research and investment in early-stage companies. www.nesta.org.uk

Part 1: The development of the public sector innovation index – an important component of NESTA's Innovation Index

1.1 There is a pressing need for innovation in the public sector and the index should enable this

This study details a pilot approach to measuring innovation in the public sector and is a part of NESTA's Innovation Index programme, a commitment under the UK Government's White paper in 2008: 'Innovation Nation'.

Since NESTA started work on the Innovation Index in March 2008, several factors have led to a need for much greater levels of innovation across the public sector. First, the Comprehensive Spending Review in October 2010 confirmed the expected dramatic reductions in public sector funding through the next four years and beyond. Second, continually rising expectations have created a need for much greater impact of remaining spend on public services – especially in a time of significant budget reductions – well beyond that of simply 'minimising the impact' of the cuts. Accordingly, new approaches to delivering public services are being considered which embrace new models of user engagement, local accountability and citizen responsibility. ¹ Finally, short-term changes in public service provision are occurring against a backdrop of long-term expensive social challenges including, for example, the impacts of an ageing and growing population.²

Furthermore, measuring innovation in the public sector is difficult. The impacts of successful innovations in the public sector tend not to be reflected in immediate financial outputs. This, combined with the nature and diversity of public sector organisations and services, makes measuring these innovations extremely challenging.

These factors create the need for a public sector innovation index that goes beyond existing measures of national public sector service delivery and productivity ³ and enables more effective consideration of how to improve innovation across the public sector. ⁴ Such an index should achieve this by:

• Helping policymakers and researchers understand levels of innovation and different drivers within different parts of the public sector.
• Helping delivery organisations understand their levels of innovation, performance and their underpinning capabilities. As such, the index should also be a practical enabler for public sector managers to take action accordingly.

Our pilot study has investigated the practical insights that can be gained through applying a survey-based tool targeting leaders in public sector organisations responsible for delivering innovation and improvements in public services.

1.2 Our pilot study has found that a survey-based index can provide practical insight in to innovation in the public sector

The project developed and piloted a survey-based approach to measuring innovation in the public sector, with participation based on a voluntary, or self-selected, approach. We developed and tested this approach during the summer and autumn of 2010, across two parts of the public sector: the NHS and Local Government. The findings reflect the sample of organisations surveyed, which are sub-sets of the overall population of organisations in these two parts of the public sector.

The feedback that we have received so far from potential users of the index, including survey participants and policymakers, suggests that the index tool can provide practical insight into innovation in these parts of the public sector. These insights are regarding the levels and types of innovation that are happening, the levels of underpinning capabilities and the sector conditions for innovation. These insights are contained in the main body of the report, from Section 2.

The Index has been designed to provide this insight through achievement of the following two objectives, in order of priority:⁵

1 Accuracy: that is, to accurately reflect how innovation happens in the public sector.

2 Comparability: that is, to enable comparisons to be drawn across different parts of the public sector and also future comparisons to be drawn between public and private sectors, primarily in the UK but also across other countries.

In doing so, NESTA's broader Innovation Index objectives were considered ⁶ – specifically metrics that balance the need for longevity and ease of collection alongside the core requirements of accuracy and comparability (something innovation indicators have traditionally placed at a premium).

The index is based on a framework that reflects how innovation happens in the public sector

In order to achieve the first objective above, accuracy, the approach was based on a framework of public sector innovation developed from NESTA's Innovation Index and broader public sector innovation research. In particular, the project compared the frameworks, findings and NESTA observations of four Innovation Index exploratory project reports produced in October 2009.⁷ These reports each proposed an innovation index for public sector organisations, based on a scorecard approach. Across the different aspects of the proposed frameworks a relatively high degree of alignment of the key factors was found and the framework below developed based on this work which was tested and refined through syndication with a group of project stakeholders (described in Appendix 1).

Importantly, the framework recognises the following:

• It reflects the innovation that is important in the public sector, whilst remaining consistent with an international standard definition of innovation.⁸

"An innovation is the implementation of a new or significantly improved product (goods or service), or process, a new marketing method, or a new organisational method in business practices, workplace organisation or external relations. ⁹

• It recognises two important differences between the public and the private sector:

  1. Differences in how value is defined. Innovation in the public sector is assessed through impact on a range of social value as well as economic value indicators.
  2. Differences in the systems in which organisations operate. The framework reflects that public sector organisations operate in a range of different systems and assesses the impact of the system conditions on innovation in organisations.

• The public sector innovation framework is shown in Figure 1.¹⁰ The coloured areas within the circle represent those aspects that are within the control of the organisation: these are 'Innovation Capability', which underpins an organisation's 'Innovation Activity', which 'Impacts on Performance'. The area outside, 'Wider Sector Conditions for Innovation', represents those aspects that are outside of the control of the organisation, but within the control of policymakers or other sector bodies of strategic influence. These conditions describe how the system, in which an organisation operates, helps or hinders innovation – that is, how the system impacts an organisation's innovation activity and capability. A consideration of these factors also allows comparisons to be made, and lessons drawn, across different systems.

It should be noted that none of the relevant current international survey instruments, in particular the Community Innovation Survey for measuring innovation within businesses ¹¹ and the MEPIN Survey public sector project, which is being piloted across the NORDIC countries, ¹² sufficiently accurately reflected the framework above to be used to achieve the Index objectives. Therefore a new public sector survey instrument aligned with the above framework was developed.

The survey instrument also enables comparisons to be made of innovation across different parts of the public sector and the private sector

In translating the framework into an appropriate survey instrument, we also sought to achieve the second objective mentioned above – comparability. This was achieved by developing a survey instrument that:

• Is based on the same definition of innovation ¹³ as NESTA's Private Sector Index Survey and the above-mentioned international survey instruments: these being the MEPIN Survey and the Community Innovation Survey.
• Where possible, aligns first with NESTA's Private Sector Index survey questionnaire ¹⁴ (through many of the questions contained within) and also the international survey instruments mentioned above.

Comparability of results is also dependent on how the survey instrument is applied. ¹⁵ Ideally, any application of the survey instrument across different types of public sector organisations should be consistent with the common guidance ¹⁶ which is used by the above-mentioned survey instruments.

Please note that at the time of writing, these guidelines are currently being revisited. ¹⁷ In the meantime, we add a note of caution to comparisons made across the participating organisations contained in the survey findings. Any comparisons will depend on the types of organisations surveyed (the different types of local authorities and the NHS Trusts), their different activities, their different sizes and their comparability as enterprises. ¹⁸

The survey results were analysed through indices (each based on a scale of 0-100 per cent) which are aligned with the framework above. A description of the index methodology adopted can be found in Appendix 3.

Accordingly, the public sector innovation indices cover the following four areas:

Future comparability with the above-mentioned private sector indices is enabled by providing the results of the survey in the form of a dataset that allows this future research to be conducted. However, please note that the results of the survey (contained in the following sections of this report) should not be compared directly with private sector indices such as NESTA's private sector index. ¹⁹ This is because although there is some commonality of the underpinning survey data gathered, the datasets are not wholly directly comparable; nor are the innovation indices we used (as they are designed to accurately reflect the framework of public sector innovation shown above). Finally, any comparisons across private and public sector organisations should also be made with reference to the previously mentioned guidelines regarding the definition of comparable enterprises. ²⁰

Our pilot application of the survey suggests that meaningful results can be achieved based on levels of voluntary participation

In piloting the survey, the aim was to test whether meaningful results (given the anticipated sample size and budget) for the users of the survey could be achieved and also to maximise learning for the application of a survey-based approach.

Two sectors, the NHS and Local Government, were selected based on prior indications of demand, potential participation and also the ability to draw meaningful sector insights from an anticipated sample size of up to 100 interviewees per sector. Support for the study was gained from both the Department of Health and the Department of Communities and Local Government, and the project team worked with the respective policy teams to agree the best approach to launching the survey and securing participation. ²¹

In each case we were seeking to survey, on a voluntary basis, one senior person per organisation (Local Authority, or NHS Trust) responsible for innovation or improvement activities. Each interviewee would be commenting on behalf of the organisation as a whole and, in return, received an individual organisation scorecard along with the overall findings.

The survey was promoted in different ways. In the case of Local Government, a joint letter on behalf of Irene Lucas the acting Permanent Secretary of the Department of Communities and Local Government and Jonathan Kestenbaum the Chief Executive of NESTA was sent to all Local Authority Chief Executives and Deputy Chief Executives, requesting their organisation's voluntary participation in the study. They were asked to forward the request to the most appropriate person in their organisation responsible for innovation and improvement. In the case of the NHS, the survey was announced and disseminated to NHS Trusts through the ten Strategic Health Authorities (SHAs), specifically through the network of SHA Innovation Leads.

The majority of respondents were in senior and cross-organisation roles. The breakdown of the types of participating organisations and interviewees is shown below: ²²

The pilot survey was conducted in the two month period during the summer of 2010, from 21st July to 24th September. This was in the period following the General Election (May 6th) and just after the publishing of the NHS White Paper ²³ (12th July).

The survey itself was conducted by Ipsos MORI using a structured Computer Assisted Telephone Interview (CATI) approach which was selected (instead of an online alternative) because of the ability to maximise benefits and minimise costs for this pilot study. In particular it was considered to be a better way of achieving the required sample size, high quality data and a better experience for respondents. Each interview took approximately 37 minutes on average.

In order to demonstrate the potential of the tool at a national level the results were analysed in three ways.

1 Through a 'sector drill-down' approach examining the index scores across the two surveyed sectors. In doing so, we tested hypotheses that innovation might differ across:

• The two sectors participating in the survey.
• The different types of organisation in each sector (types of NHS Trust; types of local authority).
• The different sizes of participating organisation.
• The different Strategic Health Authority regions (for participating NHS Trusts).
• Those organisations with innovation strategies and those without.

2 In addition to the above, we also analysed findings through the different aspects of the framework – at a thematic level. This was first as a way to understand and add richness to any patterns identified above, but also to explore the potential of the tool to explore important detailed aspects of innovation across the participating public sector organisations.

3 Finally, we also sought to test the usefulness of the tool by providing examples of how the index findings can be combined with other potential data sets to demonstrate broader policy applications.

In order to demonstrate the potential of the tool for participating organisations, we provided each organisation with their individual scorecard benchmarking their survey results with peer organisations, along with guidance for interpretation and a request for feedback. ²⁴ Please see Appendix 4 for sample scorecards.

Part 2: Pilot findings: what the survey tool tells us about how innovation is happening in the public sector

The following three sections describe the findings from the pilot application of the survey tool across the NHS and Local Authorities. In this section (Part 2) we describe what the results tell us about how innovation is happening and the lessons that can be learned. The following section (Part 3) then contains additional survey findings for the participating public sector organisations. Part 4 contains additional survey findings for policymakers, researchers and others who have roles improving the conditions for innovation in the public sector.

The findings are of course the results of a pilot, whose primary aim was to test and develop a survey tool. They should not be interpreted as a thorough diagnosis of innovation across the public sector as a whole. As mentioned earlier, differences in the institutional levels surveyed mean that these findings should not be used as a basis for comparison with those from private sector studies.

2.1 The overall index scores suggest that innovation is stifled: the key opportunity to improve innovation is the conditions in which organisations operate

In analysing the index scores ²⁵ the most useful insights can be drawn from the relative differences between the innovation indicators. Figure 2 compares the index scores across the impact, activity, capability and conditions indicators for all surveyed organisations.

These results suggest that the surveyed organisations rated their activity levels highest relative to other aspects of innovation. The conditions for innovation were rated the lowest and could potentially provide the greatest opportunity to improve innovation in the surveyed organisations.

The results also suggest slight, but consistent differences across the two sectors surveyed. Potential reasons discussed with policymakers in the two sectors included that innovation has been an explicit part of the national, regional and local NHS strategy through the Quality, Innovation Productivity and Prevention agenda (QIPP) ²⁶ underpinned by each regional Strategic Health Authority having a Regional Innovation Fund to stimulate the invention, implementation and spread of innovative ideas in their area as well as a statutory duty to innovate. ²⁷

Another reason discussed for the differences in survey scores across the two sectors are potential differences in attitudes (and hence responses) to the national use of self-assessment processes.

Drilling down in to the above indices suggests the following particular insights across three of the four indicators: impact, activity and conditions:

2.2 The index helps understand the focus of surveyed organisations' innovation efforts – survey results suggest that efficiency and localism will be the key focus

The data generated from the survey help provide insight in to the levels of different types of innovation activity across the surveyed organisations (Box 2).

Our analysis of the examples provided by interviewees (see Appendix 5) supports the spread of types of innovation activity (see A in Box 2) but also suggests a higher degree of commonality of innovation than the perceptions provided (see B in Box 2). Discussion of this with sector representatives suggested that the above perceptions could be due to organisations operating more in isolation than otherwise perceived.

The survey results suggest that the intended impact of innovation is changing across surveyed organisations (Box 3). ²⁸

Innovation Impacts – NHS

In the NHS, the picture is similar. Over the last three years, the focus has been on providing the same services better, improving access, or improving efficiency (C1,C2).

Looking ahead, over the next two years, we expect the localism agenda to also have a marked impact on innovation in the NHS (D). The results also suggest that efficiency and cost reduction will remain key drivers.

2.3 The Index helps understand how innovation is happening across the surveyed organisations – mechanisms to encourage effective sharing of ideas should be improved

As mentioned so far, overall activity indices are the highest relative to other aspects of organisational innovation. This is underpinned by relatively high levels of different types of innovation activities (Figure 3). However, levels of reported activity in 'accessing new ideas' is relatively low, especially in Local Government.

Our survey results also provide insight into how surveyed organisations are accessing new ideas. This is an important measure of the degree of open innovation that can enable more effective innovation. Survey results (across both sectors) suggest that most ideas are sourced from outside the organisation (Figure 4). The individual sources for these are examined in more detail in Box 4.

---

Footnotes:

Participants stated that access to best practice information is the key external source of ideas (A).

Organisations ranked different sources of ideas and information used (to develop new or improved services or processes) as shown. Please note the relative importance of clinical journals highlighted by NHS representatives.

Source: Survey questions 22a-f.

Within organisations, frontline staff and service users are viewed as the most important sources of ideas (B).

Please also note the differences across the two sectors of perceptions of the relative importance of managerial staff. Partner organisations are also viewed as important.

Source: Survey questions 21a,d,e,f,g,h.

Box 4: Sources of new ideas

Indeed, of all the key enablers of innovation surveyed, access to best practice information is viewed as the most important.

Figure 5 below shows surveyed views regarding the importance of key enablers of innovation across the two sectors. The key important enabler is access to best practice information from other similar organisations. Central guidance, when managed well, is also well-received.

Source: Survey questions 22b,c,d,e, 42c,e,f,i.

Despite this importance of best practice information from similar organisations, mechanisms to encourage effective sharing of ideas can be improved (Figure 6). The survey findings suggest that the organisations recognise that they are less effective in sharing best practice which may hinder wider diffusion of new ideas.

Source: Survey questions 15c, 22b.

2.4 The index helps analyse the incentives for organisations to innovate – the key incentive is customer feedback and competition is yet to take full effect as an incentive

The greatest potential opportunities to improve the conditions for organisations to innovate are their incentives and autonomy (Figure 7).

The index explores the impact of different incentives: customer feedback, transparency of performance, demand for new services and competition from alternative providers (Box 5).

Box 5: Incentives to innovate

The survey results suggest that incentives can be improved – not everyone surveyed considered that there are incentives, as a whole, to be innovative.

Source: Survey question 43h.

The survey results suggest that the key incentive for organisations to innovate is customer feedback.

Source: Survey questions 42a, 42b, 42d, 42h.

Competition is reported to exist but is not the key incentive for innovation.

In the NHS, competition from alternative providers is the weakest of the incentives surveyed (Box 5).

However, organisational views on levels of competition are significantly higher in the NHS than in Local Government (Figure 8). This suggests that although competition is reported to exist in the NHS, it is yet to fully take effect as an incentive for innovation.

Source: Survey question 49.

The survey results suggest that the impact of regulation as a barrier or as a driver for innovation differs across the two sectors (Box 6).

Box 6: Regulation and innovation

In Local Government, regulation is viewed as a double-edged sword, acting as both a barrier and a driver of innovation.

In the NHS, regulation is generally viewed as more of a driver of innovation.

Source: Survey questions 38f, 43a.

2.5 The Index helps understand the impact of autonomy – the pilot study has identified lessons from the NHS model of Foundation Trust development

The highest potential opportunities to improve the conditions for organisations to innovate are their incentives and autonomy (Figure 9).

The index explores the impact of different aspects of autonomy.

The survey results suggest that NHS Foundation Trusts (FTs) are more innovative (across all indicators) than non-Foundation Trusts (non-FTs) (Figure 10a and 10b). ¹

The results suggest that the biggest differences are in the conditions for Foundation Trusts to be innovative (incentives, autonomy, leadership and culture, and enablers) and that they have greater capability to innovate (management of innovation, leadership and culture, organisational enablers). However, the relative differences in conditions and capabilities for Foundation Trusts to innovate appears to not yet be fully translating into their innovation activities and impact on performance.

The survey results suggest that the key differences in sector conditions between Foundation Trusts and non-Foundation Trusts are their relative autonomy and access to enablers of innovation (Figure 11).

Please also note that the survey results suggest that there is very little difference in the incentives for Foundation Trusts to innovate, compared to non-Foundation Trusts and no difference in the impact of sector leadership and culture.

The different aspects of autonomy show consistent differences. Please also note the profile across the different aspects of autonomy explored through the index.

A comparison of access to different enablers of innovation suggests that there are key differences across access to comparable performance data, best practice information and innovation funds and support. Note also the consistently relatively inadequate IT systems.

Finally, the biggest differences in the 'capability' to innovate between Foundation Trusts and non-Foundation Trusts are the 'leadership and culture' indicators (particularly 'vision and spirit of senior managers' and 'attitudes to risk taking and learning').

Discussion of the above results with health sector representatives suggests that the above differences could possibly be explained by the attitudes and capabilities developed as non-Foundation Trusts progress through the Foundation Trust accreditation process.

The above results tentatively suggest that there are lessons from the NHS model of Foundation Trust development that could possibly be applied elsewhere in the public sector as well as lessons on how the impact of Foundation Trust development can be improved within the NHS.

Part 3: Pilot findings: what the survey tool tells us about how public sector organisations are managing innovation

This section contains additional key pilot survey findings for the participating public sector organisations.

3.1 Those organisations with innovation strategies have consistently higher innovation indices

The survey results suggest that, overall, those participating organisations with innovation strategies are more innovative (across all indicators) than those organisations without (Figures 14a and 14b).

For those organisations with innovation strategies, capability indicators in particular are all higher. This is mostly due to higher 'leadership and culture' factors (including the prioritisation of innovation) but also capabilities in the 'management of innovation' (Figure 15).

Perceptions of the wider sector conditions for innovation (of the surveyed organisations with innovation strategies) is higher – due to reported relatively high levels of incentives and autonomy compared with those organisations without innovation strategies (Figure 16).

Please note that the 'activity' indicators are also higher for the group of surveyed organisations with innovation strategies, which suggests that innovation capability is impacting levels of actual innovation activity in organisations.

The rest of this section shows what the pilot survey results tell us about the surveyed organisations' capabilities to innovate: how well they are managing and leading innovation in their organisations.

3.2 However there is a lack of systematic approach to managing innovation across the surveyed organisations

Thirty-seven per cent of surveyed organisations (28 per cent of Local Authorities and 53 per cent of NHS Trusts) reported having an innovation strategy as part of the overall strategy of the organisation. ²

Sector representatives suggested that the higher numbers of NHS Trusts reporting having an innovation strategy could be due to innovation being an explicit part of the national, regional and local NHS strategy through the Quality, Innovation Productivity and Prevention agenda (QIPP). ³

Low levels of innovation strategies are underpinned by a lack of 'management indicators' – particularly the existence of specific funding and objectives for innovation (Figures 17a and 17b).

Source: Survey questions 38a, 38b.

Surveyed participants also showed a lack of awareness of levels of different types of investment in innovation activities (Figures 18a and 18b).

Source: Survey question 37.

Though they did have perceptions of spend relative to their sector (Figures 19a and 19b).

Source: Survey questions 14a,b,c,d,e.

Similarly, whilst self-assessments of governance effectiveness seem encouraging (Figures 20a and 20b).

Source: Survey questions 24a,b,c,d,e,f,g,h.

The results of more objective governance indicators surveyed are mixed (Figures 21a and 21b).

Source: Survey questions 23b, 23c, 34, 38c.

3.3 Leaders are expected to create the conditions for teams to innovate

The survey findings suggest that senior leadership is expected to drive innovation and that these teams are viewed as relatively stable (Figures 22a and 22b).

Source: Survey questions 39d, 39e.

Survey results suggest that managers give high priority to developing new ways of working, and supporting the trial and error testing of new ideas. Also, that staff understand the value of innovation (Figures 23a and 23b).

Source: Survey questions 23f, 39a, 39b, 40a.

Managers appear to pay high levels of attention to the views of front line staff, middle managers and service users (Figure 24).

And when it comes to developing ideas, the most important people to involve changes slightly (Figure 25).

Source: Survey questions 21a,d,e,f,g,h.

Source: Survey questions 30a,b,d,e,f,g.

However, it is useful for organisations to validate survey findings alongside other cultural indicators. As an example, for the NHS, the staff survey results are less favourable (Figure 26).

Source: NHS Staff Survey 2009, Questions 16c, 22 a-c.

Over three-quarters of surveyed organisations stated that they conducted pilot testing for significantly improved service in the last three years and almost all used multi-disciplinary project teams from different departments. ⁴ However, the survey results suggest that project teams can be given more freedom to solve problems (Figures 27a and 27b).

Source: Survey questions 28c, 39g

The survey results also suggest that funding, skills and time allocated for innovation projects can be improved (Figures 28a and 28b).

Source: Survey questions 24f, 28g, 28h

3.4 Critical organisational enablers of innovation should be recognised and improved where necessary

Critical organisational enablers of innovation are management information, connectedness (internally and externally), access to support and skills, the use of incentives and rewards and the quality of ICT (information and communications technology) infrastructure. The survey findings suggest that some aspects are better than others.

The use of management information appears to be effective (Figures 29a and 29b).

Source: Survey questions 12a, 12b, 23a.

A range of diffusion methods are used which are important aspects of internal connectedness (Figure 30).

Source: Survey questions 31a,b,c,d,e,f,g.

---

External connectedness is important because the majority of ideas are sourced externally (all surveyed organisations) (Figure 31).

However despite high levels of use of best practice information (from similar organisations), sharing of ideas can be improved (Figure 32).

Access to external support is good from a range of bodies (Figure 33), although sector representatives commented that over-reliance can be risky.

However, access to funding, as well as the use of team-based incentives and rewards, can be improved within organisations (Figures 34a and 34b).

Part 4: Pilot findings: additional findings for policymakers and researchers

This section contains additional survey findings for policymakers, researchers and others who have roles improving the conditions for innovation in the public sector.

4.1 Additional findings for local authorities – the index findings suggest potential differences across types of authorities

Analysis of the innovation indices across the different types of participating local authorities suggests that the biggest differences might exist between participating London Boroughs and Metropolitan District Authorities, with Metropolitan District Authorities showing higher relative levels of innovation indices, in particular across impact and activity indicators (Figure 35).

The key difference in the reported impact of innovation is due to differences in the improvements in indicators of organisational performance and service evaluation (Figure 36).

The key difference in the reported innovation activity is due to differences in the levels of implementation (Figure 37).

Please also note that the activity indicators are consistently among the highest for the surveyed Metropolitan District Authorities.

The feedback received from Local Government policy officials and sector representatives was that it would be particularly helpful to be able to compare the innovation index results with other data sets. Examples of how this activity might be performed separately are presented in section 4.4.²⁹

4.2 Additional findings from the NHS – the index findings suggest that there are useful lessons from Mental Health Trusts that could be applied elsewhere in the public sector

A comparison of the innovation indices across the different types of participating NHS Trusts, suggests that Mental Health Trusts seem to be relatively more innovative than other types (Figure 38).³⁰

The results suggest that the biggest differences are the capabilities and the conditions in which Mental Health Trusts operate, relative to others.

Capability indices were consistently higher: management of innovation, leadership and culture, and organisational enablers.

Some aspects of the conditions for Mental Health Trusts scored more highly: leadership and culture, and autonomy.

Note: all the Mental Health Trusts surveyed had a new or significantly changed strategy in the last three years (this context influenced the rating for the impact of innovation).

Feedback so far has strongly re-inforced:

• The relative freedoms that Mental Health Trusts have together with similarly high degrees of staff autonomy.
• The close involvement of patients in service delivery.
• Collaborative ways of working that are cross-organisational and multidisciplinary.
• Professional and managerial philosophy including less hierarchical levels.

These observations are supported by independent staff survey findings (mentioned earlier) relating to Mental Health Trusts.

Figure 38 also suggests that the conditions for innovation are least favourable for Ambulance Trusts (please note that we interviewed three out of 12 Ambulance Trusts). The scorecard results confirm that the associated 'conditions' innovation indicators are indeed consistently lower compared to other types of trust.

Finally, as well as comparing innovation indices across different types of participating NHS Trusts, we also compared indices across regions.

Figure 39 shows that we found no major differences between the participating trusts across the Strategic Health Authority regions.

4.3 Final additional findings for policymakers – the index findings suggest that short-termism and restructuring hinder organisations from innovating

The survey findings suggest that a short-term focus on rapid return on investment hinders organisations' ability to innovate.

The charts below suggest that despite good wider leadership and cultural indicators supporting innovation, a short-term focus hinders ability to innovate. This suggests that certainty about medium-term organisational strategic objectives might be required – particularly in the NHS (Figure 40a and 40b).

The survey findings also suggest that although different attitudes to re-structuring exist across the two surveyed sectors (which could partly be due to the timing of the survey for NHS participants³¹) the majority of surveyed participants considered that re-structuring hinders more than helps their ability to innovate.

Figures 41a and 41b show that almost three-quarters of NHS participants and a third of Local Authority participants considered that restructuring hinders their ability to innovate. Another third of Local Authority participants also stated that restructuring neither helps nor hinders their ability to innovate.

The survey findings also suggest that there are not significant differences in innovation indices across the sizes of the surveyed organisations.

Figure 42 shows that any differences due to the size of the surveyed organisations are relatively minor. The results suggest that the relatively large organisations surveyed have slightly higher levels of innovation capability. This is due to consistently marginally higher indices across all aspects of innovation capability.³² This suggests that these larger organisations have a marginally more systematic approach to innovation.

Please note that although conditions for innovation are broadly similar for all sizes of public sector organisations, 'Leadership and culture' conditions are more supportive in the relatively smaller organisations.³³ In particular, this is due to 'attitudes to collaboration/working across organisational boundaries'.

4.4 Examples of future application of the Index as a tool for policy development

In this section we provide examples of how the findings from the Index can form useful ingredients, when combined with other data, of the policy development process. Please note that these are examples of how the survey findings might be used and the presentation of the results themselves should not be used for other purposes or be interpreted as causal relationships between the framework dimensions and other factors.

Example 1 - Exploring variations in innovation across Local Authorities relative to the number and range of services provided

As people involved (from policymakers and researchers to leaders of local government organisations) in considering new types of service providers or carving out provision of some services, they might seek to consider the impact of the number and range of services provided on the innovation in the affected organisations. For example, by focusing on fewer numbers or a smaller range of services, are the providers of these services more likely to be able to innovate?

Table 2 shows how the innovation indices for the surveyed Local Authorities could contribute to such wider questions. The data could begin to be analysed by comparing to the number and range of services provided.

The services provided across the different types of Local Authorities can be broken down as shown below.

Figure 43a compares the innovation indices for those authorities with a relatively higher number and range of services. Figure 43b compares the innovation indices for those authorities with a relatively lower number and range of services.

Such results suggest that there could be greater variability in organisational innovation where there are a greater number or range of services provided. In turn, this could suggest that the risk or opportunity for innovation varies depending on the number or range of services a Local Authority provides.

Example 2 - Exploring variations in innovation across Local Authorities relative to levels of deprivation

An important policy question could consider how innovation varies according to patterns of deprivation across the country: policymakers might seek to identify opportunities where innovation activity could be better targeted and if the impact of innovation is where it can be most effective.

Figures 44 and 45 provide examples of how Local Authority innovation activity and impact indices could be mapped with levels of deprivation. Similar, more detailed, analyses could be performed focusing on specific types of innovation activity from the index survey data gathered.

Innovation activity in the surveyed Local Authorities³⁴ is spread across areas of deprivation³⁵ as shown in Figure 44.

The clustering towards the top left of the chart suggests that innovation activity might currently be concentrated on Local Authority geographic areas of lower levels of deprivation.

The impact of innovation in the surveyed Local Authorities is spread across areas of deprivation as shown in Figure 45.

Similarly, the lack of clustering towards the top right of the chart suggests that innovation efforts might be better targeted on Local Authority geographic areas of higher deprivation.

As well as using such methods to provide evidence when considering a re-focusing of innovation efforts, policymakers and local government leaders might also want to ensure that existing innovation activities are having the desired impact. The innovation index data can also be used to identify where existing innovation activities could potentially be having more impact.

A comparison of the activity and impact indices, based on the survey results, is shown in Figure 46. These suggest that certain local authorities (in the top left quadrant) could possibly be achieving greater impact from their innovation activities, whilst lessons could be learned from others (in the bottom right quadrant).

Example 3 - Exploring variations in innovation activity across surveyed organisations relative to available performance data

Those people seeking to explore the relationship between innovation activity and impact on independent performance assessments might seek to understand the available data sets and the corresponding insights and limitations such as through the examples below.

We found that the ability to compare the survey findings with available performance data is limited. This could be because innovation is an indicator of future, not current, performance.

Figure 47 compares the (2010) innovation survey results for Local Authorities with (2008) performance improvement data which is available for 48 of the 111 surveyed Authorities.³⁶

The chart does not suggest a relationship between historic performance improvement and current levels of innovation.

The examples below are for the NHS.

Figures 48 and 49 compare the (2010) innovation survey results for NHS Trusts with recent (07/08 to 08/09) performance improvement data.³⁷

At most, they hint that those organisations with worsening or improving performance show greater levels of innovation activity and impact. Those with the same performance show slightly lower levels of innovation activity and impact. This suggests that the impact of recent performance improvement on organisational innovation might be useful to explore further.

Part 5: The lessons and conclusions from our pilot study

5.1 Lessons cover the limitations of a telephone-based survey tool

There are three key lessons that we have taken from the experience of piloting the survey tool using a computer assisted telephone interview approach. This is based on feedback received from survey participants (via email and a small number of participant interviews) and conversations to discuss the tool and the survey findings with stakeholders across the surveyed sectors as well as the broader public sector innovation policy and research community.

1The first lesson is the need to address engagement and accuracy limitations inherent in a telephone interview-based approach. In particular, we found that this approach limited interviewees' ability to access organisational data (and hence they answered questions based on individual perceptions rather than actual organisational evidence). Whilst access to a range of organisational data or metrics would be helpful across many parts of the framework, it would be particularly helpful in the following ways:

1. Access to data would improve interviewees' ability to comment accurately on the performance impact of their innovation efforts.
2. Access to data would improve interviewees' ability to comment on the scale of innovation activity (the number, size, types and expected impact of their innovation projects).
3. Access to data would improve interviewees' ability to draw on innovation-related capability data, such as staff survey data or other cultural indicators that they might have available.

It was found that, in some cases, participants wanted to be able to conduct the survey in small groups – to discuss the questions and their responses. As such, they would welcome a tool that has the flexibility to allow them to do this.

Similarly, we also found that participants preferred to receive their benchmark scorecard findings immediately and to have access to support (through external assistance) to analyse these – to help understand the scorecard findings, plan deeper organisational analysis and integrate with broader sector-specific tools and support that might be available.

This broader suggestion – for sector-specific access and integration of the Index tool alongside other related tools and support – was echoed by policymakers.

2The second lesson is the relevance of repeatability. This is in two ways:

1. The feedback from participants suggests that they would welcome the ability to conduct repeated self-assessments as a way of monitoring the impact of their efforts to make improvements.
2. Policy researchers have commented that repeated application could improve the richness, accuracy and coverage of the research data: * Richness (for example) by providing the ability to analyse subsequent trend data. * Accuracy, by improving the level of participation in any given part of the public sector. * Coverage, by extending application across other parts of the public sector.

3The third lesson is the importance of delivery mechanisms, timing and context in securing survey participation.

There are different delivery mechanisms across Local Government and the NHS: in Local Government the survey benefited from the endorsement both from the Acting Permanent Secretary of the Department of Communities and Local Government, and also the Chief Executive of NESTA. In the NHS, we disseminated the survey through the ten Strategic Health Authority Innovation Leads.

5.2 The tool should be developed further and made available for use across the public sector

Whilst not a study of demand for a public sector index tool, nevertheless the pilot project has provided useful insight regarding the potential applications of the index tool. This insight includes the lessons learned (described above) during the pilot project, a deeper understanding of the role of an index tool and initial evidence of demand.

A survey-based index is an enabler of a more systematic approach to innovation in the public sector

An index tool, if applied well, can be an enabler or catalyst of innovation in the public sector. Within our framework of public sector innovation, the measurement of innovation is a part of the 'Wider Conditions for Innovation'. These conditions comprise incentives for innovation, autonomy structures, leadership and culture and key enablers (such as access to comparable performance data, access to best practices and the use of innovation measurement tools).

Essentially, the index can provide a practical way to engage both national decision-makers and leaders in public sector organisations on the subject of innovation in a systematic way. The index tool promotes, stimulates and enables innovation in two ways.

1As a measurement tool for national decision-makers to engage the subject of innovation. The index provides:

1. A body of research evidence to help understand which national interventions are working and which aren’t, in improving innovation in the public sector. This research base can also be used for additional analyses, including comparisons with private sector index findings, comparisons with international indices and (for surveyed sectors) comparisons with additional national datasets such as performance or capability improvement data.
2. A way of identifying where innovation is happening well and what actual innovation activity and capability underpins this. In this respect, the tool provides a lead indicator of performance improvement and also supports the identification and diffusion of good ideas.

2As an engagement tool for organisations. The tool can support the development of a systematic approach to innovation in public sector organisations, in particular by stimulating and supporting the development of innovation strategies and by providing a mechanism for ongoing monitoring.

The pilot study’s findings suggest that the majority of surveyed organisations should take a more systematic approach to managing innovation

The survey results suggest that those organisations with innovation strategies are more innovative (across all indicators) than those organisations without (Figures 50 and 51).

However, the survey results also suggest that that there is a lack of systematic approach to managing innovation across two-thirds of the surveyed organisations and that critical organisational enablers of innovation should be recognised and improved where necessary.

The index tool, if made available, can work as a critical enabler of organisational innovation by promoting, stimulating and supporting the development of innovation strategies.

It is hoped that, by publishing this report and the materials contained within (the innovation index framework, the scorecard approach and the pilot survey findings), this stimulates and helps organisations to seriously consider the development of their innovation strategies. If organisations require support as they consider this, please do not hesitate to contact us.

The pilot study has provided early indications of demand for such a tool

The pilot application of the survey shows that the index can deliver useful results based on voluntary levels of participation. Whilst these levels of participation in the survey do not necessarily indicate levels of sustainable demand for a public sector index tool, they do provide a useful indication of what can be achieved across different parts of the public sector.

The study also provided the opportunity to have conversations with a range of future customers for the index. These customers included survey participants, policymakers and researchers with a role supporting improved innovation in the public sector, and finally a broader set of observers, researchers and commentators.

These conversations have highlighted the need for a tool that can support organisational application as a benchmarking and engagement tool and also support national measurement and research purposes. Applications could include:

1. Application of the index as a widely available, NESTA-endorsed, benchmarking and engagement tool for voluntary use by public sector organisations (and groups of organisations seeking to improve collaborative innovation efforts).
2. Application as a national co-ordinated periodic survey across all public sector services (supporting, for example, the Annual Innovation Report[^43]).
3. Application of (i) and (ii) above across parts of the public sector by the corresponding government departments or other sector bodies. The use of the index could be integrated alongside other sector-specific tools or approaches.

The index tool’s potential can best be realised by development in to an online benchmarking tool

We believe that this pilot project has provided useful lessons, indications of potential demand and some of the benefits of practical application of the index tool for public sector organisations and policymakers and researchers.

We believe that the potential of the Index tool can best be realised by further development in to a highly engaging, standardised, online tool that can be made available flexibly for use by public sector organisations through a range of existing and new channels.

An online tool not only provides a good way to address the lessons learned during this pilot project, but also a practical way to optimise its use along with a fuller range of related functions. These functions could include access to analysis tools and training materials, networks of similar organisations, sign-posting to support, access to performance data and best practice information, and access to innovation funds.

The assessment is that this pilot project has stimulated serious consideration of the use of the index tool, by those people responsible for making such tools available (policymakers, researchers and other innovation-related public sector bodies) and public sector leaders responsible for improving innovation in their organisations.

Appendix 1: Acknowledgements

This report was commissioned from Ernst & Young in partnership with Ipsos MORI. Lead authors were Alastair Hughes, Kyla Moore and Nimesh Kataria from Ernst & Young.

We would like to thank the following people in particular for their guidance and support in part across the following activities: the development of the index framework and the survey instrument, the delivery of the survey, analysis of findings and co-ordination and provision of feedback.

Department for Business, Innovation and Skills

Ray Lambert, Deputy Director, and Daniel Hodges, Economic Advisor, Science and Innovation Analysis. Annabelle Simmons, Public Sector Innovation Unit, Innovation and Enterprise Directorate. Drew Pearman, Assistant Director, Business Innovation Networks.

NHS and the Department of Health

Peter Houghton, Director of Innovation, South East Coast Strategic Health Authority. George Leahy, Senior Policy Advisor, Innovation Team. Sue Nunn, Fund Advisor, Regional Innovation Funds Advisory Service. Mark Wilkinson, Director Life Sciences Innovation, Office of Life Science Delivery Board. Jonathan Wong, NHS National Innovation Centre.

Department of Communities and Local Government

Stephen Aldridge, Director for Analysis and Innovation. Ruth Marshall, Jon White, Shreena Kotecha, Innovation Team. Ben Monks (acting in the above role), Strategy Unit, Cabinet Office.

National Audit Office

Sandy Gordon, Audit Manager.

Cabinet Office

Rupert Cryer, Capability Review Team.

HM Treasury

Felicity Harvey, Director, Prime Minister's Delivery Unit.

National Endowment for Science, Technology and the Arts (NESTA)

Mike Harris, Director of Public and Social Innovation. Philip Colligan, Executive Director, The Public Services Lab. Stian Westlake, Executive Director, Policy & Research.

Ipsos MORI

Matthew Chatterton, Director. Daniel Cameron, Research Manager. Paola Ricoldo, Senior Research Executive and Statistical Analyst

Appendix 2: Public sector innovation framework

This appendix provides a detailed description of the framework of innovation in public sector organisations, which has the four parts shown.

• The coloured areas are within the control of the organisation. These areas are innovation capability, which underpins an organisation's innovation activity, which impacts on its performance.
• The area outside, 'Wider Sector Conditions for Innovation', are outside of the control of the organisation, but within the control of policymakers. These conditions can impact an organisation's innovation activity and capability.

Appendix 3: Index methodology

This appendix describes the approach to the construction of the innovation indices aligned with the framework.

Whilst the index is based on a framework that accurately reflects how innovation happens in the public sector, in translating this framework into an appropriate survey instrument we also sought (through application) to build a data set that would not only allow comparisons to be made across the different parts of the public sector surveyed, but also future comparisons to be made across different parts of the private sector.

Accordingly, the longevity of the data collected and the ease of collection was also considered – given the practical limitations of a survey tool.

In translating the framework in to an appropriate survey instrument, comparability was achieved by:

• Basing the survey on the same definition of innovation as NESTA's Private Sector Index and relevant international survey instruments: these being the Nordic Survey ⁴⁰ and the European Community Innovation Survey. ⁴¹
• Where possible, aligning the survey instrument first with NESTA's Private Sector Index survey questionnaire ⁴² (through many of the questions used) and also the international survey instruments mentioned above.

In order to achieve this, the questions from NESTA's Private Sector Index survey questionnaire ⁴³ were used as a starting point. We tailored these questions to suit the public sector innovation framework and where required added additional questions from either international survey instruments where appropriate – the Nordic Survey ⁴⁴ (which is aligned with the European Community Innovation Survey ⁴⁵) – or completely new questions. Approximately half of the questions in the survey are adapted from NESTA's Private Sector Index survey questionnaire.

Each question was allocated to the relevant sub-category of the framework (and more than one sub-category in several cases). We prioritised and re-organised the questions in order to make the survey engaging and the length reasonable, and tested the questionnaire with a sample of users (three from the NHS and four from Local Government). Where trade-offs were made, we sought to minimise impact on accuracy, when compared with the framework.

In seeking to ensure a reasonable degree of future comparability with NESTA's private sector index, there were two trade-offs that we identified. The first was that there were some parts of the public sector framework where there were relatively more data than other parts (potentially impacting relative accuracy across the associated indices). The second was that there were different scoring mechanisms used across the range of different types of questions (potentially impacting accuracy and also user engagement with the survey). The impact of these trade-offs was minimised through judicious tailoring and testing of questions and in the subsequent approach to analysing findings.

Segmentation data was added to the survey findings. For Local Authorities, this was sourced from the Local Government Association (www.lga.gov.uk) and the Chartered Institute of Public Finance and Accountancy www.cipfastats.net). For NHS organisations, this was sourced from the NHS choices website (www.nhs.uk).

Each answer corresponded to a relevant sub-category of the innovation framework, shown below.

The different approaches to scoring for each answer were harmonised by attributing each answer either '1' or 'O' to indicate whether the associated variable was having an overall positive impact ('1') on this aspect (sub-category) of innovation or not ('0') and then averaged the scores for each of these sub-categories, representing each score as an indice of 0-100 per cent.

In order to demonstrate the potential of the tool at a national level, the results were analysed in three ways.

1. Through a 'sector drill-down' approach examining the index scores across the two surveyed sectors. In doing so, we tested hypotheses that innovation might differ across:
   • The two sectors participating in the survey.
   • The different types of organisation in each sector (types of NHS Trust; types of local authority).
   • The different sizes of participating organisation.
   • The different Strategic Health Authority regions (for participating NHS Trusts).
   • Those organisations with innovation strategies and those without.
2. In addition to the above, we also analysed findings through the different aspects of the framework – at a thematic level. This was first as a way to understand and add richness to any patterns potentially identified above, but also to explore the potential of the tool to explore important detailed aspects of innovation across the participating organisations.
3. Finally, we also sought to test the usefulness of the tool by providing examples of how the index findings can be combined with other potential data sets to demonstrate broader policy applications.

In order to demonstrate the potential of the tool for participating organisations, we provided each organisation with their individual scorecard benchmarking their survey results with peer organisations, along with guidance for interpretation and a request for feedback. Please see Appendix 4 for sample scorecards.

Appendix 4: Sample scorecards

The scorecards below are real examples of individual organisational benchmark scorecards sent to each participating organisation along with guidance for interpretation and a request for feedback. Each participating organisation will also receive a copy of the overall survey findings.

The scorecard is divided into four areas to represent the main sections of our framework: Impact on Performance, Innovation Activity, Innovation Capability and Wider Sector Conditions. Each organisation was provided with their index scores according to the categories that exist within each of these four key areas.

The organisation is able to clearly identify their position according to their peers, in relation to the mean, minimum and maximum index scores, as well as the standard deviation of their relevant sample.

Local Authorities are benchmarked according to their surveyed (CIPFA) ‘nearest neighbours', ⁴⁶ as shown below:

For Innovation Activity and Impact on Performance, a red 'X' symbol is used to mark the indices. These can then be compared to the minimum, maximum and mean innovation indices of the Local Authority's benchmark group (represented by the horizontal lines). The shaded blue regions show the standard deviation around the mean score (and so help the organisation identify if any of its indices are outliers).

The organisation can also compare their Innovation Capability and their views on Wider Sector Conditions relative to their peers. This is done by distinguishing between the blue and red lines that represent the mean innovation indices and the organisation's scores respectively.

Similarly, NHS Trusts were benchmarked according to their type. For example, all Acute Trusts were provided with a scorecard that compares their results to the other Acute Trusts that participated in the survey, as shown below:

Appendix 5: Examples of innovation

The tables below provide an (anonymised) summary of the qualitative results from Question 32 of the survey, which asked participants to describe successful examples of service or process innovations within their organisation. From the examples provided, the types of activities performed and the impacts were analysed. ⁴⁷

The following table describes the 60 NHS examples provided.

The table below shows a summary, by volume, of the service and process innovation examples provided by NHS survey participants.

The following table describes the 101 Local Government examples provided.

The table below shows a summary, by volume, of the service and process innovation examples provided by Local Government survey participants.

Table A5: Examples of innovation within Local Government by impact and activity dimensions of the framework

Finally, Figures A8 and A9 below summarise the examples provided by both sectors.

Appendix 6: Innovation Index scores

This appendix contains a set of charts that describe the innovation index scores (mean innovation indices) for the participating sectors and types of organisation. The charts cover:

1. Overall index scores for the participating organisations in the two sectors. 2. NHS index scores for the different types of trust participating in the survey. 3. NHS index scores for the Foundation and non-Foundation trusts participating in the survey. 4. NHS index scores for participating trusts across the different Strategic Health Authority regions. 5. Innovation index scores for the different types of Local Authority participating in the survey. 6. Innovation index scores for the different sizes of organisations participating in the survey. 7. Innovation index scores for those organisations participating in the survey with innovation strategies and those without. 8. Innovation index scores for those NHS organisations participating in the survey with innovation strategies and those without. 9. Innovation index scores for those Local Authorities participating in the survey with innovation strategies and those without.

Each set of charts shows the overall index scores according to the four aspects of innovation surveyed (impact, activity, capability and wider sector conditions) followed by the breakdown of each individual aspect.

1. Overall index scores for the participating organisations in the two sectors

For reference, the numbers of participating organisations are shown below.

Table A6: Population and survey sample in England

2. NHS index scores for the different types of trust participating in the survey

For reference, the numbers of participating NHS organisations, by type, are shown below.

Table A7: Breakdown of NHS Trusts by type

3. NHS index scores for the Foundation and non-Foundation trusts participating in the survey

For reference, the numbers of participating NHS organisations, by type, are shown below.

Table A8: Breakdown of NHS Trusts by Foundation status

4. NHS index scores for participating trusts across the different Strategic Health Authority regions

Table A9: Breakdown of NHS Trusts by Strategic Health Authority

5. Innovation index scores for the different types of Local Authority participating in the survey

For reference, the numbers of participating Local Authorities are shown below.

This set of charts shows the index scores for the different sizes of organisations surveyed. The charts cover the four aspects of innovation surveyed, followed by the breakdown of each individual aspect: impact, activity, capability and wider sector conditions.

6. Innovation Index scores for the different sizes of organisations participating in the survey

This set of charts shows the index scores for those organisations surveyed with innovation strategies, and those without. The charts cover the four aspects of innovation surveyed, followed by the breakdown of each individual aspect: impact, activity, capability and wider sector conditions.

7. Innovation index scores for those organisations participating in the survey with innovation strategies and those without

This set of charts shows the index scores for those NHS organisations surveyed with innovation strategies, and those without. The charts cover the four aspects of innovation surveyed, followed by the breakdown of each individual aspect: impact, activity, capability and wider sector conditions.

8. Innovation Index scores for those NHS organisations participating in the survey with innovation strategies and those without

This set of charts shows the index scores for those Local Authorities surveyed with innovation strategies, and those without. The charts cover the four aspects of innovation surveyed, followed by the breakdown of each individual aspect: impact, activity, capability and wider sector conditions.

9. Innovation Index scores for those Local Authorities participating in the survey with innovation strategies and those without

Appendix 7: Disclaimer

These materials have been prepared by Ernst & Young LLP on NESTA's instructions. The information contained in this document is derived from public and private sources (e.g. staff interviews) which we believe to be reliable and accurate but which, without further investigation, cannot be warranted as to its accuracy, completeness or correctness. The information in this report is intended to provide only a general outline of the subjects covered and should not be regarded as comprehensive or sufficient for making decisions, nor should it be used in place of professional advice.

This information is supplied on the condition that NESTA and Ernst & Young LLP, and any partner, trustee or employee of NESTA and or Ernst & Young LLP, are not liable for any error or inaccuracy contained herein, whether negligently caused or otherwise, or for any loss or damage suffered by any person due to such error, omission or inaccuracy as a result of such supply. Accordingly, NESTA and Ernst & Young LLP accept no responsibility for any loss or damage arising from any action taken or not taken by anyone using this.

In carrying out our work and preparing our report, we have worked solely on the instructions of NESTA and for NESTA's purposes. Our report may not have considered issues relevant to any third parties, any use such third parties may choose to make of our report is entirely at their own risk and we shall have no responsibility whatsoever in relation to any such use.

NESTA 1 Plough Place London EC4A 1DE [email protected] www.nesta.org.uk Published: March 2011 PSI/68