They are able to do this because their work is led and delivered by ex-offenders. This gives them the special ability to gain the trust of, access to, and insight from people within the criminal justice system.

What CSAIF funded:

User Voice was awarded £419,000 to expand the model and test its impact “through the gate”. This includes setting up five new councils and building evidence of impact to help the organisation to scale further. View the full impact evaluation.

About the evaluation

Level on Standards: Level 2 - You capture data that shows positive change, but you cannot confirm you caused this.

Evaluator: The evaluation team comprised leading academics in Desistance Theory led by Dr Monica Barry and Dr Beth Weaver (University of Strathclyde), along with Bethany Schmidt (University of Cambridge), Dr Mark Liddle (ARCS), Dr Judy Renshaw, Professor Rosie Meek (Royal Holloway, University of London) and Professor Shadd Maruna (Rutgers University).

Aim: The evaluation of User Voice aimed to assess the implementation, operation and short-term outcomes of the User Voice Prison and Community Councils (which were being implemented in six prisons and three probation Community Rehabilitation Companies across England).

Key findings:

1. Survey data indicates that User Voice Councils have a positive impact on personal outcomes such as self-efficacy, hope and motivation to change.
2. Key performance data collected by the prisons indicates that User Voice Councils have a positive impact on prison-level outcomes such as adjudications, prisoner complaints and assaults on staff.
3. Qualitative data suggested unanimous support for this unique model of Councils which improves the effectiveness and efficiency of services, develops personal and social skills, and enhances feelings of trust and respect between service users and those tasked with their care or rehabilitation. However there were mixed feelings about the level and intensity of involvement required by practitioners (prison officers and offender managers) within Councils, and suggestions that a lack of their involvement can hinder ‘buy in’ and understanding of the role of the Councils.
4. The evaluation uncovered strong evidence that User Voice activities are highly cost-effective – indeed, these activities generate a range of cost-able benefits in relation to service provision and individual change among participants, which taken together far outstrip monthly and annual running costs of User Voice.

Methodology:

A range of pre-post surveys were completed by Council participants. A range of interviews and focus groups were conducted with Council members and User Voice, prison and CRC staff and senior managers. A cost-benefit analysis was also carried out to assess value for money.

Why this Level: The evaluation includes pre-post and comparison data that indicates a positive impact on a number of prison outcomes, and a wealth of qualitative findings that indicate a positive impact on a number of prisoner outcomes. Whilst there is some evidence of limited or even negative impact on prisoner violence, and the attrition rate for the user surveys is too high to qualify for Level 2 by itself, overall the evaluation includes enough Level 2 methodology and positive findings to be validated at Level 2.

About the evidence journey

Progress: User Voice already had a wealth of qualitative research exploring the principles behind its model. This evaluation has strengthened the evidence base further through a mix of quantitative and qualitative analysis that focused on understanding the scale of User Voice’s impact. It also had two further outcomes. First, their Theory of Change has been revisited, based on the responses given in the evaluation, so that User Voice can be much clearer about the outcomes of its Council model and the data that needs to be collected. Second, a bespoke database has been created so that the evaluation team can collect this data in a more user-friendly and consistent way, that will improve its ability to evidence impact.

Lessons learned:

• Baseline data needs to be collected from individual Council participants as soon as they are engaged with User Voice, as significant improvements are made even in the first few weeks
• More regular data needs to be collected from individual Council participants to alleviate the issue of attrition given the levels of churn within a prison environment

Next steps:

1. Roll out the use of the new theory of change across the organisation
2. Embed the use of the new database
3. Provide more training and support to staff to enable them to collect data at the right points