While the majority (I should hope) of all social action projects are well intentioned, it is not always easy to prove their efficacy. And as charities, we need good evidence of impact to convince commissioners or funders to provide the resources we need to continue.

Given this pressure, it would be easy to be swayed by the desire to use a measure that you know a particular funder favours. However, as evaluators and charities we must be confident that the measures reflect the intended outcomes of our own unique programmes and we are as realistic as possible.

Evaluating the Time to Talk programme

From November 2014 and March 2016, the Cabinet Office and Nesta funded RNIB’s Time to Talk telephone peer-support service. In this time, it supported over 1,000 people.

The RNIB evaluation and impact team set out to evidence the impact of Time to Talk, working with The Social Innovation Partnership (TSIP) who supported the commissioning and oversight of the evaluation, and validated it against the Nesta Standards of Evidence.

We started by carrying out a Theory of Change with the relevant project stakeholders. This helped us to map out the outcomes we expected and to choose validated measures, in this case the Short Warwick Edinburgh wellbeing 7 levels (SWEMWBS) and in-house crafted measures to reflect the outcomes.

We used SWEMWBS to measure participants’ wellbeing before and after the Time to Talk sessions, and three months later. We also measured the wellbeing of a comparison group who chose not to take part in the programme. We were able to demonstrate an increase in all of the wellbeing measures for Time to Talk participants and the impact of peer support upon confidence building, emotional wellbeing and behavioural change. In contrast, the comparison group decreased in all of the areas of wellbeing.

The evaluation report has been used to provide strong evidence to clinicians of the social and emotional wellbeing benefits of peer-support projects. The NHS Low Vision in Wales responded to new NICE guidelines around screening patients for depression by contacting Talk and Support and our Sight Loss Counselling service, to scope a depression referral route for Low Vision patients (estimated to be around 40% of patients they support). As a result, the services have been identified as an appropriate referral route and a pilot with clinicians is underway.

Matching measures with outcomes

The evidence has also been shared with NHS commissioners in NI to support the roll-out of Time to Talk as part of the RNIB/Action Confidence building offering.

The validated measures and comparison group have been pivotal in this. If we had used another validated tool, EQ5D, which measures wellbeing, known to be favoured by NICE, we suspect we would not have been able to evidence the impact of the project we were evaluating. But we feel scoping for validated measures that match our outcomes was definitely worth the pay-off in this instance.

Using SWEMWBS to evidence outcomes has meant that NHS commissioners have paid attention to the proven benefits of peer support and RNIB are able to reach more blind and partially sighted people with support to increase their wellbeing, improve their confidence and connect with each other.

We will continue to make judgements of measures on a case-by-case basis so we can be confident the tools work to gather evidence of our intended outcomes – but if you find a match between validated measures and your programme outcomes it can ensure commissioning audiences stand up and take notice.

To read Time to Talk impact summary and full report, visit: www.rnib.org.uk/timetotalkevaluation  

To find out more about the Time to Talk service, visit: www.rnib.org.uk/time-to-talk 

This post was originally published on http://www.rnib.org.uk/. Read the original blog.   

Image courtesy of RNIB