What CSAIF funded: RNIB was awarded £190,571 (including £23,000 for evaluation) to develop an education and peer support intervention for people with sight loss or partial sight over the telephone. The peer support intervention ensures that people are given information on practical issues such rehab training and useful equipment, and also how to find support and connect with others locally.

As a result of the grant, RNIB has delivered over 4100 hours of peer support – reaching 1027 beneficiaries and activating 16 volunteers.

The funding and evidence generated through the grant has also helped the team successfully secure internal core funding to continue their intervention in the future. View the full impact evaluation.

About the evaluation

Level on Standards: Level 2 - they have captured data that shows positive change, but cannot confirm they caused this.

Evaluator: N/A (RNIB conducted their evaluation in-house).

Aim: The evaluation aimed to assess the effectiveness of the programme on five knowledge and wellbeing outcomes in 65+ year olds with visual impairment. A process evaluation also investigated what could be further improved about the delivery of the service.

Key findings:

1. Small to medium effects were found on knowledge about available support services and take-up of those services.
2. Small positive effects were found on the two well-being outcomes.
3. Very small positive effects were found on social isolation and feeling more useful, while a majority felt the service was helpful to them.

Methodology:

• RNIB employed a pre-post design with an informally matched comparison group (not robust enough for validation at Level 3) involving quantitative data from surveys and qualitative data from interviews.
• 1740 older visually impaired people signed up to ‘Time to Talk’, with 100 taking part in the evaluation. All participants consist of 65+ year olds with various levels of visual impairment. The control group included 128 participants who differed measurably from the treatment group – likely due to selection bias.

Why is it a Level 2 Evaluation?

This evaluation featured some elements of a Level 3 validation. Unfortunately these elements were not robust enough to qualify for validation at Level 3. The comparison group consisted of individuals who were offered the intervention but turned it down, which makes them systematically different from the intervention group in a way that might influence the outcomes in question.

However, RNIB have delivered an evaluation that suggests a positive change over time for the individuals that it supports, using some externally validated pre-post tools, alongside self-crafted tools. Overall response rate to the surveys was quite low, but the remaining sample size was still comparatively large in and of itself.

About the evidence journey

Progress: RNIB have moved from Level 1 to Level 2 on the Nesta Standards of Evidence, reflecting the fact that they are now able to more robustly show a positive change in outcome. The next step on the Standards would be to include a more robust comparison group (e.g. matched or randomised), to be more confident that the positive change seen is due to the programme itself (rather than other factors).

Lessons learned: Through the evaluation, they identified a number of lessons about the effective delivery of the service - the importance of building accessibility into the facilitator role from the inception of a project; sending out post course information promptly; the positive effects of peer support on wellbeing; and the desire for localised support.

Next steps: The project team will incorporate the lessons learned into the continuation of the project by trailing new methods of information delivery to ensure that the facilitator role is fully accessible, and by working with the Action ‘Living With Sight Loss’ team to ensure that there are clear referrals routes throughout the organization into both services, thereby proving a joint offer of confidence building services, either localised or over the telephone.