Face to Face service Impact and Evidence briefing Interim Findings Face to Face is an NSPCC service that seeks to increase access to independent help and emotional support for looked after children and those on edge of care. It provides up to eight sessions of support, using a solutionfocused approach. This briefing sets out the interim evaluation findings based on data collected from February 2012 to November 2013. The key findings are that: Across the whole sample on average there was a statistically significant improvement in the wellbeing scores of children and young people between their first and last Face to Face sessions. Before receiving the Face to Face service, 54 per cent of children and young people had wellbeing scores indicating clinical levels of distress, but by the end of the work this had reduced to only 15 per cent. At the end of the work children and young people reported that they felt happier, less worried, better able to cope and that they had a greater understanding of their situation. Prakash Fernandes, NSPCC April 2014
Children and young people said that the Face to Face service had helped by listening to them, helping them reflect on issues and giving ideas about ways to make things better. 2
Background of the service Face to Face is an NSPCC service for children and young people aged between 5 and 18 who would like support to deal with an issue that is affecting their emotional wellbeing or placing them at risk. Children and young people eligible for the service include those who are in care; in care of extended family; privately fostered; in custody; at risk of coming into care; recently returned from care; recently run away from home or care; adopted children; and care leavers. Young people can access the service via ChildLine, either by referring themselves or through referral from an agency with their consent. The service uses a solution-focused approach and is designed to be led by the needs of the young person. This means that the young person identifies their own goals in an initial session and then works towards achieving these with the support of a trained NSPCC practitioner. They are offered up to eight sessions but may terminate the service as soon as they feel they have met their objectives. The service is currently offered in 17 locations across the country. Methodology This briefing paper presents the interim analysis of data collected from young people who received the Face to Face service between February 2012 and November 2013. At each session the child or young person completes the Outcome Rating Scale (ORS), a four-item measure designed to track wellbeing outcomes. It assesses four dimensions: Individual: personal or symptomatic distress or wellbeing Interpersonal: the quality of the person s relationships Social: the person s view of satisfaction with work/school and relationships outside of the home Overall: a big picture or general sense of wellbeing. 3
This tool and its equivalent version for younger children aged 6-12 (the Child Outcome Rating Scale) have been validated in a variety of academic studies. The ORS measure demonstrates reasonable consistency and test-retest reliability despite its ultrabrief nature. At the end of the work the young person is also asked to complete a questionnaire to review the work done and to describe the changes they have experienced. This evaluation sample consists of 213 young people who had completed the Face to Face work and the ORS at more than one session before November 2013. The sample is representative of all users of the service in terms of age, gender and number of sessions attended. Findings Overall improvements for children and young people There was a statistically significant improvement in the average wellbeing scores of young people at the end of the work compared to their scores at the beginning. However, as there is no comparison group it is not possible to conclude whether or not this is a direct result of the Face to Face work. Figure 1 illustrates the change by age group of the young person. Figure 1: Change in average overall ORS scores from first to final session (n=213) ORS mean score 40 35 30 25 20 15 30.8 37.1 26.3 33.2 20.7 29.9 First session mean scores Final session mean scores 10 5 0 6 to 12 years 13 to 17 years 18 and 19 years (n = 83) (n = 116) (n = 14) Source: Evaluation data recorded on PIMS and Alpha data, November 2013 4
The statistical significance means that the changes in scores at the last session are very unlikely to be random. The above graph clearly shows that ORS mean scores have increased by the end of the work, indicating improvements in wellbeing. Clinical significance of change The authors of the ORS scale have identified scores that would constitute clinical distress and those that would be considered in the normal range. The score in between is termed the cutoff point. Scores below it indicate that the young person is experiencing a clinical level of distress, while those above it are considered to be in the normal range. The data indicates that 54 per cent of children and young people started below the cut-off point, but by the final session this was down to 15 percent: a reduction of 39 percentage points. Figure 2: Clinical levels of distress reported (%) n=213 100 90 85 80 Percentage 70 60 50 54 46 40 30 20 10 15 Clinical level of distress Normal range 0 First session Final session Source: Evaluation data recorded on PIMS, November 2013. Reliability of change The authors of the ORS defined a change of five points or more as reliable. In other words, the change observed is substantial enough for it to be attributable to factors other than a normal variation in people s scoring. The analysis of the ORS data of the 213 young people indicates that: 5
Among children and young people who had scores indicating clinical levels of distress (n=115) at the start, 82 per cent improved their wellbeing score by at least five points; for 72 percent this five point increase moved them out of the clinical level of distress range. Among those who started in the normal range, a quarter experienced a reliable improvement in wellbeing of at least five percentage points. Feedback from children s questionnaires at the end of the work Children and young people were asked to complete questionnaires at the end of their work to help evaluate the difference the service had made to them. This data suggests that overall, as outlined in Table 1, approximately 73 per cent of children and young people felt that the service had helped them a lot, especially in increasing their confidence to get help and making them feel more hopeful. About 20 per cent felt that the service had helped them to some extent, while about 7 per cent felt that the service had made no difference or did not help them. Table 1 summarizes their responses. Table1: Outcomes for young people in relation to the face to face service (n=101) Helped a lot (%) Helped to some extent (%) No difference (%) Solving my immediate concern 71 24 5 0 Helping me feel more in control 67 26 7 0 Increasing my confidence to get help 81 12 7 0 Making me more hopeful 73 20 5 2 Source: Questionnaire data completed at the end of the service. Did not help (%) 6
Children and young people s views on how the service helped Some comments included in the young people s written feedback revealed a change in how they felt about their situation and in their knowledge and confidence in using strategies to cope. The names of young peopled used below are not their real names. More positive about their situation or concern Young people reported feeling better, more positive and less worried about their situation: It s helped me a lot because it has lifted all the weight off my shoulders. [Andy, age 13] Better knowledge and more confidence in using new strategies Young people reported that the Face to Face sessions had helped them to better manage relationships with people around them, to know what resources are available in emergency situations and to use strategies to deal with anger: I ve learnt to walk away and how to calm down. This has helped me loads. [Beth, age 12] Aspects of the service that helped the young people make things better Young people also identified how the Face to Face service had helped them achieve positive outcomes. 7
Atmosphere of the sessions They reported that the atmosphere created during the sessions was not too serious and there was a good balance of fun and work: that I can talk about problems and I could do painting today. [Sarah, age 9] Counsellor listening to the young person Young people gave feedback that the relationship with the counsellor was important and that the feeling of being listened to was very valuable to them: (NSPCC worker) was lovely and listened without judging or making assumptions about me. [Leah, age 16] Counsellor allowing the young person to talk about positive aspects of their life Young people reported that the counsellors allowed them to talk about strengths and positive experiences, which helped to make them feel more positive and less to blame for their situation: Thinking about what I m good at actually made me realise that I m not a bad person, so yes it did help. [Connor, age 14] Specific techniques used during sessions Young people identified techniques the counsellors had used during the sessions such as mirroring, story-telling and reflecting that that had helped: 8
Liked the way the worker wrote down things on wallpaper so we could both see what we were talking about. [James, age 16] Aspects of the service that children and young people did not find helpful Some young people reported aspects of the service they did not like: the sessions did not engage them enough; there was too much talking, and they also did not like some of the ground rules for the sessions. Conclusion The data suggests that the majority of children and young people who use the service are reporting improvements in their level of wellbeing at the end of the sessions, and there is a particularly striking level of improvement for those who started with clinical levels of distress. Whilst there is no comparison group data available, and therefore it is not possible to conclusively attribute change directly to the Face to Face service, the findings are very promising. This is especially true as it might be expected that the difficulties of looked after children and those on the edge of care would be entrenched and therefore not show much change during the relatively brief intervention period. It will be important to examine if change has been sustained over a period of time by following up with these children and young people after three months of receiving the service, which we will do in the next phase of the evaluation. The next phase of will also include in-depth interviews with children and young people who have finished the work to better understand the nature of the changes they may have experienced, as well as interviews with referring agencies to understand their perceptions of change for the young person. www.nspcc.org.uk Registered charity numbers 216401 and SC037717.