Xenzone is committed to developing an evidence base for online counselling.

Transcription

1 Xenzone is committed to developing an evidence base for online counselling. Since its flagship service KOOTH.COM was launched in 2004, Xenzone has been collecting and reporting data for commissioners and the wider world to show the impact of KOOTH.COM in young people s lives. The Development of Outcome Measures on KOOTH.COM In this time, Xenzone has developed a very strong working relationship with the University of Manchester to develop research projects that evaluate the work we do on KOOTH.COM. Dr Terry Hanley, (Programme Director at University of Manchester for the Counselling Psychology Doctorate) was the first to investigate. His findings (Hanley, 2012) indicated there was a strong therapeutic alliance (a key ingredient and indicator of effective therapy) experienced by young people online. At Xenzone, we currently use outcome measures as recommended for CYP IAPT services (see and by the BACP in their outcomes toolkit for school based counselling (see At present there are no direct recommendations for online counselling, and we have adapted these to suit our needs. The key measures recommended are: A Clinical Outcomes Measure at Assessment - we use YP-CORE (for info, visit End of Session Feedback to evaluate Therapeutic Alliance A Goal based Outcome Measure we have developed this into a more interactive and youth-friendly measure called CoGS (Counselling Goals System) The development of the CoGS (Counselling Goals System) outcome tool came from an internally commissioned piece of research to evaluate the use of outcome measures online. It was published in the international peer-reviewed academic journal, Pastoral Care in Education (Sefi and Hanley, visit for abstract). Its key finding was that the YP-CORE was insufficient to tell the whole story of how a young person s life was changed through our intervention, and it recommended developing goal-based outcomes for a bigger picture of the how, what and why online counselling works. We received a research award from the British Association of Counselling and Psychotherapy (BACP) for Xenzone Alliance, our research arm, to develop the CoGS project as an innovative tool to evaluate online counselling and support, and to disseminate these findings through articles and peer-reviewed papers, due to be published in 2015.

2 Method We developed the CoGS as a goal-based outcome measure for use on KOOTH.COM. In a young person s initial contact with the service, our practitioners can identify, through collaborative assessment, what goals a young person has and wants to achieve. These can relate to anything in their lives, or specifically in the work we do with them. CoGS in action on KOOTH.COM The young person can score where they are up to in achieving this goal at any time (in chat with a counsellor or simply when they are logged onto their home page on KOOTH.COM). When they score 10, it indicates that the goal has been achieved, and this is illustrated by a gold star. 2

3 The CoGS have shown to be effective in 3 key ways (see Findings for further discussion): 1. They are a measuring tool to record how well we are helping young people to achieve their goals in counselling and in their lives. 2. They are a therapeutic tool for our practitioners to use to focus the work and for young people to take away. They are a useful way of describing their experience on KOOTH, and can demystify the process of counselling. 3. They help us demonstrate in great detail what the achievements of our service through usergenerated outcomes. The analysis of this data (shown below) tells us a lot about what young people want from and achieve through our intervention. Training As this is a new and innovative way of working, we trained all our practitioners across all our services in how to use the CoGS. The training included the key elements of: theory (the CoGS are inspired by the Pluralistic Framework for counselling see Cooper & McLeod, 2011); technical (how to use the interactive tool on KOOTH.COM); and skills practice (how to use the CoGS effectively within their practice utilising knowledge of this work derived through experience and academia, for example, Hanley et al, 2013). These training sessions were counted towards our counsellors Continuing Professional Development (CPD) and have helped the service engagement with outcomes and research. This training and on-going mentoring was backed up by a manual for using the CoGS and an Outcomes Toolkit for KOOTH Practitioners, which outlines Xenzone s ethos for KOOTH as an Outcome-Informed Service. Data Collection The CoGS were launched with our new build of KOOTH in November Before this, we had already been collecting CoGS data from our face-to-face services as well (this data forms an important part of the comparison of what goals are achieved through different media for counselling, and will be published later this year as part of a comparative study). 3

4 The CoGS data is all recorded as a key element of our integrated case notes system: Between November 2013 and August 2014, we collected goals data from the CoGS. This amounted to: 1137 goals articulated by 505 young people online through KOOTH.COM - these are the goals described in this paper. NB. This does not reflect the total work being done on KOOTH.COM as it took an initial period of bedding in to get practitioners to use the tool regularly (and since this end date, we have over 2000 goals at time of going to press) Through KOOTH face to face services (in Powys, Knowsley and Cornwall) 360 goals were articulated by 202 young people analysis of face-to-face goals in comparison to online will follow in In this time, we have conducted semi-structured interviews with six practitioners, to offer feedback about how the CoGS work from a practitioner point of view. We have also collected feedback from young people through an online focus group conducted through live group discussion, and through comments posted in a specially made forum thread. Some of this feedback is detailed below and will form basis for further research studies. 4

5 Data Analysis The project has been run in conjunction with the University of Manchester. The data analysis is being conducted under their auspices and the supervision of Dr Terry Hanley. It is being carried out by Aaron Sefi, Xenzone s Lead for Research and Outcomes, alongside Zehra Ersahin, a Doctorate student from the University of Manchester. A key part of the analysis is to codify the goals into different categories and themes. This codifying has drawn from key literature in the field of goals research namely the Berne Inventory of Goals (Grosse Holtforth and Grawe, 2002). However, as this is the first large scale research into therapy goals for children and young people, it has been necessary to adapt categories and themes. These categories and themes also represent the wide range of therapeutic approaches and models utilised by KOOTH.COM practitioners from solution focussed techniques to Dialectic Behaviour Therapy (DBT) and Cognitive Behaviour Therapy (CBT) to the Person-Centred Approach. Three Key Overarching Categories were established: 1. Intrapersonal Goals goals that are about the young person s internal world, e.g. Wanting to explore difficult feelings 2. Goals on Self relating to others goals about how a young person s internal world relates to the external world, e.g. Wanting to express myself better with my peer group 3. Interpersonal Goals goals that involve the young person s relationship to other people, e.g. Wanting a better relationship with family From these overarching categories, sub categories were formed: 5

6 In keeping with the theme of CoGS, and how the different types and categories of goals can work together as a mechanism for change in young people s lives, they can also be illustrated thus: 6

7 Findings These sub-categories have been further broken down into themes relating to that category e.g. the sub category of personal growth has six themes within it Exploring thoughts and feelings; Owing the problem; Improving confidence and self-esteem; Improving and developing skills for life and Accepting self. Our data analysis was conducted using a research methodology called Grounded Theory (Glaser and Strauss, 1967). In this approach, the researcher employs a conscious naivety when looking at the data, allowing themes and categories to emerge, as such it is a bottom-up or grounded approach to data. Through this method, Meaning Units (MUs which measure how often this goal theme is covered) are created and assigned to each theme as it emerges using qualitative data analysis coding software Nvivo 9 (Richards, QSR International: 2010). Overarching Category Intra-personal Goals Sub Categories Themes MUs Illustrative User-generated goals Personal Growth exploring thoughts and feelings 99 To explore why I feel people don t like me owning the problem 109 Wear short sleeves in public improving confidence and self-esteem 48 Process, let go and move on for a better future improving and developing skills for life 26 Identify 5 things that I like about myself accepting self 57 Acknowledge that it s okay to feel sad Emotional Wellbeing feeling happier 41 Control emotions so not crying all the time keeping calm (less anxious) & regulating emotions 49 To feel happy and more in control To manage my moods better working on anger, and grief (utilising 25 I wanna be ok with my mind not let it control me alternative ways to cope) Give permission myself for a shopping day with mum enjoying self 9 Mental Wellbeing exploring (have a better factual understanding of the mental health problem- various) 24 Read information on physical symptoms of anxiety To explore why I hurt myself for fun work on self-harm, anxiety, depression, suicidal tendency, eating difficulties, obsessive compulsive disorder, and learning disabilities by utilising a coping strategy (mindfulness & yoga, relaxation exercises, narrative work) 92 Write or draw feelings instead of self-harming Practise safe place visualisation, to control suicidal feelings Go for a walk often to help lift depression. 7

8 Physical Wellbeing self-care (i.e. sleep hygiene, stopping smoking, heaving a healthy diet) exercising to keep fit 18 keeping safe 9 60 Walk home from school for exercise Wear longer skirts/ trousers to school Start jogging at weekends Drop a size from 12 to 10 by eating healthy To shower and dress every day Inter-personal Goals Goals on Self relating to Others School/Career Improving relationships Getting help (it is okay to get help) Getting out of comfort zone (Challenging behaviours) Speaking up (communicating self better to others) Fitting in (in external world aspirational (pursuing an interest at university, planning on a professional career, accomplish self to a fame, success, wealth) exams (getting organised & getting good grades) improving relationships with family members 48 Practice online resources one hour a day to pass the RAF test To succeed in school and get a job to secure myself Become a good writer like John Green 35 I want to be in a band and tour the world. 19 Eat out with family improving relationships with friends 55 Improve relationships with family and friends To have a better relation with girlfriend improving relationships with significant others (boyfriend/girlfriend) 15 Get closer to brothers seeking health assessment (GP, nurse, 48 To ring brook for pregnancy test at clinic psychiatrist) Speak to head of year about Camhs referral again connecting to support network (family, 97 Transfer from Kooth to face to face counselling friends, teachers, tutors, pastoral care) Ask tutors for more support to improve my grade accessing f2f counselling services 10 Get through difficult day by coming online being resilient (against bullies/negative 7 Come back to chat next Monday, 8pm people) Stay off Facebook for few days until more settled being assertive in relationships 6 To learn to live with brother's Aspergers setting boundaries 6 Stop saying sorry, change to respect Practice to bat back insulting remarks committing to therapy (and thus wellbeing) 32 expressing self better and more open to 84 To talk to my girlfriend about hurt feelings significant others Let someone know who I have strong feelings for get things out of my chest 10 To talk things through and get them off my chest feeling comfortable in relationships 23 Look after granma who has early dementia Working on being accepted 27 To be comfortable making and keeping relationships Succeed and be a good person to keep grandpa attending to needs of family members and 7 proud friends 8

9 These findings can also be illustrated as charts: Overarching Categories Intra-personal Goal sub categories Intra-personal Goals Interpersonal Goals Goals on Self Relating to others Personal growth Emotional wellbeing Mental wellbeing Physical wellbeing School-Career 7% 12% 11% 32% 61% 15% 46% 16% Improving intimate relationships 17% Inter-personal Goals Improving relationship with friends 62% Improving relationships with family members 21% Goals on Self Relating to Others Challenging behaviour 14% Fitting in! (In relation to significant others) 16% Speaking up! (Communicate Self better) 26% Asking for help (and getting help) 44% 9

10 As indicated on previous page, intra-personal goals make up the largest category (61% of all the goals). Within this category, the sub category of personal growth is the largest (46%) Within this sub-category, the two predominant themes are: 1. Owning the problem these are self-help style goals focussing on what the young person can do to support their therapy 2. Exploring thoughts and feelings these are typical exploratory goals for young people goals for therapy, that are worked on through the sessions 10

11 This was the second most common category (32% of all goals articulated) The most common sub category in Self Relating to Others was: Asking for help (and getting help) at 44% of the goals in this category. The predominent theses in this sub-category are: 1. Connecting to Support Network this relates to young peoples goals around using family, friends, teachers and significant others for support in their issu and to support the therapy 2. Seeking health assessment - this reflects the level of integration and direct link we make on KOOTH.COM to health professionals, encouraging young people to link in with statutory service This category accounts for 7% of total goals articulated. The most common theme in this subcategory is: Improving Intimate relationships these relate to goals young people have around boyfriend/girlfriend issues specifically. 11

12 Distance travelled Finally, with the 1137 goals collated from the online service in the data collection period. As we chart the movement towards achieving a goal, we are able to identify which goals are achieved and to what extent, thus giving us a picture of distance travelled in our service. We collected this data in November 2014, and it indicates work in progress as many of the goals set in the data collection period are still be worked towards, but overall, it shows some very interesting results: Goals that have been fully achieved: 28% Goals that have moved by 8 points or more (i.e. nearly achieved): 40% Goals that have moved by 5 points or more (i.e. at least half-way achieved): 69% Overall, the median average movement for a goal is 7 points As mentioned, this outcome data from our CoGS research project will be published later this year once independently evaluated by the University of Manchester, alongside some movement data from our face-to-face service, and in comparison with existing research data on similar tools used in CAMHs (see Law, 2011) and other evaluation of youth counselling (see for example Rupani et al, 2013). On early reflections, this data is closely aligned, and shows that young people achieve their goals well in an online environment. We can also see a trend in young people s achievement of goals and how long they spend on the service: Score (out of 10) by Chat Hrs over 10 CHAT TIME in hours This table shows that the more time a young person spends on KOOTH.COM, the closer they get to achieving the goals they have set for themselves. 12

13 Discussion Taxonomy or frequency of goal themes and categories If we look firstly at the taxonomy of the goals (i.e. frequency of each category and sub-category), the first most striking aspect is how few inter-personal (i.e. relationship) goals were articulated in comparison to intra-personal goals (i.e. around personal growth). Whilst this can be explained to some extent by the addition of the Self Relating to Others category (which was in itself devised to differentiate or fill the gap between intra- and interpersonal), it can also be a reflection of the priorities of an online client that family relationships and friendships are not the primary focus of a young person seeking support online. With so many intra-personal goals the clear primary focus is around personal growth, and specifically, around the two key themes of Owning the problem and Exploring thoughts and feelings. Owning the problem can also be seen as a Life Goal taking responsibility for decisions and actions in life that will help achieve the goal set in therapy. Exploring thoughts and feelings is more of a Therapy Goal - setting a goal around using the counselling to explore thoughts and feelings, primarily in order to understand oneself better (see Cooper and McLeod, 2011 for more on these distinctions). The most common goal themes in the category of Goals on Self relating to others were: Connecting to support network; and Seeking health assessment. Both of these reflect a similar drive to the life goal theme of Owning the problem, in that they require the young person to make efforts outside the counselling to develop support for themselves. This reflects the skilful work of practitioners on KOOTH.COM to engage with young people around their global support network. In this, crucially, we can help young people develop an autonomous and independent role in making change in their own lives. It also reflects the anecdotal understanding of online therapeutic support that it is often independently sought and autonomously undertaken. As it requires a high level of selfmotivation to access, online therapeutic support can be tailored to enhance and develop that self-motivation. This is reflected in some of the feedback that young people and practitioners have given us around the use of the CoGS: Setting goals for myself means I can help myself and not rely on anyone else! I think it really good so everyone can help other people and they can set their own goals! Turtle Please see Appendix 1 for additional feedback from young people and practitioners on the CoGS. Participation A lot of the feedback we have received from young people has shown how collecting outcomes data can and needs to be a positive part of a young person s experience of our service. 13

14 In this way, it can be a critical part of their active participation and involvement with developing the service. This can take place on a 1-1 basis between young person and practitioner we advocate the idea of our practitioners being FIT that is to say, operating Feedback Informed Therapy (inspired by the work of Scott Miller in psychotherapy research) The CoGS also enables participation on an organisational level, as feedback also shapes service delivery. As a service, we routinely collect a range of feedback from satisfaction questionnaires, to end of session feedback forms, and these can directly inform how young people want their next session to be. The CoGS adds to this data and allows us to look at how we can improve our service based on examining the extent of achievement of goals through our interventions. Outcomes As outlined previously we can evidence positive outcomes for young people using KOOTH.COM through the CoGS. The median score of 7 points (out of maximum possible 10 points) movement in goals that young people have worked on, with 69% of goals at least half-way achieved when we took a snapshot of the work, indicates that most of the goals young people want to work on are being achieved. The next piece of work to achieve with this data is to investigate which categories and themes of goals are more achieved than others. This work will be done as part of the research project in 2015 with the University of Manchester. Next Steps We have used the taxonomy data presented here as the basis for a submission of a paper for the peer-reviewed academic journal: Counselling and Psychotherapy Research (Sefi, Ersahin and Hanley, 2015: to be published). We will be using the data to produce two further papers in 2015, evaluating the outcomes data, and the feedback from practitioners and young people. We will also be presenting the data at various research conferences in the coming years. As mentioned, all of this data will crucially continue to feed into service delivery and design, allowing for genuine user generation of the service model. We continue to collect CoGS data, and since the collection process for this part of the project has ended, we have already collected nearly 1000 more goals at time of going to press. We aspire to develop an application for young people to make it easier to use the CoGS out of session in face-to-face counselling, and we are keen to share our learning and training on how to effectively use goal-based outcome measures as an outcome measure, therapeutic tool, and basis for service design. For more information, please contact: Aaron Sefi Service Development Manager KOOTH.COM T: E: aaron@xenzone.com W: 14

15 References for this report Cooper, M. & McLeod, J. (2011). Pluralistic Counselling and Psychotherapy. Sage: London. Hanley, T., Humphrey, N. & Lennie, C. (Eds.) (2013). Adolescent counselling psychology: theory, research and practice. London: Routledge. Law, D. (2011). Goals and Goal-based Outcomes (GBOs) Some Useful Information Version 2.0 downloaded from Grosse Holtforth, M., & Grawe, K. (2002). Bern inventory or treatment goals: Part 1: Development and first application of a taxonomy of treatment goal themes. Psychotherapy Research, 12, Hanley (2012). Understanding the online therapeutic alliance through the eyes of adolescent service users. Counselling and Psychotherapy Research 12:1, Rupani, P., Cooper, M., McArthur, K.,and Pybis, J., Cromarty, K., Hill, A., Levesley, R., Murdoch, J., and Turner, N. (2014) The goals of young people in school-based counselling and their achievement of these goals. Counselling and Psychotherapy Research, 14 (4). Sefi, Ersahin and Hanley (unpublished). Examining the goals of young people in online counselling and support Sefi, A. & Hanley, T. "Examining the complexities of measuring effectiveness of online counselling for young people using routine evaluation data." Pastoral Care in Education An International Journal of Personal, Social and Emotional Development 30, no. (1)(2012): Appendix 1 Feedback from Young People and Practitioners about CoGS Here are some examples of the feedback young people gave when asked on a live forum what they thought about CoGS (punctuation added!): Setting goals for myself means I can help myself and not rely on anyone else! I think it really good so everyone can help other people and they can set their own goals! I think it s a good way of managing things breaking things down into bitesize chunks It s something to work towards, to feel like you have achieved something. And from young people on how the CoGS works for them: 15

16 The counsellors always support you when you try to get there, you get this really good feeling when you achieve them The counsellors give you some ideas of the goals you could work on, and when you have a giant problem and turn it into little ones and work on them one by one the big problem doesn t seem so bad. If you dont do it in time, the counsellors make you feel better about it. They just set it again with another deadline or they alter it a bit so it s easier to do. They always make sure you dont feel bad. They tell you it s alright Here is some comments from practitioners collated as part of semi-structured interviews that will form basis of another academic paper from the CoGS project: I think that the need to have goals (no matter how small they may seem to an observer) is a vital aspect of any therapy. The goal of 'see you again next week' may not be officially seen as a goal but in my view, it s probably THE most important goal that needs to be met. For the client, [the CoGS] can provide focus and something to identify that they are working toward (making it achievable is also important in any goalsetting). For the therapist it can show how the therapy is working and also allows he/she to reflect on their own practice and approach [The CoGS helped to] bring in more transparency of structure and objective when working with young people it seemed to help them feel more secure and focused in the work There was one young person, who felt quite distant to me throughout our work together online. But then, when I checked in with her with the cogs, she moved them along quite a lot... it was hard for her to put into words or express to me how things felt different - but through her cogs she could demonstrate to herself and me that things were different, and that there had been quite a lot of movement for her (more than I would've guessed) plus it can be really orientating for both of us - something really tangible and constant in a virtual online space! I think it clarifies for them in a real and tangible way what they actual problems are and how they can work with that. If they are just stuck with this overwhelming emotional experience that they don't know what to do with...and then you can help them to break it up into something that they can see and actually do something with. 16