Practice-Based Research for the Psychotherapist: Research Instruments & Strategies Robert Elliott University of Strathclyde Part 1: Outcome Monitoring: CORE Outcome Measure 1
A Psychological Ruler Example of a psychological frequency ruler Often used to assess level of psychological pain For measuring how much a person has had a particular experience over the past week e.g., I have felt terribly alone and isolated. Not at all Only Occasionally Sometimes Often Most or all the time Try out the measurement instruments on yourself Research Commandment 1: Never give a client an instrument you haven t already taken yourself. Research Commandment 2: Never give a client an instrument you can t stand. 2
Take the CORE-10 and see if you can stand it Exercise : I m going to show you the CORE- 10, the short form of the CORE-OM. Take a piece of paper and record your answers I m not going to collect your scores or ask you to say what they are While you do it, pay attention to (a) what it feels like to fill out; and (b) the processes you go through in deciding what number to give in rating the items. [CORE-10 pdf file] 3
Discussion questions (a) What was it like to fill out the CORE-10? How did you feel while doing it? (b) What parts of filling it out were most difficult? (c) What did you learn from doing it? (d) How valid or invalid do you think your responses are? Why? Scoring CORE-10 Add up all the numbers and divide by the number of items you actually completed. 4
Interpret your CORE-10 (1) Use the CORE-OM Pain Ruler to help interpret your mean score: Locate your mean score on this rating scale. This is how much trouble you ve been having with your average problem. If you are allergic to decimal points, you can multiply your mean scores by 10 (=Core systems Clinical Score ) Not at all Only Occasionally Sometimes Often Most or all the time 0 1 2 3 4 Information about the CORE-10 General population: mean =.47 Primary care counselling: mean = 1.97 GP Practice: mean = 2.02 5
Interpreting CORE-10: (2) Look your average All Items score up on the Table Healthy/Mild <1.25, or <1.0 Moderate 1.25/1.0 2.49 Severe/ Very severe 2.5+ Hint: This table only applies to Total scores Note: Clients starting counselling have an average score of about 1.90 on CORE Discussion questions: What do you think of labels like this? Does this fit your experience? 6
Interpreting CORE-10: (3) Look at your high score items (generally these are 3 s or 4 s) These would be issues you might want to work on if you were in counselling. The full CORE-OM works better for this purpose, because there are more items CORE-OM Now let s look at the full 34-item CORE-Outcome Measure (also known as CORE-34) Useful for pre-therapy screening/assessment, prepost assessment Not weekly outcome monitoring [CORE-OM pdf file] Note: small differences between CORE-10 & CORE- OM/34 7
CORE-34=> CORE-10 Equivalence Table CORE-34 CORE-10 Discrepancy 0 0 ## 0.5 0.492 0.008 1 1.035-0.035 1.5 1.578-0.078 2 2.121-0.121 2.5 2.664-0.164 3 3.207-0.207 3.5 3.75-0.25 4 4 ## Formula: CORE-10 mean score = CORE34 mean score X 1.086 minus.051 Data: Scottish outpatient counseling sample n = 563 administrations from 212 clients Using CORE-34 & CORE-10 for Outcome Monitoring Give CORE to your client repeatedly If you give your client CORE-OM every one or two sessions you will always have a post-test score if the client suddenly disappears Use CORE-10 for later sessions, unless you need to track risk issues 8
(1) Is the therapy on track? Use the Outcome Monitoring table to track your client s progress during psychotherapy Note: This is an outcome signalling system developed by Lambert, and adapted for the CORE- OM Progress Categories: White = Nonclinical; consider termination Green = Adequate progress Yellow = May be problematic Red = Clearly problematic; off track Draft Outcome Monitoring Criteria For CORE-OM Where my client is at the beginning of therapy: Nonclinical / mild Moderate Severe / Very severe Status or Change at Sessions 2 4 White Green Yellow Red <1.25 <1.25 < 1.25 & up to.25 worse 1.25 to 2.49 >1.25 & worse by.25 up to.5 worse <1.25 Any better No change or up to.35 worse 2.5+ <1.25 Better by at least.25 (Sessions 5+: better by at least.5) Better by less than.25 (Sessions 5+: no change or better by less than.5) >1.25 & worse by.5+ Worse by.35+ Any worse 9
Is the therapy on track? Discussion questions: Why do you think Lambert has found that providing this kind of feedback to therapists/ counsellors improves final outcome? What would you do if you found out your client was in the Red? Yellow? White? Repeat as needed Continue giving CORE to your client at oddnumbered sessions (or every session) until they finish therapy. Make a list or graph of the scores over time and your client s progress status 10
Example of low-level outcome monitoring with CORE-OM: Carol Pre-therapy After Session 8 Carol presented with severe Social Anxiety After Session 16 2.12 2.50.68 =Moderate distress =Severe distress = Red =Mild distress = White Would have been better to have monitored more frequently, such as on odd-numbered sessions Identify when the client went into the red (2) Assessing Change with CORE When your client finishes counselling, compare the pre- and post scores using the following criteria: 1. If they started >1.25, did they improve by at least.5? (.6 on CORE-10) If so, they have shown reliable improvement That is, their improvement is greater than the error of measurement = highly unlikely to be due to chance (less than 1 time out of 20) 11
Step 3: Interpreting your client s outcome scores 2. If they started >1.25, did they drop below 1.25 by the end? If so, they are no longer in the clinical range. If they met both criteria (=reliable change + moving into nonclinical range), they have shown clinically significant improvement. Example of outcome assessment with CORE-OM: Carol After Session 8 After Session 16 2.12 2.50.68 Pretherapy =Moderate clinical distress Severe distress 80% probability that client has gotten worse Nonclinical range; still has some distress 99+% probability that client has improved Clinically significant improvement 12
Interpreting your client s outcome scores: Discussion questions What does it mean if the client started <1.25? Should you still look at change? Why or why not? Is it OK is my client shows reliable change but is still in the clinical range at the end? Why can t I just ask my client? Part 2: The Personal Questionnaire Personal Questionnaire (PQ): An individualized extended target problem measure Developed by M.B. Shapiro (1961) Simplified by Elliott & Shapiro (1985) Psychometric data: see Elliott et al. (2014, submitted) Normative data: N of items: m = 9.5; sd = 2.8 Pre-therapy mean: m = 5.04; sd =.93 Typical duration of problems: 3 5 years 13
Part 3: Working Alliance Inventory Recommend WAI Short Form- Revised (Hatcher & Gillaspie, 2006, Psychotherapy Research) Watch translation of item 3: I believe likes me Based on Bordin s model of the Working Alliance: Emotional bond Agreement on goals Agreement on tasks 14
Part 4: Some Recommended Qualitative Research Tools Best approach to assessing change processes Helpful Aspects of Therapy (HAT) Form Client (or therapist) identifies most helpful or important event in a session Also hindering or unhelpful events Change Interview Semi-structured qualitative interview done after 8-10 sessions, at end of therapy, etc Also useful for assessing qualitative outcomes Can be adapted for studying any kind of personal change process Don t forget that therapy sessions are qualitative data => discourse analysis etc. Helpful Aspects of Therapy (HAT) Form 15
Helpful Aspects of Therapy (HAT) Form 1. Of the events that occurred in this session, which one do you feel was the most helpful [important] for you personally? It might be something you said or did, or something your therapist said or did 2. Can you say why it was helpful [important]? Please describe what made it helpful [important] or what you got out of it. 3. How helpful was this particular event? Rate it on this scale: HINDERING <----------- Neutral ------------> HELPFUL 1 2 3 4 5 6 7 8 9 ---+-- ---+-- ---+-- ---+-- ---+-- ---+-- ---+-- ---+-- E G M S S M G E X R O L L O R X T E D I I D E T R A E G G E A R E T R H H R T E M L A T T A L M E Y T L L T Y E L E Y Y E L Y L L Y Y Y 4. About where in the session did this event occur? 5. About how long did the event last? 16
HAT Form, continued: 6. Did anything else particularly helpful (or important) happen during this session? YES NO (If yes, please describe it briefly and rate how helpful it was: ) 7. Did anything happen during the session which might have been hindering? YES NO (If yes, please describe it briefly and rate how hindering it was: ) Example of Questionnaire- Based Data: Carol Helpful Aspects of Therapy Form, Session 8: Most helpful: Doing chair work with God Why helpful: I have been struggling with my loss of connection to God*for several months now and during this communication I remembered that there s someone who cares for me and loves me! Helpfulness rating: 9 (extremely) 17
Brief Change Interview - 1 1. General Question: What has therapy been like for you so far? How has it felt to be in therapy? 2. Changes: 2a. What changes, if any, have you noticed in yourself since therapy started? (Anything else?) 2b. Has anything changed for the worse for you since therapy started? Brief Change Interview - 2 3. Helpful Aspects: What has been helpful about therapy so far? (For example, general aspects, specific events, including both what your therapist did and what you did.) 4. Resources: What personal strengths or things in your current life situation do you think have helped you make use of therapy? (what you re good at, personal qualities; also family, job, relationships, living arrangements) 18
Brief Change Interview - 3 5. Problematic Aspects: 5a. What kinds of things about the therapy have been hindering, unhelpful, negative or disappointing for you? (For example, general aspects. specific events) 5b. Has anything been missing from your therapy? (What would make/have made your therapynmore effective or helpful?) Brief Change Interview - 4 6. Obstacles or Barriers: Is there anything about you or about your life situation that might have made it harder for you to use therapy? (things about you as a person; also family, job, relationships, living arrangements) 7. Suggestions. Do you have any suggestions regarding therapy? Is there anything else that you would like to say about it? 19
Possible Exercise: Interview each other about experience of change your personal therapy or some one person change process that you are engaged in Try out some of the Change Interview questions: What changes, if any, have you noticed in yourself since [change process] started? (Anything else?) What has been helpful about [change process] so far? What personal strengths or things in your current life situation do you think have helped you make use of [change process]? What kinds of things about the [change process] have been hindering, unhelpful, negative or disappointing for you? Was anything missing from [change process]? 20