Judging the success of feeding back outcome. information to service users

Transcription

1 Judging the success of feeding back outcome information to service users Andrew C. PAGE School of Psychology, The University of Western Australia & Perth Clinic, Perth Australia

2 Acknowledgements Geoff Hooke, Moira Munro, Kate Harwood, Elizabeth Newnham, Adelln Sng, Shannon Byrne, Fiona Ronk, Emma Restifo, Kale Dyer, Shraddha Kashyap, Nicola Flood, Ye Elah Berman Funding: Australian Research Council Medibank Private HBF

3 Rising to New Heights

4 The Team Comes First

5 The Original The Best

6 Our Journey From Measurement to Monitoring Owning the Process Engaging Staff and Patients

7 From Measurement to Monitoring at Perth Clinic

8 Perth Clinic 100 bed Inpatient hospital Western Australia Inpatient, day-patient, outpatient and outreach services 8

9 Outcomes at Perth Clinic

10 Recovered Improved Deterioration (Newnham, Harwood, & Page, 2007)

11 Need for monitoring progress No Change 57% Deteriorated 8% Recovered 14% Improved 21% Hansen, N.B., Lambert, M.J. Forman, E.M. (2006) Clinical Psychology: Science & Practice

12 Boswell et al. (2015) Non-response rate % (Hansen et al. 2002) but clinicians estimate that 85 % of their own patients improve or recover (Walfish et al. 2012) 90 % of clinicians rate themselves in the upper quartile and none see themselves as below average

13 Boswell et al. (2015) In mental health treatment there has been a glaring lack of stakeholder involvement in outcomes data initiatives and research Do patients value outcomes information? How should this information be disseminated? Does access to this information result in better care?

14 How well do clinicians predict treatment failure? (Lambert, Whipple, Bishop, Vermeersch, Gray & Finch, 2002)

15 Schulte & Eifert (2002) 30% of treatment goals were established during therapy Average number of method changes per session was 1.4 Changes in treatment direction occur when therapists are pessimistic about outcomes or believe they have little control over treatment but therapist ratings of session success were poorly correlated with final outcome

16 Schulte & Eifert (2002) Therapist mood predicted perceived success of treatment, but therapist mood deteriorated if patients did not express acceptance or signaled dislike Method changes in treatment correlated negatively (-.49) with outcome Deviating from the method guidelines in main part of treatment was harmful (r= -.41). Adherence to manual in treatment phase correlated positively (.42) with outcome

17

18 The Key Early treatment response is indicative of final outcome. Howard, Moras, Brill, Martinovich & Lutz, 1996

19 Results: DI-5 (Symptom Index) Dysfunctional 3.2% N = 43 Deteriorated Unchanged Improved Recovered DI-5 Score % N = Functional 53.4% N = % N = Day of CBT Flood, Hooke, & Page, submitted 19

20 Outpatients: Improved outcomes Lambert, M.J., Whipple, J.L., Vermeersch, D.A., Smart, D.W., Hawkins, E.J., Nielsen, S.L., & Goates, M. (2002) Clinical Psychology and Psychotherapy

21 Why DIY? Owning the Process and Making it Ours

22 Our patients are different Inpatients and daypatients Treatments tailored

23 Effects of Feedback: Mixed Meta-analyses - effects of feedback on outcome in the range of very small to very large (Knaup et al., 2009; Sapyta, 2004; Shimokawa et al., 2010) Feedback most effective for patients who are not progressing well in therapy - Not-On-Track (NOT) cases (Lambert et al., 2003; Newnham et al., 2013; Sapyta, 2004, in Sapyta et al., 2005). Carlier et al. (2010): feedback did not have a positive effect in 36% mental health trials Knaup et al. (2009) feedback had a negative effect in three of 12 studies

24 Owen et al. (2015): ORS (75%; M=7.6 sessions (6%; M=7.8 sessions) (19%; M=16.9 session

25 Pfeiffer et al. (2015): PHQ-9

26 Lutz et al (2015a): GMH

27 Symptoms (DI-5) Wellbeing (WHO-5) Observed Means of Four Groups During Inpatient Stay Observed Means of Four Groups During Inpatient Stay 25 Group 1 (30%) 15 Group 1 (30%) Symptoms (0-25) Group 2 (7%) Group 3 (37%) Group 4 (26%) W ellbeing (0-25) 10 5 Group 2 (7%) Group 3 (37%) Group 4 (26%) 0 Ad mit 1/3 2/3 3/3 Disch 0 Ad mit 1/3 2/3 3/3 Disch Stage of Inpatient Stay Stage of Inpatient Stay

28 WHO-5 Wellbeing Index Over the last day I have felt cheerful and in good spirits I have felt calm and relaxed I have felt active and vigorous I woke up feeling fresh and rested My daily life has been filled with things that interest me At no time Some of the time < half of the time > half of time Most of the time Newnham, Hooke, & Page, 2010 All of the time

29

30

31 Illustration of Not on Track Patient Responding to a Friday Feedback Session

32 Feedback

33 Inpatients & Day patients: Improved outcomes Vitality: (F(1,639) = 5.53, p =.02) Role Emotion: Newnham, E.A., Hooke, G.R., & Page, A.C. (2010) Journal of Affective Disorders (F(1,635) = 4.11, p =.04)

34 On Track patients 90% Feedback p< % No Feedback Byrne, Newnham, Hooke, & Page, 2014

35 Economic implications On track with feedback Cost effectiveness: Saving of AUD$60,000 per 100 patients (based on the WA estimates for and of average cost per patient per day: AUD$819 & average length of admission: 13 days) Australian Government Department of Health and Ageing: National Mental health Report, 2010

36 DI-5 Daily Symptom Index Over the last day At no time Some of the time < half of the time > half of time Most of the time All of the time I have felt anxious I have felt depressed I have felt worthless I have thoughts about killing myself I have felt that I am not coping

37

38 Patient Progress Graph- Combined IP and CBT

39 Adding Symptom to Well-Being Feedback Change in Distress for O ff-track Patients Change in Distress from Pre-Treatm ent W B+SX Feedback W B Feedback -4 M id -R x P ost R x Dyer, Hooke, & Page; 2015

40 13 HoNOS NOT Feedback NOT no Feedback HoNOS Pre Post

41 Engaging Staff and Patients

42 Feedback Works But! Rule Violations (+/- 3SD)

43 Less Feedback Vs More Feedback Psychoanalysis (60% success) Cognitive Therapy (60% success) Cognitive Therapy (40% success) Or Psychoanalysis (40% success) Decision? Feedback Orientation? Success?

44 Feedback Works

45 Feedback Does Not Affect Perceptions of Success

46 Self-Determination Theory Contexts that support autonomy foster internalization of the value of doing a task (Deci & Ryan, 1985) Autonomy is supported when: task appears important feelings toward the task are acknowledged choice in how to perform the task is provided

47 Moderators of Feedback Effects 44% considered outcome monitoring a waste of time (Aoun et al., 2002) 66% unwilling to use monitoring feedback even if doing so would improve outcomes (Walter et al., 1998) de Jong et al. (2012, 2014) Beneficial effect of feedback among NOT patients ONLY when therapists said they used feedback Therapists open to feedback had faster positive outcomes Simon et al. (2011) smaller effect sizes in centre with very heavy caseloads and staff demoralized by organizational changes

48 Having consumers own the process

49 Mental Health Thermometer

50

51

52

53

54 Having staff own the process Giving staff timely, relevant information

55 Patients Suicide Entry Screen Score

56 Restifo, Kashyap, Hooke, & Page, 2015

57 Day 1 Score on Suicide Question Not Known Day 2 Score on Suicide Question Estimated 8% 2% 4% 18% 8% 23% Risk Restifo, Kashyap, Hooke, & Page, 2015

58

59

60 Interpersonal Theory of Suicide Thwarted Belongingness DESIRE / IDEATION Acquired Capability Perceived Burdensomeness Risk is high

61 Interpersonal Needs.59 Acquired Capability Suicide Risk These days, I am close to other people + Thought of the least difficult way to kill myself => R 2 =.44

62 Having staff own the process Giving staff timely, relevant information Incorporating monitoring into routine care

63

64

65 Having staff own the process Giving staff timely, relevant information Incorporating monitoring into routine care Giving staff and patients clinically useful information

66 So where to now?

67 Use of CBT Skills

68 Confidence in CBT Skills

69 Correlations with Symptoms Days In Treatment Use Effectiveness Confidence

70 The Original The Best

71 Spreadsheets on ResearchGate: researchgate.net/publication/ _development_and_psychometrics_of _the_five_item_daily_index_in_a_psychiatric_sample_-_calculator?ev=prf_pub researchgate.net/publication/ _who_5_monitoring_spreadsheet_for _scoring_and_interpretation?ev=prf_pub 25 DI-5 Score Deteriorating Potential for Change Improving 5 0 Healthy Range Session

72 How to use it: Page & Stritzke (2014) Cambridge University Press

73

74 Can I use it? Yes, andrew.page@uwa.edu.au or Geoff Hooke (geoffh@perthclinic.com.au) or check my entry on for the Excel spreadsheets

75

76 Belong Interpersonal Needs.38 Burden Acquired Capability Suicide Risk