2/21/2014. Outcome Initiative Overview Is there evidence that our services are helping? PURPOSE

Transcription

1 SASKATCHEWAN CHILDREN AND YOUTH MENTAL HEALTH AND ADDICTION SERVICES Outcome Initiative Overview Is there evidence that our services are helping? C. Randy Duncan Betty Anne Rohr February 27, 2014 Park Town Saskatoon PURPOSE 1. What are the aims of the outcome initiative for Saskatchewan? 2. What are the ASIST (CDOI) outcomes? 3. What are the CAFAS outcomes? 4. Who has contributed to provincial CAFAS dataset? 5. How representative is our dataset of child and youth clients that receive Mental Health and Addiction services? 6. How are the CAFAS findings reported? 7. What are the 2013 CAFAS highlights of the provincial findings? 8. What is the typical client time-frame in treatment and how often are they assessed by the CAFAS? 10

2 WHAT ARE THE AIMS OF THE OUTCOME INITIATIVE FOR SASKATCHEWAN? Improve the quality of life for children, youth, and their families OUR GOAL Child and Youth Mental Health and Addictions Services Outcome Initiative Milestones 2006 Ministry Mandate Needs analysis, Lit review Standards of Evaluation Reference Group Recommendations 2007 Confidentiality sharing agreements reviewed Plan for Province wide client outcomes (% improved) CAFAS (web USA Host Server) CDOI (ASIST regional) 2008 Pilot test CAFAS in 4 RHAs CAFAS training in Regina by clinicians representing 11RHAs Pilot test CDOI in 5 RHAs 2009 Pilot testing progress Identify structure for provincially coordinated report Data sharing negotiations including Privacy Officers 2010 FAS SK Health License negotiations Security Evaluation: HIPA and Patriot Act CAFAS data entry commenced CDOI pilot test in Kelsey Trail 2011 First Provincial Report Nov to Sept CAFAS live data from 9 RHAs 3 Month and Exit Indicators CDOI Train the Trainer in 3 RHAs 2012 Two semi annual Provincial Reports 4 RHAs actively using CDOI however, lack of standard use Some CAFAS lag due to clinical tool restrictions on web 2013 Fiscal year Provincial Report 8 RHA Reports customized CAFAS Server move to Canada CDOI use decreased 11

3 The Canadian Server has arrived!! MHS has established a server in Ontario - Our data will be migrated to the Canadian server - Patriot Act considerations will no longer be of concern WHAT ARE THE ASIST (CDOI) OUTCOMES? CLIENT DIRECTED OUTCOME-INFORMED CDOI approach using two instruments (client rated) i. ORS (Outcome Rating Scale) 4 items measure psychological well-being personal, interpersonal, socially, overall ii. SRS (Session Rating Scale) 4 items measure status of client-therapist alliance relationship, goals, method, overall CDOI management systems 1. ASIST for Agencies (on local computer) requires upgrades or will run into technical issues 2. MyOutcomes (web-based BC server) Provincial Outcomes - Intended as a complementary indicator - % at risk for poor therapeutic outcomes - Use is not standardized enough across province to warrant aggregating data - Age level concerns - Some use with youth but mainly adult 12

4 WHAT IS THE CAFAS? Child And Adolescent Functional Assessment Scale Gold standard for assessing daily functioning Clinician-rated Inform decisions on level of care needed & treatment progress Compare change assessments over time Monitor outcomes Evaluate service programs 8 CAFAS Subscales School/Work Home Community Behaviour Toward Others Mood/Emotions Self-Harm Substance Use Thinking CAFAS (WEB-BASED) For each Subscale, items endorsed at severity level that best describes client Clinician able to get client profile immediately 13

5 CAFAS (WEB-BASED) Potential for tracking EBT List of 41 EBT and Other option Clinical Marker (PRE assessment) CAFAS Total Score ^higher score = ^dysfunction Serious Emotional Disturbance (SED) total score = 50 + Severe Behavioural Impairment (SBI) total score = 30 + AND Subscale (s) = 30 Pervasive Behavioural Impairment (PBI) School, Home AND Behaviour Toward Others > 10 CAFAS Tier classification defined by CAFAS Tiers Outcome Indicator (POST assessment) Change in Total Score Meaningful and Reliable Difference = reduction 20 points or more Change in SED total score < 50 AND all subscale scores < 30 Change In SBI Subscales < 30 Change in PBI School, Home AND Behaviour Toward Others < 20 Change in CAFAS Tier Less Salient Tier classification CAFAS TIERS CLIENT TYPES Hierarchically-arranged In Descending Order Of Severity Thinking Problems Substance Use Problems Possibly Self Harmful Delinquency Behaviour Problems with Moderate Mood Disturbance Behaviour Problems without Moderate Mood Moderate Mood/Mild Problems only No Problems Reported 14

6 SEVERITY INTERVALS (DYSFUNCTION GROUPS) 140+ Consider very intensive services, residential, or inpatient Consider intensive community based services Consider outpatient care with additional supportive services Consider Serious Emotional Disturbance (SED) and ongoing outpatient services 0 40 Consider ongoing outpatient services WHO HAS CONTRIBUTED TO PROVINCIAL CAFAS DATASET? Cypress Five Hills Heartland Prairie North Prince Albert Parkland Regina Qu Appelle Saskatoon Sun Country Kelsey Trail Target Population How Representative Is The CAFAS Dataset To All Clients Receiving Child And Youth Mental Health And Addiction Services? Sample 15

7 : Ages 6 to 17 CAFAS Dataset compared to ehealth Dataset Estimate Estimate of New and Reopened Cases using ehealth SK & Community Profile information 12 Health Regions CAFAS Assessment at Entry 8 Health Regions 4352 [VALUE] [PERCEN TAGE] There is some variability within each Health Region as to which clients are rated with the CAFAS at entry to treatment. e.g., regions differ in determining which child clients (6 to 12 years of age) are assessed with the CAFAS Some regions do NOT rate children with CAFAS if they enter into Family programming CAFAS Dataset compared to Population Age Group Total Population Total CAFAS Percentage ehealth SK 6 17 year olds 165,894 1, % Ontario Grade ,771,710 29, % Michigan Under 21 2,914,052 39, % HOW ARE THE CAFAS FINDINGS REPORTED? 16

8 CAFAS REPORTING Provincial Health Region WHAT ARE THE 2013 CAFAS HIGHLIGHTS OF THE PROVINCIAL FINDINGS OF THE CUMULATIVE DATA FROM NOVEMBER 2010 TO MARCH 31,2013? CLIENT CHARACTERISTICS BY SEX Cumulative data November 15, 2010 to March 31, Regional Health Authorities 1,958 Average Age % Total cases entered treatment 4, % 2,148 Average Age

9 CLIENT CHARACTERISTICS BY AGE GROUP Cumulative data - November 15, 2010 to March 31, 2013 Child Youth 6 to 11:11 years 12 to 17:11 years 1,606 2, % Total cases entered treatment 4, % CLINICAL MARKERS CLINICAL MARKERS: TOTAL SCORE SEVERITY INTERVALS Severity Interval at Entry to Treatment For Saskatchewan (N=4106) % of Cases % 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0 40 Consider ongoing outpatient services Consider SED and ongoing outpatient services Consider outpatient care with additional supportive service Consider intensive community based services 140+ Consider very intensive services, residential or inpatient 18

10 CLINICAL MARKERS (PRE-ASSESSMENT TO TREATMENT) TOTAL SCORE MEAN Mean Total Score CAFAS Mean Total Score at Entry to Treatment for Saskatchewan and Health Regions Mean SK (N=4106) Cypress (N=378) FiveHills (N=293) Heartland (N=74) Prairie North (N=462) Prince Albert (N=406) Regina Qu'Appelle (N=1232) Saskatoon (N=923) Sun Country (N=338) Serious Emotional Disturbance (SED) 58.5% 2,400 CLINICAL MARKERS at entry to treatment Severe Behavioural Impairment (SBI) Pervasive Behavioural Impairment (PBI) 34.0% 1, % 476 Of 4,106 cumulative data from Nov. 15, 2010 to Mar. 31, 2013 CAFAS Tier Classification 3 Most Severe Tiers Thinking Problems = 4.5% (186); Substance MisUse= 14.7% (603) Self-Harmful = 16.9% (692) 36.1% 1,481 3 most severe Serious Emotional Disturbance (SED) Severe Behavioural Impairment (SBI) 58.5% 2,400 of 4, % 34.0% 1,065 of 1,958 1,394 CLINICAL MARKER SED at entry to treatment By Sex Pervasive Behavioural Impairment (PBI) 62.2% ** 1,335 of 2, % 47.6% of 1,606 By Age CAFAS Tier Classification 3 Most Severe Tiers 65.4% ** 36.1% 1,636 of 2,500 1,481 3 most severe ** significantly higher than counterpart at p<

11 CAFAS OUTCOMES What Proportion Demonstrated Clinically Meaningful And Reliable Improvement at Exit? (i.e., Reduction in Total Score by 20+ points) Meaningful & Reliable Improvement at Exit 57.5% 771 of 1, % 369 of % 402 of % 265 of % 506 of 851 MEANINGFUL & RELIABLE IMPROVEMENT AT EXIT (i.e., Reduction in Total Score by 20+ points) % of Cases Meaningful and Reliable Difference in Overall Impairment at Exit for Saskatchewan and Health Regions SK (N=1340) Meaningful & Reliable Difference 58.8 Cypress (N=141) Five Hills (N=98) Heartland (N=not reported) Prairie North (N=227) Prince Albert (N=62) Regina QuAppelle (N=239) Saskatoon (N=374) Sun Country (N=187) 20

12 CAFAS OUTCOMES What Proportion Demonstrated Clinically Meaningful And Reliable Improvement at 3 Months? (i.e., Reduction in Total Score by 20+ points) Meaningful & Reliable Improvement at 3 Months 40.9% 450 of 1, % 214 of % 236 of % 161 of %** 289 of 631 ** significantly higher than counterpart at p<0.001 Summary Of Client Improvement Comparison From CAFAS Entry To 3-months Versus CAFAS Entry To Exit From Treatment Cumulative: November 15, 2010 to March 31, % of Cases Meaningful & Reliable No Serious Emotional No Severe Behavioural No Pervasive Behavioural Tier Improvement At Least 1 of 5 Indicators Difference Disturbance Impairment Impairment (N:3 mon=1099 exit=1340) (N:3 mon=1099 exit=1340) (N:3 mon=1099 exit=1340) (N:3 mon=697 exit=792) (N:3 mon=361 exit=437) (N:3 mon=138 exit=152) 21

13 WHAT IS THE TYPICAL CLIENT TIME-FRAME IN TREATMENT? HOW OFTEN ARE THEY ASSESSED BY THE CAFAS? Hawaii CAMHD implemented a policy of quarterly assessment using CAFAS, CALOCUS, and ASEBA. found CAFAS to serve as a model because of high response rates, broad assessment, and completion of certified professionals. across the 4 years, the average length of service was reduced by 40% to 60%, depending on what factors were statistically controlled. 866 days in days in (Daleiden, Chorpita, Dondervoet, Arensdorf, and Brogan ) Client Screened at intake & assigned to a clinician Week 2: Clinical Session 1 Week 4: Clinical Session 2 Average frequency of sessions changes after 3 months from once every 2 weeks to once every 3 weeks Week 6: Clinical Session 3 - First Data Point (Baseline Assessment) Week 8: Clinical Session 4 Week 10: Clinical Session 5 Week 12: Clinical Session 6 - Second Data Point (3-month Post-Assessment) Average CAFAS Total Score is 58.9 points Average CAFAS Total Score is 47.9 points Week 15: Clinical Session 7 Week 18: Clinical Session 8 Week 21: Clinical Session 9 Week 24: Clinical Session 10 - Third Data point (Exit from Treatment) Average length of treatment was 183 days or about 6.1 months for the 1,662 cases of the 4,106 cases that completed treatment by March 31, 2013 Average CAFAS Total Score is 36.6 points ACTIVITY -- FOOD FOR THOUGHT AND DISCUSSION 22

14 Activity & Discussion - Considering the message of the video If such a study was conducted in your region, what evidence does your Health Region have to confirm or deny the claims made? What are some of the benefits or disadvantages of the evidence provided by CAFAS outcomes for your Health Region? For Province overall? FOOD FOR THOUGHT 23