Mark A. Greenwald Director of Research and Data Integrity Laura Moneyham Assistant Secretary for Residential Services 8/21/2015 1
Discussion Topics Risk Assessment Importance of Service Matching The Disposition Matrix Continuum of Services Mapping Standardized Program Evaluation Protocol (SPEP) 2
Risk Assessment Then and Now Intuitive Actuarial & Static Dynamic Case Management Protective & Responsivity 1st Generation 2nd Generation 3rd Generation 4th Generation 4th Generation 3rd Generation 4th Generation 4th Generation 5th Generation: In Development 8/21/2015 3
Using Predictive Analytics to Improve Risk Assessment
Modeling Metrics to Outcomes NEW RECIDIVISM VARIABLE_ Total Convictions _Adj_New Defmition Node 0 Catego!Y % n NO 77.679 47440 YES 22.321 13632 Total 1 00.000 61 072 I L:: JJIS Total number of adjudicated charges pnor to program 00: = 2.000 )> 2.000 I Node 1 Node 2 % catego!:y % n 83.184 36344 16.816 7 3 47 NO YES 63.840 11 096 36.160 6285 Total 71.540 43691 Total 28.460 17381 I I L:: JJIS Total number of charges pnor to program < = 13.000 )> 13.000 I Node 3 Node 4 Catego!Y % Category % NO 70.308 6178 NO 57.226 4918 YES 29.692 2609 YES 42.774 3676 Total 14.388 8787 Total 14.072 8594 I '-= JJIS Number of Other Felony Prior Charges I Node 5 Node 121 6 Catego!:Y % Catego!Y % NO 57.330 4908 NO 30.303 YES 42_670 3653 YES 69_697 Total 14.018 8561 Total 0.054 I L= Age at Intake 10 23 33 <= 16 638 )> 16 638 I Node 6 Node 525 Catego!:Y % Catego!:Y % n NO 50.639 1 782 NO 61.999 3126 YES 49.361 1737 YES 38.001 1916 Total 5.762 3519 Total 8.256 5042 "1± I L= JJIS Total number of charges pnor to program I <= 26 000 )> 26 000 I Node 526 Node 946 Catego!:Y % n Catego!:Y % NO 64_942 2297 NO 55_083 YES 35.058 1240 YES 44.917 Total 5.792 3537 Total 2_464 L± I 8/21/2015 5
Risk Assessment at the Florida Department of Juvenile Justice Prevention Assessment Tool (PAT) Positive Achievement Change Tool (PACT) Residential Positive Achievement Change Tool (R-PACT) 8/21/2015 6
Sample PACT Overview Report PACT - Overview Report Name: Chopper Test DOB: 5/1/1990 DJJID: 804266 Created By : FL State Acct Administrator Last Modified By: FL State Acct Administrator Created Date: Aug 9 2006 1:23PM Last Modified Date: Aug 10 2006 1:49PM Overall Level of Risk tore-offend: Low Record of Referrals Risk Score: 3 Social History Risk Score: 4 Risk Factors 25% 50% Static and Dynamic Combined Protective Factors 75% 100% Domain 0% 25% 50% 75% GA : History of Relationships SA: Employment History 7A: Family History 10: Attitudes/Behaviors 3A: School History 7B: Current Uving Arrangements SA: Alcohol and Drug History 1: Record of Referrals 3B: Current School Status 4A: Historic Use of Free Time 48: Current Use of Free Time SB: Current Employment GB: Current Relationships SB: Current Alcohol and Drugs 9A : Mental Health History 9B : Current Mental Health 8/21/2015 7
PACT Validation Studies Chris Baird, Theresa Healy, Kristen Johnson, Andrea Bogie, Erin Wicke Dankert, and Chris Scharenbroch (2010). A Comparison of Risk Assessment Instruments in Juvenile Justice. U.S. Department of Justice Report # 244477. Michael T. Baglivio (2009). The Assessment of Risk to Recidivate Among a Juvenile Offending Population. Journal of Criminal Justice, 37; 596-607. Michael T. Baglivio, and Katherine Jackowski (2013). Examining the Validity of a Juvenile Offending Risk Assessment Instrument Across Gender and Race/Ethnicity. Youth Violence and Juvenile Justice, 11:1; 26-43 Kristin Parson Early, Gregory A. Hand, and Julia L. Blankenship (2012). Validity and Reliability of the Florida PACT RISK and Needs Assessment Instrument; A Three-Phase Evaluation. Justice Research Center, Inc. Contract P2085. Ira M. Schwartz, Peter York, Mark A. Greenwald, and Ana Ramos-Hernandez (2015). The Application of Predictive Analytics and Machine Learning to Risk Assessment in Juvenile Justice: The Florida Experience. (Forthcoming) 8
9
Matching Services 10
A Graduated Sanctions Model Residential Placement Redirection Probation Day Treatment Intensive PS C/R Day Treatment Redirection Intensive PS Teen Court Probation Diversion 26
EBP Mantra The right service For the right kid At the right time In the correct dosage 12
JJSIP Components Comprehensive Strategy Structured Decision Making Evaluation Standardized Program Evaluation Protocol (SPEP) 13
Tiers of Evidence The lowest form is anecdotal evidence; stories, opinions, testimonials, case studies, etc. The highest form is empirical evidence research, data, results from controlled studies, etc. We do not want to norm an entire system on anecdotal outliers 14
5 Principles of Effective Intervention Principle Risk: Need: Treatment: Responsivity: Fidelity: Intervention Target high-risk offenders. Treat risk factors associated with offending behavior. Employ evidence-based and research-proven treatment approaches and interventions. Tailor treatments to meet special needs. Monitor implementation quality and treatment fidelity. 15
Targeting High-Risk Offenders Risk Level and Treatment Recidivism Outcomes Level of Treatment Study Risk Level Minimal Intensive O Donnell et al. (1971) Baird et al. (1979) Andrews & Kiessling (1980) Bonta et al. (2000) Low High Low High Low High Low High.16.78.03.37.12.58.15.51.22.56.10.18.17.31.32.32 Source: D.A. Andrews & James Bonta (2003). The Psychology of Criminal Conduct. 3 rd Edition. Cincinnati, Ohio, Anderson Publishing Company. 16
Recidivism Rate for all Low Risk to Re-offend Youth by Placement Type Q)... ro 0:::: E V') > ""0 u Q) 0:::: 30 7S 20 15 10 5 ~...... ~ ~... / ~... _... ~ ~ All Low Risk Youth 0 Note: Data from 2012 Comprehensive Accountability Report (CAR) final files Recidivism rate for I DDS significantly lower than all other placement types for the low risk sample. Diversion and I DDS significantly lower than Probation Supervision. Probation Supervision, CBIS, Probation Enhancement rates statistically equivalent. Probation, CBIS, and Probation Enhancement rates significantly lower than Day Treatment, Redirection, Residential, and PCP. Day T reatment, Redirection, Residential, and PCP recidivism rates are statistically equivalent. Source: Michael T. Baglivio (2013). The Risk Principle. Florida Department of Juvenile Justice, Bureau of Research and Planning 17
Recidivism Rate for Low Risk Youth by "Needs" Level by Placement Type ~ Low Risk "Not Identified as High Needs" - Low Risk "Higlh Needs" Note: Data from 2012 Comprehensive Accountability Report (CAR) final files "High Needs" defined as youth greater than 1 standard deviation above the mean on t lhe Social History Score subcomponent of the PACT. Statistically significant differences found iin the recidivism rates for low risk "high needs" youth versus youth not identified as such for the following Placement Types: Diversion, I DDS, Probatiion Supervision, with llow risk "high needs" youth having significantly higher recidivism rates.. Differences in recidivism rates for Probata ion Enhancement, Day Treatment, Redirection, Residential, and Post Commitment Probation were not significant. Source: Michael T. Baglivio (2013). The Risk Principle. Florida Department of Juvenile Justice, Bureau of Research and Planning 18
Matched Low-Risk Youth Residential Probation Difference Recidivism Pre-matching 26.7% 16.1% 10.6%* Post-matching 26.7% 16.9% 9.8%* 28,681 Probation youth (low-risk) 1,726 Residential youth (low-risk) Matched on: Age at 1 st arrest Current drug/alcohol use Expulsion/drop out Violent felony Felony Antisocial peers/gang association County Race/ethnicity Gender Source: Analysis conducted by the Florida Department of Juvenile Justice, Office of Research and Data Integrity 19
Lipsey s 2009 Meta-analysis Interventions applied to high-risk delinquents produced larger recidivism reductions than when those interventions were applied to low-risk delinquents (p.23) There was no indication that there were juveniles whose risk level was so high that they did not respond to effective interventions (p.23) Source: Mark W. Lipsey (2009). The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta Analytic Overview. Victims & Offenders, 4:2; 124 147. 20
Common Risk Factors Predict Delinquency (The Big Eight) 1. Antisocial Attitudes 2. Antisocial Peers 3. Antisocial Personality Patterns (impulsivity, low self-control, risk taking) 4. History of Antisocial Behavior 5. Problems at School/Work 6. Problematic Family Circumstances 7. Problematic Leisure Activities/use of free time 8. Substance Abuse 21
Need Principle: Why Dynamic Priority Domains? Research shows a 38% reduction in recidivism when case plans contained interventions matched to assessed criminogenic needs for high risk youth. (Luong, D., & Wormith, J.S. (2011). 60 50 40 30 20 10 0 100 90 31 91 50 Interventions Matched to Needs Baseline The absence of interventions to address a domain that was ranked medium risk or higher was associated with an 82% increase in likelihood of recidivism. (Luong, D., & Wormith, J.S. (2011). 80 70 60 50 40 30 20 10 50 No Intervention For High Ranked Need Baseline 0 22
23
Disposition Recommendation Matrix Is a structured decision making tool that assists with matching youth to the appropriate level of service/supervision Is based on a matrix of risk to reoffend (PACT) and the presenting offense Consists of graduated sanctions The intensity of services increases as the risk level and offense severity increases 24
Key Points of the Disposition Matrix Low-risk offenders remain in the community with minimal supervision Moderate-risk offenders typically placed in more structured community programs, with intensive probation supervision for higher risk youth Residential placement reserved for the highest risk offenders after communitybased alternatives have been exhausted 25
Florida Department of Juvenile Justice Disposition Recommendation Matrix (Staff should begin with the least restrictive setting within a particular disposition category. See Structured Decision-Making guidelines.) Most Serious Presenting Offense Civil Citation Eligible1 Low Risk to Reoffend Level 1 PACT Risk Level to Reoffend Moderate Risk to Reoffend Level 1 Moderate-High Risk to Reoffend Level2 or 3a Level2 or 3a Level 2 or 3a-c Level 3a-c or 4 Level2 or 3a Level 2 or 3a-b Level 3a-c or 4 Level 3a-c or 4 Level 2 or 3a-b Level 2, 3a-c, or 4 Level 3a-c, 4, or 5 Level 3a-c, 4, or 5 - Eligibility for civil citation is outlined in F_S 985_12_ Youth deemed ineligible for civil citation (based on community standards) should be reviewed under the "Minor" offense category based on the PACT risk level to reoffend _ 2- All misdemeanor offenses_ 3 - Felony offenses that do not include violence_ 4 - Violent felony offenses (do not include misdemeanor assault and battery which are captured under "Minor")_ l evel1 -Alternatives to Arrest l evel 2 - Diversion & Non-DJJ Probation l evel 3 - Community Supervision l evel 4 - Non-Secure Residential Commitment (3a)- Probation Supervision level 5- Secure Residen tial Commitment (High & Maximum Risk Programs) (3b)- Probation Enhancement Services (ART, LifeSkills, etc_) (3c) - Day Treatment, MST, FFT, Minimum Risk Commitment 26
Disposition Matrix Validation 38,117 youth released in FY10-11. Below (n=691) Optimum (n=27,916) Appropriate (n=7,322) Above (n=2,188) Holds true for males, females, across race/ethnicity, and for all risk levels of youth. Source: Michael T. Baglivio, Mark A. Greenwald, and Mark Russell. (2014). Assessing the implications of a structured decision making tool for recidivism in a statewide analysis: A disposition matrix for court recommendations made by juvenile probation officers. Criminology and Public Policy, 14:1, 5 49. 27
28
Continuum Mapping Identify the available services within each county Map the identified available services according to service category within each county Identify the target population for each categorized service according to levels of the Disposition Recommendation Matrix 29
Data Collection 30
County Service Mapping Report http://www.djj.state.fl.us/research/delinquency-data/services-continuum-report/ 31
32
The Second JJSIP Component 33
Meta-Analysis: Dosage Group 5 Service (Score=30) Cognitive-Behavioral Therapy Target Weeks=15; Target Hours=45 Qualifying Supplemental Services: None (automatic 5 points added to score) Group 4 Service (Score=25) Group Counseling Target Weeks=24; Target Hours=40 Qualifying Supplemental Services: None (automatic 5 points added to score) Mentoring Target Weeks=26; Target Hours=78 Qualifying Supplemental Services: Behavioral Contracting/Management Behavioral Contracting; Contingency Management Target Weeks=24; Target Hours=72 Qualifying Supplemental Services: Mentoring, Mixed Counseling (individual, group, family, and/or vocational), Remedial Academic Program 34
Dosage (cont. 2) Group 2 Service (Score=10) Restitution; Community Service Target Weeks=12; Target Hours=60 Qualifying Supplemental Services: None (automatic 5 points added to score) Remedial Academic Program Target Weeks=26; Target Hours=100 Qualifying Supplemental Services: Job -Related Services (work experience, job preparation, and/or job training) Group 1 Service (Score=5) Individual Counseling Target Weeks=25; Target Hours=30 Qualifying Supplemental Services: None (automatic 5 points added to score) Job-Related Training Vocational Counseling Target Weeks=20; Target Hours=40 Qualifying Supplemental Services: Remedial Academic Services Job Training Target Weeks= 25; Target Hours=400 Qualifying Supplemental Services: Remedial Academic Services Work Experience Target Weeks=26; Target Hours=520 Qualifying Supplemental Services: Remedial Academic Services 35
36
Delinquency Arrests in Florida 1 120,000 110,493 100,000 97,144 80,000 85,448 78,345 74,871 60,000 40,000 20,000 0 FY 2010-11 FY 2011-12 FY 2012-13 FY 2013-14 FY 2014-15 1 Source: Florida Department of Juvenile Justice, Office of Research and Data Integrity; FY 2014-15 data is preliminary. Official delinquency arrest figures will be released in the 2015 Delinquency Profile Report (October, 2015). 37
Changes in Delinquency Arrests by Offense Seriousness, FY 2013-14 and FY 2014-15 1 Offense Category Sum of Offenses FY 2013-14 Sum of Offenses FY 2014-15 Difference % Change Felony 25,775 25,542-233 -1% Misdemeanor 34,786 31,809-2,977-9% "Other" 17,784 17,520-264 -1% Sum of Total 78,345 74,871-3,474-4% 1 Source: Florida Department of Juvenile Justice, Office of Research and Data Integrity; FY 2014-15 data is preliminary. Official delinquency arrest figures will be released in the 2015 Delinquency Profile Report (October, 2015). 38
PACT Risk to Reoffend for Youth Disposed to Commitment by Percentage of Youth Committed (FY 2010-11 through 2014-15) 1 60% 50% 40% 30% 72% 84% of current commitments involve moderate-high or high risk youth 20% 10% 0% FY 2010-11 FY 2011-12 FY 2012-13 FY 2013-14 FY 2014-15 Low Moderate Mod-High High 1 Source: Florida Department of Juvenile Justice, Office of Research and Data Integrity. 39
Operational Capacity for Residential Services at Onset of Fiscal Year Residential Commitment Beds 6,258 7,018 7,140 7,065 6,572 6,118 6,010 4,852 4,488 4,136 3,606 3,177 2,514 2,131 2,154 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 Fiscal Year Source: Florida Department of Juvenile Justice, Residential Services 40
References Donald A. Andrews and James Bonta (2003). The Psychology of Criminal Conduct. 3rd Edition. Cincinnati, Ohio, Anderson Publishing Company. Michael T. Baglivio, Mark A. Greenwald, and Mark Russell. (2014). Assessing the implications of a structured decisionmaking tool for recidivism in a statewide analysis: A disposition matrix for court recommendations made by juvenile probation officers. Criminology and Public Policy, 14:1, 5-49. Michael T. Baglivio, Katherine P. Jackowski, Mark A. Greenwald, and James C. Howell. (2014). Serious, Violent, and Chronic Juvenile Offenders: A Statewide Analysis of Prevalence and Prediction of Subsequent Recidivism Using Risk and Protective Factors. Criminology and Public Policy, 13:1, 83-116. Mark A. Greenwald, and Michael T. Baglivio (2015). Analysis of Serious, Violent and Chronic Delinquency in Florida. Florida Department of Juvenile Justice, Office of Research and Data Integrity. Mark W. Lipsey (2009). The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta- Analytic Overview. Victims & Offenders, 4:2; 124-147. 41
Other Department Resources The Office of Research and Data Integrity: http://www.djj.state.fl.us/research The Juvenile Justice System Improvement Project (JJSIP) in Florida: http://www.djj.state.fl.us/research/latest-initiatives/juvenile-justice-system-improvement-project-(jjsip) Delinquency Profile Report: http://www.djj.state.fl.us/research/delinquency-data/delinquency-profile Delinquency Briefings / Special Topics Research: http://www.djj.state.fl.us/research/fast-facts/delinquency-briefings Delinquency in Florida s Schools Research: http://www.djj.state.fl.us/research/reports/research-reports/delinquency-in-schools 42
43