Harnessing the Power of Evidence:

Transcription

1 Harnessing the Power of Evidence: Using the Results First Approach to Inform County Programming Decisions Ashleigh Holand, Manager, State Policy Pew-MacArthur Results First Initiative January 22, 2015

2 Panelists Cassaundra Friedberg Department Analyst Kern County Probation Department Andrew Davis Senior Departmental Administrative Analyst Santa Cruz Probation Department Tanja Heitman Deputy Chief Probation Officer Santa Barbara Probation Department

3 The Results First Approach: Bring Evidence into the Process 1. INVENTORY your current program investments 2. CONSIDER whether benefits justify costs 3. TARGET funds using rigorous evidence ACHIEVE dramatic improvements without increased spending

4 VT NH MA RI CT NJ DE MD AK HI WA TN MT OR ID WY CO UT NV CA AZ NM NE KS OK TX ME ND SD MN LA AR MO IA WI MI IL IN OH PA NY WV KY MS AL GA SC NC FL VA AK Participation in Results First

5 Use in the states Iowa Identify and replace ineffective programs New Mexico Identify and replace underperforming programs Massachusetts Identify gaps in service 5 States Used the approach to target $80 million in funding for good programs

6 The Results First Approach: Bring Evidence into the Process 1. INVENTORY your current program investments 2. CONSIDER whether benefits justify costs 3. TARGET funds using rigorous evidence ACHIEVE dramatic improvements without increased spending

7 The Results First Approach: Bring Evidence into the Process 1. Program Inventory 2. Benefit-Cost Analysis 3. Using your Results

8 The Results First Approach: Bring Evidence into the Process 1. Program Inventory 2. Benefit-cost Program Analysis means an intervention (program or practice) implemented to affect a discrete outcome Evidence-based means its level of effectiveness has 3. Using been your determined Results by rigorous evaluation(s)

9 Program Inventory PROGRAM INFORMATION BUDGET EVIDENCE-BASE Program Name Program Budget (FY13) % of FY13 Program Budget Evaluated in County Evaluated in Literature (Clearinghouses) Thinking for a Change $50,000 2% Vocation Education in Prison $300,000 16% Transcendental Meditation $75,000 4% Sober Living Environment $180,000 9% Adult Boot Camps $250,000 13% Veterans Courts $70,000 4% All Others $1,000,000 52%

10 Clearinghouse Database Program Area Intervention Blueprints for Healthy Youth Development California Evidence- Based Clearinghouse for Child Welfare CrimeSolutions.gov Promising Practices Network NREPP Sexual/Dating Violence 4th R Curriculum Promising Early Childhood Education Abecedarian Project Proven Mental Health: Adult Acceptance and Commitment Therapy (ACT) 1: Well-Supported 3.7 General Prevention All Stars No Effects 2.2 Juvenile Justice Functional Family Therapy (FFT) Model 2: Supported Effective Student Behavior/ Character Education Good Behavior Game Promising Effective 3.2

11 Funding by evidence base PROGRAM INFORMATION BUDGET EVIDENCE-BASE Program Name Program Budget (FY13) % of FY13 Program Budget Evaluated in Literature (Clearinghouses) Thinking for a Change $50,000 2% Evidence-based Vocational Education in Prison Transcendental Meditation $75,000 4% $300,000 16% Evidence-based Strong positive: high rigor Sober Living Environment $180,000 9% Promising Adult Boot Camps $250,000 13% No Effects Veterans Courts $70,000 4% N/A All Others $1,000,000 52% N/A 22%: Highest-rated 9%: Second-highest rated 13%: No Effects 56%: No Evidence

12 Panelist discussion 1. Program Inventory 2. Benefit-cost Analysis 3. Using your Results

13 Kern County Program Inventory Sample

14 Santa Cruz County Results First Program Inventory 1. Evidence-Based Programs with Full Match to Results First Categories Program Type Adult Programs Santa Cruz County Program Description Provider Primary Participant Population Cognitive Behavioral Therapy Moral Reconation Therapy and Thinking For a Change Multiple evidence-based curricula implemented by certificated staff Barrios Unidos, Volunteer Center, Probation AB109 and general inmate population Electronic Monitoring Custody Alternative Program EMP, including GPS and transdermal Sheriff's Office 2. Evidence-Based Programs with Partial Match to Results First Categories General inmate population screened for exclusions Inpatient/Intensive Outpatient Drug Treatment (Incarceration) RISE (Rountree) In-custody mindfulness-based substance abuse treatment Community Action Board AB109 and Rountree general population Outpatient/Non-Intensive Drug Treatment (Community) Outpatient Substance Abuse Treatment <9 hours/week individual or group SUD counseling Alto, Fenix, New Life AB109 (1170 and PRCS) assessments 3. Research-Based Promising Practices Not Matching Current Results First Categories Promising (local data) Promising (local data) Maintaining Ongoing Stability through Treatment (MOST) Downtown Accountability Program Intensive case management for severely mentally ill offenders Intensive supervision, case management, for chronic local ordinance violators 4. Non-Evidence-Based Programs (Includes Responsivity and Services that Enable Evidence-Based Programs to Operate) Non-Evidence-based DUI Classes Housing support Court-ordered 3 to 18 month class Emergency and transitional housing and support services Alto, Fenix, Janus Health Servcies Agency, Probation, Corrections City / County, Court, Encompass Homeless Service Center, River St. Shelter, Pajaro Valley Shelter Services Criminal justice Individuals with serious mental illness Individuals with chronic local ordinance violations DUI Classes

15 Panelist discussion 1. Program Inventory 2. Benefit-cost Analysis 3. Using your Results

16 The Results First Approach: Bring Evidence into the Process 1. Program Inventory 2. Benefit-Cost Analysis 3. Using your Results

17 benefit-cost model Use the best national research to identify what works Predict the impact in your county Calculate long-term costs and benefits

18 EXAMPLE: Meta-analysis of Functional Family Therapy (FFT) Recidivism Rate 80% 60% 40% RECIDIVISM RATES REDUCED BY 22% 20% 0% Without FFT (actual baseline) With FFT Follow-up Years Source: Washington State Institute for Public Policy

19 Benefit breakdown: FFT OUTCOMES FROM PARTICIPATION MAIN SOURCE OF BENEFITS Reduced crime $29,340 Lower state & victim costs Increased high school graduation $9,530 Increased earnings Reduced health care costs $398 Lower public costs Total Benefits $37,587 Cost $3,333 Net Present Value $34,254 Benefits per Dollar of Cost $11.28 Source: Washington State Institute for Public Policy

20 Build on program inventory Program Name Program Budget Participant Cost Thinking for a Change $50,000 $419 Vocational Education in Prison $300,000 $1,599 Evidence Evidencebased Evidencebased Electronic Monitoring $75,000 $1,093 Effective Sober Living Environment $180,000 $4,500 Promising Drug Court $820,000 $4,276 Promising Domestic Violence (non CBT) $600,000 $1,390 No effects Adult Boot Camps $250,000 $408 No effects Veterans Courts $70,000 $4,101 N/A

21 Predict and monetize outcomes Program Name Program Budget Participant Cost Thinking for a Change $50,000 $419 Vocational Education in Prison $300,000 $1,599 Evidence Evidencebased Evidencebased Long-term Benefits $9,954 $19,531 Electronic Monitoring $75,000 $1,093 Effective $24,840 Sober Living Environment $180,000 $4,500 Promising -- Drug Court $820,000 $4,276 Promising $10,183 Domestic Violence (non CBT) $600,000 $1,390 No effects -$7,527 Adult Boot Camps $250,000 $408 No effects -- Veterans Courts $70,000 $4,101 N/A --

22 Compare costs and benefits Program Name Program Budget Participant Cost Thinking for a Change $50,000 $419 Vocational Education in Prison $300,000 $1,599 Evidence Evidencebased Evidencebased Long-term Benefits Benefit-Cost Ratio $9,954 $24.72 $19,531 $13.21 Electronic Monitoring $75,000 $1,093 Effective $24,840 $22.72 Sober Living Environment $180,000 $4,500 Promising Drug Court $820,000 $4,276 Promising $10,183 $3.38 Domestic Violence (non CBT) $600,000 $1,390 No effects -$7,527 -$4.41 Adult Boot Camps $250,000 $408 No effects Veterans Courts $70,000 $4,101 N/A -- --

23 Panelist discussion 1. Program Inventory 2. Benefit-Cost Analysis 3. Using your Results

24 Results First Santa Cruz County Recidivism Rates - 1 to 6 year rates 50% 43.2% 40% 36.9% 39.6% 32.3% 30% 23.1% 22.6% 20% 14.6% 15.9% 18.6% 19.8% 10.6% 10% 6.4% 0% Baseline Recidivism Rate (ALL) Felony Only

25 Recidivism by Type of Crime What Crime resulted in Recidivism Behavior Murder Robbery 0.3% 0.5% Sex Crimes 0.5% Assault 5.6% Property 18.0% Misdemeanor 47.4% Drug/Other 27.8%

26

27 Program/Practice Benefits per participant Cost per participant Net Benefit per participant Benefit-Cost Ratio Recidivism Reduction Outpatient Drug/Alcohol Treatment Electronic Monitoring (in lieu of jail) Reasoning and Rehabilitation $1,849 $1,713 $137 $ % $10,301 $1,603 $8,697 $ % $4,089 $210 $3,879 $ % WAGE$$ $2,422 $193 $2,229 $ % Supervision using Risk-Need-Responsivity $7,775 N/A $7,775 N/A -18.4%

28 Using the Results First Approach 1. Program Inventory 2. Benefit-Cost Analysis 3. Using your Results

29 Results First Report Out June 2014 Page 1

30 Results First Report Out June 2014 Page 2

31 Results First Report Out Jan Page 1

32 Results First Report Out Jan Page 2

33 Using the Results First Approach 1. Program Inventory 2. Benefit-Cost Analysis 3. Using your Results

34

35

36 Panelists Cassaundra Friedberg Department Analyst Kern County Probation Department Andrew Davis Senior Departmental Administrative Analyst Santa Cruz Probation Department Tanja Heitman Deputy Chief Probation Officer Santa Barbara Probation Department

37 Ashleigh Holand, Manager