Where the Rubber Meets the Road: What do we know about what are the Evidence-Based Practices and what outcomes they impact?

Transcription

1 Where the Rubber Meets the Road: What do we know about what are the Evidence-Based Practices and what outcomes they impact? Faye S. Taxman Carolyn Watson Jennifer Lerch Amy Murphy Alese Wooditch Stephanie Ainsworth Nicole Jordan Center for Advancing Correctional Excellence! George Mason University

2 Acknowledgements NIDA U01 DA 16213; CJ-DATS1 is funded by NIDA in collaboration with: SAMHSA, CDC, NIAAA, and BJA NIDA U01 DA AHRQ HS Maryland Division of Parole and Probation

3 Paraphrased from Dr. Bennett Fletcher (NIDA)

4 We need to know what we know and what we need to develop a stronger evidence base

5 he Rubber Meets the Road: Evidence-Based Practices in the Criminal Justice System and strategies for multiple outcomes and how to handle discrep What do we know? How do we map our evidence?

6 Understanding Our Knowledge Base* Most interventions do NOT get evaluated. Consider Drug Courts: There are over 1,600 drug courts currently operating in the United States, but only 23 have been formally evaluated using minimum review standards; only 3 of these evaluations were experiments. Systemic reviews confirm that drug courts reduce recidivism but does not address whether they reduce drug use; the mechanism of action has not been identified The situation is even worse in the area of probation: despite the fact that we have over 4 million offenders on probation in the US, only a handful of research studies (and experiments) have been completed over the past thirty years. Because of this research shortfall, it is currently impossible to identify and rank the performance of various interventions for offenders Field is trying to identify the knowledge base, to advance performance of correctional agencies including resource allocation (see Sherman 2010) *Byrne & Taxman, 2010

7 Evidence-Based Reviews in Adult Corrections: A Look at the Campbell Collaborative Collection* 1. Prison-related Topics: 2 reviews: Effects of Cognitive-Behavioral Programs for Criminal Offenders: by: Mark W Lipsey, Nana A. Landenberger, Sandra Jo WilsonPublished: The Effectiveness of Incarceration-Based Drug Treatment on Criminal Behavior: by: Ojmarrh Mitchell, Doris Layton MacKenzie, David Wilson Published: Jail-related Topics: no reviews Byrne, 2010

8 Evidence-Based Reviews in Adult Corrections: A Look at the Campbell Collaborative Collection 3. Sentencing Topics: 2 reviews- Domestic Violence Interventions: by Lynette Feder, Sabrina Austin, David Wilson Published: The Effects of Custodial vs. Non-Custodial Sentences on Re-Offending: bymartin Killias, Patrice Villettaz, Isabel Zoder Published: Community corrections topics: Traditional Probation: no reviews Intensive Probation Supervision: no reviews Electronic Monitoring/ House Arrest; 1 protocol by Marc Renczemma Day Reporting Centers: no reviews Community Service: no reviews Boot camps: David Wilson, Doris Layton MacKenzie, Fawn Ngo MitchellPublished: Reentry: no reviews Residential Community Corrections: no reviews Other: 1 review of non-custodial employment programs: Impact on recidivism rates of ex-offenders Christy A Visher, Mark B Coggeshall, Laura Winterfield

9 But. We do not know how many systemic reviews are available, and how many apply to offender populations We do not know specifics about the systemic reviews and how they compare across standard measures We have underrelied on Meta-analyses or systemic reviews; they are a rich source of evidence (see Impact Factor for Cochrane Reviews)

10 But, we know there are limitations of relying on Existing Meta-Analyses * Broad vs. narrow definition of topics or program models; lack of coherent intervention model (not all programs are labeled correctly) Lack of RCTs or well-controlled studies in many areas Threshold of Methodological Quality Political and market interests (brand names) Translate into Effective information for policy makers Losel, Friedrich (2010)

11 And, Design of Systemic Reviews may not affect External Validity Design rules varies (comparative effectiveness vs treatment as usual (control)) Inclusion criteria varies type of populations Setting varies Intervention design varies Dosage seldom measured Internal validity seldom measured Lack of quality controls to ensure that intervention was implemented as intended No focus on scaling up which would include measuring within program changes

12 For example, What is inside the black box of labeling interventions Drug treatment courts: most evaluations do not consider the nature of treatment in the drug court model or the nature of judicial hearings (see Wilson, Mitchell, & MacKenzie, 2006) Motivational Interviewing for offenders: models vary from single session to MI used with group counseling (Murren, 2009) Cognitive Behavioral Therapy: multiple models; Landenburg & Lipsey (2006) teased out the components that appear more effective

13 Another Example of Limitations Study: Effects of drug substitution programs on offending among drug-addicts Authors: Martin Killias, Marcelo Aebi, Miriam Pina, Nicole Egli, Pernille Skovbo Christensen Inclusion Criteria: Randomized studies, quasi-experimental studies and before-after comparisons on the effects on reoffending of drug substitution programs will be included. Population: Had to be involved in Medications for opioids Comparison: With other medications (no control group) Programs Accepted: Interventions can be court ordered or unrelated to any involvement of the criminal justice system. Only interventions based on substitution programs (using e.g. methadone and/or opiates as substitution drugs) will be considered. Outcome Limitations: Only re-offending; no eventual medical outcomes; no comparison to behavioral or no intervention.

14 Mapping the Evidence, Mapping our Knowledge Base

15 Proposition Collect all meta-analyses & systemic reviews in the area of justice health Code to determine state of knowledge on standard set of criteria (to be determined) Map the coded data to see what we know Identify knowledge areas Identify Knowledge gaps Train students and researchers on the protocol beginning point of any literature review build a collective knowledge base Have each student/researcher publish findings Use this material for next year s meeting to review our knowledge base Build our knowledge collectively in a fashion

16 Contributors Brown University Yale University U of Mass, Lowell U of California, Irvine Ryder University York University George Mason University Sam Houston University University of Kentucky Temple University University of Denver

17 Our Topic Areas Access Focus on offenders ability to access services Programming Focus on programs (treatment/services) offered to offenders Utilization Enhancers Focus on elements that aid the effectiveness of programs how interventions work Outcomes Focus on the goals of programs, and whether they are achieved

18 Our Knowledge Base HIV Screening/Testing Access Screening Assessment Treatment Matching Referral Engagement Programming Bullying Prevention Mental Health Substance Abuse Prevention Physical Care Sex Offender Treatment HIV/AIDS Risk Behaviors Substance Abuse Tx Tobacco Cessation Community Intervention Family Intervention Correctional Intervention Working Alliance Community Coalitions Treatment Program Characteristics Utilization Enhancers Performance Contractors: P4P (Org) Quality Improvement Processes Contingency Management (Indiv) Organizational Culture Organizational Change PDSA Proximal Outcomes Continued Treatment Mental Health Criminal Behavior Distal Outcomes Relapse Prevention Mental Health Substance Abuse Risky Behaviors Recidivism Abstinence

19 Advance thinking about types of outcomes: Health Services Framework Individual Behavior Access Time to initiation Referral Washington Circle-- %within 14 days Engage Dosage Time Change in Motivation Washington Circle--% sessions in 1 month Retain in Intervention Dosage Time % Complete Outcomes Multiple Conflicting Proximal : Health, SA, MH, Employment Distal: Recidivism Interventions & Organizational Enhancers

20 What impacts criminal conduct? Health outcomes? Criminogenic needs (dynamic risk factors)? How can we map the evidence to achieve consensus and focus attention on practices?

21 Purpose of this meeting Review the recommended protocol and code sheet (Carolyn Watson & Jen Lerch) Review findings (Alese Wooditch) Review mapping process (Amy Murphy) Provide feedback on the draft materials Identify potential areas that you/your team might contribute Work towards developing a protocol Consider how to include measures of: Fidelity (Craig Henderson) Implementation (Steve Belenko) AHRQ standards (Ingrid Binswanger) Plan next year s meeting

22 Steps Identify topic area of interest Conduct Review Map the review evidence

23 Findings Overview # of reviews found Working Alliance Sex Offender Treatment School-Based Intervention Family Intervention Contingency Management

24 End Product Overview Summary of: How many reviews have been done in a topic area Whether overall findings have been positive, negative, or neutral Differences in effectiveness based on targeted population, setting, or outcome Visual display: Setting/Population, Intervention, Outcomes Three dimensional matrix (see Lum et al.) Public accessibility via ACE! website or any website

25 Lum, Telep, & Koper (2010). Available online at

26 Adapted Matrix for Justice Health Population or Setting Proximal or Distal; Access, Org Enhancers Intervention Adapted from Lum, Telep, & Koper (2010), available online at