Research Update Kevin S. Douglas, LL.B., Ph.D. Simon Fraser University ProActive ReSolutions Topics for the Day HCR-20 Research V2 Summary HCR-20 V3 Key Changes Version 3 Risk Factors Research on HCR-20 V3 Concurrent validity (between V2 and V3) Interrater reliability Predictive validity Dynamic risk Relevance ratings
SPJ Model Based on prevention paradigm Static and dynamic risk factors Link risk factors to specific interventions Logical selection of risk factors Review of scientific literature Not sample-specific Comprehensive coverage of relevant risk factors Operational definitions of risk factors Explicit coding procedures Promotes reliability Assessment Procedure Meant to be integrated into existing practice Review/use multiple sources of information File, interview, observation, collaterals Team-based approach possible Case managers, mental health professionals, nurses, social workers, occupational health and safety, substance use professionals, correctional professionals Some items require mental health professionals Dynamic items need regular updating Decision Steps The purpose is to facilitate an evidence-based, structured professional case formulation about What risk factors are present? How are they relevant to this person s violent behavior? Why did this person commit violence? What are the future scenarios of concern? What treatment, supervision, or management strategies can we apply to these risk factors? What level of risk?
Core Assumptions (V2 and V3) Risk factors and risk judgments can be made with reasonable reliability Risk factors are associated with violence Summary risk ratings (SRRs of low, moderate, high risk) are associated with violence SRRs are equally if not more strongly related to violence relative to sums or optimizations of risk factors Dynamic risk factors change over time Changes in dynamic risk factors are associated with changes in subsequent violence HCR-20 (Historical-Clinical-Risk Management 20) HCR-20 Research 1995 V1 Published 1996 1997 First 4 Studies 1998 V2 Published 2000s 2013 Hundreds More First Article V3 Published
HCR-20 Research 250 Disseminations 175 Data Sets 35 Countries 20 Translations Most Common Singh et al (2014) 12% 21% Se ttings 1% 8% 58% Forensic Prison Mixed Civil Juvenile Research Topics Dynamic Risk Risk Ratings Risk Factors Reliability and Concurrent Validity Meta-analysis (Guy, 2008; Guy, Douglas, Hart, & Edens, in prep) Meta-analysis of 166 independent SPJ studies 172 disseminations, 2903 effect sizes, N = 26,903 Evaluated SPJ numeric scores versus summary risk ratings Numerous moderators Measure-specific performance
HCR-20: Average AUCs Outcome Numeric Score Summary Risk Ratings (L,M,H) Antisocial.70 (k = 96).71 (k = 23) Vi o l en ce.70 (k = 71).74 (k = 21) Notes: 1. Not moderated by: gender, setting, country 2. SRRs tend to add incrementally to numeric use Yang et al. (2010) Meta-analysis of 9 risk instruments, including PCL measures 28 studies that used more than one measure Used PCL-R as a benchmark to compare other measures against Instr um ent # Repor ts # Par ticipants Effect Size AUC r 1. HCR-20 V1.,2 16 4200.79.71.37 2. OGRS 2 2000.78.71.36 3. RM 2000V 3 1800.76.70.35 4. VRAG 17 4900.66.68.32 5. GSIR 3 1000.67.68.25 6. PCL: SV 8 2500.65.68.31 7. PCL-R/SV F2 13 4000.61.67.30 8. H10-HCR20 18 4700.61.67.29 9. R5-HCR20 12 4000.60.66.29 10. C5-HCR20 14 4100.59.66.29 11. VRS Dynamic HCR-20 (k = 16) and OGRS (k = 2) only two measures to add incrementally to 3 1100.57.66.28 the PCL-R 12. PCL-R 16 3900.55.65.27 13. VRS 4 1100.53.65.25 14. VRS-Static 3 1100.51.65.25 15. LSI-R 3 400.51.65.25 16. PCL-R/SV (F1) 13 3900.22.56.11
Other Meta-Analyses HCR-20 performs at least as well as other instruments Campbell et al. (2009) Guy et al. (2010) Yang et al. (2010) Singh et al. (2011) Fazel et al. (2012) Incremental validity viz PCL-R/SV Guy et al. (2010) Yang et al. (2010) Dynamic Risk? Clinical Scores (Adm vs Dis) (Douglas & Belfrage, 2001) 60 50 40 30 20 Admission Discharge 10 0 0 1 2 3 4 5 6 7 8 9 10
Change and Violence (N = 174 forensic patients; Douglas, Belfrage, & Strand, 2011) Change between T1 and T2 predicted violence after T2 (Model: -2LL = 125.81; Nagelkerke R 2 =.14; p =.003) Statistically significant linear decrease, p <.001, eta 2 =.12 Do People Change Differently? If risk changes, does violence too?
If risk changes, does violence too? Michel et al. (2013) 153 general and forensic patients Five community follow-ups @ six months Item level change predicts violence For 6 of the 10 HCR-20 Clinical and Risk Management risk factors
Final Risk Judgments? Final Risk Judgments HCR-20 SPJ Judgments and Violence 20 samples (N = 2,079) Mdn AUC =.78 (0.55. 0.56, 0.63, 0.64, 0.65, 0.69, 0.7, 0.7, 0.77, 0.78, 0.78, 0.79, 0.79, 0.8, 0.81, 0.83, 0.85, 0.86, 0.89, 0.91) Forensic Psychiatric, Community Douglas, Ogloff, & Hart (2003), Psychiatric Services Research questions Reliability and validity of structured clinical risk ratings Method 100 forensic psychiatric (NCRMD) patients released from maximum security institution Overlapped coding on half of patients (n=50) to permit interrater reliability analyses Violence measured through criminal records and records of re-admission to forensic hospital
Validity: Frequency of Violence Across Risk Judgments N=100 Risk Level Any Phys. Low (n=23) 2 (9%) Mod (n=64) 12 (19%) 1 (4%) 7 (11%) High (n=13) 8 7 (62%) (54%) Base rates 22% 15% Douglas, Ogloff, & Hart (2003) SPJ vs Actuarial (Hierarchical Cox proportional hazard analysis) Physical violence H, C, and R scales entered 1 st χ 2 = 9.9, p <.05 HCR-20 summary ratings (L, M, H) entered 2 nd Significant model improvement ( χ 2 = 9.8, p<.01) Overall model χ 2 = 20.07, p <.0001 Only the summary risk ratings remain significant e B = 9.44, p <.003 Discussion of HCR-20 V2 Research As or more strongly related to violence than other measures Professionals judgments add incrementally to numeric use and actuarial instruments Works comparably across countries and continents Works comparably for men and women Sensitive to change Change predicts violence
HCR-20 Version 3 Primary Changes Increased focus on formulation Relevance ratings Item changes Sub-items PCL-R no longer required Measuring Relevance (HCR-20 V3, Douglas, Hart, Webster, & Belfrage, 2013) A risk factor is relevant to an individual s risk for violent behavior if it: a) was a material contribution to past violence; b) is likely to influence the person s decision to act in a violent manner in the future; c) is likely to impair the individual s capacity to employ non-violent problem solving techniques; d) is critical to manage this factor in order to mitigate risk.
Risk Factors H1. Violence (H1a. Child; H1b. Youth H1c. Adult) History of Problems with H2. Other Antisocial Behavior (H2a. Child; H2b. Youth H2c. Adult) H3. Personal Relationships H3a. Intimate Relationships H3b. Non-intimate Relationships H4. Employment H5. Substance Use H6. Major Mental Disorder H6a. Psychotic Disorders H6b. Major Mood Disorders H6c. Other Major Mental Disorder H7. Personality Disorder H7a. Antisocial, Psychopathic, and Dissocial H8. Traumatic Experiences H7b. Other Personality Disorders H8a. Victimization/Trauma H8b. Adverse Childrearing Experiences H9. Violent Attitudes H10. Treatment or Supervision Response C1. Insight C1a. Mental Disorder C1b. Violence Proneness C1c. Need for Treatment Recent Problems with C2. Violent Ideation or Intent C4c. Cognitive C5. Treatment or Supervision Response C5a. Compliance C4. Instability C5b. Responsiveness C3. Symptoms of Major Mental Disorder C4a. Affective C3a. Psychotic Disorders C3b. Major Mood Disorders C3c. Other Major Mental Disorders C4b. Behavioral
R1. Professional Services and Plans R2. Living Situation R3. Personal Support Future Problems with R4. Treatment or Supervision Response R4a. Potential Problems with Compliance R4b. Potential Problems with Responsiveness R5. Stress or Coping First Phase Empirical Evaluation 8 samples 5 forensic, 1 civil, 2 correctional 6 countries Canada, Netherlands, Norway, Sweden, United Kingdom (England & Wales), United States 782 participants (~40 beta-testing clinical evaluations) Reliability Concurrent Validity Predictive Validity Belfrage & Douglas (Sweden) Strub & Douglas (Canada) de Vries Robbé & de Vogel (N etherlands ) Doyle, Shaw, & Coid (England & W ales ) Douglas & Strub (Canada) Strub & Douglas (Canada) Belfrage & Douglas (Sweden) de Vries Robbé & de Vogel (N etherlands ) Smith & Edens (U SA) Eidhammer, Selmer, & Bjørkly (N orway) de Vries Robbé & de Vogel (N etherlands ) Douglas & Strub (Canada) Doyle, Shaw, & Coid (England & W ales ) N = 147 N = 395 N = 579
Concurrent Validity, V2 and V3 Total H C R SRR Range.84-.93 (Mdn.86) Range.60-.91 (Mdn.86) Range.59-.78 (Mdn.76) Range.67-.82 (Mdn.74) Range.94-.98 (Mdn.96) Douglas & Belfrage (2014) N = 35 forensic patients Three raters of each patient Interview + file review HCR-20 V2 + HCR-20 V3 Questions: Interrater reliability? Relationship between V2 and V3? Results Interrater reliability of HCR-20 V3 Scale/Judgment ICC1 ICC2 H.94.98 C.86.95 R (In).69.87 R (Out).75.90 HCR Total (In).94.98 HCR Total (Out).94.98 Final Judgment (In).81.93 Final Judgment (Out).75.90
de Vogel et al. (2014) N = 86 forensic patients Retrospective follow-up, 3 years HCR-20 V2, HCR20 V3, SAPROF Questions: Interrater reliability? Relationship between V2 and V3? Predictive validity? Results Interrater reliability ICC (single measure): HCR-20 total score =.83 HCR:V3 total score =.84 Final risk judgment 3 pt =.72 Final risk judgment 5 pt =.72 Results Predictive validity (AUC) 1 year (7 recidivists) 2 years (11 recidivists) 3 years (16 recidivists) HCR-20.80**.74**.67* HCR:V3 total.77*.75**.67* HCR:V3 average.75*.74*.66* * = p.05; ** = p <.01
Doyle et al. (2014) 387 UK forensic patients 12 mo, unselected File + interview Prospective 6 + 12 mo follow-up IRR =.73 Total Presence Ratings AUCs 6 mo:.73 12 mo:.70 Regression Significant controlling for gender and age Strub, Douglas, & Nicholls (2014) 106 civil psychiatric patients and offenders* File + interview Prospective 4-6 week, and 6-8 month, follow-ups Focus on Summary Risk Ratings SRR (Combined) r AUC 4-6 Weeks.43.81 6-8 Months.41.73 SRR (Psychiatric) r AUC 4-6 Weeks.53.91 6-8 Months.48.74 SRR (Offenders) r AUC 4-6 Weeks.34.72 6-8 Months.33.68 *Note: Regression analyses indicated no moderation by sample or gender Violence and Risk Ratings % Violent % Violent 50 80 45 40 35 30 70 60 50 25 20 15 10 5 % Violent 40 30 20 10 % Violent 0 Lo w Moderate High 0 Lo w Moderate High 4-6 Weeks 6-8 Months
Incremental Validity of SRRs As with Version 2, V3 SRRs added incrementally to the sum of risk factors Example (4-6 weeks; logistic regression) Sum of risk factors entered 1 st χ 2 = 15.37, p <.001; Negelkerke R 2 =.24 HCR-20 V3 SRRs (L, M, H) entered 2 nd Significant model improvement ( χ 2 = 4.8, p=.03) Overall model χ 2 = 20.17, p <.001 Only the summary risk ratings remain significant e B = 3.42, p <.05 Subsequent Studies Haag (2016). Interrater reliability N = 32 forensic patients (Alberta, Canada) 81% male; 75% Caucasian Four raters of each patient File review only
Results Interrater reliability of HCR-20 V3 Scale/Judgment ICC2 HCR Total.97 Final SRR.88 Final SRR (Serious).90 Final SRR (Imminent).83 Additional Interrater Reliability Study Index (N) IRR Howe et al. (2015) Total (35).64 Kötter et al. (2014) Total (30).86 Eidhammer et al. (2014) Total (20).84 Smith et al. (2014) M Scales (15).81 Holzinger et al. (2013) SRR (30).86 Schneider et al. (2015) M Scales.84 M Relevance.84 Schneider et al. (2015) Kirby Forensic Psychiatric Center Predictive validity Inpatient violence (M = 15 mo FU) N = 105 (79% male; 49% African American) Index AU C p H.67.01 C.78.01 R.64.02 Total.75.01 *Relevance ratings highly correlated with presence ratings
Penney et al. (2016) Incremental Validity of Dynamic Risk (Presence) Do changes in C and R add incrementally to baseline risk (H)? Cox regression with time dependent covariates (change in C and R scales) N = 87 forensic patients, discharged 84% male; 43% Caucasian C and R measured at baseline, 1 & 6 months Violence measures at 12 months Penney et al. (2016) Incremental Validity of Dynamic Risk (Presence) Step 1 enter H Scale scores (p <.01) Step 2 enter change in C and R C Scale hazard ratio = 1.35 (p <.03) R Scale hazard ratio = 1.62 (p <.01) Hogan et al. (2016) Incremental Validity of Dynamic Risk (Relevance) Do changes in the relevance ratings of C and R add incrementally to baseline risk (H)? 99 forensic inpatients C and R coded pre- and post-treatment
Change in HCR-20 V3 Note: Study also included START, VRS, VRAG-R, and PCL-R Incremental Validity of Change Summary of HCR-20 V3 Research Interrater reliability Strong, based on ~225 cases Concurrent validity ~.70-.90, based on ~500 cases Predictive validity (~ 1000 cases) Presence ratings Summary risk ratings Dynamic risk (presence and relevance) Incremental risk
Where to Next? Continued testing of core assumptions Relevance Formulation and scenario planning Case management Gender and ethnicity Implementation Thank You! Contact info: douglask@sfu.ca