CLINICAL VERSUS ACTUARIAL RISK ASSESSMENT OF OFFENDERS By: Gary Zajac, Ph.D. Managing Director, Justice Center for Research Senior Research Associate College of the Liberal Arts and University Outreach The Pennsylvania State University The 329 Building, Suite 222 University Park, PA 16802 Office: 814-867-3651 Fax: 814-863-3108 gxz3@psu.edu http://www.justicecenter.psu.edu/ Actuarial vs. Clinical Assessment 1
The accurate and objective screening and assessment of offender risk, needs and responsivity is one of the most important features of an effective correctional treatment system (Andrews and Bonta, 1994; Hiller, et al, 2011; Hollin, 2001). Providing a criminal justice disposition to an offender absent careful consideration of the offender s likelihood of reoffending (risk) and specific risk factors (needs) is akin to prescribing a drug to a patient without a diagnosis of what is wrong. How best to conduct such assessment has been an ongoing concern within the field of criminal justice. This question is embedded in the larger discussion in the field of psychology about the most effective approach to conducting any sort of individual assessment. Much of this discussion has focused upon the dichotomy of clinical versus actuarial assessment. Clinical assessment refers to the approach that has been used for generations to gain insight into the problems of any individual undergoing treatment. This involves a trained practitioner - the clinician - sitting down with a client who is being considered for treatment and asking that client a series of questions, or perhaps even having a more open-ended discussion. For the purposes of this discussion, a clinician may be a psychologist/psychiatrist, a social worker, a corrections counselor, a parole board member, or many other classes of human service professionals and providers. Clinical assessment, then, is essentially an interpersonal process that occurs between individuals, where one individual is charged with making a decision about, and/or providing services to, the other. The logic behind the clinical approach to assessment is that the clinician has a basis of experience, expertise and perhaps even natural insight that allows for an impressionistic interpretation of what is wrong with the client. The key limitation to this approach is that the questions that the clinician asks will often be subjective, inconsistent, unstructured and perhaps even only tangentially related to the problem under consideration; that is to say, they may or may not be good questions. Some clinicians may ask good questions most of the time, some may ask bad questions most of the time. More often than not, clinicians will be hit or miss in their assessments (some studies have found that clinical assessment produces prediction success rates that are worse than flipping a coin). The actuarial or mechanical approach to assessment attempts to control for the subjectivity and inconsistency inherent in the clinical approach by structuring a set of observations based upon discovered patterns of behavior across large numbers of cases. The observations (factors) that are recorded are driven by a statistical understanding of the relationship between the factor and the behavior in question. For example, if lung cancer is found to develop in a large percentage of cigarette smokers, but much less so in non-smokers, a cancer screening tool would likely ask questions about smoking habits (smoking would be a risk factor), among other things. An actuarial assessment tool asks the same set of questions of each individual, asks them in the same way, and interprets the answers consistently. An actuarial tool may very well involve an interview or conversation by a clinician with the individual being assessed, but the content of the interview is grounded upon known patterns in the data. Actuarial vs. Clinical Assessment 2
The clinical-actuarial dichotomy is actually a continuum, ranging from purely subjective, unstructured clinical interviewing, to clinical interviewing guided by an empirical understanding of risk factors to a fully codified assessment tool that in some cases may require little clinical expertise to administer. Most approaches to assessment, however, can be fitted into either the basic clinical or actuarial category, based upon their degree of structure and grounding in the literature. The relative outcomes of these two approaches have been studied for nearly 80 years. One of the earliest investigations of the subject was the work of Burgess, who examined parole outcomes for 3,000 Illinois offenders using a crude 21 point risk scale, compared with the predictions made by three prison psychiatrists (Burgess, 1928). On the whole, the assessment tool outperformed the psychiatrists in predicting parole failures, even though the psychiatrists had the advantage of being able to cherrypick the cases they wanted to review (the assessment tool was used on all cases). The seminal work on this topic was done by the psychologist Paul Meehl, who established the general superiority of actuarial approaches to risk prediction (Meehl, 1954). The findings of Meehl s original study have been supported by the much subsequent research. Grove and Meehl (1996) and Grove et al (2000) report on the findings of over 130 studies showing actuarial predictions to be more accurate than clinical predictions. Approximately 40 percent of the studies found substantially better predictive results for actuarial assessments compared with clinical assessments. Overall, actuarial prediction outperformed clinical prediction by about 10 percent. These studies included those with criminal justice outcomes (other types of predictive studies were included) and even studies where the clinician-raters had more information available to them than was available to raters using actuarial tools. It is telling that the clinicians education, level of experience and professional backgrounds made little difference to the accuracy of their predictions versus predictions made by actuarial tools. The actuarial predictions were the clear winner. Actuarial predictions have been found to outperform clinical judgment even in more specialized and difficult assessment settings. Hanson and Bussiere s (1998) large meta-analysis of sex offender risk prediction studies concluded that actuarial tools were more accurate than clinical approaches by a factor of four (actuarial tools produced a correlation with sexual re-offending of.46 versus.10 for clinical judgment). In a presentation to the Commonwealth of Pennsylvania Sexual Offenders Assessment Board, Thornton (2003) presented data showing that actuarial tools were nearly three times as predictive as unstructured clinical judgment. In sum, the literature offers a strong consensus that actuarial approaches outperform clinical assessments in most circumstances, even where the clinical assessors are highly trained and experienced (Jones, 1996). The primary source of superiority of actuarial approaches seems to be their grounding in the behavioral literature and the consistency and objectivity offered by standardized assessment instruments. It is difficult for even the most skilled clinician to maintain a high level of objectivity and consistency when rating large numbers of cases over time. In a study of Actuarial vs. Clinical Assessment 3
British parole decision makers who used pure clinical judgment, Bottomly (1973) found that the raters personal opinions of the prisoners personalities had more to do with their final estimates of the likelihood of parole success than did more empirically established factors such as prior record. In a sense, an actuarial tool distills the conventional wisdom about risk factors that is theoretically present in the clinician community, but imposes the rigors of scientific methods to this wisdom. The judgment of the clinician remains valuable, but it is much more so when informed and guided by an objective assessment tool. In the field of criminal risk prediction, tools such as the Level of Service Inventory-Revised (LSI-R) incorporate the substantial body of literature which finds significant correlations between re-offending and factors such as anti-social attitudes, criminal thinking, criminal history, criminal associates, employment, family stability and conventional lifestyle (Andrews and Bonta, 2001). These tools make predictions that take advantage of decades of research into the correlates of criminal behavior. The criminal justice community is well served by the use of such tools, and ignores them at its own risk. Indeed, as Meehl (1986) notes: surely we all know that the human brain is poor at weighting and computing. When you check out at a supermarket, you don t eyeball the heap of purchases and say to the clerk, Well it looks to me as if it s about $17.00 worth; what do you think? The clerk adds it up. If such rigor is to be applied to the purchase of groceries, we should expect nothing less from such a crucial arena as criminal justice risk prediction. The final decision about criminal justice dispositions, of course, remains with some individual clinician, whether it be a judge, parole board member, corrections counselor, etc. There is still much room for this clinician to interview the offender about the decision to be made. Such conversations can be better structured and validated, however, by use of the information gleaned from formal risk assessment tools. Such interviews can provide opportunities to probe more deeply into risk factors identified by an objective assessment tool and to query the offender about plans to address the assessed problems. To the extent that these clinician-offender interactions can be guided by a consistent and structured interview protocol, more accurate decisions will result. The body of evidence clearly supports such a conclusion. Sources Andrews, Donald A. and James Bonta. 2001. The Level of Service Inventory-Revised: User s Manual. North Tonawanda, NY: Multi-Health Systems, Inc. Andrews, Donald A. and James Bonta. 1994. The Psychology of Criminal Conduct. Cincinnati: Anderson Publishing. Actuarial vs. Clinical Assessment 4
Bottomly, A. 1973. Parole Decisions in a Long Term Closed Prison. British Journal of Criminology, 13, 26-40. Burgess, E.W. 1928. Factors Determining Success or Failure on Parole. In A.A. Bruce (ed.). The Working of the Indeterminate Sentence Law and the Parole System in Illinois (pp. 205-249). Springfield, IL: Illinois Committee on Indeterminate- Sentence Law and Parole. Grove, William M., David H. Zald, Boyd S. Lebow, Beth E. Snitz and Chad Nelson. 2000. Clinical Versus Mechanical Prediction: A Meta-Analysis. Psychological Assessment, 12(1), 19-30. Grove, William M. and Paul E. Meehl. 1996. Comparative Efficiency of Informal (Subjective, Impressionistic) and Formal (Mechanical, Algorithmic) Prediction Procedures: The Clinical-Statistical Controversy. Psychology, Public Policy and Law, 2(2), 293-323. Hanson, R. Karl. and Monique. T. Bussiere. 1998. Predicting Relapse: A Meta-Analysis of Sexual Offender Recidivism Studies. Journal of Consulting and Clinical Psychology, 66(2), 348-362. Hiller, Matthew L., Steven Belenko, Wayne Welsh, Gary Zajac and Roger H. Peters. 2011. Screening and Assessment: An Evidence-Based Process for the Management and Care of Adult Drug-Involved Offenders. In Carl G. Leukefeld, John. Gregrich and Thomas Gullotta. (eds.). 2011. Handbook on Evidence Based Substance Abuse Treatment Practice in Criminal Justice Settings. New York: Springer. Hollin, Clive R. (ed). 2001. Handbook of Offender Assessment and Treatment. New York: John Wiley. Jones, Peter. 1996. Risk Prediction in Criminal Justice. in Harland, Alan T. (ed.). Choosing Correctional Options that Work: Defining the Demand and Evaluating the Supply. Thousand Oaks, CA: Sage. pp.33-68. Meehl, Paul E. 1986. Causes and Effects of My Disturbing Little Book. Journal of Personality Assessment, 50, 370-375. Meehl, Paul E. 1954. Clinical versus Statistical Prediction: A Theoretical Analysis and A Review of the Evidence. Minneapolis: University of Minnesota Press. Thornton, David. 2003. Structured Risk Assessment Workshop. Presentation to the Commonwealth of Pennsylvania Sexual Offenders Assessment Board, Harrisburg, PA, March 28. Actuarial vs. Clinical Assessment 5