Assessing Violence Risk in Stalking Cases: A Regression Tree Approach

Law and Human Behavior, Vol. 29, No. 3, June 2005 ( C 2005) DOI: 10.1007/s10979-005-3318-6 Assessing Violence Risk in Stalking Cases: A Regression Tree Approach Barry Rosenfeld 1,2 and Charles Lewis 1 Advances in the field of risk assessment have highlighted the importance of developing and validating models for problematic or unique subgroups of individuals. Stalking offenders represent one such subgroup, where fears of and potential for violence are well-known and have important implications for safety management. The present study applies a Classification and Regression Tree (CART) approach to a sample of stalking offenders in order to help further the process of identifying and understanding risk assessment strategies. Data from 204 stalking offenders referred for psychiatric evaluation to a publicly-funded clinic were used to develop and assess putative risk factors. A series of nested models were used to generate tree algorithms predicting violence in this sample of offenders. Both simplified and more extensive models generated high levels of predictive accuracy that were roughly comparable to logistic regression models but much more straightforward to apply in clinical practice. Jack-knifed cross-validation analyses demonstrated considerable shrinkage in the CART, although the models were still comparable to many other actuarial risk assessment instruments. Logistic regression models were much more resilient to crossvalidation, with relatively modest loss in predictive power. KEY WORDS: stalking; violence; risk assessment. Controversies abound regarding the proper methods for assessing violence risk in mentally ill individuals and criminal defendants (Webster, Hucker, & Bloom, 2002). Fueling this controversy has been the proliferation of risk assessment techniques over the past decade, with dozens of instruments developed in the area of sex offender risk assessment alone (Barabee, Seto, Langton, & Peacock, 2001). Most of these assessment techniques fall into two basic types: Those that rely on an actuarial approach and those using structured clinical (or professional) judgment. Actuarial approaches to risk assessment seek to establish estimates of the probability of violence (either general, or a particular type of violence such as sexual violence) 1 Fordham University, Bronx, New York. 2 To whom correspondence should be addressed at Department of Psychology, Dealy Hall, Fordham University, Bronx, NY 10458; e-mail: rosenfeld@fordham.edu. 343 0147-7307/05/0600-0343/1 C 2005 American Psychology-Law Society/Division 41 of the American Psychology Association

344 Rosenfeld and Lewis based on statistical modeling, typically using an approach rooted in linear or logistic regression. Structured clinical judgment, on the other hand, involves a systematic review of known and presumed risk factors that are integrated (or weighted) by the clinician according to whatever internal algorithm the clinician deems appropriate. Not surprisingly, advocates of each approach have soundly criticized the other (e.g., Litwack, 2001; Quinsey, Harris, Rice, & Cormier, 1998). Only recently have writers argued that these two polemics can, and indeed should, be integrated together into a more holistic approach to violence risk assessment (Webster et al., 2002). For example, Webster et al. suggested that the probabilistic data rooted in actuarial risk assessment approaches represent a useful but incomplete source of data upon which to violence risk assessments are based. They, like many others, have called for further research focusing on both actuarial and clinical methods in order to more thoroughly understand the factors that may lead to violence for any particular subgroup of individuals. One area in which concerns regarding risk assessment have grown exponentially in the past few years is in cases of stalking and obsessional harassment. The scientific and legal literature pertaining to stalking offenses has risen from virtual non-existence as recently as a decade ago to the present, where dozens of empirical and theoretical articles are published annually encompassing a wide array of topics. O Connor and Rosenfeld (2004) divided this burgeoning literature into several categories, including legal analysis, experimental research (i.e., focusing on definitions and perceptions of stalking), prevalence and characteristics of stalking victims, clinical characteristics of stalking offenders, and most recently, predictors of violence in stalking offenders. Not surprisingly, the latter category has emerged only in the past few years and has been plagued by small samples, weak study methodologies, and limited (and limiting) data analytic techniques. Rosenfeld (2004) recently reviewed the literature on stalking violence, identifying a handful of variables that have been relatively well established as correlates of violence. These variables included a prior intimate relationship between victim and offender, the presence of threatening communications, a history of substance abuse on the part of the offender, the absence of a psychotic mental disorder, and the presence of a personality disorder diagnosis. In addition, demographic and clinical variables such as young age, limited education, and minority race, as well as the presence of a revenge motivation, have recently emerged as possible violence risk factors (Rosenfeld & Harmon, 2002) but have been less frequently studied. Although these variables might plausibly form the basis for a structured clinical judgment approach, these findings might also form the groundwork for an actuarial approach to assessing risk of stalking-related violence. However, the existing literature on violence risk factors in stalking has also been limited by a reliance on generalized linear models (e.g., logistic regression). Although these models may help clinicians identify the relevant correlates of violence, they are often less helpful in establishing a prediction model. For example, one limitation to the traditional multiple regression approach is that the results are framed in terms of weights that are, theoretically, to be applied to each predictor variable. Hence, practitioner interested in combining these variables must perform

Risk Factors in Stalking Cases 345 a series of calculations in order to transform the raw data into a predicted score or odds ratio. In addition, these models typically assume a one size fits all approach to prediction, whereby the same model is applied to each subject, ignoring the possibility that different variables might predict violence for different subgroups of individuals (Steadman et al., 2000). An alternative to the traditional regression approach (i.e., linear or logistic regression) that has occasionally been suggested utilizes a regression tree approach (often referred to as CART or Automatic Interaction Detection models) to identify subgroups of individuals with differing probabilities of, in this case, violence. Tree models use a sequential process to identify the predictor variables that best differentiate groups along the outcome variable of interest. The sample is then divided into two or more branches based on this predictor. Subsequent steps identify the best predictor within each of these branches and this process is repeated until no more variance can be explained with the remaining variables or some other criterion, such as a minimum group size, has been reached. The end points of these branches, referred to as nodes, represent subgroups of the original sample that differ in terms of the probability of the outcome variable. Because the same variables are not necessarily optimal for each branch of the tree, this process identifies interaction effects within the predictor variables that would typically be obscured or incomprehensible in a traditional regression approach, even when large samples are available. Although tree models do not resolve the debate between clinical and actuarial approaches, they do present actuarial data in a manner that is far more user friendly, by deriving a series of decision rules that optimize the discrimination between violent and non-violent offenders. Gardner, Lidz, Mulvey, and Shaw (1996) described the first application of CART models to the prediction of violence using data from a study of patients evaluated in a psychiatric emergency room. They compared the results of a CART model to models based on negative binomial regression and traditional regression approaches. They found that the three approaches were roughly comparable in terms of accuracy, although the CART approach was far simpler to apply in practice. More recently, Steadman and Monahan, and their colleagues (Banks et al., 2004; Monahan et al., 2000; Steadman et al., 2000) extended this approach, using an approach in which multiple classification trees were generated to optimize the discrimination among violent and non-violent psychiatric patients. They labeled tree nodes as high risk, low risk, or unclassified depending on the extent to which the probability of violence deviated from the sample base rate and then developed another tree to differentiate subgroups within the unclassified group. This approach ultimately resulted in several different decision trees that allowed for sophisticated differentiations within the sample. However, the iterative process generated a complex classification system that was arguably no simpler than a traditional regression approach. A handful of other studies have emerged in recent years focusing on predicting violence and criminal recidivism in criminal offenders (Silver & Chow- Martin, 2002; Stalens, Yarnold, Seng, Olson, & Repp, 2004), but these methods remain less frequent in forensic research compared to other clinical decision making settings (e.g., Barnes, Welte, & Dintcheff, 1991; Belle, Mendelsohn, Seaberg, &

346 Rosenfeld and Lewis Ratcliff, 2000; Boerstler & de Figueiredo, 1991; Doering et al., 1998; Lemsky, Smith, Malec, & Ivnik, 1996). The present study applies CART modeling techniques to a sample of stalking offenders and compares the resulting model to the results of a logistic regression model (Rosenfeld & Harmon, 2002). METHOD Procedure Data used in this study were collected from official records regarding criminal defendants referred for evaluation to the New York City Forensic Psychiatry Clinic between January 1, 1994 and December 31, 1998. This sample, which has been described in detail elsewhere (Rosenfeld & Harmon, 2002), will be summarized only briefly here. All offenders were referred to a city-run forensic mental health clinic for evaluation of either competence to stand trial or to aid in sentencing. Records from all offenders evaluated in this clinic during the 5-year study period were reviewed in order to identify those cases involving stalking or harassment. For the purposes of this study, the determination of a stalking case was based on the existence or report of a persistent and repetitive pattern of unwanted contact or harassment encompassing multiple contacts or attempts to contact the victim. The records reviewed included one or more psychiatric evaluations and corresponding notes (some offenders had been referred for more than one evaluation and/or were seen by more than one clinician). In addition, demographic and arrest data (i.e., rap sheets ) were obtained from the New York State Division of Criminal Justice Statistics and, in many cases, reports from the Department of Probation and victim statements describing the offender s conduct were available. Records were coded by a research assistant for a number of demographic (e.g., age, gender, race, place of birth, years of education), clinical (e.g., clinical diagnoses, estimated intelligence, history of substance abuse), and offense-related variables (e.g., type of harassment, motivation of the offender, victim offender relationship, criminal history). 3 Offenders were classified as violent if there was any reported physical contact or attempted physical contact toward the stalking victim or a third party, or if the harassment included confrontation with a weapon. A more stringent distinction was also made between violent offenders who attempted to injure the victim or used a potentially lethal weapon versus offenders who engaged in more minor violence (e.g., pushing, slapping). Fifteen percent of the cases were also reviewed by a second rater in order to assess the reliability of variable coding. Kappa coefficients for all variables exceeded.90 with the exception of clinical diagnosis, which was resolved through a review of existing data and consultation with the first author (described in more detail in Rosenfeld & Harmon, 2002). 3 A more detailed description of the coding system is presented in Rosenfeld and Harmon (2002), including information on how clinical diagnoses were established and conflicting diagnoses were resolved. Reliability analyses indicated a high degree of correspondence between the raters for all variables (kappa >.9 for all variables except diagnosis, which was resolved through consultation with the first author).

Risk Factors in Stalking Cases 347 Table 1. Variables Included in CART Models Model 1 Model 2 Model 3 Age (under 30 years) Age (under 30 years) Age (under 30 years) Education (<high school) Education (<high school) Education (<high school) Threatened victim Threatened victim Threatened victim Prior intimate relationship Prior intimate relationship Prior intimate relationship Revenge motivation Revenge motivation Revenge motivation Psychotic disorder Psychotic disorder Personality disorder Personality disorder Substance abuse history Criminal history Substance abuse history Criminal history Prior violence Below average intelligence Gender Foreign born Statistical Analysis The data were analyzed using a series of CART models (Brieman, Friedman, Olshen, & Stone, 1984) in which violence was considered the dependent variable. The model specifications required a minimum of five cases per node (the endpoint of a tree branch) and a minimum deviance of.01 in order to split any branch further (both of which are program default settings in the S-Plus classification program). Three different models were tested, each of which was nested within the next (i.e., expanding the number of predictors with each successive model but without removing any from the previous models). The most parsimonious model included five variables that had been previously identified through logistic regression analyses to be the strongest predictors of violence in this dataset (Rosenfeld & Harmon, 2002). 4 A second model was tested that included those same five variables, along with an additional four variables that have been identified in the broader empirical literature as the strongest correlates of stalking-related violence (Rosenfeld, 2004). Finally, a third model was tested that included all additional variables that had either a significant association with violence in this dataset or had been hypothesized to be significant in the existing theoretical literature. All predictors were binary. These prediction models are detailed in Tables 1 and 2. Although the modest sample size precluded formal cross-validation (i.e., applying the model to a new sample), a jack-knifed classification approach to cross-validation was utilized whereby 10% of the sample was removed without replacement and used to cross-validate the model derived from the remaining 90% and then repeated until the entire dataset had been included in the cross-validation sample. These CART models were compared with one another, as well as with the results of traditional logistic regression analyses. A Receiver Operating Charecteristic (ROC) analysis was applied to the results of each CART model, along with the corresponding logistic regression model 4 Five predictor variables were identified by Rosenfeld and Harmon (2002) using a step-wise logistic regression model. However, we opted to replace race in this model with revenge because of the practical and ethical implications of incorporating race into a risk assessment model. In addition, this variable provided the next most powerful contribution to the prediction model from the remaining pool of possible predictor variables.

348 Rosenfeld and Lewis Table 2. Predictive Accuracy of Alternative Models Model 1 Model 2 Model 3 AUC 90% CI AUC 90% CI AUC 90% CI Tree regression.787.734.840.836.791.882.848.804.892 Logistic regression.780.726.834.800.747.854.801.748.854 Tree cross-validation.659.593.725.644.578.710.649.582.715 Logistic cross-validation.744.686.802.725.664.787.706.641.771 and jack-knifed classification models, in order to quantify the overall accuracy of the prediction models. RESULTS Sample Characteristics During the 5-year study period, 204 individuals were evaluated for crimes related to stalking or obsessional harassment. The sample was predominantly male (n = 170, 82.9%), with an average age of 38.8 years (SD = 10.4) and an average of 12.9 years of education (SD = 3.1). The racial background of study participants was 43.5% Caucasian (n = 87), 30.5% black (n = 61), 20.5% Hispanic (n = 41), and 5.5% (n = 11) other (data were missing for four cases). 5 Half of the sample (50.0%, n = 102) had been previously arrested for crimes unrelated to stalking or harassment and 29.9% (n = 61) had either prior arrests or acknowledged past violent behavior. The most common psychiatric diagnoses included psychotic disorders (Schizophrenia, Delusional Disorder, and Psychotic Disorder Not Otherwise Specified, n = 81, 39.7%) and personality disorders (primarily Borderline, Antisocial, Paranoid, and mixed, n = 71, 34.8%). Alcohol and substance abuse were also frequently reported, with 16.7% (n = 34) of the sample having a substance abuse disorder diagnosis but fully half of all offenders having notations in their records suggesting alcohol or drug abuse. Not surprisingly, stalking victims in these cases were predominantly female (n = 139, 68.1%), and were often former intimate partners (n = 76, 39.1%), although non-romantic relationships, whether personal (e.g., acquaintances) or business (e.g., co-worker, attorney) were also common (n = 48, 24.8%). Strangers were the target of stalking in another quarter of these cases (n = 49, 25.3%). The stalker s motive was characterized as romantic in 65 cases (39.6%) and anger or revenge in 66 cases (40.2%). The most common forms of harassment reported were through the mail (75.5%, n = 154) and repeated telephone calls (60.8%, n = 124). Overt threats of violence were also quite common, occurring in 126 cases (61.8%). Seventy of the stalking offenders were classified as violent (34.3%) and 12 of these individuals engaged in acts of serious or life threatening violence. 5 Of note, we have previously demonstrated that this sample contains significantly more Caucasian offenders, and is significantly older than the typical New York City offender population (Rosenfeld & Harmon, 2002). However, past research has consistently noted similar findings, suggesting that this sample is likely to provide an accurate representation of the New York City stalking offender population.

Risk Factors in Stalking Cases 349 Models of Violence The first phase of this analysis involved a comparison of the three alternative, nested prediction models. The most parsimonious of these models included five dichotomous variables (prior intimate relationship, age under 30 years, high school graduate, past threats toward the victim, revenge motivation), all of which were based on either demographic characteristics of the offender or case-related variables (e.g., victim offender relationship, past threats, motivation of the offender). As noted above, these variables had all significantly differentiated violent and nonviolent offenders in univariate analyses, and are readily available to prospective decision-makers even when a clinical evaluation of the offender is not feasible. In addition, four of the five variables were statistically significant in a stepwise logistic regression model (Rosenfeld & Harmon, 2002). This analysis generated a model with 15 terminal nodes, with offender groups ranging from 4.3% violent to 70% violent (see Fig. 1). An ROC curve generated based on these estimates yielded an area under the curve (AUC) of.787 (95% CI:.734.840), indicating an adequate level of predictive accuracy (see Fig. 2). By contrast, an ROC curve based on a logistic regression model including these same five variables resulted in an AUC of.780 (95% CI:.726.834). Not surprisingly, Fig. 1. Tree model predicting violence: Model 1.

350 Rosenfeld and Lewis Fig. 2. ROC Curve predicting violence from tree, logistic regression models: Model 1. jack-knife cross-validation resulted in a substantially less accurate prediction model, with an AUC of.659 (95% CI:.593.725). The second model tested included nine predictor variables, five of which were included in the previous model. The additional predictors included three clinical variables, psychotic disorder diagnosis, personality disorder diagnosis, and substance abuse, and one historical variable, criminal history unrelated to stalking. This CART analysis yielded a model with 24 terminal nodes with considerably greater dispersion among the groups than was seen with the first model. Offenders in these 24 groups ranged from 0% violent to 100% violent (see Fig. 3). An ROC curve based on this model generated an AUC of.836 (95% CI:.791.882, Fig. 4) compared to.800 (95% CI:.747.854) for a logistic regression model based on the same variables. Despite the improvement in predictive accuracy for this model compared to the first, cross-validation indicated a slightly lower level of predictive accuracy (and therefore an even greater decrement from the model based on the full dataset), with an AUC of.644 (95% CI:.578.710). The loss of predictive power in the logistic regression model was much more modest, with an AUC of.744 (95% CI:.686.802). The final model tested included an additional cadre of variables that had either been associated with violence in univariate analyses (below average intelligence) or had theoretical relevance (e.g., gender, history of violence, foreign born). This analysis also yielded a model with 24 terminal nodes ranging from 0% violence to 100% violence. However, the resulting tree model did not improve significantly upon the previous model, generating and AUC of.848 (95% CI:.804.892), fully within the confidence interval of the previous model (see Table 2). Nevertheless, this model again compared favorably to the results of a logistic regression model incorporating the same variables, which generated an AUC of.801 (95% CI:.748.854). Cross-validation with this CART model was slightly better than with the previous model, generating an AUC of.649 (95% CI:.582.715). Cross-validation of

Risk Factors in Stalking Cases 351 Fig. 3. Tree model predicting violence: Model 2. Fig. 4. ROC Curve predicting violence from tree, logistic regression models: Model 2

352 Rosenfeld and Lewis the corresponding logistic regression model, on the other hand, generated an AUC of.725 (95% CI:.664.787). Risk Classification In order to further analyze the utility of the two best models (Models 1 and 2) we classified offenders into risk categories based on the probabilities generated by the respective tree models. Because there are no predetermined or optimal probability cut-offs for differentiated high risk and low risk individuals, we selected cut-off points that appeared to have some intuitive appeal relative to the overall probability of violence in our sample (34%). In addition, we analyzed both a threetiered classification and a five-tiered system based on the tree generated models. As evident in Table 3, both the three-tiered classifications (based on the first two tree models) divided the sample into roughly even terciles. This classification allowed for the identification of a large group of offenders who appear to represent a relatively low risk of violence (less than 15%), very few of whom engaged in acts of severe or life-threatening violence (only two of 66 offenders classified as low risk by the first model and one of 71 offenders classified as low risk by the second model). Table 3. Logistic Regression Models Model 1 Age (under 30 years) 1.01.008 Education (<high school) 1.21.001 Threatened victim 0.86.03 Prior intimate relationship 1.26.0002 Revenge motivation 0.79.07 Model 2 Age (under 30 years) 0.99.01 Education (<high school) 1.28.0007 Threatened victim 0.84.03 Prior intimate relationship 1.42.0002 Revenge motivation 0.52.15 Psychotic disorder 0.09.79 Personality disorder 0.11.80 Substance abuse history 0.29.43 Criminal history 0.57.12 Model 3 Age (under 30 years) 0.93.02 Education (<high school) 1.17.003 Threatened victim 0.88.03 Prior intimate relationship 1.40.0003 Revenge motivation 0.47.21 Psychotic disorder 0.15.66 Personality disorder 0.10.82 Substance abuse history 0.36.33 Criminal history 0.76.07 Prior violence 0.40.37 Below average intelligence 0.53.33 Gender 0.17.73 Foreign born 0.04.88 B p

Risk Factors in Stalking Cases 353 Identification of high risk offenders, on the other hand, appeared somewhat better using the first model (based on the five demographic and case-specific variables) compared to the second model (including diagnostic and historical variables). Although both models identified roughly one-third of the sample as high risk, there were substantially more of the severely violent offenders in the high risk category identified by the first model (7 of 69 offenders versus 3 of 57 in the second model). Similar findings emerged in the five-tiered classification system, with the first, more parsimonious model generating risk groups that appeared to better separate the high risk and low risk offenders. Likewise, half of the offenders who engaged in serious or life-threatening violence (6 of 12) were classified in the very high risk category based on the first tree model, which included only 41 (20.1%) of the sample. The second model, on the other hand, classified half of the most violent offenders as moderate risk and only three severely violent offenders fell in the very high risk category (Table 4). DISCUSSION Developing and validating models for differentiating violent from non-violent offenders remains a crucial task for forensic scientists and mental health clinicians. Analytic methods that rely on linear or logistic regression remain among the most popular mechanisms for pursuing these goals, although a number of limitations exist both in terms of sensitivity to complex interaction effects as well as ease of interpretation. We utilized an alternative approach based on regression tree models (also referred to as CART or Automatic Interaction Detection models) to differentiate high and low risk stalking offenders based on a number of variables that have previously emerged in the stalking literature. The results generated highly accurate classification models, with ROC analyses revealing an AUC of between.79 and.85 for the three models tested. Not only did these models perform comparably to those based on logistic regression analyses, but tree models appear far more intuitively appealing for clinical practice. It should be noted that we compared logistic regression models that contain only main effects against CART models that rely on interaction effects. Although arguably this comparison Table 4. Risk Classifications Based on Tree Models Model 1 Model 2 N/(percentage of total) Severe N/(percentage of total) Severe Three-level risk classification Low (<15% risk) 66/32.4% 2 71/34.8% 1 Moderate (15 50% risk) 69/33.8% 3 76/37.3% 8 High (>50% risk) 69/33.8% 7 57/27.9% 3 Five-level risk classification Very Low (<10% risk) 47/23.0% 1 63/30.9% 0 Low (10 20% risk) 37/18.2% 2 21/10.3% 1 Moderate (21 40% risk) 25/12.2% 2 53/26.0% 6 High (41 60% risk) 54/26.5% 1 34/16.7% 2 Very high (>60% risk) 41/20.1% 6 33/16.1% 3

354 Rosenfeld and Lewis handicaps the logistic regression models, inclusion of interaction terms renders logistic regression models untenable in clinical practice. Thus, in order to retain consistency with the extant literature on actuarial prediction (which has relied almost exclusively on main effects models), we opted to compare CART models against the logistic regression models that could feasibly be applied in clinical practice. Our comparison of three nested models revealed that the second model, containing nine variables that have previously been identified as significant predictors either in the extant literature or univariate analyses, was superior to both the more parsimonious model (five variables) and a more complex (13 variables) model. Notably, although the more parsimonious model generated a highly accurate classification tree, the second model (containing nine variables) generated a significantly better classification accuracy, as the 90% confidence intervals for their respective AUCs overlapped only slightly (and the point estimate for each model fell outside the confidence interval for the other model). The addition of several more variables, however, did not significantly improve the explanatory power of the resulting tree model. Despite the seeming superiority of the second model, we described both of the first two models for a number of reasons. First, although the second model included several seemingly important variables (e.g., psychosis, substance abuse), the addition of clinical variables to a risk assessment model limits its applicability in situations where the offender is not available for clinical evaluation. Indeed, when the offender is available for a thorough clinical evaluation, reliance on an actuarial risk assessment model such as those described here is often ill advised, since important individualized risk and protective factors cannot be considered. Second, by adding additional variables to a classification model, the stability of the resulting model typically decreases. This decrement was manifest in the decreased accuracy of the jack-knifed classification based on the second model relative to the first (although this difference was not statistically significant). Finally, despite the better overall classification accuracy, the second tree model appeared to have less utility in identifying those offenders who engaged in severe or life threatening violence whereas the first model was more effective. This finding may reflect the relatively greater significance of demographic and case-specific variables in differentiating severely violent offenders whereas diagnostic variables are less sensitive to this specific subgroup of violent offenders. However, because the subset of offenders who engaged in acts of severe violence was so small (n = 12), any conclusions regarding the variables that differentiate this subset of offenders is clearly premature. Moreover, although the relatively low rate of severe violence is perhaps heartening, large samples are obviously necessary in order to adequately derive prediction models specifically targeting these offenders. The reduction in fit from our cross-validation models also warrants some discussion, particularly given the magnitude in the decrement observed. Although our cross-validation method is arguably conservative (e.g., as compared to methods in which large samples are drawn with replacement, e.g., Steadman et al., 1998), it far exceeds the estimates offered by previous researchers (e.g., Gardner et al.,

Risk Factors in Stalking Cases 355 1996). Gardner and colleagues have suggested that the reduction in fit upon crossvalidation of CART models would likely be insignificant, estimating a decrease of roughly 1.5%. Instead, we observed a substantially greater decrease in classification accuracy based on our jack-knife analyses, with roughly 20% decrease in fit for the five-variable model and 30% for the nine-variable model. Of course, some of this decrease may reflect the smaller sample size necessary to perform such analyses (since classification models were based on only 90% of the sample size), but the relative impact of sample size issues versus model instability is simply unknown. Nevertheless, the fit estimates from cross-validation analyses were roughly comparable to the performance of other actuarial risk assessment techniques (e.g., Hanson & Thornton, 2000), suggesting that even our conservative cross-validation estimates support the utility of these models. Of course, the loss of predictive power for logistic regression models was much more modest than for the regression tree models, suggesting a possible advantage of this methodology. Nevertheless, without further analysis in novel samples, accurate conclusions regarding the relative superiority of CART versus logistic regression models is premature. In addition to unanswered questions as to which models are preferable, we presented two alternative risk classification schemes, one with three risk designations ( high, medium and low ) and one with five. Not only is there no correct answer as to whether a three-tiered or five-tiered classification system is preferable, but determining the appropriate cut-off points for making distinctions between high, moderate, and low risk individuals is inherently subjective. Hence, we elected to present two alternative classification systems, both of which relied on intuitively plausible cut-off points in order to enable clinicians and future researchers to make independent judgments regarding which designations are preferable for their particular setting or purpose. This decision reflects our belief that the initial stages of risk assessment models should, whenever possible, be descriptive in order to allow future researchers to assess the utility and adequacy of alternative models. Interpreting the actuarial models presented here requires more than simply following the branches of the tree diagrams depicted. Unquestionably, these models represent optimized predictions, based on the unique characteristics of this particular dataset. Although subsequent validation studies may support the validity of these tree models and the resulting violence risk estimates, there is little doubt that the probabilities represented here are merely estimates. Hence, despite the intuitive appeal of CART models, where determining the probability of violence associated with a particular stalking offender is relatively straightforward, appropriate use of these data demands that considerable caution be exercised in applying these results to clinical practice. Such caution may be particularly frustrating for evaluators faced with the task of assessing a stalking offender, as no risk assessment techniques specifically designed for this population exist at this time (Kropp, Hart, & Lyon, 2002). Yet these results should nevertheless be considered simply the first step in a long and complex undertaking that will, eventually, yield a reliable and valid risk assessment tool. A number of additional issues warrant note in attempting to discern the significance of these data. First, the impact of sample methodology on these results is

356 Rosenfeld and Lewis simply unknown. By studying only stalking offenders who were referred for mental health evaluation, a potential source of bias may have been introduced. Without comparison to a sample of offenders derived from police records, stalking victim reports, or other possible sources, the generalizability of these data is simply unknown. In addition, because the cases in this study included relatively few individuals who engaged in severe violence, we were unable to adequately model severe violence in particular. Thus, although we described the placement of severely violent cases in terms of risk classifications, it is not possible to separately determine the probability of severe violence from these data. Despite these limitations and caveats, this study demonstrates both the importance and utility of both CART and logistic regression models for identifying stalking offenders with a high likelihood of violence. Although clinicians are strongly cautioned against making decisions based on this first attempt at generating an actuarial model of stalking violence risk, these models can provide useful data to be integrated into the assessment process. Indeed, the estimates generated by such models may appropriately be considered a piece of data to be incorporated, along with other data, into clinical decision making. Moreover, the development of an actuarial model of stalking violence represents an important step in forming risk assessment for stalking and harassment cases. Further research will no doubt help refine and expand these models, with the goal of generating empirically valid risk assessment techniques. ACKNOWLEDGMENTS Several individuals have been instrumental in developing this database including Ronnie Harmon of the New York Forensic Psychiatry Clinic for helping facilitate the development and refinement of this database and Justine Schmullinger and Dana Levin who helped collect and code the data. In addition, Dr. Bruce Frederick and Susan Jacobson from the New York State Division of Criminal Justice Services provided official records for the sample. REFERENCES Banks, S., Robbins, P. C., Silver, E., Vesselinov, R., Steadman, H. J., Monahan, J., et al. (2004). A multiple-models approach to violence risk assessment among people with mental disorder. Criminal Justice and Behavior, 31, 324 340. Barabee, H. E., Seto, M. C., Langton, C. M., & Peacock, E. J. (2001). Evaluating the predictive accuracy of six risk assessment instruments for adult sex offenders. Criminal Justice and Behavior, 28, 490 521. Barnes, G. M., Welte, J. W., & Dintcheff, B. (1991). Drinking among subgroups in the adult population of New York State: A classification analysis using CART. Journal of Studies on Alcohol, 52, 338 344. Belle, S. H., Mendelsohn, A. B., Seaberg, E. C., & Ratcliff, G. (2000). A brief cognitive screening battery for dementia in the community. Neuroepidemiology, 19, 43 50. Boerstler, H., & de Figueiredo, J. M. (1991). Prediction of use of psychiatric services: Application of the CART algorithm. Journal of Mental Health Administration, 18, 27 34. Brieman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Boca Raton, FL: Chapman & Hall/CRC. Doering, S., Mueller, E., Keopcke, W., Pietzcker, A., Gaebel, W., Linden, M., et al. (1998). Predictors of relapse and rehospitalization in schizophrenia and schizoaffective disorder. Schizophrenia Bulletin, 24, 87 98.

Risk Factors in Stalking Cases 357 Gardner, W., Lidz, C. W., Mulvey, E. P., & Shaw, E. C. (1996). A comparison of actuarial methods for identifying repetitively violent patients with mental illnesses. Law and Human Behavior, 20, 35 48. Hanson, R. K., & Thornton, D. (2000). Improving risk assessments for sex offenders: A comparison of three actuarial scales. Law and Human Behavior, 24, 119 236. Kropp, P. R., Hart, S. D., & Lyon, D. R. (2002). Risk assessment of stalkers: Some problems and possible solutions. Criminal Justice and Behavior, 29, 590 616. Lemsky, C. M., Smith, G., Malec, J. F., & Ivnik, R. J. (1996). Identifying risk for functional impairment using cognitive measures: An application of CART modeling. Neuropsychology, 10, 368 375. Litwack, T. R. (2001). Actuarial versus clinical assessments of dangerousness. Psychology, Public Policy, and Law, 7, 409 443. Monahan, J., Steadman, H. J., Appelbaum, P. S., Robbins, P. C., Mulvey, E. P., Silver, E., et al. (2000). Developing a clinically useful actuarial tool for assessing violence risk. British Journal of Psychiatry, 174, 312 319. O Connor, M., & Rosenfeld, B. (2004). Introduction to the special issue on Stalking: Finding and filling the empirical gaps. Criminal Justice and Behavior, 31, 3 8. Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (1998). Violent offenders: Appraising and managing risk. American Psychological Association Press: Washington, DC. Rosenfeld, B. (2004). When stalking turns violent: Developments in the assessment of stalking risks. In M. Brewster (Ed.). Stalking Victims and Offenders: Treatment, Intervention and research. Kingston, NJ: Civic Research Institute (pp. 98 114). Rosenfeld, B., & Harmon, R. (2002). Factors associated with violence in stalking and obsessional harassment cases. Criminal Justice and Behavior, 29, 671 691. Silver, E., & Chow-Martin, L. (2002). A multiple-models approach to assessing recidivism risk: Implications for judicial decision making. Criminal Justice and Behavior, 29, 538 568. Stalens, L. J., Yarnold, P. R., Seng, M., Olson, D. E., & Repp, M. (2004). Identifying three types of violent offenders and predictinv violent recidivism while on probation: A classification tree analysis. Law and Human Behavior, 28, 253 271. Steadman, H. J., Mulvey, E. P., Monahan, J., Robbins, P. C., Appelbaum, P. S., Grisso, T., et al. (1998). Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Archives of General Psychiatry, 55, 393 401. Steadman, H. J., Silver, E., Monahan, J., Appelbaum, P. S., Robbins, P. C., Mulvey, E. P., et al. (2000). A classification tree approach to the development of actuarial violence risk assessment tools. Law and Human Behavior, 24, 83 100. Webster, C. D., Hucker, S. J., & Bloom, H. (2002). Transcending the actuarial versus clinical polemic in assessing risk for violence. Criminal Justice and Behavior, 29, 659 665.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.