Psychological Assessment

Transcription

1 Psychological Assessment Posttraumatic Stress Disorder in Veterans: The Utility of the MMPI 2 RF Validity Scales in Detecting Overreported Symptoms Brandee E. Goodwin, Martin Sellbom, and Paul A. Arbisi Online First Publication, April 1, doi: /a CITATION Goodwin, B. E., Sellbom, M., & Arbisi, P. A. (2013, April 1). Posttraumatic Stress Disorder in Veterans: The Utility of the MMPI 2 RF Validity Scales in Detecting Overreported Symptoms. Psychological Assessment. Advance online publication. doi: /a

2 Psychological Assessment 2013 American Psychological Association 2013, Vol. 25, No. 2, /13/$12.00 DOI: /a Posttraumatic Stress Disorder in Veterans: The Utility of the MMPI 2 RF Validity Scales in Detecting Overreported Symptoms Brandee E. Goodwin and Martin Sellbom The University of Alabama Paul A. Arbisi Minneapolis Veterans Affairs Medical Center, Minneapolis, Minnesota, and University of Minnesota The current investigation examined the utility of the overreporting validity scales of the Minnesota Multiphasic Personality Inventory 2 Restructured Form (MMPI 2 RF; Ben-Porath & Tellegen, 2008) in detecting noncredible reporting of symptoms of posttraumatic stress disorder (PTSD) in a sample of disability-seeking veterans. We also examined the effect of mental health knowledge on the utility of these scales by investigating the extent to which these scales differentiate between veterans with PTSD and individuals with mental health training who were asked to feign symptoms of PTSD on the test. Group differences on validity scale scores indicated that these scales were associated with large effect sizes for differentiating veterans who overreported from those with PTSD and for differentiating between mental health professionals and veterans with PTSD. Implications of these results in terms of clinical practice are discussed. Keywords: PTSD, MMPI 2 RF, validity scales, overreporting, response bias Supplemental materials: Individuals undergoing psychological evaluations may frequently be motivated to present themselves in an overly negative light in order to achieve a particular outcome (e.g., obtaining social security disability benefits or a not guilty by reason of insanity ruling; Sellbom & Bagby, 2008). Such response styles can result in unnecessary financial burden to society for treatment and compensation, as well as misallocation of scarce resources that could be used elsewhere (Friel, White, & Hull, 2008; Iverson, Franzen, & Hammond, 1995). Furthermore, based on the context of their evaluation, individuals may try to feign symptoms of a specific psychological disorder. For instance, veterans may try to feign posttraumatic stress disorder (PTSD) to obtain government benefits (Arbisi, Ben-Porath, & McNulty, 2006). In light of a 180% increase in PTSD claims from compensation-seeking veterans between 1999 and 2008, it is important that psychologists be able to distinguish false claims from those with merit (Doan & Morton, 2008). Self-Report Assessment of Overreporting To assist with detection of feigned psychological issues, many omnibus inventories of psychopathology (e.g., the Millon Clinical Brandee E. Goodwin and Martin Sellbom, Department of Psychology, The University of Alabama; Paul A. Arbisi, Minneapolis Veterans Affairs Medical Center, Minneapolis, Minnesota, and Departments of Psychiatry and Psychology, University of Minnesota. The original data collection was funded by a grant from the University of Minnesota Press, publisher of the MMPI 2 RF, awarded to Paul A. Arbisi. Correspondence concerning this article should be addressed to Martin Sellbom, Department of Psychology, Box , The University of Alabama, Tuscaloosa, AL msellbom@ua.edu Multiaxial Inventory [MCMI]; Millon, 1983 and the Personality Assessment Inventory [PAI]; Morey, 1991) include validity scales specifically designed to detect overreporting. The first such inventory to employ validity scales was the Minnesota Multiphasic Personality Inventory (MMPI; Baer & Miller, 2002; Hathaway & McKinley, 1940). Its successor, the Minnesota Multiphasic Personality Inventory 2 (MMPI 2; Butcher et al., 2001), also has validity scales that can be used to detect overreporting of psychopathology during psychological evaluations. Validity scales from these inventories have proven useful in detecting feigned PTSD in particular, and research to date seems to primarily support the MMPI 2 validity scales (see Demakis & Elhai, 2011, for a review). In addition to using the MMPI 2 validity scales, Demakis and Elhai (2011) highlighted the importance of examining collateral sources (e.g., official military records) and using other assessment instruments (e.g., cognitive symptom validity tests) before rendering a decision regarding malingered PTSD. The Minnesota Multiphasic Personality Inventory 2 Restructured Form (MMPI 2 RF; Ben-Porath & Tellegen, 2008) is a 338-item version of the MMPI 2 that was designed to capture the same clinically relevant construct variance as the MMPI 2 in a more psychometrically up-to-date and efficient manner. It includes revised versions of the MMPI 2 validity scales, along with two new overreporting scales, the Infrequent Somatic Complaints scale (F S ) and the Response Bias scale (RBS). The Infrequent Responses (F-r) scale combines elements of the F scale and Back F scale (F B) and provides a measure of general overreporting. The F-r scale is made up of 32 items endorsed by less than 10% of the MMPI 2 RF normative sample. The Infrequent Psychopathology Responses (F p -r) scale contains 21 items (18 of which are included on the MMPI 2 s F p scale) endorsed by less than 10% of the normative sample as well as by less than 20% of inpatient psychiatric samples. The Symptom Validity (FBS-r) scale retains 30 of 1

3 2 GOODWIN, SELLBOM, AND ARBISI the original 43 items on the MMPI 2 s FBS scale and measures overreporting of somatic and cognitive complaints. The Infrequent Somatic Complaints (F S ) Scale was designed for the MMPI 2 RF. The F S scale contains 16 items endorsed by less than 25% of the normative sample and a large sample of medical and chronic pain patients. The RBS is composed of 28 items, which distinguished between individuals who failed or passed cognitive symptom validity tests (e.g., the Word Memory Test; Green, Allen, & Astner, 1996). The utility of the MMPI 2 RF validity scales in detecting overreporting of psychological symptoms, including memory complaints (Gervais, Ben-Porath, Wygant, & Sellbom, 2010) and somatic complaints (Sellbom, Wygant, & Bagby, 2012; Wygant et al., 2009), is already well documented. The validity scales of the MMPI 2 RF have shown utility in a criminal forensic sample, with F- r and F p -r having the largest effect sizes (Sellbom, Toomey, Wygant, Kucharski, & Duncan, 2010). These scales have also shown utility in detecting overreported symptoms even when individuals are coached about the presence and purpose of validity scales (Sellbom & Bagby, 2010). Furthermore, the validity scales on the MMPI 2 RF have shown utility in detecting overreporting of specific disorders, including PTSD (Marion, Sellbom, & Bagby, 2011). Coaching and Mental Health Knowledge The utility of validity indices can be compromised when an individual has been coached about the presence of validity scales and how to feign symptoms without being detected. In the context of forensic evaluations, attorneys often provide their clients with information about validity scales before the client undergoes a psychological evaluation (Lees-Haley, 1997). For example, individuals undergoing an evaluation may be better able to avoid detection on validity scales if they receive information about the symptoms of a disorder that they are trying to feign (e.g., Bagby et al., 1997; Bagby, Marshall, & Bacchiochi, 2005; Bagby, Nicholson, Buis, & Bacchiochi, 2000). Although specifically providing information about the purpose of validity scales and their item content to participants in analogue studies decreases effect sizes between those who are instructed to feign and genuine patients on the MMPI 2 validity scales associated with overreporting, the scales are still useful in detecting dissimulators (Bacchiochi & Bagby, 2006; Bagby, Marshall, Bury, Bacchiochi, & Miller, 2006; Rogers, Sewell, Martin, & Vitacco, 2003; Sellbom & Bagby, 2008). In addition to direct coaching on both validity scales and symptoms of a particular disorder, mental health knowledge and specialized training can decrease the utility of the MMPI 2 validity scales in identifying participants in analogue studies who are feigning a psychiatric condition. (e.g., Bagby et al., 1997, 2000, 2005). Thus, individuals who have extensive training in the diagnosis and treatment of mental disorders tend to be better at avoiding detection on MMPI 2 validity scales than naïve subjects instructed to feign. Due to the large volume of information that is readily available to nonexperts (e.g., through Internet sources), it is possible that individuals who are attempting to feign PTSD could have extensive knowledge of its symptoms and presentation in veterans. Thus, examining the utility of the validity scales in identifying mental health experts asked to overreport provides the most stringent test available in terms of examining the effects of mental health knowledge. The Detection of Feigned PTSD The detection of feigned PTSD in general is particularly challenging since exposure to events likely to trigger PTSD symptoms is relatively common in certain populations (i.e., veterans), symptoms are subjective and common across psychiatric conditions (i.e., dysphoria), and the symptoms of PTSD are readily available from easily accessible media sources such as the Internet (Resnick, West, & Payne, 2008). The validity scales of the original MMPI 2 have shown utility in detecting overreporting of PTSD symptoms in a variety of populations, such as workers compensation claimants and remitted trauma victims (e.g., Efendov, Sellbom, & Bagby, 2008) as well as college students (e.g., Elhai, Gold, Sellers, & Dorfman, 2001). One study using a small sample of college students found RBS to be better than F, F B, F p, and FBS in detecting feigned PTSD symptoms (Lange, Sullivan, & Scott, 2010). Arbisi et al. (2006) examined the utility of the MMPI 2 validity scales in compensation-seeking veterans undergoing a disability evaluation. The F, F B, and F p scales were most useful in distinguishing between veterans asked to exaggerate symptoms of PTSD and those who were asked to respond candidly. Thus, the overreporting validity scales on the MMPI 2 have shown utility in detecting overreporting of PTSD symptoms. Similar to the MMPI 2 validity scales, the validity scales on the MMPI 2 RF have shown utility in detecting overreporting of PTSD symptoms. Marion et al. (2011) examined the utility of the MMPI 2 RF validity scales in detecting overreporting of PTSD by comparing scores between patients diagnosed with PTSD and two groups of simulators asked to feign PTSD: undergraduate students and individuals who had been previously diagnosed with PTSD but who no longer met diagnostic criteria. The F p -r scale was associated with the largest effect sizes in differentiating patients from simulators (ds ). The F S scale added incremental utility above and beyond F p -r when predicting group membership (patient vs. simulator), but the effect sizes were small ( R 2 range.02.06). The optimal F p -r cut score for differentiating undergraduate simulators from bona fide patients was determined to be T 100, as recommended in the administration and scoring manual (Ben-Porath & Tellegen, 2008). However, the optimal F p -r cut score for differentiating individuals who had previously met criteria from bona fide patients was determined to be T 110. Thus, the optimal cut scores for MMPI 2 RF validity scales may differ from those recommended by Ben-Porath and Tellegen (2008), especially when the degree of knowledge possessed by the test taker is taken into account. It is important to note that although the MMPI 2 RF validity scales were effective in detecting overreporting of PTSD symptoms in college students asked to feign PTSD, these findings need be replicated in other samples such as patients undergoing disability evaluations for PTSD. No studies using the MMPI 2 RF have focused on a group of trauma-exposed individuals seeking compensation for PTSD. The use of individuals who have been diagnosed with PTSD in such a context provides greater ecological validity than previous research in this area. Furthermore, and as just mentioned, there have been no studies examining the effect of professional mental health knowledge on the utility of the MMPI

4 POSTTRAUMATIC STRESS DISORDER IN VETERANS 3 2 RF validity scales when detecting feigned PTSD symptoms. Examining the ability of highly trained mental health professionals to feign PTSD and avoid detection by the MMPI 2 RF affords a conservative test of the MPI 2 RF validity scales. Information regarding the effects of coaching and mental health knowledge on the utility of validity scales is particularly important in the context of forensic or disability evaluations, as it might indicate measures that clinicians can take to maximize the likelihood that feigned PTSD will be detected during such an evaluation (e.g., use of different cut scores for validity scales or more extensive means of assessing overreporting). The Current Investigation In the current study, we sought to examine the utility of the MMPI 2 RF overreporting validity scales in detecting feigned PTSD symptoms in a sample of compensation-seeking veterans. We compared compensation-seeking veterans asked to respond honestly with two groups of simulators asked to feign PTSD symptoms on the MMPI 2 RF: compensation-seeking veterans and mental health professionals. Our goal was to determine the extent to which professionals with mental health training are better able to avoid detection by the MMPI 2 RF relative to compensation-seeking veterans who have been asked to exaggerate or feign PTSD. This would provide a direct test of the extent to which specific knowledge of PTSD and the MMPI 2 can influence the ability of MMPI 2 RF validity scales to identify overreporting of PTSD. Method Participants and Procedure For both samples (veterans and mental health professionals), protocols were excluded for failure to adhere to the scripted instructions. Invalid MMPI 2 RF protocols due to unscorable, inconsistent, and indiscriminant fixed responding were excluded based on the following recommended cut scores from Ben-Porath and Tellegen (2008): Cannot Say (CNS)-r scores 18, True Response Inconsistency (TRIN)-r scores T 80, or a Variable Response Inconsistency (VRIN)-r scores T 80). MMPI 2 RF scores were derived from MMPI 2 protocols for all participants. Veterans. This portion of the data used in this investigation was extracted from an existing database. 1 This sample included 83 men whose ages ranged from 31 to 87 years (M 58.28, SD 14.59). All participants were veterans who had completed a clinical interview and MMPI 2 as part of a compensation and pension (C&P) evaluation to determine eligibility for disability benefits as a result of PTSD in the Veterans Affairs (VA) system. Only veterans who produced valid and interpretable MMPI 2 profiles were asked to participate in the study. Participation of veterans who produced protocols marked by a Cannot Say (CNS) score greater than 30, a TRIN score greater than 80T, a VRIN score greater than 80T, or an F p score greater than 99T was not solicited. Veterans who produced a valid MMPI 2 protocol within the context of the C&P evaluation to determine service connection for PTSD were offered $10 to retake the MMPI 2 as part of a research study. Participants were randomly assigned to either the naïve simulator or the honest group. Those in the naïve simulator group were instructed to fabricate or exaggerate their symptoms of PTSD on the MMPI 2 without being detected as feigning, whereas those in the honest group were instructed to provide accurate and honest answers (see the Appendix [online supplemental material] for instructions). Participants were informed that those who followed instructions would be entered into a lottery for a one in four chance of winning $25. Upon completion of the MMPI 2, participants were administered a posttest questionnaire to determine compliance with the scripted instructions. In the naïve simulator group, 16 participants were excluded based on failure to attend to the content of the MMPI 2 RF items, 24 were excluded due to failure to follow the scripted instructions, and seven were excluded due to both a failure to attend to the content of the MMPI 2 RF items and failure to adhere to the scripted instructions. For the honest group, three participants were excluded due to failure to attend to the content of the MMPI 2 RF items, and two participants were excluded for failure to adhere to the scripted instructions. The final sample consisted of 29 veterans in the naïve simulator group and 54 veterans in the honest group. Thirty-one percent of veterans randomly assigned to the naïve simulator group and included in the analyses received a diagnosis of PTSD based on structured interview (Clinician-Administered Scale for PTSD; Blake et al., 1995) during the C&P evaluation and were ultimately awarded service connection for PTSD. In contrast, 54% of the veterans in the honest group received a PTSD diagnosis and service connection for PTSD. Mental health professionals. This sample of sophisticated simulators consisted of professionals (n 30) with mental health training. These individuals were all involved in direct patient care of veterans in a large VA medical center where reported symptoms of comorbid PTSD are quite prevalent regardless of primary psychiatric diagnosis. Participants included both men (n 12) and women (n 18) with ages ranging from 26 to 60 years (M 37.70, SD 9.67). Most specialized in psychology (80%) or psychiatry (10%), with the remaining 10% specializing in social work, occupational therapy, or clinical nursing. The average length of experience in their chosen field was 9.39 years. The majority of these professionals were psychology interns (43.3%), psychologists (30%), or psychiatrists (10%). Although the interns had less experience than licensed mental health professionals, they participated in the study at the end of their 12-month internship. Consequently, the interns all had supervised practical experience in diagnosis and assessment of PTSD in a VA clinical setting. Participants were administered the MMPI-2 under the same or similar instructions as those in the naïve simulators group in the veterans sample. Measures Minnesota Multiphasic Personality Inventory Restructured Form (MMPI 2 RF). The MMPI 2 RF includes 338 of the original 567 true/false items found on the MMPI 2. As was 1 These data were previously analyzed to examine the utility of the MMPI 2 validity scales in detecting feigned PTSD (see Arbisi et al., 2006). The current investigation extended these analyses to the MMPI 2 RF validity scales and also focused on the effects of mental health training on the utility of these validity scales.

5 4 GOODWIN, SELLBOM, AND ARBISI done in the current investigation, all scale scores for the MMPI 2 RF can be derived from an application of the MMPI 2. These scores have been shown to be equivalent to scores obtained from an administration using the MMPI 2 RF test booklet (Ben-Porath & Tellegen, 2008). Detailed reliability and validity information for the MMPI 2 RF can be found in the technical manual (Tellegen & Ben-Porath, 2008). The current investigation focused on the MMPI 2 RF overreporting validity scales: F-r, F p -r, F S, FBS-r, and RBS. Results Group Differences in Symptom Presentation We first examined symptom presentation on the MMPI 2 RF to determine if individuals who were asked to overreport symptoms actually exaggerated their symptoms on the MMPI 2 RF. We conducted one-way analyses of variance (ANOVAs) to see if scores on the MMPI 2 RF Restructured Clinical (RC) scales and the Internalizing subset of the Specific Problems (SP) scales differed across the three groups (naïve simulators, sophisticated simulators, and honest responders). Mean scale scores across groups are presented in Figure 1 (RC Scales) and Figure 2 (Internalizing SP scales). Levene s test of homogeneity of variance indicated unequal variances across groups (ps.05). The F statistic was modified using the Brown Forsythe formula to account for inequality of variances. Results indicated significant group differences (Fs ranging from 4.37 to 41.85, all ps.05) for all scales. Due to unequal variances, we conducted post hoc analyses using Figure 1. Mean scores on Minnesota Multiphasic Personality Inventory 2 Restructured Form Restructured Clinical (RC) Scale profiles for naïve simulators, sophisticated simulators, and honest responders with posttraumatic stress disorder. RCd Demoralization; RC1 Somatic Complaints; RC2 Low Positive Emotions; RC3 Cynicism; RC4 Antisocial Behavior; RC6 Ideas of Persecution; RC7 Dysfunctional Negative Emotions; RC8 Aberrant Experiences, RC9 Hypomanic Activation. Figure 2. Mean scores on Minnesota Multiphasic Personality Inventory 2 Restructured Form Internalizing Specific Problems (SP) Scale profiles for naïve simulators, sophisticated simulators, and honest responders with posttraumatic stress disorder. SUI Suicidal/Death Ideation; HLP Helplessness/Hopelessness; SFD Self-Doubt; NFC Inefficacy; STW Stress/Worry; AXY Anxiety; ANP Anger Proneness; BRF Behavior-Restricting Fears; MSF Multiple Specific Fears. Dunnett s T3 tests to examine bivariate comparisons. In regard to the RC scales, naïve simulators scored significantly higher than honest responders on all scales except RC2 (Low Positive Emotions). Sophisticated simulators, on the other hand, only scored significantly higher than honest respondents on four of the RC scales: RCd (Demoralization), RC2 (Low Positive Emotions), RC4 (Antisocial Behavior), and RC7 (Dysfunctional Negative Emotions). With regard to the Internalizing subset of the SP scales, both simulating groups scored significantly higher than honest responders on SUI (Suicidal/Death Ideation), HLP (Helplessness/ Hopelessness), AXY (Anxiety), ANP (Anger Proneness), and BRF (Behavior-Restricting Fears). Sophisticated simulators also scored significantly higher than honest responders on SFD (Self-Doubt), NFC (Inefficacy), and STW (Stress/Worry). Sophisticated simulators actually scored significantly lower than naïve simulators and honest responders on MSF (Multiple Specific Fears). Group Differences on Validity Scales We next conducted one-way ANOVAs to determine if validity scale scores differed across the three groups. Levene s test for homogeneity of variance indicated that variances were unequal across the groups (ps.05). To account for these unequal variances, we modified the F statistic using the Brown Forsythe formula. For bivariate comparisons, we used Dunnett s T3 post hoc tests to account for the unequal variances. The results of these

6 POSTTRAUMATIC STRESS DISORDER IN VETERANS 5 Table 1 Means, Standard Deviations, F Tests, and Effect Sizes for MMPI 2 RF Scales in Naïve Simulators, Sophisticated Simulators, and Patients With Posttraumatic Stress Disorder MMPI 2 RF scale Naïve veterans (n 29) Simulation Mental health professionals (n 30) Standard: Veterans with PTSD (n 54) M SD M SD M SD F d 1 d 2 d 3 F-r a a b F p -r a b c F S a b b FBS-r a a b RBS a a b Note. F statistic was modified using the Brown Forsythe formula to account for unequal variances. Dunnett s T3 post hoc tests were used for bivariate comparisons with unequal variances; means with different subscripts are significantly different at p.01. All F values are statistically significant at p.001. MMPI 2 RF Minnesota Multiphasic Personality Inventory 2 Restructured Form; PTSD posttraumatic stress disorder; F-r Infrequent Responses; F p -r Infrequent Psychopathology Responses; F S Infrequent Somatic Complaints; FBS-r Symptom Validity; RBS Response Bias Scale; d 1 effect size for naïve simulators versus sophisticated simulators groups; d 2 effect size for naïve simulators versus honest groups; d 3 effect size for sophisticated simulators versus honest groups. analyses are displayed in Table 1. We used estimates of Cohen s d effect size to characterize the magnitude of differences between groups, with 0.20 indicating a small effect, 0.50 a medium effect, and 0.80 a large effect (Cohen, 1988). All group differences were significantly different at the p.001 level. Both simulating groups had significantly higher scores than honest responders on all validity scales except for the F S scale, where only naïve simulators scored significantly higher than honest responders. Effect sizes for the statistically significant differences were primarily in the large range (ds ). Naïve simulators scored significantly higher than sophisticated simulators on F p -r and F S (large effect sizes of ds 0.90 and.93, respectively), but not on F-r, FBS-r, or RBS. Classification Accuracy We next examined classification accuracy statistics to determine the optimal cut scores for this particular sample as well as to evaluate those recommended in the test manual for F-r and F p -r. We focused on these scales because they were specifically designed for overreporting of psychopathology. For these analyses, we examined sensitivity and specificity for a range of individual cut scores that clinicians would actually consider in clinical practice. More important, we also examined positive predictive power (PPP), which indicates the probability that an individual is feigning, given a specific cut score. Similarly, negative predictive power (NPP) indicates the probability that the individual is not feigning, given a particular cut score. Because predictive powers are subject to significant change depending on base rates, we calculated classification parameters across a range of hypothetical base rates. We used base rates from.15 to.50, which are the lower and higher ends, respectively, of overreporting rates found in empirical studies, including those with combat veterans (e.g., Arbisi & Ben-Porath, 1998; Burkett & Whitley, 1998; Mittenberg, Patton, Canyock, & Condit, 2002; Rogers, Salekin, Sewell, Goldstein, & Leonard, 1998); Rogers, Sewell, & Goldstein, 1994). We provided a range of cut scores with associated classification statistics. Although optimal cut scores are locally determined based on setting and evaluation context, for purposes of selecting what we opine to be the best cut scores in the current study, we used.90 specificity as a marker of a satisfactory level of true negative classification, with a preference for a false-positive prediction error rate (1 PPP) of 20%. 2 In many types of psychological evaluations, including those to determine disability benefits, incorrectly concluding that an individual has feigned symptoms of psychopathology can have severe negative consequences for that person (e.g., denial of services to an individual who is experiencing legitimate psychological distress). Classification statistics for individual F-r and F p -r cut scores are shown in Table 2. For F-r, our findings indicated that none of the cut scores were associated with a specificity greater than.90 for either comparison. The results revealed that PPP was the highest at a score of 105T for both comparisons across hypothetical base rates, which was also associated with the best NPP rates. In comparison to the 120T cut score recommended by Ben-Porath and Tellegen (2008), the cut score of 105T was slightly better in the current sample in terms of PPP and NPP. Furthermore, the sensitivity is clearly preferable for the 105T cut score. Falsepositive prediction error rates were also better for the 105T cut score than for the recommended 120T cut score. It is our opinion that the cut score of 100T for F p -r, which was recommended by Ben-Porath and Tellegen (2008), shows good overall classification accuracy, particularly when one is examining low base rates (e.g.,.15). However, for this specific sample, we believe that lowering the cut score to 90T results in better sensitivity with little loss in specificity. This finding is particularly important in settings where the base rates may be expected to be high (i.e.,.30 or above), such as with PTSD feigning among combat veterans. For this 90T cut score, false-positive prediction 2 From a mathematical perspective, decreases in base rates will be associated with decreases in PPP rates, when everything else is held constant. Because many settings will have base rates below 30%, we allowed for a higher rate of false-positive prediction errors when evaluating the best cut scores in the current sample.

7 6 GOODWIN, SELLBOM, AND ARBISI Table 2 Classification Accuracy Statistics for MMPI 2 RF Scales F-r and F p -r in Differentiating Between Posttraumatic Stress Disorder Simulator and Honest Groups by Group Comparisons Cutoff score Sensitivity Specificity Base rate.15 Base rate.30 Base rate.50 PPP NPP PPP NPP PPP NPP F-r Naïve simulators vs. honest responders T T T T > T Sophisticated simulators vs. honest responders T T T T > T F p -r Naïve simulators vs. honest responders T T > T T Sophisticated simulators vs. honest responders T T > T T Note. Optimal cut score in bold. MMPI 2 RF Minnesota Multiphasic Personality Inventory 2 Restructured Form; F-r Infrequent Responses; F p -r Infrequent Psychopathology Responses; PPP positive predictive power; NPP negative predictive power. error rates were primarily in the acceptable to excellent range and were the best when high base rates were examined. The false prediction error rate was low for detecting mental health professionals in low base rates (e.g.,.15). Discussion Our primary goal in this study was to examine the utility of the MMPI 2 RF overreporting scales in detecting feigned PTSD in an ecologically valid sample of compensation-seeking veterans. Overall, the validity scales showed utility in differentiating between simulators and honest responders, with effect sizes primarily in the large range (ds.80 and higher), though some effect sizes for Fs, FBS-r, and RBS were in the small to moderate range (ds.02.74). Overall, F-r and F p -r scores were associated with the largest effect sizes in these differentiations across both comparisons, although we did not have sufficient statistical power to determine whether effect size magnitudes were significantly different. Further analyses indicated that Ben-Porath and Tellegen s (2008) recommended cut scores for these scales showed acceptable classification statistics (especially for F P -r); however, we deemed that cut scores of 105T (F-r) and 90T (F p -r) were associated with the best classification accuracy in this particular sample. These results are partially consistent with Marion et al. (2011), who found that F p -r was clearly the most effective scale in detecting feigned symptoms of PTSD. However, F-r performed better in the current sample of combat veterans than in Marion et al. s samples of college students and individuals who were in remission from PTSD as a result of other forms of trauma exposure. The current findings indicate that F-r scores may be of greater utility for detecting overreporting in more ecologically valid samples, such as compensation-seeking veterans. This scale also appeared to show less reduction in predictive utility relative to F P -r when simulators had expert knowledge about PTSD. In terms of evaluating cut scores, the current results regarding the F p -r scale were somewhat different from those of Marion et al. (2011), who indicated that the best cut score in their sample was the same as recommended in the test manual (i.e., 100T). While the current results confirmed that acceptable classification accuracies are associated with this recommendation, a slightly lower score (90T) appeared to better balance predictive powers in the current sample, while maintaining an acceptable rate of falsepositive prediction errors. These results are generally encouraging in terms of avoiding false-positive prediction errors in detecting feigned PTSD, and if cross-validated, clinicians can consider lower scores with better sensitivity in clients exposed to combat trauma. With regard to the F-r scale, the current results indicate that although the recommended cut score is 120T, using a cut score of 105T was deemed better in this sample, as this cut score was associated with the best predictive powers of any examined cut score across hypothetical base rates. Our second goal in this study was to determine if mental health professionals would be better able to avoid detection on the validity scales than naïve veterans. The current study indicated that while effect sizes associated with F-r and F p -r scores were lower

8 POSTTRAUMATIC STRESS DISORDER IN VETERANS 7 for detecting sophisticated simulators than for naïve simulators, they were still of large magnitude. Thus, even though individuals with professional mental health training were able to feign PTSD symptoms better than veterans with no specialized training, the MMPI 2 RF validity scales were still effective in differentiating between feigners and patients. These results are consistent with findings from previous studies, which have indicated that F p -r has shown utility in detecting feigned symptoms of psychopathology even when individuals have been coached (e.g., Sellbom & Bagby, 2010) or have previously met criteria for a diagnosis of PTSD (e.g., Marion et al., 2011). A veteran who applies for benefits due to feigned PTSD could attempt to increase the likelihood that his or her claim will be approved by carefully rehearsing the symptoms of PTSD or learning how specific psychological tests identify feigned symptoms. However, if a clinician uses the MMPI 2 RF with an emphasis on the appropriate validity scales, he or she can effectively decrease the chance that such an individual would be able to successfully feign PTSD in order to gain approval of a claim. In terms of actual symptom presentation, veterans with no training and mental health experts differed in their approach to simulating PTSD symptoms on the MMPI 2 RF. Based on different patterns across the MMPI 2 RF RC and Internalizing SP scales, mental health experts tended to report higher levels of dysfunctional negative emotions and low positive emotions than would be expected given the pattern of responses from veterans who responded candidly to the MMPI 2 RF when undergoing a disability examination for PTSD; the naïve simulators endorsed an excessive number of psychotic symptoms relative to the other groups. This information suggests that mental health experts were not only better able than naïve veterans in avoiding detection by the MMPI 2 RF validity scales but that they were actually able to produce MMPI 2 RF profiles that more closely resembled the profiles of veterans responding candidly during a disability evaluation for PTSD. Nonetheless, the experts also failed to fully and appropriately approximate the profiles of veterans with PTSD, which may be the result of misconceptions on the part of the clinicians related to preconceived notions regarding how PTSD presents in patients that is not supported by the empirical literature. For instance, within the domain of negative emotionality, the mental health experts appeared to endorse more items reflecting anger proneness relative to bona fide patients, reflecting the belief that PTSD in veterans is primarily associated with externalized behavior rather than expressed by two distinct subtypes: externalizing and internalizing (see e.g., Miller, Grief, & Smith, 2003; Miller, Kaloupek, Dillon, & Keane, 2004). Overall, the results of the current study are promising in terms of the utility of the MMPI 2 RF validity scales. Even though mental health professionals were better at feigning without detection than were veterans without training, the MMPI 2 RF validity scales nonetheless evidenced utility in detecting feigned PTSD symptoms. These results indicate that the MMPI 2 RF is a useful tool for identifying overreporting of symptoms in settings where compensation-seeking veterans may be motivated to feign symptoms of PTSD. Even if such individuals have mental health training, the validity scales of the MMPI 2 RF have shown utility in detecting feigned symptoms. Thus, clinicians who evaluate individuals in such settings should consider using the MMPI 2 RF as part of compensation evaluations (e.g., C&P evaluations in the VA system). In terms of limitations, the present study implemented an analogue simulation design in which individuals were asked to feign symptoms of PTSD. These individuals did not complete the MMPI 2 RF in the presence of the same type of external incentive that exists for individuals undergoing compensation evaluations, which has implications for external validity. However, unlike in previous research employing this methodology, the current simulators had significant first-hand knowledge of these evaluations and could therefore likely respond in a more realistic manner. Furthermore, scores on F-r and F p -r in the honest condition were comparable to scores obtained on these two scales in an inpatient sample (Marion et al., 2011). Future research into the utility of the MMPI 2 RF validity scales in detecting feigned PTSD symptoms could nonetheless employ a known-groups design, which would allow for replication and possibly greater external validity. Due to inconsistent results regarding validity scale cut scores, additional research should focus on determining the best cut scores for use with different populations and in different contexts (e.g., compensation evaluations, forensic evaluations) with larger samples. Furthermore, while F p -r and F-r both had large effect sizes for differentiating between honest responders and those feigning symptoms of PTSD, it is possible that one scale may perform better than the other. The current sample was not large enough to detect a significant difference between effect sizes; therefore, this type of analysis could be examined in a larger sample. In addition, we did not have sufficient statistical power to evaluate how these scales could be best used in conjunction, which could be accomplished via logistic regression. Future research needs to examine the additive and nonadditive effects of MMPI 2 RF validity scales in detecting overreporting (including of PTSD). Finally, in the current investigation, we were unable to examine incremental validity for the individual validity scales in predicting overreporting of PTSD symptoms due to insufficient statistical power. Future research could also investigate the unique effects of validity scales in predicting overreporting of PTSD using hierarchical logistic regression analyses. References Arbisi, P. A., & Ben-Porath, Y. S. (1998). The ability of the MMPI 2 validity scales to detect fake bad responses in psychiatric inpatients. Psychological Assessment, 10, doi: / Arbisi, P. A., Ben-Porath, Y. S., & McNulty, J. (2006). The ability of the MMPI 2 to detect feigned PTSD within the context of compensation seeking. Psychological Services, 3, doi: / Bacchiochi, J. R., & Bagby, R. M. (2006). Development and validation of the Malingering Discriminant Function Index for the MMPI 2. Journal of Personality Assessment, 87, doi: / s jpa8701_04 Baer, R. A., & Miller, J. (2002). Underreporting of psychopathology on the MMPI 2: A meta-analytic review. Psychological Assessment, 14, doi: / Bagby, R. M., Marshall, M. B., & Bacchiochi, J. R. (2005). The validity and clinical utility of the MMPI 2 Malingering Depression Scale. Journal of Personality Assessment, 85, doi: / s jpa8503_06

9 8 GOODWIN, SELLBOM, AND ARBISI Bagby, R., Marshall, M., Bury, A., Bacchiochi, J., & Miller, L. (2006). Assessing underreporting and overreporting response styles on the MMPI 2. In J. N. Butcher (Ed.), MMPI 2: A practitioner s guide (pp ). Washington, DC: American Psychological Association. doi: / Bagby, R. M., Nicholson, R. A., Buis, T., & Bacchiochi, J. R. (2000). Can the MMPI 2 validity scales detect depression feigned by experts? Assessment, 7, doi: / Bagby, R. M., Rogers, R., Buis, T., Nicholson, R. A., Rector, N. A., Schuller, D. R., & Seeman, M. V. (1997). Detecting feigned depression and schizophrenia on the MMPI 2. Journal of Personality Assessment, 68, doi: /s jpa6803_11 Ben-Porath, Y. S., & Tellegen, A. (2008). Minnesota Multiphasic Personality Inventory 2 Restructured Form: Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press. Blake, D. D., Weathers, F. W., Nagy, L. M., Kaloupek, D. G., Gusman, F. D., Charney, D. S., & Keane, T. M. (1995). The development of a clinician-administered PTSD scale. Journal of Traumatic Stress, 8, doi: /jts Burkett, B. G., & Whitley, G. (1998). Stolen valor: How the Vietnam generation was robbed of its heroes and its history. Dallas, TX: Verity Press. Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001). Minnesota Multiphasic Personality Inventory-2: Manual for Administration and Scoring (2nd ed.). Minneapolis: University of Minnesota Press. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Demakis, G. J., & Elhai, J. D. (2011). Neuropsychological and psychological aspects of malingered posttraumatic stress disorder. Psychological Injury and Law, 4, doi: /s y Doan, N. J., & Morton, B. C. (2008). A new era for establishing service connection for posttraumatic stress disorder (PTSD): A proposed amendment to the stressor verification requirement. Veterans Law Review, 2, Efendov, A. A., Sellbom, M., & Bagby, R. M. (2008). The utility and comparative incremental validity of the MMPI 2 and Trauma Symptom Inventory validity scales in the detection of feigned PTSD. Psychological Assessment, 20, doi: /a Elhai, J. D., Gold, S. N., Sellers, A. H., & Dorfman, W. I. (2001). The detection of malingered posttraumatic stress disorder with MMPI 2 Fake Bad Indices. Assessment, 8, doi: / Friel, A., White, T., & Hull, A. (2008). Posttraumatic stress disorder and criminal responsibility Journal of Forensic Psychiatry & Psychology, 19, doi: / Gervais, R. O., Ben-Porath, Y. S., Wygant, D. B., & Sellbom, M. (2010). Incremental validity of the MMPI 2 RF overreporting scales and RBS in assessing the veracity of memory complaints. Archives of Clinical Neuropsychology, 25, doi: /arclin/acq018 Green, P., Allen, L., & Astner, K. (1996). Manual for the Computerized Word Memory Test. Durham, NC: CogniSyst. Hathaway, S. R., & McKinley, J. C. (1940). A multiphasic personality schedule (Minnesota): I. Construction of the schedule. Journal of Psychology: Interdisciplinary and Applied, 10, doi: / Iverson, G. L., Franzen, M. D., & Hammond, J. A. (1995). Examination of inmates ability to malinger on the MMPI-2. Psychological Assessment, 7, doi: / Lange, R. T., Sullivan, K. A., & Scott, C. J. (2010). Comparison of MMPI 2 and PAI validity indicators to detect feigned depression and PTSD symptom reporting. Psychiatry Research, 176, doi: /j.psychres Lees-Haley, P. R. (1997). Attorneys influence expert evidence in forensic psychological and neuropsychological cases. Assessment, 4, Marion, B. E., Sellbom, M., & Bagby, R. M. (2011). The detection of feigned psychiatric disorders using the MMPI-2-RF overreporting validity scales: An analog investigation. Psychological Injury and Law, 4, doi: /s Miller, M. W., Grief, J. L., & Smith, A. A. (2003). Multidimensional Personality Questionnaire profiles of veterans with traumatic combat exposure: Internalizing and externalizing subtypes. Psychological Assessment, 15, doi: / Miller, M. W., Kaloupek, D. G., Dillon, A. L., & Keane, T. M. (2004). Externalizing and internalizing subtypes of combat-related PTSD: A replication and extension using the PSY 5 scales. Journal of Abnormal Psychology, 113, doi: / x Millon, T. (1983). Millon Clinical Multiaxial Inventory manual (3rd ed.). Minneapolis, MN: National Computer Systems. Mittenberg, W., Patton, C., Canyock, E., & Condit, D. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 24, doi: /jcen Morey, L. C. (1991). Professional manual for the Personality Assessment Inventory. Odessa, FL: Psychological Assessment Resources. Resnick, P. J., West, S., & Payne, J. W. (2008). Malingering of posttraumatic disorders. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed., pp ). New York, NY: Guilford Press. Rogers, R., Salekin, R. T., Sewell, K. W., Goldstein, A., & Leonard, K. (1998). A comparison of forensic and nonforensic malingerers: A prototypical analysis of explanatory models. Law and Human Behavior, 22, doi: /a: Rogers, R., Sewell, K. W., & Goldstein, A. M. (1994). Explanatory models of malingering: A prototypical analysis. Law and Human Behavior, 18, doi: /bf Rogers, R., Sewell, K. W., Martin, M. A., & Vitacco, M. J. (2003). Detection of feigned mental disorders: A meta-analysis of the MMPI 2 and malingering. Assessment, 10, doi: / Sellbom, M., & Bagby, R. M. (2008). Response styles on multiscale inventories. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed., pp ). New York, NY: Guilford Press. Sellbom, M., & Bagby, R. M. (2010). The detection of overreported psychopathology with the MMPI 2 RF form validity scales. Psychological Assessment, 22, doi: /a Sellbom, M., Toomey, J. A., Wygant, D. B., Kucharski, L. T., & Duncan, S. (2010). Utility of the MMPI 2 RF (Restructured Form) validity scales in detecting malingering in a criminal forensic setting: A known groups design. Psychological Assessment, 22, doi: / a Sellbom, M., Wygant, D. B., & Bagby, R. M. (2012). Utility of the MMPI-2-RF in detecting non-credible somatic complaints. Psychiatry Research, 197, Tellegen, A., & Ben-Porath, Y. S. (2008). The Minnesota Multiphasic Personality Inventory 2 Restructured Form: Technical manual. Minneapolis: University of Minnesota Press. Wygant, D. B., Ben-Porath, Y. S., Arbisi, P. A., Berry, D. T. R., Freeman, D. B., & Heilbronner, R. L. (2009). Examination of the MMPI 2 Restructured Form (MMPI 2 RF) validity scales in civil forensic settings: Findings from simulation and known group samples. Archives of Clinical Neuropsychology, 24, doi: /arclin/acp073 Received November 21, 2011 Revision received January 28, 2013 Accepted January 30, 2013