Published online: 25 Aug 2014.

Transcription

1 This article was downloaded by: [Dr Robert Denney] On: 27 March 2015, At: 10:31 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK Applied Neuropsychology: Adult Publication details, including instructions for authors and subscription information: The Detection of Feigned Impairment Using the WMT, MSVT, and NV-MSVT Patrick Armistead-Jehle a & Robert L. Denney b a Department of Behavioral Health, Munson Army Health Center, Fort Leavenworth, Kansas b School of Professional Psychology, Forest Institute, Springfield, Missouri Published online: 25 Aug Click for updates To cite this article: Patrick Armistead-Jehle & Robert L. Denney (2015) The Detection of Feigned Impairment Using the WMT, MSVT, and NV-MSVT, Applied Neuropsychology: Adult, 22:2, , DOI: / To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content ) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at

2 APPLIED NEUROPSYCHOLOGY: ADULT, 22: , 2015 Copyright # Taylor & Francis Group, LLC ISSN: print= online DOI: / The Detection of Feigned Impairment Using the WMT, MSVT, and NV-MSVT Patrick Armistead-Jehle Department of Behavioral Health, Munson Army Health Center, Fort Leavenworth, Kansas Robert L. Denney School of Professional Psychology, Forest Institute, Springfield, Missouri Downloaded by [Dr Robert Denney] at 10:31 27 March 2015 Key words: The current study sought to evaluate the sensitivity of the Word Memory Test, Medical Symptom Validity Test, and the Nonverbal Medical Symptom Validity Test (NV-MSVT) in a group of 50 well-educated individuals asked to simulate dementia. With use of the Genuine Memory Impairment Profile as operationalized by the Advanced Interpretation Program, sensitivities for single measures and the measures in combination ranged from 54% to 98% in the detection of suboptimal effort. Overall, the NV-MSVT appeared the most sensitive to feigned memory impairment in this sample. effort testing, MSVT, NV-MSVT, PVTs, simulator study, WMT INTRODUCTION The routine employment of tests and procedures designed to assess performance validity in neuropsychological evaluations is now considered common practice (Bush et al., 2005; Heilbronner et al., 2009). The importance of incorporating such measures is supported not only by the pervasive influence that effort has on cognitive ability measures (e.g., Constantinou, Bauer, Ashendorf, Fisher, & McCaffrey, 2005; Fox, 2011; Green, 2007; Green, Rohling, Lees-Haley, & Allen, 2001; Meyers, Volbrecht, Axelrod, & Reinsch-Boothby, 2011), but also by the high base rates of performance validity test (PVT) failures even among seemingly The views, opinions, and=or findings contained in this article are those of the authors and should not be construed as an official Department of the Army position, policy, or decision unless so designated by other official documentation. A preliminary review of these data were presented as a poster at the American Academy of Clinical Neuropsychology s Ninth Annual Conference & Workshops, Washington, DC, in June Address correspondence to Patrick Armistead-Jehle, Behavioral Health, Munson Army Health Center, 550 Pope Ave., Fort Leavenworth, KS patrick.j.armistead-jehle.civ@mail.mil unbiased samples (e.g., An, Zakzanis, & Joordens, 2012). Generally speaking, PVTs can be dichotomized into one of two forms: (a) stand-alone instruments designed and standardized to evaluate effort, or (b) indexes embedded in cognitive ability tests. Most available stand-alone tests simulate measures of memory and employ a forced-choice recognition paradigm to gauge a test taker s willingness to engage in a consistent and effortful manner (Frederick & Foster, 1991; Frederick & Speed, 2007). However, there have been concerns that measures of this nature have potential limitations secondary to their ease in coaching (Rohling & Boone, 2007; Suhr & Gunstad, 2007) as well as the possibility of false positives in low-functioning individuals (Dunn, Shear, Howe, & Ris, 2003; Merten, Green, Henry, Blaskewitz, & Brockhaus, 2005). Select PVTs have been developed that differ from their counterparts in that a series of subtests of varying difficulties are administered in conjunction with the more traditional forced-choice recognition measures. This paradigm allows for a profile analysis where scores of dissimilar difficulty levels can be evaluated to determine the credibility and reliability of performances (Brockhaus & Merten, 2004). This methodology has

3 148 ARMISTEAD-JEHLE & DENNEY been applied to three PVTs: (a) the Word Memory Test (WMT; Green, Allen, & Astner, 1996; Green & Astner, 1995); (b) Medical Symptom Validity Test (MSVT; Green, 2004); and (c) Nonverbal Medical Symptom Validity Test (NV-MSVT; Green, 2008b). Across each of these measures, various subtests are objectively easier than others and comparisons between the easy and the hard subtests have been shown to provide useful information regarding test-taker effort and engagement regardless of the potential cognitive abilities or clinical diagnoses of the test taker (Carone, 2009; Wager & Howe, 2010). The MSVT and WMT have been subject to several simulator studies with results suggesting good classification accuracy across cultures. For instance, Tan, Slick, Strauss, and Hultsch (2002) administered three forced-choice tests (including the forced-choice recognition subtests of the WMT) to 52 undergraduate students, with roughly half of the participants instructed to feign believable impairment following brain injury from a motor vehicle accident and the other half asked to perform as if they had recovered from such an injury. The administered WMT subtests correctly classified all participants in the respective groups, thus demonstrating 100% sensitivity and 100% specificity. Unfortunately, these authors did not administer all WMT subtests, and as such, the profile analysis could not be employed. Merten et al. (2005) evaluated an oral version of the MSVT in a sample of 36 healthy Germanspeaking individuals randomized into two groups: experimental malingerers and full-effort participants. The MSVT forced-choice recognition subtests correctly classified 97% of the sample; however, as with the Tan et al. study, no profile analysis was employed. In a Russian simulator study, Tydecks, Merten, and Gubbay (2006) administered the WMT to 20 participants, half of whom were instructed to feign memory deficits in the context of litigation and the other half told to provide their best effort. Results demonstrated that the WMT forced-choice recognition subtests had perfect classification accuracy across the sample. Finally, Weinborn, Woods, Nulsen, and Leighton (2012) evaluated the NV-MSVT and MSVT among 103 healthy young-adult volunteers who were randomly assigned to various coached and noncoached conditions. Their results demonstrated sensitivity values ranging from 83% to 100% for the MSVT and 88% to 96% for the NV-MSVT. Several other studies have come to similar conclusions regarding the sensitivities of these measures (e.g., Blaskewitz, Merten, & Kathmonn, 2008; Gervais, Rohling, Green, & Ford, 2004). As these studies demonstrate, the WMT, MSVT, and NV-MSVT are sensitive to feigned impairment in simulator studies; however, the issue of specificity in these PVTs is also of import and deserves mention here. Green, Montijo, and Brockhaus (2011) elaborated on a response pattern encountered when applying the WMT to individuals diagnosed with dementia. A subsection of this sample failed the easy subtests of the WMT and thus would potentially be considered false positives for poor effort if one relied solely on the established easy subtest cut scores. However, impairment from Alzheimer s disease probably explained their failed WMT performances. Criterion D of the Slick, Sherman, and Iverson (1999) criteria for malingering requires that before concluding poor effort and invalid test results, one must exclude neurological or psychiatric conditions that could explain the PVT failure. Criterion D states that malingering cannot be concluded from failure on any test if behaviors meeting Criteria B and C are fully explained by psychiatric, neurological or developmental factors (p. 554). The diagnosis of Alzheimer s disease in the Green et al. (2011) participants is of course one such neurological condition. Green et al. (2011) observed a marked tendency for the scores on harder subtests to be far lower than the easier subtest scores in patients diagnosed with dementia. None of these patients who failed the easy subtests had an easy hard difference of less than 30 points. Thus, if someone fails the easy WMT subtests and there is an easy hard difference of 30 points or more, in keeping with Criterion D of the Slick et al. criteria, the results are not considered indicative of poor effort unless one can rule out impairment secondary to known psychiatric, neurological, or developmental factors, akin to that seen in dementia of the Alzheimer s type. A similar profile analysis has been applied to the MSVT and the NV-MSVT (Henry, Merten, Wolf, & Harth, 2010; Howe & Loring, 2009; Singhal, Green, Ashaye, Shankar, & Gill, 2009). These studies demonstrate the achievement of high levels of specificity in samples of patients diagnosed with dementia by applying profile analysis, taking into account both the level of scores on easy subtests and the magnitude of the differences between scores on easier and harder subtests. Singhal et al. (2009), for example, found 100% specificity for profile analysis on the NV-MSVT in hospitalized patients with advanced dementia. A potential problem with aspects of the current PVT literature is that high specificity was achieved at the cost of a low sensitivity in simulators when the poor-effort criteria were applied without consideration of all other clinical information. For instance, with some frequency, the simulators in the Singhal et al. (2009) study managed to produce profiles resembling those of patients with dementia on the MSVT or the NV-MSVT alone. However, by combining the tests and classifying a participant as showing suboptimal effort if a poor-effort profile was found on either test, sensitivity increased to 80%. A question that has not yet been addressed in the current

4 PVT literature is the level of sensitivity that could be achieved if the WMT, MSVT, and NV-MSVT were all applied to participants asked to simulate impairment. The primary purpose of the current investigation was to explore the answer to this inquiry. METHODS Participants/Procedures The study sample consisted of 50 volunteers, all of whom were psychology graduate students enrolled in a Midwestern American Psychological Association (APA)-accredited Psy.D. program. There were 33 women and 17 men with an average age of 27.0 years (SD ¼ 5.7 years) and an average of 17.0 years of education (SD ¼ 1.2 years). The volunteers were recruited to attempt simulation of dementia across a series of PVTs. Participants were instructed to complete the measures in a manner that would deceive the examiner into believing that he or she had a diagnosis of dementia when in fact they did not. Participants were not asked to feign dementia across all interactions with the researchers, but rather only in their PVT performances. Although study participants were provided no coaching on how to accomplish this task, all had some degree of graduate training in psychopathology and could thus be expected to have at least a rudimentary understanding of dementia. At the conclusion of data collection, all participants completed a questionnaire that verified that they understood the test instructions and in fact attempted to simulate dementia in their test responses. All participants who were able to produce plausible results suggesting dementia across the measures were entered into a drawing for a cash prize of $100. None of the participants had firsthand knowledge of the WMT, MSVT, or NV-MSVT. The study was approved by the Forest Institute of Professional Psychology Institutional Review Board. Measures Study participants completed the MSVT, NV-MSVT, and WMT consecutively in this fixed order. Tests were administered per standardized instructions with one exception; for the sake of convenience, the interval between the Immediate Recognition (IR) and Delayed Recognition (DR) trials of each test was simulated. Prior to the DR trials, participants were instructed to imagine that they had been performing other unrelated cognitive or psychological testing for a period of 30 min (in the case of the WMT) and 10 min (in case of the MSVT and NV-MSVT). Primary data analysis to determine participant performances was done via Green s Advanced Interpretation (AI) Program (Green, 2008a). PVTS IN SIMULATORS 149 Word Memory Test. The WMT is a computeradministered verbal memory test with multiple subtests designed to assess verbal memory, effort, and response consistency. Twenty semantically related word pairs are presented twice for examinees to learn. Directly after the learning trials, IR memory is tested. Following a delay, DR memory is assessed. This is then followed by a series of subtests including Multiple Choice (MC), Paired Associates (PA), and Free Recall (FR) formats. In addition to these five subtests, a Consistency (CNS) score is calculated to gauge recall consistency across select trials. Failure on the WMT was defined as performance below the cut score on the IR, DR, and=or CNS subtests as defined in the test manual (Criterion A). The IR, DR, and CNS subtests are considered the initial measures of symptom validity and have been termed the easy subtests. As a result of their difficulty relative to the easy subtests, the MC, PA, and FR subtests are referred to as the hard subtests. When one or more easy subtests are failed, further analysis of the complete WMT profile and presenting clinical factors is then recommended to determine if genuine memory impairment or suboptimal effort is more likely. When at least one easy subtest is failed, the mean of the easy subtests is subtracted from the mean of the hard subtests. The magnitude of this difference is high (i.e., greater than or equal to 30) in true impairment but low in simulators, and the examination of the easy hard subtest difference (Criterion B) has been found to discriminate adequately between those with and without genuine memory impairment. Further, order violations (i.e., superior performance on relatively difficult subtests compared with easier ones) have also been shown to differentiate between patients with genuine memory impairment and those offering suboptimal effort (Chafetz, 2008; Green, 2003; Howe & Loring, 2009). Examinees performing below the cut scores on the easy subtests had their WMT profiles analyzed to determine if their performances met criteria for a Genuine Memory Impairment Profile (GMIP), defined as a difference between the easy and the hard subtests of 30 or more. In other words, a GMIP was concluded if the participant failed Criterion A (i.e., poor performance on the easy subtests) and passed Criterion B (easy hard difference of 30 or more). See Table 1 for a summary of Criteria A and B for all PVTs included in this study. Medical Symptom Validity Test. The MSVT (Green, 2004) is an abbreviated version of the WMT and thus follows a very similar format. In the MSVT, 10 semantically related word pairs representing a single common object are shown across two trials. Afterward, IR memory is tested. Following a short delay, DR memory (is tested. A PA trial is then administered and the measure

5 150 ARMISTEAD-JEHLE & DENNEY TABLE 1 Criteria A and B Definitions for the WMT, MSVT, and NV-MSVT PVT Criterion A Criterion B WMT IR, DR, or CNS < specified cut points y Easy Subtests mean Hard Subtests mean < 30 points MSVT IR, DR, or CNS < specified cut points y Easy Subtests mean Hard Subtests mean < 20 Points NV-MSVT A1: Mean of the IR, DR, CNS, DRA, DRV, and PA or A2: Mean of DR, CNS, DRA, and DRV < specified cut points y B1: PA must not be 11 points or more below the mean of DR, CNS, DRA, and DRV B2: Easy Subtests mean Hard Subtests mean < 20 points B3: The standard deviation of scores on the five easiest subtests is 12 PVT ¼ Performance Validity Test; WMT ¼ Word Memory Test; MSVT ¼ Medical Symptom Validity Test; NV-MSVT ¼ Nonverbal Medical Symptom Validity Test; IR ¼ Immediate Recognition; DR ¼ Delayed Recognition; CNS ¼ Consistency; PA ¼ Paired Associates; DRA ¼ Delayed Recognition-Archetypes; DRV ¼ Delayed Recognition-Variation. y Specified cut points can be located in the respective PVT test manual. Downloaded by [Dr Robert Denney] at 10:31 27 March 2015 is concluded with an FR trial. In addition to these four subtests, a CNS score is calculated to gauge recall consistency across select trials. Failure on this measure is defined as performance below the cut score on the IR, DR, and=or CNS subtests as outlined in the test manual (Criterion A; see Table 1). As with the WMT, a GMIP has been established and is defined as a difference between easy and hard subtests of at least 20 points (Criterion B). A number of studies have demonstrated the utility of this measure in the discrimination between those with genuine memory impairment and those simulating impairment in a range of patient samples (e.g., Chafetz, 2008; Howe & Loring, 2009; Singhal et al., 2009). Nonverbal Medical Symptom Validity Test. The NV-MSVT (Green, 2008b) is a brief automated nonverbal memory screening with several subtests designed to measure nonverbal memory and response consistency. Ten artist-drawn colored images representing an intuitive pair of items (e.g., a baseball and a baseball bat) are shown across two trials. Afterward, IR memory is tested via forced-choice format involving 20 pairs of targets and foil items (e.g., baseball vs. car). Following a short delay, there are three forced-choice tasks asking the patient to select items seen before on the computer. The DR subtest is similar to the IR and involves the presentation of the 20 target items seen before being paired with a novel foil (e.g., baseball vs. desk). The Delayed Recognition-Variation (DRV) subtest involves 10 presentations of the complete image from the original list (e.g., baseball and baseball bat) paired with the similar image that differs only by one small detail (e.g., baseball with stitching missing and a baseball bat). The Delayed Recognition-Archetypes (DRA) subtest involves 20 presentations of items that were previously used as a foil during the IR trial (e.g., car) paired with a vivid archetypal image, such as a coiled snake with its mouth open and teeth viciously exposed, which has not been presented before. The DR tasks are then followed by a PA recall subtest, where the examinee is presented with one part of the pair (e.g., baseball) and is asked to respond with the corresponding item (e.g., baseball bat). The final subtest is an FR task in which the examinee is asked to recall from memory as many items from the original list of paired items as possible. Failure of this measure was determined as specified in the test manual and the AI program (Green, 2008a). The sensitivity and specificity of the NV-MSVT as a measure of symptom validity has been demonstrated in a variety of patient samples (e.g., Green, 2008b; Green, Flaro, Brockhaus, & Montijo, 2012; Henry et al., 2009; Singhal et al., 2009). When evaluating performance on the NV-MSVT, the test manual describes various decision rules designed to discriminate between genuine memory impairment and suboptimal performance (Green, 2008b). Criterion A is failed if the mean of the IR, DR, CNS, DRA, DRV, and PA subtests or the mean of the DR, CNS, DRA, and DRV subtests is below the cut points specified in the test manual. In essence, these are the means of the easy subtests that are passed by various clinical groups (e.g., children with a diagnosis of fetal alcohol syndrome, children with a diagnosis of autism, children with an average Full-Scale IQ of 74, adults with a history of moderate-to-severe brain injury), and a failure is only expected if the individual is offering suboptimal effort or if the individual is unable to pass secondary to severe impairment on par with dementia. The second set of criteria are to be applied only when Criterion A is failed (i.e., easy subtests are failed), and these criteria were designed to discriminate between genuine memory impairment and poor effort. Criterion B1 specifies that the PA score must be at least 11 points below the mean of the DR, CNS, DRA, and DRV. If not, the profile suggests poor effort as the individual s test performance is not consistent with the objective difficulty levels of the subtests (this constellation of scores is also known as the paradoxical profile ). Criterion B2 states that

6 PVTS IN SIMULATORS 151 Downloaded by [Dr Robert Denney] at 10:31 27 March 2015 the mean of the easiest subtests must be at least 20 points higher than the mean of the hardest subtests. Finally, Criterion B3 reads that the standard deviation of the examinee s scores on the five easiest subtests must be less than 12. With Criterion B1 and B2, one is essentially examining if there are inconsistencies across the NV-MSVT subtests that demonstrate differing levels of objective complexity; Criterion B3 identifies excess variability on select subtests. If Criterion A and any two or three of Criteria B1, B2, or B3 are failed, the examinee is judged to be giving poor effort. The sensitivity and specificity of these decision rules have been supported across multiple studies (Green, 2008b; Green, Flaro, Brockhaus, & Montijo, 2012; Henry et al., 2009; Singhal et al., 2009). Advanced Interpretation Program. The AI Program (Green, 2008a) is a computerized scoring system for the WMT, MSVT, and NV-MSVT. The program employs the scoring criteria outlined to determine if a WMT, MSVT, or NV-MSVT profile is suggestive of genuine memory impairment or invalid test data. The program relies entirely on the A and B criterion for each measure. Although the program notes the presence of order violations (i.e., superior performance on relatively difficult subtests compared with easier ones), it does not employ them in determining a genuine memory impairment versus nongenuine memory impairment. According to the program designer (Dr. Paul Green), the reasoning for this was to minimize false positives for poor effort TABLE 2 WMT and MSVT Subtest Performances in Those Failing the Measures and, in turn, to improve specificity in cases where dementia or other severe impairment is potentially present (P. Green, personal communication, August 8, 2013). RESULTS Tables 2 and 3 outline the average performances of the sample across the WMT, MSVT, and NV-MSVT in those who failed Criterion A of these measures. Across the various PVTs, few simulators passed the easy subtests. The WMT had 1 individual pass the easy subtests, while the MSVT had 3 and the NV-MSVT had 2. Table 4 outlines the Criterion A failure rates across these measures. Such a finding would indicate that the participants were attempting to appear impaired as demanded by the experimental instructions. Of the 49 cases who failed the WMT easy subtests, 13 were able to produce a GMIP. As such (when including the participant with easy subtest scores too high to be considered reasonable for dementia), the sensitivity of this measure to malingered impairment using profile analysis was 74%. Of the 47 who failed the MSVT easy subtests, 23 produced a GMIP. Consequently (when including the 3 participants with easy subtests scores too high to be considered reasonable for dementia), the sensitivity of the MSVT was 54%. In regard to the NV-MSVT, of the 48 participants who failed the easy subtests (Criterion A), 3 failed only one B criterion and thus produced a GMIP. When including the 2 participants WMT Scores Mean Scores in % Correct Standard Deviation Range Fail WMT (N ¼ 49) IR DR CNS MC PA FR Easy Subtests Hard Subtests Easy Hard Subtests MSVT Scores Mean Scores in % Correct Standard Deviation Range Fail MSVT (N ¼ 47) IR DR CNS PA FR Easy Subtests Hard Subtests Easy Hard Subtests WMT ¼ Word Memory Test; MSVT ¼ Medical Symptom Validity Test; IR ¼ Immediate Recognition; DR ¼ Delayed Recognition; CNS ¼ Consistency; MC ¼ Multiple Choice; PA ¼ Paired Associates; FR ¼ Free Recall.

7 152 ARMISTEAD-JEHLE & DENNEY TABLE 3 NV-MSVT Scores in Those Failing Easy Subtests (N ¼ 48) NV-MSVT Subtests Mean Scores in % Correct Standard Deviation Range IR DR CNS DRA DRV PA FR NV-MSVT ¼ Nonverbal Medical Symptom Validity Test; IR ¼ Immediate Recognition; DR ¼ Delayed Recognition; CNS ¼ Consistency; DRA ¼ Delayed Recognition-Archetypes; DRV ¼ Delayed Recognition-Variation; PA ¼ Paired Associates; FR ¼ Free Recall. TABLE 4 Criterion A Failure on the WMT, MSVT, and NV-MSVT WMT N ¼ 49 (98%) MSVT N ¼ 47 (94%) NV-MSVT N ¼ 48 (96%) WMT and MSVT N ¼ 49 (98%) WMT and NV-MSVT N ¼ 49 (98%) MSVT and NV-MSVT N ¼ 49 (98%) WMT, MSVT, and NV-MSVT N ¼ 49 (98%) WMT ¼ Word Memory Test; MSVT ¼ Medical Symptom Validity Test; NV-MSVT ¼ Nonverbal Medical Symptom Validity Test. with easy subtests too high to be considered reasonable for dementia, the NV-MSVT had a sensitivity of 94%. Table 5 outlines a breakdown of Criteria A, B1, B2, and B3. This breakdown was very similar to the data published in the NV-MSVT manual (Green, 2008b; Table 6.1 on p. 68). When the measures were analyzed in combination, as anticipated, an overall increase in sensitivity was demonstrated. Of the 13 participants who were able to produce a GMIP on the WMT, 6 were unable to produce the same on the MSVT. As such, the sensitivity of combining performances on the WMT and the MSVT, relative to the WMT alone, increased from 74% to 84%. When these measures are examined in the reverse order, we see that of the 23 participants who produced a GMIP TABLE 5 Criteria A, B1, B2, and B3 Applied to the NV-MSVT Failure on A N ¼ 48 (96%) Failure on A and B1 N ¼ 46 (92%) Failure on A, B1, and B2 N ¼ 41 (82%) Failure on A, B1, and B3 N ¼ 29 (58%) Failure on A, B2, and B3 N ¼ 25 (50%) Failure on A, B1, B2, and B3 N ¼ 25 (50%) NV-MSVT ¼ Nonverbal Medical Symptom Validity Test. on the MSVT, 16 were unable to produce a GMIP on the WMT and the sensitivity increased from 54% to 84%. With regard to the NV-MSVT, 3 participants were able to fail the easy subtests and pass two of the three B criteria; however, 2 were unable to simulate a GMIP on either the WMT or the MSVT, and the other was unable to simulate a GMIP on the WMT. As such, the sensitivity of the NV-MSVT in combination with the WMT or the MSVT was 100%. Across the three tests, no participant was able to produce GMIPs across all measures. However, 1 participant was able to produce a GMIP on the WMT and the MSVT and create a passing NV-MSVT that evidenced deficient memory. In a clinical sense, this was the lone participant in the sample who would have likely been considered to evidence genuine cognitive impairment (and he was awarded the $100 prize). As noted, the primary method of PVT analysis came via the AI Program, and this program does not incorporate order violations into the GMIP versus non-gmip analysis. When order violations were taken into account, 6 of the 13 participants who obtained a GMIP on the WMT demonstrated order violations, and 3 of the 23 participants who obtained a GMIP on the MSVT evidenced order violations. Consequently, with the inclusion of order violations, the sensitivities of the WMT and MSVT increase to 86% and 60%, respectively. In regards to the NV-MSVT, of the 3 participants who failed Criterion A and demonstrated a GMIP by failing only one of the B criteria, 2 had order violations. With the inclusion of order violations, the sensitivity of the NV-MSVT was then increased to 98%. DISCUSSION In the current study, the administered measures were able to detect simulated performances with a generally high degree of sensitivity. When the data were examined by employing only Criterion A, the sensitivity rates ranged from 94% to 98%. Among these tests, when Criteria A and B were employed for profile analysis, the NV-MSVT was best able to detect the attempted imitation of memory impairment among sophisticated simulators with sensitivity of 94%. The WMT had a 74% sensitivity rate, while the MSVT was the least sensitive of the administered measures with a 54% sensitivity rate using profile analysis. However, when profile analysis of both the MVST and WMT were analyzed in combination, they achieved 84% sensitivity. When the NV-MSVT was used in combination with either the WMT or the MSVT, the sensitivity increased to 100%. It is of note that one participant was able to produce a pattern of test results that would likely have been interpreted clinically as valid cognitive impairment.

8 In this case, the participant produced a GMIP on the WMT and the MSVT and passed the NV-MSVT. Technically speaking, this participant did not fail the NV-MSVT with a profile consistent with genuine memory impairment, but considering the concurrent WMT and MSVT performances, most clinicians would have likely considered this performance valid and suggestive of weak memory. When interpreted in this conservative manner, the total sensitivity of the three measures in combination was 98%. Stated differently, 98% of simulators were unable to produce a profile of scores across the WMT, MSVT, and NV-MSVT that would be considered suggestive of genuine severe impairment in a clinical case. Among the limitations of the current study was the condensed nature of the measure administration. Given the simulator design of this study, the aim of the investigation was made apparent to the participants, and it is debatable if the altered testing procedures affected participant test performances. Additionally, given the very high scores on DR subtests in cognitively impaired samples (e.g., Carone, 2008), it could be deduced that the delay has little impact on these measures, which demonstrate a rather low ceiling. However, future investigations may augment internal validity by administering the measures with the standardized delay between subtests. Additionally, as the majority of the sample had experience with graduate-level training in psychology and a generally high level of education, the sample could be considered to possess greater sophistication than the general population. High education was also noted as a possible limitation in the Merten et al. (2005) simulator study. This higher level of knowledge about psychometrics could have augmented their abilities to produce a GMIP, and in turn, it could have driven down the overall sensitivity of some of the administered measures. An additional limitation to this study was the absence of a group with genuine cognitive impairment, the inclusion of which would have resulted in the evaluation of specificity for the administered measures. Although these data were not available in the current investigation, several published studies have worked to establish the high degree of specificity of these PVTs. For example, in a group of 52 (mostly elderly) adults referred to a memory disorders clinic, Howe and Loring (2009) demonstrated a specificity of 91% on the MSVT. Singhal et al. (2009) demonstrated 100% specificity in the MSVT and NV-MSVT in 10 cases of severe dementia. Henry et al. (2009) evaluated, via the NV-MSVT, 65 adult patients diagnosed with neurologic dysfunction and 50 healthy adult volunteers. They reported a specificity of 99% in the neurological group and 100% in the control group. Green et al. (2011) demonstrated a specificity of more than 98% for the PVTS IN SIMULATORS 153 WMT and MSVT in a group of adults with possible or probable dementia (n ¼ 125). Moreover, several studies have demonstrated the very low failure rates of these measures (even on Criterion A) in impaired samples. For instance, Carone (2008) found that children with moderate-to-severe brain damage=dysfunction from a range of injuries including strokes, moderate-tosevere traumatic brain injury, and encephalitis easily passed the easy subtests of the MSVT with average scores greater than 96% on each subtest. In addition, Green et al. (2012) tested children with developmental disabilities and reported that, on average, only 7.9% of these disabled children failed Criterion A of the WMT, MSVT, or NV-MSVT. None failed Criterion A on all three tests, whereas this occurred in almost all cases in the simulators of the current study. When Criteria A and B were applied, poor-effort profiles (i.e., not GMIP) were found in only 5.3% of the children on the WMT, 2.6% of children on the MSVT, and 4.1% of the children on the NV-MSVT in the Green et al. (2012) data. Such data stand in marked contrast to the very high rate of poor-effort profiles on all three tests in the current simulators. Consequently, although the current study did not include a group of participants by which to determine the specificity of the administered PVTs, previous research clearly demonstrates the high levels of specificity in these measures. Ideally, future investigations would include simulator and genuinely impaired groups. An important caveat to this research design is that should any of these individuals have presented for neuropsychological assessment, by virtue of their independent functioning, they would not be eligible for the GMIP analysis in the first place. As the current investigation was a simulator study, this criterion was suspended, but the caveat is of import when discussing the derived classification statistics in the real-world setting. As noted in Table 3, when applying Criterion A alone, there were a high number of failures across the three measures. As failures of this type across the easy subtests can occur in individuals with severe impairment secondary to conditions like dementia, profile analysis becomes essential because it allows false positives in such individuals to be greatly reduced. An additional caveat to consider in the current study is that the AI Program does not employ subtest order violations in determining the GMIP versus non-gmip nature of an examinee s performances. The current data suggest that the WMT, MSVT, and NV-MSVT sensitivities can be increased if order violations are considered. Further research could evaluate the incremental utility of evaluating order violations in examinees without dementia. Despite the study s limitations, the current data demonstrate, in a simulator design, not only the high sensitivities of specific measures, but the increased sensitivity gained by employing multiple PVTs. The current data

9 154 ARMISTEAD-JEHLE & DENNEY deserve replication, but the study fills a gap in the extant literature by demonstrating the high sensitivity of profile analysis among simulators. When the already published studies on the administered PVTs are examined in conjunction with the current data, it can be concluded that regardless of the presence of a condition meeting the Slick et al. (1999) Criterion D, if there is failure of Criterion A and there is not a GMIP, then the neurocognitive data are exceedingly unlikely to represent a reliable estimation of cognitive ability irrespective of diagnosis. In conclusion, a number of studies demonstrate that examinee effort accounts for more variance in cognitive test scores than does neurologic insult (Constantinou et al., 2005; Fox, 2011; Green, 2007; Green et al., 2001; Meyers et al., 2011) and that clinicians qualitative analysis of effort is, at best, limited (Faust, Hart, Guilmette, & Arkes, 1988; Heaton, Smith, Lehman, & Vogt, 1978). In the instance of suboptimal effort on behalf of the examinee, should the neuropsychologist erroneously attribute poor performances across cognitive ability measures to a neurologic etiology rather than to the correct source, there is then remarkable potential for the misallocation of clinical and financial resources. The present data suggest that the sensitivity of detecting feigned memory impairment is notably increased by employing the WMT, MSVT, and NV-MSVT. In the current health care environment, market forces work against adding time to evaluations in an effort to contain costs; however, given the limited amount of time needed to administer and interpret these procedures, a cogent argument can be made that the gain in objectively determining the validity of test data is worth the expense. In other words, the cost-benefit ratio argues in favor of employing the WMT, MSVT, and NV-MSVT in the completion of a neuropsychological assessment. ACKNOWLEDGMENTS The authors would like to acknowledge the data collection assistance of Kathryn Dunham, Jessica Mack, and Anna Sparks. Neither author has any financial interest in the tests used in this study. FUNDING The only line of funding in the current investigation came with the donation of Advanced Interpretation credits by Green s Publishing. REFERENCES An, K. Y., Zakzanis, K. K., & Joordens, S. (2012). Conducting research with non-clinical healthy undergraduates: Does effort play a role in neuropsychological test performance. Archives of Clinical Neuropsychology, 27, Blaskewitz, N., Merten, T., & Kathmonn, N. (2008). Performance of children on the symptom validity tests: TOMM, MSVT, and FIT. Archives of Clinical Neuropsychology, 23, Brockhaus, R., & Merten, T. (2004). Neuropsychologische Diagnostik suboptimalen Leistungverhaltens mit dem Word Memory Test [Diagnosing suboptimal effort on neuropsychological tests with the Word Memory Test]. Nervenartz, 75, Bush, S. S., Ruff, R. M., Troster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H.,... Silver, C. H. (2005). Symptom validity assessment: Practice issues and medical necessity. NAN Policy and Planning Committee. Archives of Clinical Neuropsychology, 20, Carone, D. A. (2008). Children with moderate=severe brain damage= dysfunction outperform adults with mild-to-no brain damage on the Medical Symptom Validity Test. Brain Injury, 22, Carone, D. A. (2009). Test review of the Medical Symptom Validity Test. Applied Neuropsychology, 16, Chafetz, M. (2008). Malingering on the Social Security Disability Consultative Exam: Predictors and base rates. The Clinical Neuropsychologist, 22, Constantinou, M., Bauer, L., Aahendorf, L., Fisher, J. M., & McCaffrey, R. J. (2005). Is poor performance on recognition memory effort measures indicative of generalized poor performance on neuropsychological tests? Archives of Clinical Neuropsychology, 20, Dunn, T. M., Shear, P. K., Howe, S., & Ris, M. D. (2003). Detecting neuropsychological malingering: Effects of coaching and information. Archives of Clinical Neuropsychology, 18, Faust, D., Hart, K., Guilmette, T., & Arkes, H. R. (1988). Neuropsychologists capacity to detect adolescent malingerers. Professional Psychology: Research and Practice, 19, Fox, D. D. (2011). Symptom validity test failure indicates invalidity of neuropsychological tests. The Clinical Neuropsychologist, 25, Frederick, R. I., & Foster, H. G. (1991). Multiple measures of malingering on a forced-choice test of cognitive ability. Psychological Assessment, 3, Frederick, R. I., & Speed, F. M. (2007). On the interpretation of below-chance responding in forced-choice tests. Assessment, 14, Gervais, R. O., Rohling, M. L., Green, P., & Ford, W. (2004). A comparison of the WMT, CARB, and TOMM failure rates in non-head injury disability claimants. Archives of Clinical Neuropsychology, 19, Green, P. (2003). Green s Word Memory Test for Windows: User s manual. Edmonton, AB, Canada: Green s Publishing. Green, P. (2004). Green s Medical Symptom Validity Test (MSVT) for Microsoft Windows: User s manual. Edmonton, AB, Canada: Green s Publishing. Green, P. (2007). The pervasive influence of effort on neuropsychological tests. Physical Medicine and Rehabilitation Clinics of North America, 18, Green, P. (2008a). Advanced Interpretation Program. Edmonton, AB, Canada: Green s Publishing. Green, P. (2008b). Manual for the Nonverbal Medical Symptom Validity Test. Edmonton, AB, Canada: Green s Publishing. Green, P., Allen, L., & Astner, K. (1996). Manual for the computerized Word Memory Test. Durham, NC: Cognisyst. Green, P., & Astner, K. (1995). Manual for the Oral Word Memory Test. Durham, NC: Cognisyst. Green, P., Flaro, F., Brockhaus, R., & Montijo, J. (2012). Performance on the WMT, MSVT, & NV-MSVT in children with developmental disabilities and in adults with mild traumatic brain injury. In C. R. Reynolds & A. Horton (Eds.), Detection of malingering during head injury litigation (2nd ed.). New York, NY: Plenum. Green, P., Montijo, J., & Brockhaus, R. (2011). High specificity of the Word Memory Test and Medical Symptom Validity Test in groups with severe verbal memory impairment. Applied Neuropsychology, 18,

10 Green, P., Rohling, M. L., Lees-Haley, P. R., & Allen, L. M. (2001). Effort has a greater effect on test scores than severe brain injury in compensation claimants. Brain Injury, 15, Heaton, R. K., Smith, H. H., Lehman, A. W., & Vogt, A. T. (1978). Prospects for faking believable deficits on neuropsychological testing. Journal of Consulting and Clinical Psychology, 46, Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. L., Millis, S. R., & Conference Participants. (2009). American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23, Henry, M., Merten, T., Wolf, S. A., & Harth, S. (2010). Nonverbal medical symptom validity test performance of elderly healthy adults and clinical neurology patients. Journal of Clinical and Experimental Neuropsychology, 32, Howe, L. L., & Loring, D. W. (2009). Classification accuracy and predictive ability of the Medical Symptom Validity Test s Dementia Profile and General Memory Impairment Profile. The Clinical Neuropsychologist, 23, Merten, T., Green, P., Henry, M., Blaskewitz, N., & Brockhaus, R. (2005). Analog validation of German-language symptom validity tests and the influence of coaching. Archives of Clinical Neuropsychology, 20, Meyers, J. E., Volbrecht, M., Axelrod, B. N., & Reinsch-Boothby, L. (2011). Embedded symptom validity tests and overall neuropsychological test performance. Archives of Clinical Neuropsychology, 26, PVTS IN SIMULATORS 155 Rohling, M. L., & Boone, K. B. (2007). Future directions in effort assessment. In K. B. Boone (Ed.), Assessment of feigned cognitive impairment (pp ). New York, NY: Guilford. Singhal, A., Green, P., Ashaye, K., Shankar, K., & Gill, D. (2009). High specificity of the Medical Symptom Validity Test in patients with very severe memory impairment. Archives of Clinical Neuropsychology, 24, Slick, D., Sherman, E., & Iverson, G. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. Clinical Neuropsychology, 13, Suhr, J. A., & Gunstad, J. (2007). Coaching and malingering: A review. In G. J. Larrabee (Ed.), Assessment of malingered neuropsychological deficits (pp ). New York, NY: Oxford University Press. Tan, J. E., Slick, D. J., Strauss, E., & Hultsch, D. F. (2002). How d they do it? Malingering strategies of symptom validity tests. The Clinical Neuropsychologist, 16, Tydecks, S., Merten, T., & Gubbay, J. (2006). The Word Memory Test and the One-in-Five Test in an analogue study with Russian speaking participants. International Journal of Forensic Psychology, 1, Wager, J. G., & Howe, L. L. S. (2010). Nonverbal Medical Symptom Validity Test: Try faking now! Applied Neuropsychology, 17, Weinborn, M., Woods, S. P., Nulsen, C., & Leighton, A. (2012). The effects of coaching on the verbal and nonverbal medical symptom validity tests. The Clinical Neuropsychologist, 26,