Published online: 12 Dec 2014.

Transcription

1 This article was downloaded by: [ ] On: 26 August 2015, At: 23:28 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: Registered office: 5 Howick Place, London, SW1P 1WG Applied Neuropsychology: Adult Publication details, including instructions for authors and subscription information: Results From Three Performance Validity Tests (PVTs) in Adults With Intellectual Deficits Paul Green a & Lloyd Flaro a a Private Practice, Edmonton, Alberta, Canada Published online: 12 Dec Click for updates To cite this article: Paul Green & Lloyd Flaro (2015) Results From Three Performance Validity Tests (PVTs) in Adults With Intellectual Deficits, Applied Neuropsychology: Adult, 22:4, , DOI: / To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content ) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at

2 APPLIED NEUROPSYCHOLOGY: ADULT, 22: , 2015 Copyright # Taylor & Francis Group, LLC ISSN: print/ online DOI: / Results From Three Performance Validity Tests (PVTs) in Adults With Intellectual Deficits Paul Green and Lloyd Flaro Private Practice, Edmonton, Alberta, Canada Downloaded by [ ] at 23:28 26 August 2015 Previous studies of performance on the Word Memory Test (WMT; Green, 2003; Green & Astner, 1995) in adults with very low intelligence have provided conflicting evidence. Most studies suggest that a Full-Scale IQ (FSIQ) less than 70 cannot explain failure on the WMT, but Shandera et al. (2010) suggest that many adults with mental retardation (MR) cannot pass the WMT. If so, we would expect adults with such low intelligence to fail the WMT at a high rate, even if they were motivated to perform well. In the current study, parents with an FSIQ of 70 or less, who were seeking custody of their children, rarely failed the WMT or the Medical Symptom Validity Test (MSVT; Green, 2004). They did not fail the WMT or MSVT any more often than adults of higher intelligence. On the other hand, adults with an external incentive to appear impaired scored significantly lower on the WMT and MSVT than did parents with an incentive to look good. The data strongly suggest that MR with an FSIQ in the range of 46 to 70 is not sufficient to explain failure on these performance validity tests by adults. Key words: effort, intellectual disability, malingering, Medical Symptom Validity Test, mental retardation, MSVT, performance validity test, PVT, WMT, Word Memory Test INTRODUCTION A fundamental principle of performance validity tests (PVTs) is that although they were designed to be easy for most people with genuine cognitive impairment, any PVT so far devised will be failed by some people who are at the extreme lower end of the ability spectrum. A false positive for poor effort occurs when a person fails a PVT because of actual cognitive impairment and not because of suboptimal effort. Hence, those with the most severe cognitive impairment are the most likely to be false positives on PVTs. Merten, Bossink, and Schmand (2007) called this the limit to effort testing and cited PVT failures in people hospitalized with obvious and very severe neurological diseases. In such cases, low PVT results could Dr. Flaro has no vested interest and no financial interest in the tests used in this retrospective analysis of his clinical data. Dr. Paul Green is the inventor of the performance validity tests used in this study and his company is the sole supplier of these tests. Address correspondence to Paul Green, Ave., Edmonton, AB T5S 1K7, Canada. paulgreen@shaw.ca be valid and might reflect the person s actual impairment. However, in other cases, failures of the same PVT will occur because of suboptimal effort or because of deliberate effort to perform badly, and they would be true positives for suboptimal effort. In such cases, it has been shown by many independent studies that other neuropsychological test results are likely to be invalid and to underestimate true ability (Fox, 2011; Green, Rohling, Lees-Haley, & Allen, 2001; Meyers, Volbrecht, Axelrod, & Reinsch-Boothby, 2011; Stevens, Friedel, Mehren, & Merten, 2008). It is important that we should be able to discriminate between PVT failures owing to actual severe impairment versus feigned impairment or just poor effort. Within the Slick, Sherman, and Iverson (1999) criteria for malingering, Criterion D requires a critical informed decision by the examiner about whether the person has a neurological, psychiatric, or developmental condition that is sufficient to account fully for any PVT failure. Failure owing to response bias is identifiable, in principle, because the person fails a PVT but does not have a diagnosis that can explain failure on such very easy tests. However, Slick Criterion D assumes

3 294 GREEN & FLARO that we have good empirical evidence showing which diagnostic conditions are sufficient to account for failure on any given PVT and which are not. This assumption is often not justified by the existing data, either because people with genuine severe impairment and a specific diagnosis have not been tested with a particular PVT or because such people have been tested but it is not known whether effort was maximal among those who failed the PVT. One diagnosis that is known to be associated with a high rate of failures on PVTs is dementia of the Alzheimer s type. A substantial proportion of cases with dementia in the test manual failed the Test of Memory Malingering (Tombaugh, 1996, pp ). Similarly, Green, Montijo, and Brockhaus (2011) found some patients with dementia failed the easy subtests of the Word Memory Test (WMT; Green, 2003; Green & Astner, 1995) although they all displayed dementia profiles (i.e., they scored much higher on the easy subtests than on the harder subtests). Singhal, Green, Ashaye, Shankar, and Gill (2009) found that some patients with dementia failed the recognition memory subtests of the Medical Symptom Validity Test (MSVT; Green, 2004), but it was presumed that their effort was satisfactory. They were all producing profiles of scores on the MSVT that are typical of people with dementia. On the Nonverbal Medical Symptom Validity Test (NV-MSVT), some people with dementia failed the recognition measures, but once again, they all produced profiles of scores typical of people with dementia (Henry, Merten, Wolf, & Harth, 2010; Singhal et al., 2009). Such dementia-like profiles are defined operationally within the Advanced Interpretation (AI) computer program (Green, 2008a), in which they are called Genuine Memory Impairment Profiles (GMIP). The data are automatically considered to be unreliable if both Criteria A and B are failed on the WMT or MSVT or if Criterion A and two or more B criteria are failed on the NV-MSVT. These are called poor effort profiles, which are typical of a substantial proportion of simulators but are almost nonexistent in people with dementia (Iverson, Green, & Gervais, 1999; Singhal et al., 2009). For more details of profile analysis and the typical profiles seen in people who are genuinely unable to pass these PVTs, please see Green, Flaro, Brockhaus, and Montijo (2012). Mental retardation (MR), which will soon be called intellectual disability in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (Regier, Kuhl, & Kupfer, 2013), is a condition in which intellectual abilities are very significantly below the normal mean and day-today functioning is very low. By definition, if people have a Full-Scale IQ (FSIQ) of 70, their intelligence is lower than that of 49 out of 50 people in the general population, and therefore, they could be at high risk for failing many PVTs. However, suitable empirical data are needed to help us to decide whether or not failure on any given PVT is typical of people with an FSIQ of 70 or less. Carone (2014) presented the case of a 9-year-old girl whose FSIQ was only 58. She had a major developmental loss of brain tissue on magnetic resonance imaging and suffered from chronic epilepsy, which was poorly controlled despite multiple anticonvulsants, including high-dose benzodiazepines. She scored perfectly on the recognition memory subtests of the MSVT and almost perfectly on the WMT recognition subtests, whereas her scores on the more difficult memory subtests were severely impaired. Carone s study showed that even an FSIQ less than 60 in a child combined with impaired memory does not necessarily lead to failure on the WMT or MSVT. In the study by Brockhaus and Merten (2004), with only one exception, the institutionalized adults with MR passed the recognition subtests of the WMT. The sole case failing the WMT was later found to be a person with a borderline personality disorder who was malingering MR to obtain accommodation. There were no failures in the healthy control group, but all 100 simulators failed the WMT, implying 100% specificity in people with MR and 100% sensitivity to poor effort. When there is a 0 failure rate on a PVT in a given group, as in the Brockhaus and Merten study, there is no need to control for external incentives. It is clear that the test has 100% specificity in that group. However, when failures occur, the question of motivation becomes important. Have they failed because they want to appear impaired or because they really cannot pass? Alternatively, have they failed merely because of a lack of a strong positive incentive to try hard? Roughly half of the low-intelligence adults studied by Chafetz, Prentkowski, and Rao (2011) had an FSIQ of less than 70, and in those with a disability incentive, failure of the MSVT was frequent. In contrast, no MSVT failures occurred when these adults were trying to regain legal custody of their children. They had an incentive to appear cognitively intact. Chafetz et al. concluded that it was not low ability but motivation to appear impaired that caused failure on the MSVT. This finding echoed the conclusions of an earlier study by Flaro, Green, and Robertson (2007), who reported WMT results from parents seeking custody of their children from the court. Usually, such parents are highly motivated to appear cognitively intact, and in keeping with this expectation, only 2 cases out of 118 failed the WMT (1.7% of cases). The 2 cases who failed the WMT turned out to be women of normal-range intelligence who admitted that they no longer wanted their children to be returned to their care. Those with an FSIQ less than 70 did not fail the WMT. Shandera et al. (2010) reported lower-than-normal mean scores on the recognition memory subtests of the WMT in adults with mild MR and a mean FSIQ of 63. Yet these adults were residents in the community and they were their own legal guardians, meaning that they were presumably much higher functioning in daily life than the institutionalized adults with MR studied by Brockhaus and Merten

4 (2004), all of whom passed the WMT. This discrepancy leads us to ask whether adults diagnosed with MR in Shandera et al. s study failed the easy WMT subtests because of poor effort or because they really could not pass even if they made their best possible effort. Relevant to their motivation, the authors noted that the adults were not seeking disability payments, but this does not necessarily imply that they had a strong incentive to try their best on testing. Often, authors assume that effort is good, but the assumption of good effort without convincing evidence is not a sound criterion on which to base scientific findings. An, Zakzanis, and Joordens (2012) reported that more than half of their undergraduate student volunteers failed effort tests and produced invalid cognitive test results. They had no disability or litigation incentive nor did they have a strong incentive to try their best. One of the authors of the Shandera et al. study reported that most or all of the community-dwelling adults with MR in the latter study were already collecting disability payments (Jordan Harp, personal communication, March 20, 2014). Thus, they might have had an incentive to perform poorly to maintain their disability. Most of the people studied by Green et al. (2001) were also not seeking disability. They were already receiving disability payments, but many of them failed the WMT and effort explained 50% of the variance in the whole test battery. Similarly, adults with fibromyalgia who already received disability payments failed the WMT far more often than did those with no disability claim (Gervais et al., 2001). Already being in receipt of disability payments was strongly associated with WMT failure in this study. Another study revealed that a simple step to enhance motivation made a big difference in the WMT scores of certain developmentally disabled children (Green & Flaro, 2003). The children who failed the WMT the first time they took it were asked to take it a second time and to try their best in return for the small incentive of a candy or pop that was visible on a shelf. They all passed except for a child with oppositional defiant disorder who scored worse than chance on retesting. Children with a diagnosis of MR were not analyzed separately at that time because the MR sample was too small, but the study showed how easily motivation to perform simple tasks can be affected by external incentives. Shandera et al. (2010) did not employ the standard method for interpreting WMT scores, which includes profile analysis and the use of both Criteria A and B. A reanalysis of the raw data provided by Anne Shandera (personal communication, October 10, 2013) showed that 33% of their WMT failures had a poor effort profile, which is very rarely found in people with genuine cognitive impairment (Green et al., 2011). Such a profile was typical of the simulators studied by Iverson et al. (1999) and it strongly suggests invalid data. Most of the papers on the WMT and MSVT to date point toward a very low rate of failure in adults with MR, and the PVT RESULTS IN ADULTS WITH INTELLECTUAL DEFICITS 295 Shandera et al. (2010) study stands out because it raises a question about a high rate of false positives on the WMT in adults with MR. One way to address the conflicting findings would be to use the method of Chafetz et al. (2011) and to study adults of very low intelligence who have a strong positive incentive to appear intellectually intact. In the current study, WMT, MSVT, and NV-MSVT (Green, 2003, 2004, 2008b) results were available from a consecutive series of more than 500 adults who had been ordered by the court to undergo assessment of their fitness to be parents and who were seeking custody of their children. A subgroup of these parents had an FSIQ of 70 or less. A separate group consisted of adults with some external incentive to appear impaired, such as seeking disability or academic accommodations. If an FSIQ of 70 or less were sufficient to explain failure on the tests used, we would expect to see more failures on these tests in those with an FSIQ of 70 or less compared to those with an FSIQ greater than 70 (Hypothesis 1). We would expect parents with an FSIQ greater than 70 to produce significantly higher mean scores on the WMT, MSVT, and NV-MSVT recognition subtests compared to parents with an FSIQ of 70 or less (Hypothesis 2). We would expect the parents seeking custody to score higher on the PVTs (Hypothesis 3) and also to fail the PVTs less often than those who had an external incentive to exaggerate impairment (Hypothesis 4). Hypothesis 5 was that the mean WMT recognition scores in our parents with low FSIQ would be in the same range as those of the adults with MR in the Shandera et al. (2010) study. If failure did occur, a secondary question was: What proportion of these cases would automatically be classified as producing results of questionable reliability because they showed a poor effort profile, using the B criteria operationalized within the AI program (Green, 2008a). Would MR create a profile of scores similar to those previously observed in people with Alzheimer s disease? METHOD Parents Seeking Custody The second author provided data from a consecutive series of 515 parents who were seeking to regain custody of their children and who were required by the court to undergo assessment of their fitness to be parents during 1996 to The series included the 118 parents on which the original report by Flaro et al. (2007) was based. The mean age of the sample was 33 years (SD ¼ 8) and their mean FSIQ on the Wechsler Adult Intelligence Scale-Third Edition (WAIS-III) was 95 (SD ¼ 15; Wechsler, 1997). Thirty-three cases had a WAIS-III FSIQ of 70 or less (M ¼ 64.4, SD ¼ 6.3), and 468 cases had an FSIQ greater than 70 (M ¼ 97, SD ¼ 13), bringing the total to 501 cases for

5 296 GREEN & FLARO whom an FSIQ was recorded. Fourteen cases were dropped from the study because their FSIQ was not measured for various reasons, such as refusal to complete testing. Reading level was assessed with the Wide-Range Achievement Test (Wilkinson & Robertson, 2006). There was a highly significant difference (t ¼ 10.2, p <.0001) between the mean reading level of those cases with an FSIQ greater than 70 (M grade ¼ 10.5, SD 2.5, n ¼ 467) versus those with an FSIQ of 70 or less (M grade ¼ 5.7, SD ¼ 2.7, n ¼ 33). Three cases in the low FSIQ group had less than a Grade 3 reading level and the reading score was missing for one case. The primary reasons for the court questioning the parenting abilities of the group as a whole were recorded as follows: major psychiatric disorder and=or drug=alcohol addiction (59.7% of cases), domestic disharmony (9.2%), alleged physical abuse (9.2%), alleged neglect of children (6%), failure to provide protection from predators (3.7%), other concerns regarding parental competence (11.9%) and missing data (0.2%). In those with an FSIQ of 70 or less, the primary reasons for the court questioning parenting abilities were major psychiatric disorder and=or drug=alcohol addiction (61.3% of cases), alleged abuse or neglect of children (12.9%), domestic disharmony (6.5%), and other concerns regarding parental competency (19.4%). Other Adults For comparison with the parents undergoing assessment of their parenting skills for legal purposes, we studied another group composed of 111 adults who were sent for assessment in the context of external incentives to appear impaired. The parenting assessment group and the other adult group did not differ from each other in terms of age, FSIQ, or reading levels (Table 1). Some of the other adults were trying to obtain medical disability status, which would involve being paid financial benefits while not working. Others were being assessed to determine eligibility for financial support through a system called Assured Income for the Severely Handicapped or through the government department called People with Developmental Disabilities. However, the clients referred by the Department of Social Services often had not been told that the assessment would have a bearing on their future benefits. Some were trying to obtain accommodations for learning disabilities in postsecondary education, and there were some with compensation claims for injury at work and who were referred by the local Workers Compensation Board. All of these cases had no strong external incentives to try to appear cognitively intact, and they often had external incentives to appear impaired. Procedures The WMT, MSVT, and=or the NV-MSVT were all administered in the standard manner described in their respective test manuals (Green, 2003, 2004, 2008b), and they were used to assist interpretation of the validity of other tests given in the same assessment. Usually these tests are not recommended for people with less than a Grade 3 reading level. Of three parenting assessment cases with less than a Grade 3 reading level, only one was given the MSVT (and passed) and two were given the NV-MSVT (one passed and one failed). The results of neuropsychological tests, including PVTs, were routinely entered by the second author or his assistant into a spreadsheet to assist in clinical interpretation by providing relevant local normative data. All adults were informed and gave consent for the database to be created and used anonymously for research purposes. None objected to the data being used in this way. The primary outcome measures were (a) whether each person failed the easy recognition memory subtests of each instrument, known in the AI program as Criterion A; and (b) in those who did fail, whether the resulting profile failed the B criteria, as defined in the AI program (Green, 2008a), meaning that they displayed a noncredible poor effort profile, as opposed to a GMIP, which could be credible in someone with sufficiently severe impairment. The AI computer program was designed to assist in the standardization of interpretation of the results from these tests by laying out rules of interpretation explicitly. The program applies the rules automatically to the results of the WMT, MSVT, and NV-MSVT, using a flow chart and written explanations of how to interpret certain patterns of scores. In the adults in the current study who failed the A criterion, the results would TABLE 1 No Differences in Mean Age, FSIQ, or Reading Levels Between Parents With an Incentive to Look Good and Adults With an Incentive to Appear Impaired Reason for Assessment N Mean Age SD Mean Reading Level SD Mean FSIQ SD Fitness to parent Other t value df * 610 Significance ns ns ns FSIQ ¼ Full-Scale IQ. * Differences in df values reflect slight differences in sample sizes for different variables owing to very few cases with missing values.

6 automatically be described as unreliable if the B criterion were failed (or at least two out of three B criteria in the case of the NV-MSVT). However, if the B criteria were not failed, the overall profile on that test would be classified as a GMIP, also known as a severe impairment profile or a dementia-like profile. The B criteria are all based on whether a person scores higher on an objectively easy subtest(s) than on harder subtests or whether they score in approximately the same narrow range on several subtests of equivalent difficulty. Genuine severe impairment leads to much higher scores on easier subtests and to a fairly uniform pattern of scores across subtests of equal difficulty. On the other hand, invalid and unreliable test data tend to incorporate inconsistencies between subtests, such as scoring higher on the more difficult Free Recall (FR) subtest than on the relatively easy Paired Associates Recall (PA) subtest. In people with genuine impairment, scores on the WMT or MSVT FR subtests are invariably lower than on the PA subtests, with the exception of people tested in a language that is completely foreign to them (Richman et al., 2006). Whereas nearly all cases of dementia tested to date who fail Criterion A do not fail the B criteria (Green et al., 2011; Henry et al., 2010) and so have a Genuine Memory Impairment profile (GMIP), it is known that some simulators can achieve a GMIP (e.g., Singhal et al., 2009). What the presence of a GMIP means is that, in accordance with Slick et al. (1999) Criterion D, steps must be taken to decide whether or not the person has a diagnosis that has previously been established empirically to cause failure on the easy recognition memory subtests, such as Alzheimer s disease (Green et al., 2012) or severe dyslexia (Larochette & Harrison, 2012) or a disease that is very likely to be sufficient to do so, such as olivopontocerebellar atrophy, progressive supranuclear palsy, advanced Parkinson s disease, or other conditions meeting Merten et al. s (2007) concept of obvious and severe neurological impairment. In such cases, genuine severe impairment would be concluded as the probable cause of the failure on the easy recognition subtests (Criterion A) if a GMIP is present. If there is no obvious and severe impairment for example, in the case of an adult of premorbid average intelligence with a mild traumatic brain injury (TBI) poor effort would be concluded as the probable cause of the failure on the easy recognition subtests, even if a GMIP is present. The parents seeking custody of children were given selected cognitive and personality tests as part of the evaluation of their fitness to be parents, in addition to interviewing and file review. The adults seen for other reasons were given more comprehensive cognitive testing. The WMT was used in 332 parents, the MSVT was used in 405 cases, and the NV-MSVT was used in 142 cases. In those parents with an FSIQ of 70 or less (n ¼ 33), the WMT was used in 14 cases, the MSVT was used in 17 cases, and the NV- MSVT was used in only 4 cases. In the other adults, the PVT RESULTS IN ADULTS WITH INTELLECTUAL DEFICITS 297 WMT was used in 88 cases, the MSVT was used in 98 cases, and the NV-MSVT was used in 69 cases. In the other adults with an FSIQ of 70 or less (n ¼ 10), the WMT was used in 6 cases, the MSVT was used in 9 cases, and the NV-MSVT was used in 6 cases. RESULTS Table 2 includes the mean scores and the rates of failure on each PVT for all parents seeking custody of children, contrasted with adults with some external incentive to appear impaired. As hypothesized, the adults being assessed for parenting skills rarely failed the recognition memory subtests, and they failed significantly less often than those seen for other types of assessment both on the WMT (chi-square ¼ 6.7, p ¼.009) and the MSVT (chi-square ¼ 13.2, p ¼.0002). There was no significant difference between their failure rates on the NV-MSVT (chi-square ¼ 0.6, p ¼.4). The parents scored significantly higher on the WMT and MSVT primary effort subtests (Immediate Recognition [IR], Delayed Recognition [DR], and Consistency [CNS]) than did the adults with an external incentive to appear impaired, but the differences were only significant for one primary effort subtest of the NV-MSVT (i.e., Delayed Recognition Archetypes [DRA]; see Table 2). Overall, these data show that external incentive is important in determining the results on PVTs. Table 3 shows the results from parents seeking custody broken down into those with an FSIQ of greater than 70 (M ¼ 97, SD ¼ 13) versus those with an FSIQ of 70 or less (M ¼ 64.4, SD ¼ 6.3). The lower FSIQ group also had significantly lower reading skills. The failure rate on the WMT in parents with an FSIQ greater than 70 (11 fail WMT, 212 pass) was not significantly different from the failure rate in parents with an FSIQ of 70 or less (2 fail, 12 pass; chi-square ¼ 2.2, p ¼.13). Similarly, there was no significant difference between the high and low FSIQ groups in MSVT failure rates (chi-square ¼ 0.3, p ¼.6). Too few low-fsiq parents were given the NV-MSVT to allow for an examination of the significance of any difference in NV-MSVT failure rates. On the WMT recognition subtests, the differences between the mean scores of the lower- versus the higher- FSIQ groups were statistically significant, but the magnitudes of the differences were clinically insignificant. For example, the mean score was 94.1% correct on the WMT IR in those with an FSIQ of 70 or less compared with 96.8% on the DR in those with an FSIQ greater than 70 (Table 3). Both of these recognition scores are well above the cutoffs for poor effort defined in the WMT test manual. In the low-fsiq parents who passed the WMT (i.e., the vast majority), the mean IR score was 96.3% correct (SD ¼ 4) and the mean DR score was 96.7% correct (SD ¼ 4). Such scores are indistinguishable from the means produced by healthy adult volunteers of average intelligence listed in the WMT

7 298 GREEN & FLARO TABLE 2 Adults Undergoing Parenting Assessments Failed the WMT and MSVT Significantly Less Often Than Adults With an External Incentive to Appear Impaired N Mean for Parents Seeking Custody Std. Dev. % Failing Criterion A N Mean for Other Adults Std. Dev. % Failing Criterion A F p Value Downloaded by [ ] at 23:28 26 August 2015 Age ns FSIQ ns WMT subtests IR DR CNS % % MC ns PA (n ¼ 13=238) (n ¼ 13=88) 2.0 ns FR MSVT subtests IR DR % % CNS PA (n ¼ 6=299) (n ¼ 10=98) FR NV-MSVT subtests IR ns DR ns CN ns DRA % % DRV ns PA (n ¼ 2=69) (n ¼ 4=72) 1.6 ns FR ns Note. Criterion A is failed if the person scores below the cutoffs on the easy recognition memory subtests. The abbreviation ns means that the difference is not significant at p <.05. WMT ¼ Word Memory Test; MSVT ¼ Medical Symptom Validity Test; FSIQ ¼ Full-Scale IQ; IR ¼ Immediate Recognition subtest; DR ¼ Delayed Recognition subtest; CNS ¼ Consistency subtest; MC ¼ Multiple Choice subtest; PA ¼ Paired Associates subtest; FR ¼ Free Recall subtest; NV-MSVT ¼ Nonverbal Medical Symptom Validity Test; CN ¼ consistency between IR and DR; DRA ¼ Delayed Recognition Archetypes; DRV ¼ Delayed Recognition Variations. program (Green, 2003). There was also no significant difference between the lower-fsiq versus the higher-fsiq parent groups in their mean MSVT IR and DR scores (Table 3). However, consistent with a genuine difference between these groups in memory ability, there were significantly higher scores among the parents with high versus low FSIQ on the more difficult WMT memory subtests (Multiple Choice [MC], PA, and FR) and on the relatively hard MSVT PA and FR subtests. Thus, the parents with lower versus higher FSIQ did differ from each other in memory abilities, but this was not evident to any clinically relevant degree in the recognition memory subtest scores. One of the two cases with a low FSIQ who failed the WMT was incarcerated in a women s prison for drug trafficking and antisocial behaviors, including assault. She failed the WMT, MSVT, and NV-MSVT, and when asked about it, she freely admitted that she did not want to regain custody of her children. This was one of the 2 women out of 118 parents seeking custody previously reported by Flaro et al. (2007). The second parent diagnosed with MR failing the WMT demonstrated significant negative impression management in various parts of the assessment and had a victimized mentality related to her past history of reported abuse and trauma. She demonstrated very passive-aggressive behavior and probably suffered from a borderline personality disorder. She was one of the parents declared to be unfit to parent her children. She was felt to be exaggerating memory complaints and learning problems. In the parents who failed the WMT (n ¼ 13), 6 cases had a poor effort profile indicating internal inconsistencies between subtest scores, which is usually interpreted to mean unreliable data irrespective of diagnosis (Green et al., 2012). In those with an external incentive to appear impaired, 13 cases failed WMT Criterion A and 5 had a poor effort profile. On the MSVT, 2 out of 5 of the parents who failed Criterion A had a poor effort profile. Also on the MSVT, 4 out of 10 parents with an external incentive to appear impaired and who failed Criterion A had a poor effort profile. Of the two parents who failed the NV-MSVT, both cases had profiles indicating poor effort (meeting Criterion A and all three B criteria). Of the 4 adults with an external incentive to appear impaired and who failed NV- MSVT Criterion A, 3 failed all three B criteria and 1 failed two B criteria. Thus, all 4 would be classified as poor effort within the AI program (Green, 2008a). Table 4 shows all of the raw data on the WMT from the 10 community volunteers from the study of Shandera et al.

8 PVT RESULTS IN ADULTS WITH INTELLECTUAL DEFICITS 299 TABLE 3 There Were No Significant Differences in Rates of Failure on the WMT and MSVT in Parents With Low Versus Higher FSIQ N Mean for Parents With FSIQ 70 SD % Failing Criterion A N Mean for Parents With FSIQ > 70 SD >% Failing Criterion A F p Value Value Size of Difference Downloaded by [ ] at 23:28 26 August 2015 Age ns 0.2 Reading FSIQ WMT subtests IR DR CNS ns 2.4 MC % % PA (n ¼ 2) (n ¼ 11) FR MSVT subtests IR ns 0.1 DR ns 0 CNS % % 0.01 ns 0.1 PA (n ¼ 0) (n ¼ 5) FR NV-MSVT subtests IR DR CN DRA % % DRV (n ¼ 0) (n ¼ 2) PA FR (2010), who were asked to make a good effort and were assumed by the authors to be making a good effort. The PA and FR subtests were not administered. It can be seen that failure occurred in half of these cases using the rules of interpretation in the AI program (Green, 2008a). For test Sample too small to calculate significance of differences. WMT ¼ Word Memory Test; MSVT ¼ Medical Symptom Validity Test; FSIQ ¼ Full-Scale IQ; IR ¼ Immediate Recognition subtest; DR ¼ Delayed Recognition subtest; CNS ¼ Consistency subtest; MC ¼ Multiple Choice subtest; PA ¼ Paired Associates subtest; FR ¼ Free Recall subtest; NV-MSVT ¼ Nonverbal Medical Symptom Validity Test; CN ¼ consistency between IR and DR; DRA ¼ Delayed Recognition Archetypes; DRV ¼ Delayed Recognition Variations. security purposes, the rules are not specified here. In absolute terms, the mean WMT scores from these supposedly healthy community volunteers fall below the means from the parents with an average FSIQ of 64 in the current study (Table 3). This is the opposite of what we would expect if TABLE 4 Raw Data From Community Volunteers Asked to Make a Good Effort in the Study by Shandera et al. (2010) Case Number WMT Scores in % Correct WMT IR WMT DR WMT CNS WMT MC Overall Pass or Fail on WMT Pass * Fail Pass * 70 * Fail * * 50 * Fail * * 65 * Fail Pass Pass Pass * 75 * 55 * Fail Mean 90.5% 94.5% 88% 76.5% 5 passes 5 failures Note. Raw data supplied by Dr. Anne Shandera. Data from the Paired Associates and Free Recall subtests are missing, which prevents profile analysis. WMT ¼ Word Memory Test; IR ¼ Immediate Recognition subtest; DR ¼ Delayed Recognition subtest; CNS ¼ Consistency subtest; MC ¼ Multiple Choice subtest. * Denotes a score that is below the cutoff for poor effort used in the Advanced Interpretation program (Green, 2008a).

9 300 GREEN & FLARO an FSIQ less than 70 caused WMT failure. The mean WMT scores from Shandera et al. s community-dwelling adults are also about 1 standard deviation lower than the mean scores from the group of patients with severe TBI with a mean Glasgow Coma Scale score of 5, listed in the WMT computer program (Green, 2003), who scored as follows: IR ¼ 97.3% correct (SD ¼ 3.5); DR ¼ 96.6% (SD ¼ 4); CNS ¼ 95.7% (SD ¼ 4); MC ¼ 88.2% (SD ¼ 14); PA ¼ 79% (SD ¼ 20); and FR ¼ 45.2% (SD ¼ 18.4). The WMT data from the community volunteers in the Shandera et al. study are of questionable validity, and as a whole, they are not consistent with full effort in healthy community-dwelling adults. DISCUSSION Hypothesis 1 was that if low intelligence were sufficient to cause failure on the WMT, we would expect the parents in the current study with an FSIQ of 70 or less (M ¼ 64) to fail the WMT recognition memory subtests significantly more often than parents of higher FSIQ (M ¼ 97). They did not do so, as shown in Table 3. There was also no clinically significant difference between the mean WMT recognition scores in the lower- versus higher-fsiq groups, contrary to Hypothesis 2. Similarly, we might expect more failures on the MSVT in the parents with lower FSIQ versus higher FSIQ, but this was not found either (Table 3). We would expect MSVT recognition subtest scores to be significantly lower in the parents with low FSIQ compared with those of much higher FSIQ, but there was no significant difference (Table 3). Thus, low FSIQ did not have the hypothesized effects on the main effort measures of the WMT or the MSVT. On the other hand, differences between groups were noted based on differences in motivation. In keeping with Hypotheses 3 and 4, the parents seeking custody of their children scored significantly higher on the WMT and MSVT effort subtests and failed less often than adults with an external incentive to appear impaired. The results show that external incentives do make a difference in performance on these PVTs (Table 2), but FSIQ does not have an appreciable effect (Table 3). The adults with MR in Shandera et al. (2010) were of similar intelligence to the adults with low FSIQ in the current study (M ¼ 63 vs. 64). Hypothesis 5 was that the mean WMT scores in our parents with low FSIQ would be in the same range as those of the MR group in the Shandera et al. study, who had mean scores of 78% on WMT IR and 80% on WMT DR, but this did not happen. The observed mean scores in the parents with low FSIQ shown in Table 3 (94.1% on WMT IR and 94.6% on DR) were substantially higher than the mean scores from the adults with MR in the Shandera et al. study, and they were close to the mean scores of 97% correct expected from healthy adults (Green, 2003). Even though the two groups were of equally low intelligence, the mean WMT IR score of Shandera et al. s MR adults was 2.2 standard deviations lower than that of our parent group with low FSIQ. Similarly, Shandera et al. s adults with MR scored more than 2 standard deviations lower on WMT DR than did our parents with low FSIQ. Not only did the current parents with low FSIQ perform very well on the WMT recognition subtests, but they also did well on the recognition memory subtests of the MSVT. They scored, on average, only 1 percentage point lower than the mean MSVT IR and DR scores of 99% correct from a sample of healthy adults with a mean of 17 years of education and 1 point lower than the mean from German university students trying their best (Richman et al., 2006). The differences between Shandera et al. s (2010) adults with MR and the parents with low FSIQ in the current study cannot be explained on the basis of a difference in FSIQ because there was no such difference. Where the two groups did differ was in terms of motivation. The parents in the current study had a high incentive to try their best to obtain custody of their children, and they performed relatively well on the WMT, whereas the adults in the Shandera et al. study did not have a strong positive incentive and they performed poorly. They probably had an incentive not to try too hard for fear of losing their benefits. A lack of positive incentives to do well, combined with an incentive to underperform to avoid losing benefits, would explain why the adults in Shandera et al. s study with MR scored so much lower than the adults with low FSIQ in the current study. An unexpected observation was that the current low-fsiq parent sample scored higher on WMT IR than Shandera et al. s (2010) healthy community volunteers, whose mean WMT IR score was only 90.5% correct. This is a remarkable finding. There are various indicators in Table 4 that the community volunteers in the Shandera et al. study were either not making a valid effort or were not healthy community volunteers. The cutoffs for failure on the WMT recognition subtests were based on scores 3 standard deviations below the mean from that of patients with moderate to severe brain injury (Allen & Green, 1999). Yet half of the community volunteers failed the WMT, using the rules laid out in the AI program. The mean MC score from the Shandera et al. community volunteers was only 76.5%. This is worth mentioning because in a group of mixed neurological patients selected for having impaired memory and cited in the WMT program (Green, 2003), the mean MC score was 81% (SD ¼ 12). We would not expect Shandera et al. s (2010) healthy community volunteers to score lower than patients with TBI, lower than neurological patients with impaired verbal memory, and lower than our sample of parents with an FSIQ of 70 or less. The fact that Shandera et al. s community volunteers scored so low leads us to question their motivation. It appears that most or all of them were adults recruited from an unemployment office

10 and were offered money to take part in a psychological study. It is optimistic to select such a low-functioning group and then to assume that they will make a full effort when the money is not contingent upon passing the effort tests. The raw WMT scores from the community volunteers shown in Table 4 suggest that they did not make a full effort. The usual interpretation of such WMT results would be that half of the group was probably not putting forth sufficient effort to produce valid test results. As Chafetz et al. (2011) stated in their study of adults with low FSIQ, it is motivation and not low ability that leads to failure on these PVTs. Because motivation has such a large influence on test scores, future studies of motivation and effort would ideally use not only people with an incentive to do poorly, but also people who have a strong positive incentive to do well. Only a small percentage of the hundreds of parents currently tested failed the PVTs. In the parents seeking custody who failed, it is not certain in some cases whether poor effort or actual impairment led to the failure. Retrospectively, we did not have thorough information about the particular motivational factors affecting every case. However, in parents seeking custody, the failure rates were only 5% on the WMT, 2% on the MSVT, and 2.8% on the NV- MSVT. Even if all these cases were false positives for poor effort, which is very unlikely, these tests would still have very acceptable specificity. Moreover, the failure rate in the parents of low FSIQ did not differ from the failure rate in those with higher FSIQ. Unlike the study by Green and Flaro (2003), we did not manipulate motivation by asking those who failed the PVTs to take them again with an external incentive to pass. However, we were able to show that adults with a strong positive incentive to do well scored higher than those who had no such incentive and who might have had incentives to do poorly. It might be asked why there was no significant difference in failure rates on the NV-MSVT between parents seeking custody and adults with external incentives. The NV-MSVT was only given to 28% of the whole sample. In that particular subgroup, there were also no differences in failure rates on the WMT or MSVT between parents and other adults. In that subgroup, the failure rate on the NV-MSVT was only 5% in those with an external incentive, and this is much lower than typically seen in adult groups with disability claims or in litigation. One possibility is that whereas there were monetary awards contingent on results, many of the adults in the current study were not fully aware of the possible consequences of the assessment. For example, the Department of Social Services would often make a referral because they were considering an application on behalf of the client for longterm support from the government but the client had not been made aware of this possibility. For such people, there was an external incentive available, but they were unaware of it and not motivated to do poorly. It is hard to tell in PVT RESULTS IN ADULTS WITH INTELLECTUAL DEFICITS 301 retrospect, but it is possible that more of those cases fell into the subgroup given the NV-MSVT than into the remainder of the adult group. This would explain why the atypical subgroup did not differ significantly from the parents seeking custody in terms of failure on the NV-MSVT and the other PVTs. All cases in this study were tested by one psychologist or his assistants. There is always the possibility that some psychologists are particularly good at motivating adults to work hard, and in theory, failure rates might differ from one tester to another. Ideally, we would study the performances of several groups tested by different people. In addition, Shandera et al. (2010) used a WMT format that is not recommended for routine testing. They used the oral form of the WMT, not the computerized form. The oral form introduces room for error because different testers pronounce words differently, they may differ in rates of presentation, and their verbal and nonverbal responses to the client may affect performance. The examiner may inadvertently affect stimulus presentation and thereby alter the results. In contrast, the computer program used in the current study presents stimuli in an identical way to everyone, with perfect timing such that all cases get exactly the same stimuli and scoring is error-free. Computerized WMT testing is more standardized and is preferable to oral testing. However, it may be noted that Brockhaus and Merten (2004) also used the oral WMT, but they found no failures in those with genuine MR. The probability must be borne in mind that if FSIQ is low enough, a person will be expected to fail these PVTs, but it is unclear at present how low the FSIQ needs to be to cause failure. The range of FSIQ scores in the parents of low intelligence in this study was from 46 to 70. It appears that FSIQ scores in this range are not sufficient to lead to failure on the WMT, MSVT, or NV-MSVT. It is likely that some adults with low FSIQ will fail the recognition subtests of the WMT, MSVT, and NV-MSVT if they have certain comorbidities. One condition that is known to affect such scores is severe dyslexia (Larochette & Harrison, 2012), but this would not explain failures of the WMT in the current study because none of the parents failing the WMT had less than a Grade 5 reading level and Grade 3 is the minimum required reading level (Green & Flaro, 2003). It is also likely that some children with temporal-lobe epilepsy and an FSIQ less than 70 could fail these PVTs, although they would be expected to produce GMIP-type profiles in most cases. Severe executive impairment might also have an impact, but this remains to be demonstrated. In summary, there is no evidence from this study of any greater risk for failure on the WMT, MSVT, or NV-MSVT in those with an FSIQ of 70 or less than in parents of higher intelligence. On average, the parents with low FSIQ scored well above the cutoffs for poor effort on the WMT, the MSVT, and the NV-MSVT, as defined in the AI program.

11 302 GREEN & FLARO Their recognition memory scores on all tests were hardly distinguishable from the mean scores from healthy adults, although their FR scores were much lower than the healthy adult mean. When a person fails these recognition memory tests and an attempt is made to apply Criterion D of the Slick et al. criteria (1999), the current results suggest that mild MR in adults is not a good explanation of test failure. The results of the current study, combined with those of Brockhaus and Merten (2004), Carone (2014), Green and Flaro (2003), and Flaro et al. (2007) suggest that the adults with MR in the Shandera et al. (2010) study were probably not making a full effort. The results reinforce studies done by others (e.g., An et al., 2012; Chafetz et al., 2011), which emphasize the importance of establishing the actual motivation affecting test performance. It is not sufficient to assume that effort is satisfactory because the participants are not in litigation or are not trying to use the results of the assessment to obtain compensation. When motivation is not properly controlled, we can expect to see effects that are superficially paradoxical, such as adults with low FSIQ scoring higher than healthy adult volunteers. The best explanation for this and other paradoxical effects summarized in this article is that Shandera et al. s adults with MR and their healthy volunteers were not well motivated when taking the tests. We can conclude firmly that having an FSIQ of less than 70 does not explain low scores on the WMT, MSVT, or NV- MSVT recognition memory subtests in any of the groups studied. Following the same logic as Carone (2014), we may conclude that if an FSIQ of 70 or less does not lead to failure on these PVTs, then other conditions involving less impairment also cannot explain failure on these PVTs. One such condition is mild TBI, which is widely reported to involve no measurable cognitive impairment even weeks postinjury (McCrea, 2008). If mild TBI creates no measurable impairment, then it is not logical to argue that it can cause failure on tests that people with MR can easily pass. More precisely, within the Slick et al. (1999) Criterion D, because mild TBI does not involve cognitive impairment, it is not a condition that can explain failure on the recognition memory subtests of the WMT, MSVT, or NV-MSVT. REFERENCES Allen, L., & Green, P. (1999). Severe TBI sample performance on CARB and the WMT in the supplement manuals to the CARB 97 and Word Memory Test. Raleigh, NC: Cognisyst. An, K., Zakzanis, K., & Joordens, S. (2012). Conducting research with non-clinical healthy undergraduates: Does effort play a role in neuropsychological test performance? Archives of Clinical Neuropsychology, 27, Brockhaus, R., & Merten, T. (2004). Neuropsychologische diagnostic suboptimalen leistungsverhaltens mit dem Word Memory Test [The diagnosis of suboptimal effort using the Word Memory Test]. Nervenarzt, 75, Carone, D. (2014). Young child with severe brain volume loss easily passes the Word Memory Test and Medical Symptom Validity Test: Implications for mild traumatic brain injury. The Clinical Neuropsychologist, 28, Chafetz, M., Prentkowski, E., & Rao, A. (2011). To work or not to work: Motivation (not low IQ) determines symptom validity test findings. Archives of Clinical Neuropsychology, 26, Flaro, L., Green, P., & Robertson, E. (2007). Word Memory Test failure 23 times higher in mild brain injury than in parents seeking custody: The power of external incentives. Brain Injury, 21, Fox, D. (2011). Symptom validity test failure indicates invalidity of neuropsychological tests. The Clinical Neuropsychologist, 25, Gervais, R. O., Russell, A. S., Green, P., Allen, L. M., Ferrari, R., & Pieschl, S. D. (2001). Effort testing in patients with fibromyalgia and disability incentives. Journal of Rheumatology, 28, Green, P. (2003). Green s Computerized Word Memory Test for Windows. User s manual. Edmonton, AB, Canada: Green s Publishing. Green, P. (2004). Green s Medical Symptom Validity Test: User s manual. Edmonton, AB, Canada: Green s Publishing. Green, P. (2008a). Advanced Interpretation computer program. Edmonton, AB, Canada: Green s Publishing. Green, P. (2008b). Green s Nonverbal Medical Symptom Validity Test: User s manual. Edmonton, AB, Canada: Green s Publishing. Green, P., & Astner, K. (1995). Oral Word Memory Test: User s manual. Raleigh, NC: Cognisyst. Green, P., & Flaro, L. (2003). Word Memory Test performance in children. Child Neuropsychology, 9, Green, P., Flaro, L., Brockhaus, R., & Montijo, J. (2012). Performance on the WMT, MSVT and NV-MSVT in children with developmental disabilities and in adults with mild traumatic brain injury. In C. Reynolds, & A. Horton (Eds.), Detection of malingering during head injury litigation (2nd ed., pp ). New York, NY: Springer. Green, P., Montijo, J., & Brockhaus, R. (2011). High specificity of the Word Memory Test and Medical Symptom Validity Test in groups with severe verbal memory impairment. Applied Neuropsychology, 18, Green, P., Rohling, M. L., Lees-Haley, P. R., & Allen, L. M. (2001). Effort has a greater effect on test scores than severe brain injury in compensation claimants. Brain Injury, 15, Henry, M., Merten, T., Wolf, S. A., & Harth, S. (2010). Nonverbal Medical Symptom Validity Test performance of elderly healthy adults and clinical neurology patients. Journal of Clinical and Experimental Neuropsychology, 32, Iverson, G., Green, P., & Gervais, R. (1999). Using the Word Memory Test to detect biased responding in head injury litigation. Journal of Cognitive Rehabilitation, 17, 4 8. Larochette, A., & Harrison, A. (2012). Word Memory Test performance in Canadian adolescents with learning disabilities: A preliminary study. Applied Neuropsychology: Child, 1, McCrea, M. (2008). Mild traumatic brain injury and postconcussion syndrome. New York, NY: Oxford University Press. Merten, T., Bossink, L., & Schmand, B. (2007). On the limits of effort testing: Symptom validity tests and severity of neurocognitive symptoms in nonlitigant patients. Journal of Clinical and Experimental Neuropsychology, 29, Meyers, J., Volbrecht, M., Axelrod, B., & Reinsch-Boothby, L. (2011). Embedded symptom validity tests and overall neuropsychological test performance. Archives of Clinical Neuropsychology, 26, Regier, D., Kuhl, E., & Kupfer, D. (2013). The DSM-5: Classification and criteria changes. World Psychiatry, 12, Richman, J., Green, P., Gervais, R., Flaro, L., Merten, T., Brockhaus, R., & Ranks, D. (2006). Objective tests of symptom exaggeration in