The Test of Memory Malingering (TOMM): normative data from cognitively intact, cognitively impaired, and elderly patients with dementia

Transcription

1 Archives of Clinical Neuropsychology 19 (2004) The Test of Memory Malingering (TOMM): normative data from cognitively intact, cognitively impaired, and elderly patients with dementia Gordon Teichner, Mark T. Wagner Department of Neurology, Medical University of South Carolina, 222 W. Coleman Blvd., Mt. Pleasant, SC 29464, USA Accepted 22 May 2003 Abstract This research adds to the psychometric validation of the Test of Memory Malingering (TOMM) by providing data for samples of elderly patients who are cognitively intact, cognitively impaired (non-dementia), and with dementia. Subjects were 78 individuals referred for evaluation of memory complaints. Significant group differences emerged between the dementia group and the two other groups (normals and cognitively impaired), although the latter two did not differ from each other. One hundred percent of normals and 92.7% of the cognitively impaired group made fewer than five errors (the suggested cut-off) on Trial 2 or the Retention trial of the TOMM, yielding an overall correct classification rate of 94.7%. However, the rate of misclassification for persons with dementia was high whether using a cut-point score of five, eight, or ten errors. This investigation extends the validity and clinical utility of this instrument. Results suggest that the TOMM is an useful index for detecting the malingering of memory deficits, even in patients with cognitive impairment, but only when dementia can be ruled out National Academy of Neuropsychology. Published by Elsevier Ltd. All rights reserved. Keywords: Malingering; Test validity; Measurement; Dementia; Geriatrics; Test of Memory Malingering The detection of malingering or purposeful exaggeration of symptoms is a complex and critical issue that neuropsychologists may encounter, especially in the medicolegal arena. The Diagnostic and Statistics Manual of Mental Disorders Fourth Edition (1994) defines malingering as, Corresponding author. Tel.: ; fax: address: teichneg@musc.edu (G. Teichner) /$ see front matter 2003 National Academy of Neuropsychology. doi: /s (03)

2 456 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) the intentional production of false or grossly exaggerated physical or psychological symptoms, motivated by external incentives such as avoiding military duty, avoiding work, obtaining financial compensation, evading criminal prosecution, or obtaining drugs. (p. 683) Patients who malinger symptoms are motivated by an external incentive. Common incentives include receiving compensation, avoidance of responsibility, threat of punishment, or retaliation. The possibility of symptom exaggeration or feigning should be objectively examined when: (1) the patient presents in a medicolegal context such as litigation or application for disability; (2) identifiable incentives for exaggeration are present; (3) symptoms do not make medical, neuropsychological, or psychological sense; (4) the claimed disability is in excess of objective findings; (5) a lack of cooperation during the evaluation occurs; (6) inconsistencies between complaints and behavior are observed; or, (7) contradictions between self-report and medical records are present. The issue of detecting exaggerated or deliberately faked cognitive impairment is paramount in many clinical, medicolegal, and forensic cases. Assessment results often determine whether a patient receives financial or personal gains, or determines the initial course of treatment. Recent times have seen the increased involvement of neuropsychologists in various forensic contexts. This has incited considerable discussion regarding the detection of malingering in the research literature (e.g., Franzen & Iverson, 1997; van Gorp et al., 1999; Gouvier, Hayes, & Smiroldo, 1998; Hartlage, 1998; Heubrock & Petermann, 1998; Pankratz & Binder, 1997; Slick, Sherman, & Iverson, 1999; Williams, 1998), while a plethora of formulas and actuarial effort assessment tools have been developed and put to the empirical method (e.g., Donders, 1999; Frederick & Crosby, 2000; Golden & Grier, 1998; Green, Iverson, & Allen, 1999; Grote et al., 2000; Iverson, Slick, & Franzen, 2000; McKinzey, Podd, Krehbiel, Mensch, & Trombka, 1997; McKinzey & Russell, 1997; Meyers & Volbrecht, 1999; Mittenberg, Rotholc, Russell, & Heilbronner, 1996; Reitan & Wolfson, 1998; Scott-Killgore & DellaPietra, 2000). One such actuarial assessment of testing effort is the Test of Memory Malingering (TOMM), originated by Tombaugh (1996). The TOMM is a 50-item recognition test designed for adults to discriminate between true memory-impaired patients and malingerers. The initial validation studies of the TOMM (see Rees, Tombaugh, Gansler, & Moczynski, 1998; Tombaugh, 1996, 1997) indicate that this instrument is a useful test for detecting exaggerated or malingered memory impairment. Using a criterion cut-off score of 45 on Trial 2 (i.e., 90% correct responding), Tombaugh s investigations (see Rees et al., 1998; Tombaugh, 1996, 1997) revealed correct classification of 95% of all non-demented patients as not malingering. A specificity of 91% was achieved when considering all patients, including those with dementia. Specificity was 100% in an experiment of controls versus simulated malingerers when the same cut-point strategy was employed. In a separate investigation, Tombaugh assessed the performance of patient controls, cognitively intact controls, and litigating versus non-litigating TBI patients on the TOMM. Results indicated that the litigating TBI group scored significantly lower than the other groups, while the non-litigating TBI patients did not differ from controls. Tombaugh s initial validation studies of the TOMM support the validity and clinical utility of this instrument in detecting cognitive malingering. However, our review of the literature revealed no other published studies assessing the psychometric properties of this test or its clinical utility; the only studies published to date appear to be authored by the test originator and

3 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) one of his students (Rees et al., 1998). Independent examination of the psychometric properties of actuarial assessment tools, across the age span and with varying clinical groups, is an assertion commonly echoed throughout the literature. Independent psychometric evaluation of the TOMM, with a clinical sample, appears warranted given its current state of limited empirical support outside of the author s laboratory. The purpose of the current research was to add to the psychometric validation of the TOMM by providing data for samples of cognitively intact elderly subjects, patients with dementia (primarily Alzheimer s Disease and Vascular Dementia), and mixed cognitive impairment. Of particular interest was the rate of specificity (i.e., correct classification as not malingering) of the TOMM in this population. Normative and clinical data in older adults will extend the clinical utility of this tool. Additionally, data derived from elderly subjects with dementia or neurological impairment will strengthen the practitioner s confidence in the TOMM to assist in evaluating the testing motivation of younger patients presenting with cognitive complaints. 1. Methods 1.1. Subjects Subjects were 78 elderly patients referred for neuropsychological and neurological evaluation to a Neurology outpatient clinic located at a large University Hospital. All subjects were referred for a dementia examination given complaints of memory loss, functional decline, or other change in cognition. Data was collected on a serial basis over a 12-month period of time. Most (93.6%) were Caucasian, while the remaining sample was African American (6.4%). The sample s mean age was 70.5 (S.D. = 8.5) and they were well educated (M = 14.0 years, S.D. = 3.3). A total of 33 males and 45 females comprised the group. The majority were married (67.9%), with others being widowed (19.2%), divorced (6.4%), separated (2.6%), or never married (2.6%). Data pertaining to the marital status of two subjects (2.6%) was missing Procedure Subjects underwent comprehensive neurological, neuropsychological, and nursing evaluations. Standard metabolic panels were obtained as part of the dementia workup on all subjects to assist in ruling out competing etiologies that may account for cognitive change. ApoE genotyping and neuroradiological studies (i.e., MRI, SPECT, or CT) were obtained to aid in diagnostic clarification. Neurological evaluation included a basic neurological exam, physical exam, and laboratory studies as described above. Neuropsychological examination was comprised of obtaining a clinical history, neurobehavioral examination, and administration of a comprehensive neuropsychological battery. 1 The nursing exam objectively assessed Activities 1 The standard neuropsychological battery was comprised of the Wechsler Adult Intelligence Test Third Edition (WAIS-III), Wechsler Memory Scale Revised (WMS-R), CERAD Neuropsychological Battery, Mini-Mental State Examination, Trails B, Draw-a-Clock, Hooper, Geriatric Depression Scale, Sentence Completion, and Revised Memory and Behavior Problems checklist. Additional neuropsychological tests were given in selected cases to gain further diagnostic clarification (e.g., Wisconsin Card Sorting Task, Grooved Pegboard, Personality Assessment Inventory).

4 458 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) of Daily Living and Independent Activities of Daily Living by having the subject perform specific tasks in the clinic. Diagnosis was determined by team consensus (neurologist, neuropsychologist, geriatric nurse) based on data collected. Clinical criteria for dementia minimally met the Diagnostic and Statistical Manual Fourth Edition (1994) guidelines. For Dementia of the Alzheimer s Type, an extended ADRDA-NINDS (McKhann et al., 1984) clinical criteria was adopted. The State of California Alzheimer s Disease Diagnostic and Treatment Centers (ADDTC) Ischemic Vascular Disease criteria were used in the determination of Vascular Dementia (Chui et al., 1992). Mild Cognitive Impairment (MCI), Age-Associate Cognitive Impairment, and Cognitive Disorder Not Otherwise Specified were determined by criteria established by the NIH and/or DSM-IV. Other psychiatric illness was determined via DSM-IV diagnostic criteria. On this basis, subjects were divided into the following categories: Dementia (N = 21); Cognitive Impairment, but not meeting criteria for dementia (N = 36); and Cognitively Intact (N = 21). 2 Subjects comprising each group are described in Table Measures Subjects underwent comprehensive evaluation as described above. Given the specific aims of this research, selected tests were chosen for examination; these instruments are described in the following Test of Memory Malingering (TOMM) Tombaugh (1996) The TOMM is a 50-item recognition test designed for adults to discriminate between true memory-impaired patients and malingerers. Subjects are shown 50 drawings for 3 s each, at 1 s intervals. Immediately thereafter, they are given 50 recognition panels, with each panel containing the target picture and a new picture. This is a forced choice task, and the subject is required to indicate the picture previously viewed. Feedback regarding the correctness of the subject s response is offered. Two learning trials are given, and a retention trial is administered after a 10 min delay. Scores range from 0 to 50 for each trial. Tombaugh (1996, 1997) indicates that five or more errors on Trial 2 or the Retention trial indicates the possibility of malingering Wechsler Adult Intelligence Scale Third Edition (WAIS-III) Wechsler (1997) The WAIS-III is the latest version of the Wechsler Adult Intelligence scale. Its psychometric properties are documented in the manual (Wechsler, 1997). This instrument consists of several subtests measuring different facets of intelligence, and its inclusion was to illustrate the general intellectual functioning of the three groups. The three composite scores Verbal, Performance, and Full Scale Intelligence Quotients were used for this study. 2 As indicated in Table 1, 7 (33.3%) subjects were free of psychiatric or neurological illness, while the other 14 (66.6%) subjects had an identifiable psychiatric diagnosis or neurological condition. This mixed group of worried well, psychiatric, and neurological (cognitively intact) patients is a better representation of individuals who may present to a real outpatient clinic. Thus, we feel they served as a more appropriate normal comparison group.

5 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) Table 1 Diagnoses of the sample by group Dementia (N = 21) Probable Alzheimer s Disease (AD) 13 Vascular Dementia (VD) 3 Mixed AD/VD 2 Cortico Basal Ganglionic Degeneration 1 Tumor 1 Frontotemporal Dementia 1 Cognitive Impairment (N = 36) Cognitive Disorder Not Otherwise Specified (vascular etiology) 14 Age Associated Cognitive Decline 11 Parkinson s Disease 3 Closed Head Injury 2 Progressive Supranuclear Palsy 1 Primary Progressive Aphasia 2 Alcohol Encephalopathy 1 Herpes Encephalopathy 1 Probable Pick s Disease 1 Cognitively Intact (N = 21) Major Depressive Disorder or Dysthymia 9 No diagnosis 7 Closed Head Injury 2 Bipolar Disorder 1 Seizure Disorder 1 Lymes Encephalopathy 1 N Wechsler Memory Scale Third Edition (WMS-III); Orientation Wechsler (1997) The Orientation subtest of the WMS-III assesses orientation to person, biographical information, place, time, location, and knowledge of the current and previous Presidents. Scores range from 0 to 14. This measure was used to obtain a general measure of orientation Mini-Mental State Examination (MMSE) Folstein, Folstein, and McHugh (1975) The MMSE is a popular screen for cognitive impairment. Items assess aspects of orientation, attention, language, visuoconstruction, and immediate and delayed recall. Scores range from 0 to 30. A cut-off score of <24 may be strongly suggestive of dementia (see Petersen et al., 2001 for a review). Its inclusion was to illustrate the differences between the three clinical groups in terms of global cognitive functioning. 2. Results 2.1. Demographic variables Demographic group differences were analyzed by ANOVA or Chi-square as appropriate. The groups significantly differed for age, with subjects in the dementia group being

6 460 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) Table 2 Group differences across demographic and neuropsychological variables Dementia (N = 21) Cognitively Impaired (N = 36) Normal (N = 21) Variable M S.D. M S.D. M S.D. F (2, 74) P AGE 75.3 a b b <.01 EDUC ns TOMM Trial a b b <.001 Trial a b b <.001 Retention trial 39.5 a b b <.001 IQ FSIQ 80.6 a b c <.001 VIQ 86.3 a b b <.01 PIQ 77.6 a b c <.01 MMSE 19.9 a b c <.001 ORIEN 9.4 a b b <.001 Note: Means in the same row that share the same superscript letters do not differ significantly from one another at P<.05; means that do not share subscripts differ at P<.05. EDUC: Education; TOMM: Test of Memory Malingering; IQ: Wechsler Adult Intelligence Scale Third Edition, FSIQ: Full Scale Intelligence Quotient; VIQ: Verbal Intelligence Quotient; PIQ: Performance Intelligence Quotient; MMSE: Mini-Mental State Examination; ORIEN: Orientation subtest from the Wechsler Memory Scale Revised. significantly older than the other two groups (see Table 2). Age was correlated with performance on Trial 2 (r =.40, P<.01) and Retention trial (r =.50, P<.01) of the TOMM. Education was not significantly related to performance on Trial 2 (r =.03) or Retention trial (r =.04). No other significant demographic group differences or correlations with performance on the TOMM were observed TOMM and neuropsychological group differences Age was used as a covariate in all analyses given significant group differences. A series of Multivariate Analysis of Covariance (MANCOVA), with age as a covariate, were performed between the three groups. Three separate MANCOVAs were calculated: TOMM trials 1, 2, retention; IQ FSIQ, VIQ, PIQ; Mental status Orientation, MMSE. Univariate tests were calculated using Wilks Lambda. Post hoc group comparisons were then performed using the Tukey procedure to reduce the possibility of Type II error given multiple comparisons. These results are displayed in Table 2. The MANCOVA for the two learning and retention trials of the TOMM was significant, F(6, 144) = 11.02, P<.001. Univariate analyses and pairwise comparisons indicated that the Cognitively Intact and Cognitively Impaired groups did not differ from each other across TOMM trials, while both groups scored significantly better than the Dementia group. Group differences were observed for intellectual functions, F(6, 144) = 3.15, P<.01, as normals fared better than the Cognitively Impaired group who in turn performed superior to those in

7 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) Table 3 Percent of correct responses on the TOMM Percent of correct responses Trials <40 Dementia Trial Trial Retention trial Other neurological Trial Trial Retention trial Normals Trial Trial Retention trial the Dementia group for FSIQ and PIQ. Verbal intelligence did not differ significantly between the Cognitively Intact and Cognitively Impaired groups, yet both did better than individuals in the Dementia group. Not surprisingly, the groups differed in relation to mental status, F(4, 146) = 14.6, P<.001, as the MMSE distinguished the three groups, while performance on the WMS-R Orientation subtest was similar for normals and cognitively impaired; the Dementia group scored significantly lower as would be expected Classification rates using Trial 2 and Retention trial of the TOMM Table 3 presents data on the percent of correct responses across TOMM trials for the three groups. Table 4 displays the false positive rates using various cut-points on Trial 2 and the Retention trial. All elderly normals (100%), and the majority (91.7%) of cognitively impaired elderly made fewer than five errors on these TOMM trials. Using eight errors as a cut point improved correct classification of the Cognitively Impaired group to 94.4% using either Trial Table 4 False positive (FP) rates for cut-off points of five, eight, and ten errors on the TOMM Normals (N = 21) Retention trial Trial 2 Cognitively Impaired (N = 36) Dementia (N = 21) Retention trial Trial 2 Retention trial Trial 2 Cut point (number of errors) N (%FP) N (%FP) N (%FP) N (%FP) N (%FP) N (%FP)

8 462 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) or the Retention trial. In contrast, the Dementia group performed poorly on both Trial 2 and Retention trial. The same cut-point strategy revealed a very high false positive rate whether cut-points of five, eight, or ten errors were used. Thus, if dementia can be ruled out, this data indicates that a cut-point score of five errors yields a high level of specificity (i.e., 94.7% of all non-demented patients are correctly classified), and slightly improved when a cut-off of eight errors is used (96.5%). If dementia cannot be ruled out, then the specificity of the TOMM in this elderly sample was poor whether using five errors (76.9%), eight errors (82.1%), or ten errors (85.9%) on Trial Discussion This research adds to the psychometric validation of the TOMM by examining its performance in a sample of cognitively intact elderly, dementia patients, and older adults with cognitive impairment who do not meet diagnostic criteria for dementia. Patients were extensively evaluated by means of neurological, neuropsychological, and nursing examinations to assure accurate diagnostic clarification. As expected, groups significantly differed across specific measures of cognition (e.g., intellectual functioning, MMSE) adding to the confidence of the diagnostic decision. Results add to the validity of the TOMM as a viable utility in the assessment of testing motivation. Non-demented individuals (normals and cognitively impaired) performed similarly on the TOMM independent of their cognitive status. Results are analogous to those documented in the test manual (Tombaugh, 1996) albeit the original validation samples were considerably younger. A cut-point score of five or more errors on the second learning or retention trials yielded a high rate of correct classification, again comparable to Tombaugh s initial studies. Cut-points of eight or ten errors did not significantly improve the test s specificity. Persons with dementia performed significantly poorer than the other two groups across all trials of the TOMM. In contrast to the initial validation studies, a cut-point of five or more errors on Trial 2 or the Retention trial yielded an unacceptably high rate of false positives. More conservative cut-offs of eight or ten errors did not significantly improve the correct classification of individuals with dementia as non-malingering. Findings of the current initiative extend the clinical utility and validity of the TOMM across the age span to include elderly patients. Further, these results add to the neuropsychologist s confidence of making a correct decision regarding the possibility of cognitive malingering in younger adults (e.g., when presenting for the evaluation of Traumatic Brain Injury, Mild Head Injury) given the performance of elderly psychiatric and neurological patients in the current sample. Data indicate that the specificity for the TOMM is high when dementia can be ruled out. However, the TOMM appears sensitive to detect the severe cognitive dysfunction associated with dementia, and is not a useful measure of assessing test motivation for persons with such deficits. It is not clear why the Dementia group in our sample performed so poorly on the TOMM as compared to Tombaugh s Dementia group who performed relatively well. In the test manual, Tombaugh (1996) indicates that the four individuals with dementia who scored below 40 on Trial 2 were moderately to severely demented, with MMSE scores of 7, 15, 16, and 19. The

9 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) manual does not go into further detail regarding the remaining sample s MMSE performance. The range of MMSE scores of our sample was 11 28, with 9 (42.9%) scoring below 19. Thus, it is possible that our dementia sample was more impaired, as a group, as compared to Tombaugh s sample, resulting in poorer performance on the TOMM. Another possible explanation for the poorer performance of our Dementia group may lie in the method of diagnostic classification. Tombaugh (1996) states that the diagnostic status of his clinical sample was based on DSM-III-R or DSM-IV criteria. Our diagnostic strategy utilized the ADRDA-NINDS or ADDTC criteria for Probable Dementia of the Alzheimer s Type or Vascular Dementia rather than the more liberal definitions offered by the DSM. This leads us to speculate that Tombaugh s adoption of the this more stringent classification strategy may have led to similar findings. At present, this research appears to be the only independent study adding to the psychometric validity of the TOMM. Normative and clinical data are provided for a sample of elderly outpatients presenting with memory complaints. Intact and cognitively impaired (without dementia) elderly performed near perfect on Trial 2 and the Retention trials, similar to younger adults with and without cognitive impairment (Tombaugh, 1996, 1997). The specificity of detecting malingering or poor testing motivation across the adult age span is strengthened in light of these results. Research with other clinical samples and larger groups will continue to add to the clinical utility and psychometric validation of this promising instrument. References American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders Fourth edition. Washington, DC. Chui, H. C., Victoroff, J. I., Margolin, D., Jagust, W., Shankle, R., & Katzman, R. (1992). Criteria for the diagnosis of ischemic vascular dementia proposed by the State of California Alzheimer s Disease Diagnostic and Treatment Centers. Neurology, 42, Donders, J. (1999). Specificity of a malingering formula for the Wisconsin Card Sorting Test. Journal of Forensic Neuropsychology, 1, Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-Mental State. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, Franzen, M. D., & Iverson, G. L. (1997). The detection of biased responding in neuropsychological assessment. In A. M. Horton, D. Wedding, & J. Webster (Eds.), The neuropsychology handbook, Vol. 1: Foundations and assessment(2nd ed., pp ). New York: Springer Publishing Co. Frederick, R. I., & Crosby, R. D. (2000). Development and validation of the Validity Indicator Profile. Law and Human Behavior, 24, Golden, C. J., & Grier, C. A. (1998). Detecting malingering on the Luria Nebraska Neuropsychological Battery. In C. R. Reynolds (Ed.), Detection of malingering during head injury litigation. Critical issues in neuropsychology (pp ). New York: Plenum Press. van Gorp, W. G., Humphrey, L. A., Kalechstein, A., Brumm, V. L., McMullen, W. J., Stoddard, M., & Pachana, N. A. (1999). How well do standard clinical neuropsychological tests identify malingering? A preliminary analysis. Journal of Clinical and Experimental Neuropsychology, 21, Gouvier, W. D., Hayes, J. S., Smiroldo, B. B. (1998). The significance of base rates, test sensitivity, test specificity, and subjects knowledge of symptoms in assessing TBI sequelae and malingering. In C. R. Reynolds (Ed.), Detection of malingering during head injury litigation. Critical issues in neuropsychology (pp ). New York: Plenum Press.

10 464 G. Teichner, M.T. Wagner / Archives of Clinical Neuropsychology 19 (2004) Green, P., Iverson, G. L., & Allen, L. (1999). Detecting malingering in head injury litigation with the Word Memory Test. Brain Injury, 13, Grote, C. L., Kooker, E. K., Garron, D. C., Nyenhuis, D. L., Clifford, A., & Mattingly, M. L. (2000). Performance of compensation seeking and non-compensation seeking samples on the Victoria Symptom Validity Test: Cross-validation and extension of a standardization study. Journal of Clinical and Experimental Neuropsychology, 22, Hartlage, L. C. (1998). Clinical detection of malingering. In C. R. Reynolds (Ed.), Detection of malingering during head injury litigation. Critical issues in neuropsychology (pp ). New York: Plenum Press. Heubrock, D., & Petermann, F. (1998). Neuropsychological assessment of suspected malingering: Research results, evaluation techniques, and further directions of research and application. European Journal of Psychological Assessment, 14, Iverson, G. L., Slick, D. J., & Franzen, M. D. (2000). Evaluation of a WMS-R malingering index in a non-litigating clinical sample. Journal of Clinical and Experimental Neuropsychology, 22, McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlam, E. M. (1984). Clinical diagnosis of Alzheimer s disease: Report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer s Disease. Neurology, 34, McKinzey, R. K., Podd, M. H., Krehbiel, M. A., Mensch, A., & Trombka, C. (1997). Detection of malingering on the Luria Nebraska Neuropsychological Battery: An initial and cross-validation. Archives of Clinical Neuropsychology, 12, McKinzey, R. K., & Russell, E. W. (1997). Detection of malingering on the Halstead Reitan Battery: A cross-validation. Archives of Clinical Neuropsychology, 12, Meyers, J. E., & Volbrecht, M. (1999). Detection of malingerers using the Rey Complex Figure and Recognition trial. Applied Neuropsychology, 6, Mittenberg, W., Rotholc, A., Russell, E., & Heilbronner, R. (1996). Identification of malingered head injury on the Halstead Reitan Battery. Archives of Clinical Neuropsychology, 11, Pankratz, L., & Binder, L. M. (1997). Malingering on intellectual and neuropsychological measures. In R. Rogers (Ed.), Clinical assessment of malingering and deception (2nd ed., pp ). New York: Guilford Press. Petersen, R. C., Stevens, J. C., Ganguli, M., Tangalos, E. G., Cummings, J. L., & DeKosky, S. T. (2001). Practice parameter: Early detection of dementia: Mild cognitive impairment (an evidence-based review). Report of the quality standards subcommittee of the American Academy of Neurology. Neurology, 56, Rees, L. M., Tombaugh, T. N., Gansler, D. A., & Moczynski, N. P. (1998). Five validation experiments of the Test of Memory Malinger (TOMM). Psychological Assessment, 10, Reitan, R. M., & Wolfson, D. (1998). Detection of malingering and invalid test results using the Halstead Reitan Battery. In C. R. Reynolds (Ed.), Detection of malingering during head injury litigation. Critical issues in neuropsychology (pp ). New York: Plenum Press. Scott-Killgore, W. D., & DellaPietra, L. (2000). Using the WMS-III to detect malingering: Empirical validation of the Rarely Missed Index (RMI). Journal of Clinical and Experimental Neuropsychology, 22, Slick, D. J., Sherman, E. M., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. Clinical Neuropsychologist, 13, Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM). New York: Multi-Health Systems, Inc. Tombaugh, T. N. (1997). The Test of Memory Malingering (TOMM): Normative data from cognitively intact and cognitively impaired individuals. Psychological Assessment, 9, Wechsler, D. (1997). Wechsler adult intelligence scale Third edition: Administration and scoring manual. San Antonio, TX: The Psychological Corporation. Williams, J. M. (1998). The malingering of memory disorder. In C. R. Reynolds (Ed.), Detection of malingering during head injury litigation. Critical issues in neuropsychology (pp ). New York: Plenum Press.