Medical Symptom Validity Test Performance Following Moderate-Severe Traumatic Brain Injury: Expectations Based on Orientation Log Classification

Transcription

1 Archives of Clinical Neuropsychology 32 (2017) Medical Symptom Validity Test Performance Following Moderate-Severe Traumatic Brain Injury: Expectations Based on Orientation Log Classification Abstract Stephen N. Macciocchi 1,2, *, Ronald T. Seel 2, Angela Yi 2, Sarah Small 2 1 Atlanta Neuropsychology, GA, USA 2 Shepherd Center, GA, USA *Corresponding author at: P.O. Box 189, Marble Hill, GA USA. Tel.: ; fax: address: snmphd@windstream.net (S.N. Macciocchi). Accepted 12 December 2016 Objective: This study examined performance on the Medical Symptom Validity test (MSVT) during acute rehabilitation for moderatesevere traumatic brain injury (TBI) stratified by Orientation Log (O-Log) scores. Method: Participants were 77 prospectively enrolled persons who sustained moderate-severe TBI and were acutely hospitalized secondary to the cognitive, medical and physical sequelae of their TBI. Participants were administered neuropsychological metrics, the O-Log and the MSVT a mean of 44 days post injury. Results: Significantly lower neurocognitive test scores were observed among participants who remained in post-traumatic amnesia (O-Log scores ranging from 20 to 24) versus those who were oriented (O-Log scores ranging from 25 to 30). MSVT performance was lower among participants who remained in post-traumatic amnesia. When participants O-Log scores were unimpaired (30), performance on the MSVT was also unimpaired on immediate recognition (IR) and delayed recognition (DR). A small percentage of participants performed below MSVT interpretive expectations on CNS. As O-Log scores decreased, MSVT performance also declined on some, but not all MSVT metrics. The sample as a whole performed at or above expectations on MSVT criterion B2 (IR) = 96.6%; (DR) = 94.8%; consistency (CNS) = 92.9%; paired associate (PA) = 86.4% and delayed free recall (FR) = 46.8%. Conclusions: MSVT performance stratified by O-Log scores provides basal expectation levels for persons with acute, moderate-severe impairment in cognitive skills secondary to TBI. Our data demonstrate that persons with significant neurocognitive impairment who are oriented generally perform at or above MSVT interpretive guidelines. Keywords: Malingering/symptom validity testing; Forensic neuropsychology; Head injury; Traumatic brain injury Introduction Performance validity is a topic of significant importance for neuropsychologists working in a variety of settings. The NatIonal Academy of Neuropsychology (NAN) initially addressed the importance of assessing performance validity in 2005 (Bush et al., 2005). The American Academy of Clinical Neuropsychology (AACN) subsequently provided a comprehensive framework for assessing neuropsychological test validity in 2009 based on recommendations from a national and international consensus panel (Heilbronner et al., 2009). Among other recommendations for evaluating the validity of neuropsychological test findings, the AACN and NAN consensus panels recommend the use of empirically validated performance validity tests, in large part based on the amount of variance neuropsychological test scores explained by performance validity metrics (Gidley-Larson et al., 2015; Green, Rohling, Lees-Haley, & Allen, 2001; Kirkwood, Yeates, Randolph, & Kirk, 2012; Stevens, Friedel, Mehren, & Merten, 2008). The Author Published by Oxford University Press. All rights reserved. For permissions, please journals.permissions@oup.com. doi: /arclin/acw112 Advance Access publication on 31 January 2017

2 340 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); There are numerous free standing performance validity tests currently available for use in neuropsychological examinations (Larrabee, 2012). One metric that has been widely used in clinical practice is the Medical Symptom Validity Test (MSVT). The MSVT (Green, 2004) is a computer administered performance validity metric that assesses verbal learning and memory as well as response consistency. The MSVT has a learning phase that involves two presentations of 10 word pairs, one every 6 s, followed by an immediate recognition trial (IR). During the IR trial, the examinee must select a previously viewed word from a list of 20 word pairs. After a 10 min delay, the examinee is administered a delayed recognition trial (DR) during which they must select words viewed during the initial two learning trials from a second, different set of 20 word pairs. Once the IR and DR trials are complete, a consistency index (CNS) is computed. The CNS metric reflects the degree to which the examinee correctly or incorrectly chose words during IR and DR (Green, 2004). CNS does not reflect accuracy, but reflects consistency of correct responding during the IR and DR trials (Green, 2004). In addition to IR, DR and CNS scores, the MSVT has two other metrics. The Paired Associate (PA) trial involves providing the first word of each initially presented word pairs and requires the examinee to provide the correct matching word. Finally, during free recall (FR), the examinee is required to freely recall as many words as possible from the initial word pairs in any order. Once the entire metric has been administered, the computer generates scores for IR, DR, CNS, PA and FR. These scores are presented in printout along with comparative group data for interpretive purposes. For instance, the MSVT summary sheet provides 13 pages of comparative group data, including scores of persons with major psychiatric and neurological disorders as well as children and adults with no history of psychiatric or neurological disorders. There are also data on persons simulating cognitive impairment and persons who have been determined by other metrics such as the Word Memory Test (Green, 2003 [2005]) to have expended optimal and suboptimal effort. According to the MSVT interpretive manual, IR, DR and CNS scores <90% correct are consistent with suboptimal effort (Criteria A). There are also criteria to determine whether the MSVT scores are the product of severe cognitive impairment including Criteria B 1 and B 2. Criteria B 1 stipulates that when the mean of IR, DR and CNS compared to the mean of PA and FR is 20 then the MSVT data are deemed to be unreliable. Conversely, when the difference between the mean of IR, DR and CNS is greater than 30, then MSVT scores may be due the presence of severe cognitive impairment (Singhal, Green, Ashaye, Shankar, & Gill, 2009) and may represent a genuine memory impairment profile (GMIP). GMIP is calculated by the MVST interpretive program. In cases where GMIP > 30, additional clinical and psychometric information is required to determine if severe cognitive impairment is present. Numerous investigators have contributed data that is represented in the MSVT comparative group printout, including data from several groups of persons reported to have sustained moderate-severe traumatic brain injury (TBI). In most cases either the age, education, neuroimaging and neurological condition of examinees is reported in the comparative group tables, but no comparative group is listed based on age, education, acuity and severity of TBI as well as neuropsychological test scores (Green, 2004). Based on the tables contained in the comparative group printout, there is substantial data that shows the MSVT can be easily performed by children and adults with and without psychiatric and neurological conditions. For instance, persons with moderate to severe TBI (N = 19) have been shown to score well above the 90% threshold on IR, DR and CNS (Green, 2004). More recent studies have also shown that impaired intellectual skills do not negatively affect MSVT performance. For instance, (Axelrod & Schutte, 2011) showed that parents with an FSIQ 70 who were pursuing custody of their child did not fail the MSVT more often than adults with higher intelligence. The authors concluded that a FSIQ score ranging from 46 to 70 does not explain failure on the MSVT (Axelrod & Schutte, 2011). In a similar, but different study, (Green & Flaro, 2016) found that children with intellectual disability and at least a third grade reading level did not fail the MSVT more frequently than children with much higher intellectual test scores. Another study revealed generally equivalent results in adults (Green & Flaro, 2015). Based on these two studies, the authors concluded that intellectual disability in children and adults cannot explain failure on the MSVT (Green & Flaro, 2015, 2016). Despite the extensive data on MSVT performance in different clinical groups, there are few studies contained in the existing comparative group database that document MSVT performance of persons with acute, moderate to severe TBI, particularly in a prospective, unselected cohort where neuroimaging and neuropsychological test data are also available. Providing population specific MSVT performance expectations for persons with acute moderate-severe TBI has the potential to assist neuropsychologists who perform clinical and/or forensic examinations, especially in cases where detecting suboptimal effort is important for clinical decision making and particularly in cases where an empirical foundation for diagnosing suboptimal effort is required by law (Heilbronner et al., 2009). Our current study assessed MSVT performance during acute, inpatient neurorehabilitation in a consecutively admitted cohort of adults who had sustained moderate-severe TBI. Our research objectives were to describe MSVT performance following acute, moderate to severe TBI, determine the extent to which TBI severity factors affect MSVT test

3 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); performance and lastly to identify which neuropsychological metrics are most strongly associated with MSVT performance. Based on past research and expectations from comparative group data, we hypothesized that persons with acute, moderate to severe TBI would perform at or above empirically derived MSVT performance expectations ( 90% on IR, DR and CNS) when they were fully oriented. Based on prior research, we also hypothesized that MSVT performance would be negatively affected when neuropsychological test performance was >2.0 standard deviations below normative expectations. Method Setting and Participants Persons participating in the study were consecutive admissions to a specialized, CARF and JCAHO accredited neurorehabilitation hospital located in Atlanta, Georgia. All persons in the study were admitted to a dedicated acute brain injury rehabilitation program secondary to a medically documented history of moderate-severe TBI. Participants were prospectively enrolled over a one year period of time. Persons with a primary diagnosis of cerebral vascular accident or anoxic brain injury were excluded from participation. All persons admitted with a primary diagnosis of a moderate-severe TBI were considered for participation (N = 101). Persons with severe TBI who did not emerge from coma, could not follow commands or who had profound visual or motor dysfunction that precluded administration of any neuropsychological tests were excluded from participation (N = 24). No participants were known to be actively involved in litigation at the time of enrollment. Neuropsychological testing was conducted as part of a standardized clinical assessment completed prior to discharge from the acute TBI neurorehabilitation program Procedure and Measures Demographic data and test scores were prospectively collected and subsequently extracted from the general neuropsychology department clinical measurement database. Legally designated family members provided proxy consent to use deidentified, group clinical data for research and evaluation purposes. Participants also provided assent for participation. The study was approved by the Shepherd Center IRB. Testing was conducted in a dedicated assessment environment by psychometricians who were blinded to study hypotheses. All psychometricians participating in the study had graduate degrees (Masters) and were educated, trained and experienced in assessing persons with moderate-severe TBI. Neuropsychological and performance validity testing was completed a mean of 43.9 days post-injury (SD = 17.5) in a single testing session. Following test administration, scores were calculated for each test and the neuropsychology and research staff performed quality control checks on the data. Following quality control checks, a research coordinator entered the test data in a secure data file. Prior to data entry, identifying information was removed from the research data file to protect patient privacy in compliance with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations. Because the participants were in the acute recovery stage following TBI and because the neuropsychological data obtained was also used for treatment and disposition purposes, only neuropsychological measures assessing executive, processing speed, language and memory skills were administered. Participants were first administered the Orientation Log (Novack, Dowler, Bush, Glen, & Schneider, 2000) in order to determine whether they continued to be in operationally defined posttraumatic amnesia (PTA). There are several methods for determining PTA status, but the Orientation Log (O-Log) assesses orientation in a standardized format. The O-Log is a quantified assessment of PTA comprised of questions focused on knowledge of the date, time, city, place, hospital, month, date, year, day of week, time, injury etiology and current deficits (Novack et al., 2000). The O-Log is scored based on whether responses to questions are spontaneous (3 points), require logical cuing (2 points), require multiple choicephonemic cuing (1 point) or are incorrect despite cuing (0 points). The maximum score is 30. The O-Log is recommended by the Center for Outcome Measurement in Brain Injury (COMBI/tbims.org) and an O-Log score >25 has been empirically shown to correlate 0.91 with scores on other PTA measures such as the Galveston Orientation and Amnesia Test (GOAT), although measures like the O-Log and the GOAT(Levin, O Donnell, & Grossman, 1979) may underestimate the duration of PTA due to progressive resolution of cerebral dysfunction over at least 3 months following moderate-severe TBI. In the current study, persons with O-Log scores <20 were not included for participation because they were unable to reliably engage in formal neuropsychological or performance validity assessment. Participants were stratified into three groups based on O-Log scores, including persons currently in PTA (O-Log = 20 24), persons not in PTA (O-Log = 25 29) and persons who

4 342 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); were fully oriented (O-Log = 30) according to O-Log criteria (Novack et al., 2000). This stratification is similar to the PTA and loss of consciousness duration comparison conducted by Rohling, Meyers and Millis (2003) and our stratification is supported by differences in overall neuropsychological test battery means (OTBM) described in Table 2 for each group. In addition to the O-Log, the Visual Reproduction subtests (VR I and II) from the Weschler Memory Scale-IV were also administered (Wechsler, 2009). Participants were also administered the Letter Numbering Sequencing (LNS) from the Wechsler Adult Intelligence Scale-IV (Wechsler, 2008). Other measures administered included the California Verbal Learning Test II (Woods, Delis, Scott, Kramer, & Holdnack, 2006), the Symbol Digit Modalities-Test Oral Version (Smith, 1982), the Booklet Category Test (DeFilippis & McCampbell, 1997), the Delis-Kaplan Executive Functions Trail Making Test (Delis, Kaplan, & Kramer, 2001), the Multilingual Aphasia Exam (MAE) Token Test (Benton, Hamsher, Rey, & Sivan, 1989) and MSVT (Green, 2004). Data Analyses Neuropsychological test scores were demographically corrected based on normative information contained in individual test manuals. All data entry and analyses were conducted using the Statistical Package for the Social Sciences (SPSS) version Descriptive statistics were computed to characterize TBI etiology, Glasgow Coma Scale (GCS) scores, neuroimaging findings and O-Log scores. Means and standard deviations were computed for appropriate demographic variables (age-education-days post-injury) and standardized neuropsychological test scores. MSVT test scores were computed and classified based on level of orientation defined by O-Log scores ranging from 20 to 30. Neuropsychological test scores were also computed based on O-Log scores. Neuropsychological test performance based on O-Log scores was compared using ANOVA. An Overall Test Battery Mean (Miller & Rohling, 2001) was computed for the entire sample as well specific cohorts based on O-Log scores. All neuropsychological test data were analyzed using standardized, normative scores. ANOVA was used to identify main effects and Tukey s HSD was used for post hoc analyses to identify specific group differences in performance (p <.05). Due to evidence of heterogeneity of variance (p <.10), the Welch test was used to evaluate main effects and Dunnett s C was used for post hoc analyses in order to identify specific group differences (p <.05). Finally, Pearson correlations were computed for MSVT scores and neurocognitive measures. Results Demographic and Injury Characteristics The current sample was predominately male (70%). Mean participant age was 30.0 years (SD = 13.8). Mean education was 12.4 years (SD = 2.2). Injury mechanism included motor vehicle accident (82%), falls (8%) violence (1%) and other (9%). Neuroimaging (CT scan) documented 100% of the current sample had one or more intracranial lesions. Participants had a range of intracranial lesions, including diffuse axonal injury (33%), subdural hematoma (36%), subarachnoid hemorrhage (35%) and focal cortical contusions (48%). When initial Glasgow Coma Scores (GCS) were available (64%), 71% of participants were found to have initial GCS scores that fell in the severe range (3 8). Participants age, gender, education, traumatic etiology, Glasgow Coma Scale (GCS) scores, imaging findings, days between injury and assessment as well as O-Log scores are documented in Table 1. Neuropsychological Test Performance Table 2 lists standardized neuropsychological test scores and standard deviations for the entire sample as well as cohorts based on O-Log Scores. Performance on the Token Test was normatively average for the entire sample (T = 48.9) and did not significantly differ based on O-Log scores or severity of impairment, which indicates participants were able to adequately comprehend test instructions and follow commands (see Table 2). The Overall Test Battery Mean (OTBM) for the entire sample was mildly-moderately impaired and fell approximately two standard deviations below normative expectations (T = 31.7) based on classification criteria. Participants with the lowest O-Log scores (20 24) had moderately-severely impaired test performance (OTBM T = 24.4) with their most prominent impairment occurring on the Symbol Digit Modalities Test (T = 17) and the California Verbal Learning Test-II Long Delay Recall (T = 20), but all test scores of persons with O-Log scores of were significantly impaired relative to normative expectations. The OTBM of participants with O-Log scores of was mildly-moderately impaired (T = 32.4) and the most prominent impairment was also observed on the Symbol Digit

5 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); Table 1. Patient characteristics (N = 77) Characteristics Values Age (years) 30.0 ± 13.8 Gender (men) 70% Education Years completed 12.4 ± 2.2 % high school degree or less 34% Traumatic etiology Vehicular 82% Falls 8% Violence 1% Other 9% Glasgow Coma Scale Score, acute admission % % % Unknown 36% Positive imaging, acute admission (n = 69) a DAI 33% SDH 36% SAH 35% Focal cortical contusion 48% Days post-injury, neuropsychological assessment 43.9 ± 17.5 Orientation Log (raw score), neuropsychological assessment % % 30 22% Note: Values are mean ± standard deviation or percentage. a 100% of Sample had positive imaging findings. Table 2. Mean neuropsychological test performance and confidence intervals (95%) by orientation level Neuropsychological tests O-Log scores p-value Post hoc All (N = 77) (n = 9) (n = 51) 30 (n = 17) SDMT-Oral 25.3 ( ) 17.0 ( ) 25.8 ( ) 28.2 ( ) < 2, 3 LNS 41.3 (39.0, 43.7) 32.7 ( ) 42.6 ( ) 42.2 ( ) < 2 WMS-IV VR I 36.3 ( ) 27.7 ( ) 38.1 ( ) 35.3 ( ) < 2 WMS-IV VR II 36.2 ( ) 28.1 ( ) 37.9 ( ) 35.3 ( ) < 2 CVLT 1 5 a 31.2 ( ) 20.7 ( ) 32.2 ( ) 33.5 ( ) < 2, 3 CVLT short delay a 29.7 ( ) 22.2 ( ) 30.0 ( ) 32.9 ( ) < 2, 3 CVLT long delay a 27.1 (24.0, 30.2) 20.0 ( ) 27.1 ( ) 30.9 ( ).049 NS D-KEFS numbers a 28.8 ( ) 22.4 ( ) 30.0 ( ) 28.3 ( ).060 NS D-KEFS letters a 28.1 ( ) 21.3 ( ) 28.1 ( ) 31.9 ( ) < 2, 3 Booklet category 32.4 ( ) 30.0 ( ) 31.8 ( ) 35.4 ( ).453 NS Overall Test Battery Mean 31.6 ( ) 24.2 ( ) 32.4 ( ) 33.3 ( ) < 2, 3 Note: P values and Post hoc analyses denote statistically significant differences among group 1 (O-Log = 20 24), group 2 (O-Log = 25 29) and group 3 (O-Log = 30). O-Log, Orientation Log. At least 75 inpatients completed each neuropsychological test; 72 inpatients completed all 10 tests. All neuropsychological test data are standardized scores with mean and (95% confidence intervals). ANOVA was used to identify main effects and Tukey s HSD was used post hoc (p <.05) to identify specific group differences except as indicated. a Due to evidence of heterogeneity of variances (p <.10), the Welch test was used to evaluate main effects and Dunnett s C was used post hoc (p <.05) to identify specific group differences. Modalities Test (T = 25.8) and the California Verbal Learning Test-II Long Delay Recall (T = 27.1). The only score that fell in the unimpaired range was the WAIS-IV Letter Number Sequencing (LNS) score (T = 42.6). Finally, the OTBM of participants with an O-Log score of 30 was also mildly-moderately impaired (T = 33.4) with Symbol Digit Modalities Test (SDMT) scores being most impaired score (T = 28.2). Again, LNS was the least impaired score (T = 42.2).

6 344 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); Based on analyses, significant differences in test performance were observed as O-Log scores increased (Table 2). Participants with O-Log scores of performed significantly less proficiently than participants with O-Log scores ranging from 25 to 30 on the SDMT, LNS, WMS-IV Visual Reproduction-I and II as well as the California Verbal Learning Test-II 1 5 (p <.05). The test performance of persons with O-Log scores of and those with an O-Log score of 30 did not significantly differ on any metric (see Table 2). MSVT Performance and O-Log Scores Participant s MSVT scores are reported in Tables 3, 4 and 5. Table 3 describes performance on IR, DR and CNS based on O-Log scores. No participant scored below 65% correct on IR, DR or CNS and Table 3 list cumulative percentages of scores on IR, DR and CNS ranging from 65% to 100% correct. Table 3 reveals that mean performance of the entire sample (N = 77) was above 90% on IR, DR and CNS. Confidence intervals (95%) were 95.5% 97.7% on IR, 93.0% 96.6% on DR and 91.1% 94.8% on CNS. When examining performance patterns IR, DR and CNS, no participants with O-Log = 30 scored below 90% on IR. When participants O-Log scores were 25 29, 94% of participants scored above the 90% threshold and the lowest score was 85%. When O-Log scores were 20 24, 78% of participants scores above 90% correct on IR. The lowest score on IR for persons with an O-Log = was 75%. When examining DR scores, 100% of participants with an O-Log = 30 scored above the 90% threshold. When O-Log scores ranged from 25 to 29, 90% of participants scored above 90% correct. The lowest score obtained on DR by persons with O-Log = was 70%, which was observed in 3% of participants (N = 2). Participants with O-Log scores = performed less proficiently and only 66% scored above the 90% threshold. The lowest score was 70% correct (N = 1). On the CNS measure, 88% of persons with O-Log scores of 30 scored above 90% correct. The lowest score was 80% (N = 1). When O-Log scores = 25 29, 74% of participants scored above 90% correct. The lowest score was 70% (N = 2/3%). Only 56% of persons with O-Log scores of scored at or above 90% on CNS. The lowest score was 75% (N = 1/1%). Table 4 provides cumulative levels of performance on PA and FR. The mean performance for the entire sample 86.4% on PA and 46.8% on FR. Participants with O-Log = 30 had a PA score of 88.8% and persons PA scores with O-Log = was 89%. In contrast, persons with O-Log = had mean correct PA score of 66.7%. A similar pattern of results was observed on FR. Participants with O-Log = 30 and O-Log = performed equivalently (52.7% and 51.2), whereas participants with O-Log scores performed significantly less proficiently (11.1%). Table 5 reports cumulative percentages of persons with GMIP scores ranging from 0 to 89. No participants with O-Log scores = had GMIP scores less than 30% and 56% of this group had GMIP scores that were greater than 40. In contrast, 64% of persons with O-Log scores = had GMIP scores less than 30% and 58% of participants with O-Log scores = 30 had GMIP scores less than 30. Based on MSVT guidelines all persons with O-Log scores = would meet criteria for GMIP. In addition 36% of persons with O-Log scores = had GMIP scores 30. Moreover, 42% of persons with O-Log = 30 had GMIP scores 30, Table 3. MSVT immediate and delayed recognition, and consistency cumulative percentages of patients with scores at or below the indicated raw score by O-Log score MSVT immediate recognition MSVT delayed recognition MSVT consistency (CNS) O-Log Score All All All N = n = 9 n = 51 n = 17 n = 77 n = 9 n = 51 n = 17 MSVT Raw Mean = % CI = (95.5 (87.0 (95.8 (95.0 (93.0 (76.9 (93.6 (95.9 (91.1 (80.0 (91.0 ( ) 98.6) 98.3) 99.1) 96.6) 93.2) 97.4) 99.9) 94.8) 93.3) 95.6) 98.2) Note: O-Log, Orientation Log; Percentages in within column groupings may not add up to 100% due to rounding.

7 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); Table 4. MSVT test paired-associates and free recall cumulative percentages of patients with scores at or below the indicated raw score by O-Log score MSVT paired associates MSVT free recall O-Log score = All All N = MSVT raw = Mean % CI ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Note: Percentages in within column groups may not add up to 100% due to rounding. Higher GMIP scores indicate more severe impairment. Table 5. MSVT genuine memory impairment profile cumulative percentages of particpants with scores at or below the indicated raw score by O-Log score GMIP O-Log score = All N = GMIP = Mean %CI ( ) ( ) ( ) ( ) Note: Percentages in within column groupings may not add up to 100% due to rounding. Higher GMIP scores indicate more severe impairment. Associations of MSVT Scores with Neuropsychological Test Scores Table 4 provides a matrix with Pearson correlations between neuropsychological test scores and MSVT measures. Table 4 highlights the highest two correlations among neuropsychological test scores and MSVT measures at p <.01. The largest correlations were observed between memory measures (WMS-IV VR I and II, CVLT-II 1 5, CVLT-II Short Delay and CVLT-II Long Delay) and MSVT FR. Correlations ranged from 0.49 to There were no significant correlations between memory measures and MSVT IR, DR or CNS measures. MSVT PA was modestly correlated with WMS-IV VR I (0.33) and II (0.34) (Table 6).

8 346 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); Table 6. Correlation between MSVT and neuropsychological test scores Neuropsychological tests MSVT subscales IR DR CNS PA FR SIP Token SDMT-Oral LNS WMS-IV VR I WMS-IV VR II CVLT CVLT short delay CVLT long delay D-KEFS numbers D-KEFS letters Booklet category All correlations are Pearson r scores; italics indicates a significant correlation at p <.01; bold italics indicate top two significant correlated tests for each MSVT subtest. Discussion Participants in our study were admitted to a dedicated, acute TBI program following moderate-severe TBI. Participants in our study were prospectively enrolled and administered neuropsychological measures, the O-Log and the MSVT a mean of 44 days post-injury. Only eligible participants who were in a coma, were minimally conscious or were untestable secondary to extreme confusion or severe medical problems were excluded from this study. No participants were excluded based on O-Log scores even though a small cohort of participants did not meet O-Log criteria for resolution of PTA (N = 9). All our participants had one or more intracranial lesions, at least mild-moderate impairment on neuropsychological tests (Heaton, Miller, & Taylor, 2004) and were unable to function independently in a hospital environment at the time of their examination. In other words, our participants MSVT performance provides a basal level of expectations following moderate-severe TBI. Performance on the MSVT during the acute recovery period following moderate-severe TBI was consistent with studies showing that persons with significant neurological disorders generally perform above diagnostic thresholds established by Green (2004) with a few exceptions. When the entire sample s MSVT performance was examined, the mean scores on IR (96.6), DR (94.8), CNS (92.9), PA (86.4) and FR (46.8) fell above performance expectations described in MSVT interpretive guidelines and declined linearly as a function of brain injury severity, as established by PTA severity, which is expected according to MSVT Criteria B 2 (Green, 2004). When examining specific subgroups of participants with moderate-severe TBI based on O-Log scores, no participant with an O-Log score of 30 (N = 17) performed below 90% correct on the MSVT IR and DR measures. In other words, when participants were oriented in time and place and were able to correctly answer questions regarding their injury status and current deficits, no participant scored below the empirically derived thresholds on IR or DR, despite the presence of neuropsychological test performance that was at the 4th percentile (T = 33) relative to normative expectations and a need for constant supervision in activities of daily living (ADL). In contrast, 12% of persons with O-Log scores of 30 scored below the 90% threshold on the CNS metric. Although IR and DR are unlikely to be negatively affected by acute, moderate-severe TBI, a small percentage of persons with acute moderatesevere TBI with O-log scores of 30 evidenced CNS scores below 90%. When O-Log scores were a 30, the lowest score observed on the CNS metric was 80%, which was observed in 6% of the subsample. This finding suggests that the CNS metric is not significantly affected in persons with moderate-severe TBI, despite prominent neurocognitive impairment and functional deficits, not to mention 48% of this subsample has GMIP >30. Given the high levels of performance on IR and DR, having 12% of CNS scores fall between 80% and 85% is likely due to the number of permutations in consistency when IR and DR scores are so high. In other words, all persons with O-Log scores of 30 recall test stimuli at a high level, but some persons are inconsistent from trial to trial, which is not unexpected following moderate-severe TBI. In any case, the base rate of CNS scores <90% is low and our data establish an absolute basal level of performance on CNS (80+) in a very small percentage of persons, which should help clinicians who observe deviations in MSVT performance following less acute and severe TBI. As O-Log scores and verbal memory scores decreased, performance on some MSVT measures also decreased. For instance, 6% of persons who had O-Log Scores between 25 and 29 scored below the 90% threshold on IR, but the lowest score on IR was 85%, which suggests that failure on IR reflects only a mild performance decline in a small number of

9 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); persons. A slightly greater percentage of persons with similar O-Log scores scored below the established MSVT threshold on DR (10%) even though as a group these participants had moderately impaired delayed verbal memory scores on the CVLT-II (T = 27.1). The lowest DR score observed in 10% of participants with O-Log scores of was 70%, which suggests that a small number of persons with acute moderate-severe TBI not only score below the 90% threshold, but also score somewhat lower on DR compared to IR, which make sense given the nature of the DR metric and the finding that 36% of this subsample had GMIP scores >30. CNS scores of persons with O-Log Scores of were more negatively affected. Despite high levels of performance on IR and DR, 26% of participants with O-Log scores had CNS scores below 90%. Again, the lowest CNS score in this cohort was also found to be 70%. Participants with O-Log scores of performed well on IR and DR despite moderately impaired verbal memory scores and functional impairment, but as was observed in persons with O-Log Scores of 30, CNS performance was less proficient although not dramatically less proficient then persons with O-Log scores of 30 because the lowest score in this group was 70% compared to a lowest score of 80% in the group with an O-Log score of 80%. Finally, 36% of this cohort had GMIP scores >30. In a small cohort of persons with O-Log scores of and an OTBM = 24.4, 11% of participants scored below the established 90% MSVT threshold on IR. In contrast, 55% of these persons fell below the 90% threshold on DR, despite the presence of severely impaired short (T = 22) and long delayed verbal memory scores (T = 20) on the CVLT-II. The lowest score on IR, DR and CNS was 75% and 70% respectively. CNS performance was similarly impaired with 44% scoring below the 90% threshold, but the lowest score on CNS was 75%. Most importantly, 100% of this small cohort had MSVT GMIP profiles that were 30. In other words, the mean of IR, DR and CNS compared to the mean of PA and FR was 30, which has been observed in other populations with severe cognitive impairment such as dementia (Howe, Anderson, Kaufman, Sachs, & Loring, 2007). Although the assessment of neuropsychological performance validity is more complex than only administering a standalone or embedded performance validity measure, performance validity measures are an important component of assessing overall neuropsychological test validity (Bush et al., 2005; Heilbronner et al., 2009; Slick, Sherman, & Iverson, 1999). Optimally, clinicians would prefer to use performance validity tests that have sensitivity and specificity close to 100%. These optimal psychometric characteristics are rarely realized and most performance validity tests are designed to sacrifice sensitivity for specificity in order to minimize the number of false positives, while optimizing positive predictive power (Larrabee, 2012). Our data shows that performance on the MSVT is only affected when persons are in PTA as defined by the O-Log and have severe co-occurring cognitive impairment that is at least two standard deviations below normative expectations. Based on our data, when performing examinations in outpatient settings where the severity of TBI or other neurological disorders is significantly less severe, MSVT performance would not be expected to deviate from Criteria A, B 1 or B 2. The most applicable cohort for comparison in outpatient settings would be our group with O-Log scores of 30. No one with a valid O-Log score of 30 would be expected to have IR and DR scores lower than 90%. The probability of a CNS score lower than 90% is small (0.12), but the CNS has a zero probability of falling below 80%. These scores reflect performance levels and a pattern of MSVT scores that should establish expectations in populations of outpatients who are not in acute recovery from a moderatesevere TBI. Our data provides basal performance expectations for persons who would be highly unlikely to be referred for neuropsychological assessment in outpatient settings due to the severity and acuity of their neuropsychological impairment. When assessing persons with less acute moderate-severe TBI and especially persons who have suffered less severe injuries such as documented or suspected mild traumatic brain injury (MTBI), MSVT performance would not be expected to fall anywhere close to the parameters established in the current cohort. Conflict of Interest None declared. References Axelrod, B. N., & Schutte, C. (2011). Concurrent validity of three forced-choice measures of symptom validity. Applied Neuropsychology, 18, doi: / Benton, A., Hamsher, K. D. S., Rey, J., & Sivan, A. (1989). Multilingual aphasia examination (3rd ed.). Iowa City, IA: AJA Associates. Bush, S. S., Ruff, R. M., Troster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., et al (2005). Symptom validity assessment: Practice issues and medical necessity NAN policy & planning committee. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 20, doi: /j.acn

10 348 S.N. Macciocchi et al. / Archives of Clinical Neuropsychology 32 (2017); DeFilippis, N. A., & McCampbell, E. (1997). The booklet category test. Odessa, Florida: Psychological Assessment Resources. Delis, D., Kaplan, E., & Kramer, J. (2001). Examiner s manual of the Delis-Kaplan executive functioning system. San Antonio, TX: The Psychological Corporation. Gidley-Larson, J. C., Flaro, L., Peterson, R. L., Connery, A. K., Baker, D. A., & Kirkwood, M. W. (2015). The Medical Symptom Validity Test measures effort not ability in children: A comparison between mild TBI and fetal alcohol spectrum disorder samples. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 30, doi: /arclin/acv012. Green, P. (2003 [2005]). World Memory Test for Windows; Users Manual and Program. Edmonton, Canada: Green s Publishing. Green, P. (2004). Green s Medical Symptom Validity Test (MSVT) for Microsoft windows user s manual. Edmonton, Canada: Green s Publishing. Green, P., & Flaro, L. (2015). Results from three Performance Validity Tests (PVTs) in adults with intellectual deficits. Applied Neuropsychology Child, 22, doi: / Green, P., & Flaro, L. (2016). Results from three performance validity tests in children with intellectual disability. Applied Neuropsychology Child, 5, doi: / Green, P., Rohling, M. L., Lees-Haley, P. R., & Allen, L. M., III (2001). Effort has a greater effect on test scores than severe brain injury in compensation claimants. Brain Injury, 15, doi: / Heaton, R., Miller, W., & Taylor, M. (2004). Revised comprehensive norms for an expanded Halstead-Reitan battery: Demographically adjusted neuropsychological norms for African American and Caucasian ADults. Odessa, FL: PAR. Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Conference, P. (2009). American Academy of Clinical Neuropsychology Consensus Conference Statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23, doi: / Howe, L. L., Anderson, A. M., Kaufman, D. A., Sachs, B. C., & Loring, D. W. (2007). Characterization of the Medical Symptom Validity Test in evaluation of clinically referred memory disorders clinic patients. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 22, doi: /j.acn Kirkwood, M. W., Yeates, K. O., Randolph, C., & Kirk, J. W. (2012). The implications of symptom validity test failure for ability-based test performance in a pediatric sample. Psychological Assessment, 24, doi: /a Larrabee, G. L. (2012). Forensic neuropsychology: A scientific approach. Oxford, New York: Oxford University Press. Levin, H. S., O Donnell, V. M., & Grossman, R. G. (1979). The Galveston Orientation and Amnesia Test. A pratical scale to assess cognition after head injury. Journal of Nervous and Mental Disease, 167, Miller, L. S., & Rohling, M. L. (2001). A statistical interpretive method for neuropsychological test data. Neuropsychology Review, 11, Retrieved from Novack, T. A., Dowler, R. N., Bush, B. A., Glen, T., & Schneider, J. J. (2000). Validity of the Orientation Log, relative to the Galveston Orientation and Amnesia Test. Journal of Head Trauma Rehabilitation, 15, Retrieved from Rohling, M. L., Meyers, J. E., & Millis, S. R. (2003). Neuropsychological impairment following traumatic brain injury: A dose response analysis. Clinical Neuropsychol, 17, Singhal, A., Green, P., Ashaye, K., Shankar, K., & Gill, D. (2009). High specificity of the medical symptom validity test in patients with very severe memory impairment. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 24, doi: / arclin/acp074. Slick, D. J., Sherman, E. M., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13, doi: / (199911)13:04;1-y;ft545. Smith, A. (1982). Symbol digits modalities test western psychological services. Los Angeles: WPS Publishing. Stevens, A., Friedel, E., Mehren, G., & Merten, T. (2008). Malingering and uncooperativeness in psychiatric and psychological assessment: Prevalence and effects in a German sample of claimants. Psychiatry Research, 157, Wechsler, D. (2008). Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV). San Antonio, TX: Psychological Corporation. Wechsler, D. (2009). Wechsler Memory Scale-Fourth Edition (WMS-IV) technical and interpretive manual. San Antonio, TX: Pearson. Woods, S. P., Delis, D. C., Scott, J. C., Kramer, J. H., & Holdnack, J. A. (2006). The California Verbal Learning Test second edition: Test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 21, doi: /j.acn