Psychometric properties and factor structure of an expanded CERAD neuropsychological battery in an elderly VA sample

Transcription

1 Archives of Clinical Neuropsychology 21 (2006) Abstract Brief report Psychometric properties and factor structure of an expanded CERAD neuropsychological battery in an elderly VA sample Scott N. Jones a,b,, Catherine R. Ayers a,1 a VA Maryland Health Care System, Perry Point Division, Perry Point, MD, USA b Department of Psychiatry, University of Maryland Medical School, Baltimore, MD, USA Accepted 30 March 2006 Recent research by Strauss and Fritsch [Strauss, M.E., & Fritsch, T. (2004). Factor structure of the CERAD neurophysiological battery. Journal of the International Neuropsychological Society, 10, ] suggested additions to the original Consortium to Establish a Registry for Alzheimer s Disease (CERAD) neuropsychological battery. In the present study, the data from a clinical series of military veteran patients referred for neuropsychological evaluation of suspected dementia using an Expanded CERAD battery was explored. Logistic regression analysis and exploratory factor analysis with quantitative methods for determining the number of factors strongly supported the addition of the Indiana University Token Test. A one-factor model best accounted for the demented subjects data and a two-factor model emerged when the total sample (demented and non-demented subjects) data were factor analyzed. The benefit of using computational methods for determining the number of underlying factors was discussed. Published by Elsevier Ltd on behalf of National Academy of Neuropsychology. Keywords: Dementia; Screening evaluation; Factor analysis 1. Introduction Research and clinical practice in the last 20 years to address the increase in prevalence and incidence of dementias has resulted in the development of numerous rating scales and brief neuropsychological batteries to aid in diagnosis. The Consortium to Establish a Registry for Alzheimer s Disease (CERAD) efforts to develop such a brief battery has met with considerable success judged by the number of studies (100+) published using it to evaluate people with Alzheimer s disease (AD), other types of dementia, and cognitive status of psychiatric disorders in late life (e.g. McGurk et al., 2000). The original tests were chosen to tap the principal cognitive manifestations of AD (e.g. memory, language, praxis, and general intellectual status) (Morris et al., 1989) and to avoid ceiling effects early in the disorder and floor effects late in its progression. The original developers of the CERAD battery sought to balance brevity (the core battery takes min to administer) with the scope and depth necessary for clinical diagnosis and to further our knowledge of AD in a research context. Corresponding author at: VA Maryland Health Care System, Perry Point Division (116B), Perry Point, MD 21902, USA. Tel.: x6130; fax: address: scott.jones2@va.gov (S.N. Jones). 1 Present address: VA San Diego Healthcare System, University of California at San Diego, USA /$ see front matter. Published by Elsevier Ltd on behalf of National Academy of Neuropsychology. doi: /j.acn

2 360 S.N. Jones, C.R. Ayers / Archives of Clinical Neuropsychology 21 (2006) Ongoing research with CERAD battery has revealed that subtest scores are differentially affected by age, education, and gender (Welsh et al., 1994) and that the underlying factor structure may vary depending on sample and methods of statistical analysis. Strauss and Fritsch (2004) argued that exploratory factor analysis is preferred to principal components analysis to examine underlying factor structure and that older rules of thumb pertaining to numbers of factors to retain such as Eigenvalues 1 and scree plots tend to overestimate the actual number of factors. O Connor (2000) further summarizes technical difficulties using factor analysis due to the various statistical approaches to the procedure and the subjectivity of decisions inherent in the process. Under-extraction of factors compresses variables into distorted factors, and over-extraction splits factors into unintelligible or insignificant constructs. Both Strauss and Fritsch (2004) and O Connor allow that scree plots are more accurate than Eigenvalues >1 as a retention scheme but remain less accurate than computational methods when objectively compared in Monte Carlo simulations. Strauss and Fritsch (2004) used a method termed parallel analysis first developed by Horn (1965) and revised by Glorfeld (1995). This test compares Eigenvalues from random data sets with the same number of cases and variables as the actual data. An alternative computational approach, Velicer s minimum average partial (MAP) test, compares the relative amounts of systematic and unsystematic variance remaining in a correlation matrix after extraction of increasing numbers of components (O Connor, 2000; p. 396). Pertaining to the present research, Strauss and Fritsch re-analysis of the original CERAD data (i.e. Morris et al., 1989), found that one factor accounted for the reliable variance in the correlation matrix of those variables with a confirmed AD patient population. Furthermore, Andel et al. (2003) employing a nonpatient sample, found differences on the CERAD word list learning task between a university sample and a community sample even with education and age statistically controlled. They speculated that subtle motivational differences may have accounted for these differences but their study design did not permit exploration of this hypothesis. Thus, there is some evidence pointing to the need for population-specific norms to ensure valid interpretation of CERAD data. In this regard, recent research by Jones and Votolato (2005) indicated that a clinical series of elderly veterans administration (VA) patients referred for neuropsychological evaluation differed substantially from the original Morris et al. (1989) sample on education and racial identification but not on age. Jones and Votolato (2005) also found a pattern of heavy alcohol and tobacco use in their sample as well as other risk factors for vascular/subcortical dementias, and they concluded that most of their demented subjects had vascular or mixed etiologies rather than pure dementias of the Alzheimer s type. The large and growing segment of the VA population referred for evaluation of possible dementia will severely tax clinical resources and require accurate and efficient assessment methodologies to meet this demand. The purpose of the present investigation was to explore the psychometric properties, including factor structure, of an expanded CERAD battery with an elderly VA population. The expanded battery in part addresses issues raised by Strauss and Fritsch (2004) both in terms of content (they suggested adding additional tests of confrontation naming and praxis tasks to help discriminate subtypes of AD) and methodology. They noted that multiple indices derived from individual tests have the potential to confound method and construct variance. Therefore, a balance was sought in additional task selection between this more research-oriented concern and the clinical concern for battery length. The expanded battery included a measure of reading ability, an additional measure of verbal fluency, a simple test of mental set acquisition and set-shifting, a test of aural verbal comprehension, a brief screening test of depression in late life, tests of immediate span and working memory, and complex visual perception. 2. Methods The current investigation is a retrospective analysis of a sample of patients referred to the Neuropsychology Clinic of a mid-atlantic Veterans Administration Medical Center. Primary care physicians, psychiatrists, or other direct care specialists referred patients for a full neurocognitive evaluation based on a suspicion of dementia. One hundred thirty-five cases were selected from a clinical series of referrals over a 4-year period ( ). At the time of referral, the complete medical records were reviewed and demographic variables including age, education, gender, ethnicity, and handedness were recorded. Clinical variables including neuroimaging findings, substance use history, medical diagnoses, psychiatric diagnoses, and medications were also documented. As part of the evaluation each individual was given a neurocognitive evaluation including an expanded CERAD battery, which, in addition to the original CERAD battery, included the following: Wide Range Achievement Test-3 (WRAT-

3 S.N. Jones, C.R. Ayers / Archives of Clinical Neuropsychology 21 (2006) ; Wilkinson, 1993, Reading subtest, raw score range 0 57) as an estimate of premorbid intelligence, Controlled Oral Word Association test (CFL; Benton, Hamsher, & Sivan, 1983; raw score range 0 high) as another test of verbal fluency less influenced by retained semantic structure/ability than animal naming, Wechsler Adult Intelligence Scale-III (WAIS III; Wechsler, 1997) digit span as a test of immediate memory span (digits forward, raw score range 0 16) and working memory (digits backward, raw score range 0 14), the Indiana University Token Test (IUTT; Unverzagt, Farlow, & Hendrie, 1999, raw score range 0 24) as a test of language comprehension/working memory/visual motor spatial integration, Poppelreuter-type superimposed figures test (POP; Luria, 1980, raw score range 0 4) as a test of complex visual identification, reciprocal motor movements (RECMM, raw score range 0 10) and a go/no-go (G/N-G, raw score range 0 10) task as a measure of cognitive set acquisition and set-shifting, and the Geriatric Depression Scale-Revised (GDS-R; Burke, Roccaforte, & Wengel, 1991, raw score range 0 15). These additional measures added approximately min to the administration time required for the original CERAD battery. Further additions or alterations to the original battery were also made. Three drawings to copy of line figures were added as a test of constructional praxis (PRAXIS, score range 0 3). These figures were more complex than the original CERAD stimuli and one involved a two-dimensional representation of a three-dimensional object (house). An additional drawing of a Greek cross under special instructions ( complete the drawing without lifting pencil from the paper, score range 0 1) was included as a test of planning. The MMSE was included and divided into five subscales suggested by Collie et al. (1999); (1) Orientation (MORIENT), comprised of items to identify the year, season, month, date, day, place, county, state, floor of the building, and address (score range 0 10), (2) Registration (MREGIST), required subjects to repeat the names of three objects (score range 0 3), (3) Attention (MATTN), comprised of spelling world backward (score range 0 5), (4) Recall (MRECALL), consisted of recalling three objects (score range 0 3), and (5) Language/Praxis (MLANG), comprised of naming two objects, repeating a phrase, following a three-step aural verbal command, writing a sentence, following a one-step written command, and copying a design (score range 0 9). A memory savings (SAVINGS) score, computed by dividing the number of words retrieved on delayed recall from the CERAD word list by the number of words recalled on the third learning trial (score range, ), was also included in the data analyzes. Based on the clinical interview and medical history data, diagnoses of dementia were made using Diagnostic and Statistical Manual-IV (1994) criterion. When a diagnosis of dementia was made, the disorder was staged as mild (and assigned a value of 1), moderate (assigned a value of 2) or severe (assigned a value of 3 using criterion developed by Hughs, Berg, Danziger, Coben and Martin (1982) clinical dementia rating [CDR]). 3. Subject characteristics The sample s mean age in years was (S.D. = 8.91), with a range of years. All but two of the subjects were male (133 male, two female). Seventy-three percent were Caucasian and 27% African-American. The average educational attainment in years was (S.D. = 3.54). All patients were referred for evaluation because they were either suspected of dementia or already carried a dementia diagnosis. Of the 135 patients, 76 (56%) were diagnosed with dementia. 4. Statistical analyses Statistical analyses were conducted using SAS (version 6.12) and included the following: 1) t-tests were utilized to examine the differences between demented and non-demented individuals on general demographic and battery variables. 2) Step-wise logistic regression was used to predict a clinical diagnosis of dementia using all battery variables except RECMM as there was virtually no variance in scores on this variable. 3) Factor structure of the expanded CERAD battery was explored in two ways; first, through Velicer s minimum average partial (MAP) test which we felt was computationally simpler and typically provides the same result as parallel analysis and using SAS syntax described in O Connor (2000), and, second, through exploratory factor analysis examining the number of factors specified by the MAP.

4 362 S.N. Jones, C.R. Ayers / Archives of Clinical Neuropsychology 21 (2006) Table 1 Demographic and neuropsychological characteristics of sample Variable Mean (S.D.) T-value p Demented (n = 76) Non-demented (n = 59) Age 71.7 (8.5) 70.4 (9.5) Education (years) 10.7 (4.0) 11.0 (2.8) Tobacco use (14274) (11706) Alcohol use (130865) (99215) GDS-R (score range 0 15, scores of 6 suggest possible depression) 4.7 (3.6) 6.3 (4.6) MMSE (normal range 24 30) 20.9 (5.2) 26.3 (3.4) Tobacco use = estimate lifetime total packs of cigarettes (or equivalent), reference-one pack per day for 40 years = 14,600; alcohol use = estimated lifetime total alcoholic drinks (one drink = one 12 oz beer, one 4 oz glass of wine, or 1 oz of liquor), reference-2 drinks per day for 40 years = 29,200; GDS-R = Geriatric Depression Scale-Revised. 5. Results Demented and non-demented subjects did not differ in age, use of tobacco or alcohol, years of education or GDS-R score (see Table 1). Alcohol and tobacco use were heavy in both subsamples. GDS-R scores were near the suggested cut-off for suspicion of depression in the demented group and were above the cut-off for the non-demented subjects. Point biserial correlations of GDS-R scores and clinical diagnosis of any depression disorder (i.e. major depressive disorder, adjustment disorder with depressed mood, etc.) was r = (p = ). Nearly 40% of the total sample was found to have periventricular white matter changes and 34% had lacunes on noncontrast computed tomography scan of the brain, 28% had a diagnosis of coronary artery disease, 30% had a diagnosis of diabetes mellitus type II, and 54% had a diagnosis of hypertension. However, χ 2 analysis revealed demented subjects were no more likely than non-demented subjects to have these conditions. Initial t-tests comparing demented and non-demented subjects revealed each subtest differentiated the two groups (p 0.01) except for Digit Span-forward, as expected (i.e. immediate memory span is typically preserved early in a dementing process). Results of step-wise logistic regression to predict a clinical diagnosis of dementia is shown in Table 2. The CERAD word list delayed recall, the IUTT, and orientation questions from the MMSE were all significant (p < 0.001). The SAVINGS scores also contributed unique, significant variance to predicting the presence and severity of dementia. Twenty-one variables were included in the EFA (see Table 3). Per recommendations from Delis et al. (2003) Larrabee (2003), the EFA was conducted in two parts, first with demented subjects only and second with the entire sample. Velicer s MAP test revealed that for the demented subjects a single factor should be retained. The EFA produced an Eigenvalue associated with this factor of 6.59, accounting for 31% of the variance. Table 3A contains the factor loadings, with only one memory variable (word list learning trial 3) loading above The factor appeared to be a general severity of dementia construct. Table 2 Step-wise logistic regression to predict dementia Variable d.f. Parameter estimate Standard error Wald χ 2 P r > χ 2 Intercept Intercept Intercept WLDELR IUTT MORIENT SAVINGS Note: WLDELR = CERAD word list delayed recall; IUTT = Indiana University Token Test; MORIENT = MMSE orientation items; SAV- INGS = CERAD word list learning savings score.

5 Table 3 Rotated factor loadings for expanded CERAD battery S.N. Jones, C.R. Ayers / Archives of Clinical Neuropsychology 21 (2006) Expanded CERAD items Factor 1 (A) Dementia patients only MRECALL 0.25 WRATREAD 0.65 ANIMNAM 0.56 CFL 0.66 BNTA 0.62 DIGSPF 0.52 DIGSPB 0.72 WLRT CROSLFT 0.42 WLDELR 0.02 WLDYES 0.20 WLDNO 0.10 IUTT 0.81 POP 0.56 GNG 0.61 MORIENT 0.52 MREGIST 0.47 MATTN 0.74 MLANG 0.85 SAVINGS 0.17 PRAXIS 0.72 Expanded CERAD items Factor 1 Factor 2 (B) Dementia and non-dementia patients MRECALL WRATREAD ANIMNAM CFL BNTA DIGSPF DIGSPB WLRT CROSLFT WLDELR WLDYES WLDNO IUTT POP GNG MORIENT MREGIST MATTN MLANG SAVINGS PRAXIS Note: WRATREAD = Wide Range Achievement Test-3, reading subtest; ANIMNAM = animal naming; CFL = Controlled Oral Word Association C, F, and L); BNTA = Boston Naming Test-Abbreviated version; DIGSPF = WAIS-III Digit Span forward; DIGSPB = WAIS-III Digit Span backwards; WLRT3 = CERAD word list learning trial 3; CROSLFT = Greek cross drawing; WLDELR = CERAD word list delayed recall; WLDYES = CERAD word list recognition trials score yes ; WLDNO = CERAD word list recognition trials scored no ; IUTT = Indiana University Token Test; POP = Poppelreuter superimposed figures test; GNG = go/no-go test; MORIENT = MMSE orientation items; MREGIST = MMSE registration items; MRECALL = MMSE recall items; MATTN = MMSE attention items; MLANG = MMSE language/praxis items; SAVINGS=CERAD word list learning savings score; PRAXIS = drawing to copy of three figures.

6 364 S.N. Jones, C.R. Ayers / Archives of Clinical Neuropsychology 21 (2006) Of note, with this dementia patients only data, there were seven factors with and Eigenvalue >1 accounting for 75% of the variance in the correlation matrix. The values for the other six factors were 2.46, 1.72, 1.52, 1.22, 1,17, and Thus, scree plot analysis would have suggested retention of two or perhaps three factors. Examination of the variable loadings on the first three rotated (Varimax) factors revealed a first factor marking general dementia severity, a second factor with high loadings on the word production measures, and a third factor with high loadings on the MMSE orientation and recall items, the third learning trial from the CERAD as well as the recognition memory trial from this test. Given these factor loadings, the one-factor solution also identified by the MAP procedure was considered the most coherent. When the data from all subjects was included, the MAP test indicated two factors should be retained for rotation. Although there were five initial factors with Eigenvalues >1, a scree plot examination of these values (i.e. 7.54, 2.37, 1.44, and 1.17) was largely consistent with MAP results suggesting a two-factor solution. Table 3B contains the rotated (Varimax) factor loadings. Eigenvalue for Factor 1 was 6.18 (accounting for 30% of the variance) and for Factor 2 was 3.72 (accounting for 17% of the variance). Variable loadings on the first factor suggested it was again a severity of dementia factor. All of the variable loadings on Factor 2 were measures of memory function. 6. Discussion The tests added to the original CERAD battery in the present study appear to have added practical utility with minimal additional effort and time invested in the assessment situation. The findings offer strong support for the addition of the IUTT in an expanded CERAD battery. In addition to delayed recall from the CERAD word list and orientation items from the MMSE, the IUTT was the only other variable in the logistic regression analysis with a significant, unique contribution to predicting a clinical diagnosis of dementia. Its loadings in factor analysis with both demented and total samples were high, as well. The IUTT results are often useful in clinically predicting how patients will manage real world contexts where following the direction of others (caregivers) can mean home placement versus institutionalization. Of the other added measures, the test of attention and working memory (Digit Span) and word production (CFL) had relatively high factor loadings. The test of complex perception (Poppelreuter figures) had moderate loadings on the severity of dementia factor. The revised praxis measures used here also had high factor loadings on this factor. It did not emerge as a separate factor from the language/praxis on the MMSE, as it did in the Strauss and Fritsch (2004) results. Of note, among the memory tests, only the third trial of the CERAD word list had an even moderate loading on the single factor that emerged from the demented only subjects analysis. With the total sample, the second factor was marked only by variables that tap different aspects of verbal memory including learning, retrieval, and recognition. These results may have varied from Strauss and Fritsch (2004) as they had dropped all delayed recall measures from their analysis due to skewed distributions of these variables, instead relying on standardized learning and recognition scores form the CERAD word list only. The strong loading of a second verbal learning and recall task from a separate test (i.e. recall on the MMSE) speaks to the concern raised by Strauss and Fritsch (p. 560) about method variance when all measures are derived from a single test. Although SAVINGS scores were nonetheless a derivative measure from CERAD learning and recall performance, this score can be clinically useful to differentiate patients with deficient new learning but retained retrieval capacity from those with the opposite pattern of deficits or with both deficiencies. In addition to the obvious use of the WRAT-3 Reading subtest as a measure of word recognition and reading ability, it also proved valuable to help estimate the patient s global abilities and general intelligence, thereby helping to predict the patient s performances on the other measures. Somewhat unexpectedly, tests of executive functioning including the set-shifting measure (go/no-go) and planning (cross) did not achieve strong factor loadings or add unique variance in predicting dementia. The usefulness of the MAP test to determine the number of factors to retain was demonstrated here. Traditional noncomputational methods (Eigenvalues, scree plots) would have overestimated the number of factors in the data. As noted, the factor analysis of the battery data produced by the demented patients yielded seven factors with Eigenvalues >1. An examination of these rotated factors revealed some with easily interpretable variable loadings (e.g. Factor 2 had highest loadings on tests of executive function animal naming, CFL, and go/no-go) and others that had no clear interpretation (e.g. Factor 6 was marked only by the Greek cross variable; Factor 7 was marked by loadings on BNT-A and the registration items from the MMSE). Developing strained factor names for these apparently artifactual factors adds little to our understanding of the diagnostic process with these patients.

7 S.N. Jones, C.R. Ayers / Archives of Clinical Neuropsychology 21 (2006) The limits of the present research are also some of its strengths results from this nearly all male, veteran subject sample, with many vascular risk factors, may not be generalizable to a civilian population with a predominantly primary degenerative etiology of dementia. However, these results from a naturalistic (i.e. clinical series) sample may be more informative for the many clinical settings serving veterans and other groups where mixed dementias are common. Acknowledgements The authors acknowledge Dwight Dickinson, PhD for assisting with gaining UMMS IRB approval for this research and for helpful comments on an earlier draft of this paper. Karen M. Jones, M.S. provided statistical consultation. References Andel, R., McCleary, C. A., Murdock, G. A., Fiske, A., Wilcox, R. R., & Gatz, M. (2003). Performance on the CERAD word list memory task: a comparison of university-based and community-based groups. International Journal of Geriatric Psychiatry, 18, Benton, A. L., Hamsher, K., & Sivan, A. B. (1983). Multilingual aphasia exam (3rd ed.). Iowa City Iowa: AJA Associates. Burke, W. J., Roccaforte, W. H., & Wengel, S. P. (1991). The short form of the geriatric depression scale: a comparison with the 30-item form. Journal of Geriatric Psychiatry and Neurology, 4, Collie, A., Shafiq-Antonacci, R., Maruff, P., Tyler, P., & Currie, J. (1999). Norms and the effects of the demographic variables on a neuropsychological battery for use in a healthy ageing Australian population. Australian and New Zealand Journal of Psychiatry, 33, Delis, D. C., Jacobson, M., Bondi, M. W., Hamilton, J. M., & Salmon, D. P. (2003). The myth of testing construct validity using factor analysis or correlations with normal or mixed clinical populations: lessons from memory assessment. Journal of the International Neuropsychological Society, 9, American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association. Glorfeld, L. W. (1995). An improvement on Horn s parallel analysis methodology for selecting the correct number of factors to retain. Educational and Psychological Measurement, 55, Horn, J. L. (1965). A rationale and test for the number of factors in a factor analysis. Psychometrika, 30, Hughs, C. P., Berg, L., Danziger, W., Coben, L. A., & Martin, R. L. (1982). A new clinical scale for the staging of dementia. British Journal of Psychiatry, 140, Jones, S. N., & Votolato, R. C. (2005). Neuropsychological correlates of mini-mental state exam scores in an elderly VA sample. Washington, DC: American Psychological Association. Larrabee, G. L. (2003). Lessons on measuring construct validity: a commentary on Delis, Jacobson, Bondi, Hamilton, and Salmon. Journal of the International Neuropsychological Society, 9, Luria, A. R. (1980). Higher cortical functions in man (2nd ed.). New York: Consultants Bureau. McGurk, S. R., Moriatry, P. J., Harvey, P. D., Parrella, M., White, L., & Davis, K. L. (2000). The longitudinal relationship of clinical symptoms, cognitive, functioning, and adaptive life in geriatric schizophrenia. Schizophrenia Research, 42, Morris, J. C., Heyman, A., Mohs, R. C., Hughes, J. P., van Belle, G., Fillenbaum, G., et al. (1989). The Consortium to Establish a Registry for Alzheimer s Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer s disease. Neurology, 39, O Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer s MAP test. Behavior Research Methods, Instruments & Computers, 32, Strauss, M. E., & Fritsch, T. (2004). Factor structure of the CERAD neuropsychological battery. Journal of the International Neuropsychological Society, 10, Unverzagt, F. W., Farlow, M. R., & Hendrie, H. C. (1999). Clinical utility of new visual learning memory and language subtests for use in the CERAD neuropsychological battery. Journal of the International Neuropsychological Society, 5, 129. Wechsler, D. (1997). Wechsler Adult Intelligence Scale-III. San Antonio, TX: The Psychological Corporation. Welsh, K. A., Butters, N., Mohs, R. C., Beekly, D., Edland, S., Fillenbaum, G., et al. (1994). The Consortium to Establish a Registry for Alzheimer s Disease (CERAD) Part V. A normative study of the neuropsychological battery. Neurology, 44, Wilkinson, G. S. (1993). Wide Range Achievement Test-3. Wilmington, DE: Wide Range Inc.