Short Cognitive Performance Test: Diagnostic Accuracy and Education Bias in Older Brazilian Adults

Transcription

1 Archives of Clinical Neuropsychology 24 (2009) Short Cognitive Performance Test: Diagnostic Accuracy and Education Bias in Older Brazilian Adults Mariana K. Flaks a, Orestes V. Forlenza a, Fernanda S. Pereira a, Luciane F. Viola a,mônica S. Yassuda a,b, * a Psychogeriatric Unit, Laboratory of Neuroscience (LIM 27), Department and Institute of Psychiatry, Faculty of Medicine, University of São Paulo, São Paulo SP, Brazil b Gerontology, School of Arts, Sciences and Humanities (EACH), University of São Paulo, São Paulo SP, Brazil Accepted 11 June 2009 Abstract The Short Cognitive Performance Test (Syndrom Kurztest, SKT) is a cognitive screening battery designed to detect memory and attention deficits. The aim of this study was to evaluate the diagnostic accuracy of the SKT as a screening tool for mild cognitive impairment (MCI) and dementia. A total of 46 patients with Alzheimer s disease (AD), 82 with MCI, and 56 healthy controls were included in the study. Patients and controls were allocated into two groups according to educational level (8 years or.8 years). ROC analyses suggested that the SKT adequately discriminates AD from non-demented subjects (MCI and controls), irrespective of the education group. The test had good sensitivity to discriminate MCI from unimpaired controls in the sub-sample of individuals with more than 8 years of schooling. Our findings suggest that the SKT is a good screening test for cognitive impairment and dementia. However, test results must be interpreted with caution when administered to less-educated individuals. Keywords: Neuropsychology; Screening tools; Diagnosis; Dementia; Cognitive aging; Education Introduction The Short Cognitive Performance Test (Syndrom Kurztest, SKT) (Erzigkeit, 2001) is a bedside cognitive screening test capable of detecting deficits that characterize the early stages of dementia, as it evaluates memory and attention skills, taking into account the speed of information processing. The total score provides information about the severity of the disease. Results can also be interpreted in terms of attention and memory sub-scores to evaluate the homogeneity or discrepancy of deficits in these two domains. According to the National Institute for Communicative Disorders and Stroke-Alzheimer s Disease and Related Disorders Association (NINCDS-ADRDA) guidelines (McKhann et al., 1984), the use of cognitive screening tools is recommended prior to extensive neuropsychological testing. This suggestion is based on the fact that screening is an important step for the identification and characterization of candidate patients for cognitive impairment and dementia. Hence, the use of screening instruments, with high sensitivity for the diagnosis of mild cognitive impairment (MCI) and Alzheimer s disease (AD), is vital for the early detection of cognitive decline. Studies using the original German version, as well as studies with culturally validated forms in the United Kingdom, the USA, the Netherlands, France, Italy, Belgium (Erzigkeit, 2001), suggested that the SKT is not influenced by education. As to developing countries, similar results were described by Fornazzari and colleagues (2001), in a Chilean Spanish-speaking population, with high levels of illiteracy. Studies with Mexican Spanish-speaking population reported opposite outcomes in a sample of illiterate subjects or low educational level, as the SKT lost sensitivity and specificity, despite maintaining its * Corresponding author at: Laboratory of Neuroscience (LIM 27), Department and Institute of Psychiatry, Rua Dr. Ovídio Pires de Campos, 785 Cerqueira César, São Paulo SP, Brazil. Tel.: þ ; fax: þ address: yassuda@usp.br (M.S. Yassuda). # The Author Published by Oxford University Press. All rights reserved. For permissions, please journals.permissions@oxfordjournals.org. doi: /arclin/acp033 Advance Access publication on 29 July 2009

2 302 M.K. Flaks et al. / Archives of Clinical Neuropsychology 24 (2009) capacities for subjects with medium and high educational level (Ostrosky-Solís et al., 1999). These studies have also shown that the SKT is an appropriate cognitive instrument to detect MCI and mild dementia. According to Erzigkeit (2001), it loses its capacity for precise staging in cases of severe cognitive deficits when comprehension of instructions is markedly impaired. In a preliminary study conducted in Brazil, the psychometric properties of the SKT were assessed in a sample of elderly patients with varying degrees of cognitive impairment, indicating that the test had good internal consistency (Cronbach s a ¼ 0.80) and significant correlations were found between the total score and the nine SKT sub-tests (ranging from a ¼ 0.75 to a ¼ 0.84, p, 0.01). In addition, the SKT scores were shown to be significantly correlated with the mini-mental state examination (MMSE) (Folstein et al., 1975) (r ¼ 20.66, p, ) and the clock drawing test (CDT) (Sunderland et al., 1989) (r ¼ 20.57, p, ) (Flaks et al., 2006). The objective of the present study was to evaluate the sensitivity and specificity of the SKT for the diagnoses of MCI and AD, in two groups of different educational levels (low: 8 years; or high:.8 years), using the Brazilian version of the SKT (Flaks et al., 2006). Materials and Methods Subjects and Material A total of 184 older adults (137 women, 74.5%) were assessed at the outpatient clinic of the Laboratory of Neuroscience (LIM-27), Institute of Psychiatry, Faculty of Medicine, University of São Paulo, Brazil. Illiterate subjects were not included in the sample, due to concerns about floor effects. Patients with severe dementia and those with evidence of neurological or psychiatric comorbidities were also excluded. Written informed consent was obtained from all participants in the study. Cognitively unimpaired subjects signed the document. AD patients were accompanied by relatives or care givers who provided the consent. The study was part of a larger multidisciplinary research program on aging and cognition, which included volunteer participation, anonymous handling of data, safety guidelines, and the accomplishment of good clinical practice at the local institution. This study was approved by the Ethics Committee (CAPPesq HCFMUSP) and was performed in accordance with the Helsinki Declaration. All tests were administered in Portuguese. Subjects were clinically assessed with the aid of the Brazilian version of the Cambridge Examination for Mental Disorders of the Elderly (CAMDEX) (Bottino et al., 1999; Roth et al., 1986). Objective information on cognitive and functional abilities was addressed with the Informant Questionnaire on Cognitive Decline in the Elderly (IQCode; Jorn & Korten, 1988), in addition to the CAMDEX informant-based sections. Neuropsychological examinations included the Rivermead Behavioral Memory Test (RBMT) (Wilson, Baddeley, & Cockburn, 1991), the Fuld Object Memory Evaluation (Fuld, 1980), the Nelson s Modified Cards Sorting Test (Zubicaray & Ashton, 1996), the Trail Making Test (Army Individual Test Battery, 1944), and the Wechsler Adult Intelligence Scale Revised Vocabulary and Block Design sub-tests (WAIS-R; Wechsler, 1981). The WAIS-R Block Design and Vocabulary sub-tests were used to assess the estimated intelligence coefficient. Estimated IQ scores and age were needed because original SKT normative tables are stratified according to age and IQ levels. Erzigkeit (2001) suggested that years of education and professional achievement may be used as proxy measures of IQ, when suitable tests are not available. In Brazil, years of formal education and professional achievement are not good correlates of intellectual capacities because of social inequalities (Nitrini et al., 2005). Therefore, we understood and it was best to use an objective measure of IQ to correct raw scores. International norms from Germany were used to score the SKT (normative tables available in the English manual), in which case the characterization of impairment was further submitted to clinical judgment, in the light of other clinical and functional information gathered at consensus sessions. Clinical diagnoses were established at consensus sessions, by an expert multidisciplinary team, taking into account clinical, neuropsychological, and neuroimaging data. The multidisciplinary team, including the neuropsychologists, was blind to the SKT results in the consensus sessions to establish the clinical diagnoses. Subjects were classified as either cognitively unimpaired (normal controls; NC), as with AD or MCI. The latter two diagnoses were made according to NINCDS-ADRDA criteria (McKhann et al., 1984) and Petersen s criteria (Petersen et al., 1999), respectively. Thus, the diagnosis of MCI in this test group subsumes the presence of subjective complaints, objective impairment of cognitive abilities (but not severe enough to characterize dementia), along with preserved global intellectual function, and ability to perform activities of daily living. The definition of impairment was guided by the performance on cognitive tests per se (i.e., 1.5 SD lower than the expected age and education corrected norms) in addition to the clinical interpretation of this finding, taking into account social and cultural aspects of the studied population. Although patients in the MCI group were sub-classified into three different MCI sub-types, according to the

3 M.K. Flaks et al. / Archives of Clinical Neuropsychology 24 (2009) predominantly affected cognitive domain (24 single-domain amnestic MCI; 48 multipledomain amnestic; and 10 non-amnestic MCI), the subtype sample sizes were too small to be compared. Considering this limitation the statistical analyses took into account the MCI group without considering its subtypes. Subjects without evidence of cognitive or functional impairment were regarded as NC, irrespective of the presence of subjective memory complaints. The SKT The SKT is composed of nine sub-tests. In Sub-test I, the patient is shown a board depicting 12 pictures which are to be named and memorized at the same time. In Sub-test II, which assesses immediate recall, the patient is asked to name as many pictures from the board shown in Sub-test I as he or she can remember. The pictures are shown again for 5 s as a brief learning phase. In Sub-test III, 10 two-digit numbers on a board are to be read out loud as quickly as possible by the patient. These block numbers should be placed in increasing order in Sub-test IV and replaced in their original positions in Sub-test V by the patient. In Sub-test VI, the patient is required to count out loud how often a particular target symbol occurs on a board with several others symbols. The target symbol is always depicted at the top of the board. Subtest VII assesses cognitive rigidity. Two rows composed of letters A and B are to be read out loud as quickly and as correctly as possible. When A appears, the patient must say B and vice versa. Subtest VIII assesses delayed recall, as the patient is asked to recall the 12 pictures depicted on the board shown in Sub-test I. And in the Sub-test IX, recognition memory is tested. The patient is required to identify the 12 pictures memorized in Sub-test I that are now mixed with distractors (Erzigkeit, 1992). The full assessment takes 10 min, and score calculation takes 3 min. The SKT has five parallel versions (A to E) for repeated testing, so learning effects can be minimized. Total scores fall between 0 and 27 with higher scores indicating more severe cognitive impairment. Score tables are available for six age groups (17 44, 45 54, 55 64, 65 74, 75 84, and 85 years and above) and three IQ levels (,90, , and.110) (Erzigkeit, 2001). For this study, only version A was used. The SKT instructions were translated from English into Portuguese by two neuropsychologists involved with this project. The only cultural adaptation required was the replacement of 2 of the 12 pictures that appear in the board (version A) of the original test, which were replaced by more culturally suitable alternatives: the cherry was replaced by a bunch of grapes, and the picture of a hammer was substituted by one more similar to hammers used in Brazil. Statistical Analysis ROC curve analyses, using consensus diagnosis as the gold standard, estimated the area under the curve (AUC), sensitivity, and specificity between diagnostic group pairs. Cut-off scores were also proposed to estimate the highest sensitivity and specificity values possible. When it was not possible to have high sensitivity and specificity, preference was given to high sensitivity as the SKT is a screening test. These data were provided for two educational levels (low: 8 years; or high:.8 years). To address the interaction between the SKT scores and other variables, we adopted a generalized linear model with negative binomial response, given the fact that the SKT scores were skewed (i.e., non-normal distribution) and the superdispersion was identified in the Poisson s model. Results The demographic characteristics of AD, MCI, and NC according to age, gender, and years of schooling are provided in Table 1. As to cognitive status, 46 subjects were diagnosed as AD, 82 as MCI, and 56 as NC. Age ranged from 42 to 88 years, with a mean of years. Educational level ranged from 1 to 26 years, with a mean of years of schooling. Eighty-two subjects (44.56%) had 8 years of formal education. Table 2 presents the descriptive statistics of AD, MCI, and NC performance on the SKT according to total scores and attention and memory sub-scores. To evaluate the diagnostic accuracy of the SKT, we performed ROC curve analyses for the comparison between diagnostic group pairs. Consensus diagnoses were defined as gold standard. In both educational strata (low and high, respectively), ROC curve analyses suggested that the SKT adequately discriminated AD from NC and AD from MCI. The SKT had a 95.5% sensitivity to discriminate between NC and MCI among subjects with more than 8 years of schooling (AUC 0.8, p, 0.001), albeit with low specificity. The test was less accurate to discriminate MCI from NC among seniors with limited education (Table 3).

4 304 M.K. Flaks et al. / Archives of Clinical Neuropsychology 24 (2009) Table 1. Demographic characteristics of the sample AD, mean (+ SD) MCI, mean (+ SD) NC, mean (+ SD) p-value a Multiple comparison b Gender (women) 32 (69.6) 61 (74.4) 44 (78.6).0.05 Age (7.9) (6.2) (6.1),0.001 AD=MCI, AD=NC Schooling (years) 8 (5.6) 9.2 (4.9) 13.9 (5.5),0.001 AD=NC, MCI=NC Education level c Lower 29 (62.2) 42 (51.2) 12 (21.4) Higher 17 (37.8) 40 (48.8) 44 (78.6) Notes: a ANOVA. b Tukey post hoc test. c Lower: 8 years; higher:.8 years of formal schooling. Table 2. Descriptive statistics of the total score and sub-scores of the SKT Consensus diagnosis SKT AD, median (IQR) a MCI, median (IQR) a NC, median (IQR) a p-value b Multiple comparisons c Attention 7 (3,25 9,75) 2 (1 5) 1 (0 2),0.001 AD = MC AD = NC MCI = NC Memory 5 (3 7) 1 (0 3) 0 (0 1),0.001 AD = MCI AD = NC MCI = NC Total 11 (8 15) 4 (2 7) 2 (1 3),0.001 AD = MCI AD = NC MCI = NC Notes: a IQR, Interquartile ranges. b Kruskal Wallis test. c Dunn test. Table 3. ROC curve analyses of SKT total score and normative scores suggestions between the diagnosis groups with low and high education n AUC p Sensitivity (%) Specificity (%) Cut off NC AD 8 years , years , MCI AD 8 years , years , NC MCI 8 years years , Notes: n ¼ sample size; AUC ¼ area under the curve, p ¼ significance level. Discussion The SKT is a quick and easy-to-use screening test designed to identify signs of pathological cognitive decline. Differently from most cognitive screening tools, the SKT takes into account the information processing time, which is computed in addition to the correctness of the answers provided by the patient in the calculation of scores. Such properties may render the SKT more sensitive for mild cognitive deficits, and able to separately estimate the performance on memory and attention skills. In fact, previous studies suggested that the SKT can contribute to the early diagnosis of dementia, among other cognitive disorders. In a preliminary study, we demonstrated that the psychometric properties of the Brazilian version of the SKT were similar to the original test (Flaks et al., 2006), suggesting that it adequately discriminates the occurrence of cognitive decline in older Brazilian adults.

5 M.K. Flaks et al. / Archives of Clinical Neuropsychology 24 (2009) Accordingly, we found a strong correlation between the SKT scores and other well-established cognitive screening tools, such as the MMSE and CDT. However, in this preliminary exploration, the SKT scores were shown to be affected by education, which is in disagreement with previous publications (Erzigkeit, 2001; Fornazzari et al., 2001) In general, the present results confirm the validity of the test as a screening tool for cognitive decline. Our data reinforce the notion that the SKT has an excellent power to differentiate demented (AD) from non-demented patients (NC and MCI). Very few studies have addressed the diagnostic properties of the SKT in the screening of cognitive impairment at predementia stages. Our data indicate that the SKT may have a good sensitivity to discriminate cases of MCI from NC in the sub-sample of individuals with more than 8 years of formal education. Given the long prodromal phase of most dementing illness, AD included, longitudinal re-assessment is a clinical need to ascertain the progression of cognitive impairment. Thus, the existence of five parallel forms of the SKT renders it a more suitable screening instrument, less liable to learning effects than other cognitive screening tests. However, the interpretation of these results must be made with caution in the assessment of less-educated individuals. The fact that some of the current analyses have yielded low specificity values, in particular when comparing NC and MCI patients, suggests that a representative number of patients may be incorrectly identified as having cognitive decline, due to mild deficits associated to lower education, general cognitive or motor slowing, sensory deficits, among other factors. Bearing in mind that the SKT is a screening tool, high sensitivity is prioritized when potential cutoff scores are analyzed, because false positives should be detected in more comprehensive evaluations which follow positive screening results. Based on the analysis of the current sample, we provide evidence that the SKT scores may be affected by education, albeit the correlations tend to be weak. This notion is in accordance with that by Ostrosky-Solís and colleagues (1999), who showed that education affected the SKT scores in a sample of Mexican elderly individuals, but is in partial disagreement with the original guidelines provided by the author (Erzigkeit, 2001) and further observations portraying the SKT as an instrument not affected by education (Kim, Nibbelink, & Overall, 1993; Lehfeld & Erzigkeit, 1997; Fornazzari et al., 2001). Although this interaction did not seem to affect the diagnostic accuracy of the test when discriminating demented from non-demented individuals, education bias may affect the diagnostic accuracy of the test when screening for milder degrees of cognitive impairment. Limitations of the current study include: the relatively small sample size, particularly after stratification according to diagnostic groups and education level, the absence of normal distribution of data requiring non-parametric analysis; the use of international norms for the interpretation of neuropsychological tests, which may have over-emphasized the occurrence of subtle deficits; and finally, the current study was based on a clinical sample, limiting the interpretation of SKT scores collected in community-based studies. Thus, it is advisable that the results regarding the diagnostic properties of the SKT be replicated in larger samples, and compared with data drawn from epidemiological studies, preferentially including older adults from different regions of the country. Funding This work was supported by grants from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), and from the Associação Beneficiente Alzira Denise Hertzog (ABADHS). Conflict of Interest None declared. References Army Individual Test Battery (1944). Manual of directions and scoring. Washington, DC: War Department, Adjunt General s Office. Bottino, C. M. C., Almeida, O. P., Tamai, S., Forlenza, O. V., Scalco, M. Z., & Carvalho, I. A. M. (1999). CAMDEX: The Cambridge Examination for Mental Disorders of the Elderly: tradução e adaptação para o português. Projeto Terceira Idade (PROTER). São Paulo: Instituto e Departamento de Psiquiatria do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo. Erzigkeit, H. (1992). SKT: a short cognitive performance test for assessing memory and attention. In SKT manual. Erlangen, Germany: Beltz Test. Erzigkeit, H. (2001). SKT: a short cognitive performance test for assessing deficits of memory and attention. In User s manual. (23rd ed.). Erlangen, Germany: Geromed GmbH. Flaks, M. K., Yassuda, M. S., Regina, A. C. B., Cid, C. G., Camargo, C. H. P., Gattaz, W. F., et al. (2006). The Short Cognitive Performance Test (SKT): a preliminary study of its psychometric properties in Brazil. International Psychogeriatrics, 18 (1), Folstein, M. F., Folstein, S. E., & Mchugh, P. R. (1975). Mini-mental State: a pratical method for grading the cognitive state of patients of the clinician. Journal of Psychiatic Research, 12,

6 306 M.K. Flaks et al. / Archives of Clinical Neuropsychology 24 (2009) Fornazzari, L., Cumsille, F., Quevedo, F., Quiroga, P., Rioseco, P., Klaasen, G., et al. (2001). Spanish validation of the Syndrom Kurztest (SKT). Alzheimer Disease and Associated Disorders, 15 (4), Fuld, P. (1980). Guaranteed stimulus processing in the evaluation of memory and learning. Córtex, 16 (2), Jorn, A. F., & Korten, A. E. (1988). Assessment of cognitive decline in the elderly by informant interview. The British Journal of Psychiatry, 152, Kim, Y. G., Nibbelink, D. W., & Overall, J. E. (1993). Factor structure and scoring of the SKT Test Battery. Journal of Clinical Psychology, 49 (1), Lehfeld, H., & Erzigkeit, H. (1997). The SKT: a short cognitive performance test for assessing deficits of memory and attention. International Psychogeriatrics, 9 (1), Mckhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlan, E. M. (1984). Clinical diagnosis of Alzheimer s disease: report of the NINCDS-ADRDA Work Group under the auspice of the Department of Health and Human Services Task Force on Alzheimer s Disease. Neurology, 34, Nitrini, R., Caramelli, P., Bottino, C. M., Damasceno, B. P., Brucki, S. M., & Anghinah, R. (2005). [Diagnosis of Alzheimer s disease in Brazil: cognitive and functional evaluation. Recommendations of the Scientific Department of Cognitive Neurology and Aging of the Brazilian Academy of Neurology]. Arquivos de Neuro-Psiquiatria, 63 (3A), Ostrosky-Solís, F., Dávila, G., Ortiz, X., Veja, F., Ramos, G. G., Celis, M., et al. (1999). Determination of normative criteria and validation of the SKT for use in Spanish-speaking populations. International Psychogeriatrics, 11 (2), Petersen, R. C., Doody, R., Kurz, A., Mohs, R. C., Morris, J. C., Rabins, P. V., et al. (2001). Current concepts in mild cognitive impairment. Archives of Neurology, 58, Petersen, R. C., Smith, G. E., Waring, S. C., Ivnik, R. J., Tangalos, E. G., & Kokmen, E. (1999). Mild cognitive impairment. Archives of Neurology, 56, Roth, M., Tym, E., Mountjoy, C. Q., Huppert, F. A., Hendrie, H., Verma, S., et al. (1986). CAMDEX: a standard instrument for the diagnosis of mental disorder in the elderly with special reference to the early detection of dementia. British Journal of Psychiatry, 149, Sunderland, T., Hill, J. L., Mellow, A. M., Lawlor, B. A., Gundersheimer, J., Newhouse, P. A., et al. (1989). Clock Drawing in Alzheimer s disease: a novel measure of dementia severity. Journal of American Geriatrics Society, 37, Wechsler, D. I. (1981). Examiner s manual: Wechsler adult intelligence scale revised. New York: Psychological Corporation. Wilson, B. A., Baddeley, A. D., & Cockburn, J. M. (1991). The Rivermead behavioural memory test. In R. M. Oliveira, & S. L. Schmidt (Eds.), Teste Comportamental de Memória Rivermead (2a ed. Tradutores). Suffolk, Rio de Janeiro: Thames Valley Test Company. Zubicaray, G., & Ashton, R. (1996). Nelson s modified cards sorting test: a review. Clinical Neuropsychology, 10,