An evaluation of screening measures for cognitive impairment after stroke

Transcription

1 Age and Ageing 2002; 31: # 2002, British Geriatrics Society An evaluation of screening measures for cognitive impairment after stroke HOLLY BLAKE, MICHELLE MCKINNEY, KAREN TREECE, ELIZABETH LEE, NADINA B. LINCOLN School of Psychology, University of Nottingham, Nottingham NG7 2RD, UK Address correspondence to: N. Lincoln. Fax: (q44) nbl@psychology.nottingham.ac.uk Abstract Objectives: to assess the sensitivity and specificity of a screening battery for detecting cognitive impairment after stroke. Design: a randomized controlled trial. Methods: stroke patients were recruited from hospitals in three centres. Patients were screened for cognitive impairment on the Mini-Mental State Examination, the Sheffield Screening Test for Acquired Language Disorders and Raven s Coloured Progressive Matrices and received a further battery of assessments of cognitive function. Sensitivity and specificity values were calculated for the three screening measures for overall conclusions regarding cognitive impairment reached from a comprehensive assessment. Receiver Operating Characteristic Curves were plotted. Conclusion: the Mini-Mental State Examination was not a useful screen for memory problems or overall cognitive impairment after stroke. The Sheffield Screening Test for Acquired Language Disorders was an appropriate screen for language problems. The Raven s Coloured Progressive Matrices was appropriate as a screen for perceptual problems and visual inattention but not for executive deficits. Keywords: stroke, cognitive impairment, sensitivity, specificity, screening Introduction Stroke results in motor and cognitive impairments, with the severity of the stroke associated with the degree of impairment [1]. The type and severity of the cognitive impairments varies according to the site of neurological damage and its magnitude. Cognitive problems after stroke are common [2] and need to be assessed. However, this is a time consuming process and therefore it is advantageous to use screening measures to select those patients who require further evaluation. The value of neuropsychological assessment is increasingly being recognized in the management of stroke patients [3]. In the National Clinical Guidelines for Stroke [4] the importance of rapid neuropsychological assessment in stroke rehabilitation services is emphasized. The benefits are wide ranging, including the provision of prognostic information, providing a basis on which to plan cognitive remediation and providing advice and recommendations to other members of the rehabilitation team, patients, families and the social services [5]. The cost of this provision can be high, both financially and in time, as many of the assessments are lengthy to administer. It is therefore important to target such assessment where it is most needed. Screening tests are needed to highlight problem areas [6]. They must be sensitive enough to detect all those with problems and specific, so as not to identify anyone as having cognitive problems when they do not. It is an advantage if they are quick and easy to administer by clinical personnel who may have little training in neuropsychological testing. The Mini-Mental State Examination (MMSE) [7], Raven s Coloured Progressive Matrices (RCPM) [8] and the Sheffield Screening Test for Acquired Language Disorders (SST) [9] are brief, easily administered and potentially suitable as screening measures for cognitive impairment after stroke. The MMSE assesses cognitive aspects of mental function [7] and consists of two sections, one covering orientation, memory and attention and requiring only verbal responses, the second addressing the ability to follow verbal and written commands, name, write a sentence spontaneously and copy a complex polygon figure [7]. As an assessment of cognitive state it shows good discrimination between the cognitively impaired and normals [10] and has demonstrable validity and reliability [7, 11]. The MMSE has been recommended as a routine cognitive screening instrument for elderly persons in the community [12] with 80% sensitivity and 98% specificity for detecting cognitive problems. A comparison between the MMSE and the Cognitive Section of the Cambridge 451

2 H. Blake et al. Examination for Mental Disorders of the Elderly (CAMCOG) concluded that both screening tests were able to detect dementia, but, receiver operating characteristic analysis suggested that the CAMCOG was a more accurate screening assessment [13]. However, the MMSE is more frequently used in clinical practice. The SST [9] was developed as a non-specialist clinical aid to help identify dysphasia and to enable an appropriate referral to a speech and language therapist [14]. It has two sections, one assessing receptive skills and one assessing expressive skills. It is as short and simple to administer as the Frenchay Aphasia Screening test but has the advantage that it requires no stimulus cards and is able to detect milder communication problems [14]. The RCPM [8] is a non-verbal assessment of intelligence based on visual perceptual abilities and analogical reasoning. It is a shortened version of the Standard Progressive Matrices and is suitable for the assessment of elderly people. The aim of the study was to investigate the sensitivity and specificity of the MMSE, SST and RCPM in screening for cognitive impairment. Methods Stroke patients were recruited within 4 weeks of admission to hospitals in Nottingham, Derby and Mansfield, UK, as part of a randomized controlled trial evaluating the impact of cognitive assessment in multi-disciplinary stroke rehabilitation [15]. Patients giving informed consent were included provided that they were conscious on admission to hospital (thus excluding patients who were unlikely to survive), could sit and co-operate with the assessments for 30 minutes at a time and had no significant visual and hearing impairments which prevented them from completing all of the screening assessments. Basic demographic information was collected including age, sex, residence prior to the stroke, pre-stroke Barthel Index [16] and carer support. Information was collected on level of consciousness, side of stroke, degree of weakness in arm and leg, swallowing status and continence of bowel and bladder. Functional independence was assessed using the Barthel Index [16]. The MMSE, SST and RCPM were administered to all recruited patients. The total score was calculated for the MMSE and also the orientation, memory and attention items. The receptive, expressive and total scores were calculated for the SST. The total score was calculated for the RCPM as an indicator of visuospatial and executive deficits. In addition the proportion of right and left answers was calculated to indicate visual inattention using the formula (R L)/(RqL)3100 and ignoring the sign, so that the scale ranged from 0, responses equally distributed between the right and left, and 100, extreme bias of responses to one side. After the screening assessment, patients were randomly allocated to one of two groups as part of the evaluation of the value of cognitive assessment after stroke [15]. The intervention group received a detailed cognitive assessment. The results were summarized in a report made available to all staff involved with patient rehabilitation including nurses, therapists and doctors and the patient s GP, the patient themselves and with consent, their relatives. Patients in the intervention group who had received both a screening assessment and detailed assessment battery, were included in this study. A battery of detailed assessments was administered to all patients in the intervention group within 3 months of the screening assessment. This included the Shortened National Adult Reading Test (NART) [17] as an assessment of pre-morbid intellectual abilities. The Rey-Osterreith Complex Figure [18] was used to assess spatial perception, and the Behavioural Inattention Test (BIT) Star Cancellation [19] to assess visual inattention. Recognition memory was assessed on the Salford Objective Recognition Test (SORT) [20] and immediate and delayed verbal recall on the Adult Memory and Information Processing Battery (AMIPB) Story Recall [21]. The Apraxia Test [22] was used to assess for apraxia. Language and executive function were assessed on the Controlled Oral Word Association Test [23] and executive function on the Cognitive Estimation Test [24]. Based on the findings from the standard battery of cognitive assessments, supplementary tests were selected to clarify the nature of any cognitive deficits identified. These assessments were tailored to individual needs and provided information to be incorporated into the overall conclusions in the psychology reports. Perceptual deficit was assessed further using the Visual Object and Space Perception Test [25], BIT [19] or the Birmingham Object Recognition Test [26]. Memory was assessed in more detail on the AMIPB [21], Recognition Memory Test (RMT) [27], Doors and People [28] or the Revised Weschler Memory Scale (WMS-R) [29]. Attention was assessed on the Test of Everyday Attention (TEA) [30]. Language was assessed on the Graded Naming Test [31], Token Test [32] and the Psycholinguistic Assessments of Language Processing in Aphasia (PALPA) [33]. Executive function was assessed on the Modified Card Sorting Test [34] and Verbal and Spatial Reasoning Test [35]. The Stroke Driver s Screening Assessment [36] was used for those patients who wished to return to driving. The information obtained from testing was summarized in a detailed structured written report and recommendations were provided for approaches to the patient s care. This was made available to hospital staff involved in the patient s care and the GP. From the conclusions of these reports, patients were classified as impaired or not impaired in spatial perception, visual inattention, memory, (verbal and visual), apraxia, executive function and language. The assessments were conducted by an assistant psychologist, supervised by a chartered clinical psychologist. Data were analyzed using SPSS Release 9. The sensitivity and specificity of the screening measures were determined in relation to the presence of impairment as 452

3 Screening measures for cognitive impairment after stroke given in the overall conclusions of the written reports. Receiver Operating Characteristic (ROC) curves [37] were plotted to show the sensitivity and false positive rates resulting from various cut-offs for the screening tests. The MMSE was evaluated as a screening measure for impairments of memory and overall cognitive impairment. The SST was evaluated with respect to impairments of language and verbal memory. The RCPM was evaluated in relation to impaired spatial perception, visual inattention and impaired executive functioning. Results There were 112 stroke patients recruited and randomly allocated to receive a cognitive assessment. Of these, 64 were men and 48 were women. They were aged (mean age 70.8 S.D years). Left weakness occurred in 50 patients, right weakness in 56 patients, bilateral weakness in one patient; no signs of weakness were detected in one patient and in two patients, the presence of weakness was unknown. The mean Barthel Index at recruitment was 10.5 (S.D. 5.8). The descriptive results from the screening and detailed assessments are shown in Table 1. The number of patients impaired on each assessment according to recommended cut-offs was calculated. Results are shown in Table 1. About a third of patients were impaired on each of the measures used. Impairment on recognition memory for words was less frequent (11%) and visuospatial impairment on the Rey Figure was more frequent (66%). According to the overall conclusions from the written reports, 43 (40%) of patients had language problems, 49 (51%) had difficulties with spatial perception, 25 (26%) had visual inattention, 48 (52%) had executive deficits, 50 (56%) had verbal memory problems and 38 (42%) visual memory problems. The sensitivity and specificity of the screening battery was then determined in relation to overall impairment classifications from the written reports. ROC curves were formed by plotting the range of sensitivity and specificity pairs for the screening assessments with impairments. A test is more accurate in classifying patients as impaired or not impaired the closer the curve is to the left-hand corner of the graph. ROC curves showing the sensitivity and specificity for the full range of possible cut-offs are shown in Figures 1, 2 and 3. Optimum cut-offs were identified from the ROC curves for each comparison. When evaluating the usefulness of a screening measure to identify those individuals with cognitive impairment, good sensitivity ()80%) is desirable. Therefore, cut-offs were selected that maximized the sensitivity of the tests whilst maintaining an acceptably low false positive rate (specificity)60%). No optimum cut-offs could be identified from the ROC curves for the MMSE with overall visual and verbal memory problems. Separating the orientation memory and attention questions did not provide an acceptable cut Table 1. Descriptive results from the screening and detailed assessments Assessments n Mean S.D. Range % impaired Screening Tests MMSE Sheffield Screening Test RCPM Total Laterality Index Detailed Assessments SORT Words SORT Faces Immediate Story Recall Delayed Story Recall Word Fluency Rey Figure Copy Star Cancellation Cognitive Estimation Test Figure 1. Comparisons between MMSE and overall conclusions for impairment. MMSE = Mini-Mental State Examination. RCPM = Raven s Coloured Progressive Matrices. SORT = Salford Objective Recognition Test. Figure 2. Comparisons between SST and overall conclusions for impairment. 453

4 H. Blake et al. Figure 3. Comparisons between RCPM and overall conclusions for impairment. off with good sensitivity and specificity to identify those with memory problems. However comparing the MMSE with cognitive impairment in any area identified an optimum cut-off of -24 with good specificity (88%) and moderate sensitivity (62%). A cut-off of -15 on the SST was optimum for the detection of overall language impairment with good sensitivity (89%) and specificity (88%). Separate analysis of the receptive and expressive sub-sections showed that a cut-off of -7 on the receptive scale had a sensitivity of 79% and a specificity of 85%. For the expressive section the optimum cut-off was -9 indicating language problems with a sensitivity of 82% and specificity of 89%. The ROC curve for the RCPM as a screening for executive deficit showed no optimum cut-off. Curves for RCPM with spatial deficit identified a cut-off of -23 with good sensitivity (87%) and adequate specificity (69%). When RCPM was compared with visual inattention, a cut-off of -19 gave high sensitivity (91%) and adequate specificity (72%). Using the laterality index as a screening measure for inattention gave an optimum cut-off of -25% lateralised, which had a sensitivity of 76% and a specificity of 78% for detecting inattention. Discussion In comparison with the overall conclusions from reports, the MMSE overall score was not a good screening measure to detect memory problems after stroke because there was no clear cut-off score to indicate a problem requiring further evaluation. This also applied when only the orientation, memory and attention questions were considered. It may be useful to evaluate whether an alternative measure of memory such as the SORT [20] or the R-CAMCOG [38] provide clearer cut-offs for impairment. However, the MMSE does not exclusively measure memory. It was able to detect those with any type of cognitive problems with moderate sensitivity and specificity. The SST was a useful screening measure for language problems. The optimum cut-off of -15 was both sensitive and specific. This corresponds with that specified in the test manual and is consistent with other research on screening measures for aphasia [14]. Using the full SST was more accurate than either the receptive or expressive subscales alone. The SST is therefore recommended as a brief screening measure for identifying language difficulties after stroke. The concern that the SST might also detect verbal memory problems was not supported. The RCPM was found to be a sensitive and specific measure of both visual inattention and spatial perception deficits. The cut-off of -23 for spatial perception problems had good sensitivity and adequate specificity. Although the Rey Figure Copy has previously been recommended as a screening instrument for perceptual deficit following stroke [6], the RCPM has the advantage in that it does not require motor skills for drawing. The RCPM is recommended as an easily administered bedside screening assessment for the detection of perceptual problems. The RCPM performed poorly as a screening measure for executive deficit. This is surprising given that the test was developed as both a measure of perceptual difficulties and analogical reasoning. This is possibly because executive function is complex and consists of a collection of functions rather than being a unitary deficit [39]. Therefore, the RCPM may have been measuring a single deficit in executive function and was unsatisfactory as a screen for the presence of any deficit of executive function. This implies that several measures of executive deficit are needed rather than a single test. The laterality index of the RCPM was no more sensitive or specific than the simple total for detecting inattention. Therefore the simple cut-off of the total RCPM of -19 as indicative of visual inattention should be used. One limitation of the study is that all patients did not complete all of the cognitive assessments. If they had it would have been possible to select a gold standard for each cognitive ability, such as the PALPA [33] for language problems and the BIT [19] for visual neglect, and assess the performance of the screening measures against these gold standards. However, this was impractical because it would have required all patients to tolerate a very lengthy assessment schedule. The procedure was designed to be representative of cognitive assessment in clinical practice and therefore not all abilities were assessed in detail. Assessments were done on patients selected for a randomized controlled trial [15] and therefore may not be representative of all stroke patients admitted to hospital. Patients who were not able to tolerate the assessment were excluded and they may have had cognitive deficits. It may be necessary to develop simpler shorter screening measures for this group of patients. 454

5 Screening measures for cognitive impairment after stroke Nevertheless, the patients included were probably representative of the majority who would receive rehabilitation following stroke. It was also not possible to determine whether all cognitive deficits were a consequence of the stroke and not a pre-existing cognitive impairment. However, this would not necessarily have affected the relationship between the screening measure and overall detailed assessment. The MMSE was a moderately useful screening measure for general cognitive impairment. However it lacked sufficient sensitivity to be recommended. The SST and RCPM are recommended as useful measures for screening for language and perceptual impairments. However, further studies are needed to identify appropriate measures to screen for memory impairment and executive deficits after stroke. Key points. The MMSE lacks sensitivity as a screening measure for cognitive impairment after stroke.. The SST is a useful screening measure for language disorders.. RCPM is a useful screening measure for visual inattention and spatial problems but not executive deficits. Acknowledgements The authors would like to thank all the patients in the study, Liz Peel for help with data collection, participating hospitals for access to the stroke registers and the Stroke Association for financial support. References 1. Chambers BR, Norris JW, Shurvell BL, Hachinski VC. Prognosis of acute stroke. Neurology 1997; 37: Desmond DW, Moroney JT, Sano M, Stern Y. Recovery of cognitive function after stroke. Stroke 1996; 27: Wade D. National Guidelines for Stroke. London: The Royal College of Physicians Clinical Effectiveness and Evaluation Unit, Royal College of Physicians. National Clinical Guidelines for Stroke, Section on Cognitive Impairment. Prepared by the Intercollegiate Working Party for Stroke. London: Royal College of Physicians, Davies S. Neuropsychological assessment of the older person. In Woods RT ed. Handbook of the Clinical Psychology of Ageing. Chichester: John Wiley & Sons 1996: Lincoln NB, Drummond AER, Edmans JA, Yeo D, Willis D. The Rey Figure Copy as a screening instrument for perceptual deficits after stroke. Br J Occup Ther 1998; 61: Folstein MF, Folstein SE, McHugh PR. Mini-Mental State: A practical method of grading the cognitive status of patients for the clinicians. J Psychiatric Res 1975; 12: Raven JC. Guide to using the Coloured Progressive Matrices. London: H.K. Lewis, Syder D, Body R, Parker M, Boddy M. Sheffield Screening Test for Acquired Language Disorders. Windsor: NFER-NELSON, Ebrahim S, Harwood R. Stroke: Epidemiology, Evidence and Clinical Practice. Oxford: Oxford University Press, Engedal K, Gilje K, Laake K. Prevalence of dementia in a Norwegian sample aged 75 years and over and living at home. Compr Gerontol Series A 1988; 2: MacKenzie DM, Copp P, Shaw RJ, Goodwin GM. Brief cognitive screening of the elderly: a comparison of the Mental Status Examination (MMSE), Abbreviated Mental Test and Mental Status Questionnaire (MSQ). Psycholog Med 1996; 26: De Koning I, Van Kooten F, Dippel DWJ, Van Harskamp F, Grobbee Koudstaal PJ. The CAMCOG: a useful screening instrument for dementia patients. Stroke 1998; 29: Al-Khawaja I, Wade DT, Collin CF. Bedside screening for aphasia: a comparison of two methods. Br J Occupat Ther 1998; 61: McKinney M, Blake H, Treece K, Lincoln NB, Playford ED, Gladman JRF. Evaluation of cognitive assessment in stroke rehabilitation. Clin Rehab 2002; 16: Wade DT, Collin C. The Barthel ADL Index: a standard measure of physical disability? Int Disabil Stud 1988; 10: Beardsall L, Brayne C. Estimation of verbal intelligence in an elderly community: a prediction analysis using a shortened NART. Br J Clin Psychol 1990; 29: Lezak MD. Neuropsychological Assessment. New York: Oxford University Press, Wilson B, Cockburn J, Halligan PW. Behavioural Inattention Test. Titchfield, UK: Thames Valley Test Company, Burgess IS, Dean M, Lincoln NB, Pearce MJ. Salford Objective Recognition Test. Nottingham: Nottingham Rehabilitation, Coughlan AK, Hollows SE. The Adult Memory and Information Processing Battery. Leeds: St James University Hospital, Kertesz A, Ferro JM. Lesion size and location in ideomotor apraxia. Brain 1984; 107: Benton AL, Hamsher K. Controlled Oral Word Association Test. Iowa City, IO: AJA Associates Inc., Shallice T, Evans ME. The involvement of the frontal lobes in cognitive estimation. Cortex 1978; 14: Warrington EK, James M. The Visual Object and Space Perception Battery. Titchfield, UK: Thames Valley Test Company, Riddoch MJ, Humphreys GW. Birmingham Object Recognition Battery. Hove: Lawrence Erlbaum Associates, Warrington EK. Recognition Memory Test. Windsor: NFER-NELSON,

6 H. Blake et al. 28. Baddeley A, Emslie H, Nimmo-Smith I. A Test of Visual and Verbal Recall and Recognition: Doors and People. Titchfield, UK: Thames Valley Test Company, Weschler D. Weschler Memory Scale Revised. New York: The Psychological Corporation, Robertson IH, Ward T, Ridgeway V, Nimmo-Smith I. The Test of Everyday Attention. Titchfield, UK: Thames Valley Test Compan, McKenna P, Warrington EK. Graded Naming Test. Windsor: NFER-NELSON McNeil MR, Prescott TE. Revised Token Test. Austin, TX: PRO-ED, Kay J, Lesser R, Coltheart M. Psycholinguistic Assessments of Language Processing in Aphasia (PALPA). Hore: Psychology Press, Nelson HE. A modified card sorting test sensitive to frontal defects. Cortex 1976; 12: Langdon DW, Warrington EK. VESPAR: Verbal and Spatial Reasoning Test. Hove: Lawrence Erlbaum Associates, Nouri FM, Lincoln NB. Stroke Driver Screening Assessment. Nottingham: Nottingham Rehabilitation, Altman DG, Bland JM. Diagnostic tests 3: receiver operating characteristic plots. Br Med J 1994; 309: De Koning I, Dippel DWJ, Van Kooten F, Koudstaal PJ. A short screening instrument for post-stroke dementia: the R-CAMCOG. Stroke 2000; 31: Sbordone RJ. The executive functions of the brain. In Groth-Marnet G ed. Neuropsychological Assessment in Clinical Practice: A Guide to Test Interpretation and Integration. New York: John Wiley, Received 4 February 2002; accepted in revised form 5 July