It is estimated that up to three quarters of acute and subacute

Transcription

1 Topical Review Section Editors: Amanda Thrift, PhD, and Barbara G. Vickrey, MD, MPH Are Cognitive Screening Tools Sensitive and Specific Enough for Use After Stroke? A Systematic Literature Review Downloaded from by guest on May 12, 2018 Renerus J. Stolwyk, DPsych; Megan H. O Neill, BPsych(Hons); Adam J.D. McKay, PhD; Dana K. Wong, PhD It is estimated that up to three quarters of acute and subacute stroke survivors exhibit cognitive impairment, with many experiencing ongoing problems. 1,2 Cognitive impairment can significantly compromise functional recovery, quality of life, and social engagement after stroke. 2 4 Encouragingly early detection and rehabilitation can improve functional recovery of stroke-related impairments. 5 Unfortunately, however, a significant amount of cognitive dysfunction is not detected by health professionals in acute and subacute settings. 6 Comprehensive neuropsychological assessment using reliable and valid tools to measure multiple cognitive domains is considered the gold standard method of detecting and characterizing cognitive dysfunction after stroke. However, neuropsychological assessments are often considered too expensive and lengthy to be routinely administered to patients with stroke. In an attempt to improve detection of cognitive impairments, while managing expense, many national stroke clinical management guidelines now recommend the use of screening measures to detect cognitive impairment. 7 9 If cognitive difficulties are detected during this screening process, comprehensive assessment and intervention is then recommended. The Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA) are 2 screening tools that are regularly used in clinical practice. Although these tests are commonly used to detect cognitive impairment in dementia settings, neither was specifically designed for use after stroke. The profile of cognitive impairment after stroke is heterogeneous, and focal impairments such as dysphasia, dyspraxia, unilateral inattention, and agnosia are often observed. Therefore, we cannot assume that reliability and validity of cognitive screening tools found in other clinical populations will be comparable in stroke. It is acknowledged that numerous reliability and validity indices are important to consider when evaluating neuropsychological measures. However, when considering the use of cognitive screening measures, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) are particularly important to ensure patients with cognitive impairment are not missed, and patients without cognitive impairment do not undergo comprehensive neuropsychological evaluation unnecessarily. Several studies have investigated the sensitivity and specificity of cognitive screening tools within stroke populations. However, a range of different methodologies have been used, and results seem to vary considerably across studies. Thus, the aims of this review were (1) to systematically review the sensitivity, specificity, PPV, and NPV of a range of cognitive screening tools used in stroke and (2) to critically evaluate methodologies used within these studies. It is intended that findings from this review will inform clinicians regarding suitability of these screening tools for clinical use and direct best practice for future research in this field. Methods Search Strategy This systematic literature review was conducted and reported in line with the current Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Articles were identified through MedLine, PsychInfo, Scopus, PubMed, and CINAHL databases. Keywords included stroke, cerebrovasc*, cognit*, screen*, sensitivity, and specificity. Common screening measure names were also used. See Figure I in the online-only Data Supplement for an example of key words and search strategy. The search was limited to studies of adult humans published in English. The electronic search was conducted on December 27, Reference lists of articles included in this review and other relevant publications were also used to identify any studies overlooked in the electronic search. Study Selection Articles were included in this review if they met 3 key criteria: (1) male or female participants aged 18 years; (2) confirmed ischemic or hemorrhagic stroke, and (3) analysis of the sensitivity and specificity of a cognitive screening measure compared with a gold standard neuropsychological assessment. If >1 clinical population was included Received February 18, 2014; final revision received May 27, 2014; accepted July 1, From the School of Psychological Sciences, Monash University, Melbourne, Australia (R.J.S., M.H.O., A.J.D.M., D.K.W.); and Epworth Rehabilitation, Melbourne, Australia (A.J.D.M.). The online-only Data Supplement is available with this article at Correspondence to Renerus J. Stolwyk, DPsych, School of Psychological Sciences, Building 17, Clayton Campus, Monash University, Melbourne, Victoria, 3800, Australia. rene.stolwyk@monash.edu (Stroke. 2014;45: ) 2014 American Heart Association, Inc. Stroke is available at DOI: /STROKEAHA

2 3130 Stroke October 2014 Downloaded from by guest on May 12, 2018 in a study (eg, transient ischemic attack and stroke), stroke-specific data must have been available. Cognitive screening measures were included if they were designed to screen for cognitive impairment or had been used for that purpose and typically took <30 minutes to administer. Gold standard neuropsychological assessments were included if they used multiple domain-specific neuropsychological assessments with established reliability and validity. 10 Some studies identified during the literature search investigated screening tools that aim to detect just 1 cognitive domain, such as dysphasia or dyspraxia. However, understandably these studies typically only included 1 cognitive domain within their gold standard assessment and thus did not meet our eligibility criteria. In line with PRISMA guidelines, 2 authors (R.J.S. and M.H.O.) separately reviewed results from the electronic search and identified potentially relevant titles and abstracts. If the abstract suggested the article met the inclusion criteria, the full-text article was obtained and evaluated. Full-text articles were then compared across authors, and contrasting/ambiguous studies were discussed to determine whether they met criteria for inclusion. Articles that met the inclusion criteria were included for subsequent data extraction. Data Extraction The following data were extracted from each article: author, year, title, participant data (sample size, age, sex, education), stroke data (mechanism, location, hemisphere, severity), recruitment procedures (inclusion/exclusion criteria, participant attrition), cognitive screening measure used, domains and tests included in gold standard cognitive assessment, time poststroke of screening and gold standard cognitive assessments, and sensitivity, specificity, PPV, and NPV results. In some studies, multiple sets of sensitivity, specificity, PPV, and NPV data were presented at different screening measure cut points. To limit the amount of data presented, the cut point that resulted in the most favorable sensitivity and specificity results was selected. This was based on commonly used criteria of sensitivity >80% and specificity >60%. 11,12 Results Search Results Electronic and additional searching returned records; duplicate ones were removed. Sixty-six records remained following title and abstract screening. A further 50 records were excluded during full-text review for the following reasons: review articles (n=2), combined transient ischemic attack/stroke data (n=3), full neuropsychological battery used instead of screening tool (n=4), screening measure did not meet inclusion criteria (n=3), not written in English (n=2), no sensitivity and specificity data (n=6), nonstroke samples used (n=11), and gold standard neuropsychological battery did not meet inclusion criteria (n=19). Sixteen articles were found to meet our inclusion criteria and were retained for analysis. See Figure II in the online-only Data Supplement for a summary of the above. Study Participants Summary of study descriptions and evaluations is presented in Tables I and II in the online-only Data Supplement. All studies adequately reported sample size; however, few justified the sample size used or reported whether assumptions for statistical analyses were met. Based on key references within this field, 13,14 it seems that many studies did not use sufficient sample sizes, particularly those that used samples of 50 people This may have contributed to the large confidence intervals of sensitivity and specificity results noted across studies. With regard to demographic variables, all studies provided adequate information regarding age and sex. Four studies failed to include education information. 11,15,16,19 Most study samples appeared representative of stroke populations. However, most studies excluded nonnative language speakers. With regard to stroke variables, most studies reported key information such as stroke mechanism (12 of 16), location (12 of 16), and severity (11 of 16). Stroke mechanism and lesion location variables across studies appeared generally representative of stroke populations. However, most studies excluded people with severe stroke. Only 2 studies provided adequate statistical analysis of whether the final study sample used for analysis was representative of the wider patient group within their clinical setting. 20,21 There was significant heterogeneity across studies regarding time since stroke, with mean/median times ranging from 6 days 17 to 29 months 18 across studies. Two studies failed to report this information, 15,22 and others provided only limited information. Sensitivity and Specificity Methodology All studies calculated sensitivity and specificity using receiver operative characteristics curve analysis. Only 9 of 16 studies calculated PPV or NPV data. Gold standard assessments used to classify the cognitive status of participants differed across studies. Cognitive domains such as language, visual/ space perception, attention, memory, and executive functions were generally well represented. However, cognitive functions such as calculic function, praxis, and mental speed were less well represented, included in 4 of the 16 studies. Most studies used age- and education-based normative data to interpret gold standard test performances, using a criterion ranging from fifth to tenth percentile as an indicator of impaired performance. However, studies differed regarding whether impairment on single or multiple cognitive domains was required to classify participants as impaired on gold standard assessments. Furthermore, some studies did not use psychometric criteria at all, instead relying on clinician opinion taken from neuropsychological reports. 11,15,20 To reliably and validly assess sensitivity and specificity, it was expected that screening and gold standard assessments would be conducted within a short period of each other. Unfortunately, 5 studies did not sufficiently report this information, 16,18,20,23,24 and 3 reported the mean time interval between assessments as >10 days. 11,15,25 Only 6 studies stratified sensitivity or specificity results according to demographic or stroke variables. 20,22,23,25 27 Sensitivity and Specificity Results As seen in Table, the MMSE and MoCA were the most commonly studied screening measures. With regard to the MMSE, 11 studies investigated the sensitivity and specificity of the measure, with just 3 reporting sensitivity and specificity at or above commonly regarded acceptable levels (sensitivity >80% and specificity >60%). 23,25,27 All 3 studies obtained these levels using cut points of either 26/30 or 27/30. Bour et al 25 achieved acceptable sensitivity and specificity levels only when the gold standard assessment impairment criterion was increased to 2 cognitive domains. Of the 3 studies that reported adequate sensitivity and specificity, PPVs were generally >80%; however, NPVs were less impressive, ranging from 65% to 73% across studies.

3 Stolwyk et al Cognitive Screening After Stroke 3131 Table. Results From Included Studies Study Screening Measure Screening Measure Cut Point SE, % (CI) SP, % (CI) PPV, % (CI) NPV, % (CI) Agrell et al 20 MMSE 23/ Blake et al 11 MMSE < Boosman et al 26 BNI Age <55, <47 80 (49 94) 39 (20 61) Age >55, <41 92 (67 99)* 84 (58 96)* Bour et al 25 MMSE 27/28 1 impaired gold standard domain 27/28 2 impaired gold standard 80* 70* 86 domains 26/27 4 impaired gold standard 82* 75* 72 domains Cartoni et al 15 MEAMS 3 failed subtests 52 (32 71) 100 (29 100) 5 failed subtests 26 (11 46) 100 (29 100) Cumming et al 27 MMSE 26/27 82* 76* MoCA 23/24 92* 67* Downloaded from by guest on May 12, 2018 Godefroy et al 23 MMSE 26 80* 77* MoCA 23 88* 71* MMSE adj 27 89* 61* MoCA adj 23 84* 81* (adjusted s created using control group to account for age and education) Grace et al 22 MMSE < Modified MMSE < Green et al 28 RBANS 83/84 84* 90* Morris et al 12 ACE-R MMSE Nøkleby et al 16 Cognistat 8/9 81 (68 93)* 67 (22 96)* SINS 2/3 71 (57 85) 67 (22 96) CDT 9/10 63 (49 78) 67 (22 96) Nys et al 17 MMSE < Salvadori et al 21 MoCA 21 91* 76* Schweizer et al 18 MMSE <27 MoCA <26 Domain-specific data provided. No global sensitivity/ specificity data; MMSE: sensitivity=0 for all cognitive domains, specificity=1.00 for all cognitive domains; MoCA: sensitivity ranged from 40 to 100 across domains; specificity ranged from 54 to 70 across domains Srikanth et al 24 MMSE-S (4 32) 100 (92 100) 100 (40 100) 63 (50 74) Wong et al 19 MoCA 2 4 wk 17/18 75 (43 95) 95 (87 99) 75 (41 95) 95 (87 99) 1 y 21/ (74 100)* 75 (63 85)* 41 (24 61) 100 (93 100) MMSE 2 4 wk 23/24 75 (43 95) 90 (80 96) 60 (32 84) 95 (86 99) 1 y 23/24 58 (28 85) 84 (73 92) 39 (17 64) 92 (82 97) ACE-R indicates Addenbrooke Cognitive Examination, Revised; AUC, area under the receiver operating curve; BNI, Barrow Neurological Institute (screen for higher cerebral functions); CDT, clock drawing task; CI, confidence interval; MEAMS, Middlesex Elderly Assessment of Mental State; MMSE, Mini-Mental State Examination; MMSE-S, MMSE-Standardized; MoCA, Montreal Cognitive Assessment; NPV, negative predictive value; PPV, positive predictive value; RBANS, Repeatable Battery for the Assessment of Neuropsychological Status; SE, sensitivity; SINS, Screening Instrument for Neuropsychological Impairments in Stroke; and SP, specificity. *Results reach traditionally acceptable levels of 80% SE and 60% SP. Improved AUC for MMSE noted when z criterion increased to 1.5.

4 3132 Stroke October 2014 Downloaded from by guest on May 12, 2018 Five studies investigated the sensitivity and specificity of the MoCA. Three of these reported adequate sensitivity and specificity 21,23,27 ; 1 further study reported adequate sensitivity and specificity at 1 year poststroke, but not 2 to 4 weeks poststroke, 19 and the final study reported adequate sensitivity and specificity for only 2 of the 13 gold standard tests included (naming and verbal learning). 18 Acceptable sensitivity and specificity were found at different MoCA cut points across the studies, ranging from 21 to 26. Of the 4 studies that reported adequate sensitivity and specificity, only 2 studies also reported PPVs and NPVs >80%. 21,27 Four studies directly compared the MMSE and MoCA. Using area under the receiver operating curve s, 2 studies reported no significant differences between the MMSE and MoCA, 23,27 whereas another reported higher MoCA area under the receiver operating curve s compared with the MMSE at 1 year poststroke only. 19 The final study reported superior sensitivity of the MoCA compared with the MMSE. 18 The general trend noted across these 4 studies was of somewhat better sensitivity and slightly poorer specificity of MoCA compared with the MMSE. The sensitivity and specificity of other screening measures have been investigated in only a single study that met our inclusion criteria. Both the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) and Cognistat demonstrated typically accepted sensitivity and specificity levels; however, NPV was only 53% for the RBANS, and no PPV or NPV data were presented for the Cognistat. 16,28 The Barrow Neurological Institute (BNI) screen for higher cerebral functions returned acceptable sensitivity, specificity, PPV, and NPV levels in people aged >55 years but not <55 years. 26 The Middlesex Elderly Assessment of Mental State (MEAMS), Addenbrooke Cognitive Examination, Revised (ACE-R), Screening Instrument for Neuropsychological Impairments in Stroke (SINS), and Clock Drawing Test all failed to achieve adequate levels of sensitivity and specificity. 12,15,16 In addition to providing total s, some cognitive screening measures also provided domain-specific subs. Four studies included in this review examined the sensitivity and specificity of these subs to detect domain-specific cognitive impairment assessed using gold standard measures. 12,15,16,28 This information is presented in Table III in the online-only Data Supplement. Three of the 5 subs from the RBANS achieved acceptable sensitivity and specificity levels (immediate memory, language, visuospatial) with the other 2 subtests only just under acceptable thresholds (delayed memory, attention). 28 Results from the Cognistat, MEAMS, ACE-R, and SINS were less impressive. The Cognistat and MEAMS both achieved acceptable results for their naming subs; however, all other subs did not reach acceptable levels. 15,16 No subs from ACE-R or SINS reached acceptable levels. 12,16 Although memory, language, and visuospatial domains were regularly assessed, attention, processing speed, praxis, and executive function were seldom examined. Few studies specifically investigated whether methodological factors or patient variables impacted on sensitivity and specificity results. Qualitatively there was no consistent evidence to suggest time poststroke significantly affected sensitivity and specificity results. Although Wong et al 19 reported acceptable MoCA sensitivity and specificity at 1 year but not 2 to 4 weeks after stroke, other studies reported favorable MoCA, Cognistat, RBANS, and BNI sensitivity and specificity ranging from 1 week to 1 year poststroke. With regard to time interval between screening and gold standard assessments, almost all studies that reported favorable sensitivity and specificity of the MoCA, Cognistat, RBANS, and BNI used mean assessment time intervals within 1 week. Studies varied regarding their criteria for impairment on gold standard assessment. Those who used a multiple cognitive domain criterion were more likely to report adequate sensitivity and specificity compared with those who used a singledomain criterion. A minority of studies stratified sensitivity and specificity results according to demographic and stroke variables. One study reported better sensitivity and specificity results in older stroke participants. 26 Lesion hemisphere effects were equivocal. One study reported no effect, 25 whereas others reported better results in right hemisphere 20,27 or left hemisphere groups. 22 No studies specifically investigated the impact of premorbid cognitive function, stroke severity or mechanism, or cultural factors on screening measure performance. Few studies directly compared screening measure performance across different gold standard cognitive domains. The MoCA was shown to be relatively more sensitive to naming and verbal learning difficulties compared with other cognitive domains in 1 study, 18 whereas performance was higher for language and visuospatial impairments in another. 19 Discussion Sixteen articles that investigated the sensitivity and specificity of cognitive screening tools in stroke met our inclusion criteria. Eleven of these studies investigated the MMSE, and most reported inadequate sensitivity and specificity. MoCA results were somewhat better, with 3 of 5 studies reporting consistent acceptable sensitivity and specificity results. It is not clear why the MoCA performed better than the MMSE. However, possible reasons include the fact the MoCA contains items assessing executive functions, which are often affected by stroke and the total of MoCA can be adjusted for education level, albeit crudely. Interestingly, 2 relatively more recently developed measures, the RBANS and Cognistat, demonstrated traditionally acceptable levels of sensitivity and specificity. There is also some preliminary support for the use of BNI within older stroke populations. Furthermore, analysis of RBANS subs highlighted promising sensitivity and specificity results to detect a range of focal cognitive difficulties, including memory, language, and visuospatial difficulties. However, it is noted that the RBANS, Cognistat, and BNI were only investigated in 1 study that met our eligibility criteria, and further research confirming these initial findings is warranted. The above findings provide some preliminary support for the use of the MoCA, BNI, Cognistat, and RBANS as screening measures for stroke. However, these findings should be considered in the context of some key methodological issues. First, of the 7 studies that reported adequate sensitivity and specificity of MoCA, BNI, Cognistat, and RBANS, 3 either failed to report PPVs and NPVs 16 or reported NPVs <80% (indicating

5 Stolwyk et al Cognitive Screening After Stroke 3133 Downloaded from by guest on May 12, % false-negative rates). 23,28 Second, adequate sensitivity and specificity of the MoCA were found at different cut points, making recommendations for clinical practice difficult. Third, most studies did not include calculation, praxis, and speed of information processing within gold standard assessments. Thus, capacity for screening measures to detect these cognitive difficulties remains unknown, which is problematic considering that impairments of calculic, praxis, and mental speed functions are not uncommon after stroke and can significantly impact functional recovery. 1 Fourth, most studies that reported adequate sensitivity and specificity used a criterion of multiple cognitive impairments ( 2 domains) within their gold standard assessments. Studies have shown higher screening measure sensitivity for multiple-domain versus single-domain cognitive impairments. 25,29 Thus sensitivity results from studies that used multiple-domain impairment as a gold standard criterion may have been lower if a single-domain criterion was used (although equally specificity results may have been higher). Finally, few studies stratified sensitivity and specificity results according to demographic and stroke variables. This can be problematic for several reasons. For example, many screening measures do not account for age, education, or premorbid intelligence. Thus, it is possible that sensitivity of these screening measures for young and highly intelligent people may be limited, and specificity may be limited in older people and those with lower premorbid intelligence. Furthermore, people with severe stroke and those from culturally and linguistically diverse backgrounds were often excluded from studies altogether. Additional research is required within these groups before use of these screening measures is warranted. See Figure for recommendations for future research. With regard to more general methodological issues, significant heterogeneity and poor reporting regarding time interval between screening and gold standard assessments and time of assessment since stroke were noted across studies. Unless long-term predictive validity is a specific research aim, we recommend screening and gold standard assessments be conducted as time-congruent as possible. Cognitive function can change significantly during the course of stroke recovery, and results from early screening cannot be assumed to be Figure. Recommended methodological considerations for future studies. CALD indicates culturally and linguistically diverse; NPV, negative predictive value; PPV, positive predictive value; SE, sensitivity; and SP, specificity. an accurate picture of longer-term cognitive function. Thus, we recommend further research investigating the potential impact of time since stroke on the sensitivity and specificity of screening measures. On another note, although it is important to report PPV and NPV data, we acknowledge these values vary according to prevalence of impairment in the population. Thus, direct comparison of these values across studies is not valid. Importantly, however, PPVs and NPVs can be calculated based on sensitivity, specificity, and prevalence data. 30 Thus, clinicians and researchers alike may choose to use data presented in this review to estimate PPV and NPV across a range of stroke populations where prevalence data are known. Finally, few studies have investigated which specific cognitive domains are more or less likely to be detected by these screening measures. Further research is warranted. Many researchers have previously suggested that 80% sensitivity and 60% specificity of cognitive screening measures is considered adequate for clinical practice. However, the significant negative impact of cognitive impairment in stroke survivors has been consistently demonstrated. 2 4 As such, 20% nondetection of patients with cognitive impairment seems unacceptable for clinical practice. Further research is required to more comprehensively examine existing screening measures that show initial promise (MoCA, Cognistat, RBANS, BNI) addressing previous methodological weaknesses noted above. Further development of more appropriate stroke-specific screening measures may be warranted if future research does not generate positive results. Furthermore, it is important to evaluate how current recommended guidelines (cognitive screening followed by comprehensive assessment) are being implemented in clinical practice. There is evidence to suggest good adherence to cognitive screening protocols, but limited provision of further comprehensive assessment when indicated by screening results. 31 Further research exploring potential modifications to screening processes is also warranted. For example, benefits of including patient, close other, or clinician reports of cognitive difficulties, in conjunction with screening measures, to improve detection of cognitive difficulties could be explored. Addition of items not included in current screening measures, but often affected by stroke such as calculation, praxis, and mental speed, should also be considered. It would be particularly helpful for these cognitive measures to be incorporated as standard measures within stroke trials to ensure ongoing comprehensive investigation of their utility across research and clinical settings. Although beyond the scope of this review, it is also important to consider cognitive screening in other cerebrovascular disorders. For example, some screening protocols have been specifically developed for small-vessel disease and have demonstrated encouraging results. 32 This may be because of the relatively more homogeneous neuropathology and associated cognitive profile seen in this population, compared with the relatively more heterogeneous cognitive profile across the stroke population, which seems to present as a challenge for some existing screening measures to accommodate. In conclusion, a limited number of studies have adequately investigated the sensitivity and specificity of cognitive screening measures after stroke. Although most studies do not support the MMSE for clinical use, the MoCA, Cognistat,

6 3134 Stroke October 2014 Downloaded from by guest on May 12, 2018 RBANS, and BNI show some initial promise. However, further research addressing key methodological considerations and further discussion regarding what is considered acceptable sensitivity and specificity for clinical practice is required before use of these screening measures can be fully supported. None. Disclosures References 1. Leśniak M, Bak T, Czepiel W, Seniów J, Członkowska A. Frequency and prognostic value of cognitive disorders in stroke patients. Dement Geriatr Cogn Disord. 2008;26: Nys GM, van Zandvoort MJ, de Kort PL, van der Worp HB, Jansen BP, Algra A, et al. The prognostic value of domain-specific cognitive abilities in acute first-ever stroke. Neurology. 2005;64: Hochstenbach JB, Anderson PG, van Limbeek J, Mulder TT. Is there a relation between neuropsychologic variables and quality of life after stroke? Arch Phys Med Rehabil. 2001;82: Hommel M, Miguel ST, Naegele B, Gonnet N, Jaillard A. Cognitive determinants of social functioning after a first ever mild to moderate stroke at vocational age. J Neurol Neurosurg Psychiatry. 2009;80: Paolucci S, Antonucci G, Grasso MG, Morelli D, Troisi E, Coiro P, et al. Early versus delayed inpatient stroke rehabilitation: a matched comparison conducted in Italy. Arch Phys Med Rehabil. 2000;81: Jaillard A, Naegele B, Trabucco-Miguel S, LeBas JF, Hommel M. Hidden dysfunctioning in subacute stroke. Stroke. 2009;40: Intercollegiate Stroke Working Party. National Clinical Guidelines for Stroke. 4th ed. London, UK: Royal College of Physicians; National Stroke Foundation. Clinical Guidelines for Stroke Management Melbourne, Australia: National Stroke Foundation; Lindsay MP, Gubitz G, Bayley M, Hill MD, Davies-Schinkel C, Singh S, Phillips S. Canadian Best Practice Recommendations for Stroke Care (Update 2010). Ottawa, Canada: Canadian Stroke Network; Lezak MD, Howieson DB, Bigler ED, Tranel D. Neuropsychological Assessment. 5th ed. New York, NY: Oxford University Press; Blake H, McKinney M, Treece K, Lee E, Lincoln NB. An evaluation of screening measures for cognitive impairment after stroke. Age Ageing. 2002;31: Morris K, Hacker V, Lincoln NB. The validity of the Addenbrooke s Cognitive Examination-Revised (ACE-R) in acute stroke. Disabil Rehabil. 2012;34: Carley S, Dosman S, Jones SR, Harrison M. Simple nomograms to calculate sample size in diagnostic studies. Emerg Med J. 2005;22: Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143: Cartoni A, Lincoln NB. The sensitivity and specificity of the Middlesex Elderly Assessment of Mental State (MEAMS) for detecting cognitive impairment after stroke. Neuropsychol Rehabil. 2005;15: Nøkleby K, Boland E, Bergersen H, Schanke AK, Farner L, Wagle J, et al. Screening for cognitive deficits after stroke: a comparison of three screening tools. Clin Rehabil. 2008;22: Nys GM, van Zandvoort MJ, de Kort PL, Jansen BP, Kappelle LJ, de Haan EH. Restrictions of the Mini-Mental State Examination in acute stroke. Arch Clin Neuropsychol. 2005;20: Schweizer TA, Al-Khindi T, Macdonald RL. Mini-Mental State Examination versus Montreal Cognitive Assessment: rapid assessment tools for cognitive and functional outcome after aneurysmal subarachnoid hemorrhage. J Neurol Sci. 2012;316: Wong GK, Lam SW, Wong A, Ngai K, Poon WS, Mok V. Comparison of montreal cognitive assessment and mini-mental state examination in evaluating cognitive domain deficit following aneurysmal subarachnoid haemorrhage. PLoS One. 2013;8:e Agrell B, Dehlin O. Mini mental state examination in geriatric stroke patients. Validity, differences between subgroups of patients, and relationships to somatic and mental variables. Aging (Milano). 2000;12: Salvadori E, Pasi M, Poggesi A, Chiti G, Inzitari D, Pantoni L. Predictive value of MoCA in the acute phase of stroke on the diagnosis of mid-term cognitive impairment. J Neurol. 2013;260: Grace J, Nadler JD, White DA, Guilmette TJ, Giuliano AJ, Monsch AU, et al. Folstein vs modified Mini-Mental State Examination in geriatric stroke. Stability, validity, and screening utility. Arch Neurol. 1995;52: Godefroy O, Fickl A, Roussel M, Auribault C, Bugnicourt JM, Lamy C, et al. Is the Montreal Cognitive Assessment superior to the Mini-Mental State Examination to detect poststroke cognitive impairment? A study with neuropsychological evaluation. Stroke. 2011;42: Srikanth V, Thrift AG, Fryer JL, Saling MM, Dewey HM, Sturm JW, et al. The validity of brief screening cognitive instruments in the diagnosis of cognitive impairment and dementia after first-ever stroke. Int Psychogeriatr. 2006;18: Bour A, Rasquin S, Boreas A, Limburg M, Verhey F. How predictive is the MMSE for cognitive performance after stroke? J Neurol. 2010;257: Boosman H, Visser-Meily JM, Post MW, Duits A, van Heugten CM. Validity of the Barrow Neurological Institute (BNI) screen for higher cerebral functions in stroke patients with good functional outcome. Clin Neuropsychol. 2013;27: Cumming TB, Churilov L, Linden T, Bernhardt J. Montreal Cognitive Assessment and Mini-Mental State Examination are both valid cognitive tools in stroke. Acta Neurol Scand. 2013;128: Green S, Sinclair E, Rodger E, Birks E, Lincoln N. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) for post-stroke cognitive impairment screening. Int J Ther Rehabil. 2013;20: Pendlebury ST, Mariz J, Bull L, Mehta Z, Rothwell PM. Impact of different operational definitions on mild cognitive impairment rate and MMSE and MoCA performance in transient ischaemic attack and stroke. Cerebrovasc Dis. 2013;36: Smith JE, Winkler RL, Fryback DG. The first positive: computing positive predictive value at the extremes. Ann Intern Med. 2000;132: McClure JA, Salter K, Foley N, Mahon H, Teasell R. Adherence to Canadian Best Practice Recommendations for Stroke Care: vascular cognitive impairment screening and assessment practices in an Ontario inpatient stroke rehabilitation facility. Top Stroke Rehabil. 2012;19: O Sullivan M, Morris RG, Markus HS. Brief cognitive assessment for patients with cerebral small vessel disease. J Neurol Neurosurg Psychiatry. 2005;76: Key Words: cognition neuropsychology sensitivity and specificity stroke

7 Are Cognitive Screening Tools Sensitive and Specific Enough for Use After Stroke?: A Systematic Literature Review Renerus J. Stolwyk, Megan H. O'Neill, Adam J.D. McKay and Dana K. Wong Downloaded from by guest on May 12, 2018 Stroke. 2014;45: ; originally published online July 29, 2014; doi: /STROKEAHA Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX Copyright 2014 American Heart Association, Inc. All rights reserved. Print ISSN: Online ISSN: The online version of this article, along with updated information and services, is located on the World Wide Web at: Data Supplement (unedited) at: Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office. Once the online version of the published article for which permission is being requested is located, click Request Permissions in the middle column of the Web page under Services. Further information about this process is available in the Permissions and Rights Question and Answer document. Reprints: Information about reprints can be found online at: Subscriptions: Information about subscribing to Stroke is online at:

8 SUPPLEMENTAL MATERIAL Are cognitive screening tools sensitive and specific enough for use following stroke? A systematic literature review. *Renerus J. Stolwyk, DPsych 1 ; Megan H. O Neill, BPsych(Hons) 1 ; Adam J.D. McKay, PhD 1, 2 ; Dana K.Wong, PhD 1 1 School of Psychological Sciences, Monash University, Melbourne, Australia 2 Epworth Rehabilitation, Melbourne, Australia *Corresponding author: Dr Renerus J. Stolwyk, School of Psychological Sciences, Building 17, Clayton Campus, Monash University, Melbourne, Victoria, 3800, Australia. Phone: Fax: rene.stolwyk@monash.edu 1

9 Supplemental Table I. Descriptions of research methodologies Study Screening Measure Sample Size (included in statistical analyses) Inclusion and exclusion criteria Agrell et MMSE 116 Inclusion: Stroke al. 1 patients on geriatric rehabilitation unit Exclusion: Previous diagnosis of dementia of cognitive impairment Blake et al. 2 MMSE 112 Inclusion: could sit and co-operate with an assessment of 30 minutes Exclusion: Significant hearing or sight issues Boosman et al. 3 BNI 54 Inclusion: Recently suffered a stroke (<12 months), sufficient command of the Dutch language, Barthel Score 19 Demographic Information Age: Female M=77 SD=7; Male M=76 SD=7 Sex: Male N=65, Female N=51 Education: Primary school only N=85, higher-level schooling N=31 Age: M=70.8, SD=12.2 Sex: Male N=64, Women N=48 Education: Age: M= 53.8, SD=12.3 Sex Male=64.8%, Female=35.2% Education: High=33.3%, Low=66.6% Stroke Information Mechanism: Infarction N=84, Haemorrhage N=10, Unspecified N=22. Location: Hemisphere: Right N=60, Left N=39, Infratentorial N=17 Severity: Barthel Index M=67, SD=22 Mechanism: Location: Hemisphere: Left weakness N=50 right weakness N=56, Bilateral N=1, no signs N=1, presence of weakness unknown N=2 Severity: Barthel Index M=10.5 (SD.5.8) Mechanism, Location and Hemisphere: Ischaemic=46.3%, Left=44%, Right=44%, Bilateral=8%. Unknown=4%. Haemorrhagic=11.1%, Time poststroke 2-8 weeks post stroke Recruited 4 weeks poststroke Weeks: M=15.0, SD=12.8 2

10 Exclusion: Bour et al. 4 MMSE 194 Inclusion: First ever hemispheral stroke, age above 40, native speaker Exclusion: Severe aphasia, pre-stroke dementia, and other major neurological or psychiatric disorders that could interfere with testing. Cartoni et MEAMS 30 Inclusion: Stroke al. 5 patients within rehabilitation ward Exclusion: no other neurological impairments, no mental Age: M=68.4, SD=12.5 Sex: Male N=107, Female N=89 Education: high/low=86/107 Age: M=75.80, SD=7.98 Sex: 17 Male, 13 Female Education: Left=50%, Right=33.3%, Bilateral=16.7%. Subarachnoid haemorrhage=42.6%, Anterior circulation aneurysm=47.8%, posterior circulation aneurysm=30.4%, Nonaneurysmal=21.7% Severity: Mechanism and location: Territorial=39.2%, Lacunar=52.6%, Haemorrhagic=8.2 % Hemisphere: Left= 41.6%, Right=56.8%, Bilateral=1.6% Severity: Rankin M=3.1, SD=1.4 Mechanism: Location: TACI N=6, PACI N=13, POCI N=8, LACI N=3 Hemisphere: Right N=13, Left N=14, Bilateral N=3. Severity: Within 1 month 3

11 Cumming et al. 6 Godefroy et al. 7 MMSE MoCA MMSE MoCA MMSEadj MoCAadj (Adjusted s created using control confusion/dementia prior to stroke. No aphasia, visual impairment or auditory impairment such that they could not be tested 60 Inclusion: Ischaemic or intracerebral haemorrhagic stroke. Exclusion: younger than 18 years of age, unconscious on admission to hospital, required an interpreter, major visual/hearing/ language impairment 95 Inclusion: Recent infarct or haemorrhage (< 3 weeks) Exclusion: Severe general and neurological conditions precluding neuropsychological assessment, illiteracy, mental Age: M=72.1, SD=13.9 Sex: Male N=44, Female N=16 Education: M=10.5, SD=3.9 Age : M=68.2, SD=13.7 Sex: Male N=60, Female N=35 Education: primary N=55, secondary N=24, high school N=16 Mechanism: Infarct N=55, Haemorrhage N=5 Location: TACI N=4, PACI N=21, POCI N=20, LACI N=8 Hemisphere: Right N=31, Left N=20, Bilateral N=2, Unknown N=7 Severity: NIHSS median = 5 Mechanism: Infarct N=88, Haemorrhage N=7 Location: ACA N=1 MCA N=65, PCA N=4, Posterior Fossa N=18. Haemorrhages: Lobar N=3, Deep N=4. Hemisphere: Right N=44, Left N=39, Bilateral N=12 Severity: NIHSS M=2.7, SD=3.2 Days: M=98.3 SD=12.0 Days: M=6.6, SD=3.5 4

12 Grace et al. 8 group to account for age and education) MMSE Modified MMSE retardation, nonnative speaker, other neurological and psychiatric conditions 70 Inclusion: Patients with stroke undergoing treatment on a medical rehabilitation unit Age: M=74.5, SD=7.7 Sex: Male N=27, Female N=43 Education: M= 10.4, SD= 2.6. Mechanism: Location: Hemisphere: Right N=41 Left N=15, bilateral N=8, Non-focal lesion N=6. Severity: Exclusion: Green et RBANS 60 Exclusion: unable to al. 9 provide consent, aphasia, visual/auditory impairments precluding completion of tests, poor English, diagnosis of dementia, older than 80 years Morris et ACE-R 61 Inclusion:. al. 10 Recruited from MMSE stroke wards and identified from medical notes Exclusion: History of psychiatric Age: M=67.7, SD=13.2 Sex: Male N=35, Female N=25 Education: M= 10.6, SD= 3.0 Age: Median=76.0, IQR Sex: Male N=31, Female N=30 Education: Median=9.0, IQR 9-11 Mechanism: Ischemia: N=45, Haemorrhage N=6, Unspecified N=9. Location: TACS N = 17, PACS N = 15, POCS = 1, LACS N =18 Hemisphere: Right N=36, Left N=20, Bilateral: 2, Unknown = 2 Severity: Mechanism: Location: TACS N=15, PACS N=10, POCS N=9, LACS N=21, Unknown N=6 Hemisphere: Right N=36, Left N= 22, Unknown N= 3 Days: M=24, SD=16. Days: Median= 18, IQR

13 Nøkleby et al. 11 Cognistat SINS Clock Drawing problems, blind/deaf, too ill, too drowsy, poor English, moderate or severe dysphasia 49 Inclusion: Diagnosis of stroke, age > 18, stable medical condition, ability to give informed consent Exclusion: Nys et al. 12 MMSE 34 Inclusion: Ischaemic or haemorrhagic stroke Exclusion: High degree of handicap, non-native speaker, severe disturbance in communication and consciousness, blind Salvadori et MoCA 80 Inclusion: Diagnosis al. 13 of stroke (ischemic or haemorrhagic), age 18+ years, native speaker, informed consent and availability for bedside evaluation Age: Median=62, IQR= Sex: Male N=32, Female =17 Education: Age: M=64.7, SD=11.5 Sex: Male=41.2% Education (scale): Median=4 Age: M=68.2, SD=14.6 Sex: Male=66%, Female=33% Education: No PCSI M=10.9, SD=4.3, PSCI M=7.9, SD=4.1 Severity: Barthel Index Median=9.0, IQR 6-12 Mechanism: Infarct N= 40, Intracerebral Haemorrhage N=6, Subarachnoid Haemorrhage N=3 Location: Hemisphere: Right N=24, Left N=19, Other N=6 Severity: Mechanism: Infarct N=33, Haemorrhage N=1 Location: PACI 5, LACI 21, POCI 6, TACI 1, POCH - 1 Hemisphere: Right N=17, Left N=17 Severity: Modified Rankin Scale: Median=3 Mechanism: No PSCI Ischemic=82%, PSCI Ischaemic=89% Location: No PSCI Supratentorial=85%, PSCI Supratentorial=85% Hemisphere: No PSCI Left=55%, Days: Median 38, IQR Days: M = 6.5, SD = days post admission 6

14 Schweizer et al. 14 MMSE MoCA between 5th and 9th day after stroke. Exclusion: Patients living outside the Florence area 32 Inclusion: aneurysmal subarachnoid haemorrhage, classified as good outcome or moderate disability according to Glasgow Outcome Scale Srikanth et al. 15 S-MMSE 79 Inclusion: First ever stroke, spoke English, not severely dysphasic, 2 on the NIHSS, adequate vision and hearing. Wong et MoCA 74 at 2-4 week al. 16 assessment Exclusion: Inclusion: spontaneous subarachnoid Age : M=55.2, SD=7.8 Sex : Male N=13, Female N=19 Education: M=15.8, SD=3.8 Age: M=69.0, SD=14.4 Sex: Male N=47, Female N=32 Education: Cognitively intact M=9.7 SD=2.2, Cognitive impairment no dementia M=10.0 SD=2.3, Dementia M=9.9 SD= weeks Age: Median=58 IQR=49-66 PSCI Left=51% Severity: NIHSS: M=3.6, SD=4.8 Mechanism: Aneurysm=100% Location: Anterior circulation N=18, MCA N=4, Posterior circulation N=9, Unknown N=1 Hemisphere: Severity: WFNS grade: I N=22, II N=3, III N= 3, IV N=2 Mechanism: Ischaemic N= 72, Haemorrhagic N=7 Location: LACI N=26, PACI N=27, POCI N=17, TACI N=2 Hemisphere: Right N=39, Left N=25, Bilateral N=15 Severity: Barthel Cognitively intact M=19.1 SD=1.7, Cognitive impairment no dementia M=19.6 SD=1.0, Dementia M=17.6 SD=3.0 Mechanism: Haemorrhage=100% 2-4 weeks Months: M=29.3, SD=17.5 Days: M=381.6, SD= weeks & 1 year 7

15 MMSE 80 at 1-year assessment haemorrhage, hospital admission within 96 hours of ictus, between 21 and 75 years of age, native speaker, willing and able to provide informed consent Exclusion: history of previous cerebrovascular or neurological disease other than unruptured intracranial aneurysm, a history of neurosurgery before ictus, inability to cooperate with cognitive assessments (unable to obey commands). Sex: Female=50% Education: 1-year Age: Median=52 IQR=47-61 Sex: Female=55% Education: Location: ICA other than PComA N= 11, PComA N=13, ACA N=24, MCA N=18, PCA N=8 Severity: WFNS Grade 1 N= 48, 2 N=15, 3 N=4, 4 N=6, 5 N=1 1-year Location: ICA other than PComA N=16, PComA N=16, ACA N=26, MCA N=20, PCA N=12 Severity: WFNS grade 1 N=45, 2 N=21, 3 N=1, 4 N=9, 5 N=4 Hemisphere: ACA Anterior Cerebral Artery, ACE-R Addenbrooke s Cognitive Examination Revised, AUC Area Under Curve, BNI Barrow Neurological Institute screen for higher cerebral functions, CDT Clock Drawing Task, ICA IntraCerebal Aneurysm, IQR Interquartile Range, LACI/S Lacunar Infarct/Syndrome, M Mean, MEAMS Middlesex Elderly Assessment of Mental State, MCA Middle Cerebral Artery, MMSE Mini- Mental State Examination, MMSE-S Standardised Mini-Mental State Examination, MoCA Montreal Cognitive Assessment, NIHSS National Institute of Health Stroke Severity, N Sample Size, Not reported, PACI/S Partial Anterior Circulation Infarct/Syndrome, PCA Posterior Cerebral Artery, PComA Posterior Communicating Artery, POCH Posterior Circulation Haemorrhage, POCI/S Posterior Circulation 8

16 Infarct/Syndrome, PSCI, Post Stroke Cognitive Impairment, RBANS Repeatable Battery for the Assessment of Neuropsychological Status, SD Standard Deviation, SINS Neuropsychological Impairments in Stroke, TACI/S Total Anterior Circulation Infarct/Syndrome, WFNS World Federation of Neurological Surgeons 9

17 Supplemental Table II. Evaluation of Research Methodologies Study Sample size justificat ion Documented recruitment and attrition Agrell et al. 1 Yes. No differences in sex, education or diagnosis. However, study sample had less left sided lesions and were younger than nonparticipants Stroke diagnosis confirmed with brain imaging Diagnosis was determined by CT in 60% of participants. The remainder were determined by clinical investigation Time between screening and gold-standard assessment Blake et Within 3 al. 2 months Cognitive domains included in gold standard assessment for SE and SP analyses Spatial memory Arithmetic* Verbal Memory Spatial Function Visual reasoning* Spatial perception Visual inattention Memory (verbal and visual) Apraxia Executive functioning Language Boosman et al. 3 Same day Perception Language Memory Attention Reasoning Criteria for goldstandard impairment From neuropsychological tests an estimate was made blindly of whether a patients was mentally unimpaired or impaired Clinician classified patient as impaired on at least one cognitive domain Below 5 th percentile or within the first decile on one or more cognitive domains. Stratified results for demographic and stroke variables No lesion hemisphere effect for SE. SP higher in right hemisphere group. Higher SE and SP in people aged over

18 Bour et al. 4 Yes between participants and nonparticipants. Those who did not complete follow up were older, less educated and had lower original MMSE s Cartoni et Yes. However, al. 5 no statistical comparison between original and final sample. Cumming Yes Yes. However, et al. 6 no statistical comparison between original and final sample. Yes Within 1 month Days: M=20.73, SD= Days: M=8.1, SD 2.4 Godefroy Not directly et al. 7 reported. Executive Functioning Memory Mental Speed Executive function Calculation* Visuospatial* Orientation* Attention* Praxis* Reasoning* Language* Memory Language Perception Apraxia Executive functioning Visuospatial Memory Executive Language Attention Visual neglect Language Visuoconstructive Dutch norms used when possible. Performance below 10 th percentile compared to normative group. Separate analyses for 1,2, and 4 impaired domains Overall clinician conclusion from report Domain z-s of < 1 compared to age/education normative data in 2 or more domains Follow up analyses included more stringent criterion of z- <-1.5. Impairment of at least 2 cognitive Results comparable across left and right lesions MMSE: No lesion hemisphere effect MoCA: better predictive validity in right lesion participants When adjusting for 11

19 Estimated M=17.5 days abilities Working memory Long term memory Action speed Executive function domains, at a criterion that corresponded to the 5% level (whether compared to age and education normative data not reported) OR age and education, MMSE SE slightly increased but SP slightly decreased. MoCA SE and SP showed opposite trend. Grace et Within 1- al. 8 week, across three sessions Green et al. 9 Yes. However, no statistical comparison between original and final sample Days: M=4, SD = 5 Orientation and mental control Language Verbal fluency Visuospatial ability Memory Language Perception Attention Memory Executive abilities MMSE lower than 23 Performance <2 SDs below published norms in two or more cognitive domains. Memory performance was defined as impaired when it fell at least 1.5 SDs below published norms Performance < 5 th percentile on any subtest published normative data Morris et Limited Days: Verbal memory Below 5 th No lesion hemisphere effects for the modified MMSE. Higher SE and SP in left hemisphere lesions for standard MMSE. 12

20 al. 10 Nøkleby et al. 11 Limited rationale provided Limited Nys et al. 12 Yes. However, no statistical comparison between original and final sample. Salvadori et al. 13 Yes. Patients not included in analyses had lower premorbid cognitive status, higher stroke severity and lower MoCA s Schweizer Yes. However, et al. 14 no statistical comparison WHO criteria used Median=3, IQR 2 7. Visual memory Executive functioning Attention Perception Language Visuospatial Function Attention and neglect Apraxia Speed in unaffected arm Memory Yes Same day Abstract reasoning Verbal memory Executive function Visual perception and construction Visual memory Language Yes Months: M=8.4, SD 2.2 Verbal memory Focalised and maintained attention Divided and selective attention Language Attention Executive function Verbal learning and percentile on one or more cognitive domains At least one cognitive domain impaired (at least SD from estimated level before stroke) Performance below z- in at least one cognitive domain compared to matched controls Abnormal performance (below 5 th percentile on age and education adjusted norms) on one test and borderline performance on at least one other test. 2 SD below age/education normative data 13

21 between original and final sample. Srikanth et al. 15 Yes. However, no statistical comparison between original and final sample. memory Naming Motor function Yes General intellect General intellect/executive ability Verbal memory Everyday memory Spatial ability Language/executive ability Wong et al. 16 Yes During same session or in 2 sessions on consecutive days Verbal Memory Visuospatial skills and memory Attention and working memory Executive functioning and psychomotor speed Naming Each cognitive domain investigated separately Scores more than 1 SD below age- and educationcorrected norms in at least two tests measuring the same domain. Participants with dementia excluded from ROC analysis A cognitive domain deficit was defined as a cognitive domain z Cognitive impairment was defined as two or more cognitive domain deficits * Limited assessment of these domains noted. IQR Interquartile Range, Not reported, M Mean, MMSE Mini-Mental State Examination, MoCA Montreal Cognitive Examination, ROC Receiver Operating Characteristics, SD Standard Deviation, WHO World Health Organisation 14

22 Supplemental Table III. Sensitivity and Specificity Data for Focal Cognitive Domains. Screening Tool MEAMS Cartoni et al. 5 RBANS Green et al. 9 ACE-R Morris et Gold Standard Assessment Domain Memory Immediate Memory Delayed Memory Language Subtest Cut-point SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV Orientation Pass raw Remembering Pass raw Pictures Name Learning Pass raw Naming Pass raw Comprehension Pass raw Verbal fluency Pass raw Spatial Pass raw construction Fragmented Pass raw letter Unusual views Pass raw Usual views Pass raw Motor Pass raw perseveration Immediate 79/ Memory Delayed 79/ Memory Language 79/ Visuospatial 76/77 Attention 83/84 Visuospatial 14 Fluency 8 15

23 SINS Nokleby et al. 11 al. 10 Memory Attention and 16 Orientation Cognistat Memory >65yo 7/ Nokleby et al yo 9/10 Attention 5/ Similarities >65y 3/ y 4/5 Comprehension 4/ Naming 6/ Repetition 10/ Visuo- >65yo 2/2 construction 65yo 3/4 Aphasia 11/ Visuocognitive 17/18 Apraxia 20/21 16

24 Supplemental Table III. Sensitivity and Specificity Data for Focal Cognitive Domains (continued). Screening Tool MEAMS Cartoni et al. 5 Subtest Orientation Remember Pictures Name Learning Naming Comprehen. Verbal fluency Spatial construction Fragmented letter Unusual views Usual views Cutoff Pass raw Pass raw Pass raw Pass raw Pass raw Pass raw Pass raw Pass raw Pass raw Pass raw Gold Standard Assessment Domain Perception/Visuospatial Executive Function Attention/Neglect Speed Praxis SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV SE SP PPV NPV

25 RBANS Green et al. 9 ACE-R Morris et al. 10 Cognistat Nokleby et al. 11 SINS Nokleby et al. 11 Motor persever. Pass raw 79/ Immediate Memory Delayed 79/80 Memory Language 79/80 Visuospatial 76/ Attention 83/ Visuospatial Fluency Memory 20 Attention and Orientation Memory >65y 7/8 65y 9/10 Attention 5/ Similarities Age depen dent Comprehen. 4/5 Naming 6/ Repetition 10/11 Visuoconstruction >65y 2/3 65y 3/ Aphasia 11/12 Visuocog. 17/ Apraxia 23/

26 All results presented as percentages. Due to large amount of data Confidence Intervals are not included in this table. Bolded data indicates results reach traditionally acceptable levels of 80% SE and 60% SP. MEAMS Middlesex Elderly Assessment of Mental State, NPV Negative Predictive Value, Not Reported, PPV Positive Predictive Value, RBANS Repeatable Battery for the Assessment of Neuropsychological Status, SINS Neuropsychological Impairments in Stroke, SE Sensitivity, SP Specificity 19

27 Supplemental Figure I. Search strategy example OVID MEDLINE 20

28 Supplemental Figure II. Flowchart of Systematic Literature Review 21