Diagnostic accuracy of clinical stroke scores for distinguishing stroke subtypes: a systematic review Reviewers Clifford Mwita 1,2 MD Contact: cmwita@gmail.com Duncan Kajia 1 MBChB Contact: kajiahtree@yahoo.com Samson Gwer 1,3 MBChB, MRCPCH, PhD Contact: samgwer@gmail.com Anthony Etyang 4 MBChB, MMed Contact: aetyang@kilifi.kemri-wellcome.org Charles Newton 4,5,6 MBChB, MRCP, MD, FRCPCH Contact: cnewton@kilifi.kemri-wellcome.org 1. Joanna Briggs Institute Affiliate Center, Kenya 2. Thika Level-5 District Hospital, Thika, Kenya 3. Afya Research Africa, Nairobi, Kenya 4. Kenya Medical Research Institute (KEMRI)/ Wellcome Trust Research Programme, Centre for Geographic Medicine Research (Coast), Kenya 5. Neurosciences Unit, Institute of Child Health, University College London, United Kingdom 6. Department of Psychiatry, University of Oxford, Oxford, United Kingdom 1
Review question/objective This review aims to examine the sensitivity and specificity of clinical stroke scores in distinguishing ischemic and hemorrhagic stroke subtypes in patients with acute stroke and determine the score best suited for use in resource poor settings. The specific review question is: what is the sensitivity and specificity of clinical stroke scores in distinguishing ischemic from hemorrhagic stroke in patients with acute stroke, compared with non-contrast computed tomography? Background Stroke is a major cause of morbidity globally and is associated with up to 5.54 million deaths every year, two thirds of which occur in resource poor countries (RPC). 1 The prevalence of stroke is reported to be higher in developed countries, probably due to a higher proportion of elderly individuals in their population. However, it is possible that in resource poor settings, a lower prevalence is apparent due to higher case fatality. The World Health Organization (WHO) defines stroke as a clinical syndrome of rapid onset of focal cerebral deficit lasting more than 24 hours (unless interrupted by surgery or death) with no apparent cause other than a vascular one. There are two main subtypes, ischemic or hemorrhagic. Of the two types, ischemic stroke is more common, occurring in up to 80% of patients. 2 Infarction takes several hours to occur because following occlusion of a cerebral artery, anastomotic channels from other arterial territories open up to restore perfusion to its territory. This allows for restoration of blood supply and reversal of the process, with subsequent reduction in neurological deficits, disability and other complications. 2-3 Hemorrhagic stroke accounts for 15%-20% of all stroke cases and is associated with a higher mortality. 2, 4 There is bleeding directly into brain parenchyma due to damaged cerebral blood vessels. 5 Blood in the brain parenchyma causes disruption of neurons with localized cessation of function. The hemorrhage may expand and cause progression of neurological deficits. If large enough, it may cause a shift of intracranial contents and lead to rapid death. This type of stroke presents in a similar manner to ischemic stroke but has a more dramatic preceding clinical picture, usually headache, altered mental status, seizures, nausea and vomiting, and marked hypertension. However, none of these clinical features have been shown to reliably distinguish between the two sub-types of stroke. 6-7 Clinically, stroke may evolve from a transient ischemic attack (TIA), progressing stroke, to a completed stroke, based on the duration and evolution of symptoms. 3 In TIA, symptoms resolve within 24 hours of onset while in progressing stroke, symptoms worsen, perhaps due to increasing volume of infarct or haemorrhage. In a completed stroke, the neurological deficits persist but do not progress. Neurological deficits can be elicited from the history and, if persistent, from neurological examination. For optimal management of acute stroke, a distinction must be made between ischemic and hemorrhagic stroke since the therapy is different. 2 Ischemic stroke warrants institution of thrombolytic and/or antiplatelet therapy. Antiplatelet therapy has been shown to improve outcome and be cost effective even in RPC. 8-9 In the management of acute hemorrhagic stroke, haemostatic therapy may be given but there is conflicting evidence on its benefit. 5 Inadvertent administration of anti-platelet or anti-thrombotic therapy in hemorrhagic stroke sub-type may aggravate the clinical course. Conversely, use of haemostatic therapy in 2
ischemic stroke may promote vascular occlusion and worsen the infarction. Ideally, either thrombolytic or haemostatic therapy should be given soon after the onset of stroke in order to improve outcome. Index tests and reference test Non-contrast computed tomography (CT) scan is the most widely used brain imaging modality in patients with stroke and it reliably detects cerebral haemorrhage. It is the gold standard for distinguishing stroke sub-types. 10-11 It is cheaper than magnetic resonance imaging (MRI), but it is still expensive and inaccessible in many RPCs. To overcome the difficulties in accessing CT scan in the diagnosis of stroke and to enhance clinical bedside diagnosis, clinical stroke scores have been developed. The most commonly used ones include the Guy s hospital score (GHSS), 6 the Besson score, 12 the Greek stroke score 13 and the Siriraj stroke score (SSS). 14 In developing these scores, clinical variables that could potentially distinguish ischemia from haemorrhage in patients with acute stroke were used. A validation study was then performed to test the scores in patient populations other than the ones used in their development. The Greek score (GSS) was found to be 97% sensitive and 99% specific for the diagnosis of hemorrhagic stroke 13 while the Siriraj stroke score was 89.3% sensitive for haemorrhage and 93.2% sensitive for ischemia. 14 The Besson score had a positive predictive value of 100% in the diagnosis of ischemic stroke 12 while the Allen score is reported to have accurately diagnosed 90% of vascular lesions. 6 For the Siriraj score a score above 1 indicates hemorrhagic stroke while a score below -1 indicates ischemic stroke. A result between -1 and 1 indicates an equivocal result needing a CT scan. A score of below 4 for the Allen score indicates ischemic stroke while a score of above 24 indicates hemorrhagic stroke. For the Greek score, a score of 3 or less points to ischemic stroke while a score of 11 and above points to hemorrhagic stroke. A score of less than 1 for the Besson score diagnoses ischemic stroke with no cut off reported for hemorrhagic stroke. Rationale for the review CT scan is both expensive and largely unavailable in resource poor settings. There is also a paucity of specialists in the field of neurology with a large number of hospitals lacking a specialised stroke unit. Most patients with stroke in these parts of the world are inadequately diagnosed, resulting in poor outcomes. These constraints highlight the need for clinical stroke scores to distinguish between the stroke sub types. While these scores may not be sensitive enough to replace neuro-imaging, they are simple, cheap and practical, and do not require the presence of a specialist to administer and interpret. However, their true accuracy and value in the diagnosis of stroke remains unknown. We intend to conduct a systematic review of available literature to examine the evidence on the accuracy of clinical stroke scores in distinguishing between stroke sub types. Further, we aim to compare the accuracies of the clinical scores in order to determine which score is best suited for use in RPCs. Thus, this will be a review of diagnostic test accuracy. The main outcome measures will be sensitivity and specificity of the scores compared to CT scan as the reference standard. For the purpose of this review, sensitivity will be defined as the probability that a patient 3
with the disease will have a positive test result while specificity will be defined as the probability that a patient without the disease will have a negative test result. We searched PubMed, EMBASE, and the Cochrane and the Joanna Briggs Institute Libraries of systematic reviews in July 2012 and we did not identify any reviews either published or underway on this topic. Inclusion criteria Types of participants This review will consider patients admitted to hospital with a diagnosis of acute stroke according to the WHO criteria. 2 There will be no age or sex limitation and we will consider participants from all ethnic backgrounds. Types of interventions This review will consider studies that evaluate the Siriraj, Allen (Guy s Hospital), Besson and Greek stroke scores with CT-Scan as the reference standard. Details on the calculation of test scores have been described previously. 6, 12-14 Studies that compare two or more of these scores simultaneously will also be included. Types of outcomes The review will consider studies that report on the sensitivity and specificity of stroke scores compared to CT scan diagnosis in distinguishing between stroke sub-types. Studies that do not report on sensitivity and specificity but have sufficient information to calculate these will also be considered for inclusion. Types of studies The review will consider studies of diagnostic test accuracy in which the index and reference tests are interpreted independently of one another on the same group of participants. Search strategy The search strategy aims to find both published and unpublished studies in English within the period 1983-2012 A three step strategy will be implemented. First, we will conduct an initial search of PubMed and EMBASE with analysis of text words in the title and abstract and of index terms used to describe the article. A second search with all identified keywords and index terms across all included databases will then be done. Finally, the reference list of all included reports and articles will be searched for additional studies. Databases to be searched include: PubMed, EMBASE, Cochrane central register of controlled trials (CENTRAL), Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Science. The search for unpublished studies will include: Virtual Health Library, System for Information for Grey Literature in Europe (SIGLE), Scientific Electronic Library Online (SciELO), MedNar and ProQuest. Initial keywords include: stroke, acute stroke, cerebrovascular accident, clinical stroke score, Siriraj stroke score, Guy s hospital stroke score, Allen score, Besson stroke score, Greek stroke score. 4
Assessment of methodological quality Papers selected for review will be assessed by two independent reviewers for methodological validity prior to inclusion in the review using the QUality Assessment of Diagnostic Accuracy Studies (QUADAS) criteria 15 (Appendix I). Three other reviewers will randomly sample five or more included papers and assess them. Disagreements will be resolved through discussion. Data collection A modified Joanna Briggs Institute (JBI) data extraction form (Appendix II) will be used to collect details from included studies. Two authors will independently extract data from each study. Disagreements will be resolved through discussion. For each study, the following data will be obtained: author information, year of publication, study site, setting, study design, number and characteristics of patients (age, sex, ethnicity), reference standard, index test(s), information on clinicians who administered the scores and clinicians who interpreted the reference standard i.e. background specialty. Sensitivity and specificity, number of patients with equivocal scores, true positive (TP), false positive (FP), true negative (TN) and false negative (FN) data for each test will be taken directly from source papers. If this is not possible, they will be calculated from provided data. Extracted data with consensus from both authors will then be entered in a separate form and transferred to a spreadsheet. Data synthesis We aim to generate 2 X 2 contingency tables with true positive (TP), false positive (FP), true negative (TN) and false negative (FN) cases. We will consider patients with ischemic stroke as false positives or true negatives when analysing the performance for detecting hemorrhagic stroke and count patients with hemorrhagic stroke as false positives or true negatives when analysing performance for detection of ischemic stroke. We will calculate sensitivity and specificity with 95% confidence intervals for each clinical score for each study as well as predictive values and likelihood ratios. Meta-analysis will be undertaken using a bivariate mixed effects binomial regression model as described by Harbord et al. 16 Summary estimates for sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios (LR) as well as diagnostic odds ratio, will be generated. Where meta-analysis is not possible, a narrative synthesis will be undertaken. Heterogeneity will be assessed graphically using forest plots. For statistical heterogeneity we will use the quantity I 2 which describes the percentage of total variation across studies that is attributable to heterogeneity rather than chance. A value of 50% and above will be considered substantial heterogeneity. Analysis will be performed on STATA v.11 (Stata Corp, TX) using the midas 17 command and Review Manager Software version 5. 18 Conflicts of Interest None declared. 5
References 1. WHO. The world health report 2004-Changing history. The world health report 2004 (Statistical annex). 2004. 2. WHO. Stroke. Neurological disorders: public health challenges Geneva: WHO Press; 2006. 3. Boon NA, Colledge NR, Walker BR, Hunter JAA, editors. Davidson's Principles and Practice of Medicine. 20 ed. Philadelphia: Elsevier; 2006. 4. Broderick J, Connolly S, Feldmann E, Hanley D, Kase C, Krieger D, et al. Guidelines for the management of spontaneous intracerebral hemorrhage in adults: 2007 update: a guideline from the American Heart Association/American Stroke Association Stroke Council, High Blood Pressure Research Council, and the Quality of Care and Outcomes in Research Interdisciplinary Working Group. Circulation. 2007 Oct 16;116(16):e391-413. 5. Elliott J, Smith M. The acute management of intracerebral hemorrhage: a clinical review. Anesth Analg. 2010 May 1;110(5):1419-27. 6. Allen CM. Clinical diagnosis of the acute stroke syndrome. Q J Med. 1983 Autumn;52(208):515-23. 7. von Arbin M, Britton M, de Faire U, Helmers C, Miah K, Murray V. Accuracy of bedside diagnosis in stroke. Stroke. 1981 May-Jun;12(3):288-93. 8. The International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19435 patients with acute ischaemic stroke. International Stroke Trial Collaborative Group. Lancet. 1997 May 31;349(9065):1569-81. 9. Heller RF, Langhorne P, James E. Improving stroke outcome: the benefits of increasing availability of technology. Bull World Health Organ. 2000;78(11):1337-43. 10. Sandercock P, Molyneux A, Warlow C. Value of computed tomography in patients with stroke: Oxfordshire Community Stroke Project. Br Med J (Clin Res Ed). 1985 Jan 19;290(6463):193-7. 11. Wadhwani J, Hussain R, Raman PG. Nature of lesion in cerebrovascular stroke patients: clinical stroke score and computed tomography scan brain correlation. J Assoc Physicians India. 2002 Jun;50:777-81. 12. Besson G, Robert C, Hommel M, Perret J. Is it clinically possible to distinguish nonhemorrhagic infarct from hemorrhagic stroke? Stroke. 1995 Jul;26(7):1205-9. 13. Efstathiou SP, Tsioulos DI, Zacharos ID, Tsiakou AG, Mitromaras AG, Mastorantonakis SE, et al. A new classification tool for clinical differentiation between haemorrhagic and ischaemic stroke. J Intern Med. 2002 Aug;252(2):121-9. 14. Poungvarin N, Viriyavejakul A, Komontri C. Siriraj stroke score and validation study to distinguish supratentorial intracerebral haemorrhage from infarction. British Medical Journal. 1991;302(6792):1565-7. 6
15. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003 Nov 10;3:25. 16. Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JA. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007 Apr;8(2):239-51. 17. Dwamena BA. midas: A program for Meta-analytical Integration of Diagnostic Accuracy Studies in Stata. Division of Nuclear Medicine, Department of Radiology, University of Michigan Medical School, Ann Arbor, Michigan. 2007. 18. RevMan 5 download and installation.[cited; Available from: http://ims.cochrane. org/revman/download. 7
Appendix I: Critical Appraisal Tool QUADAS TOOL 15 Item Yes No Unclear 1 Was the spectrum of patients representative of the patients who will receive the test in practice? 2 Were selection criteria clearly described? 3 Is the reference standard likely to correctly classify the target condition? 4 Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the two tests? 5 Did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis? 6 Did patients receive the same reference standard regardless of the index test result? 7 Was the reference standard independent of the index test (i.e. the index test did not form part of the reference standard)? 8 Was the execution of the index test described in sufficient detail to permit replication of the test? 9 Was the execution of the reference standard described in sufficient detail to permit its replication? 10 Were the index test results interpreted without knowledge of the results of the reference standard? 11 Were the reference standard results interpreted without knowledge of the results of the index test? 12 Were the same clinical data available when test results were interpreted as would be available when the test is used in practice? 13 Were uninterpretable/ intermediate test results reported? 14 Were withdrawals from the study explained? 8
APPENDIX II: DATA EXTRACTION FORM (Adapted from JBI MAStARI data extraction tool) Record Number Author Year Journal Study site Reviewer Setting Study design Participants (age, gender, ethnicity) Number of participants Index test (IT)1 Index test 2 Index test 3 Background of clinician(s) administering index test(s) Reference standard (RS) Background of clinician(s) interpreting reference standard Index test 1 Sensitivity(95%CI) Sensitivity(95%CI) Specificity(95%CI) Specificity(95%CI) Number with equivocal scores 9
Index text 2 Sensitivity(95%CI) Sensitivity(95%CI) Specificity(95%CI) Specificity(95%CI) Number with equivocal scores Index test 3 Sensitivity (95%CI) (95%CI) Specificity (95%CI) (95%CI) Sensitivity Specificity Number with equivocal scores k 10