A Comparison of Five Simplified Scales to the Out-of-hospital Glasgow Coma Scale for the Prediction of Traumatic Brain Injury Outcomes

A Comparison of Five Simplified Scales to the Out-of-hospital Glasgow Coma Scale for the Prediction of Traumatic Brain Injury Outcomes Michelle Gill, MD, Robert Steele, MD, Ryan Windemuth, MD, Steven M. Green, MD Abstract Background: The 15-point Glasgow Coma Scale (GCS) frequently is used in the initial evaluation of traumatic brain injury (TBI) in out-of-hospital settings. We hypothesized that the GCS might be unnecessarily complex for out-of-hospital use. Objectives: To assess whether a simpler scoring system might demonstrate similar accuracy in the prediction of TBI outcomes. Methods: We performed a retrospective analysis of a trauma registry consisting of patients evaluated at our Level 1 trauma center from 1990 to 2002. The ability of out-of-hospital GCS scores to predict four clinically relevant TBI outcomes (emergency intubation, neurosurgical intervention, brain injury, and mortality) by using areas under receiver operating characteristic curves (AUROCs) was calculated. The same analyses for five simplified scales were performed, and compared with the predictive accuracies of the total GCS score. Results: In this evaluation of 7,233 trauma patients over a 12-year period of time, the AUROCs for the total GCS score were 0.83 (95% confidence interval [CI] = 0.81 to 0.84) for emergency intubation, 0.86 (95% CI = 0.85 to 0.88) for neurosurgical intervention, 0.83 (95% CI = 0.82 to 0.84) for brain injury, and 0.89 (95% CI = 0.88 to 0.90) for mortality. The five simplified scales approached the performance of the total GCS score for all clinical outcomes. Conclusions: In the evaluation of injured patients, five simplified neurological scales approached the performance of the total GCS score for the prediction of four clinically relevant TBI outcomes. ACADEMIC EMERGENCY MEDICINE 2006; 13:968 973 ª 2006 by the Society for Academic Emergency Medicine Keywords: Glasgow Coma Score, traumatic brain injury, prehospital care, out-of-hospital care Despite advances in the assessment and management of traumatic brain injury (TBI) 1 8 and efforts aimed at its prevention, 9 13 TBI remains the number one cause of death and disability among children and young adults in the United States. 14 Patients with TBI require rapid assessment and resuscitation, followed by out-of-hospital personnel communication with the base From the Department of Emergency Medicine, Loma Linda University School of Medicine (MG, RS, RW, SMG), Loma Linda, CA. Dr. Windemuth is now at Inland Valley Medical Center, Wildomar, CA. Received September 4, 2005; revisions received April 16, 2006, April 26, 2006, and April 28, 2006; accepted May 1, 2006. Reprints not available. Address for correspondence: Michelle Gill, MD, Loma Linda University Medical Center, 11234 Anderson Street, P.O. Box 2000, Room A-108, Loma Linda, CA 92354. Fax: 909-558-0121; e-mail: mgill@ahs.llumc.edu. station regarding management and transportation. One such assessment, originally developed for serial use in the ICU setting, is the three-parameter scoring system known as the Glasgow Coma Scale (GCS). 15 This neurological scale has been adopted for broader use since its original introduction in 1974 and has become the standard tool for communication of mental status in both traumatized and nontraumatized patients; however, it was created neither for the purpose of performing a single neurological assessment of patients with TBI nor as a tool for triaging these patients. Recently, Gill and colleagues noted that the GCS exhibits only moderate interrater reliability when applied by two emergency physicians within 5 minutes of each other in the emergency department (ED) setting. 16 In another recent study, these same investigators demonstrated that simplified versions of the GCS motor and GCS verbal scales as assessed on ED arrival exhibited essentially equivalent test performance to the total GCS 968 ISSN 1069-6563 PII ISSN 1069-6563583 ª 2006 by the Society for Academic Emergency Medicine doi: 10.1197/j.aem.2006.05.019

ACAD EMERG MED September 2006, Vol. 13, No. 9 www.aemj.org 969 in predicting trauma outcomes. 17 These two studies suggest that this ubiquitous scoring system may be less reliable than is widely assumed and may be more complicated than is necessary for its intended purpose. The true utility for any simplified GCS scale would be in the out-of-hospital rather than ED setting, however, and so we hypothesized that the GCS is unnecessarily complex for the initial assessment of TBI in the out-ofhospital setting and that a simpler scoring system might demonstrate similar predictive accuracy. We compared the predictive value of the total GCS score provided by out-of-hospital personnel with the predictive value of its components (eye, verbal, and motor) alone in the identification of clinically relevant outcomes in TBI. We also compared the predictive value of the GCS with any GCS component simplifications suggested by our analyses. METHODS Study Design We performed a retrospective analysis of a trauma registry. The study was approved by our hospital s institutional review board. Study Setting and Population Our facility is the only Level 1 trauma center and children s hospital serving a region of approximately three million people, and it has an annual ED census of 64,000 patients. The trauma registry consisted of patients of all ages presenting to our trauma center from 1990 to 2002, who met standard trauma alert criteria as specified by the institution and in accordance with the American College of Surgeons. 18 Study Protocol When out-of-hospital personnel arrived to the ED with a trauma victim, they verbally reported clinical information including GCS components to one ED nurse who was assigned to document this information onto a standardized trauma record. Trauma registry staff later transcribed the clinical information into a computerized database and also recorded prespecified information on hospitalization course and outcome. Final diagnoses and all procedures are coded in the registry according to the International Classification of Diseases, Ninth Revision (ICD-9). 19 We excluded patients if the out-of-hospital GCS or its components (eye, verbal, or motor) were missing from the database. Outcome Measures We selected four outcomes a priori that we believed to be clinically relevant to clinicians caring for patients in the acute phase of TBI. The first was ED intubation, as recorded by the ED documentation nurse by using a yes no checkbox on the trauma record. The second outcome, neurosurgical intervention, was defined as the performance of the following selected procedures on the brain (as identified by ICD-9 code): cranial puncture; diagnostic procedures on skull, brain, or meninges; craniectomy or craniotomy; incision of brain or cerebral meninges; operations on the thalamus or globus pallidus; other excision or destruction of brain or meninges; excision of skull lesions; cranioplasty; repair of cerebral meninges; ventriculostomy; placement of extracranial ventricular shunt; revision, removal or irrigation of ventricular shunt; and other operations of the brain, skull, and cerebral meninges. Clinically significant brain injury was chosen as the third outcome and defined as one of the following diagnoses (as identified by ICD-9 code): skull fracture or basilar skull fracture with brain laceration, hemorrhage, or contusion; cerebral laceration or contusion; subarachnoid hemorrhage; subdural hemorrhage or extradural hemorrhage; and other unspecified intracerebral hemorrhage after injury. Skull fractures without intracranial injury, unspecified head injuries, and concussions were not considered clinically significant brain injuries for the purpose of this study. 6 The fourth outcome was mortality, as defined by death of the patient during the index hospitalization. Data Analysis We constructed receiver operating characteristic (ROC) curves and measured the areas under these curves (AUROCs) to compare the predictive value of the outof-hospital GCS score and its components against the four outcomes (intubation, neurosurgical intervention, clinically significant brain injury, and mortality). We also calculated the 95% confidence intervals (CIs) of the AUROCs for each outcome. It was our a priori intention to analyze the ROC curves of the GCS components to determine whether simplifications of these might retain a similar degree of predictive value. From these analyses we planned to generate simplified scales that then could be tested for accuracy in predicting the four chosen outcomes. All analyses were performed by using Stata 8 (Stata Corporation, College Station, TX). RESULTS During the study period of interest, 8,648 trauma alerts occurred, with the number of exclusions and available outcomes detailed in Figure 1. The study sample analyzed consisted of 7,233 cases, of which 5,156 (70%) were male, 2,163 were younger than age 18 years (30%), 5,070 were >18 years (70%), and median age was 24 years (interquartile range = 16 to 38 yr). Descriptive statistics for study outcomes are shown in Table 1. For comparison, the excluded study sample (n = 1,415) consisted of 1,019 male patients (72%), with 532 aged younger than 18 years (38%) and 883 aged 18 years or older (62%), and a median age of 22 years (interquartile range = 12 to 35 yr). Of these excluded patients, 12% underwent neurosurgery, 20% had brain injury, 30% were known to be intubated in the ED (missing intubation data: n = 3), and 13% died during the incident hospitalization (missing mortality data: n = 109). Receiver operating characteristic curves for the four outcomes are shown in Figure 2A-D. The 95% CIs of the verbal component of the GCS overlapped those of the total GCS in three of the outcomes, and the motor component of the GCS overlapped it in one outcome. The absolute differences between these areas were of low magnitude, all less than 10%, and most less than 5% (Table 2).

970 Gill et al. COMPARISON OF FIVE SIMPLE SCALES TO THE GLASGOW COMA SCALE Figure 1. Flow diagram of study subjects. GCS = Glasgow Coma Scale; ED = emergency department. We deferred further evaluation of the GCS eye scale, given that it demonstrated the weakest predictive value of the three components. However, we noted that both the GCS verbal and GCS motor scales had the principal portion of their ROC curves defined by the top two thresholds, and we created simplified three-item versions of these scales (Table 3). These two simplified scales, a Simplified Motor Scale (SMS) and a Simplified Verbal Scale, also demonstrated essentially the same predictive value as the total GCS. However, we noted that the 95% CIs of the SMS overlapped those of the total GCS for the hospital mortality outcome (Table 2). DISCUSSION The GCS often is used in the out-of-hospital setting to communicate the extent of TBI to other health care professionals and is one of the items that out-of-hospital personnel are required to learn to standardize assessment and outcome of TBI. 20 In this study, we found that outof-hospital GCS scores performed only marginally better than five simpler alternative scores for the assessment Table 1 Descriptive Statistics for Outcome Variables (N = 7,233), Including Prevalence Rates and Their 95% Confidence Intervals Outcomes Yes* No* ED intubation 1,853 (26%; 5,356 (74%; 95% CI = 24, 27%) 95% CI = 72, 76%) Traumatic 1,200 (17%; 6,033 (83%; brain injury 95% CI = 16, 18%) 95% CI = 81, 86%) Neurosurgical 638 (9%; 6,595 (91%; intervention 95% CI = 8, 10%) 95% CI = 89, 93%) Hospital 728 (10%; 6,432 (91%; mortality 95% CI = 9, 11%) 95% CI = 89, 93%) * Percentages calculated after excluding missing data. Missing Data 24 0 0 73 and prediction of TBI. Because the interrater reliability of the GCS in the ED setting is only moderate 16 and this study reveals that simpler scores approach the test performance of the GCS when predicting outcomes of TBI, we question the popularity and widespread acceptance of the GCS as the de facto system for neurological monitoring of out-of-hospital patients. 21 It appears that the popularity of the GCS is disproportionate to the evidence that supports its role as a tool in the out-of-hospital evaluation of TBI. 21 Our findings coincide with other recent reports suggesting that a simpler neurological scoring system may approach the same test performance as the entire GCS for out-of-hospital and ED care. 22,23 Meredith and colleagues discovered that a single threshold, that is, a GCS motor component score of less than 6 obtained in the out-of-hospital setting, allowed them to predict mortality with a sensitivity of 59% and specificity of 97% in 29,000 trauma patients. 22 Ross and colleagues reported that a GCS motor component score of less than 6 obtained by out-of-hospital personnel was nearly as accurate (sensitivity 91%, specificity 85%) as was a GCS total of less than 14 (sensitivity, 92%; specificity, 85%) when predicting severe head injury. 23 Emergency department based studies similarly have proposed the use of single components of the GCS as simpler alternatives to using the total GCS score in trauma patients. 24 26 Al-Salamah et al. found that the initial ED GCS motor score predicted survival (AUROC = 0.81) and admission to the intensive care unit (ICU) as well as the total GCS score (AUROC = 0.82) and Revised Trauma Score (AUROC = 0.83), revealing that the Revised Trauma Score, which contains the GCS as one of its components, did not show any advantage over the total GCS or GCS motor scores in its ability to predict these outcomes. 24 Healey et al. found that the motor component of the GCS performed as well when predicting mortality (AUROC = 0.873) as did the total GCS score (AUROC = 0.891) in more than 200,000 trauma patients. 25 Holmes et al. prospectively evaluated the GCS and its components in 2,043 children presenting to the ED with blunt head trauma and discovered that the performance of the individual components of the GCS was as good as the total GCS for predicting TBI on computed tomography, as well as TBI in need of acute intervention. 26 Contrary to other studies, 17,22 25 however, the Holmes et al. study revealed that the motor component of the GCS displayed worse test performance than the eye and verbal components in children, with the verbal component of the GCS performing the best. 26 Although the accuracy of the out-of-hospital GCS and its components has been tested previously, 22,23 to our knowledge there have been no out-of-hospital studies specifically testing their precision. Gill and colleagues recently have found that the SMS displays better interrater reliability (k = 0.70) than both the total GCS score (k = 0.32) and the GCS motor score (k = 0.54) when prospectively evaluated in ED patients. 27 We propose that the three-point SMS replace the GCS score in the outof-hospital setting, because it revealed test performance approaching that of the total GCS in this study and in our previous ED study 17 and because it displays improved precision over the GCS and its components in this recent ED study. 27

ACAD EMERG MED September 2006, Vol. 13, No. 9 www.aemj.org 971 Figure 2. (A) Emergency department intubation. (B) Neurosurgical intervention. (C) Traumatic brain injury. (D) Hospital mortality. The replacement of the GCS by the SMS apparently would demand reconstruction of additive scoring systems such as the Revised Trauma Score and Trauma and Injury Severity Score that incorporate the GCS as one of their arms. However, other studies have already found that the total GCS 28 and GCS motor scores 24 have as much predictive value as these additive scores when predicting mortality 24,28 and admission to intensive-care units. 24 Some may believe that substitution of the SMS for the GCS requires similar testing of its test performance in comparison to chosen additive scoring systems. We do not believe that this is mandatory, because it already has been argued that most additive scoring systems provide little value to clinical care of individual patients and are not necessarily clinically useful. 24,28 33 LIMITATIONS Although the data used for the conduct of this study were prospectively assembled, the study itself was retrospective in nature, and the data were not collected specifically Table 2 Areas Under the ROC Curve for the Total GCS, Its Three Components, and the Simplified Verbal and Motor Scales, with 95% Confidence Intervals Item ED Intubation Neurosurgical Intervention Traumatic Brain Injury Hospital Mortality Total GCS 0.83 (0.81, 0.84) 0.86 (0.85, 0.88) 0.83 (0.82, 0.84) 0.89 (0.88, 0.90) Eye alone 0.77 (0.76, 0.78) 0.83 (0.82, 0.85) 0.78 (0.77, 0.80) 0.85 (0.83, 0.86) Verbal alone 0.81 (0.80, 0.82) 0.85 (0.84, 0.86) 0.82 (0.81, 0.83) 0.87 (0.85, 0.88) Motor alone 0.79 (0.78, 0.80) 0.84 (0.82, 0.85) 0.79 (0.78, 0.81) 0.88 (0.87, 0.89) SVS 0.82 (0.79, 0.81) 0.84 (0.82, 0.85) 0.80 (0.77, 0.80) 0.84 (0.85, 0.88) SMS 0.79 (0.77, 0.80) 0.83 (0.81, 0.84) 0.79 (0.77, 0.80) 0.86 (0.86, 0.89) SVS = Simplified Verbal Scale; SMS = Simplified Motor Scale.

972 Gill et al. COMPARISON OF FIVE SIMPLE SCALES TO THE GLASGOW COMA SCALE Table 3 Simplifications of the GCS Verbal and Motor Scales Alternative Score Score GCS Equivalent Simplified Verbal Scale Oriented 2 Verbal = 5 Confused conversation 1 Verbal = 4 Inappropriate or less response 0 Verbal %3 Simplified Motor Scale Obeys commands 2 Motor = 6 Localizes pain 1 Motor = 5 Withdrawal to pain or less response 0 Motor %4 for the purpose of this study. Because of incomplete records, we were forced to exclude 12% of the potential subjects from the study. We have shown that these excluded subjects were similar in characteristics to included subjects; however, the exclusion of them may have affected our overall results. We noted that the 95% CIs of the total GCS and the verbal component of the GCS overlapped in three of the four outcomes and that both the motor component of the GCS and the SMS overlapped with the total GCS for the mortality outcome. This may indicate that some test performance could be a result of chance alone. However, despite these instances of overlapping confidence intervals, we believe that the small absolute differences in AUROCs observed (most less than 5%) are unlikely to have clinical significance. 17 Indeed, much or all of these slight differences in observed test performance may result from simple geometry rather than true clinical effect, given that scales with smaller numbers of elements naturally will oversimplify their curvature. This characteristic effectively truncates the convexity of each portion of the simpler scales curves and decreases the observed area, automatically rendering a smaller AUROC for the simplest scales. The out-of-hospital environment is a chaotic, fastpaced scene in which complex assessment tools such as the GCS are subject to error. This study reveals that simpler scoring systems may approach the test performance of the GCS when predicting outcomes from the out-ofhospital setting. Although we advocate for the use of a simpler alternative to the GCS, any simplified system requires vigorous testing before acceptance for clinical application. CONCLUSIONS In the evaluation of injured patients, five simplified neurological scales applied in the out-of-hospital setting approached the same test performance as the total GCS for the prediction of four clinically relevant TBI outcomes. References 1. Clifton GL, Miller ER, Choi SC, et al. Lack of effect of induction of hypothermia after acute brain injury. N Engl J Med. 2001; 344:556 63. 2. Narayan RK. Hypothermia for traumatic brain injury a good idea proved ineffective. N Engl J Med. 2001; 344:602 3. 3. Haydel MJ, Preston CA, Mills TJ, Luber S, Blaudeau E, DeBlieux PM. Indications for computed tomography in patients with minor head injury. N Engl J Med. 2000; 343:100 5. 4. Edwards P, Arango M, Balica L, et al. Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury outcomes at 6 months. Lancet. 2005; 365: 1957 9. 5. Roberts I, Yates D, Sandercock P, et al. Effect of intravenous corticosteroids on death within 14 days in 10,008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo-controlled trial. Lancet. 2004; 364:1321 8. 6. Stiell IG, Wells GA, Vandemheen K, et al. The Canadian CT Head Rule for patients with minor head injury. Lancet. 2001; 357:1391 6. 7. Hollander JE, Go S, Lowery DW, et al. Interrater reliability of criteria used in assessing blunt head injury patients for intracranial injuries. Acad Emerg Med. 2003; 10:830 5. 8. Davis DP, Peay J, Serrano J, et al. The impact of aeromedical response to patients with moderate to severe traumatic brain injury. Ann Emerg Med. 2005; 46: 115 22. 9. American Academy of Pediatrics Committee on Injury and Poison Prevention. Skateboard and scooter injuries. Pediatrics. 2002; 109:542 3. 10. American Academy of Pediatrics Committee on Injury and Poison Prevention. Bicycle helmets. Pediatrics. 2001; 108:1030 2. 11. Pilkington P, Kinra S. Effectiveness of speed cameras in preventing road traffic collisions and related casualties: systematic review. BMJ. 2005; 330:331 4. 12. Mace SE, Gerardi MJ, Dietrich AM, et al. Injury prevention and control in children. Ann Emerg Med. 2001; 38:405 14. 13. Hackney RG. ABC of sports medicine. Nature, prevention, and management of injury in sport. BMJ. 1994; 308:1356 9. 14. Centers for Disease Control and Prevention. Traumatic Brain Injury in the United States: A Report to Congress. Available at: http://www.cdc.gov/ncipc/tbi/ tbi_congress/02_intro_servprog89 98.htm. Accessed May 1, 2006. 15. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974; 2:81 4. 16. Gill MR, Reiley DG, Green SM. Interrater reliability of Glasgow Coma Scale scores in the emergency department. Ann Emerg Med. 2004; 43:215 23. 17. Gill M, Windemuth R, Steele R, Green SM. A comparison of the Glasgow Coma Scale score to simplified alternative scores for the prediction of traumatic brain injury outcomes. Ann Emerg Med. 2005; 45:37 42. 18. American College of Surgeons Committee on Trauma. Resources for Optimal Care of the Injured Patient. Chicago, IL: American College of Surgeons, 1999. 19. World Health Organization. International Classification of Diseases, Ninth Revision (ICD-9). Geneva, Switzerland: World Health Organization, 1977.

ACAD EMERG MED September 2006, Vol. 13, No. 9 www.aemj.org 973 20. Watts DD, Hanfling D, Waller MA, Gilmore C, Fakhry SM, Trask AL. An evaluation of the use of guidelines in prehospital management of brain injury. Prehosp Emerg Care. 2004; 8:254 61. 21. Brain Trauma Foundation. Guidelines for Prehospital Management of Traumatic Brain Injury. Available at http://www2.braintrauma.org/guidelines/index.php. Accessed Apr 13, 2006. 22. Meredith W, Rutledge R, Hansen AR, et al. Field triage of trauma patients based upon the ability to follow commands: a study in 29,573 patients. J Trauma. 1995; 38:129 35. 23. Ross SE, Leipold C, Terregino C, O Malley KF. Efficacy of the motor component of the Glasgow Coma Scale in trauma triage. J Trauma. 1998; 45:42 4. 24. Al-Salamah MA, McDowell I, Stiell IG, et al; OPALS Study Group. Initial emergency department trauma scores from the OPALS study: the case for the motor score in blunt trauma. Acad Emerg Med. 2004; 11: 834 42. 25. Healey C, Osler TM, Rogers FB, et al. Improving the Glasgow Coma Scale score: motor score alone is a better predictor. J Trauma. 2003; 54:671 8. 26. Holmes JF, Palchak MJ, MacFarlane T, Kuppermann N. Performance of the pediatric Glasgow coma scale in children with blunt head trauma. Acad Emerg Med. 2005; 12:814 9. 27. Gill M, Martens K, Lynch E, Salih A, Green SM. Interrater reliability of three simplified neurological scales applied to adults presenting to the emergency department with altered levels of consciousness. Ann Emerg Med. In press. 28. Grmec S, Gasparovic V. Comparison of APACHE II, MEES and Glasgow Coma Scale in patients with nontraumatic coma for prediction of mortality. Crit Care. 2001; 5:19 23. 29. Chesnut RM. Glasgow Coma Score versus severity systems in head trauma. Crit Care Med. 1998; 26: 10 1. 30. Plant JR, MacLeod DB, Kortbeek J. Limitations of the Prehospital Index in identifying patients in need of a major trauma center. Ann Emerg Med. 1995; 26: 133 7. 31. Baxt WG, Berry CC, Epperson MD, Scalzitti V. The failure of prehospital trauma prediction rules to classify trauma patients accurately. Ann Emerg Med. 1989; 18:1 8. 32. Mulholland SA, Gabbe BJ, Cameron P, Victorian State Trauma Outcomes Registry and Monitoring Group (VSTORM). Is paramedic judgment useful in prehospital trauma triage? Injury. 2005; 36: 1298 305. 33. Gabbe BJ, Cameron PA, Wolfe R. TRISS: does it get better than this? Acad Emerg Med. 2004; 11:181 6. Dynamic Emergency Medicine After field testing during the last few months, the editors of Academic Emergency Medicine are pleased to announce a new section of our journal, Dynamic Emergency Medicine. This section will present video articles that are relevant to the research, practice, and teaching of emergency medicine. We anticipate these will include instructional videos related to laboratory or clinical procedures or techniques; videos of cases where bedside imaging helped in management or diagnosis; interviews of emergency medicine leaders on concepts; history, or practice, or any other material best presented in video form. Videos of lectures or other didactic presentations will not be considered. Each submission must be accompanied by a brief written description of the video contents. High-quality still images will be published in the paper journal and will link to the video in the electronic journal. Videos should not exceed four minutes in length and will undergo peer review. Information on preferred formats is listed below; in all cases, the highest possible quality is required. Preferred format: MPEG -1 or -2 (.mpg extension) Also acceptable: Apple Quick Time (.mov), and Microsoft Audio/Video interfaced format (.avi) Please contact the AEM office by email with questions regarding the submission process.