Evaluation of the Military Acute Concussion Evaluation for Use in Combat Operations More Than 12 Hours After Injury

Transcription

1 MILITARY MEDICINE, 175, 7:477, 2010 Evaluation of the Military Acute Concussion Evaluation for Use in Combat Operations More Than 12 Hours After Injury COL Rodney L. Coldren, MC USA * ; Mark P. Kelly, PhD ; MAJ Robert V. Parish, MS USA ; CPT Michael Dretsch, MS USA ; LTC Michael L. Russell, MS USA ABSTRACT The diagnosis and management of concussion can be difficult in a combat environment, especially in the absence of loss of consciousness or post-traumatic amnesia. As no validated test exists to diagnose or grade neurocognitive impairment from a concussion, the military currently employs the Military Acute Concussion Evaluation (MACE) in Iraq. This is a two-part test, which incorporates the standardized assessment of concussion (SAC) as its objective score, although it has not been shown to be valid unless administered shortly after injury. A research team deployed to Iraq between January and April 2009 to examine the validity of several tests of neurocognitive function following a concussion, including the MACE. When administered more than 12 hours after the concussive injury, the MACE lacked sufficient sensitivity and specificity to be clinically useful. INTRODUCTION Estimates of the number of service members suffering from a mild traumatic brain injury (mtbi), hereafter referred to as a concussion, while deployed to Operation Iraqi Freedom/ Operation Enduring Freedom (OIF/OEF) vary widely, but some estimates suggest the number is as high as 320, Based on the definition adopted by several organizations, the DoD defines this condition as: A traumatically induced structural injury and/or physiological disruption of brain function as a result of an external force that is indicated by new onset or worsening of at least one of the following clinical signs, immediately following the event: any period of loss of or a decreased level of consciousness; any loss of memory for events immediately before or after the injury; any alteration in mental state at the time of injury (confusion, disorientation, slowed thinking, etc.); neurological deficits (weakness, loss of balance, change in vision, praxis, paresis/plegia, sensory loss, aphasia, etc.) that may or may not be transient; intracranial lesion 2 * Uniformed Services University of the Health Sciences, Department of Preventive Medicine and Biometrics, 4301 Jones Bridge Road, Bethesda, MD Walter Reed Army Medical Center, Department of Psychology, Building 6, Borden Pavilion, 6900 Georgia Avenue NW, Washington, DC U.S. Army Aeromedical Research Laboratory, Building 6901, Fort Rucker, AL AMEDD Neuropsychology Consultant, Lincoln Center, Suite 300, 7800 Interstate 10 West, San Antonio, TX The views expressed in this manuscript are those of the authors and does not necessarily reflect the position or policy of the United States Army Medical Research Acquisition Activity, the Walter Reed Army Medical Center, the Uniformed Services University of the Health Sciences, the United States Army Aeromedical Research Laboratory, the Army Medical Department, the Department of the Army, the Department of Defense, or the United States government and no official endorsement should be inferred. This definition encompasses a wide range of injury severity. In the presence of loss of consciousness (LOC) or posttraumatic amnesia (PTA), the diagnosis of concussion is straightforward. In contrast, there are questions of whether altered consciousness alone is a sufficient diagnostic criterion, 3 particularly in the chaotic context of combat or a blast injury, which can be disorienting even to a nonconcussed individual. Making return to duty or referral determinations in patients without LOC or PTA is one of the most challenging clinical dilemmas facing providers in the theater of operations, much as it is to athletic trainers and sports medicine physicians on the sidelines of athletic events. 3 Even in the setting of documented LOC or PTA, decisions about the necessity for referral to a higher level of care are not always easy. Further complicating the clinical decision-making process is a lack of available tests to assist the clinician. For an instrument to be a valid screening test for this application, it must have at a minimum sufficient sensitivity and specificity to accurately differentiate individuals who have suffered concussions from those who have not. There are currently no published studies validating any specific testing methodology to detect neurocognitive dysfunction secondary to a concussion incurred in the theater of operations. The test currently in use is the Military Acute Concussion Evaluation (MACE). The MACE is, essentially, a two-part test with an historical and an objective portion. The first section of the MACE gathers historical data about the nature of the concussive event and the signs and symptoms noted at the time of injury. The second examination portion of the MACE is the Standardized Assessment of Concussion (SAC). The SAC is a scorable test of orientation, immediate and delayed recall, and concentration. Patients are given a score of 0 to 30 on the basis of their performance, with this score commonly referred to as the MACE score in the military setting, a term we will use for the remainder of this article. Multiple forms of this assessment are used to decrease practice effect. Whereas MILITARY MEDICINE, Vol. 175, July

2 these various forms were designed to be comparable, there is evidence showing that individuals routinely score higher on some forms than on others. 6 The SAC is a screening test developed and validated for use only in the immediate aftermath of a concussive event, as most evidence has shown that concussion-related neurocognitive deficits usually resolve within days to a few weeks, 4,5 and was validated only in the context of a sports-related injury, not a combat- or blast-related injury. 3,7 12 Furthermore, most validation studies of the SAC made use of changes in score from baseline, not a one-time individual score to detect neurocognitive impairment from a concussion. 3,8,10,12 There is no standardized training or evaluation of administration competence on the MACE for medical providers in Iraq. Furthermore, service members do not routinely receive a baseline MACE, either in the continental U.S. (CONUS) or in theater, from which a change can be determined; rather guidelines are based on an absolute score of less than 24 being a criteria for referral. Despite these significant limitations, medical personnel and commanders are relying heavily on MACE scores hours to weeks after an injury event to determine diagnostic and treatment decisions. In fact, the MACE is currently being recommended, by MEDCOM policy, as a tool for return-to-duty (RTD) assessments during the medical evaluation board (MEB) process. 13 Given these considerations, we hypothesize that the MACE is not an appropriate tool in its current use to evaluate concussions after the immediate postinjury period. A team of researchers deployed to Iraq from January to April 2009 to study the validity of several neurocognitive screening tests, including the MACE, the Automated Neurocognitive Assessment Metrics for Traumatic Brian Injury 4 (ANAM-IV- TBI), and traditional neurocognitive testing in the evaluation of neurocognitive deficits secondary to concussive events. The team was composed of two senior neuropsychologists, a neuropsychology fellow, a research psychologist, an epidemiologist, and two noncommissioned officer technicians. Here we present the results of the evaluation of the MACE. The results of the evaluation of other testing modalities will be presented in future publications. METHODS Study Design During the period January 11, 2009 April 10, 2009, the study team deployed to Iraq to validate the ability of several tests to detect the presence of neurocognitive dysfunction secondary to concussion. Neurocognitive functioning of U.S. Army soldiers presenting for medical care within 72 hours of suffering a concussive event in Iraq was assessed by the MACE and traditional neurocognitive testing measures with the results being compared to those of the same measures administered to comparable controls. Subjects and controls were enrolled from Victory Base Complex, Joint Base Balad, and Mosul. Sample size calculations determined that 60 cases and a total of 120 controls would achieve a power of 0.90 to detect betweengroup and within-group effects at an α of This study was approved by the institutional review board of Brooke Army Medical Center. Subjects Cases All U.S. Army soldiers meeting eligibility criteria who presented for care to a medical facility within 72 hours of suffering a concussive event, whether combat-related or not, were offered enrollment. Eligible subjects had to be between 18 and 50 years of age, meet the DoD definition of a concussion, be free of psychoactive medication, have no significant psychiatric diagnosis requiring ongoing therapy, report pain not greater than 7 on a scale of 1 to 10, and give consent to be included in the study. Individuals with any history of severe TBI, of moderate TBI within the previous 3 years, or of any concussion within 90 days of current injury were excluded. Because of the impact of fatigue on traditional neurocognitive testing results, all cases were required to have had a full night s rest before administration of the tests to prevent fatigue-related neurocognitive deficits from confounding the results. Therefore, testing was administered more than 12 hours after the event in all cases. Controls Two control groups were enrolled. The first was the healthy control group: U.S. Army soldiers from OIF-deployed units that volunteered for participation. In addition to the inclusion/ exclusion criteria for the cases, these control subjects could not have suffered any concussive event during this deployment. The second group was acutely injured (within 72 hours) U.S. Army soldiers presenting for care who were neither head-injured nor exposed to a blast. This group was recruited to control for any negative effects of nonconcussive injury on neurocognitive function. Inclusion and exclusion criteria were the same as for healthy controls. Measures The neurocognitive tests administered include the MACE, the ANAM-IV-TBI, and a set of traditional neurocognitive tests, including the Test of Memory Malingering, which was administered to control for poor effort. As many soldiers may have memorized form A of the MACE and because even more may have had it recently administered, forms B and C were used in this study, with subjects being assigned to take one or the other form in an alternating fashion. Data Analysis Data were entered into a Microsoft Office Excel 2007 (Microsoft Corp., Redmond, WA) spreadsheet. Statistical analysis was performed using Stata v8.2 (Stata Corp., College Station, TX.) Group means were compared using the Student s t -test. Dichotomous and categorical data were compared using 478 MILITARY MEDICINE, Vol. 175, July 2010

3 the Pearson c 2 statistic. Results are expressed as p values. Receiver operating characteristic (ROC) curves were plotted to explore the appropriateness of this instrument as a screening test. RESULTS A total of 71 cases, 102 healthy controls, and 64 injured controls were enrolled. Data from two cases were excluded from the analysis because of evidence of poor effort. Sensitivity analysis demonstrated that their exclusion did not significantly alter the results. As both control groups were statistically indistinguishable in their performance on traditional neurocognitive testing and the MACE, they were combined into a single control group to increase analytical power. Analyses were conducted in aggregate and further separated into those with and those without either LOC or PTA. This subgroup analysis was necessary because of the difficulty in making the diagnosis in those individuals who had not experienced LOC or PTA. Thus, we examine the validity of the MACE in all individuals meeting the DoD diagnostic criteria for concussion and for those with more definitive evidence of concussion as evidenced by LOC or PTA. The 15 cases with indeterminate LOC and PTA histories were included in the aggregate statistics, but not in the subanalysis by concussion severity. Cases were moderately significantly more likely to be male ( p = 0.07) with 96% of cases vs. 88% of controls being male. Cases and controls were similar in age ( p = 0.44) with the mean age of cases 26.5 and controls 27.3 years old. Cases had slightly lower education levels ( p = 0.02) with cases having an average of 12.5 years education compared to 13.1 years for controls. Both cases and controls were predominately junior enlisted (E1 E4) with no cases being officers. There were small statistically significant, although not clinically significant, differences in the mean MACE scores between cases and controls ( p = 0.02) with cases having a mean MACE score of 26.0 and controls Cases with known LOC or PTA had significantly lower scores ( p < 0.01) with a mean score of There was no significant difference between MACE scores of cases without LOC or PTA and controls ( p = 0.63) with a mean score of There was no difference in mean MACE form B and MACE form C scores. Distribution of MACE scores is graphically presented in Figure 1. The distribution curves of MACE scores were far too similar for this test to be clinically useful to determine whether a subject could be said to be impaired secondary to a concussion. Figures 2 and 3 are the ROC curves for all cases vs. controls and cases with LOC/PTA vs. controls, respectively. The area under the curve for all cases vs. controls is and for those with LOC/PTA vs. controls is , both well below the generally acceptable value of 0.8. Sensitivity and specificity to differentiate concussed from nonconcussed soldiers for a range of MACE scores is presented in Table I. The results demonstrate that the MACE lacks sufficient sensitivity or specificity to differentiate cases from controls in aggregate FIGURE 1. FIGURE 2. controls. Distribution of MACE scores. ROC curve of MACE scores for all concussed subjects vs. FIGURE 3. ROC curve of MACE scores for concussed subjects with LOC/PTA vs. controls. or even when comparing only those cases with LOC or PTA to controls. DISCUSSION The results of this study are consistent with the published literature on the SAC, which shows rapid return to baseline MILITARY MEDICINE, Vol. 175, July

4 TABLE I. Sensitivity and Specificity of MACE Score in Differentiating all Cases From Controls Score % Sensitivity (all cases) % Sensitivity (LOC/PTA) % Specificity < < < < < performance.3,9,10 Although there were minor differences in mean MACE scores among the groups, with the more severely concussed subjects having lower scores, simple inspection of the data and ROC curve analysis demonstrate that this test lacks sensitivity and specificity when the test was administered more than 12 hours after the concussive event, a finding that again is consistent with previous studies. 14 It would seem that the use of the MACE score to determine or guide treatment and referral decisions more than 12 hours after the injury is not appropriate. A limitation of the study is the lack of baseline scores for comparison. Since most SAC validation studies are based on changes from baseline, access to baseline scores would have been useful in assessing validity of that MACE. However, service members do not receive baseline MACE administration. Baselining service members would raise several issues, such as whether baselines obtained in CONUS would be valid in theater and how to record baseline MACE scores in a manner that would be readily available and helpful to a combat medic. Because the requirement of a full night s rest before test administration, we cannot comment on the validity or utility of the MACE when administered closer to the time of injury, another significant limitation of the current study. The literature could potentially be extrapolated to support the use of the MACE as a tool to assist in return-to-duty or referral decisions at the time of injury. Another major limitation of this study is a lack of followup. We cannot correlate MACE scores from our study to final outcomes. Further study is required to determine whether concussed individuals with lower MACE scores have different ultimate outcomes from those with higher scores. The MACE is currently being used to determine return to duty vs. referral for further evaluation/rest in emergency departments and aid stations, hours to days after an injury event. Thus, soldiers are potentially being returned to duty in a vulnerable state 15 based on normal MACE scores or potentially inappropriately referred to higher levels of care based on abnormal MACE scores. Further study is required to determine whether and when the MACE is a valid tool for diagnosis of concussion in the combat environment. Until then, caution should be taken in utilizing MACE scores for diagnostic or treatment decisions. Instead, increased reliance on the patient s and witnesses accounts of the injury event is indicated. Theater healthcare providers should be trained to recognize, properly diagnose, and appropriately respond to minimal TBI using historical information. Performance on neurocognitive testing might then be used to further inform treatment and return-to-duty decisions. CONCLUSION There is no evidence supporting use of the examination section of the MACE as a tool to evaluate the presence of concussion if administered more than 12 hours after the injury, a use for which it was not intended. It is likely that because of the rapid natural recovery curve of concussions, the MACE score is not sensitive enough to detect an underlying injury distant in time from its occurrence. Further study is required to determine whether it is a valid test for use sooner after an injury in a combat setting. Until it is properly validated, the current results suggest that medical personnel and commanders should not utilize MACE scores per se to distinguish concussed from nonconcussed individuals. Nevertheless, missing the diagnosis of concussion is not acceptable in the case of a deployed soldier who continues to have undetected neurocognitive impairment and who remains vulnerable to more significant neurocognitive injury. Instead, increased reliance on the patient s and eyewitnesses accounts of the patient s injury/exposure and subsequent sequelae is recommended to make diagnostic, referral, and treatment decisions. Thus, at this juncture, the history section of the MACE might be significantly more useful to the combat medic or physician than the MACE score. ACKNOWLEDGMENTS The authors acknowledge the assistance of numerous individuals whose efforts were critical for the performance of this study. We thank SSG David Lopez and SGT Pedro Cruz for their diligent work on this study. REFERENCES 1. Tanielian T, Jaycox L : Invisible Wounds of War: Summary and Recommendations for Addressing Psychological and Cognitive Injuries. Contract no. MG-720/1-CCF. Santa Monica, CA, RAND Corporation, Department of Defense. Health Affairs Memorandum Traumatic Brain Injury: Definition and Reporting, Available at health.mil/about_mhs/ha_policies_guidelines.aspx?policyyear= 2007 ; accessed May 3, McCrea M : Standardized mental status testing on the sideline after sportrelated concussion. J Athl Train 2001 ; 36 (3) : Collins MW, Grindel SH, Lovell MR, et al : Relationship between concussion and neuropsychological performance in college football players. JAMA 1999 ; 282 (10) : McClincy MP, Lovell MR, Pardini J, Collins MW, Spore MK : Recovery from sports concussion in high school and collegiate athletes. Brain Inj 2006 ; 20 (1) : Valovich McLeod TC, Perrin DH, Guskiewicz KM, Shultz SJ, Diamond R, Gansneder BM : Serial administration of clinical concussion assessments 480 MILITARY MEDICINE, Vol. 175, July 2010

5 and learning effects in healthy young athletes. Clin J Sport Med 2004 ; 14 (5) : McCrea M, Kelly JP, Kluge J, Ackley B, Randolph C : Standardized assessment of concussion in football players. Neurology 1997 ; 48 (3) : Barr WB, McCrea M : Sensitivity and specificity of standardized neurocognitive testing immediately following sports concussion. J Int Neuropsychol Soc 2001 ; 7 (6) : McCrea M, Kelly JP, Randolph C, Cisler R, Berger L : Immediate neurocognitive effects of concussion. Neurosurgery 2002 ; 50 (5) : , discussion McCrea M, Guskiewicz KM, Marshall SW, et al : Acute effects and recovery time following concussion in collegiate football players: the NCAA Concussion Study. JAMA 2003 ; 290 (19) : McCrea M, Barr WB, Guskiewicz K, et al : Standard regression-based methods for measuring recovery after sport-related concussion. J Int Neuropsychol Soc 2005 ; 11 (1) : Daniel JC, Nassiri JD, Wilckens J, Land BC : The implementation and use of the standardized assessment of concussion at the U.S. Naval Academy. Mil Med 2002 ; 167 (10) : Office of the US Army Surgeon General. OTSG/MEDCOM Policy Memo : Optimal Use of Psychological/Neurological Assessment, Naunheim RS, Matero D, Fucetola R : Assessment of patients with mild concussion in the emergency department. J Head Trauma Rehabil 2008 ; 23 (2) : Giza CC, Hovda DA : The neurometabolic cascade of concussion. J Athl Train 2001 ; 36 (3) : MILITARY MEDICINE, Vol. 175, July