ORIGINAL ARTICLE. Association Between Trauma Quality Indicators and Outcomes for Injured Patients

Transcription

1 ONLINE FIRST ORIGINAL ARTICLE Association Between Trauma Quality Indicators and Outcomes for Injured Patients Laurent G. Glance, MD; Andrew W. Dick, PhD; Dana B. Mukamel, PhD; Turner M. Osler, MD Objective: To examine the association between the American College of Surgeons Committee on Trauma (ACSCOT) quality indicators and outcomes. Design: Cross-sectional study. Setting: Data from the Pennsylvania Trauma Outcome Study. Patients: We studied data from patients admitted to 35 trauma centers in Pennsylvania between 2000 and Main Outcome Measures: Regression analyses were performed to examine the association between ACSCOT quality indicators and in-hospital mortality and death or major complications. Results: Seven of the ACSCOT quality indicators were associated with either increased (1) in-hospital mortality or (2) death or major complications. No head computed tomography scan in patients with a Glasgow Coma Scale score less than 13 was associated with a 4-fold increase in mortality (adjusted odds ratio [AOR], 4.39; 95% confidence interval [CI], ) and a nearly 3-fold increased risk of death or major complications (AOR, 2.76; 95% CI ). Gunshot wounds to the abdomen managed nonoperatively were associated with a nearly 5-fold increase in mortality (AOR, 4.80; 95% CI, ). Femoral fractures treated with nonfixation were also strongly associated with mortality (AOR, 4.08; 95% CI, ) and death or major complications (AOR, 2.54; 95% CI, ). Conclusion: Several current ACSCOT quality indicators have a strong association with clinical outcomes. These findings should be interpreted with caution because some measures may lack face validity for identifying poorquality care in complex patients with multiple injuries. Arch Surg. 2012;147(4): Published online December 19, doi: /archsurg Author Affiliations: Departments of Anesthesiology and Community and Preventive Medicine, University of Rochester School of Medicine, Rochester, New York (Dr Glance); RAND Health, RAND, Santa Monica (Dr Dick), and Center for Health Policy Research, Department of Medicine, University of California, Irvine (Dr Mukamel); and Department of Surgery, University of Vermont Medical College, Burlington (Dr Osler). TRAUMATIC INJURIES ARE THE leading cause of death in patients younger than 45 years and are the fifth most common cause of mortality overall in the United States. 1 In 2000, 10% of hospital discharges were owing to injuries, and the direct cost of treating 50 million injury cases was $80.2 billion, with an estimated additional $326 billion in indirect costs. 2 In light of the substantial mortality, morbidity, and cost of caring for injured patients, improving the care of trauma patients is an important national priority. 3 CME available online at The release of the landmark Institute of Medicine report on medical errors 4 has resulted in public and private initiatives to improve health care quality through performance measurement and promoting adherence to evidence-based practices. 5,6 The wide variability in clinical outcomes across hospitals treating trauma patients, 7,8 even among designated level I trauma centers, 9 is evidence of large gaps in trauma care quality across hospitals. It is unknown to what extent this variability is caused by differences in clinical practice across trauma centers. More than 10 years before the release of the Institute of Medicine report, the American College of Surgeons Committee on Trauma (ACSCOT) created a set of quality indicators (audit filters), based on expert consensus, 10,11 to measure adherence to best practices and facilitate quality improvement. 12 In 2006, ACSCOT created the Trauma Quality Improvement Program (TQIP) to develop a national reporting and quality improvement See Invited Critique at end of article program for trauma patients. 3 As part of its mandate, TQIP will seek to identify and validate best practices in trauma care 13 and has proposed several candidate process measures for evaluation and inclusion in TQIP. 3 Because process measures directly measure clinical practice, they are actionable; 308

2 Table 1. ACSCOT Audit Filters Indicator Description Applicable Trauma Patient Population Ambulance scene 20 min All patients transported from injury scene Absence of ambulance report on medical record All patients transported from injury scene Admission GCS score 13 without a head CT scan ED GCS score 13 Absence of sequential neurological documentation on ED record in patient with skull fracture or intracranial injury or spinal cord injury ICD-9-CM codes: 800.xx, 801.xx, 803.xx, 804.xx, 850.xx, 851.xx, 852.xx, 853.xx, 854.xx, 806.xx, 952.0x, 952.1x, , 952.8, Absence of hourly documentation of vital signs in ED Patients admitted through ED No definitive airway in patient with GCS score 8 ED GCS score 8 and post-ed destination is not the morgue GSW to abdomen managed nonoperatively GSW to abdomen Patient requiring laparotomy that is performed 2 h after ED arrival Patients requiring exploratory laparotomy Epidural or subdural brain hematoma receiving craniotomy 4 h after ED arrival ICD-9-CM codes: 800.2x, 800.7x, 801.2x, 801.7x, 803.2x, 803.7x, 804.2x, 804.7x, and ICD-9-CM codes: , , 01.41, 01.42, , 01.6, 02.0x, 02.1x, , Patient transferred in after 3hatinitial hospital Transfer patients Interval 8 h between ED arrival and initial treatment of blunt open tibial fracture ICD-9-CM codes: , , , , , and ICD-9-CM codes: 78.0x, 79.xx Admitting physician is not a surgeon All patients Nonfixation of femoral diaphyseal fracture ICD-9-CM codes: , and ICD-9-CM codes: 78.15, 78.55, 79.05, 79.15, Required reintubation within 48 h of extubation Intubated patients Patient with any complications All patients Cervical spine injury or neurological deficit not addressed on admission Patients with discharge diagnosis of cervical spine fracture, subluxation, or neurological deficit Abbreviations: ACSCOT, American College of Surgeons Committee on Trauma; CT, computed tomography; ED, emergency department; GCS, Glasgow Coma Scale; GSW, gunshot wound; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification. poorly performing hospitals can focus on improving compliance with a specific measure (eg, timely administration of antibiotics). Process measures are also an effective mechanism for achieving rapid adoption of new evidence into clinical practice and promoting standardization. 14 However, process measures must first be validated to ensure that improved adherence to a recommended practice standard is associated with optimal patient outcomes. Recent negative findings from studies examining the results of 2 national patient safety projects, Leapfrog Safe Practices Scores 6 and the Surgical Care Improvement Project, 15 highlight the need to establish a strong evidence-based link between practice standards and clinical outcomes before adopting them as performance measures. In addition to the process measures proposed by the American College of Surgeons TQIP, recent comprehensive reviews of existing trauma quality indicators have also identified candidate process measures for evaluating trauma center quality. 16,17 Many of the proposed candidate measures are similar to existing ACSCOT audit filters. The 2 largest previous studies investigating trauma audit filters are based on data nearly 20 years old and may not reflect contemporary clinical practice. 18,19 As part of an Agency for Healthcare Research and Quality funded study to help develop the national infrastructure for trauma quality reporting, 7 the current study examined the association between existing quality indicators and clinical outcomes using a large population-based trauma outcomes registry. We used data from Pennsylvania trauma centers to examine whether ACSCOT audit filters are associated with mortality and morbidity. Data on the validity of these expert-based performance measures may help inform the development of future trauma performance measures. METHODS DATA SOURCE This study was based on data from the Pennsylvania Trauma Outcome Study (PTOS), obtained from the Pennsylvania Trauma Systems Foundation, on patients with traumatic injuries admitted to Pennsylvania trauma centers between 2000 and The Pennsylvania population, which makes up 4.2% of the US total population and includes rural and urban areas, provides a representative case mix of injured patients. 20 The PTOS database is a statewide trauma registry that includes data on all trauma admissions at accredited trauma centers meeting PTOS inclusion criteria: admission to the intensive care unit or stepdown unit, hospital length of stay greater than 48 hours, hospital admissions transferred from another hospital, and transfers to an accredited trauma center. 21 The PTOS database includes deidentified data on patient demographics, Abbreviated Injury Score codes and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes, mechanism of injury (based on ICD-9-CM E codes), comorbidities, physiology information, mechanisms of injury, in-hospital mortality and complications, transfer status, processes of care, and encrypted hospital identifiers. PATIENT POPULATION The study population consisted of trauma patients older than 16 years in the PTOS database, after excluding patients with burns, hypothermia, isolated hip fractures, superficial injuries, unspecified injuries, and nontraumatic mechanism of injury. From this initial cohort of patient observations, we excluded patients with missing information on transfer status (291), demographics (174); invalid Abbreviated Injury Score codes (12 732); and patients transferred out to another hospital (4712). The final study cohort consisted of pa- 309

3 Table 2. Unadjusted Outcomes in Patients ged With Quality Indicator Indicator Description No Audit Filter Missing % Eligible Patients, No. (%) Mortality, % No Death or Major Complication, % Ambulance scene 20 min (92.6) a Absence of ambulance report on medical record (75.8) a Admission GCS score 13 without a head CT scan (12.4) Absence of sequential neurological documentation on (42.4) ED record in patient with skull fracture or intracranial injury or spinal cord injury Absence of hourly documentation of vital signs in ED (96.4) No definitive airway in patient with GCS score (12.2) a GSW to abdomen managed nonoperatively (1.99) Laparotomy performed 2 h after ED arrival (5.48) Epidural or subdural brain hematoma receiving (2.53) craniotomy 4 h after arrival Patient transferred in after 3hatinitial hospital (27.3) Interval 8 h between ED arrival and initial treatment of (2.06) blunt open tibial fracture Admitting physician is not a surgeon (100) Nonfixation of femoral fracture (2.53) Required reintubation within 48 h of extubation (19.6) Patient with any complications (100) NA Cervical spine injury or neurological deficit not addressed on admission (98.8) Abbreviations: CT, computed tomography; ED, emergency department; GCS, Glasgow Coma Scale; GSW, gunshot wound; NA, not applicable. a P value.05; all other P values are.05 No tients in 35 trauma centers. This study was approved by the institutional review board at the University of Rochester School of Medicine. ANALYSIS The goal of this study was to estimate the association between the ACSCOT audit filters (Table 1) and (1) in-hospital mortality and (2) death or major complication. The unit of analysis was the patient. Initial exploratory analyses were conducted using univariate logistic regression. In the first set of analyses, separate logistic regression models were estimated to test the independent effect of each of the ACSCOT audit filters on in-hospital mortality. We used the previously validated Trauma Mortality Probability Model (TMPM-AIS) 22 modified by the addition of age, sex, comorbidities, mechanism of injury (based on E codes), transfer status, the Glasgow Coma Scale (GCS) motor component score, systolic blood pressure, and year of admission to control for confounding. Backward stepwise selection was used to determine which of the comorbidities to include. Fractional polynomial analysis was used to determine the optimal specification for continuous predictor variables. 23 The STATA implementation of the MICE method of multiple imputation described by van Buuren et al 24 was used to impute missing values of the motor component of the GCS and systolic blood pressure. Model parameters estimated in the 5 imputed data sets were combined using Rubin s rule. 25 Each analysis was based on the applicable trauma patient population eligible for a particular audit filter (Table 2). In the second set of analyses, we tested the independent association between each of the ACSCOT audit filters and death or major complications. We defined this composite complication outcome if any of the following occurred after hospital admission: death, acute respiratory distress syndrome, acute myocardial infarction, acute respiratory failure requiring more than 48 hours of ventilatory support, aspiration pneumonia, pneumonia, pulmonary embolism, fat embolism syndrome, acute renal failure, central nervous system infection, progression of original neurological insult, liver failure, sepsis, septicemia, empyema, dehiscence, gastrointestinal bleeding, small-bowel obstruction, compartment syndrome, arterial occlusion, and postoperative hemorrhage. We controlled for age, sex, injury severity, mechanism of injury, the motor component score of the GCS, systolic blood pressure, comorbidities, transfer status, and year of admission, using the modified TMPM-AIS, as described earlier. Data management and statistical analyses were performed using STATA SE/MP version 11.0 (StataCorp). Robust variance estimators were used to account for the nonindependence of observations within hospitals. All statistical tests were 2-tailed and P values less than.05 were considered significant. The performance of the logistic regression models (for mortality) was assessed using measures of discrimination (C statistic) and calibration (calibration curves). RESULTS BASELINE PATIENT CHARACTERISTICS AND CLINICAL OUTCOMES The study sample consisted of patients admitted to 35 trauma centers in Pennsylvania between 2000 and Population characteristics are displayed in Table 3. The median age of patients in the study sample was 47 years and 63% were male. Twenty-seven percent of the patients were transferred from another hospital. The 3 leading causes of injury were blunt trauma (41.4%), 310

4 motor vehicle collisions (26.4%), and low falls (13.7%). The overall mortality rate for this population was 6.3% and the rate of major complications was 7.2%. The median length of stay was 4 days. Gunshot wounds were associated with the highest overall mortality rates (24%) and the highest rate of death or major complications (34%). Patients with GCS motor component scores between 1 and 4 had mortality rates ranging from 22% to 46%. Patients with significant hypotension (systolic blood pressure 90 mm Hg) had mortality rates between 21% and 95%. AUDIT FILTER OUTCOME ASSOCIATION The number of patients eligible for each of the audit filters and proportion of patients flagged for nonadherence are listed in Table 2. The rate of missing data exceeded 10% for 5 of the audit filters, primarily because of missing time elements. The results of the analyses for these audit filters are presented in the Tables but will not be discussed further. Unadjusted and adjusted outcomes for patients flagged by the ACSCOT audit filters compared with unflagged patients are displayed in Table 2 and Table 4. The statistical performance of the mortality and major complication or death models used to adjust for confounding were examined. Both models exhibited excellent calibration, as shown in Figure 1 and Figure 2. Both the TMPM mortality model and the TMPM death or major complication model had excellent discrimination, with C statistics of 0.96 and 0.89, respectively. Process of Care Filters Twenty percent of patients with an admission GCS score less than 13 did not receive a head computed tomography (CT) scan. This filter was associated with large increases in mortality (adjusted odds ratio [AOR], 4.39; 95% confidence interval [CI], ) and death or major complications (AOR, 2.76; 95% CI, ). Of those patients admitted with a gunshot wound to the abdomen, 8% were flagged because they were managed nonoperatively. These patients had a nearly 5-fold increase in their risk of mortality (AOR, 4.80; 95% CI, ). Fifteen percent of trauma patients with femoral diaphyseal fractures were treated with nonfixation. These patients had a 4-fold increased risk of mortality (AOR, 4.08; 95% CI, ) and greater than 2-fold greater risk of death or major complication (AOR, 2.54; 95% CI, ). Less than 1% of patients with an emergency department (ED) discharge GCS score of 8 or less left the ED without a definitive airway. ged patients were at higher risk of death (AOR, 1.33; 95% CI, ) but were less likely to experience the composite outcome of death or major complication (AOR, 0.63; 95% CI, ). Only 6% of trauma patients were admitted by nonsurgeons. This audit filter was an independent predictor of a lower risk of death or major complication (AOR, 0.75; 95% CI, ). Table 3. Patient Characteristics No. (%) of Patients Delay in Care Filter Death, % The percentage of patients requiring a laparotomy that was not performed within 2 hours of ED arrival was 19%. Delayed laparotomy was associated with reduced risk of mortality (AOR, 0.74; 95% CI, ) but a higher risk of death or major complication (AOR, 1.42; 95% CI, ). Documentation Filters Death or Major Complications, % Demographics Age, y, median (IQR) 47 (29, 70) NA NA Sex M (62.9) F (37.1) Transfer from other hospital Transferred (27.3) Not transferred (72.7) GCS motor component score (7.90) (0.25) (0.36) (1.23) (2.77) (80.6) Missing (6.91) Systolic BP, mm Hg (1.54) (0.32) (2.66) (75.4) (17.4) (0.62) Missing 4458 (2.11) Mechanism of trauma Blunt (41.4) Motor vehicle collision (26.4) GSW (5.99) Stab 7994 (3.79) Pedestrian trauma (8.74) Low fall (13.7) Mortality 6.33 Major complication 7.20 Length of stay, d, median (IQR) 4 (2, 7) Abbreviations: BP, blood pressure; GCS, Glasgow Coma Scale; GSW, gunshot wound; IQR, interquartile range; NA, not applicable. Nearly 40% of patients transported to the hospital from the hospital scene by ambulance or helicopter were missing an ambulance record in the medical record. Patients flagged with missing ambulance records had a lower in-hospital mortality rate (AOR, 0.79; 95% CI, ). Hourly documentation of vital signs in the ED was absent in almost 4% of the patients. Missing ED documentation was associated with lower mortality (AOR, 0.75; 95% CI, ) and a lower rate of death or major complications (AOR, 0.76; 95% CI, ). 311

5 Table 4. Adjusted Outcomes in Patients ged With ACSCOT Audit Filter Mortality Death or Major Complication Indicator Description AOR (95% CI) P Value AOR (95% CI) Ambulance scene 20 min 0.97 ( ) ( ).001 Absence of ambulance report on medical record 0.79 ( ) ( ).49 Admission GCS score 13 without a head CT scan 4.39 ( ) ( ).001 Absence of sequential neurological documentation on ED 0.81 ( ) ( ).13 record in patient wi th skull fracture or intracranial injury Absence of hourly documentation of vital signs in ED 0.75 ( ) ( ).001 No definitive airway in patient with GCS score ( ) ( ).003 GSW to abdomen managed nonoperatively 4.80 ( ) ( ).59 Laparotomy performed 2 h after ED arrival 0.74 ( ) ( ).001 Epidural or subdural brain hematoma receiving craniotomy 0.46 ( ) ( ) h after ED arrival Patient transferred in after 3hatinitial hospital 0.90 ( ) ( ).79 Interval 8 h between ED arrival and initial treatment of 0.77 ( ) ( ).007 blunt open tibial fracture Admitting physician is not a surgeon 0.88 ( ) ( ).001 Nonfixation of femoral fracture 4.08 ( ) ( ).001 Required reintubation within 48 h of extubation 0.45 ( ) ( ).001 Patient with any complication 1.18 ( ).07 NA Cervical spine injury not addressed on admission 1.30 ( ) ( ).001 Abbreviations: ACSCOT, American College of Surgeons Committee on Trauma; AOR, adjusted odds ratio; CI, confidence interval; CT, computed tomography; ED, emergency department; GCS, Glasgow Coma Scale; GSW, gunshot wound; NA, not applicable. P Value Complication Filters Approximately 7% of patients experienced a major complication. There was a nonsignificant trend toward a higher mortality rate in this patient group (AOR, 1.18; 95% CI, ). Patients who were flagged because they required reintubation within 48 hours of extubation (8%) were less likely to die (AOR, 0.45; 95% CI, ) but experienced a nearly 5-fold increase in the risk of the composite outcome of death or major complication (AOR, 4.46; 95% CI, ). Missed Diagnosis Filter Less than 1% of patients were flagged because of a discharge diagnosis of cervical spine injury that was not diagnosed on admission. These patients had a 2-fold higher risk of death or major complications (AOR, 1.96; 95% CI, ). COMMENT In this large population-based observational study of the association between the ACSCOT audit filters and clinical outcomes, we find evidence that 6 of the audit filters are predictive of increased mortality or the composite outcome death or major complications. For some of the filters, the clinical impact appears to be very strong. For example, trauma patients with an admission GCS score less than 13 who did not receive a head CT scan had a 4-fold increased risk of mortality and a nearly 3-fold higher risk of death or major complications. Similarly, patients admitted with a gunshot wound to the abdomen who were managed nonoperatively experienced a 5-fold higher odds of mortality compared with those undergoing surgery. In comparison, there is no evidence that incomplete documentation is associated with worse outcomes. The ACSCOT audit filters, first introduced in 1987 and then subsequently revised in 1990 and 1993, are based on expert consensus. 11,18 In comparison with many of the process measures developed by the American College of Cardiology, trauma process measures are not based on data from large multicenter randomized controlled trials. 16 Process measures can be an effective means of promoting best practices 26 and improving quality 27,28 and have been adopted by several large national initiatives to improve quality of care across a wide spectrum of medical and surgical conditions. 15,28 However, in some cases, the absence of a strong evidence base linking specific processes of care and clinical outcomes limits the value of performance measures as a tool to improve the quality of health care. 15,29,30 Viewed in this light, ongoing efforts by the ACSCOT to evaluate process measures for inclusion in TQIP may benefit from the findings of our exploratory analysis examining the association between existing ACSCOT trauma audit filters and outcome. COMPARABLE STUDIES To our knowledge, there are only 2 other large-scale observational studies that have investigated the link between the ACSCOT audit filters and outcomes. 18,19 Both of these studies were conducted more than 15 years ago, and one did not adjust for differences in case mix. 18 The other study, by Copes and colleagues, 19 was also based on the PTOS database and used a case-control design to control for case mix differences. However, 80% of the au- 312

6 dit filters examined by Copes et al did not have matches for 20% or more of the patients flagged by the audit filter. Ignoring these observations in the case-control study may have introduced significant bias. 31 The lack of risk adjustment in one study, and the large amount of missing matches in the second, makes it difficult to compare our study findings with those of these prior studies. In addition, the period separating our study from prior studies limits the utility of such a comparison. ADDITIONAL CONSIDERATIONS Observed Mortality Rate, % Predicted Mortality Rate, % Figure 1. Calibration curve for the modified TMPM mortality model. Vertical bars represent 95% confidence intervals. Observed Death or Complication Rate, % Predicted Death or Complication Rate, % Figure 2. Calibration curve for the modified TMPM death or major complications model. Vertical bars represent 95% confidence intervals. Process measures can be used to improve quality and clinical outcomes by promoting adherence to best practices. 26 Unlike outcome measures, which can identify a quality problem but not its root cause, process measures are directly actionable because they quantify adherence to a practice guideline based on best practices. 32 However, the Achilles heel of process measures is the lack of strong evidence supporting large areas of clinical practice. 32 Even in the field of cardiology, which has a very strong evidence base grounded in many large experimental trials, nearly 50% of the practice guidelines developed by the American College of Cardiology are based on the lowest quality of evidence. 33 The increasing pressure to create process measures for hospitals and physicians and link these to financial incentives through pay for performance and Centers for Medicare & Medicaid Services value-based purchasing 34 may have important unintended consequences. 35 Recent research findings on the effectiveness of measures designed to reduce the incidence of surgical infections, 15 control blood glucose levels, and improve perioperative cardiac outcomes through the use of -blockers 36 highlight the potential downside of creating performance measures based on incomplete evidence. The challenge facing the ACSCOT in creating process measures for TQIP is to move beyond expert consensus. To do so, the National Trauma Databank should be expanded to include more information on processes of care to facilitate comparative effectiveness research. Beyond this, trauma surgeons may need to consider testing the efficacy of best practices, based on the initial results of comparative effectiveness research, using large multicenter randomized controlled trials. Results of these trials may help provide the evidence base for creating the next generation of ACSCOT quality measures. The implications of our findings with respect to the future development of process measures for trauma care are not straightforward. Although some of the American College of Surgeons audit filters are strongly associated with increased mortality, these same audit filters may not be suitable as performance measures. For example, despite the finding that there is a strong association between mortality and no head CT scan, it is likely that in many cases CT scanning was not performed because other aspects of care were assigned a higher clinical priority. In particular, patients requiring emergent laparotomy to control hemorrhage may not have time for a head CT scan on admission. Nonfixation of femur fractures may also reflect the need to prioritize clinical care in complex patients with multiple injuries. These issues will need to be addressed by TQIP as they evaluate several proposed measures such as the use of intracranial pressure monitoring in patients with severe traumatic brain injury, time to operative fixation in patients with long bone fractures, and time to hemorrhage control. 37 Failure to carry out specific best practices in individual patients may reflect appropriate clinical decision making in complex patients with multiple injuries as opposed to substandard care. LIMITATIONS This study has several limitations. First, this study is based on data from Pennsylvania and may not be generalizable to the rest of the United States. Alternative data sources such as the National Trauma Databank and the Healthcare Cost and Utilization Project Nationwide Inpatient Sample, however, do not include the necessary patient-level information on the ACSCOT audit filters. Second, this study, like other observational studies, is potentially biased owing to unmeasured confounders. For example, the findings that patients with an abnormal GCS score who do not receive a head CT scan are more likely to die may reflect confounding by indication; patients with abnormal neurological examination findings who did not undergo CT scanning may have been too unstable to spend time in the radiology suite as opposed to dying because they did not receive indicated care. It is unlikely that a 313

7 propensity-based analysis would eliminate this potential source of bias, since propensity scoring can only balance groups using available data and cannot adjust for unmeasured risk factors. Although it is impossible to rule out bias as a threat to the internal validity of this study, the excellent statistical performance of our risk adjustment model reduces the likelihood of this type of bias. Third, we did not explore the association between the ACSCOT audit filters and functional outcomes. As captured by the Institute of Medicine definition of quality, the goal of health care is to increase the likelihood of desired health outcomes. 38 Future work to validate revised trauma performance measures should include functional outcomes as one of the quality domains. Fourth, the ACSCOT audit filters were first designed as quality indicators to identify patients for peer review, as opposed to performance measures to use for hospital benchmarking. However, as is the case with the Agency for Healthcare Research and Quality Patient Safety Indicators, quality indicators originally intended to facilitate peer review are now routinely used for performance measurement. 39 Thus, from a practical standpoint, assessing the validity of the ACSCOT audit filters as performance measures is reasonable. Finally, we examined the association between audit filters that were developed more than 20 years ago and whose relevance to modern trauma management may be challenged. However, some of the process measures currently under review by TQIP overlap with existing ACSCOT measures (eg, deep venous thrombosis prophylaxis and time to operative fixation for fractures). Since TQIP intends to incorporate process measures as part of its reporting system, our exploratory analysis to examine the validity of the ACSCOT audit filters using a contemporary data set should help facilitate the development of an updated set of trauma performance measures. The primary challenge to the construction of process measures for TQIP is the same today as it was 20 years ago: identifying best practices in trauma care using the best available evidence, which unfortunately remains quite limited. Despite its limitations, we believe that this study will help inform efforts by the American College of Surgeons to develop new performance measures for trauma care. The large variability in outcomes across specialized trauma centers challenges the trauma community to develop standardized treatment approaches, based on best practices, to improve the overall quality of trauma care in the United States. Although performance measurement is an integral part of efforts to improve trauma outcomes, it is imperative that we focus our measurement and standardization efforts on clinical practices that have been demonstrated to lead to better outcomes. The next version of ACSCOT process measures should be based on the best available evidence and should be carefully validated before accepting them as the basis for trauma center evaluation and quality improvement. Because of the complexity of trauma care, the goal of creating evidence-based and clinically valid process measures is likely to prove very challenging. Accepted for Publication: September 8, Published Online: December 19, doi: /archsurg Corrected on March 13, Correspondence: Laurent G. Glance, MD, University of Rochester Medical Center, 601 Elmwood Ave, Box 604, Rochester, NY (laurent_glance@urmc.rochester.edu). Author Contributions: Study concept and design: Glance, Dick, Mukamel, and Osler. Acquisition of data: Glance. Analysis and interpretation of data: Glance, Dick, Mukamel, and Osler. Drafting of the manuscript: Glance and Dick. Critical revision of the manuscript for important intellectual content: Glance, Dick, Mukamel, and Osler. Statistical analysis: Glance, Dick, Mukamel, and Osler. Obtained funding: Glance, Dick, and Mukamel. Administrative, technical, and material support: Glance. Financial Disclosure: None reported. Funding/Support: This project was supported by grant RO1 HS from the Agency for Healthcare and Quality Research. Additional Information: These data were provided by the Pennsylvania Trauma Systems Foundation, Mechanicsburg. The foundation specifically disclaims responsibility for any analyses, interpretations, or conclusions. RFERENCES 1. Table 122. Deaths and death rates by leading causes of death and age: Accessed April 1, Finkelstein EA, Corso PS, Miller TR. The Incidence and Economic Burden of Injuries in the United States. New York, NY: Oxford University Press; Hemmila MR, Nathens AB, Shafi S, et al. The Trauma Quality Improvement Program: pilot study and initial demonstration of feasibility. J Trauma. 2010;68 (2): Committee on Quality of Health Care in America; Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; Werner RM, Bradlow ET. Relationship between Medicare s hospital compare performance measures and mortality rates. JAMA. 2006;296(22): Kernisan LP, Lee SJ, Boscardin WJ, Landefeld CS, Dudley RA. Association between hospital-reported Leapfrog Safe Practices Scores and inpatient mortality. JAMA. 2009;301(13): Glance LG, Osler TM, Dick AW, Mukamel DB, Meredith W. The Survival Measurement and Reporting Trial for Trauma (SMARTT): background and study design. J Trauma. 2010;68(6): Glance LG, Dick AW, Meredith JW, Mukamel DB. Variation in hospital complication rates and failure-to-rescue for trauma patients. Ann Surg. 2011;253 (4): Shafi S, Friese R, Gentilello LM. Moving beyond personnel and process: a case for incorporating outcome measures in the trauma center designation process. Arch Surg. 2008;143(2): Evans C, Howes D, Pickett W, Dagnone L. Audit filters for improving processes of care and clinical outcomes in trauma systems. Cochrane Database Syst Rev. 2009;(4):CD Willis CD, Gabbe BJ, Cameron PA. Measuring quality in trauma care. Injury. 2007; 38(5): Committee on Trauma of the American College of Surgeons. Hospital and prehospital resources for optimal care of the injured patient. Bull Am Coll Surg. 1986; 71(10): Pasquale MD. Outcomes for trauma: is there an end (result) in sight? J Trauma. 2008;64(1): Lee TH. Eulogy for a quality measure. N Engl J Med. 2007;357(12): Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to Surgical Care Improvement Project measures and the association with postoperative infections. JAMA. 2010;303(24): Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. Quality indicators for evaluating trauma care: a scoping review. Arch Surg. 2010;145(3): Stelfox HT, Straus SE, Nathens A, Bobranska-Artiuch B. Evidence for quality indicators to evaluate adult trauma care: a systematic review. Crit Care Med. 2011; 39(4): Nayduch D, Moylan J, Snyder BL, Andrews L, Rutledge R, Cunningham P. American College of Surgeons trauma quality indicators: an analysis of out- 314

8 come in a statewide trauma registry. J Trauma. 1994;37(4): , discussion Copes WS, Staz CF, Konvolinka CW, Sacco WJ. American College of Surgeons audit filters: associations with patient outcome and resource utilization. J Trauma. 1995;38(3): Mohan D, Rosengart MR, Farris C, Cohen E, Angus DC, Barnato AE. Assessing the feasibility of the American College of Surgeons benchmarks for the triage of trauma patients. Arch Surg. 2011;146(7): Pennsylvania Trauma Systems Foundation annual report Accessed April 1, Osler T, Glance L, Buzas JS, Mukamel D, Wagner J, Dick A. A trauma mortality prediction model based on the anatomic injury scale. Ann Surg. 2008;247(6): Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: parsimonious parametric modeling. Appl Stat. 1994;43: doi: / van Buuren S, Boshuizen HC, Knook DL. Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med. 1999;18(6): Rubin D. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley & Sons; Chassin MRL, Loeb JM, Schmaltz SP, Wachter RM. Accountability measures: using measurement to promote quality improvement. N Engl J Med. 2010; 363(7): Peterson ED, Roe MT, Mulgund J, et al. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA. 2006;295(16): Werner RM,BradlowET.Publicreportingonhospitalprocessimprovementsislinked to better patient outcomes. Health Aff (Millwood). 2010;29(7): Ingraham AM, Cohen ME, Bilimoria KY, et al. Association of surgical care improvement project infection-related process measure compliance with riskadjusted outcomes: implications for quality measurement. J Am Coll Surg. 2010; 211(6): Hawn MT. Surgical care improvement: should performance measures have performance measures. JAMA. 2010;303(24): Newgard CD. The validity of using multiple imputation for missing out-ofhospital data in a state trauma registry. Acad Emerg Med. 2006;13(3): Glance LG, Neuman M, Martinez EA, Pauker KY, Dutton RP. Performance measurement at a tipping point. Anesth Analg. 2011;112(4): Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC Jr. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009;301(8): Roadmap for implementing value driven healthcare in the traditional Medicare Fee-for-Service Program. /downloads /VBPRoadmap_OEA_1-16_508.pdf. 35. Krumholz HM, Normand SL, Spertus JA, Shahian DM, Bradley EH. Measuring performance for treating heart attacks and heart failure: the case for outcomes measurement. Health Aff (Millwood). 2007;26(1): Devereaux PJ, Yang H, Yusuf S, et al; POISE Study Group. Effects of extendedrelease metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial. Lancet. 2008;371(9627): Shafi S, Nathens AB, Cryer HG, et al. The Trauma Quality Improvement Program of the American College of Surgeons Committee on Trauma. J Am Coll Surg. 2009;209(4): , e Committee on Quality of Health Care in America; Institute of Medicine. Crossing the Quality Chasm. Washington, DC: National Academy Press; Romano PS, Mull HJ, Rivard PE, et al. Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data. Health Serv Res. 2009;44(1): ONLINE FIRST Time to Turn the Page INVITED CRITIQUE Moving on to Write New Chapters for Trauma Care G lance et al1 have provided an excellent and provocative analysis of the relationship between process measures and outcomes for patients with traumatic injuries. Briefly summarized, their study analyzed data collected in a statewide trauma registry for patients hospitalized in trauma centers. The data covered the first decade of our new millennium ( ) and comprised more than patients, allowing for robust statistical analysis. Their aim was to examine the relationship between the current ACSCOT quality indicators vs outcomes (measured by death or major complications) of the patients. Their conclusion: many of these old process measures simply do not work to measure quality or have any relationship to the patient s outcomes. This is an important bit of science, as there is a substantial national push for developing quality measures that will then be both driven down to individual surgeons, for value-based payment, but also available for public reporting of quality at the hospital or surgeon level. Thus, developing a list of quality metrics that are valid and associated with detection of poor quality of care and outcomes is critically important. In the final analysis, patients, for the most part, really do not care what processes we do or do not do to them; they only care about how well they do under our care. Not surprising, the quality improvement world is moving away from process measures (what actions we take) toward outcomes measures (whether the patient did well) as a measure of quality. The current ASCOT quality indicators (and even some trauma center verification standards) are mostly process measures, developed from a consensus process. The strength of process measures lies in their use as tools to understand differences between trauma centers with good risk-adjusted outcomes and those with poor outcomes and to promote best-of-breed approaches to solve poor outcomes. In this respect, they are great to have around. However, good process tools do not necessarily make good quality measures. The weakness of using process measures as surrogates for quality is made obvious by this current analysis, clearly outlining many reasons why we need to refine and evolve our trauma quality metrics. While good and innovative in their day, these process audit filters and standards belong to the history of trauma care. It is now time for us to turn the page and go on to write new chapters. 315