The Society of Thoracic Surgeons: 30-Day Operative Mortality and Morbidity Risk Models

Transcription

1 The Society of Thoracic Surgeons: 30-Day Operative Mortality and Morbidity Risk Models A. Laurie W. Shroyer, PhD, Laura P. Coombs, PhD, Eric D. Peterson, MD, Mary C. Eiken, MSN, Elizabeth R. DeLong, PhD, Anita Chen, MS, T. Bruce Ferguson, Jr, MD, Frederick L. Grover, MD, and Fred H. Edwards, MD Denver Department of Veterans Affairs Medical Center, and University of Colorado Health Sciences Center, Denver, Colorado, Duke Clinical Research Institute, Durham, North Carolina, The Society of Thoracic Surgeons, Chicago, Illinois, LSU Health Sciences Center, New Orleans, Louisiana, and University of Florida Health Sciences Center, Jacksonville, Florida Background. Although 30day risk-adjusted operative mortality (ROM) has been used for quality assessment, it is not sufficient to describe the outcomes after coronary artery bypass grafting (CABG) surgery. Risk-adjusted major morbidity may differentially impact quality of care (as complications occur more frequently than death) and enhance a surgical team s ability to assess their quality. This study identified the preoperative risk factors associated with several complications and a composite outcome (the presence of any major morbidity or 30-day operative mortality or both). Methods. For CABG procedures, the 1997 to 1999 Society of Thoracic Surgeons (STS) National Adult Cardiac Surgery Database was used to develop ROM and riskadjusted morbidity (ROMB) models. Risk factors were selected using standard STS univariate screening and multivariate logistic regression approaches. Risk model performance was assessed. Across STS participating sites, the association of observed-to-expected (O/E) ratios for ROM and ROMB was evaluated. The evaluation of patient outcomes has become increasingly accepted as a first step to assess and improve quality of patient care. As noted by Donabedian [1], quality may be judged based on improvements in patient status obtained, as compared to those changes reasonably anticipated based on the patient s severity of illness, presence of comorbidity, and the medical services received. Historically, the Society of Thoracic Surgeons (STS) National Adult Cardiac Surgery Database has reported riskadjusted 30-day operative mortality (based on either death in hospital or within 30 days of surgical procedure) for participating cardiac surgical member sites as a measure of quality of care. The goal of this project was to update and expand STS risk-adjusted 30-day operative mortality reports that have been used to compare relative surgical group performance to regional and national benchmarks. As part of a national continuous quality improvement program, new STS risk-adjusted morbidity reports are Presented at the Thirty-eighth Annual Meeting of The Society of Thoracic Surgeons, Fort Lauderdale, FL, Jan 28 30, Address reprint requests to Dr Shroyer, Denver Department of Veterans Affairs Medical Center, 1055 Clermont St (112R), Denver, CO 80220; laurie.shroyer@med.va.gov. Results. The 30-day operative death and major complication rates for STS CABG procedures were 3.05% and 13.40%, respectively (503,478 CABG procedures), including stroke (1.63%), renal failure (3.53%), reoperation (5.17%), prolonged ventilation (5.96%), and sternal infection (0.63%). Risk models were developed (c-indexes for stroke [0.72], renal failure [0.76], reoperation [0.64], prolonged ventilation [0.75], sternal infection [0.66], and the composite endpoint [0.71]). Only a slight correlation was found, however, between ROMB and ROM indicators. Conclusions. Used in combination, ROMB and ROM may provide the surgical team with additional information to evaluate the quality of their care as well as valuable insights to allow them to focus on areas for improvement. (Ann Thorac Surg 2003;75: ) 2003 by The Society of Thoracic Surgeons planned to provide additional clinically relevant and timely information to the cardiac surgical team for use in quality assessment and quality improvement activities. Although operative mortality is obviously the most deleterious clinical endpoint, limitations exist to using mortality alone to evaluate a surgical team s quality of care. Over the past decade, 30-day operative mortality rates for CABG-only procedures have declined significantly. In spite of a significant increase in the preoperative risks of the patients [2], this indicates a very significant improvement in the quality of cardiac surgical care rendered by STS members. It is clearly recognized that complications of cardiac surgery may not be fatal but can significantly impact a patient s functional status and quality of life. The ability of CABG surgery to improve overall health-related quality of life has been demonstrated and is considered a major indication for a CABG operation [3]. Therefore, the addition of risk-adjusted morbidity reports for CABG procedures may provide valuable insights on areas to focus improved quality of care. Similar to 30-day mortality, complications are influenced by preoperative patient characteristics that must be accounted for in any comparisons between surgical 2003 by The Society of Thoracic Surgeons /03/$30.00 Published by Elsevier Inc PII S (03)

2 Ann Thorac Surg SHROYER ET AL 2003;75: OPERATIVE RISK MODELS 1857 programs and benchmarks. Using the STS National Adult Cardiac Surgery Database, a primary objective of this study was to determine the frequency of major complications and to provide a new composite morbidity endpoint (based on the presence of either 30-day operative mortality or the presence of any major complication). The key preoperative clinical risk factors associated with a set of major morbidities were identified. Logistic regression risk models for each major morbidity endpoint (as well as the summary composite endpoint for mortality and morbidity) were developed and their performance assessed. Using regression analyses of the observed to expected ratios for 30-day operative death in comparison with this set of major morbidities (each assessed individually), the supplementary information provided by using these two different types of riskadjusted outcome measures together in combination was assessed across the STS participating member sites. Material and Methods Study Population Since its inception, the STS National Cardiac Database has grown to 1.5 million records by For data obtained during the study period from 1997 to 1999, there were 497 participating member sites (representing approximately 589 unique hospitals) submitting data (668,386 total records). Of these records, 505,645 records indicated that a CABG-only procedure was performed. From the set of records with CABG-only procedural designation, records with missing age (or out-of-range age) or missing sex were excluded from this analysis. After exclusion criteria were applied, the resultant population for this study s risk modeling analyses consisted of 503,478 CABG-only patient records from 495 participating sites throughout the United States. Selection of Major Morbidity Endpoints In collaboration with the Duke Clinical Research Institute (DCRI) National Database Warehouse activities, the STS National Database Risk Stratification Sub-Committee identified a subgroup of major complications (either life threatening or potentially resulting in permanent functional disability) that appeared to be most uniformly reported. Five STS major morbidity endpoints were selected: permanent stroke, renal dysfunction or renal failure requiring dialysis, any cardiac surgery reoperation, prolonged ventilation greater than 48 hours, and deep sternal wound infection. For clarification, the renal morbidity endpoint was defined as acute postoperative renal insufficiency resulting in one or more of the following: an increase of serum creatinine to more than 2.0; 50% or greater increase in creatinine over base line preoperative value; a new requirement for dialysis. Although several other STS morbidity outcomes might normally been considered as major (such as septicemia and perioperative myocardial infarction), a review of the data for these adverse events revealed too much variation in reporting, indicating a problem with definition or interpretation or uniformity of tests. Therefore, these other major morbidity endpoints were not included in this analysis. Rigorous data quality standards for the coding of these five major perioperative complications used for this analysis were previously established [4, 5]. Analytic Methods With the exception of body surface area (where a genderspecific median value was used), all missing or out-ofrange values were imputed using the variable-specific median value. Records were initially randomly assigned based on an approximate split into an 80/20 learning and test groupings to determine model test-retest reliability. For the final 1997 to 1999 CABG model, the learning population included 403,325 records and the testing population included 100,153 records. Among the 403,325 patients in the learning data set there were 12, day operative deaths and 54,156 cases where either a major morbidity or 30-day operative death occurred. In spite of the large sample size of the STS National Cardiac Database, there were no clinically important differences between characteristics of patients in the learning and testing subpopulations. Before proceeding with developing a multivariate analysis, univariate screening of all model-eligible risk factors was performed. Based on the univariate screening, 30 potential risk factors were identified. A multivariate stepwise logistic regression analysis was then performed for each of the seven dependent variables: (1) 30-day operative death; (2) permanent stroke; (3) renal dysfunction or renal failure requiring dialysis; (4) any reoperation; (5) prolonged ventilation; (6) deep sternal wound infection; and (7) the summary composite endpoint (the presence of any major morbidity or 30-day operative mortality). For all STS models developed, the proposed limit for number of model eligible variables was not exceeded [6]. Model Performance For all statistical risk models developed, traditional approaches were used to assess model performance [7]. The logistic risk model s accuracy for prediction was measured using the c-index (a measure of model discrimination). Generally, the value for a c-index ranges from 0.5 (ie, 50/50 with no discrimination better than chance alone) to 1.0 (perfect prediction). Model calibration (the degree to which observed outcomes are similar to the predicted outcomes from the model across patients) was examined by comparing average observed and predicted values within each of 10 equal-sized subgroups arranged in increasing order of patient risk. To evaluate model calibration, the Hosmer-Lemeshow (H-L) test for the lack of goodness of fit was applied. As noted in the literature, the value for the H-L test is limited because of the large STS sample size [8]. Evaluating Association Between STS Member Site Mortality and Morbidity Observed-to-Expected Ratios The final STS risk-models were applied to the CABGonly records to calculate the risk for 30-day operative

3 1858 SHROYER ET AL Ann Thorac Surg OPERATIVE RISK MODELS 2003;75: Table 1. Key Descriptive Data, Continuous and Categorical Variables Variable/Response Percentage for All CABG Study Records n 503,478 Percentage for 30-Day Operative Deaths n 15,364 Percentage for Records With Major Morbidity Composite Endpoint n 67,453 Gender Female Race Caucasian Status Elective Chronic lung disease Yes Cerebrovascular disease Yes Peripheral vascular disease Yes Diabetes Yes Number of diseased vessels Three Two One Number of records 503,478 CABG-only records included ( ): variable, age; mean, 64.9; median, 66.0; standard deviation, 10.7; percentage missing, CABG coronary artery bypass graft procedure. mortality and morbidity. For each STS hospital that performed at least 20 CABG procedures during the study period, an observed-to-expected (O/E) ratio was calculated (with 95% confidence intervals [CI]) for 30-day operative mortality and for each of five major morbidities. Based on these STS member site-specific O/E ratios, a Spearman rank correlation coefficient was used to evaluate whether the five O/E ratios for major morbidities were associated with the O/E ratios for 30-day operative mortality. Using this approach, the degree of association was evaluated to determine if these different risk-adjusted major morbidity measures provided duplicative or complementary information to the traditional risk-adjusted mortality information. Results Risk Profile for Study Population For the STS CABG patient population studied (total STS CABG-only patient records 503,478), the average age was 64.9 years (median, 66.0) with a standard deviation of 10.7 years. This generally male CABG population (29.1% female) was predominantly noted to have three-vessel disease (69.9%). There were 6.6% that required an emergent or salvage procedure and 31.1% that required an urgent procedure. Preoperatively, there was a relatively high level of other comorbidities including peripheral vascular disease (14.9%), diabetes (31.6%), cerebrovascular disease (10.5%), and chronic lung disease (15.0%). An abbreviated risk profile for the CABG study population (as well as for the patient subgroups with a 30-day operative death or major morbidity) is shown in Table 1. Outcome Rates The STS CABG study population had 30-day operative death and major composite morbidity rates for CABG procedures of 3.05% and 13.40%, respectively. The specific major morbidity rates included stroke (1.63%), renal failure (3.53%), reoperation (5.17%), prolonged ventilation (5.96%), and sternal infection (0.63%; Table 2). Model Results Seven different risk models were developed and the final logistic model results odds ratios (OR) and 95% CIs for the ORs are identified in Table 3. Generally, there was a high degree of overlap among these seven models in the risk factors that were statistically significant predictors for the study 30-day mortality and morbidity outcomes. For example, the variables for age, sex, body surface area Table 2. CABG-Only Outcomes ( ) Outcome Percentage for All CABG Records n 503, day operative mortality 3.05 Permanent stroke 1.63 Renal failure requiring dialysis 3.53 Prolonged ventilation 5.96 Deep sternal wound infection 0.63 Cardiac surgery reoperation 5.17 Composite major morbidity/mortality Number of records 503,478 CABG-only records. CABG coronary artery bypass graft procedure.

4 Table 3. Odds Ratios With 95% Confidence Intervals for CABG-Only Risk Models ( ) Variable Mortality OR Stroke OR Renal OR Prolonged Ventilation OR Deep Sternal OR Reoperation OR Composite OR Age (years) 1.05 (1.05, 1.05) 1.06 (1.05, 1.06) 1.05 (1.05, 1.06) 1.03 (1.03, 1.03) 1.02 (1.01, 1.03) 1.02 (1.02, 1.02) 1.03 (1.03, 1.04) Age 66 years (1.01, 1.03) Aortic stenosis 1.40 (1.21, 1.61) 1.25 (1.03, 1.50) 1.27 (1.11, 1.46) 1.37 (1.23, 1.53) 1.34 (1.19, 1.50) 1.39 (1.28, 1.50) Black 1.34 (1.23, 1.45) 1.38 (1.24, 1.53) 1.41 (1.31, 1.52) 1.37 (1.29, 1.45) 0.98 (0.81, 1.17) 1.15 (1.08, 1.23) 1.30 (1.24, 1.36) Body surface area a 0.91 (0.89, 0.93) 0.93 (0.91, 0.95) 1.04 (1.02, 1.05) 0.94 (0.93, 0.95) M 1.16 (1.07, 1.25) 0.91 (0.89, 0.92) 0.95 (0.94, 0.95) F 1.38 (1.35, 1.42) c Congestive heart failure 1.18 (1.11, 1.25) 1.06 (1.02, 1.11) 1.04 (1.01, 1.07) Chronic lung disease 1.41 (1.35, 1.48) 1.31 (1.26, 1.37) 1.67 (1.61, 1.73) 1.62 (1.47, 1.78) 1.32 (1.27, 1.37) 1.41 (1.38, 1.45) Cerebrovascular accident 1.10 (1.04, 1.17) 1.43 (1.33, 1.53) 1.15 (1.09, 1.21) 1.11 (1.08, 1.15) Diabetes, oral treatment 1.15 (1.09, 1.21) 1.36 (1.28, 1.45) 1.35 (1.29, 1.42) 1.17 (1.13, 1.22) 1.53 (1.38, 1.70) 0.99 (0.96, 1.04) 1.14 (1.11, 1.17) Ejection fraction 50% 0.98 (0.98, 0.98) 0.99 (1.00, 0.99) 0.99 (0.99, 0.99) 0.98 (0.98, 0.99) 0.99 (0.99, 0.99) 0.99 (0.98, 0.99) First reoperation 2.76 (2.62, 2.91) 1.26 (1.15, 1.37) 1.55 (1.46, 1.64) 1.97 (1.89, 2.05) 1.40 (1.33, 1.47) 1.75 (1.70, 1.81) Hispanic 1.04 (0.92, 1.17) 1.07 (0.92, 1.26) 1.11 (1.00, 1.24) 1.14 (1.05, 1.24) 1.06 (0.83, 1.36) 1.10 (1.00, 1.20) 1.09 (1.02, 1.15) Hypercholesterolemia 0.82 (0.79, 0.86) 0.90 (0.88, 0.93) 0.95 (0.93, 0.97) Hypertension 1.12 (1.08, 1.17) 1.30 (1.22, 1.38) 1.45 (1.39, 1.51) 1.16 (1.13, 1.20) 1.17 (1.07, 1.28) 1.06 (1.03, 1.10) 1.17 (1.15, 1.20) Intraaortic balloon pump 1.46 (1.37, 1.55) 1.13 (1.03, 1.24) 1.54 (1.45, 1.64) 2.26 (2.17, 2.36) 1.43 (1.25, 1.64) 1.36 (1.29, 1.43) 1.78 (1.72, 1.84) Immunosuppressive therapy 1.75 (1.57, 1.95) 1.22 (1.03, 1.44) 1.48 (1.33, 1.64) 1.35 (1.24, 1.48) 1.49 (1.18, 1.89) 1.32 (1.20, 1.46) 1.34 (1.26, 1.43) Insulin 1.50 (1.42, 1.58) 1.48 (1.37, 1.59) 2.26 (2.16, 2.37) 1.53 (1.47, 1.59) 2.74 (2.47, 3.03) 1.22 (1.17, 1.28) 1.59 (1.54, 1.64) Left main artery 50% 1.18 (1.14, 1.24) 1.11 (1.05, 1.18) 1.06 (1.02, 1.10) 1.11 (1.07, 1.14) 1.10 (1.07, 1.14) 1.10 (1.08, 1.13) stenosis Male 0.84 (0.80, 0.89) 0.92 (0.86, 0.99) 1.06 (1.00, 1.12) 0.78 (0.75, 0.82) 0.66 b (0.58, 0.75) 1.20 (1.15, 1.25) 0.96 (0.93, 0.98) Mitral insufficiency 1.22 (1.17, 1.28) 1.21 (1.09, 1.35) 1.29 (1.24, 1.35) 1.40 (1.32, 1.48) 1.39 (1.17, 1.65) 1.31 (1.23, 1.40) 1.39 (1.33, 1.45) Multiple reoperations 4.19 (3.61, 4.86) 0.92 (0.65, 1.28) 1.60 (1.33, 1.92) 2.30 (2.01, 2.64) 1.69 (1.49, 1.97) 2.13 (1.92, 2.36) NYHA functional class IV 1.15 (1.10, 1.20) 1.11 (1.05, 1.17) 1.16 (1.11, 1.20) 1.23 (1.20, 1.27) 1.17 (1.07, 1.28) 1.13 (1.10, 1.17) 1.19 (1.16, 1.21) level Other race 1.12 (1.01, 1.25) 1.33 (1.17, 1.53) 1.22 (1.11, 1.35) 1.24 (1.15, 1.34) 1.34 (1.08, 1.67) 1.28 (1.19, 1.39) 1.28 (1.22, 1.35) Prior myocardial infarction 1.18 (1.16, 1.21) 1.06 (1.03, 1.09) 1.10 (1.08, 1.12) 1.17 (1.16, 1.19) 1.07 (1.03, 1.12) 1.07 (1.05, 1.08) 1.11 (1.10, 1.12) PTCA 6 hrs 1.32 (1.18, 1.48) 1.46 (1.29, 1.66) 1.32 (1.20, 1.45) 1.42 (1.28, 1.58) 1.33 (1.23, 1.43) PVD/CVD 1.29 (1.25, 1.34) 1.50 (1.44, 1.56) 1.30 (1.27, 1.34) 1.23 (1.20, 1.26) 1.19 (1.11, 1.26) 1.22 (1.19, 1.25) 1.26 (1.23, 1.28) Renal failure/dialysis 1.88 (1.80, 1.96) 1.49 (1.37, 1.62) 4.30 (4.09, 4.52) 1.95 (1.86, 2.04) 1.27 (1.14, 1.41) 1.38 (1.33, 1.44) 2.49 (2.41, 2.58) Shock 2.04 (1.90, 2.19) 1.36 (1.21, 1.52) 1.60 (1.48, 1.72) 1.95 (1.85, 2.06) 1.46 (1.37, 1.56) 1.86 (1.78, 1.95) Smoker 1.14 (1.10, 1.18) 1.05 (1.03, 1.08) 1.19 (1.16, 1.21) 1.19 (1.12, 1.25) 1.06 (1.03, 1.08) 1.09 (1.08, 1.11) Status 1.96 (1.88, 2.05) 1.38 (1.29, 1.48) 1.38 (1.31, 1.45) 1.46 (1.41, 1.52) 1.25 (1.19, 1.30) 1.58 (1.53, 1.63) Triple-vessel disease 1.21 (1.17, 1.26) 1.20 (1.14, 1.26) 1.19 (1.14, 1.23) 1.21 (1.18, 1.25) 1.12 (1.04, 1.21) 1.03 (1.00, 1.06) 1.14 (1.12, 1.16) a Odds ratio is based on a 0.1 unit change in body surface area (BSA). b Due to BSA by sex interaction, OR for males is based on BSA 1.8. c Due to BSA by sex interaction, OR for BSA is identified separately for males and females for deep sternal wound infection. Number of records 403,325 CABG-only records (learning data set). For odds ratio (OR) only statistically significant risk factors are noted for each model with 95% confidence interval (CI) listed. NYHA New York Heart Association; PTCA percutaneous transluminal coronary angioplasty; PVD/CVD peripheral vascular disease or cerebrovascular disease or both. Ann Thorac Surg SHROYER ET AL 2003;75: OPERATIVE RISK MODELS 1859

5 1860 SHROYER ET AL Ann Thorac Surg OPERATIVE RISK MODELS 2003;75: Fig 1. (A) Thirty-day operative mortality risk model calibration. (B) Stroke risk model calibration. (C) Reoperation risk model calibration. (D) Prolonged ventilation risk model calibration. (E) Renal failure risk model calibration. (F) Deep sternal wound infection risk model calibration. (BSA), race (ie, categories for black, Hispanic, and other), previous myocardial infarction, cerebrovascular and peripheral vascular disease, diabetes (ie, both insulin and oral treatment categories), renal failure requiring dialysis, immunosuppressive treatment, triple vessel disease, mitral valve insufficiency, New York Heart Association functional class IV, hypertension, and use of intraaortic balloon pump were significant predictors in all of these models. In contrast, the OR impact for each of these risk factors

6 Ann Thorac Surg SHROYER ET AL 2003;75: OPERATIVE RISK MODELS 1861 Fig 2. Composite endpoint (either 30-day operative mortality or complication) risk model calibration. was different for each outcome studied. For example, the impact of the base line risk for renal failure requiring dialysis had the highest OR for perioperative renal failure (OR of 4.30 with 95% CI of 4.09 to 4.52) in comparison with other adverse outcomes such as deep sternal wound infection (OR of 1.27 with 95% CI of 1.14 to 1.41). Model Performance To evaluate model performance, both a c-index (measure of model discrimination) and H-L test (measure of model calibration across risk groups to evaluate goodness of fit ) were calculated. Generally, there was generally an acceptable degree of model predictive power. The c- index was lower for deep sternal wound infection and reoperation models. Based on a comparison of the splitsample learning and testing sets, the predictive accuracy of these models was acceptable. Figures 1 and 2 demonstrate the calibration of models or how well the predicted event rates match the observed event rates among patient subgroups of risk. As noted by the close agreement between these results, these models appear to be relatively accurate across the ranges of patient risk subgroups. The details of model performance metrics are provided in Table 4. Relationship Between Risk-Adjusted Mortality and Risk-Adjusted Morbidity The intra-hospital association between the O/E ratios between 30-day operative mortality with the O/E ratios for each of the five major morbidity endpoints are listed in Table 5. As a measure of association, the value for Spearman s correlation coefficient ranged from 0.03 (almost no correlation) for deep sternal wound infection (where there was not a statistically significant association with the operative mortality model, p ) to 0.26 (modest correlation, but statistically significant with p 0.001) for prolonged ventilation. Figure 3 graphically displays these associations. Given the low degree of association between the information provided by riskadjusted mortality and risk-adjusted mortality, there is Table 4. CABG-Only Risk Model Performance Metrics ( ) no direct overlap of the information provided. Thus, the two outcomes used in combination provide a more comprehensive assessment. Comment C-Index H-L Test H-L Test p Value Final mortality model Final stroke model Final renal failure model Final prolonged ventilation model Final deep sternal wound infection Final reoperation model Final composite model Number of records in test set 100,153 CABG-only records. CABG coronary artery bypass graft surgery; C-Index a measure of model discrimination; H-L Hosmer-Lemeshow. Ultimately, the aim of performing a cardiac surgical procedure is to get the patient through their hospital experience both alive and well with a substantial improvement in functional and overall health. Historically, risk-adjusted 30-day operative mortality rates have been a predominant focus of STS national quality improvement reporting. These STS 30-day risk models were developed with the goal of providing surgical teams clinically relevant risk-adjusted mortality and morbidity reports to use in their local self-assessment and selfimprovement activities. The previously published STS 30-day operative mortality risk model was updated using Table 5. Correlation Coefficients, Association With Risk-Adjusted Mortality ( ) Risk-adjusted stroke vs risk-adjusted mortality Risk-adjusted renal failure vs risk-adjusted mortality Risk-adjusted prolonged ventilation vs risk-adjusted mortality Risk-adjusted deep sternal wound infection vs risk-adjusted mortality Risk-adjusted reoperation vs risk-adjusted mortality Spearman s Correlation Coefficient p Value Number of records 503,478 CABG-only records. Number of sites with observed-to-expected ratios calculated 562 hospitals.

7 1862 SHROYER ET AL Ann Thorac Surg OPERATIVE RISK MODELS 2003;75: Fig 3. (A) Mortality observed-to-expected (O/E) ratio versus stroke O/E ratio by hospital. (B) Mortality O/E ratio versus reoperation O/E ratio by hospital. (C) Mortality O/E ratio versus prolonged ventilation O/E ratio by hospital. (D) Mortality O/E ratio versus renal failure O/E ratio by hospital. (E) Mortality O/E ratio versus deep sternal wound infection O/E ratio by hospital. the most recent data reported in National STS Adult Cardiac Surgery Database. The updated mortality model is very similar to previously published STS models. Although the field of cardiac surgery has rapidly evolved over the past decade (with the introduction of new surgical techniques, medications, monitoring equipment, and devices), there appears to be an inherent stability among the many risk factors impacting 30-day operative mortality. Across the 1995, 1996, and 1999 STS 30-day operative mortality models published, the preoperative risk factors of age, renal dysfunction, emergent status, stability upon arrival to the operating room (eg, cardio-

8 Ann Thorac Surg SHROYER ET AL 2003;75: OPERATIVE RISK MODELS 1863 genic shock or hemodynamic instability variables), and reoperation (including the variable for multiple reoperations) have consistently remained among the significant predictors of operative mortality [8, 9]. Although mortality is obviously the most extreme adverse clinical endpoint, limitations exist to using operative mortality alone to evaluate a surgical team s quality of care. Major complications often impact not only the perioperative period but also the patient s quality of life in the future and may often pose serious threats to a patient s longer-term survival, functional capabilities, and overall well being after the CABG procedure. Complementary outcome measures for risk-adjusted major morbidities were developed. This supplementary analysis created a window of opportunity to examine the patient risk characteristics that are most likely to influence major morbidity, as well as to examine the overall impact of morbidity models on the evaluation of current CABG outcomes as part of the STS national continuous quality improvement reporting initiative. For future STS research, morbidity risk models also may play important role in evaluating different clinical treatment strategies. As part of several ongoing STS studies evaluating different cardiac surgical approaches (eg, the use of internal mammary artery grafting in the elderly), these new STS morbidity models have been implemented successfully. Even if a short-term survival benefit is demonstrated for a specific approach, the probability for the occurrence of a major morbid event needs to be an important consideration in the surgeon s decision-making process weighing the advantages and disadvantages before proceeding toward implementation in daily clinical practice. From the results in Table 3, there appears to be a high degree of overlap between the risk factors that are predictive for both the 30-day operative mortality and the major morbidity risk models. For example, the variables for age, sex, BSA, previous myocardial infarction, cerebrovascular and peripheral vascular disease, diabetes, and use of intraaortic balloon pump are a subset of the risk factors that were predictive across all of the seven mortality and morbidity models developed. A smaller set of patient clinical characteristics may be identified in the future to constitute a core set of patient risk factors for the most serious adverse events [10]. Generally, the previously reported CABG morbidity models in the literature had been developed from populations of small size (usually single-center studies), focused on a single major morbidity outcome, or developed using data from high-risk patient groups [11 14]. From an STS national reporting perspective, therefore, these existing morbidity models had limited applicability as broad risk-evaluation tools for STS CABG patients. Overall, the set of risk models developed in this study performed acceptably. The clinical implications are that preoperative patient risk factors have a significant influence on the likelihood for major morbid events (ie, older, sicker patients are at higher risk for occurrence of perioperative complications). For both deep sternal wound infection and reoperation morbidity endpoints, there are several potential reasons for the lower model discrimination that was observed, including (1) the patient preoperative risk factors most relevant to these two endpoints may not be captured adequately; (2) these two complications may have reporting reliability challenges; or (3) patient preoperative factors may not have as a high impact upon the likelihood for these two events occurring. That is, these two major morbidities may be potentially more related to processes and structures of cardiac surgical care (for example, reoperation may potentially be related to surgeon technical skill and surgical infection may possibly be related to operative time or technique) rather than a patient s inherent risk profile. It may be of interest to note, that the directionality and impact of a given risk factor may vary across the risk models developed for different outcomes. As previously reported, female gender is an important predictor of 30-day operative mortality [15, 16]. In the major morbidity risk models developed, however, the impact of female gender is not consistent across different complications. For example, the impact of female gender appears to be slightly protective for the major morbidity endpoints of renal failure requiring dialysis and reoperation. However, female gender appears to predispose to higher adverse event rates for the outcomes of 30-day operative death, stroke, prolonged ventilation, and deep sternal wound infection. Moreover, female gender appears to maintain this overall adverse impact when the summary composite endpoint (combining both mortality and major morbidity) is assessed. As might be anticipated clinically, the impact of emergent salvage, emergent, urgent, or elective status are in that order predictive of an adverse event. In reviewing the odds ratio for the status variable across models, the impact on mortality appears higher than the impact on major morbidity endpoints. Importantly, the incidence of the composite endpoint (an adverse outcome combining both mortality and morbidity) even for relatively low risk the elective and urgent patient subgroups is sufficiently high (as noted in Table 1) to suggest that focusing on the combined endpoint may have value for this subgroup of CABG patients. A key question addressed by this study was if calculating an O/E ratio for five individual major morbidities added additional information that may be potentially helpful to individual surgeons and groups beyond the information obtained from risk-adjusted mortality alone. A high degree of association between mortality and morbidity O/E ratios would indicate that one approach might be a substitute for the other. A low degree of association, however, would indicate that both measures together provide complementary information. From the results in Table 5, it is clear that there was not a statistically significant association between O/E ratios for deep sternal wound infection with 30-day operative mortality. For the other four major morbidity outcomes (stroke, renal failure, prolonged ventilation, and reoperation), there was a statistically significant association between these morbidities and 30-day operative mortality. Given the low degree of association found between O/E

9 1864 SHROYER ET AL Ann Thorac Surg OPERATIVE RISK MODELS 2003;75: ratios for mortality with major morbidity (correlation coefficients ranging from 0.03 to 0.26), it appears that reporting risk-adjusted morbidity provides additional information, but not duplicative of, risk-adjusted mortality information currently provided to all STS member sites. This low degree of association may be due in part to the fact that mortality is an objective hard endpoint, whereas there may be greater variability in the collection of morbidity uniformly across hospitals. There was a higher association between risk-adjusted mortality and the three major complications of permanent stroke (Spearman s correlation coefficient 0.20), renal failure requiring dialysis (Spearman s correlation coefficient 0.21), and prolonged ventilation (Spearman s correlation coefficient 0.26) in comparison with reoperation (Spearman s correlation coefficient 0.13). Again, the reasons for this discordance may well be related to other clinical factors (eg, the conduct of the operation) that contribute to the occurrence of these postoperative complications. Although the reliability of STS morbidity reporting cannot be readily assessed, the first step toward improving morbidity data is to begin to provide local surgical teams with their risk-adjusted morbidity results. In summary, these new individual morbidity models and the new composite model complement to a high degree the previous CABG 30-day operative mortality model generated for the STS National Adult Cardiac Surgery Database. Together, these risk-adjusted mortality and risk-adjusted morbidity data provide a broader range of outcomes to screen and focus quality of care improvement activities after CABG, both at an individual site and nationwide. Based on these analyses, the STS Database has documented that 87% of patients coming to CABG can have the expectation of surviving the procedure without a major morbid event. Doctor Shroyer s participation in this project was supported in part by funding from the Department of Veterans Affairs Health Services Research and Development Office (Grant IHY , Dr Shroyer Principal Investigator), the VA Office of Patient Care Services, and the VA Office of Quality and Performance, VA Headquarters, Washington, DC. The authors wish to thank all of the participants of the STS National Database Committee for their support to make this risk-adjusted mortality/morbidity study possible. The authors are grateful to Bradley G. Hammill, MS, and Amy M. Krambrink, BS, at the Duke Clinical Research Institute (DCRI) for their outstanding efforts to provide support for this project. References 1. Donabedian A. Criteria and standards for quality assessment and monitoring. Qual Rev Bull 1986;12: Ferguson TB Jr, Hammill BG, DeLong ER, Peterson ED, Grover FL. A decade of change: risk profiles and outcomes for isolated CABG procedures, Ann Thorac Surg 2002;73: Rumsfeld JS, Magid D, O Brien MM, et al. Changes in health-related quality of life following coronary artery bypass graft surgery. Ann Thorac Surg 2001;72: Grover FL, Shroyer AL, Edwards FH, et al. Data quality review program: The Society of Thoracic Surgeons Adult Cardiac National Database. Ann Thorac Surg 1996;26: Shroyer AL, Edwards FH, Grover FL. Updates to the data quality review program: The Society of Thoracic Surgeons Adult Cardiac National Database. Ann Thorac Surg 1998;65: Hosmer DW, Lemeshow S. Applied logistic regression. New York: Wiley & Sons, 1989: Ash A, Schwartz M. Evaluating the performance of riskadjustment methods: dichotomous variables. In: Iezzoni L, ed. Risk adjustment for measuring health care outcomes. Ann Arbor, MI: Health Administration Press, 1994: Shroyer AL, Grunwald GK, Edwards FH, Grover FL. Response to Ms. Ivanov s Inquiry [Letter]. Ann Thorac Surg 1998;66: Shroyer AL, Grover FL, Edwards FE Coronary artery bypass risk model: The Society of Thoracic Surgeons Adult Cardiac National Database. Ann Thorac Surg 1998;65: Shroyer AL, Plomondon ME, Grover FL, Edwards FH Coronary artery bypass risk model: The Society of Thoracic Surgeons Adult Cardiac National Database. Ann Thorac Surg 1999;67: Jones RH, Hannan EL, Hammermeister KE, et al. Identification of preoperative variables needed for risk adjustment of short-term mortality after coronary artery bypass graft surgery. J Am Coll Cardiol 1996;28: Rady MY, Ryan T, Starr NJ. Perioperative determinants of morbidity and mortality in elderly patients undergoing CABG surgery. Crit Care Med 1998;26: McKhann GM, Goldsborough MA, Borowicz LM Jr, et al. Predictors of stroke risk in coronary artery bypass patients. Ann Thorac Surg 1997;63: Almassi GH, Summers T, Moritz TE, et al. Stroke in cardiac surgical patients: determinants and outcome. Ann Thorac Surg 1999;68: Hammermeister KE, Burchfiel C, Johnson R, Grover FL. Identification of patients at greatest risk for developing major complications of CABG surgery. Circulation 1990; 82(Suppl 4): Edwards FH, Carey JS, Grover FL, Bero JW, Hartz RS. Impact of gender on coronary bypass operative mortality. Ann Thorac Surg 1998;66: Hartz RS, Rao AV, Plomondon ME, Grover FL, Shroyer AL. Effects of race, with or without gender, on operative mortality after coronary artery bypass grafting: a study using The Society of Thoracic Surgeons National Database. Ann Thorac Surg 2001;71: DISCUSSION DR ROGER J. F. BASKETT (Halifax, Nova Scotia, Canada): I have a couple of what I think are fundamental questions. How do you validate the data from the various centers? In other words, how do you insure that they re reporting 100% of their cases and that their data is correct? In addition, how do you deal with missing data, particularly ejection fraction? There is always a huge number missing. And when you get up to 10% or 20% of missing data points, it becomes pretty difficult to use the variable. So I wanted to know how you dealt with missing variables.

10 Ann Thorac Surg SHROYER ET AL 2003;75: OPERATIVE RISK MODELS 1865 DR EDWARDS: DCRI has an audit protocol to insure high quality of the data. That has considerably cleaned up the data as compared with 5 or 6 years ago. Now, as far as knowing that each one of the individual participating organizations reports every case they have done: there is simply not a way to do that in a voluntary database. As you know, some of the other databases are involuntary, like the VA system, and the New York State system in particular. They have ways of actually going out to the institutions to make sure that if a case was done it gets entered into the database. With a voluntary database, you simply do not have that luxury and clearly that is one of the disadvantages of a voluntary database. In regard to missing data, there are standard ways that we can use imputation techniques or substitution techniques to allow us to use a patient with some degree of missing data. Ejection fraction, as you point out, is often missing, often for very good clinical reasons, and that is one of the ideal circumstances in which we use imputation data. DR JOHN C. CHEN (Honolulu, HI): As you are developing this model, are you going to use the same variables for on pump versus off pump for all calculations? DR EDWARDS: We did not take that into account with the models developed here. I think it would make a very good follow-up study to be able to break the population into on-pump and off-pump groups and develop different models for each. You could also just run simple univariate and multivariate analyses of the risk factors to see if perhaps the risk factors are different for each one of the populations. But up to this point we have not done that.