Risk Score for Predicting In-Hospital/30-Day Mortality for Patients Undergoing Valve and Valve/ Coronary Artery Bypass Graft Surgery

Transcription

1 Risk Score for Predicting In-Hospital/3-Day Mortality for Patients Undergoing Valve and Valve/ Coronary Artery Bypass Graft Surgery Edward L. Hannan, PhD, Michael Racz, PhD, Alfred T. Culliford, MD, Stephen J. Lahey, MD, Andrew Wechsler, MD, Desmond Jordan, MD, Jeffrey P. Gold, MD, Robert S. D. Higgins, MD, and Craig R. Smith, MD State University of New York at Albany and Albany College of Pharmacy and Health Sciences, Albany, New York; New York University Medical Center, New York, New York; University of Connecticut Health Center, Storrs, Connecticut; Drexel University College of Medicine, Philadelphia, Pennsylvania; Columbia-Presbyterian Medical Center, New York, New York; and Medical University of Ohio, Toledo, and Ohio State University, Columbus, Ohio Background. Risk scores are simplified linear formulas for predicting mortality or other adverse outcomes at the bedside without personal digital assistants or calculators. Although risk scores are available for valve surgery, they do not predict short-term mortality (within 3 days of surgery) after hospital discharge. Methods. New York s Cardiac Surgery Reporting System 27 to 29 data were matched to vital statistics data to identify valve surgery with and without concomitant coronary artery bypass graft (CABG) surgery deaths occurring in the index admission or within 3 days after the procedure in any location. Risk scores were created to easily predict these outcomes by modifying more complicated logistic regression models. Results. There were 13,4 isolated valve surgery patients and 8,373 valve/cabg surgery patients in the study. The respective in-hospital/3-day mortality rates were 4.3% and 6.6%. There are 11 risk factors comprising the isolated valve surgery score, with risk factor scores ranging from 1 to 8, and the highest observed total score is 28. There are 14 risk factors comprising the valve/cabg surgery score, with risk factor scores ranging from 1 to 6, and the highest observed total score is 19. The scores accurately predicted mortality in 27 to 29 as well as in 24 to 26, and were strongly correlated with complications and length of stay. Conclusions. The risk scores that were developed provide quick and accurate estimates of patients chances of short-term mortality after cardiac valve surgery. (Ann Thorac Surg 213;9:1282 9) 213 by The Society of Thoracic Surgeons Organizations such as The Society of Thoracic Surgeons (STS) and the New York State Department of Health have created statistical models for predicting short-term mortality for cardiac valve surgery patients for many years [1, 2]. These models can be used at the hospital level to assess and compare risk-adjusted mortality rates among hospitals, and can be used at the patient level to predict a patient s chance of surviving the surgery. The statistical (logistic regression) models used to predict patient mortality are complex and not easily translated into probabilities of short-term mortality at the bedside. The Society of Thoracic Surgeons has developed a risk calculator that can be used on a computer or personal digital assistant. However, this is not convenient for everyone to use, and for many people a simple addition of risk scores for patient risk factors combined with a table that assigns mortality probabilities to risk scores is preferable. Accepted for publication Nov 12, 212. Address correspondence to Dr Hannan, School of Public Health, State University of New York at Albany, One University Pl, Rensselaer, NY ; elh3@health.state.ny.us. Risk scores that accomplish this for coronary artery bypass graft (CABG) surgery and valve surgery patients have been available for several years, but these scores use in-hospital mortality as the outcome measure, and in recent years it has been established that many patients die shortly after discharge from the hospital after valve surgery [3 13]. The purpose of this study is to develop a risk score that can be used to predict short-term (in the index admission at any time or within 3 days anywhere) mortality during or after cardiac valve surgery, defined as either isolated valve surgery, or valve surgery combined with CABG surgery. We will also apply the risk score to another time frame in New York for purposes of validation, and examine the association of the index with length of stay. Patients and Methods Databases The data in the study were obtained from the Cardiac Surgery Reporting System (CSRS) in the years 27 to 29, with the years 24 to 26 used for purposes of validation. The CSRS is a population-based registry 213 by The Society of Thoracic Surgeons 3-497/$36. Published by Elsevier Inc

2 Ann Thorac Surg HANNAN ET AL 213;9: NEW YORK VALVE SURGERY RISK SCORE 1283 founded in 1989 by the New York State Department of Health that contains detailed information on all major cardiac surgery procedures performed in nonfederal hospitals in New York. Information in the registry includes demographics, risk factors, complications, discharge destination, and types of procedures performed, as well as hospital and physician identifiers. In-hospital mortality is one of the options for discharge status. The completeness of CSRS data are assured by matching it to New York s administrative database, the Statewide Planning and Research Cooperative System (SPARCS). Accuracy of in-hospital mortality is confirmed by matching with SPARCS, and the accuracy of risk factor reporting is assured by auditing samples of records from hospitals. To obtain deaths that occur with 3 days of the index procedure after discharge, the CSRS was linked to New York vital statistics data and to the Social Security Administration Master File. Patients Patients in the system who were candidates for the study consisted of all patients undergoing valve surgery with or without concomitant CABG surgery in nonfederal hospitals in New York in 27 to 29. Patients in 24 to 26 were also used for validation purposes. A total of 22 shock patients undergoing isolated valve surgery and 37 shock patients undergoing valve/cabg surgery in 27 to 29 were excluded from the study. The total number of 27 to 29 patients in the study was 21,828; of those, 13,4 underwent isolated valve surgery and 8,373 underwent valve/cabg surgery. The respective in-hospital/3- day mortality rates were 4.3% and 6.6%. A total of 4 hospitals were represented in the models. Statistical Analyses First, for each type of patient (valve, valve/cabg), a logistic regression model was used to predict in-hospital/ 3-day mortality (the dependent variable) based on all of the available risk factors in the CSRS. The first part of this process consisted of identifying which of these risk factors had a significant (p.1) bivariate relationship with the dependent variable. Risk factors that were tested included age, sex, body mass index, body surface area, preprocedural myocardial infarction, hemodynamic state, ventricular function, number of vessels diseased, left main disease, and numerous comorbidities. Continuous variables like age, body mass index, and body surface area were tested using t tests and Wilcoxon Table 1. Logistic Regression Equation for Isolated Valve Surgery In-Hospital/3-Day Deaths in New York State, (n 13,4) a Risk Factor Prevalence (%) Coefficient Odds Ratio (9% CI) p Value Type of valve surgery Aortic valve replacement (reference) Mitral valve replacement ( ).1 Mitral valve repair ( ).98 Multiple valve repair or replacement ( ).1 Age, number of years ( ).1 Female sex ( ).1 Hemodynamically unstable ( ).1 Ejection fraction 3% % ( ).4 Preprocedural MI No previous MI Previous MI within 7 days ( ).13 Previous MI 8 days ( ).22 Endocarditis ( ).1 Cerebrovascular disease ( ). Renal failure No renal failure Creatinine 1.3 to 1. mg/dl ( ).1 Creatinine 1. to 2. mg/dl ( ).1 Creatinine 2. mg/dl (2.27.3).1 Requiring dialysis ( ).1 Left main disease ( ).29 Previous open heart surgery ( ).1 a Intercept.133; C statistic.781. CI confidence interval; MI myocardial infarction.

3 1284 HANNAN ET AL Ann Thorac Surg NEW YORK VALVE SURGERY RISK SCORE 213;9: rank-sum tests; all other variables were tested using 2 tests. Risk factors that were significantly related to the mortality measure were then used as candidate independent variables in subsequent logistic regression analyses. Significant variables from each model (p.) comprised the final model for each type of patient (valve, valve/cabg). The fit of the final model was evaluated on the basis of discrimination using the C statistic [14], and calibration using the Hosmer- Lemeshow statistic []. The next step consisted of converting the logistic regression models into corresponding valve and valve/ CABG risk scores first described by Sullivan and colleagues [16] that were used in our earlier studies [17,18]. The constant corresponding to one point in the risk score methodology was identified by finding the lowest weight associated with a one unit increase in each significant risk factor using the logistic regression coefficients. The total risk score for each patient was obtained by summing the scores of all of the risk factors the patient had. For each risk score, the probability of in-hospital/3-day mortality was obtained by developing a new logistic regression model using the risk score as the single independent variable and mortality as the binary dependent variable. The predicted value for each risk score was then obtained by plugging the score into the model. Table 2. Logistic Regression Equation for Valve/Coronary Artery Bypass Graft Surgery In-Hospital/3-Day Deaths in New York State, 27 to 29 (n 8,373) a Risk Factor Prevalence (%) Coefficient Odds Ratio (9% CI) p Value Type of valve/cabg surgery Aortic valve replacement/cabg (reference) Mitral valve replacement/cabg ( ).1 Mitral valve repair/cabg ( ).4 Multiple valve repair or ( ).1 replacement/cabg Age, years 7 (reference) ( ) ( ).1 Sex/BSA.24 Male and BSA ( ).26 Female and BSA ( ).21 Female and BSA 1. to ( ).1 Female and BSA ( )... All others (reference) Hemodynamically unstable ( ).71 Ejection fraction 3% % ( ).1 Preprocedural MI No MI within 14 days (reference) Previous MI within 1 day ( ).3 Previous MI 1 to 14 days ( ).1 Endocarditis ( ).73 Peripheral vascular disease ( ).1 Renal failure No renal failure Creatinine mg/dl ( ).26 Creatinine 2. mg/dl ( ).1 Requiring dialysis ( ).1 Left main disease ( ).1 Previous open heart operations ( ).1 Previous PCI before this admission (1. 1.7).13 a Intercept ; C statistic.77. BSA body surface area; CABG coronary artery bypass graft; CI confidence interval; MI myocardial infarction; PCI percutaneous coronary intervention.

4 Ann Thorac Surg HANNAN ET AL 213;9: NEW YORK VALVE SURGERY RISK SCORE 128 Table 3. Risk Scores for In-Hospital/3-Day Mortality for Isolated Valve Surgery a Risk Factor Score Type of valve surgery Aortic valve replacement Mitral valve replacement 2 Mitral valve repair Multiple valve repair or replacement 4 Age, years Female 3 Hemodynamically unstable Ejection fraction 3% 2 Previous MI within 7 days 3 Previous MI 8 days or more 2 Endocarditis 4 Cerebrovascular disease 2 Renal failure Creatinine mg/dl 2 Creatinine mg/dl 4 Creatinine 2. mg/dl Requiring dialysis 8 Left main disease 4 Previous open heart operations 2 a Range of total score, to 44. The highest observed total risk score was 28 in 27 to 29 New York data. MI myocardial infarction. For each group of patients (isolated valve and valve/ CABG), the accuracy of the risk score in predicting mortality was examined by comparing, for each individual score, the predicted mortality rate for all 27 to 29 patients with that score with the observed mortality rate for that risk score. Confidence intervals were calculated for the observed rates for each risk score and the predicted value for each score was inspected to see if it fell inside the confidence interval for the observed value. The same process was then used to see how well the 27 to 29 risk scores predicted mortality for patients undergoing valve surgery in New York in 24 to 26 after recalibrating the 24 to 26 scores to adjust for the fact that the underlying mortality rate had dropped between the two periods. Because the mortality rate in 27 to 29 was considerably lower than in 24 to 26, and the predicted rate for 24 to 26 based on the 27 to 29 model was significantly lower than the 27 to 29 observed rate, the 27 to 29 model was then recalibrated and the fit of the risk score on 24 to 26 data was reassessed [17]. For each type of patient, the correspondences between the mortality risk score and length of stay was also examined for each measure by plotting adverse outcome rates for each value of risk score to determine if there were higher values of the adverse outcome measure as the risk score increased. All statistical analyses were conducted in SAS version 9.1 (SAS Institute, Cary, NC). Results The significant risk factors in the logistic regression model for isolated valve surgery that was altered to create a risk score are contained in Table 1. In the logistic regression model for valve/cabg surgery (see Table 2), the variables were very similar to those in the isolated valve model. C statistics for the models were very good (.781 and.77) as were Hosmer-Lemeshow statistics (p.9 and p.32, respectively). Scores for the various risk factors are presented in Tables 3 and 4. As indicated, the highest scores for isolated valve surgery patients were associated with patients requiring renal dialysis (8 points) and patients at least 8 years old (7 points). The highest point totals for patients undergoing valve/cabg surgery were for patients requiring renal dialysis (6 points), patients with Table 4. Risk Scores for In-Hospital/3-Day Mortality for Valve/Coronary Artery Bypass Graft Surgery a Risk Factor Score Type of valve/cabg surgery Aortic valve replacement/cabg Mitral valve replacement/cabg 2 Mitral valve repair/cabg Multiple valve repair or replacement/cabg 4 Age, years Between 7 and or more Sex and BSA Male and BSA 2. 2 Female and BSA 1. 2 Female and BSA 1. to Female and BSA 2. All others Hemodynamically unstable 3 Ejection fraction 3% 2 Previous MI within 1 day 4 Previous MI 1 to 14 days 2 Endocarditis 3 Peripheral vascular disease 2 Renal failure Creatinine mg/dl 2 Creatinine 2. mg/dl Requiring dialysis 6 Left main disease 1 Previous open heart operations 2 Previous PCI before this admission 1 a Range of total score, to 38. The highest observed total risk score was 19 in 27 to 29 New York data. BSA body surface area; CABG coronary artery bypass graft; MI myocardial infarction; PCI percutaneous coronary intervention.

5 1286 HANNAN ET AL Ann Thorac Surg NEW YORK VALVE SURGERY RISK SCORE 213;9: Table. Predicted Risk of In-Hospital/3-Day Mortality Associated With Individual Risk Scores for Isolated Valve Surgery Total Risk Score Predicted Risk (%) (.7% of all patients) 78 creatinine greater than 2. mg/dl not requiring dialysis ( points), females with a body surface area of at least 2. ( points), and patients at least 8 years old ( points). The minimum possible risk score was points for each type of patient, and it was associated with patients having none of the risk factors listed in Tables 3 and 4. As noted in Table, the predicted probabilities of in-hospital/3-day mortality for isolated valve surgery patients ranged from less than 1% for a risk score of, to 78% for the risk scores of 22 and higher. Table 6 indicates that the predicted probabilities of in-hospital/3-day mortality for valve/cabg patients ranged from 2% for risk scores of and 1, to 87% for risk scores of 14 and higher. As an example of how the risk score can be used, a 62-year-old female isolated valve surgery patient undergoing aortic valve replacement with no hemodynamic instability, an ejection fraction of %, a previous myocardial 2 months earlier, with no endocarditis, cerebrovascular disease, no renal failure, left main disease, and no previous open heart operations would have a total risk score of From Table, the predicted probability of in-hospital/3-day mortality for this patient would be %. Figures 1 and 2 demonstrate the correspondence between observed and predicted rates for each risk score where observed and predicted rates were obtained from 27 to 29 data. Figure 1 applies to patients with isolated valve surgery, and Figure 2 to patients with valve/cabg surgery. As the figures demonstrate, the observed and predicted values are quite close together, and the predicted rates fall within the 9% confidence interval for the observed rates for each of the risk score ranges in the figure. Figures 3 and 4 contrast the observed rates for each risk score in the year 24 to 26 with the predicted values based on the 27 to 29 risk model after recalibrating the 27 to 29 risk score probabilities to reflect the differences in performance between 24 to 26 and 27 to 29. Figure 3 applies to isolated valve surgery and Figure 4, to valve/cabg surgery. For patients with isolated valve surgery, the predicted values and observed values again demonstrate a very good correspondence, with the predicted values for all risk score ranges falling within the corresponding 9% confidence interval for the observed value for all ranges except in the last range of scores, for which the predicted value barely exceeds the confidence interval for the observed value. For valve/ CABG patients, the fit was reasonably good, with six of the eight ranges having predicted mortality rates within the confidence intervals for the observed rates. Figures and 6 show that there are reasonably good correspondences between each of the two 27 to 29 risk scores (isolated valve surgery and valve/cabg surgery) and the postprocedural length of stay. Except for a risk score of 7 for patients undergoing valve/cabg surgery, there is a monotonic rise in length of stay that accompanies increases in risk scores. Comment Risk scores continue to be valuable and relevant because for many clinicians they are the preferred way of making a quick, accurate decision about patient risk for purposes of informed consent and determination of the appropriate treatment. However, risk scores for valve surgery are rare and all that have been published to date used Table 6. Predicted Risk of In-Hospital/3-Day Mortality Associated With Individual Risk Scores for Valve/Coronary Artery Bypass Graft Surgery Total Risk Score Predicted Risk (%) (.8% of all patients) 87

6 ) Ann Thorac Surg HANNAN ET AL 213;9: NEW YORK VALVE SURGERY RISK SCORE day Mortality (%) sk of In-Hospital\3 Ri 3 2 Fig 1. Observed (with 9% confidence interval [open bars]) and predicted risks of in-hospital/3- day mortality by total risk score for isolated valve surgery patients in New York State, 27 to 29 (n 13,4). (Circles/line predicted by risk score.) % of pa ents Cumula ve % in-hospital mortality as the adverse outcome to be predicted. A recent New York report found that nearly 18.3% of all short-term deaths (in-hospital or out-of-hospital within 3 days of surgery) occurred after discharge [2]. The purpose of our study was to develop valve risk scores (one for isolated valve surgery and another for valve/ CABG surgery) using this expanded mortality measure as the outcome. Future efforts should perhaps be devoted to even longer time intervals after surgery, such as 9 days. Results of our study show that the isolated valve risk score based on New York s 27 to 29 data contains a total of 11 risk factors, with the risk factor scores range from to 8, with a total risk score from to 34, and the highest score observed for any patient was 28. A total of 91% of the patients had scores of 14 or lower. The valve/cabg risk score contains a total of 13 risk factors, with the risk factor scores range from to 6, with a total risk score from to 38, and the highest score observed for any patient was 19. A total of 91% of the patients had scores of 8 or lower. Our valve risk score models proved to have excellent calibration in that the predicted and observed mortality rates for various risk score ranges were extremely 3-day Mortality (% %) Risk of In-Hospital\3 2 1 Fig 2. Observed (with 9% confidence interval [open bars]) and predicted risks of in-hospital/3- day mortality by total risk score for valve and coronary artery bypass graft surgery patients in New York State, 27 to 29 (n 8,373). (Circles/line predicted by risk score.) % of pa ents Cumula ve %

7 ) 1288 HANNAN ET AL Ann Thorac Surg NEW YORK VALVE SURGERY RISK SCORE 213;9: Fig 3. Observed (with 9% confidence interval [open bars]) and rescaled predicted risks of inhospital/3-day mortality by total risk score for isolated valve surgery patients in New York State, 24 to 26 (n 12,34). (Circles/ line predicted by risk score.) -day Mortality (%) sk of In-Hospital\3 Ri % of pa ents Cumula ve % similar, with no expected rates falling outside of the 9% confidence intervals for the observed rates. With regard to validation, we were only able to validate the risk scores based on New York data in a different time interval because we do not have access to any other populations of valve surgery data. When that was done, we found that after recalibration to account for a relatively large reduction in mortality between the two databases, the 27 to 29 risk scores predicted 24 to 26 mortality rates reasonably well, all but one (isolated valve surgery) or two (valve/cabg surgery) ranges having predicted rates falling inside of the 9% confidence intervals for observed rates. Nevertheless, the isolated valve and valve/cabg risk scores developed in this study should also be validated by testing them against non-new York populations. We were not able to do this because we do not have access to other databases. It should also be noted that the New York risk scores for valve surgery are based on data from all 4 nonfederal hospitals in which cardiac surgery was performed in New York from 27 to 29. Thus, we used Fig 4. Observed (with 9% confidence interval [open bars]) and rescaled predicted risks of in-hospital/3-day mortality by total risk score for valve and coronary artery bypass graft surgery patients in New York State, 24 to 26 (n 8,6). (Circles/line predicted by risk score.) 3-day Mortality (% %) Risk of In-Hospital\ % of pa ents Cumula ve %

8 ) Ann Thorac Surg HANNAN ET AL 213;9: NEW YORK VALVE SURGERY RISK SCORE 1289 ength of Stay (days) Le Fig. Mean length of stay by total risk score for isolated valve surgery patients in New York State, 27 to 29 (n 13,4) %ofpa ents Cumula ve % population-based data, whereby all patients in a given region are included in the analyses. That is important because it should contribute to the accuracy and generalizability of our findings. Furthermore, the completeness of the database has been verified by matching it to New York s administrative database, and the accuracy of the data has been enhanced by matching to administrative data and vital statistics data, and by extensive auditing by New York s utilization review agent. The use of vital statistics data to confirm 3-day deaths outside of the hospital is particularly important because we found that hospital-reported 3-day deaths are grossly underreported. We also found that the short-term risk scores we used for isolated valve and valve/cabg surgery were strongly associated with postprocedural length of stay in the same period. Thus, the risk scores developed for predicting mortality also work as reasonably good markers for length of stay. The authors would like to thank Kimberly S. Cozzens, Rosemary Lombardo, Cynthia Johnson, and the cardiac surgery departments and cardiac catheterization laboratories of the participating hospitals for their tireless efforts to ensure the timeliness, completeness, and accuracy of the registry data. s) Length of Stay (days L Fig 6. Mean length of stay by total risk score for valve and coronary artery bypass graft surgery patients in New York State, 27 to 29 (n 8,373) % of pa ents Cumula ve %

9 129 HANNAN ET AL Ann Thorac Surg NEW YORK VALVE SURGERY RISK SCORE 213;9: References 1. O Brien SM, Shahian DM, Filardo G, et al. The Society of Thoracic Surgeons 28 cardiac surgery risk models: part 2 isolated valve surgery. Ann Thorac Surg 29;88(Suppl): Adult cardiac surgery in New York State: Albany, NY: New York State Department of Health, February Hannan EL, Wu C, Bennett EV, et al. A risk index for predicting in-hospital mortality for cardiac valve surgery. Ann Thorac Surg 27;83: Nashef SA, Roques F, Sharples LD, et al. EuroSCORE II. Eur J Cardiovasc Surg 212;41: Higgins TL, Estafanous FG, Loop FD, Beck GJ, Blum JM, Paranandi L. Stratification of morbidity and mortality outcome by preoperative risk factors in coronary artery bypass patients: a clinical severity score. JAMA 1992;267: Wouters SC, Noyez L, Verheugt FW, Brouwer RM. Preoperative prediction of early mortality and morbidity in coronary bypass surgery. Cardiovasc Surg 22;1:. 7. Nashef SA, Roques F, Michel P, et al. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16: Nashef SA, Roques F, Hammill BG, et al. Validation of European system for cardiac operative risk evaluation (EuroSCORE) in North American cardiac surgery. Eur J Cardiothorac Surg 22;22: Immer F, Habicht J, Nessensohn K, et al. Prospective evaluation of 3 risk stratification scores in cardiac surgery. Thorac Cardiovasc Surg 2;48: Heijmans JH, Maessen JG, Roekaerts PM. Risk stratification for adverse outcome in cardiac surgery. Eur J Anaesthesiol 23;2: Geissler HJ, Holzl P, Marohl S, et al. Risk stratification in heart surgery: comparison of six score systems. Eur J Cardiothorac Surg 2;17: Gogbashian A, Sedrakyan A, Treasure T. EuroSCORE: a systematic review of international performance. Eur J Cardiothorac Surg 24;: Parsonnet V, Dean D, Bernstein AD. A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease. Circulation 1989;79(Suppl 1): Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143: Hosmer DW, Lemeshow S. Applied logistic regression. New York: John Wiley & Sons, Sullivan LM, Massaro JM, D Agostino RB. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med 24;23: Hannan EL, Wu C, Bennett EV, et al. Risk stratification of in-hospital mortality for coronary artery bypass graft surgery. J Am Coll Cardiol 26;47: Hannan EL, Farrell LJS, Wechsler A, et al. The New York Risk Score for in-hospital/3-day mortality for coronary artery bypass graft surgery. Ann Thorac Surg. In press.