The European System for Cardiac Operative Risk. Validation of EuroSCORE II in Patients Undergoing Coronary Artery Bypass Surgery

Transcription

1 Validation of EuroSCORE II in Patients Undergoing Coronary Artery Bypass Surgery Fausto Biancari, MD, PhD, Francesco Vasques, MS, Reija Mikkola, MS, Marta Martin, MS, Jarmo Lahtinen, MD, PhD, and Jouni Heikkinen, MD, PhD Department of Surgery, Oulu University Hospital, Oulu, Finland Background. The European System for Cardiac Operative Risk Evaluation II (EuroSCORE II) has been recently developed to improve the performance of the original EuroSCORE. Herein we evaluated its discriminatory ability in predicting the immediate and late outcome after coronary artery bypass grafting (CABG). Methods. Complete data on 1,027 patients who underwent isolated CABG were available for validation of EuroSCORE II and to compare its discriminatory ability with the original EuroSCORE and its Finnish modified version. Results. EuroSCORE II performed somewhat better (area under the curve [AUC] 0.852, Brier score 0.031) than the original logistic EuroSCORE (AUC 0.838, Brier score 0.034) and its Finnish modified version (AUC 0.825, Brier score 0.034) in predicting operative mortality. The overall expected-to-observed operative mortality ratio for the original logistic EuroSCORE was 1.8, for its Finnish modified version was 0.6, and for EuroSCORE II was 1.2. EuroSCORE II showed expected-to-observed ratios ranging from 1.05 to 1.17 in its highest third quintiles. The best cutoff of EuroSCORE II in predicting operative postoperative mortality was 10% (21.5% vs 1.6%, p < ; sensitivity 91.5%, specificity 60.5%, negative predictive value 98.4%, accuracy of 90.3%). The EuroSCORE II was predictive of de novo dialysis (AUC 0.805), prolonged use of inotropes (AUC 0.748), and intensive care unit stay 5 days or greater (AUC 0.793). The risk of late mortality significantly increased across increasing quintiles of EuroSCORE II (p < ). Conclusions. The EuroSCORE II performs better than its original version in predicting operative mortality and morbidity after isolated CABG. Its ability to predict 30-day mortality in high-risk patients is of particular importance. The EuroSCORE II is also a good predictor of late postoperative survival. (Ann Thorac Surg 2012;93:1930 5) 2012 by The Society of Thoracic Surgeons The European System for Cardiac Operative Risk Evaluation (EuroSCORE) [1] has served us well during the last 12 years [2]. The EuroSCORE has been a magnificent instrument in clinical research as it has been adopted as a measure of the operative risk in adult cardiac surgery in over 1,000 studies. Its recent widespread use in selecting patients for transcatheter aortic valve implantation confirmed its important value in the daily clinical activities [3]. As adult cardiac surgery has gone through major changes in terms of increasing operative risk of patients undergoing cardiac surgery and in improvements of surgical techniques and perioperative care [4], preoperative risk prediction has been shown to be a moving target. Because of this, despite its validity to stratify the operative risk [5], EuroSCORE seems to significantly overestimate it [6, 7]. The EuroSCORE investigators recently developed a modified version of this risk scoring method [8]. The aim of this study was to evaluate its efficacy in predicting the immediate and late outcome of patients undergoing coronary artery bypass grafting (CABG). Accepted for publication Feb 20, Address correspondence to Dr Biancari, Division of Cardiothoracic and Vascular Surgery, Department of Surgery, Oulu University Hospital, PO Box 21, Oulu, Finland; faustobiancari@yahoo.it. Material and Methods This study includes a consecutive series of 1,027 patients who underwent isolated CABG at the Oulu University Hospital, Finland between June 2006 and April These patients belong to a series of 2,001 patients who underwent isolated CABG during the same study period and whose 30-day operative mortality rate was 2.8% and postoperative stroke rate was 2.1%. The analysis was restricted to these 1,027 patients because of lack of data on New York Heart Association classes and Canadian Cardiovascular society classes in the other patients. A further restriction to patients inclusion was the lack of specific data in a number of patients with left ventricular ejection fraction less than No attempt to replace missing values was made. Patients characteristics and operative data are summarized in Table 1. Baseline and operative data were provided by our local institutional clinical registry, which collects information in a computerized database. Intraoperative, anesthesiologic, as well as nursing care data are prospectively collected in specific charts and databases. Furthermore, the full medical records of the eligible patients were reviewed in order to determine the preoperative comorbidities and the incidence of major operative complications. Operative risk was estimated by the EuroSCORE II risk scoring method [8], the original EuroSCORE [1], as 2012 by The Society of Thoracic Surgeons /$36.00 Published by Elsevier Inc doi: /j.athoracsur

2 Ann Thorac Surg BIANCARI ET AL 2012;93: EUROSCORE II 1931 Table 1. Preoperative Characteristics and Operative Data in 1,027 Patients Who Underwent Isolated Coronary Artery Bypass Surgery Clinical and Operative Variables Validation Dataset No. (%) Dataset With Missing Data No. (%) p Value Age (years) Females 228 (22.2) 189 (19.4) Diabetes Non-insulin dependent diabetes 173 (16.8) 695 (16.4) Insulin dependent diabetes 125 (12.2) 110 (11.3) egfr ml/min/1.73 m 2 (Cockroft Gault method) egfr ml per min/1.73 m 2 (MDRD method) Dialysis 13 (1.3) 3 (0.3) Body mass index Pulmonary disease 104 (10.1) 89 (9.1) Atrial fibrillation 110 (10.7) 98 (10.1) Stroke 54 (5.3) 40 (4.1) Neurologic dysfunction 17 (1.7) 20 (2.1) Extracardiac arteriopathy 106 (10.3) 83 (8.5) Prior percutaneous coronary intervention 95 (9.3) 89 (9.1) Prior cardiac surgery 12 (1.2) 20 (2.1) Recent myocardial infarction 472 (46.0) 462 (47.4) New York Heart Association classes I 8 (0.8) II 167 (16.3) III 417 (40.6) IV 435 (42.4) Canadian Cardiovascular Society class IV a 174 (16.9) Left ventricular ejection fraction (70.0) 654 (76.9) (22.5) (5.6) (1.9) Unstable angina a 174 (16.9) 112 (11.5) Critical preoperative status 111 (10.8) 63 (6.5) Type of surgery Elective surgery 466 (45.4) 446 (45.8) Urgent surgery 471 (45.9) 484 (49.7) Emergency surgery 90 (8.8) 44 (4.5) Salvage surgery 0 0 Systolic pulmonary pressure 60 mm Hg 20 (1.9) 8 (0.8) mm Hg 37 (3.6) mm Hg 393 (38.3) At least 1 internal mammary artery graft 983 (95.7) 942 (96.7) Radial artery graft 127 (12.4) 198 (20.3) Heart beating surgery 558 (54.3) 548 (56.3) No. of distal anastomoses a Defined as nitrates infusion at operating room arrival. Continuous variables are reported as mean and standard deviation. egfr estimated glomerular filtration rate; MDRD Modification of Diet in Renal Disease. well as its Finnish modified version [6]. Risk factors recorded in our institutional registry were categorized according to the original EuroSCORE criteria. The Euro- SCORE II introduced some new variables, which were collected in our registry only in a number of patients; namely, the New York Heart Association classes and the Canadian Cardiovascular Association classes. Because the Canadian Cardiovascular class 4 may widely vary during the preoperative period, herein we decided to define it as nitrates infusion at operating room arrival.

3 1932 BIANCARI ET AL Ann Thorac Surg EUROSCORE II 2012;93: The latter clearly depicts a status of severe angina at rest and it has been previously shown to be an important risk factor of the original EuroSCORE. Herein we defined the variable poor mobility with the same criteria of the original EuroSCORE variable neurological dysfunction. Indeed, these 2 definitions are, from a clinical point of view, similar. Operative Techniques Intermittent antegrade and retrograde cold blood cardioplegia was used during conventional CABG. Proximal anastomoses were sutured to the ascending aorta during cross clamping, when the latter was considered safe. Epiaortic ultrasound was performed according to the surgeon s preference and the ascending aorta was left untouched in case of a severely diseased aorta. Octopus stabilizer, in some instances Starfish stabilizer, and intracoronary shunts (Medtronic, Minneapolis, MN) were used in patients who underwent off-pump coronary artery bypass. Outcome Endpoints The primary outcome measure of this study was a 30-day postoperative mortality, death within 30 days from operation or later than 30 days if still in hospital. Late all-cause mortality was also considered as a main outcome measure. Secondary outcome endpoints of this study were 30-day postoperative mortality, stroke, low cardiac output syndrome, prolonged use of inotropes, de novo dialysis, resternotomy for mediastinitis, reexploration for excessive bleeding, and length of stay in the intensive care unit. Data on any postoperative mortality was retrieved from the Finnish National Registry Statistics Finland. Statistical Analysis Statistical analysis was performed using PASW v. 18 statistical software (IBM SPSS Inc, Chicago, IL). Continuous variables are reported as the mean standard deviation. Univariate analysis of dichotomous variables was performed with the 2 or Fisher exact test. The Spearman test was used for correlation analysis of continuous variables. Receiver operating characteristic (ROC) curve analysis was performed to estimate the discriminatory ability of these risk scoring methods in predicting immediate postoperative adverse events. Calibration of each risk scoring method was estimated as the area under the ROC curve (AUC) with its 95% confidence interval (95% CI). Accuracy of these risk scoring methods was assessed by the Brier score [9], which is the average squared difference between the predicted probability and the true occurrence of operative mortality. A Brier score should be as close to 0 as possible, with 0.25 as an acceptable upper cutoff. We compared the Brier score of these risk scoring methods with the Wilcoxon test. Furthermore, the expected-to-observed operative mortality ratios were calculated to further estimate the performance of each risk scoring method. A ratio of 1 indicates optimal performance of the risk score. Survival analysis was performed by the Kaplan-Meier method. The ROC curve analysis was used to estimate the performance of these risk scores in predicting mortality at 1-year followup. In the latter case, we excluded from the analysis those patients with a possible follow-up shorter than 1 year. As these scoring methods included most of the herein considered risk factors, multivariable analysis was not performed. A p value less than was considered statistically significant. Results Operative Mortality In-hospital or 30-day mortality rate in this series was 3.7% (38 of 1,027 patients). The predicted mortality rate by the original EuroSCORE was 6.6% 9.9%, by its Finnish modified version was 2.1% 0.3%, and by EuroSCORE II was 4.5% 6.7%. EuroSCORE II performed somewhat better (AUC 0.852, 95% CI to 0.910) than the original logistic EuroSCORE (AUC 0.838, 95% CI to 0.903) and its Finnish modified version (AUC 0.825, 95% CI to 0.891) in predicting operative mortality. The Brier score was for the original EuroSCORE, for its Finnish modified version, and for EuroSCORE II. The Brier scores of these risk scores significantly differed from each other (p ). In the overall series, the expected-to-observed operative mortality ratio for the original logistic EuroSCORE was 1.8, for its Finnish modified version was 0.6, and for EuroSCORE II was 1.2. Figure 1 summarizes the observed and predicted in-hospital or 30-day mortality rates according to quintiles of EuroSCORE II. Figure 2 summarizes the expected-to-observed operative mortality ratios for each quintile of EuroSCORE II, which indicated optimal performance for EuroSCORE II in its 3 highest quintiles (range in these quintiles: 1.05 to 1.17). Immediate postopera ve mortality (%) Observed immediate postop. mortality Original logis c EuroSCORE Finnish modified EuroSCORE EuroSCORE II pts 207 pts 201 pts 205 pts 205 pts EuroSCORE II quin les Fig 1. Observed and predicted operative mortality according to the original logistic European system for cardiac operative risk evaluation (EuroSCORE), Finnish modified score, and EuroSCORE II according to quintiles of the EuroSCORE II. (Pts patients.)

4 Ann Thorac Surg BIANCARI ET AL 2012;93: EUROSCORE II Original logis c EuroSCORE 35 De novo dialysis p< Expected- to- observed mortality ra o Finnish modified EuroSCORE EuroSCORE II Incidence (%) Stroke p=0.087 Low cardiac output syndrome p< ICU stay 5 days p< EuroSCORE II quin les Fig 2. Predicted-to-observed operative mortality ratios for the original logistic EuroSCORE, Finnish modified score, and European system for cardiac operative risk evaluation II (EuroSCORE II) according to quintiles of the EuroSCORE II. A ratio of 1 (dotted line) indicates optimal performance of the risk score. Almost optimal predictive performance was observed for EuroSCORE II in 3 highest quintiles. The best cutoff of EuroSCORE II in predicting operative mortality was 10% ( 10%, 15 of 920 patients: 1.6% vs 10%: 23 of 107 patients: 21.5%, p ). This cutoff value had a sensitivity of 91.5%, a specificity of 60.5%, a negative predictive value of 98.4%, and an accuracy of 90.3%. Immediate Postoperative Adverse Events The EuroSCORE II was predictive of all major adverse postoperative events, but reexploration for excessive bleeding (Table 2; Fig 3). This risk scoring method performed particularly well in predicting de novo dialysis (AUC 0.805, 95% CI to 0.877), prolonged use of EuroSCORE II quin les Fig 3. Major postoperative adverse events according to quintiles of European system for cardiac operative risk evaluation II (EuroSCORE II). (ICU intensive care unit.) inotropes (AUC 0.748, 95% CI to 0.781), intensive care unit stay 5 days or greater (AUC 0.793, 95% CI to 0.834), and combined adverse endpoint (AUC 0.747, 95% CI to 0.779). Correlation between the length of intensive care unit and EuroSCORE II was statistically significant (rho: 0.431, p ) and it was somewhat better than the original logistic EuroSCORE (rho: 0.411, p ) and its Finnish modified version (rho: 0.355, p ). The EuroSCORE II 10% or greater was a powerful predictor of a number of postoperative adverse events (low cardiac output syndrome: 27.1% vs 10.7%, p ; prolonged use of inotropes: 67.3% vs 25.0%, p ; de novo dialysis: 9.3% vs 1.2%, p ; stroke: 5.6% vs 2.1%, p 0.034; intensive care unit stay 5 days: 43.0% vs 7.7%, p ). Table 2. Postoperative Adverse Events After Isolated Coronary Artery Bypass Surgery a Outcome Endpoints No. (%) Original Logistic EuroSCORE (AUC, 95% CI) Finnish Modified EuroSCORE (AUC, 95% CI) EuroSCORE II (AUC, 95% CI) 30-day mortality 38 (3.7) ( ) ( ) ( ) In-hospital mortality 28 (2.7) ( ) ( ) ( ) One-year mortality 88 (10.5) b ( ) ( ) ( ) Stroke 25 (2.4) ( ) ( ) ( ) Low cardiac output syndrome 127 (12.4) ( ) ( ) ( ) Prolonged use of inotropes 302 (29.4) ( ) ( ) ( ) De novo dialysis 21 (2.0) ( ) ( ) ( ) Resternotomy for mediastinitis 18 (1.8) ( ) ( ) ( ) Resternotomy for bleeding 57 (5.6) ( ) ( ) ( ) Intensive care unit stay 5 days 117 (11.4) ( ) ( ) ( ) Combined endpoint 377 (36.7) ( ) ( ) ( ) a The predictive ability of the original logistic EuroSCORE, its modified finnish version, and EuroSCORE II in predicting postoperative complications is defined as the area under the receiver operating characteristics curve. b Analysis included 842 patients with a possible follow-up longer than 1 year; Combined endpoint: 30-day death, stroke, prolonged inotropes, low cardiac output syndrome, mediastinitis, de novo dialysis, or intensive care unit stay 5 days). AUC area under the receiver operating characteristics curve; CI confidence interval; EuroSCORE European System for Cardiac Operative Risk Evaluation.

5 1934 BIANCARI ET AL Ann Thorac Surg EUROSCORE II 2012;93: Fig 4. Kaplan-Meier estimate of overall survival after coronary artery bypass surgery according to quintiles of European system for cardiac operative risk evaluation II (EuroSCORE II) (log-rank test: p ). Late Survival The mean length of follow-up was years. Four-year overall survival rate was 84.3% (56 patients entering this interval). The ROC curve analysis showed that these 3 risk scoring methods performed equally well in predicting 1-year mortality (Table 2). Kaplan-Meier analysis showed a marked decrease in overall survival in increasing quintiles of EuroSCORE II (log-rank test: p , Fig 4). Patients in the highest quintile of Euro- SCORE II had decreased survival, particularly during the first postoperative year. Patients with a EuroSCORE II 10% or greater had a significantly poorer 4-year overall survival (67.0% vs 86.0%, p ). Comment EuroSCORE is one of the most successful stories of modern adult cardiac surgery and, without any doubt, contributed significantly to improve the quality of clinical research and patients care. Recently, its ability to stratify the operative risk has been criticized as it has been shown to markedly overestimate the individual patient risk. This may severely affect any critical assessment of clinical results and may lead to a false sense of reassurance, underperformance may go undetected, and patient welfare may be compromised [10]. Because of this, Euro- SCORE investigators recently developed a revised version, the EuroSCORE II, which was kept as simple as the original. This study confirms that, in patients undergoing isolated CABG, its simplicity is also associated with a satisfactory discriminatory ability to predicting operative mortality. Importantly, it is more accurate than the previous version in predicting other postoperative complications and this makes it a valuable assessment tool of the main postoperative adverse events occurring after adult cardiac surgery. It is important that the present findings suggest that EuroSCORE II is particularly accurate in predicting the operative mortality risk of high-risk patients, a weakness area of the original EuroSCORE. In fact, EuroSCORE II optimally performed in its 3 highest quintiles, the expectto-observed ratios herein having ranged from 1.05 to 1.17 (Fig 2). However, this new risk scoring method did not perform equally well in predicting operative mortality in low-risk patients. However, this may be due to a higher prevalence of comorbidities in this validation dataset (Table 1). Therefore, low-risk patients are underrepresented in this series and EuroSCORE not adequately validated in low-risk patients. This likely does not decrease its value as mortality in low-risk patients may not be easily predictable because it is not always strictly related to a patient s preoperative conditions [11]. Furthermore, the predicted mortality rates by EuroSCORE II in low-risk patients still fall within acceptable levels (herein, the mean EuroSCORE II was 0.8% and 1.3% in the 2 lowest quintiles). It is our opinion that better grading of the severity of renal failure and of pulmonary artery pressure might have contributed to the observed improved discriminatory ability of EuroSCORE II. We previously showed that estimated glomerular filtration rate as calculated by the Modification of Diet in Renal Disease (MDRD) formula is a better predictor of adverse events than serum creatinine [6, 12]. Despite that the MDRD formula for calculation of estimated glomerular filtration rate is considered the most reliable among currently available ones [13], EuroSCORE investigators adopted an old formula proposed in the 1970s by Cockroft and Gault [14], which herein performed better than the MDRD formula. This might have significantly strengthened the predictive ability of EuroSCORE II. The introduction of 3 different degrees of pulmonary artery hypertension (p 0.001) instead of the previous dichotomous one (p 0.533) may have also significantly contributed to improve this risk scoring method. The EuroSCORE II has somewhat limited our possibilities to include a larger number of patients in the present analysis. In fact, we could not include a few patients with left ventricular ejection fraction less than 0.30, as often this parameter was not better quantified below this value. Furthermore, we were able to include only those patients with preoperative data on clinical status as graded by the New York Heart Association classification, but we did not have the possibility to retrospectively grade the other patients. We decided to grade angina at rest as nitrates infusion at operating room arrival as the status of angina may markedly vary during the preoperative period. Indeed, very often patients with acute coronary syndrome and requiring nitrates infusion at hospital arrival may recover promptly and become asymptomatic before surgery. The herein adopted criteria of nitrates infusion at operating room arrival may therefore restrict this definition of angina at rest to only those patients suffering severe, unstable angina.

6 Ann Thorac Surg BIANCARI ET AL 2012;93: EUROSCORE II 1935 In conclusion, the present results suggest that the EuroSCORE investigators had succeeded in improving the discriminatory ability of the original EuroSCORE, at least in patients undergoing isolated CABG; its ability to predict 30-day mortality in high-risk patients is of particular importance. The EuroSCORE II is also a good predictor of immediate postoperative adverse events as well as of late survival. The discriminatory ability of this new risk scoring method should be confirmed in cardiac procedures other than isolated CABG. References 1. Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16: Ngaage DL. The EuroSCORE has served us well. Eur J Cardiothorac Surg 2010;38: Lefèvre T, Kappetein AP, Wolner E, et al. One year follow-up of the multi-centre European PARTNER transcatheter heart valve study. Eur Heart J 2011;32: Biancari F, Kangasniemi OP, Mahar AAM, et al. Changing risk of patients undergoing coronary artery bypass surgery. Interact Cardiovasc Thorac Surg 2009;8: Nilsson J, Algotsson L, Höglund P, Lührs C, Brandt J. Comparison of 19 pre-operative risk stratification models in open-heart surgery. Eur Heart J 2006;27: Nissinen J, Biancari F, Wistbacka JO, et al. Is it possible to improve the accuracy of EuroSCORE? Eur J Cardiothorac Surg 2009;36: Collart F, Feier H, Kerbaul F, et al. Valvular surgery in octogenarians: operative risk factors, evaluation of Euro- SCORE and long term results. Eur J Cardiothorac Surg 2005;27: Nashef SA, Roques F, Sharples LD, et al. EuroSCORE II. Eur J Cardiothorac Surg 2011; in press. 9. Brier GW. Verification of forecasts expressed in terms of probabilities. Monthly Weather Review 1950;78: Choong CK, Sergeant P, Nashef SA, Smith JA, Bridgewater B. The EuroSCORE risk stratification system in the current era: how accurate is it and what should be done if it is inaccurate? Eur J Cardiothorac Surg 2009;35: Freed DH, Drain AJ, Kitcat J, Jones MT, Nashef SA. Death in low-risk cardiac surgery: the failure to achieve a satisfactory cardiac outcome (FIASCO) study. Interact Cardiovasc Thorac Surg 2009;9: Kangasniemi OP, Mahar MA, Rasinaho E, et al. Impact of estimated glomerular filtration rate on the 15-year outcome after coronary artery bypass surgery. Eur J Cardiothorac Surg 2008;33: Levey AS, Greene T, Kusek J, Beck G. A simplified equation to predict glomerular filtration rate from serum creatinine. J Am Soc Nephrol 2000;11:155A. 14. Cockcroft DW, Gault MH. Prediction of creatinine clearance from serum creatinine. Nephron 1976;16: INVITED COMMENTARY The authors [1] have compared the recently released Euro- SCORE II model against the original EuroSCORE model using a local patient population. Recognizing that the original EuroSCORE model consistently over-estimated operative risk, the authors used a sample from their local population to compare results of the two EuroSCORE models as well as the Finnish modified version. They have shown that in high-risk patients, the EuroSCORE II predicted risk more accurately than the EuroSCORE. Comparing EuroSCORE II results against EuroSCORE results in a real-world population is necessary to determine whether the shortcomings of the original model are adequately addressed. The authors are to be commended for recognizing this and conducting a study to shed light on the performance of the new model. With the emerging use of transcatheter valve therapy (TVT), risk assessment models such as this take on an even more important role than in the past. Comparison of EuroSCORE II against The Society of Thoracic Surgeons (STS) risk models will also be particularly important in the search for the most meaningful models to assess risk in the TVT population. It should be noted that the STS/ACC TVT Registry plans to develop new models based exclusively on the TVT population in order to tailor predictions to this highly select group of patients. Several models will then vie for the most accurate assessment of the TVT population. EuroSCORE will clearly be in the competition; therefore, additional validation of Euro- SCORE II will be necessary in larger populations and in other geographic regions. Fred H. Edwards, MD University of Florida W 8th St Jacksonville, FL fred.edwards@jax.ufl.edu Reference 1. Biancari F, Vasques F, Mikkola R, Martin M, Lahtinen J, Heikkinen J. Validation of EuroSCORE II in patients undergoing coronary artery bypass surgery. Ann Thorac Surg 2012;93: by The Society of Thoracic Surgeons /$36.00 Published by Elsevier Inc doi: /j.athoracsur