The Aristotle Comprehensive Complexity Score Predicts Mortality and Morbidity After Congenital Heart Surgery

Transcription

1 ORIGINAL ARTICLES: SURGERY: The Annals of Thoracic Surgery CME Program is located online at To take the CME activity related to this article, you must have either an STS member or an individual non-member subscription to the journal. The Aristotle Comprehensive Complexity Score Predicts Mortality and Morbidity After Congenital Heart Surgery Mirela Bojan, MD, MS, Sébastien Gerelli, MD, Simone Gioanni, MD, Philippe Pouard, MD, and Pascal Vouhé, MD, PhD Departments of Pediatric Cardiac Anesthesiology and Pediatric Cardiac Surgery, Necker Hospital, and Paris Descartes University, School of Medicine, Paris, France Background. The Aristotle Comprehensive Complexity (ACC) score has been proposed for complexity adjustment in the analysis of outcome after congenital heart surgery. The score is the sum of the Aristotle Basic Complexity score, largely used but poorly related to mortality and morbidity, and of the Comprehensive Complexity items accounting for comorbidities and procedure-specific and anatomic variability. This study aims to demonstrate the ability of the ACC score to predict 30-day mortality and morbidity assessed by the length of the intensive care unit (ICU) stay. Methods. We retrospectively enrolled patients undergoing congenital heart surgery in our institution. We modeled the ACC score as a continuous variable, mortality as a binary variable, and length of ICU stay as a censored variable. For each mortality and morbidity model we performed internal validation by bootstrapping and assessed overall performance by R 2, calibration by the calibration slope, and discrimination by the c index. Results. Among all 1,454 patients enrolled, 30-day mortality rate was 3.4% and median length of ICU stay was 3 days. The ACC score strongly related to mortality, but related to length of ICU stay only during the first postoperative week. For the mortality model, R , calibration slope 0.98, c index 0.86, and 95% confidence interval was 0.82 to For the morbidity model, R , calibration slope 0.94, c index 0.64, and 95% confidence interval was 0.62 to Conclusions. The ACC score predicts 30-day mortality and length of ICU stay during the first postoperative week. The score is an adequate tool for complexity adjustment in the analysis of outcome after congenital heart surgery. (Ann Thorac Surg 2011;91: ) 2011 by The Society of Thoracic Surgeons The analysis of congenital heart surgery outcomes is challenging owing to the large number and to the varying complexity of surgical procedures. One method proposed for complexity-adjusted outcome analysis is the Aristotle Complexity Score [1]. It offers a precise and universal scoring system that can be applied to all congenital cardiac procedures performed worldwide. It was developed by the Aristotle Committee, involving 50 experienced surgeons from around the world. It reflects complexity through three components: the potential for mortality, the potential for morbidity, and surgical technical difficulty. The Aristotle Basic Complexity (ABC) score has been used since 2002 both by The Society of Thoracic Surgeons (STS) and the European Association of Cardiothoracic Surgery (EACTS) in their reports of outcome after congenital heart surgery. Nevertheless, analyses of their databases showed that the ABC score has limited ability to predict mortality and morbidity [2]. The Aristotle Committee also developed the Aristotle Comprehensive Complexity (ACC) score, a sum of the ABC score and Comprehensive Complexity score, which takes into account comorbidities and procedure-specific and anatomic dissimilarities. The ACC score is still under development, but has already been shown to relate to postoperative outcome in the Norwood procedure [3] and in a German case-mix population [4]. However, there has been no statistical validation of the ACC score to date. The present study aims to assess the ability of the ACC score to predict 30-day mortality and morbidity in our institution. Accepted for publication Oct 26, Address correspondence to Dr Bojan, Department of Pediatric Anaesthesia, Necker Enfants Malades Hospital, 149, rue de Sèvres Paris, France; mirela.bojan@nck.aphp.fr. Material and Methods This project was reviewed and approved by the ethics committee of the French Society of Thoracic and Cardio by The Society of Thoracic Surgeons /$36.00 Published by Elsevier Inc doi: /j.athoracsur

2 Ann Thorac Surg BOJAN ET AL 2011;91: ARISTOTLE COMPREHENSIVE COMPLEXITY SCORE 1215 vascular Surgery, which waived the need for patient consent. Scoring System The methodological details of each scoring system have been previously described [1, 5, 6]. The Aristotle Committee involved experts from 50 centers in 23 countries. They developed a scoring system accounting for the potential for mortality, potential for morbidity, and technical difficulty for every congenital procedure, each contributing up to 5 points to a continuous score ranging from 1.5 to 15. Four levels of complexity were identified. The comprehensive score introduces patient-adjusted complexity. Anatomic variants (76 items), associated procedures (85 items), and age (6 groups) accounted for procedure-dependent factors. Nonspecific patient condition (prematurity, weight), clinical factors (31 items), extracardiac factors (39 items), or surgical factors (8 items) accounted for procedure-independent factors. The comprehensive score added 10 points and two levels of complexity to the basic score scale. Patient Population All patients referred to our unit between January 1, 2007, and March 31, 2009, were eligible if they had undergone a congenital heart procedure identified by the International Heart Surgery Nomenclature [7]. Only index procedures were counted, and cases with combinations of cardiac procedures performed simultaneously were placed into the category defined by the highest basic complexity value. Data Sources The pediatric cardiovascular surgery and anesthesiology databases of Necker Hospital are prospective databases maintained daily by clinical staff. They contained the ABC score for each performed procedure, assigned by the surgeon in charge of each patient. Both databases contained all data required to calculate the ACC score according to its latest version (Aristotle Institute, Denver, CO; available at Accessed August 28, 2009). The ACC score was the sum of the ABC score plus the procedure-dependent and procedure-independent factors. Both databases contained all information concerning mortality and morbidity. The study focused on two clinical outcomes: (1) 30-day mortality, in-hospital or after discharge from the intensive care unit (ICU) or hospital, regardless of cause; and (2) morbidity, defined by length of ICU stay. Statistical Analysis Nonparametric methods were used to assess the predictive ability of the score. Internal validation was performed by bootstrapping with 200 repeated samples. The analysis focused on overall performance, calibration (ie, the distance between actual and predicted outcome) and discrimination (ie, ability to separate subjects who experienced the outcome event from the others) [8]. The ACC score was modeled as a continuous variable, 30-day mortality as a binary variable, and length of ICU stay as a censored variable. As several patients had more than one index operation during the study period, coefficient estimates and the adequacy index for the mortality model were compared with Huber-White robust sandwich estimates of the variance covariance matrix, a statistical procedure penalizing for clustering by patient [9]. We plotted log-odds ratios (OR) and log-hazard ratios to visualize the strength of the relationship between the score and outcome [9]. As a measure of overall accuracy of prediction we calculated the R 2 and the Nagelkerke s R 2 (ie, the proportion of variability in the dataset that is accounted for by the model). Calibration was assessed by plotting actual outcome versus predicted outcome (ideally with predictions on the 45-degree line) and using bootstrapping repeated samples to correct for overfitting. The calibration slope was calculated as a measure of calibration (ideally 1). We assessed discrimination by the c index, a rank correlation index calculated as the proportion of pairs of subjects with the responder having a higher predicted probability of the response than the nonresponder. A c of 0.5 indicates random prediction, whereas c of 1 indicates perfect prediction; 0.8 is the cutoff for a useful prediction model. The 95% confidence intervals (95% CI) were estimated from 200 repeated samples. Two supplementary models were generated for mortality. The first modeled the score as a six-level categorical variable to assess the consistency of the six complexity levels and tested for heterogeneity in the ORs using Breslow-Day statistic. Finally, a comparison of estimated risk versus expert consensus was done for the five most commonly recorded comprehensive complexity factors. Measured risk was identified using the coefficient of the multivariable model including ABC and the five factors, and the 95% CI of the ORs were compared with the expert consensus scoring. No adjustment was made for year of operation. The R statistical package, Design, Hmisc, boot, and ROCR libraries ( were used for all analyses. Results An ACC score could be assigned for 1,454 (99%) of the 1,468 index operations recorded in the database. Surgical closure of the patent ductus arteriosus was not performed in premature neonates during the study period. A total of 1,368 patients were enrolled: 83 patients (6.1%) underwent more than one index operation, and 3 patients (0.2%) underwent three index operations. The characteristics of the patients and crude outcomes are summarized in Table 1. Twenty-seven patients required reoperations within 24 hours for postoperative hemorrhage or a second operation, and 59 patients required the following delayed reoperations within the same hospital stay: pericardial drainage (n 8), mediastinitis (n 10), chylothorax drainage (n 13), and other surgical procedures (n 28).

3 Table 1. Characteristics of the Study Population Characteristic Levels of the ACC Score 1(n 257) 2 (n 261) 3 (n 309) 4 (n 476) 5 (n 124) 6 (n 27) Age (days) Mean (SD) 1,105.9 (1,488.3) 1,078.9 (1,723.8) 1,058.3 (1,755.6) 1,095.2 (1,819.6) (1100.0) (562.6) Median (IQR) 363 (83, 1,751) 192 (91, 1,644) 169 (42, 1,312) 176 (12, 1,425) 19 (7, 143) 20 (7, 129) n (%) 30 days 37 (14.4) 34 (13.0) 73 (23.6) 154 (32.3) 70 (56.4) 15 (55.6) 30 days 1 year 92 (35.8) 125 (47.9) 117 (37.9) 132 (27.7) 31 (25.0) 6 (22.2) 1 18 years 127 (49.4) 98 (37.5) 112 (36.2) 185 (38.9) 22 (17.7) 6 (22.2) 18 years 1 (0.4) 4 (1.5) 7 (2.3) 5 (1.0) 1 (0.8) 0 Length of ICU stay (days) Mean (SD) 3.2 (3.8) 3.9 (4.9) 5.0 (7.8) 5.9 (7.4) 8.7 (8.8) 9.6 (5.1) Median (IQR) 2 (1, 4) 2 (1, 4) 3 (2, 5) 4 (2, 6) 6 (3, 1) 8 (7, 12.5) Length of mechanical ventilation (days) Mean (SD) 0.9 (2.7) 1.2 (3.2) 1.7 (4.6) 2.6 (5.4) 4.9 (6.3) 7.4 (5.4) Median (IQR) 0 (0, 0) 0 (0, 1) 0 (0, 1) 0 (0, 2) 3 (1, 7) 7 (3.5, 10) Mortality, n (%) 0 1 (0.4) 1 (0.3) 20 (4.2) 20 (16.1) 8 (29.6) Reoperation n, (%) 8 (3.1) 15 (5.7) 15 (4.8) 31 (6.5) 13 (10.5) 4 (14.8) Most common procedures, n VSD repair, 83 ASD repair, 52 Coarctation repair, 29 Pacemaker procedure, 23 PA banding, 20 PAVC repair, 10 VSD repair, 69 TOF repair, 57 Coarctation repair, 41 PA banding, 12 Aortic stenosis, subvalvar repair, 10 BDCPA, 8 TOF repair, 42 ASO, 39 VSD repair, 33 CAVC repair, 25 BDCPA,18 Coarctation repair, 17 TAPVC repair, 13 ASO, 54 TCPA, 39 Pulmonary atresia, VSD, 31 ASO and VSD repair, 29 Conduit reoperation, 22 REV, 21 BDCPA, 15 Konno procedure, 11 Anomalous origin of coronary artery repair, 10 Truncus arteriosus repair, 10 Interrupted aortic arch repair, 7 Pulmonary atresia, VSD, 15 ASO, 13 ASO and VSD repair, 11 TAPVC repair, 11 Norwood procedure, 10 MBTS, 7 ASO and VSD repair, 4 Norwood procedure, 3 Interrupted aortic arch repair, 4 ACC Aristotle Comprehensive Complexity; ASD atrial septal defect; ASO arterial switch operation; BDCPA bidirectional cavopulmonary anastomosis; CAVC common atrioventricular canal; ICU intensive care unit; IQR interquartile range; MBTS modified Blalock-Taussig shunt; PA pulmonary artery; PAVC partial atrioventricular canal; REV Réparation a l Etage Ventriculaire procedure; SD standard deviation; TAPVC total anomalous pulmonary venous connection; TCPA total cavopulmonary anastomosis; TOF tetralogy of Fallot; VSD ventricular septal defect BOJAN ET AL Ann Thorac Surg ARISTOTLE COMPREHENSIVE COMPLEXITY SCORE 2011;91:

4 Ann Thorac Surg BOJAN ET AL 2011;91: ARISTOTLE COMPREHENSIVE COMPLEXITY SCORE 1217 Table 2. Surgical Procedures Performed During the Study Period Surgical Procedures n (%) VSD repair 208 (14.3) TOF repair, no ventriculotomy 133 (9.15) Coarctation repair 119 (8.18) ASO 109 (7.50) ASD repair 63 (4.33) CAVC repair 52 (3.58) TCPC, external conduit, fenestrated 46 (3.16) ASO and VSD repair 44 (3.02) PA banding 44 (3.02) Bidirectional Glenn 42 (2.89) Shunt, systemic to pulmonary 36 (2.48) Pacemaker procedure 35 (2.41) Conduit reoperation 30 (2.06) TAPVC repair 28 (1.92) Valvuloplasty, aortic 27 (1.86) Pulmonary atresia VSD, RV-PA conduit 27 (1.86) Partial atrioventricular canal repair 25 (1.72) Aortic stenosis, subvalvar, repair 25 (1.72) REV 24 (1.65) Pulmonary atresia VSD repair 23 (1.58) Valvuloplasty, mitral 19 (1.31) Norwood procedure 17 (1.17) PAPVC repair 16 (1.10) Konno procedure 15 (1.03) PA plasty, main and branch 15 (1.03) Aortic stenosis, supravalvar, repair 15 (1.03) Anomalous origin of coronary artery from 14 (0.96) pulmonary artery repair Valve replacement, mitral 14 (0.96) Truncus arteriosus repair 13 (0.89) Aortic arch repair 13 (0.89) PDA closure, surgical 12 (0.82) Absent pulmonary valve repair 12 (0.82) Interrupted aortic arch repair 9 (0.62) Transplant, heart 9 (0.62) Ross-Konno procedure 7 (0.48) Hemitruncus repair 7 (0.48) Arterial switch, VSD closure and aortic 6 (0.41) arch repair Valvuloplasty, pulmonic 6 (0.41) Pericardial drainage procedure 6 (0.41) Aortic root replacement 6 (0.41) DORV repair 5 (0.34) Ross procedure 5 (0.34) ECMO procedure 5 (0.34) Others 38 (2.61) ASD atrial septal defect; ASO arterial switch operation; CAVC common atrioventricular canal; DORV double outlet right ventricle; ECMO extracorporeal membrane oxygenation; PA pulmonary artery; PAPVC partial anomalous pulmonary venous connection; PDA patent ductus arteriosus; REV Réparation a l Etage Ventriculaire procedure; RV right ventricle; TAPVC total anomalous pulmonary venous connection; TOF tetralogy of Fallot; VSD ventricular septal defect. The analysis involved 79 cardiovascular procedures (Table 2). The following were identified among the procedure-independent factors: (1) 3 general factors: weight less than 2.5 kg, prematurity, and severe prematurity; (2) 21 clinical factors: 6 cardiac, 2 pulmonary, 2 infectious, 3 gastrointestinal, 1 hematologic, 2 renal, 4 neurologic, and 1 endocrine; (3) 29 extracardiac factors: 1 central nervous system, 7 respiratory, 4 gastrointestinal, 3 renal, 7 genetic and chromosomal, 4 spatial anomalies, and 3 other; and (4) 4 surgical technique factors. We identified 149 procedure-dependent factors. There was no significant difference between estimates of the mortality model and the Huber-White sandwich estimates: coefficient estimates, (standard deviation) versus , respectively; likelihood ratio (LR), ; 1 degree of freedom for both. Thus, the number of index operations per patient was small and did not impede on independence assumptions. Consequently, only unadjusted logistic model estimates were used for further analysis. Mortality The 30-day mortality rate was 3.4%. When modeled as a continuous variable, the ACC score was strongly related to 30-day mortality (regression coefficient, ; p ; and R showed good overall performance). When plotting the log ORs of 30-day morality versus the ACC score expanded by a restricted cubic spline function, as shown in Figure 1, the ACC score was seen to have a strong effect on 30-day mortality; the log-ors were found to be less than 0, and the ORs did not equal 1 for nearly every value of the score [9]. The calibration curve shown in Figure 2, using the loess smoother, and the calibration slope of 0.98 indicate good calibration. The model slightly overestimated the Fig 1. Estimated relationship between the Aristotle Comprehensive Complexity (ACC) score and the log-odds ratios (OR) for 30-day mortality. Solid line is a spline fit, knots are marked with arrows. Thirty-day mortality was strongly related to the Aristotle Comprehensive Complexity score. (CI confidence interval.)

5 1218 BOJAN ET AL Ann Thorac Surg ARISTOTLE COMPREHENSIVE COMPLEXITY SCORE 2011;91: Fig 2. Calibration curve of the regression model describing 30-day mortality as a function of the Aristotle Comprehensive Complexity score. Overall calibration was good, predicted mortality was overestimated for high actual mortality rates. The bias-corrected curve was constructed by bootstrap estimates. Triangles represent the actual mortality rate within deciles of the predicted mortality rate, a graphic illustration of the Hosmer-Lemeshow goodness-of-fit test. Rug plot at the top of the graph indicates the distribution of predicted probabilities. probability of 30-day mortality when actual mortality rate exceeded For the sake of completeness we also plotted actual mortality by deciles of predicted mortality. The receiver operating characteristics curve shown in Figure 3 indicates good discrimination; the c index was 0.864, with 95% CI of to Fig 3. Receiver operating characteristics curve illustrating the discrimination accuracy of the Aristotle Comprehensive Complexity score for 30-day mortality. The c index was 0.864, bootstrap calculated 95% confidence intervals were to Fig 4. Odds ratio (OR) for 30-day mortality within the levels of the Aristotle Comprehensive Complexity (ACC) score. Odds ratios are shown with 95% confidence intervals (CI). The first, second, and third level were combined into a composite level for analysis. Breslow-Day statistics show a linear trend ( ; df, 3; p 0.337). When analyzing the levels of the ACC score, the first and second levels were combined with the third level into a composite level because of their low mortality rates. The fourth level was chosen as the reference. The unadjusted ORs for mortality were 0.05, 4.33, and 9.58 for the composite level and the fifth and sixth levels versus the fourth level, respectively (95% CI, 0.01 to 0.18, 2.24 to 8.38, 3.59 to 24.05, respectively; overall p 0.001), as shown in Figure 4. The Breslow-Day statistic was not significant ( ; 3 degrees of freedom; p 0.337) and confirmed a linear trend. The ORs for the factors of the comprehensive score chosen to compare measured risk versus expert consensus risk were weight less than 2.5 kg (n 43), OR, 1.94; 95% CI, 0.62 to 6.06; prematurity 32 to 35 weeks (n 53), OR, 1.22; 95% CI, 0.38 to 3.95; mechanical ventilation to treat cardiorespiratory failure (n 75), OR, 2.57; 95% CI, 1.06 to 6.25; Down syndrome (n 64), OR, 2.69; 95% CI, 0.87 to 8.35; and redo sternotomy 1, 2, or 3 (n 335), OR, 0.57; 95% CI, 0.23 to All had an expert consensus risk of 2, except for Down syndrome, which had a consensus risk of 1. All except redo sternotomy 1, 2, or 3 had the expert consensus risk included in the 95% CI of measured risk. Morbidity Median length of ICU stay among the 30-day survivors was 3 days; interquartile range, 2 to 6 days. The median length of ICU stay in nonsurvivors was 5 days, interquartile range, 1 to 9 days. Analysis of length of stay was first conducted including only 30-day survivors, than both 30-day survivors and nonsurvivors. We will first focus on analysis including 30-day survivors.

6 Ann Thorac Surg BOJAN ET AL 2011;91: ARISTOTLE COMPREHENSIVE COMPLEXITY SCORE 1219 Fig 5. Interval-specific log-hazard ratios (HR) of Aristotle Comprehensive Complexity score for the length of intensive care unit (ICU) stay. Length of intensive care unit stay was strongly related to the Aristotle Comprehensive Complexity score during the first postoperative week. (CI confidence interval.) Overall length of ICU stay was weakly related to ACC score (coefficient, ; p ; Nagelkerke s R ). In Figure 5, showing the interval-specific log-hazard ratios of the score for the length of ICU stay, the ACC score has a strong effect on length of ICU stay during the first postoperative week. After day 7, the log-hazard ratio was close to 0 and the hazard ratio close to 1. For calibration, Kaplan-Meier estimated survival was plotted versus Cox predicted survival for eight groups of patients on the third postoperative day. The curve, shown in Figure 6, indicates good calibration for actual probabilities greater than The model overestimated by up to 15% the probability of a length of stay of 3 days in patients with an actual probability of less than In other words, it underestimated length of stay in patients with a long length of stay. The calibration slope was Harrell s overall c statistic, a generalization of the c index for censored data, was (95% CI, to 0.660) [9]. When including both 30-day survivors and nonsurvivors, there was no significant change in the result, the overall performance of the model remained low (Nagelkerke s R ), the calibration curve suggested a similar relationship between the ACC score and length of ICU stay, and the discrimination index was not significantly different (0.638; 95% CI, to 0.656). Comment The ABC score has been used since 2002 by both the STS and the EACTS in their yearly analysis and reporting of outcomes. O Brien and colleagues [2] analyzed 35,862 operations from the STS and EACTS congenital databases between January 1, 2002, and December 31, 2004, and showed the ability of the ABC score to discriminate between low-risk and high-risk congenital procedures. However, the results fell below expectations because the performance of the score as a predictor of mortality and prolonged stay was blunted with an area under the receiver operating characteristics curve of only 0.70 for mortality and 0.65 for overall postoperative length of stay of 21 days. Sinzobahamvya and associates [3] showed the usefulness of the ACC score as a predictor of outcome after the Norwood operation. Mortality increased sixfold when the score was greater than 20. The same authors recently analyzed a case-mix population of 758 patients and demonstrated good correlation between the ACC score and mortality, length of ICU stay, and surgical technical difficulty [4]. Furthermore, there was an almost perfect correlation between the ACC score and the German cost-weight system, implemented as the basis for hospital reimbursement [10]. Besides, the ACC score has been shown to correlate to cardiac output early after the Norwood procedure [11]. Hence, the ACC score appeared to be a useful covariate for analyzing outcome after congenital heart surgery. Surprisingly, it has never been used to analyze outcome in the STS and EACTS databases, but a further modification of the score, based on the outcome data contained in these databases, is already underway [12]. When analyzing the performance of the ACC score, Heinrichs and coworkers [4] modeled each component of the score separately and conducted separate analysis of procedure-specific mortality and morbidity. However, Fig 6. Calibration curve for the Cox model describing the length of the intensive care unit (ICU) stay as a function of the Aristotle score. Overall calibration of the model was good, but predicted length of stay was overestimated for short actual length of stay. Validation of the calibration curve was performed by bootstrap repeat sampling in six prognostic groups. Dots represent the apparent predictions for each group, X s represent bootstrap-corrected estimates.

7 1220 BOJAN ET AL Ann Thorac Surg ARISTOTLE COMPREHENSIVE COMPLEXITY SCORE 2011;91: the score was developed as a sum of these components, and the resulting value was intended to be used as a continuous variable for adjustment in outcome analysis. We used a similar approach, enabling us to avoid the difficulty related to the large number of surgical procedures, one of the major difficulties encountered in congenital cardiac surgery scoring systems [2]. The analysis was conducted according to the recommendations of Steyerberg and associates [8] for validation of predictive models. With its multitude of items, the score contains a huge amount of information, and the assumption was made that it already contained all the information required for prediction. Therefore, mortality and morbidity were modeled as a function of the nude score, without further adjustments, and if this assumption is true, the generated models would have good predictive ability. Mortality The ACC score was strongly related to 30-day mortality. The unadjusted mortality model was well calibrated and demonstrated the good predictive ability of the score. There was a slight overestimation of mortality when actual mortality rates exceeded The paucity of cases with such a high observed mortality rate may account for the calibration error in this area. However, the calibration index was high. Discrimination was very good; the c index was (95% CI, to 0.906), much better than 0.70 found for the basic score by both O Brien and colleagues [2] when analyzing the STS and EACTS databases, and by Al Radi and associates [13] when analyzing a Canadian database. Thus, the assumption that the score contained enough information to make a good prediction of 30-day mortality was verified. The ACC score is therefore a more useful covariate for case-mix adjustment in 30-day mortality analyses than the ABC score. The expert chosen division of the ACC into six complexity levels was empiric [5]. The small sample size allowed only incomplete analysis, with levels 1, 2, and 3 merged into a composite level, but the not significant Breslow-Day statistic suggested consistency of the partition, with constant increase in risk across complexity levels. Heinrichs and coworkers [4] reported similar results with an excellent Spearman s correlation coefficient of calculated between complexity levels and mortality rates. For the five most frequently recorded complexity factors, measured risk assessed by regression-based ORs and expert consensus risk were compared. In all but one, the 95% CI of the measured risk included the expert estimated risk. However, this only allowed intuitive comparisons. When using a regression approach, risk is identified as incremental. In contrast, the comprehensive score provides an additive risk. Besides, as in previous studies, our efforts were hindered by the large number of procedures with small patient sample sizes, making it impossible to analyze each comprehensive complexity item. Morbidity The ACC score performed less well for the prediction of morbidity and was only weakly related to overall length of ICU stay. However, Figure 5 shows a strong effect of the score during the first postoperative week. The calibration curve of the Cox model in Figure 6 gave similar results, with good calibration of the model for an actual probability of having a length of ICU stay of 3 days of greater than Thus, the ACC score could accurately predict patients with a short length of stay, but not those with a long length of stay; up to 15% of predictions were wrong in patients with a long ICU stay. Variables other than those included in the ACC score may relate to length of stay in patients requiring a long duration of ICU care. Discrimination was assessed by Harrell s overall c statistic, a generalization of the c index for censored data, indicating the proportion of all pairs of subjects that could be ordered such that the subject with a higher predicted ICU stay was the one who stood longer. Discrimination was less than in the mortality model. Brown and associates [14] conducted an analysis of factors related to the length of ICU stay after congenital heart surgery in children. The cutoff for a long length of stay was 14 days, the upper interquartile range of overall length of stay. Long ICU stay was multifactorial and was related to preoperative, perioperative, and postoperative factors such as operative class (a locally defined surgical complexity staging system), requirement for delayed sternal closure, reoperation, postoperative extracorporeal membrane oxygenation or renal replacement therapy, diaphragmatic palsy, chylothorax, and sepsis. These authors included postoperative factors in a complication score. Increasing complication score was related to a higher risk for long ICU stay. Similarly, a morbidity index based on the proportion of ICU stays exceeding 1 week was used by Heinrichs and coworkers [4] in their analysis of performance according to the ACC score. Their morbidity index correlated well with the incidence of postoperative complications and the duration of mechanical ventilation, ICU, and hospital stay. The occurrence of postoperative complications might also explain an ICU stay of more than 1 week in our study. Several authors have pointed out the need for an objective assessment of morbidity after congenital heart surgery [12, 15, 16]. Assessment of morbidity is still controversial despite efforts during the last decade to develop objective definitions, and no outcome variable other than length of stay has been proposed to account for morbidity. Clark and colleagues [12] proposed the assessment of morbidity using a score including length of hospital stay, mechanical ventilation, postoperative extracorporeal membrane oxygenation or ventricular assistance, and major complications such as reoperation, atrioventricular block, neurologic disorders, and renal failure. However, such a definition of morbidity has not been validated using independent databases. With respect to the initial publication, we assessed morbidity by the length of ICU stay [1].

8 Ann Thorac Surg BOJAN ET AL 2011;91: ARISTOTLE COMPREHENSIVE COMPLEXITY SCORE 1221 Limitations The main limitation of this study is that it was institution based and had a small sample size. In addition, because of a retrospective collection of variables accounting for the comprehensive complexity score from a database that was not initially designed for the study, the possibility of missing data cannot be excluded. Conclusions In conclusion, the ACC score can be used to predict 30-day mortality after congenital heart surgery. The score can also be used to predict length of ICU stay during the first postoperative week, but other factors intervene when ICU stay exceeds 1 week. The ACC score appears to be an adequate tool for complexity adjustment in the analysis of outcome after congenital heart surgery and should allow for a fair comparison of performance according to case-mix complexity. The authors would like to thank Mrs Sylvie Escolano and Dr Jean-Philippe Empana, Department of Cardiovascular Epidemiology and Sudden Death, Georges Pompidou Hospital, Paris, France, for thoughtful scientific advice on statistical analysis. References 1. Lacour-Gayet F, Clarke D, Jacobs J, et al. The Aristotle score: a complexity-adjusted method to evaluate surgical results. Eur J Cardiothorac Surg 2004;25: O Brien SM, Jacobs JP, Clarke DR, et al. Accuracy of the Aristotle Basic Complexity Score for classifying the mortality and morbidity potential of congenital heart surgery operations. Ann Thorac Surg 2007;84: Sinzobahamvya N, Photiadis J, Kumpikaite D, et al. Comprehensive Aristotle score: implications for the Norwood procedure. Ann Thorac Surg 2006;81: Heinrichs J, Sinzobahamvya N, Arenz C, et al. Surgical management of congenital heart disease: evaluation according to the Aristotle score. Eur J Cardiothorac Surg 2010;37: Lacour-Gayet F, Clarke D, Jacobs J, et al. The Aristotle score for congenital heart surgery. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu 2004;7: Lacour-Gayet F, Clarke DR; Aristotle Committee. The Aristotle method: a new concept to evaluate quality of care based on complexity. Curr Opin Pediatr 2005;17: Mavroudis C, Jacobs JP. Congenital Heart Surgery Nomenclature and Database Project: overview and minimum dataset. Ann Thorac Surg 2000;69(Suppl):S Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York, NY: Springer; 2009: Harrell FE Jr. Regression modeling strategies with applications to linear models, logistic regression, and survival analysis. New York, NY: Springer; 2001:193, , Sinzobahamvya N, Photiadis J, Arenz C, Kopp T, Hraska V, Asfour B. Congenital heart disease: interrelation between German diagnosis-related groups system and Aristotle complexity score. Eur J Cardiothorac Surg 2010;37: Li J, Zhang G, Holtby H, et al. Significant correlation of comprehensive Aristotle score with total cardiac output during the early postoperative period after the Norwood procedure. J Thorac Cardiovasc Surg 2008;136: Clarke DR, Lacour-Gayet F, Jacobs JP, et al. The assessment of complexity in congenital cardiac surgery based on objective data. Cardiol Young 2008;18(Suppl 2): Al-Radi OO, Harrell FE Jr, Caldarone CA, et al. Case complexity scores in congenital heart surgery: a comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system. J Thorac Cardiovasc Surg 2007;133: Brown KL, Ridout DA, Goldman AP, Hoskote A, Penny DJ. Risk factors for long intensive care unit stay after cardiopulmonary bypass in children. Crit Care Med 2003;31: Benavidez OJ, Gauvreau K, Del Nido P, Bacha E, Jenkins KJ. Complications and risk factors for mortality during congenital heart surgery admissions. Ann Thorac Surg 2007;84: Lacour-Gayet F, Jacobs ML, Jacobs JP, Mavroudis C. The need for an objective evaluation of morbidity in congenital heart surgery. Ann Thorac Surg 2007;84:1 2.