Major Infection After Pediatric Cardiac Surgery: External Validation of Risk Estimation Model

Major Infection After Pediatric Cardiac Surgery: External Validation of Risk Estimation Model Andrzej Kansy, MD, PhD, Jeffrey P. Jacobs, MD, PhD, Andrzej Pastuszko, MD, PhD, Małgorzata Mirkowicz-Małek, MD, PhD, Małgorzata Manowska, MD, PhD, Elżbieta Jezierska, MD, Przemyesław Maruszewski, MD, Piotr Burczyński, MD, PhD, and Bohdan Maruszewski, MD, PhD The Children s Memorial Health Institute, Warsaw, Poland, and University of South Florida, St. Petersburg, Florida Background. A multivariable risk estimation model, in which the primary outcome was major infection, was recently developed and published using The Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database. We have applied this risk estimation model to our congenital heart surgery program over a 16-year time interval to validate this risk estimation model and verify its specific risk factors for major infection. Methods. Using complete and verified data, we selected patients in whom major procedures had been classified using both Aristotle Basic Score and Risk Adjustment for Congenital Heart Surgery (RACHS-1) and created a multivariable model in which primary outcome was major infection (septicemia, mediastinitis, or endocarditis). We checked the STS risk estimation model for major infection. We also assessed the significance of the STS risk factors in our program. Results. A total of 6,314 patients were analyzed. We identified 197 (3.1%) major infections (septicemia 3%, endocarditis 0.015%, mediastinitis 0.09%). Hospital mortality, ventilation time, and length of stay were greater in patients with major infections. The following preoperative risk factors identified by the STS risk estimation model were significant in multivariate analysis in our patients: young age, high complexity, medium complexity, previous operation, and preoperative ventilation (p < 0.0001). Estimated infection risk ranged from 0.32% to 11.58%. The model discrimination was good (c index, 0.808). Risks of infections after most common congenital heart surgery procedures were similar in both studies (r s 0.952, p 0.0003). Conclusions. Our external validation study confirmed that the STS model can be used as a preoperative risk stratification tool for major infection risk at the single institutional level. (Ann Thorac Surg 2012;94:2091 6) 2012 by The Society of Thoracic Surgeons Serious postoperative infections after congenital heart surgery result in significant morbidity, and thus increase the cost of treatment and may lead to higher mortality [1 3]. Especially, septicemia, mediastinitis, and endocarditis may have an important negative impact on overall outcome of treatment [4 7]. The Society of Thoracic Surgeons (STS) has recently developed and published a multivariable risk estimation model and risk scoring system that could be used as a bedside tool for predicting the risk of major postoperative infections [8]. This model used the STS Congenital Heart Surgery Database, the largest such a registry in North America to identify risk factors for major postoperative infection in children after cardiac surgery, and the study was based on the multiinstitutional dataset of 30,078 patients treated in 48 institutions between 2002 and 2006. Barker and colleagues [8] confirmed that the following variables Accepted for publication July 26, 2012. Presented at the Poster Session of the Forty-eighth Annual Meeting of The Society of Thoracic Surgeons, Fort Lauderdale, FL, Jan 28 Feb 1, 2012. Address correspondence to Dr Kansy, The Children s Memorial Health Institute, Department for Pediatric Cardiothoracic Surgery, Al Dzieci Polskich 20, 04-830 Warsaw, Poland; e-mail: ankansy@wp.pl. were associated with increased risk of major infections: age, reoperation, preoperative length of stay longer than 1 day, preoperative respiratory support or tracheostomy, genetic abnormality, and medium or high complexity score. The aim of our study was to validate externally the STS risk estimation model and verify its specific risk factors using a single institution s complete and verified dataset. Patients and Methods The study was approved by the ethical committee of the Children s Memorial Health Institute Because the individual patients were not identified, the need for parental consent was waived. Data Source For our retrospective analysis we have used the data collected in the registry of our department for pediatric cardiac surgery at the Children s Memorial Health Institute, Warsaw, Poland. Our database originates from the European Association of Cardio-Thoracic Surgery Congenital Heart Surgery Database and uses International Nomenclature for Pediatric and Congenital Heart Sur- 2012 by The Society of Thoracic Surgeons 0003-4975/$36.00 Published by Elsevier Inc http://dx.doi.org/10.1016/j.athoracsur.2012.07.079

2092 KANSY ET AL Ann Thorac Surg OUTCOMES OF NEONATAL CARDIAC SURGERY 2012;94:2091 6 Table 1. Clinical Characteristics of the Patient Population and Univariate Analysis Variable Level Total N (6,314) Overall % Number With Infection (n 197) Percentage With Infection p Value Aristotle Basic Complexity level 1 1,290 20.9 8 0.62 0.0001 2 3,354 53.5 84 2.5 3 1,182 18.3 61 5.16 4 488 7.3 44 9.02 RACHS-1 category 1 1,519 24.7 10 0.66 0.0001 2 2,734 44.0 43 1.57 3 1,647 25.4 91 5.53 4 363 5.13 49 13.5 6 51 0.77 4 7.84 Age 30 days 1,037 15.4 95 9.16 0.0001 1 3 mo 386 5.75 34 8.81 4 12 mo 1,390 21.9 50 3.6 1 10 y 2,807 45.6 15 0.53 10 y 694 11.3 3 0.43 Weight (kg) 2.5 122 1.67 20 16.40 0.0001 2.5 4.99 1,626 24.4 133 8.18 5 4,565 73.9 43 0.94 Sex Male 3,284 52.01 109 3.32 0.34 Female 3,030 47.99 88 2.90 Operative type Bypass 5,020 79.8 140 2.79 0.003 Nonbypass 1,294 20.2 57 4.40 Procedure stratum a Low complexity 4,644 74.4 92 1.98 0.0001 Medium complexity 1,182 18.33 61 5.16 High complexity 488 7.26 44 9.02 Previous cardiac operation No 5,883 93.25 179 3.04 0.19 Yes 431 6.75 18 4.18 Preoperative ventilation No 6,145 97.65 172 2.8 0.0001 Yes 169 2.35 25 14.8 Any genetic abnormality No 5,822 92.21 181 3.11 0.86 Yes 492 7.79 16 3.25 Preoperative length of stay (days) Missing 2,572 40.73 55 2.14 0.0001 1 1,173 18.58 82 6.99 1 2,569 40.69 60 2.34 Surgery year 1995 394 6.24 5 1.27 0.012 1996 424 6.72 6 1.42 1997 398 6.30 7 1.76 1998 458 7.25 9 1.97 1999 482 7.64 7 1.45 2000 482 7.64 15 3.11 2001 425 6.73 12 2.82 2002 421 6.67 26 6.18 2003 389 6.16 9 2.31 2004 338 5.35 29 8.58 2005 368 5.83 27 7.34 2006 335 5.31 11 3.28 2007 302 4.78 9 2.98 2008 333 5.27 11 3.30 2009 393 6.23 6 1.53 2010 371 5.88 8 2.16 a High complexity is an Aristotle Basic Complexity Score 3 or RACHS-1 4; medium complexity is an Aristotle Basic Complexity Score of 3 or RACHS-1 of3to4. RACHS-1 risk adjustment for congenital heart surgery.

Ann Thorac Surg KANSY ET AL 2012;94:2091 6 OUTCOMES OF NEONATAL CARDIAC SURGERY 2093 Table 2. Full Model Variable OR (95% CI) Age 0.99 (0.99 1.0) Genetic abnormality 1.1 (0.8 1.3) Preoperative stay 1 day 0.9 (0.6 1.3) Preoperative ventilator support 3.1 (1.7 5.6) Surgery year 1.0 (0.9 1.1) Previous cardiac operation 2.4 (1.2 4.7) High complexity a 1.7 (1.0 2.7) Medium complexity a 2.5 (1.7 3.7) a High complexity is an Aristotle Basic Complexity Score 3 or RACHS-1 4; medium complexity is an Aristotle Basic Complexity Score of 3 or RACHS-1 of 3 to 4. CI confidence interval; OR odds ratio; RACHS-1 risk adjustment for congenital heart surgery. gery and a dataset the same as used by the STS Congenital Heart Surgery Database [9 11]. Patient Population We have included patients 18 years or younger at our institution who between 1995 and 2010 underwent major congenital heart surgery procedures classified using both Aristotle Basic Score (ABC) and Risk Adjustment for Congenital Heart Surgery (RACHS-1). As in the STS study [8], we have excluded all solely thoracic operations and subsequent operations within the same admission, as well as those with preoperative endocarditis or septicemia and patent ductus arteriosus ligation in neonates weighing less than 2.5 kg. Finally, 6,314 patients were enrolled into the study group. The major infections were defined as septicemia, mediastinitis, or endocarditis after surgery but before hospital discharge, or after discharge if related to the operation using the same definitions as in the STS study [8]. These definitions were adopted in 2006. Our data were reinvestigated retrospectively to confirm the incidence of major infections based on the new definition. Model Validation and Statistical Analysis The following candidate risk factors for infection after cardiac surgery, previously identified and testified in the STS study [8], were selected in our full model: complexity of procedure measured by ABC level or RACHS-1 category (grouped into categories), age at surgery, sex, year of surgery, type of operation (bypass, nonbypass), procedure stratum (low, medium, or high complexity), previous cardiac operation, preoperative ventilation, genetic abnormality, preoperative length of stay of more than 1 day. We did not include race, preoperative acidosis, shock and circulatory support, and preoperative tracheostomy as those variables were either not available or not routinely verified. Also weight was not included as a candidate variable because of its correlation with age. According to the STS study we then developed a logistic regression model including all candidate variables. To simplify procedure complexity grouping, a three-strata model was adopted from the STS study (low complexity, defined as ABC 3 and RACHS-1 3; high complexity, defined as ABC 4 or RACHS-1 5; and medium complexity, defined as all others). When there were multiple procedures per operation, they were assigned to the most complex procedure score. To the reduced model we included the candidate variables using a significance criterion of 0.05. The age was categorized into three groups (younger than 90 days, between 90 days and 3 years, and older than 3 years). Despite the fact that the year of surgery was not significant in the full model analysis, we included it in the reduced model to ensure that the weighting of patient-level risk factors would not be confounded with time. To create the bedside risk tool, as in the STS study, we multiplied each regression coefficient by 5 and rounded to the nearest integer. The year of surgery was omitted from the bedside risk tool. The risk score for each patient was defined by summing the points across risk factor. We set the relationship between risk score and infection risk using logistic regression. We validated the models internally by calculating measures of calibration and discrimination. We have also compared the rates of infections in the STS and our datasets using the eight most common procedures (ventricular septal defect repair, atrial septal defect repair, modified Blalock-Taussig shunt, tetralogy of Fallot repair, coarctation of the aorta repair, complete atrioventricular septal defect repair, bidirectional Glenn procedure, and Norwood). Statistical analysis was performed using Statistica for Windows and MedCalc Software. Descriptive statistics are reported as mean value and 95% confidence interval (CI). Comparisons between groups were made either using the unpaired Student s t test or the Mann-Whitney U test as appropriate. Discrete variables were compared with Yates 2 test. Univariate and multivariate logistic regression was used to assess the risk factors for major postoperative infection in our dataset. The calibration of the model was assessed using the Hosmer-Lemeshow test and the discrimination by calculating the c-statistic. Table 3. Reduced Model Variable OR (95% CI) Points p Value Age 90 days 33.8 (14.6 78.7) 18 0.0001 Age 90 days 3 years 7.7 (3.3 17.9) 10 0.0001 High complexity a 1.6 (1.1 2.4) 2 0.0279 Medium complexity a 2.4 (1.7 3.4) 4 0.0001 Previous cardiac operation 2.7 (1.6 4.7) 5 0.0002 Preoperative ventilator support 2.1 (1.3 3.3) 4 0.0032 a High complexity is an Aristotle Basic Complexity Score 3 or RACHS-1 4; medium complexity is an Aristotle Basic Complexity Score of 3 or RACHS-1 of 3 to 4. CI confidence interval; OR odds ratio; RACHS-1 risk adjustment for congenital heart surgery.

2094 KANSY ET AL Ann Thorac Surg OUTCOMES OF NEONATAL CARDIAC SURGERY 2012;94:2091 6 Fig 1. (A) Reduced model of predicted risk in relation to risk score. Solid line shows model estimate. Dotted line represents 95% prediction interval. (B) Distribution of study population by risk score value. Rank correlation was used to compare the infection rate in the eight most common procedures. Results From the cohort of 6,314 patients, 197 had major postoperative infections (3.1%; 3.0% septicemia, 0.09% mediastinitis, 0.016% endocarditis). Three patients had more than one type of infection. The mortality in patients who experienced major infections was 25.38% compared with 3.91% in those who did not (p 0.0001). The mortality in patients with septicemia was 25.9% (50 of 193) and in those with mediastinitis was 16.7% (1 of 6); 1 patient survived endocarditis. Postoperative length of stay in patients with major infections was 46.5 days (95% CI, 40.9 to 52.1) compared with 14 days (95% CI, 13.4 to 14.5) in those without infections (p 0.001). In univariate analysis the following variables were associated with major infection: higher ABC complexity level and higher RACHS-1 category, as well as higher procedure stratum, younger age at surgery (younger than 30 days and 1 to 3 months), lower body weight (less than 2.5 kg and 2.5 to 4.99 kg), nonbypass surgery, preoperative ventilator support, preoperative length of stay, and year of surgery. Previous cardiac operation, genetic abnormalities, and sex did not have increased risk of major infections in univariate analysis (Table 1). In multivariable analysis, in the full model, only the age, high complexity score, medium complexity score, previous cardiac operation, and preoperative ventilatory support were associated with increased infection risk (Table 2). These significant variables were included in the reduced model, on the basis of which the infection risk score was calculated (Table 3). The infection risk score ranged between 0 and 22 (mean, 9.41; 95% CI, 9.24 to 9.59). Estimated risk of infection demonstrated a nonlinear relationship with risk score and ranged from 0.32% to 11.58% (Fig 1A). Distribution of risk score is shown in Figure 1B. Fig 2. Receiver operating characteristic curve for the reduced model as predictor for major infection. Fig 3. Comparison of infection rates between The Society of Thoracic Surgeons (STS) and our data using the eight most common procedures.

Ann Thorac Surg KANSY ET AL 2012;94:2091 6 OUTCOMES OF NEONATAL CARDIAC SURGERY 2095 Table 4. Comparison of Infection Rates Between Our Data and The Society of Thoracic Surgeons Congenital Heart Surgery Database for Eight Common Procedures Procedure Number of Procedures Infection Rate Our (%) (95% CI) STS (%) Ventricular septal defect repair 1,318 1.06 (0.5 1.6) 1.3 Atrial septal defect repair 1,185 0.08 (0.008 0.2) 0.2 Modified Blalock-Taussig shunt 600 5.80 (3.9 7.7) 5.9 Tetralogy of Fallot repair 473 2.10 (0.8 3.4) 2.2 Coarctation repair, end to end extended 385 3.40 (1.5 5,1) 2.3 Complete atrioventricular septal defect repair 360 7.80 (4.9 10.5) 3.6 Bidirectional Glenn procedure 177 2.20 (0.04 4.5) 2.1 Norwood procedure 51 7.80 (0.2 15.4) 13.4 CI confidence interval; STS The Society of Thoracic Surgeons. The reduced model had good predictive ability (cindex, 0.808 95% CI, 0.798 to 0.818). Internal calibration of the risk tool was good, with close accordance between predicted and observed infection rates (goodness of fit 2 11.0; p 0.2; Fig 2). Comparison of infection rates between STS and our data using the eight most common procedures exhibited close correlation (r s 0.952; 95% CI, 0.753 to 0.992; p 0.0003; Fig 3). Table 4 presents our and the STS rates of major infection in the eight most common procedures. our experience, whereas the Norwood procedure is associated with a lower risk of infection than in the STS study. The model developed using STS Congenital Heart Surgery data has an important clinical impact as a tool for predicting serious postoperative infections and providing information that can and should be used for prevention and neutralization of the risk of this major complication. Our external validation showed that the STS model can be applied to single-institutional data. Comment Our aim was to validate the STS postoperative infection risk model and verify its specific factors for major infections after congenital heart surgery. The STS model was developed based on a large number of multicenter data coming from 48 North American institutions. This model was validated to see whether it can be applied to singleinstitutional data. The strength of our study is owing to completeness of validated data collected between 1995 and 2010. Our study has validated the STS tool to identify children at high risk of major infection after cardiac surgery. The overall risk of major postoperative infection is similar in both cohorts of patients (STS, 2.8%; our study, 3.1%). It should be noticed that our study time frame goes 15 years backward to 1995, whereas the STS database included patients operated on between 2002 and 2006. The frequency of septicemia was higher in our experience, and mediastinitis and endocarditis were rare when compared with the STS database. The weakness of our analysis is related to the fact that we could not include some important risk factors as preoperative acidosis, shock, tracheostomy, and preoperative circulatory support. These factors were either too rare or not verified. Our study confirmed that younger age, higher complexity of procedure, preoperative ventilator support, and previous cardiac procedure are associated with higher risk of major postoperative infections. This remains consistent with the STS study conclusions. Regarding the eight most common procedures, the risk of infection after complete atrioventricular septal defect repair is higher in References 1. Allpress AL, Rosenthal GL, Goodrich KM, Lupinetti FM, Zerr DM. Risk factors for surgical site infections after pediatric cardiovascular surgery. Pediatr Infect Dis J 2004;23: 231 4. 2. Nateghian A, Taylor G, Robinson JL. Risk factors for surgical site infection following open-heart surgery in Canadian pediatric population. Am J Infect Control 2004;32:397 401. 3. Shah SS, Kagen J, Lautenbach E, et al. Bloodstream infections after median sternotomy at a children s hospital. J Thorac Cardiovasc Surg 2007;133:435 40. 4. Vida VL, Leon-Wyss J, Larrazabal A, Cruz S, Castaneda AR. Mediastinitis in pediatric cardiac surgery: treatment and cost-effectiveness in low-income country. Pediatr Cardiol 2007;28:163 6. 5. Long CB, Shah SS, Lautenbach E, et al. Postoperative mediastinitis in children: epidemiology, microbiology and risk factors for Gram negative pathogens. Pediatr Infect Dis J 2005;24:315 9. 6. Tortoriello TA, Friedman JD, McKenzie ED, et al. Mediastinitis after pediatric cardiac surgery: a 15-year experience at a single institution. Ann Thorac Surg 2003;76:1655 60. 7. Di Filippo S, Delahaye F, Semiond B, et al. Current patterns of infective endocarditis in congenital heart disease. Heart 2006;92:1490 5. 8. Barker GM, O Brien SM, Welke KF, et al. Major infection after pediatric cardiac surgery: a risk estimation model. Ann Thorac Surg 2010;89:843 50. 9. International Paediatric and Congenital Cardiac Code. Available at: http://www.ipccc.net. Accessed August 13, 2012. 10. European Association of Cardio-Thoracic Surgery Congenital Database. Available at: http://www.eactscongenitaldb. org. Accessed August 13, 2012. 11. The Society for Thoracic Surgeons Congenital Heart Surgery Database. Available at: http://www.sts.org/nationaldatabase. Accessed August 13, 2012.