Validation of the acute leukemia-ebmt score for prediction of mortality following allogeneic stem cell transplantation in a multi-center GITMO cohort

Transcription

1 Received: 16 December 2016 Revised: 22 January 2017 Accepted: 3 February 2017 DOI: /ajh RESEARCH ARTICLE Validation of the acute leukemia-ebmt score for prediction of mortality following allogeneic stem cell transplantation in a multi-center GITMO cohort Roni Shouval 1,2,3,4 Francesca Bonifazi 5 Joshua Fein 1,3 Cristina Boschini 6 Elena Oldani 6 Myriam Labopin 7 Roberto Raimondi 8 Nicoletta Sacchi 9 Osamah Dabash 1 Ron Unger 4 Mohamad Mohty 10 Alessandro Rambaldi 6 * Arnon Nagler 1,3 * 1 Hematology and Bone Marrow Transplantation Division, Chaim Sheba Medical Center, Tel-Hashomer, Israel; 2 Dr. Pinchas Bornstein Talpiot Medical Leadership Program, Chaim Sheba Medical Center, Tel-Hashomer, Israel; 3 Sackler School of Medicine, Tel-Aviv University, Israel; 4 Bar-Ilan University, Ramat Gan, Israel; 5 S. Orsola-Malpighi University Hospital, University of Bologna, Bologna, Italy; 6 Hematology and BMT Unit, Ospedale Papa Giovanni XXIII, Bergamo, Italy; 7 Acute Leukemia Working Party-Paris Office, Hospital Saint-Antoine, EBMT, Paris, France; 8 Hematology and BMT Unit, Ospedale San Bortolo, Vicenza, Italy; 9 Italian Bone Marrow Donor Registry E.O. Galliera-Genova, Italy; 10 Hematology Department, Saint-Antoine Hospital, AP-HP, Universite Pierre et Marie Curie, Paris, France Correspondence Roni Shouval, Hematology and Bone Marrow Transplantation Division, Sheba Medical Center, Tel-Hashomer, Ramat-Gan, , Israel. shouval@gmail.com Abstract Predictive models may help in determining the risk/benefit ratio of allogeneic hematopoietic stem cell transplantation (HSCT) in acute leukemia (AL). Using a machine-learning algorithm we have previously developed the AL- European Society for Blood and Marrow Transplantation (EBMT) score for prediction of mortality following transplantation. We report here the first external validation of the AL-EBMT score in a cohort of AL patients from the Italian national transplantation network. A total of 1848 patients transplanted between the years were analyzed. The median age was Indications for HSCT were Acute Myeloid Leukemia (68.1%) and Acute Lymphoblastic Leukemia (31.9%). The majority of patients were in first complete remission (60.4%), and received myeloablative conditioning (81.3%). Median follow-up was 2 years. The score was well-calibrated for prediction of day 100 mortality and 2-year overall survival (OS), leukemia free survival (LFS), and nonrelapse related mortality, with corresponding area under the receiveroperator curves of 0.698, 0.651, 0.653, and 0.651, respectively. Increasing score intervals were associated with a decreasing probability of 2-year OS and LFS. The highest scoring group was associated with a hazard ratio of 3.16, 2.8, and 2.27 for 2-year OS, LFS, and NRM, respectively. In conclusion, the AL-EBMT score identified three distinct risk groups and was predictive of OS. It is a valid tool for stratifying the risk of acute leukemia patients undergoing allogeneic HSCT. 1 INTROUDCTION Allogeneic hematopoietic stem cell transplantation (HSCT) is an intensive therapy used to treat high-risk malignant hematological disorders and other life-threatening hematological diseases. 1 Nonrelapse mortality (NRM) following transplantation is attributed to graft-versus-host disease, infection, organ toxicity, and other causes. A complex network of parameters related to the patient (eg, age and comorbidities), disease (eg, remission status at transplantation and cytogenetics), donor (eg, human leukocyte antigen [HLA] matching), and procedure (eg, conditioning) all have meaningful effects. 2 4 Although the risk for NRM differs AR and AN had contributed equally to this work. considerably among individuals, the establishment of a robust prognostic system is still needed to guide patient counseling before transplant. Several prognostic scoring systems, including the European Society for Blood and Marrow Transplantation (EBMT) risk score and the Hematopoietic Cell Transplantation-Comorbidity Index (HCT-CI), are utilized in prediction of allogeneic HSCT outcomes. 3,5 These scores may aid therapeutic decision. However, their predictive capacity varies. 3,5 7 Using the Alternating Decision Tree (ADT) machine learning algorithm, 8,9 we have recently developed and internally validated a prognostic model, the Acute Leukemia (AL)- EBMT score, for prediction of mortality following allogeneic HSCT in a large cohort of AL patients (n ). 10,11 The AL-EBMT score includes 10 variables related to patient, donor, type of transplantation, and center characteristics. They Am J Hematol. 2017;92: wileyonlinelibrary.com/journal/ajh VC 2017 Wiley Periodicals, Inc. 429

2 430 SHOUVAL ET AL. are disease stage, Karnofsky Performance Status score at the time of HSCT, age, interval between diagnosis and HSCT, conditioning regimen, donor type, center annual allogeneic HSCT transplant activity, year in which the procedure took place, donor-recipient cytomegalo virus (CMV) serostatus combination, and diagnosis (Acute Myeloid Leukemia [AML] vs Acute Lymphoblastic Leukemia [ALL]). The score provides an individualized estimated probability of day-100 mortality following transplantation for AL patients; eg, for a patient receiving a score of 6 the probability of death at day 100 is 6%. In addition, the score was categorized into 10 intervals which were shown to be predictive of overall survival (OS), leukemia free survival (LFS) and NRM 2 years after transplantation; eg, patients with scores in the range of 5.55 to 6.51 had a probability of 67.3 ( , 95% confidence interval [CI]) for 2-year OS. 10 The AL-EBMT score has yet to be validated on an independent data set. In the present study, we aimed to validate the predictive capacity of the AL-EBMT score on an independent data set of acute leukemia patients from the Italian transplantation network (Gruppo Italiano Trapianto di Midollo Osseo [GITMO]). We explored the score s abilityto give a probabilistic estimation of mortality at 100 days post-hsct. In addition, we divided the original cohort s scores into terciles, which were evaluated for validity in the predicting 2-year OS, LFS and NRM in the GITMO cohort. 2 METHODS 2.1 Study design This was a retrospective validation study of the AL-EBMT score on a cohort of acute leukemia patients. The principles of Transparent Reporting of a prediction model for Individual Prognosis Or Diagnosis (TRIPOD) were followed. 12,13 Data from the GITMO registry were analyzed for external validation of the AL-EBMT score. GITMO is the Italian national bone marrow transplantation network comprising more than 80 centers required to report consecutive HSCT and annual follow-up in a standardized manner. Permission to perform this study was obtained from the GITMO Clinical Studies Board. All patients provided informed consent in accordance with the Declaration of Helsinki for the analysis of their clinical data. 2.2 Population variables and AL-EBMT score calculations Patients included underwent first allogeneic transplants, performed from 2000 to 2014, using peripheral blood stem cells or bone marrow as cell source, in adults (age 18 years) diagnosed with de-novo acute leukemia. Donors were either siblings or unrelated. Patients who were previously included in the score development study, 10 and those receiving an umbilical cord blood transplants, were excluded. The AL- EBMT score was calculated for all patients in the GITMO cohort using the formula published by Shouval et al., ( ~bondi/web1.html). 10 The score accommodates missing predictor values and requires the following data at time of HSCT: recipient age, diagnosis (AML or ALL), year of transplantation (2003, >2003), days from diagnosis to HSCT, disease status (1 st complete remission [CR], 2 nd CR, other disease status), Karnofsky performance status at HSCT (<80, 80), recipient s and donor s CMV serostatus (both negative, at least 1 positive), conditioning regimen defined by the criteria set out by Bacigalupo et al., (myeloablative, reduced intensity conditioning), 14 donor type (sibling or unrelated HLA-matched donor), and annual number of allogeneic HSCTs, averaged across 3-year intervals, performed in the center where the patient was transplanted in the same year of transplantation, (<20, 20 HSCT/year). The Hematopoietic Cell Transplant-Co-morbidity Index (HCT-CI) score was calculated for 893 patients in accordance with definitions published by Sorror et al Outcomes All outcomes were measured from the time of allogeneic HSCT. NRM was defined as death without previous relapse/progression. LFS was defined as survival without leukemia progression or relapse. Competing risk analysis was used to calculate cumulative incidence of 24-month NRM, using the Gray test to test differences between AL-EBMT score groups. OS and LFS were estimated using the Kaplan-Meier method; hazard ratios were computed between subgroups using Cox regression for OS and competing risk regression for NRM. Competing risk regression was used to compute NRM cumulative incidence rates, considering relapse/progression as the competing event. Calibration and discrimination, assessed with the time-dependent area under the receiver operating characteristic curve (AUC), 13,15 were used to evaluate the score s quality. Calibration reflects the agreement between outcome predictions from the model and the observed outcomes. Discrimination refers to the ability of a prediction model to differentiate between those who do or do not experience the outcome event, and ranges from 0.5 (ie, prediction is not better than chance) to 1 (ie, the predicted risk for all individuals who develop the outcome is higher than for all individuals who do not experience the outcome). 13,16 All statistical analysis were performed using SPSS (v.21) and R (v.3.2.4). 3 RESULTS 3.1 Patient population Characteristics of 1848 analyzed patients are listed in Table 1. Median follow-up was 2 years ( ). The majority of patients had AML (68.1%), were in CR1 (52.3%) and received myeloablative conditioning (MAC) (78.0%). Grafts from matched sibling donors were used in 50.4% of patients. Stem cell source was bone marrow in 30.1% of patients. More than half of the patients were transplanted after OS at 2 years was 55.2% ( ), LFS at 2 years was 48.9% (46.4%-51.6%), and NRM at 2 years was 19.9% ( ). The cohort differed (P <.05) from the EBMT data set in all parameters included in the ADT score (Supporting Information Table S1). In a subanalysis of patients with available comorbidity data (n 5 893) the HCT- CI score ranged from 0 to 7, with 61.6%, 24.0%, and 14.4% having scores of 0,1-2, and 3, respectively.

3 SHOUVAL ET AL. 431 TABLE 1 Population characteristics Variable Value Missing Median age (IQR), years 46 (35-55) 0% Median time between dx and HSCT (IQR), days 236 ( ) 0% Median annual allo-hsct 27 (18-44) 0% experience (IQR) a Diagnosis (%) % AML 1258 (68.1) ALL 590 (31.9) Disease stage (%) % CR1 966 (60.4) CR2 334 (20.9) Advanced 300 (18.8) Patient sex (%) % Male 1060 (57.6) Female 781 (42.4) Karnofsky at HSCT % (95.6) <80 65 (4.4) R CMV serostatus % Negative 289 (18.6) Positive 1265 (81.4) Donor type % MSD 931 (50.4) UD b 917 (49.6) Donor sex % Male 1036 (59.7) Female 700 (40.3) D-R sex combination % M >M 621 (35.9) F >F 330 (19.1) M >F 411 (23.7) F >F 369 (21.3) D-R CMV serostatus combo % - > (10.7) 1 > (54.8) - < > (34.5) Year of HSCT % (8.0) (11.5) (14.6) (65.9) Conditioning % MAC 1441 (81.3) RIC 331 (18.7) Stem cell source % PB/BM1PB 1285 (69.9) BM 554 (30.1) a HLA allelic level compatibility: 10 of 10 (n 5 389), 9 of 10 (n 5 323), <9 of 10 (n 5 115), missing (n 5 97). b Averaged over 3 years. ALL, acute lymphoblastic leukemia; AML, acute myeloid leukemia; BM, bone marrow; CR1, 1 st complete remission; CR2, 2 nd complete remission; IQR, inter-quartile range; MAC, myeloablative conditioning; MSD, matched sibling donor; PB, peripheral blood; RIC, reduced intensity conditioning; UD, unrelated donor. 3.2 Prediction of day-100 mortality in the GITMO cohort As in the original publication, 10 we first evaluated the AL-EBMT score as a continuous parameter to predict overall mortality 100 days following allogenic HSCT in the GITMO cohort. The AL-EBMT score shows similar performance to the original EBMT cohort, with an AUC of and good calibration (r 5 0.9) (Supporting Information Figure S1). 3.3 The AL-EBMT s risk-score groups The large sample size, including almost patients in the original AL-EBMT score development study, 10 allowed for categorization of the AL-EBMT score into 10 intervals and assessment of their impact on 2-year outcomes. Using the same 10 categories in the GITMO cohort (n ) was not reasonable owing to the smaller sample size. We therefore went back to original EBMT cohort (n ) and categorized the patients scores as terciles (low [<8.5] intermediate [8.6-10], and high [>10]), corresponding to 9337, 9222, and 9671 patients in each interval, respectively. Intervals corresponded with a 63.7% (62.8%-64.6%), 56.7% (55.1%-58.2%), and 36.7% (35.7%- 37.8%) probability of overall survival at 2 years. Intermediate vs. low and high vs. low score intervals were associated with hazard ratios of 1.33 ( ) and 2.63 ( ), P <.001, for 2-year overall survival. 3.4 Risk stratification by the AL-EBMT score in the GITMO cohort The newly-determined tercile intervals from the EBMT cohort were used to categorize patients in the GITMO data set, corresponding with 410, 847, and 591 patients, in the low, intermediate, and high score groups, respectively. Increasing score intervals were associated with a decreasing probability for 2-year OS ranging from 73.2% ( ) to 38.4% ( ), and decreasing probability for 2-year LFS, ranging from 65.9% ( to 31.1% ( ). In contrast, the 2-year NRM probability corresponded directly with an increasing AL-EBMT score, ranging from 13.7% ( ) to 27.2% ( ) (Supporting Information Table S2, Figure 1). When the 3 categories of the AL-EBMT score were studied in a Cox regression model for 2-year OS and LFS and a competing risk regression model for NRM, the categories stratified patients into distinct risk groups (Table 2). The hazard ratio for 2-year OS of the intermediate vs low and high vs low categories was 1.63 ( ) and 3.16 ( ), respectively. Hazard ratios for 2-year NRM were 1.31 ( ) and 2.27 ( ). In a multivariate subgrop analysis including patients the patients with available HCT-CI score (n 5 893), the AL-EBMT remained predictive of the outcomes assessed (Table 3). 3.5 Performance measures The categorized AL-EBMT score was well-calibrated for all outcomes assessed (Supporting Information Figure S2). The score s discrimination

4 432 SHOUVAL ET AL. FIGURE 1 Kaplan-Meyer survival/cumulative incidence curves for AL-EBMT terciles in the GITMO cohort. AL-EBMT, Acute Leukemia European Society of Blood and Marrow Transplantation score; GITMO, Gruppo Italiano Trapianto di Midollo Osseo; LFS, Leukemia free survival; NRM, Nonrelapse mortality (AUC) for 2-year OS, LFS, and NRM over the complete GITMO cohort was 0.651, 0.653, and 0.651, respectively. Discrimination varied by the particular setting as shown in Supporting Information Table S3: the AUC for prediction of 2 year NRM was for patients transplanted after a reduced intensity conditioning (RIC) regimen versus after MAC, or for patients transplanted from an HLA matched sibling donor versus in transplants from matched unrelated donors. Discrimination was also better for younger (age 46 years) versus older patients (age > 46 years) with AUCs of and 0.629, respectively. 4 DISCUSSION Acute leukemia is the leading indication for allogeneic HSCT. Thus, assessing the risk associated with transplantation in this population is of great importance, and can be used for informed decision-making and, potentially, personalization of the transplant procedure. In this retrospective analysis, we have externally validated the AL-EBMT score, a prognostic scoring system for prediction of outcomes following allogeneic HSCT in acute leukemia. An increasing score correlated with increasing probability of mortality at 100 days after HSCT. Furthermore, when categorized into 3 intervals, the score was associated with an increase in hazard of OS, LFS, and NRM at 2 years following HSCT. The AL-EBMT score was developed on registry data from the Acute Leukemia Working Party of the EBMT. Validating the score on the GITMO registry was appealing as the registry comprises detailed data on transplantations from over 80 centers. Hence, performance of the score on the GITMO cohort is likely to be generalizable. Furthermore, the high data quality provided by GITMO has been the source of several major publications which have had meaningful effect on HSCT practice. 7,17 20 Most centers in the GITMO network additionally report transplantation data to the EBMT registry. To assure that the validation cohort was independent, we excluded patients who were previously included in original EBMT cohort. Furthermore, the EBMT cohort included transplantations performed from 2000 to 2011, while most transplantations included in the current analysis were performed after 2011 (72%). Hence, we see that AL-EBMT score predictions remain consistent with contemporary practice. The GITMO cohort differed from the EBMT cohort in every parameter tested (Supporting Information Table S1). Patients in the current dataset were older (median age of 46 vs 43, P <.001), more likely to receive a graft from an unrelated donor (49.6% vs 46.1%, P 5.004) and to receive a myeloablative conditioning regimen (81.3% vs 71.5%, P <.001). Despite these differences, the AL-EBMT maintained its performance. Overall, our results indicate the robust nature of the score. The HCT-CI is a valid tool for risk stratification of HSCT patients. In our population, the index was only available for 893 (48%). The majority of patients had a score of 0, differing from the HCT-CI distribution presented by Sorror et al., and others, who reported a higher comorbidity burden. 3,21 However, similar frequencies were reported in other populations, 7,22 24 likely reflecting differing patients characteristics between the various cohort. In a multivariate Cox regression model (Table 2), considering only patients with complete HCT-CI and AL- EBMT score information, both scores were independent predictors of TABLE 2 Hazard ratios by AL-EBMT score terciles in the GITMO cohort OS at 2 years LFS at 2 years NRM at 2 years AL-EBMT Score (interval) HR p HR p HR p Low (<8.5) Reference <0.001 reference <0.001 reference <0.001 Intermediate (8.6-10) 1.63 ( ) 1.44 ( ) 1.31 ( ) High (>10) 3.16 ( ) 2.8 ( ) 2.27 ( ) AL-EBMT, Acute Leukemia European Society of Blood and Marrow Transplantation score; CI, 95% confidence interval; GITMO, Gruppo Italiano Trapianto di Midollo Osseo; HR, hazard ratio; LFS, leukemia-free survival; NRM, nonrelapse mortality; OS, overall survival.

5 SHOUVAL ET AL. 433 TABLE 3 Hazard ratios for multivariate analysis incorporating both AL-EBMT and HCT-CI scores in the GITMO cohort OS at 2 years LFS at 2 years NRM at 2 years Hazard Ratio p Hazard Ratio p Hazard Ratio p AL-EBMT Score (interval) Categories Low (<8.5) Reference <0.001 Reference <0.001 Reference Intermediate (8.6-10) 1.39 ( ) 1.21 ( ) 1.24 ( ) High (>10) 2.79 ( ) 2.52 ( ) 1.84 ( ) HCT-CI Categories HCT-CI 0 Reference <0.001 Reference reference <0.001 HCT-CI ( ) 1.11 ( ) 1.56 ( ) HCT-CI ( ) 1.6 ( ) 2.3 ( ) AL-EBMT, Acute Leukemia European Society of Blood and Marrow Transplantation score; CI, 95% confidence interval; GITMO, Gruppo Italiano Trapianto di Midollo Osseo; HCT-CI, Hematopoetic cell transplant comorbidity index; HR, hazard ratio; LFS, leukemia-free survival; NRM, nonrelapse mortality; pv, p-value;. OS, LFS and NRM risk. Interestingly, prognostic strength differed between the three assessed outcomes. Increasing levels of the AL- EBMT were associated with greater hazard for 2-year OS and LFS, while the HCT-CI levels stratified NRM risk more effectively than the AL-EBMT score. Thus, the two scores are complementary, reflecting different discriminant points in the complexity of transplant outcome, such as disease type, transplantation characteristics and patient related aspects (eg, comorbidities). Several features are noteworthy with regards to the methodology underlying this validation study. First, traditional prognostic models in stem cell transplantation (eg, the EBMT, Pre-transplantation Assessment of Mortality [PAM] and HCT-CI scores) are based on Cox modelisation. 3,25,26 The AL-EBMT score was developed using the Alternating Decision Tree machine learning algorithm, which does not depend on prior assumptions regarding distribution of data and readily detects interactions between features. 10,11,27 Our findings support the incorporation of machine learning algorithms and the data mining approach as tools for analyzing and modeling clinical data. Second, in order to comply with good practice of prognostic modeling, we adhered to the TRI- POD statement, requiring among other things, reporting on the models discrimination (AUC) and calibration. 12,13 When reviewing the vast majority of the HCT-CI and EBMT score validation studies, few meet these standards. 3,5 7,28 31 To ensure an unbiased appraisal of prediction models quality and utility, the process of model development and validation should be transparent, as proposed by TRIPOD. The study has several limitations. First, despite being an independent validation set, GITMO centers report to the EBMT, hence, validation in non-european centers is still warranted. Nevertheless, as discussed above, there are several features, including year of transplantation and population characteristics that are unique to the population analyzed (Supporting Information Table S1). Second, we were only able to calculate the HCT-CI for 893 patients. Thus, the sample size did not allow for development of an integrated AL-EBMT HCT-CI prognostic system, which may further improve discrimination. Finally, this was a retrospective registry analysis, subject to the typical biases and confounders of such studies such as selection and measurement biases. Therefore, prospective validation is warranted. 5 CONCLUSIONS In conclusion, in this retrospective analysis we have externally validated the prognostic utility of the AL-EBMT score in a cohort of acute leukemia patients. The continuous score can be used to estimate the individual probability for overall mortality at day 100 following HSCT. Furthermore, when categorized into 3 intervals, the score was discriminatory for OS, LFS, and NRM at 2 years. Overall, the score s potential applications include pre-transplantation risk assessment and stratification, patient counseling during informed consent sessions, and tailoring transplantation regimens or referring to alternative treatments according to transplantation risk. Future elaborations of this model should include other types of input such as genetic data, which could further improve the performance of the AL-EBMT score. 32 Moreover, our analysis suggests that integrating data on comorbidities with the AL- EBMT score could further promote the risk stratification. ACKNOWLEDGMENTS This study was supported by The Varda and Boaz Dotan Research Center in Hemato-Oncology affiliated with the CBRC of Tel Aviv University. REFRENCES [1] Copelan EA. Hematopoietic stem-cell transplantation. N Engl J Med. 2006;354: [2] Stone RM. Acute myeloid leukemia in first remission: to choose transplantation or not? J Clin Oncol. 2013;31: [3] Sorror ML, Maris MB, Storb R, et al. Hematopoietic cell transplantation (HCT)-specific comorbidity index: a new tool for risk assessment before allogeneic HCT. Blood. 2005;106:

6 434 SHOUVAL ET AL. [4] Magenau J, Couriel DR. Hematopoietic stem cell transplantation for acute myeloid leukemia: to whom, when, and how. Curr Oncol Rep. 2013;15: [5] Gratwohl A, Stern M, Brand R, et al. Risk score for outcome after allogeneic hematopoietic stem cell transplantation a retrospective analysis. Cancer. 2009;115: [6] Versluis J, Labopin M, Niederwieser D, et al. Prediction of nonrelapse mortality in recipients of reduced intensity conditioning allogeneic stem cell transplantation with AML in first complete remission. Leukemia. 2015;29: [7] Raimondi R, Tosetto A, Oneto R, et al. Validation of the hematopoietic cell transplantation-specific comorbidity index: a prospective, multicenter GITMO study. Blood. 2012;120: [8] Freund Y, Mason L. The alternating decision tree learning algorithm. ICML; pp [9] Freund Y, Schapire R, Abe N. A short introduction to boosting. J Jpn Soc Artif Intell. 1999;14:1612 [10] Shouval R, Labopin M, Bondi O, et al. Prediction of allogeneic hematopoietic stem-cell transplantation mortality 100 days after transplantation using a machine learning algorithm: a European group for blood and marrow transplantation acute leukemia working party retrospective data mining study. J Clin Oncol. 2015;33: [11] Shouval R, Nagler A, Labopin M, et al. Interpretable boosted decision trees for prediction of mortality following allogeneic hematopoietic stem cell transplantation. J Data Mining Genomics Proteomics. 2015;6:2. [12] Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015;162: [13] Moons KG, Altman DG, Reitsma JB, et al. Transparent reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015; 162:W1 W73. [14] Bacigalupo A, Ballen K, Rizzo D, et al. Defining the intensity of conditioning regimens: working definitions. Biol Blood Marrow Transplant. 2009;15: [15] Blanche P, Dartigues JF, Jacqmin-Gadda H. Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks. Statist Med. 2013;32: [16] Harrell F. Regression modelling strategies with applications to linear models, logistic regression, and survival analysis. New York: Springer- Verlag; [17] Patriarca F, Bacigalupo A, Sperotto A, et al. Allogeneic hematopoietic stem cell transplantation in myelofibrosis: the 20-year experience of the Gruppo Italiano Trapianto di Midollo Osseo (GITMO). Haematologica. 2008;93: [18] Crocchiolo R, Zino E, Vago L, et al. Nonpermissive HLA-DPB1 disparity is a significant independent risk factor for mortality after unrelated hematopoietic stem cell transplantation. Blood. 2009;114: [19] Della Porta MG, Alessandrino EP, Bacigalupo A, et al. Predictive factors for the outcome of allogeneic transplantation in patients with MDS stratified according to the revised IPSS-R. Blood. 2014; 123: [20] Todisco E, Ciceri F, Oldani E, et al. The CIBMTR score predicts survival of AML patients undergoing allogeneic transplantation with active disease after a myeloablative or reduced intensity conditioning: a retrospective analysis of the Gruppo Italiano Trapianto Di Midollo Osseo. Leukemia. 2013;27: [21] Sorror ML, Sandmaier BM, Storer BE, et al. Comorbidity and disease status-based risk stratification of outcomes among patients with acute myeloid leukemia or myelodysplasia receiving allogeneic hematopoietic cell transplantation. J Clin Oncol. 2007;25: [22] Teixeira GM, Bittencourt H, de Macedo AV, et al. Assessing the influence of different comorbidities indexes on the outcomes of allogeneic hematopoietic stem cell transplantation in a developing country. PloS One. 2015;10:e [23] Tanaka Y, Kurosawa S, Tajima K, et al. Analysis of non-relapse mortality and causes of death over 15 years following allogeneic hematopoietic stem cell transplantation. Bone Marrow Transplant. 2016;51: [24] Wang Y, Liu DH, Liu KY, et al. Impact of pretransplantation risk factors on post transplantation outcome of patients with acute myeloid leukemia in remission after haploidentical hematopoietic stem cell transplantation. Biol Blood Marrow Transplant. 2013;19: [25] Gratwohl A, Hermans J, Goldman JM, et al. Risk assessment for patients with chronic myeloid leukaemia before allogeneic blood or marrow transplantation. Chronic Leukemia Working Party of the European Group for Blood and Marrow Transplantation. Lancet. 1998;352: [26] Parimon T, Au DH, Martin PJ, et al. A risk score for mortality after allogeneic hematopoietic cell transplantation. Ann Intern Med. 2006; 144: [27] Shouval R, Bondi O, Mishan H, et al. Application of machine learning algorithms for clinical predictive modeling: a data-mining approach in SCT. Bone Marrow Transplant. 2014;49: [28] Sorror ML, Giralt S, Sandmaier BM, et al. Hematopoietic cell transplantation specific comorbidity index as an outcome predictor for patients with acute myeloid leukemia in first remission: combined FHCRC and MDACC experiences. Blood. 2007;110: [29] DeFor TE, Majhail NS, Weisdorf DJ, et al. A modified comorbidity index for hematopoietic cell transplantation. Bone Marrow Transplant. 2010;45: [30] Barba P, Martino R, Perez-Simon JA, et al. Combination of the hematopoietic cell transplantation comorbidity index and the European Group for blood and marrow transplantation score allows a better stratification of high-risk patients undergoing reducedtoxicity allogeneic hematopoietic cell transplantation. Biol Blood Marrow Transplant. 2014;20: [31] Pitombeira BS, Paz A, Pezzi A, et al. Validation of the EBMT risk score for south brazilian patients submitted to allogeneic hematopoietic stem cell transplantation. Bone Marrow Res. 2013;2013: [32] Shouval R, Labopin M, Unger R, et al. Prediction of hematopoietic stem cell transplantation related mortality-lessons learned from the in-silico approach: a European society for blood and marrow transplantation acute leukemia working party data mining study. PloS One. 2016;11:e SUPPORTING INFORMATION Additional Supporting Information may be found in the online version of this article. How to cite this article: Shouval R, Bonifazi F, Fein J, et al. Validation of the acute leukemia-ebmt score for prediction of mortality following allogeneic stem cell transplantation in a multicenter GITMO cohort. Am J Hematol. 2017;92: