The Karakiewicz Nomogram Is the Most Useful Clinical Predictor for Survival Outcomes in Patients With Localized Renal Cell Carcinoma

Transcription

1 The Karakiewicz Nomogram Is the Most Useful Clinical Predictor for Survival Outcomes in Patients With Localized Renal Cell Carcinoma Min-Han Tan, MBBS, MRCP 1,2,3 ; Huihua Li, PhD 4 ; Caroline Victoria Choong 1 ; Kee Seng Chia, MBBS, MSc(OM), MD 2 ; Chee Keong Toh, MBBS, MRCP 1 ; Tiffany Tang, MBBS, MRCP 1 ; Puay Hoon Tan, MBBS, FRCPA, MD 5 ; Chin Fong Wong, MBBS 5 ; Weber Lau, MBBS, FRCS, FAMS 6 ; and Christopher Cheng, MBBS, FRCS, FAMS 6 BACKGROUND: Outcomes after surgical removal of localized renal cell carcinoma (RCC) are variable. There have been multiple prognostic nomograms and risk groups developed for estimation of survival outcomes, with different models in use for evaluating patient eligibility in ongoing trials of adjuvant therapy. The authors aimed to establish the most useful prognostic model for patients with localized RCC to guide trial design, biomarker research, and clinical counseling. METHODS: A total of 390 consecutive patients who underwent nephrectomy for sporadic localized RCC in a tertiary institution ( ) with 65 months median follow-up were retrospectively evaluated. The Karakiewicz nomogram, the Kattan nomogram, the Sorbellini nomogram, and the Leibovich model were compared in predicting survival outcomes (overall survival, cancer-specific survival, and freedom from recurrence) using likelihood analysis, adequacy indices, decision curve analysis, calibration, and concordance indices. RESULTS: Overall, the Karakiewicz nomogram outperformed the Kattan nomogram, the Sorbellini nomogram, and the Leibovich model. Highly improved accuracy was seen using the Karakiewicz nomogram in survival prediction, using likelihood ratio analysis in bivariate models including the competing prognostic models. The Karakiewicz nomogram showed higher adequacy and concordance indices and improved clinical benefit relative to all other nomograms. All 4 models were reasonably calibrated. Exploratory comparisons of prespecified discretized Karakiewicz nomograms and the SORCE trial recruitment criteria (a discretized Leibovich model) of high-risk patients favored the discretized Karakiewicz nomograms. CONCLUSIONS: The Karakiewicz nomogram was shown to be superior to the other tested nomograms and risk groups in predicting survival outcomes in localized RCC. Routine integration of this model into trial design and biomarker research should be considered. Cancer 2011;117: VC 2011 American Cancer Society. KEYWORDS: renal cell carcinoma, nomograms, prognosis, adjuvant chemotherapy, algorithms, sunitinib, sorafenib, likelihood functions, decision curve analysis. Approximately 70% of patients with renal cell carcinoma (RCC) present with localized disease, for which nephrectomy is usually curative. 1 However, about 30% of these patients eventually relapse and develop metastases during the course of the disease. 2 There are several prognostic risk groups and nomograms developed to estimate outcomes of patients with localized RCC. 3 Both approaches are methods to predict outcomes for patients, 4 but nomograms differ from risk groups in several relevant ways, as briefly explained in Table 1. These models are used in clinical practice to aid in counseling, follow-up planning, and most recently, patient classification into groups for trials of adjuvant therapy. 5 To illustrate the current use of risk grouping, the UK Medical Research Council SORCE trial is recruiting patients with intermediate- and high-risk Leibovich scores for randomization between sorafenib and placebo, 1 whereas the Adjuvant Corresponding author: Min-Han Tan, MBBS, MRCP, Department of Medical Oncology, National Cancer Centre Singapore, 11 Hospital Drive, Singapore , Republic of Singapore; Fax: (65) ; tan.min.han@nccs.com.sg 1 Department of Medical Oncology, National Cancer Centre Singapore, Singapore; 2 Department of Epidemiology and Public Health, National University of Singapore, Singapore; 3 NCCS-VARI Laboratory of Translational Cancer Research, National Cancer Centre Singapore, Singapore; 4 Department of Clinical Research, Singapore General Hospital, Singapore; 5 Department of Pathology, Singapore General Hospital, Singapore; 6 Department of Urology, Singapore General Hospital, Singapore The first 3 authors contributed equally to this work. DOI: /cncr.26193, Received: September 24, 2010; Revised: March 21, 2011; Accepted: March 24, 2011, Published online May 12, 2011 in Wiley Online Library (wileyonlinelibrary.com) 5314 Cancer December 1, 2011

2 Karakiewicz Nomogram in Kidney Cancer/Tan et al Table 1. Comparison of Risk Groups and Nomograms in Predicting Survival Outcomes Objective and approach Scales of measurement Clinical use Examples in renal cell carcinoma Risk Groups To predict outcomes for the individual patient by classifying similar but not identical patients into risk groups, where all patients in the risk group have the same predicted outcome Relatively fewer groups on a semiquantitative or ordinal scale (eg, classification into groups of low-, medium-, and high-risk patients) Widespread, eg, International Prognostic Index in lymphoma UCLA Integrated Staging System, Leibovich score Nomograms To predict outcomes for patients using formulae that computes predictions for the individualized patient, rather than for risk groups; points on a semicontinuous scale are assigned to individual variables A directly calculated quantitative outcome on a numerical scale (eg, predicted overall survival of 69% at 5 years for a patient) Limited, most commonly used in prostate cancer Karakiewicz nomogram, Kattan nomogram, Sorbellini nomogram Sorafenib or Sunitinib for Unfavorable Renal Carcinoma (ASSURE) and the Sunitinib Treatment of Renal Adjuvant Cancer (S-TRAC) trials are selecting patients based on modified UCLA Integrated Staging System (UISS) criteria. Although it is recognized that nomograms often outperform risk grouping, 6 nomograms have not been integrated into RCC clinical trials, and no study has comprehensively examined nomogram performance relative to that of risk grouping in predicting survival in RCC. Several studies evaluating nomogram performance have imposed multiple thresholds on the nomogram outcomes, 3,7 with no disclaimer of exploratory analysis or expanded justification presented for the selection of these thresholds. Although discretization of a continuous variable inevitably results in a loss of statistical information 8, this is practically difficult to avoid in a trial setting, where thresholds for patient recruitment are used. For implementation of a trial design using nomograms, a single threshold for discretizing risk, so as to divide patients into 2 risk groups, is usually most useful for establishing clear inclusion criteria and for decision making by clinicians and patients. Several postoperative clinical nomograms have been recently developed for RCC. 9 The Kattan nomogram was developed to predict 5-year freedom from recurrence (FFR) in patients undergoing radical nephrectomy for nonmetastatic RCC. 10 The variables used in this postoperative nomogram were symptoms, histologic subtype,- umor size, and TNM classification. More recently, the Karakiewicz nomogram was developed to predict 1-, 2-, 5-, and 10-year cancer-specific survival (CSS) of patients undergoing nephrectomy for RCC of all stages. The Karakiewicz nomogram was designed as a postoperative nomogram with the variables TNM classification, size, Fuhrman grade, histologic subtype, age, and symptom classification. 11 The Sorbellini nomogram was developed to predict 5-year FFR for patients undergoing surgical treatment for localized clear cell RCC, using tumor size, TNM classification, Fuhrman grade, tumor necrosis, vascular invasion, and symptom presentation. 12 Despite the multiple RCC nomograms published in the literature, none are currently in widespread clinical use. Instead, ongoing clinical trials employ risk grouping for risk estimation and patient selection. In evaluating risk-group performance, we have previously shown that the SORCE trial criteria (a discretized Leibovich model) performs better in discrimination than the ASSURE trial criteria (a discretized modified-uiss model). 13 The purpose of the current study was to clarify which clinical model is most useful for survival prediction in localized RCC, so as to establish a standard for guiding trial design and biomarker research. MATERIALS AND METHODS We identified 423 patients who underwent nephrectomy for sporadic nonmetastatic unilateral RCC in Singapore General Hospital from 1990 to Institutional review board approval was obtained. All specimens were reviewed by a pathologist for histologic subtype, tumor grade, lymphovascular invasion, and necrosis. Survival status and cause of death, if any, were obtained from a national registry. To ensure the most accurate comparisons between nomograms, we used an approach to select common selection criteria. Broadly, the Karakiewicz nomogram had the least restrictive selection criteria, and similar to the Kattan nomogram, it was applicable to all RCC subtypes. The Sorbellini nomogram and the Leibovich score were restricted to clear cell RCC. Therefore, in comparing the Karakiewicz nomogram with the Kattan nomogram, we excluded patients with large tumors Cancer December 1,

3 (pt4), Eastern Cooperative Oncology Group (ECOG) >1, and patients with subtypes other than clear cell RCC, papillary RCC, or chromophobe RCC (n ¼ 33), with a remaining data set of 390 subjects. The ECOG limitation was imposed to ensure that this comparison would be useful in the appropriate patient set for trial design and is consistent with our previous approach. 13 All comparisons with the Sorbellini nomogram (n ¼ 329) and the Leibovich score (n ¼ 322) similarly were restricted to clear cell subtype only, and selection mirrored the more restrictive criteria of each score to ensure a fair comparison. The tumor size of pathologic specimens was determined as the greatest dimension in centimeters, and the Fuhrman grading scheme was used to determine the nuclear grade of tumors. Pathologic staging was determined in accordance with the American Joint Committee on Cancer (AJCC) 2002 primary tumor TNM classification, 14 except when scoring the Kattan nomogram, where AJCC 1997 stage grouping was used. The symptoms were classified as incidental, local (hematuria, flank pain, palpable mass), or systemic (weight loss, anorexia, asthenia, fever). 15 Statistical Analysis We chose 3 study end points to evaluate in common across the different models, these being CSS, FFR, and overall survival (OS). CSS was defined as the interval between diagnosis date and cancer-related death date or last follow-up date for censored patients. FFR was defined as the interval between surgery date and relapse date or the date of last follow-up for censored patients. OS was defined as the interval between diagnosis date and death date or last follow-up date for censored patients. The Leibovich score was developed on metastasis-free survival (MFS). Because there is considerable overlap between the definition of FFR, MFS, and disease-free survival (DFS), and hence replicated analyses did not show material differences between these 3 outcomes, we selected to present FFR here. Outcomes were estimated by the Kaplan-Meier approach, and Cox regression was used to evaluate the effects of covariates. We used likelihood ratio (LR) chisquare of nested models to perform pairwise comparisons of the models involved. An adequacy index using LR methods was used to quantify the percentage of the variation explained by a subset of the individual predictors compared with the information contained in the full set of predictors by means of log likelihood. 16,17 Harrell s c-index was calculated to evaluate the concordance between predicted and observed responses of individual subjects separately. 13 Calibration is useful for evaluating Table 2. Baseline Characteristics of the Singapore Data Set (N ¼ 390) Characteristics n (%) Age at diagnosis, y, median (range) 57 (16-91) Sex Male 256 (66) Female 134 (34) Symptoms Incidental 170 (44) Local only 179 (46) Systemic only 15 (4) Local and systemic 26 (7) ECOG performance status (70) (30) Nephrectomy type Radical 355 (91) Partial 34 (9) Unknown 1 (0) Approach Open 275 (71) Laparoscopic 115 (29) Histology Clear cell 334 (86) Papillary 44 (11) Chromophobe 12 (3) TNM staging I 214 (55) II 65 (17) IIIA 69 (18) IIIB/C 42 (11) Tumor size/cm (35) (34) > (31) Fuhrman grade 1 57 (15) (53) 3 99 (25) 4 28 (7) Histologic tumor necrosis Yes 152 (39) No 236 (61) Events Deaths, all causes 86 (22) Deaths, disease related 63 (16) Deaths, other causes 23 (6) RCC recurrence 98 (24) RCC distant metastases 95 (24) ECOG indicates Eastern Cooperative Oncology Group; RCC, renal cell carcinoma Cancer December 1, 2011

4 Karakiewicz Nomogram in Kidney Cancer/Tan et al Figure 1. Kaplan-Meier survival curves show the outcomes of the Singapore patient cohort with localized renal cell carcinoma by the various outcomes. Shown are (A) overall survival; (B) cancer-specific survival; (C) freedom from recurrence; and (D) metastasis-free survival. whether actual outcomes approximate predicted outcomes for each model in our data set. For calibration comparisons, we evaluated each model by its defined 5-year survival outcome (Karakiewicz, CSS; Kattan and Sorbellini, FFR; Leibovich, MFS), collapsing each nomogram to approximate the risk grouping of the Leibovich score for greater comparability (risk thresholds 0.9 and 0.6), with expected outcome in each risk group determined by the median scorer. The Leibovich score was calibrated by prespecified low-, intermediate- and high-risk groups. 18 Decision curve analyses were performed to determine the clinical net benefit derived by examining the theoretical relation between the threshold probability of developing an event and the relative value of false-positive and falsenegative results as described by Vickers et al. 19 To evaluate whether the nomogram has potential to outperform current standards of risk evaluation, we further tested several prespecified Karakiewicz nomogram thresholds (estimated 5-year CSS of 0.90, 0.85, and 0.80) against the SORCE trial criteria (a discretized Leibovich score) using similar methodology on an exploratory basis. The SORCE trial criteria divided patients into low-risk (0-2) and intermediate-/high-risk (3) individuals by Leibovich score. A separate decision analytic approach was also performed to determine estimated cutoff, 19 based on a threshold benefit of 0.05, similar to considerations of adjuvant therapy in gastric, 20 colorectal, 21 and breast cancer, 22 with a 0.5 risk reduction. 23 STATA 11 (StataCorp, College Station, TX) and R were used for analysis, and all tests were 2-sided with a significance level of.05. RESULTS Baseline characteristics and survival graphs of the full 390-patient data set are reported (Table 2 and Fig. 1). Predicted survival outcomes and concordance indices are presented (Tables 3 and 4). We show that the Karakiewicz nomogram is overall the best-performing nomogram Cancer December 1,

5 Table 3. Pairwise Comparison of Nomograms in Estimating Outcomes in Localized RCC No. of Events/ No. of Patients Hazard Ratio (95% CI) P of Log-Rank Test 5-Year Outcomes Concordance Index Kattan vs Karakiewicz OS 86/ Kattan nomogram 1.02 ( ) a < Karakiewicz nomogram 1.02 ( ) a < CSS 63/ Kattan nomogram 1.03 ( ) a < Karakiewicz nomogram 1.03 ( ) a < FFR 98/ Kattan nomogram 1.02 ( ) a < Karakiewicz nomogram 1.02 ( ) a < Sorbellini vs Karakiewicz OS 70/ Sorbellini nomogram 1.02 ( ) a < Karakiewicz nomogram 1.02 ( ) a < CSS 51/ Sorbellini nomogram 1.02 ( ) a < Karakiewicz nomogram 1.03 ( ) a < FFR 80/ Sorbellini nomogram 1.02 ( ) a < Karakiewicz nomogram 1.02 ( ) a < RCC indicates renal cell carcinoma; CI, confidence interval; OS, overall survival; CSS, cancer-specific survival; FFR, freedom from recurrence. a Both the Leibovich score and the Karakiewicz nomogram are represented as a continuous score. when individually compared against each of the other models (Fig. 2). The Karakiewicz nomogram had consistently higher adequacy and concordance indices for all tested outcomes. Its inclusion in a full model resulted in highly statistically significant accuracy improvements for all outcomes when tested with LR analysis against the Kattan nomogram (P <.001), the Sorbellini nomogram (P <.001), with marginal improvements over the Leibovich score (P ¼.04 for CSS, P ¼.03 for DFS, with equivalent performance for OS). This supports our conclusion that the Karakiewicz nomogram is a superior predictor to the Kattan or Sorbellini nomograms (Table 3), and in addition, that it is superior to the Leibovich score (Table 4), which we had previously identified as the most useful approach within risk-grouping approaches. 13 On inspection of the calibration plots, all 4 models were reasonably calibrated with best calibration seen in the Karakiewicz nomogram and the Leibovich model (Fig. 3). Maximum departure from ideal outcomes was as follows: Karakiewciz (4%), Kattan (11%), Sorbellini (15%), Leibovich score (17%), and the Leibovich model with risk groupings (3%). For the decision curve analyses (Fig. 4A-I), the Karakiewicz nomogram showed consistent clinical net benefit over the Kattan and Sorbellini nomogram across the range of threshold probabilities for example, at 20% threshold probability, the differential clinical benefit for CSS using Karakiewicz nomogram was over Kattan nomogram and over Sorbellini nomogram relative to the assumption that no patient will relapse. The net benefit of the Karakiewicz and the Leibovich models were quite similar across the range of threshold probabilities. In exploratory pairwise comparisons, we noted that several discretized Karakiewicz nomograms generally performed better than the ongoing SORCE trial criteria (a discretized Leibovich score) in terms of LR analysis, concordance indices, and clinical net benefit in multiple survival outcomes (Table 4), particularly at the probability threshold of.90 (Fig. 4J-L). Superiority of the Karakiewicz threshold remained but reduced as the threshold was reduced. Similarly, LR analysis showed consistently higher adequacies for the Karakiewicz nomograms at the 0.85 and 0.80 discretizations for multiple survival outcomes, supporting superiority of the discretized Karakiewicz nomogram (Table 5). A decision analytic method for threshold derivation yielded a similar approximate threshold (corresponding to predicted 5-year CSS of 0.89, or 99 points). Although for all outcomes, the 0.85 and 0.80 discretization thresholds yielded improved net benefit relative to the SORCE criteria over a higher range of threshold probabilities, the SORCE criteria showed benefit over the 0.85 and 0.80 Karakiewicz thresholds in a more limited range of lower threshold probabilities Cancer December 1, 2011

6 Karakiewicz Nomogram in Kidney Cancer/Tan et al Table 4. Comparison of Leibovich Score and Karakiewicz Nomogram in Estimating Outcomes in Localized RCC Evaluated Models No. of Events/ No. of Patients Hazard Ratio (95% CI) P of Log-Rank Test 5-Year Outcomes OS 72/ Leibovich score a 1.47 ( ) a < Karakiewicz nomogram a 1.02 ( ) a < Discretized Leibovich b 0.67 Low risk 11/142 Reference 0.95 Intermediate/high risk 61/ ( ) < (cutoff 0.90) Low risk 13/178 Reference 0.96 High risk 59/ ( ) < (cutoff 0.85) Low risk 27/222 Reference 0.94 High risk 45/ ( ) < (cutoff 0.80) Low risk 35/256 Reference 0.93 High risk 37/ ( ) < CSS 52/ Leibovich score a 1.63 ( ) a 0.83 Karakiewicz nomogram a 1.02 ( ) a 0.84 Discretized Leibovich b 0.70 Low risk 5/142 Reference 0.99 Intermediate/high risk 47/ ( ) < (cutoff 0.90) Low risk 5/178 Reference 0.99 High risk 47/ ( ) < (cutoff 0.85) Low risk 12/222 Reference 0.98 High risk 40/ ( ) < (cutoff 0.80) Low risk 19/256 Reference 0.97 High risk 33/ ( ) < FFR 83/ Leibovich score a 1.57 ( ) a < Karakiewicz nomogram a 1.02 ( ) a < Discretized Leibovich b 0.70 Low risk 8/142 Reference 0.95 Intermediate/high risk 75/ ( ) < (cutoff 0.90) Low risk 13/178 Reference 0.95 High risk 70/ ( ) < (cutoff 0.85) Low risk 24/222 Reference 0.92 High risk 59/ ( ) < (cutoff 0.80) Low risk 39/256 Reference 0.88 High risk 44/ ( ) < Concordance Index RCC indicates renal cell carcinoma; CI, confidence interval; OS, overall survival; CSS, cancer-specific survival; FFR, freedom from recurrence. a Both the Leibovich score and the Karakiewicz nomogram are represented as a continuous score. b The discretized Leibovich score corresponds to the existing Leibovich score discretization used in the SORCE trial, where intermediate- and high-risk patients are recruited. Cancer December 1,

7 Figure 2. Likelihood analyses compare the various models in predicting overall survival, cancer-specific survival, and freedom from recurrence. Shown are (A) comparison of the Karakiewicz nomogram and the Kattan nomogram; (B) comparison of the Karakiewicz nomogram and the Sorbellini nomogram; (C) comparison of the Karakiewicz nomogram and the Leibovich prognostic score; and (D) comparison of the discretized Karakiewicz nomogram (5-year cancer-specific survival cutoff 0.90) and the discretized Leibovich prognostic score (SORCE trial criteria). DISCUSSION Over the last decade, several prognostic models have been developed to predict survival outcomes in RCC patients. Many investigators have generally preferred to develop new models on their own data sets, rather than compare pre-existing models. Based on our comparisons, the Karakiewicz nomogram is the most useful clinical predictor of survival outcomes in our data set, showing superior accuracy in terms of LR analyses and concordance indices. Although the Karakiewicz nomogram was designed to test for CSS and the Sorbellini and Kattan nomograms for FFR, the Karakiewicz nomogram discriminated better in 5320 Cancer December 1, 2011

8 Karakiewicz Nomogram in Kidney Cancer/Tan et al Figure 3. Plots depict the calibration of each model in the Singapore data set in terms of agreement between predicted and observed 5-year outcomes. Shown are (A) the Karakiewicz nomogram and cancer-specific survival; (B) the Kattan nomogram and freedom from recurrence; (C) the Sorbellini nomogram and freedom from recurrence; and (D) Leibovich prognostic score and metastasis-specific survival. (E) In view of the multiple risk groups of the Leibovich prognostic score, which increase perceived variability, we inspected the calibration of corresponding low- (2), intermediate- (3-5), and high-risk (6) groups. Vertical bars are 95% confidence intervals. Model performance is shown by the plot, relative to the 45 line, which represents perfect prediction. Cancer December 1,

9 Figure 4. Decision curve analyses depict the clinical net benefit in pairwise comparisons across the different outcomes. The Karakiewicz nomogram is compared with the Kattan nomogram in terms of overall survival (OS), cancer-specific survival (CSS), and freedom from recurrence (A-C, respectively). The Karakiewicz nomogram is compared with the Sorbellini nomogram in similar terms of OS, CSS, and disease-free survival (D-F), as well as the Leibovich score (G-I). The Karakiewicz nomogram discretized at several prespecified intervals (Karakiewicz 1, estimated 5-year CSS cutoff at 0.90; Karakiewicz 2, cutoff at 0.85; Karakiewicz 3, cutoff at 0.80) is compared against the discretized Leibovich score or the SORCE trial criteria (J-L). Dashed lines indicate the net benefit of using the Karakiewicz nomogram in all figures across a range of threshold probabilities. The horizontal solid black line represents the assumption that no patients will experience the event; the solid gray line represents the assumption that all patients will recur. predicting CSS, FFR, and OS than the other nomograms and the Leibovich model. Thus, in terms of individual counseling, the Karakiewicz nomogram, which provides excellently calibrated CSS estimates, is likely to be more useful than the other models, which provide less discriminatory and less calibrated estimates of FFR. As a survival outcome, CSS may also be more relevant than other outcomes influenced by surveillance strategies. We have used a wide array of tools for comparing the various models, including comparisons of discrimination 5322 Cancer December 1, 2011

10 Karakiewicz Nomogram in Kidney Cancer/Tan et al Table 5. Comparison of Nomograms and the SORCE Trial Criteria (Discretized Leibovich Score) Karakiewicz Cutoff Karakiewicz1 Leibovich Likelihood P Adequacy Index Leibovich Karakiewicz Leibovich a Karakiewicz b Leibovich Karakiewicz 0.90 OS < % 100.0% CSS < % 100.0% FFR < % 94.9% 0.85 OS % 83.6% CSS < % 94.9% FFR < % 88.1% 0.80 OS < % 84.1% CSS < % 89.0% FFR <.0001 < % 74.0% OS indicates overall survival; CSS, cancer-specific survival; FFR, freedom from recurrence. a Comparison of the full model including both predictors with the nested model with only the discretized Karakiewicz nomogram. b Comparison of the full model including both predictors with the nested model with only the discretized Leibovich score (likelihood, adequacy, concordance indices) as well as calibration analyses. In addition to these comparisons, we have used decision curve analysis to show that the Karakiewicz nomogram is consistently superior to the other nomograms across a range of clinical threshold probabilities, with equivalent clinical benefit as the Leibovich score. Our work provides a framework for considering the choice of model in an adjuvant therapy setting in RCC, best highlighted by the clinical net benefit analysis. It is critical to note that evaluation of clinical net benefit will evolve with information on benefits and toxicities of adjuvant therapy. Practically, the selection of a cutoff in a clinical trial is in addition driven by trade-offs between event rate and accrual rate. 24 We considered it interesting that the best-performing cutoff for our discretization (estimated 5-year CSS, 0.90) coincided with one derived by decision analysis. Nonetheless, we caution here that although the Karakiewicz nomogram at all tested discretizations compares favorably to the SORCE trial criteria, prespecified thresholds should be considered exploratory, and external validation of such thresholds are required for definitive conclusions on discretization. Multiple studies have documented the superior performance of nomograms when compared with risk groups using rank-concordance methods, and our results reinforce this point. Other than improved individualized prediction, the immediate real-world implications of our findings would be a more efficient accrual of patients for the ongoing adjuvant therapy RCC trials. For example, of practical real-world interest is the finding that if the discretized Karakiewicz nomogram, rather than the standard trial criteria using the Leibovich score, had been used to select patients in our center for the SORCE trial, 151 patients instead of 187 would have been recruited for 47 events in terms of cancer-specific survival (Table 4). Other than clinical trials, the use of the Karakiewicz nomogram as a standard in biomarker research will allow for the development of more rigorous biomarkers when the strongest available clinical model is incorporated into multivariable analysis. The main limitation of our study is its nature as a retrospective, single-institution study, raising the usual concern about generalization of results to other populations. However, the excellent calibration demonstrated suggests that this is likely of relatively minor concern. Although populationbased data, such as the Surveillance, Epidemiology, and End Results (SEER) registries, would be most appropriate for evaluation and comparison of models, the absence of systematic collection of all evaluated variables precludes this possibility. It is important to note that we have focused on postoperative nomograms rather than pre-operative nomograms, as postoperative models have yielded better predictions than pre-operative models. 3 We have addressed the issue of ethnic differences in our previous work using high-throughput expression profiling, suggesting that there is little or no difference in RCC outcomes resulting from ethnic variation between Asian and Western populations, 30 and once again, the excellent calibration of the Western models as presented here suggests that ethnic differences are not an issue. Cancer December 1,

11 Conclusions The Karakiewicz nomogram is superior to the Kattan nomogram, Leibovich prognostic score, and the Sorbellini nomogram by discrimination in our cohort of patients with localized RCC. All 4 models showed reasonable calibration, with best calibration seen in the Karakiewicz nomogram and the Leibovich model, the latter 2 models showing equivalent performance by decision curve analysis. Notably, discretization of the Karakiewicz nomogram shows superior performance relative to the SORCE trial recruitment criteria (a discretized Leibovich score) in terms of discrimination and clinical benefit. Hence, future routine integration of the Karakiewicz nomogram into RCC trial design and biomarker research should be investigated. CONFLICT OF INTEREST DISCLOSURES This research is supported by grants from the National Kidney Foundation Singapore (NKFRC ) and the National Cancer Center Singapore (NRFGC09101). Dr. Tan is supported by a fellowship from the Singapore Millenium Foundation. REFERENCES 1. Eisen T. Adjuvant therapy in renal cell carcinoma: where are we? Eur Urol. 2007;6(suppl): Lane BR, Kattan MW. Prognostic models and algorithms in renal cell carcinoma. Urol Clin North Am. 2008;35: ; vii. 3. Cindolo L, Patard J, Chiodini P, et al. Comparison of predictive accuracy of four prognostic models for nonmetastatic renal cell carcinoma after nephrectomy: a multicenter European study. Cancer. 2005;104: Kattan MW. Do we need more nomograms for predicting outcomes in patients with prostate cancer? Nat Clin Pract Urol. 2008;5: Galfano A, Novara G, Iafrate M, et al. Mathematical models for prognostic prediction in patients with renal cell carcinoma. Urol Int. 2008;80: Di Blasio CJ, Rhee AC, Cho D, Scardino PT, Kattan MW. Predicting clinical end points: treatment nomograms in prostate cancer. Semin Oncol. 2003;30: Liu Z, Lv J, Ding K, Fu Q, Cao Q, Wang F. Validation of the current prognostic models for nonmetastatic renal cell carcinoma after nephrectomy in Chinese population: a 15- year single center experience. Int J Urol. 2009;16: Morgan TM, Elashoff RM. Effect of covariate measurement error in randomized clinical trials. Stat Med. 1987;6: Galfano A, Novara G, Iafrate M, et al. Mathematical models for prognostic prediction in patients with renal cell carcinoma. Urol Int. 2008;80: Kattan MW, Reuter V, Motzer RJ, Katz J, Russo P. A postoperative prognostic nomogram for renal cell carcinoma. J Urol. 2001;166: Karakiewicz PI, Briganti A, Chun FK, et al. Multi-institutional validation of a new renal cancer-specific survival nomogram. J Clin Oncol. 2007;25: Sorbellini M, Kattan M, Snyder M, et al. A postoperative prognostic nomogram predicting recurrence for patients with conventional clear cell renal cell carcinoma. J Urol. 2005;173: Tan MH, Kanesvaran R, Li H, et al. Comparison of the UCLA Integrated Staging System and the Leibovich score in survival prediction for patients with nonmetastatic clear cell renal cell carcinoma. Urology. 2010;75: ;e Edge S, Page, DL, Balch CM, Fleming ID, Morrow M, eds. AJCC cancer staging manual. 6th ed. New York, NY: Springer-Verlag; Patard J, Leray E, Rodriguez A, Rioux-Leclercq N, Guille F, Lobel B. Correlation between symptom graduation, tumor characteristics and survival in renal cell carcinoma. Eur Urol. 2003;44: Al-Radi OO, Harrell FE Jr, Caldarone CA, et al. Case complexity scores in congenital heart surgery: a comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system. J Thorac Cardiovasc Surg. 2007;133: Harrell FE. Regression modelling strategies: with applications to linear models, logistic regression and survival analysis. New York, NY: Springer-Verlag; Leibovich BC, Blute ML, Cheville JC, et al. Prediction of progression after radical nephrectomy for patients with clear cell renal cell carcinoma: a stratification tool for prospective clinical trials. Cancer. 2003;97: Vickers AJ, Cronin AM, Kattan MW, et al. Clinical benefits of a multivariate prediction model for bladder cancer: a decision analytic approach. Cancer. 2009;115: Paoletti X, Oba K, Burzykowski T, et al. Benefit of adjuvant chemotherapy for resectable gastric cancer: a metaanalysis. JAMA. 2010;303: Baddi L, Benson A 3rd. Adjuvant therapy in stage II colon cancer: current approaches. Oncologist. 2005;10: Seruga B, Hertz PC, Wang L, et al. Absolute benefits of medical therapies in phase III clinical trials for breast and colorectal cancer. Ann Oncol. 2010;21: Motzer R, Hutson T, Tomczak P, et al. Sunitinib versus interferon alfa in metastatic renal-cell carcinoma. N Engl J Med. 2007;356: Kattan MW. Editorial comment. Urology. 2010;75:1371; author reply Chun FK, Karakiewicz PI, Briganti A, et al. A critical appraisal of logistic regression-based nomograms, artificial neural networks, classification and regression-tree models, look-up tables and risk-group stratification models for prostate cancer. BJU Int. 2007;99: Kattan MW. Comparison of Cox regression with other methods for determining prediction models and nomograms. J Urol. 2003;170:S6-9; discussion S Kattan MW. Nomograms are superior to staging and risk grouping systems for identifying high-risk patients: preoperative application in prostate cancer. Curr Opin Urol. 2003;13: Shariat SF, Karakiewicz PI, Palapattu GS, et al. Nomograms provide improved accuracy for predicting survival after radical cystectomy. Clin Cancer Res. 2006;12: Shariat SF, Karakiewicz PI, Suardi N, Kattan MW. Comparison of nomograms with other methods for predicting outcomes in prostate cancer: a critical analysis of the literature. Clin Cancer Res. 2008;14: Tan MH, BT T. Gene expression profiling predicts survival in clear cell renal cell carcinoma. Proc Am J Soc Clin Oncol Cancer December 1, 2011