ava ilable at www.sciencedirect.com journa l homepage: euoncology.europeanurology.com Internal and External Validation of a 90-Day Percentage Erection Fullness Score Model Predicting Potency Recovery Following Robot-assisted Radical Prostatectomy Linda M. Huynh a, Douglas Skarecky a, Timothy Wilson b, Clayon Lau b, Christian Wagner c, James Porter d, Jorn H. Witt c, Thomas E. Ahlering a, * a Department of Urology, University of California Irvine, Orange, CA, USA; b Department of Urology, City of Hope National Medical Center, Duarte, CA, USA; c Urology Department, St. Antonius-Hospital Gronau GmbH, Gronau, Germany; d Swedish Urology Group, Swedish Medical Center, Seattle, WA, USA Article info Article history: Accepted August 31, 2018 Associate Editor: Gianluca Giannarini Keywords: Erectile function Radical prostatectomy Validation Fullness Erections Potency Abstract Background: We previously reported a new post radical prostatectomy (RP) prediction model for men with normal baseline erectile function (EF) using 90-d postoperative erection fullness to identify men who might benefit from early EF rehabilitation. Objective: To prospectively internally and externally validate the use of this risk assessment model in predicting 1- and 2-yr post-rp EF recovery. Design, setting, and participants: We randomly assigned 297 patients with a preoperative International Index of Erectile Function 5 score of 22 25 undergoing robotassisted RP by a single surgeon to a training set and internal validation set at a ratio of 2:1. A prospective external validation set included 91 patients treated by five highvolume surgeons. Outcome measurements and statistical analysis: Potency was defined as erections sufficient for intercourse. To predict 1- and 2-yr potency recovery, logistic regression models were developed in the training set based on 90-d erection fullness of 0 24% or 25 100%. The resultant models were applied to the internal and external validation sets to calculate risk scores for 1- and 2-yr potency for each patient. Predictive validity was assessed using receiver operating characteristic (ROC) curves. Results and limitations: Percentage erection fullness was an independent predictor of 1- and 2-yr potency recovery in all data sets. Internal validation confirmed strong reliability in predicting 2-yr potency outcomes (area under the ROC curve [AUC] 0.87) and external validation illustrated similar reliability in predicting 1-yr potency outcomes (AUC 0.80). In the external validation, the model predicted a mean 1-yr potency recovery rate of 39.7% (standard deviation 3.2%), compared to the actual rate of 36.26%. Limitations include the short follow-up for this cohort. Conclusions: We present internal and external validation of a 90-d percentage erection fullness score, confirming that this metric is a robust predictor of post-rp EF recovery. Patient summary: Percentage erection fullness at 3 mo after radical prostatectomy discriminates patients with a low or a high probability of recovery of erectile function (EF), which can facilitate identification of a need for early EF rehabilitation. 2018 Published by Elsevier B.V. on behalf of European Association of Urology. * Corresponding author. Department of Urology, University of California Irvine, 333 City Boulevard West, Orange, CA 92868, USA. Tel.: +1 714 4566068; Fax: +1 888 3784524. E-mail address: tahlerin@uci.edu (T.E. Ahlering). https://doi.org/ 2588-9311/ 2018 Published by Elsevier B.V. on behalf of European Association of Urology.
2 1. Introduction Erectile dysfunction (ED) after radical prostatectomy (RP) has a significant impact on patient quality of life during the recovery process [1 3]. While the International Index of Erectile Function-5 (IIEF-5) is a well-accepted method for diagnosing preoperative erectile function (EF), there is no universal standard of assessment for recovery of erections satisfactory for intercourse (ESI) post-rp [4,5]. It is well established that EF recovery takes up to 12-24 months, a factor that necessitates long-term patient follow-up [6 9]. In this regard, an incomplete longitudinal patient response can significantly limit the accurate assessment of and time to recovery of EF. If there were an early means to identify a patient s individual probability of potency recovery, we could preemptively intervene in those who would benefit most from early rehabilitation. In 2018 we introduced the percentage erection fullness (PEF) score as a qualitative adjunct to the IIEF-5 [10]. Subsequently, we realized that low PEF at 90 d after RP reliably predicted poor 2-yr potency recovery in preoperatively potent men. While patients reporting 25 100% PEF at 90 d reliably recovered potency by 2 yr, patients reporting 0 24% PEF were approximately six times more likely to suffer from long-term impotence [10,11]. Using this threshold of 25% and its predictive capability, we sought to internally and externally validate this PEF scale in multicenter patient populations. 2. Patients and methods 2.1. Definition of potency and ED According to a systematic review by Ficarra and associates [8], the definition of potency most commonly used is ESI, or two affirmative answers to the following questions: Are your erections firm enough for penetration? and Are they satisfactory? On this basis, we assessed patients for potency recovery at 3, 9, 15, and 24 mo after RP using an affirmative ESI as the primary outcome measure. 2.2. Data collection Data collection was conducted in compliance with the Health Insurance Portability and Accountability Act after institutional review board approval from the University of California, Irvine (HS#1998-84). Patients were assessed for EF at all time points; if a patient had achieved potency at an early visit, he was still reassessed at subsequent follow-up visits throughout the 24-mo study period. 2.3. Patient population 2.3.1. Internal validation We queried 667 consecutive patients with a preoperative IIEF-5 score of 22-25 undergoing robot-assisted RP (RARP) at our institution for inclusion. Patients undergoing non-nerve-sparing surgery (n = 2) and those lost to follow-up at any time point were excluded (n = 241); only patients with at least 2 yr of follow-up were included. The management protocol at our institution included a daily low dose of a phosphodiesterase-5 (PDE-5) inhibitor (tadalafil or sildenafil) to enhance blood flow and encourage EF recovery after RP. All men using any other ED therapies, including penile rehabilitation and/or ED medications beyond the daily dose of PDE-5 inhibitors, were excluded from the analysis (n = 127). Overall, the internal validation included 297 patients with a preoperative IIEF-5 score of 22-25 (without the use of PDE-5 inhibitors) undergoing RARP performed by a single surgeon (T.E.A.). Pairwise comparisons between the 297 men included versus the full cohort confirmed that there were no significant biases in demographic characteristics, covariates, or recovery. 2.3.2. External validation External validation included a prospective cohort of patients undergoing RARP performed by five external surgeons at three academic institutions. Patients from the internal validation cohort were not included in the external validation and all data were collected by a blinded third-party contractor. Surgeons had experience of between 500 and 3500 cases each performed >100 procedures per year. Preoperative demographics and patient-reported outcomes were systematically collected at 3, 6, and 12 mo after RP as part of a 1-yr randomized control trial that was ultimately negative. Of the 199 undergoing RARP, 91 (45.7%) had a preoperative IIEF-5 score of 22 25, followed the same management protocol as described above, and were ultimately included in the external validation cohort. Final data collection occurred between May and October 2014. 2.4. Statistical analysis 2.4.1. Development of the risk prediction model The internal cohort of 297 patients was split 2:1 via random number generation to generate a training data set (n = 202) and a validation data set (n = 95). For the training data set, descriptive statistics were used to characterize the cohort and candidate variables were then evaluated by examining their bivariate associations with ED using the Fisher exact test. This examination included an analysis of risk factor interactions, and variables demonstrating an association with ED in unadjusted bivariate analysis (p < 0.05) were included in the final multivariable logistic regression model. These included age, nerve-sparing status, PEF (<25% vs 25 100%), preoperative IIEF-5 score, preoperative prostate-specific antigen (PSA), and clinical stage. 2.4.2. Estimating the risk of ED Two logistic regression models were developed in the training data set to estimate the 1- and 2-yr risk of impotence for each patient. Both risk models were then applied to the internal and external validation cohorts. A risk score was calculated by multiplying each variable in the multivariate model by its b coefficient (natural logarithm of the odds ratio) and summing the products. This risk score (RS) was then used to compute the odds of 2-yr potency recovery according to the formula odds(2-yr potency) = e RS. Odds were then converted to a probability [Pr (2-yr potency)] according to Pr(2-yr potency) = odds(2-yr potency)/1 + odds(2-yr potency). Receiver operating characteristic (ROC) analysis was then used to evaluate the performance of each risk model in discriminating patients who would or would not ultimately recover potency. All statistical analyses were performed using SPSS v.25 (IBM, Armonk, NY, USA). 3. Results Table 1 lists the demographic characteristics of the training data set and the internal and external validation cohorts. There were no significant differences between these cohorts for any of the covariates analyzed.
3 Table 1 Baseline characteristics of patients in the internal and external validation cohorts a Training set Internal validation set External validation set Patients (n) 202 95 91 Age (yr) 58.34 7.11 59.38 6.35 60.58 9.53 Preoperative IIEF-5 score 24.33 0.88 22.51 5.74 24.15 0.91 Prostate-specific antigen (ng/ml) 5.52 3.53 6.73 5.96 9.29 7.61 Body mass index (kg/m 2 ) 26.7 3.21 27.05 3.00 27.01 4.75 Clinical stage, n (%) T1 149 (74.50) 64 (68.82) 38 (42.70) T2 50 (25.00) 28 (30.11) 51 (57.30) T3 1 (0.50) 1 (1.08) 0 (0.00) Clinical Gleason group, n (%) a Grade group 1 (3 + 3) 118 (58.13) 51 (53.13) 31 (34.07) Grade group 2 (3 + 4) 44 (21.67) 25 (26.04) 28 (30.77) Grade group 3 (4 + 3) 18 (8.87) 13 (13.54) 21 (23.08) Grade group 4 (4 + 4, 3 + 5, 5 + 3) 18 (8.87) 3 (3.13) 6 (6.59) Grade group 5 (4 + 5) 5 (2.46) 2 (2.08) 5 (5.49) Nerve sparing, n (%) Unilateral 27 (13.37) 12 (12.63) 17 (18.68) Bilateral 178 (88.12) 83 (87.37) 74 (81.32) Lymph node dissection, n (%) b 75 (36.95) 30 (31.25) 25 (27.47) IIEF-5 = International Index of Erectile Function-5. a Data for continuous variables are reported as mean standard deviation. b Clinical Gleason group reported according to the International Society for Urological Pathology 2014 Gleason grading system. c All surgeons followed standard clinical care guidelines and preoperative nomograms to determine the extent of lymph node dissection. 3.1. Model training The internal validation cohort of 297 patients was split into a training set and an internal validation set. In the training set, we identified two risk groups (those reporting 0 24% vs 25 100% PEF at 90 d after RP). Of note, 78% and 82% of men reporting 25 100% PEF at 90 d recovered ESI at 1 and 2 yr after RP, respectively. By contrast, only 17% and 30% of men reporting 0 24% PEF at 90 d recovered ESI at 1 and 2 yr, respectively. With this threshold of 25% PEF, two risk models were generated, one for prediction of 1-yr and one for prediction of 2-yr potency recovery. Table 2 illustrates these multivariable model generated from the training set. ROC curve analysis demonstrated good discrimination in potency recovery in both models (area under the ROC curve [AUC] 0.887 and 0.895; Figs. 1 and 2), with PEF as an independent predictor of sexual function recovery, after consideration of other covariates. 3.2. Internal validation The multivariable model for 2-yr potency prediction (Table 2) was applied to the internal validation set and demonstrated good discrimination in predicted potency versus impotency (AUC 0.870). ROC curve analysis identified sensitivity of 97.3% and specificity of 87.3% at the PEF threshold of 25%. The mean standard deviation (SD) for the predicted probability of 2-yr potency recovery was Table 2 Logistic regression model of factors contributing to potency at 24 mo (AUC 0.887) and 12 mo (AUC 0.895) after adjusting for significant preoperative and 3-mo postoperative characteristics Parameter b SE Wald p value OR (95% CI) Potency at 24 mo Age 0.035 0.038 0.852 0.356 0.965 (0.895 1.041) Nerve sparing (unilateral vs bilateral) 0.370 0.734 0.255 0.614 1.448 (0.344 6.105) Percentage fullness (<25% vs 25 100%) 1.767 0.583 9.175 0.002 5.854 (1.866 18.364) Preoperative IIEF-5 score 0.047 0.285 0.027 0.869 0.954 (0.546 1.667) Preoperative prostate-specific antigen 0.029 0.049 0.349 0.554 0.971 (0.882 1.070) Clinical stage 0.647 0.553 1.372 0.241 0.523 (0.177 1.546) Constant 1.371 7.804 0.031 0.861 3.940 Potency at 12 mo Age 0.044 0.036 1.460 0.227 0.957 (0.892 1.028) Nerve sparing (unilateral vs bilateral) 0.494 0.702 0.494 0.482 1.639 (0.414 6.491) Percentage fullness (<25% vs 25 100%) 1.800 0.556 10.495 0.001 6.049 (2.036 17.971) Preoperative IIEF-5 score 0.007 0.288 0.001 0.981 1.007 (0.573 1.769) Preoperative prostate-specific antigen 0.021 0.042 0.256 0.613 0.979 (0.902 1.063) Clinical stage 0.403 0.516 0.609 0.435 0.668 (0.243 1.838) Constant 0.981 7.713 0.016 0.899 2.666 AUC = area under the receiver operating characteristic curve; SE = standard error; OR = odds ratio; CI = confidence interval; IIEF-5 = International Index of Erectile Function-5.
4 4. Discussion Fig. 1 Receiver operating characteristic curve of 90-d percentage erection fullness predicting 24-mo potency. An erection fullness threshold of 25% has sensitivity of 97.3% and specificity of 87.3% for predicting recovery (area under the curve 0.870). Fig. 2 Receiver operating characteristic curve of 90-d percentage erection fullness predicting 12-mo potency. An erection fullness threshold of 25% has sensitivity of 97.3% and specificity of 87.3% for predicting recovery (area under the curve 0.801). 69.0% 3.4%, compared to the actual 70.8% of patients recovering potency at 2 yr. 3.3. External validation The multivariable model for 1-yr potency prediction (Table 2) was applied to the external validation set and demonstrated similarly good discrimination (AUC 0.801), with sensitivity of 86.0% and specificity of 83.8% at the PEF threshold of 25%. The mean SD predicted probability of 1- yr potency recovery was 39.7% 3.2%, compared to the actual 36.26% of patients recovering potency at 1 yr. Although multiple models are available for predicting potency recovery after RP, these nomograms rely exclusively on preoperative metrics and are dedicated to preoperative counseling [3 5]. The average survival following RP is 12 yr [12 14], which underscores the critical need to address improvements in patient urinary and sexual quality-of-life outcomes after RP. After continence, recovery of EF has a significant impact in most men, and particularly among younger men with normal baseline EF (IIEF-5 score of 22 25 without the use of PDE-5 inhibitors). Historically, at 3 mo after RARP, approximately 70% of preoperatively potent men report that they do not have ESI [8], and of those assessed with the IIEF-5, 85% report scores between 1 and 6 [10]. Although ESI and IIEF-5 provide accurate assessment of satisfactory recovery of erections at 3 mo after RARP, they do not reliably detect partial recovery or predict the longterm propensity for recovery [15 19]. Given that EF can take up to 24 mo to recover, an early metric that could predict the probability of recovery would significantly help in patient counseling and improve management strategies. In this prospective study, we carried out internal and external validation of a 90-d PEF scale as a metric for EF recovery at 1 and 2 yr after RARP. In our efforts to identify a threshold, we noted that men reporting 0 24% PEF recovered ESI just 30% of the time at 2 yr. In stark contrast, 82 99% of patients reporting 90-day 25 100% PEF recovered ESI by 2 yr [8]. Using this threshold of 25%, we noted an approximately sixfold improvement in potency recovery for patients reporting 25 100% PEF as compared to those with 0 24% PEF. This was strongly supported in external validation: men reporting 25 100% PEF were also 5.85 times more likely to recover 1-yr ESI as compared to those reporting 0 24% PEF. Furthermore, the actual proportion of patients who recovered ESI fell within 1SD of the predicted proportion, confirming good predictability of both models. Despite heterogeneity in the external cohort in terms of five different surgeons, their varying surgical techniques and experience, and patient demographics, the predictive capability of PEF was maintained. These results have several practical clinical implications for men with normal baseline EF (preoperative IIEF-5 score of 22 25). First, our internal and external validations confirm that 90-d PEF of 0 24% is a valid metric predicting poor 2-yr recovery of ESI. Given their low predicted probability of recovery, these men would be good early candidates for consideration of pre-emptive rehabilitation and/or aggressive interventions. Such interventions are not only more effective when pursued early [20], but early intervention can also significantly improve patient quality of life during the post-rp recovery process [21,22]. Conversely, men reporting 25 100% PEF at 90 d have a significant chance of ESI recovery and it would be logical to manage them accordingly. Overall, PEF facilitates 90-d risk stratification so that a surgeon can quickly and accurately identify the patients at high and low risk of long-term ED. Furthermore, since 90-d outcomes correlate
5 well with long-term outcomes, surgeons could use PEF as a real-time metric of surgical outcomes. In this regard, PEF is an independent predictor of potency recovery, regardless of typical confounders for EF recovery (age, preoperative PSA, and Gleason grade), making it a recovery model that can be readily applied to different patient populations, surgeons, and disease stages. As an independent predictor of potency recovery patterns, PEF accurately models nerve regrowth and functional recovery [10,11]. Limitations of this study include the short follow-up time of 1 yr and the small patient numbers per surgeon in the external validation cohort. Because external validation was performed on data from a randomized control trial, only 1-yr follow-up was available. However, our results are consistent with previous studies in showing linear progression of recovery between years 1 and 2, supporting interpolation [8]. Furthermore, recommendations for validation of patient-reported outcome measures range from two to 20 subjects per item, with a mean subject/item ratio of 28 [23]. Given the relatively large sample size of our cohort, we were able to restrict the analysis to patients with preoperative IIEF-5 scores of 22 25 and pre-emptively minimize significant variation among each of the surgeons and patient populations. Historically, men with a preoperative IIEF-5 score of <21 have not been in focus, largely because of the significantly lower probability of EF recovery [6,8,10]. Overall, the fact that PEF maintains its discriminant validity in small patient populations (40 patients per surgeon) supports its strength as an early metric for longterm outcomes. Confirmation in a large-scale prospective multicenter study (currently under way) would be optimal. 5. Conclusions We presented internal and external validation data for PEF as a robust predictor of post-rp EF recovery. While the IIEF- 5 historically discriminates well among potent men, PEF is able to discriminate post-rp partial erection recovery versus a low probability of overall recovery, which is a unique advantage at 90 d after RP that can be easily applied to varying surgeons and patient populations. Author contributions: Thomas E. Ahlering had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Ahlering, Huynh. Acquisition of data: Skarecky. Analysis and interpretation of data: Huynh, Ahlering, Skarecky. Drafting of the manuscript: Huynh, Ahlering. Critical revision of the manuscript for important intellectual content: Ahlering. Statistical analysis: Huynh. Obtaining funding: None. Administrative, technical, or material support: Wilson, Lau, Wagner, Porter, Witt. Supervision: None. Other: None. Financial disclosures: Thomas E. Ahlering certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None. Funding/Support and role of the sponsor: None. Acknowledgements: We are grateful to Dr. Sheldon Greenfield and Dr. Corey Hugen for their invaluable support. This work was accomplished with special regards to Drs. Edward and Arthur Lui, in memory of their parents Mr. & Mrs. L.H.M. Lui. References [1] Bianco Jr FJ, Scardino PT, Eastham JA. Radical prostatectomy: longterm cancer control and recovery of sexual and urinary function ( trifecta ). Urology 2005;66:83 94. [2] Wagner A, Link R, Pavovich C, et al. Use of validated quality of life questionnaire to assess sexual function following laparoscopic radical prostatectomy. Int J Impot Res 2006;18:69 76. [3] Alemozaffar M, Regan M, Cooperberg M, et al. Prediction of erectile function following treatment for prostate cancer. JAMA 2011;306:1 7. [4] Rosen RC, Cappellerini JC, Smith MD, et al. Development and evaluation of an abridged, 5-item version of the International Index of Erectile Function (IIEF-5) as a diagnostic tool for erectile dysfunction. J Sex Med 1999;11:319 26. [5] Rosen RC, Riley A, Wagner G, et al. The International Index of Erectile Function (IIEF): a multidimensional scale for assessment of erectile dysfunction. Urology 1997;9:822 30. [6] Ficarra V, Borghesi M, Suardi N, et al. Long-term evaluation of survival, continence, and potency (SCP) outcomes after robotassisted radical prostatectomy (RARP). BJU Int 2013;112:338 45. [7] Walsh PC, Marschke, Ricker D, et al. Patient-reported urinary continence and sexual function after anatomic radical prostatectomy. Urology 2000;5:58 61. [8] Ficarra V, Novara G, Ahlering TE, et al. Systematic review and metaanalysis of studies reporting potency rates after robot-assisted radical prostatectomy. Eur Urol 2012;62:418 30. [9] Lardas M, Liew M, Bergh R, et al. Quality of life outcomes after primary treatment for clinically localised prostate cancer: a systematic review. Eur Urol 2017;72:869 85. [10] Huynh LM, Osann K, Skarecky D, Ahlering TE. Predictive modelling of 2-year potency outcomes using a novel 90-day erection fullness scale after robot-assisted radical prostatectomy. BJU Int 2018;122:249 54. [11] Huynh LM, Skarecky D, Ahlering TE. A prospective, multi-institutional trial of 90-day percent fullness-predictive modeling of potency recovery post-robot assisted radical prostatectomy (RARP). J Endourol 2017;31(Suppl 2), MP29-16A. [12] Jeong BC, Chalfin HJ, Lee SB, et al. The relationship between extent of extraprostatic extension and survival following radical prostatectomy. Eur Urol 2015;67:342 6. [13] Abdollah F, Boorjian S, Cozzarini, et al. Survival following biochemical recurrence after radical prostatectomy and adjuvant radiotherapy in patients with prostate cancer: the impact of competing causes of mortality and patient stratification. Eur Urol 2013;64:557 64. [14] Roehl KA, Han M, Ramos CG, et al. Cancer progression and survival rates following anatomical radical retropubic prostatectomy in 3,478 consecutive patients: long-term results. J Urol 2004;172:910 4. [15] Vertosick E, Vickers A, Cowan J, et al. Interpreting patient-reported urinary and sexual function outcomes across multiple validated instruments. J Urol 2017;197:1239 40.
6 [16] Carlucci JR, Nabizada-Pace F, Samadi DB. Robot-assisted laparoscopic radical prostatectomy: technique and outcomes of 700 cases. Int J Biomed Sci 2009;5:201 8. [17] Murphy DG, Kerger M, Crowe H, Peters JS, Costello AJ. Operative details and oncological and functional outcome of robotic-assisted laparoscopic radical prostatectomy: 400 cases with a minimum of 12 months follow-up. Eur Urol 2009;55:1358 67. [18] Samadi DB, Muntner P, Nabizada-Pace F, Brajtbord JS, Carlucci J, Lavery HJ. Improvements in robot-assisted prostatectomy: the effect of surgeon experience and technical changes on oncologic and functional outcomes. J Endourol 2010;24:1105 10. [19] Dowrick AS, Wootten AC, Murphy DG, Costello AJ. We used a validated questionnaire : what does this mean and is it an accurate statement in urologic research? J Urol 2015;85:1306 10. [20] Capogrosso P, Salonia A, Briganti A, Montorsi F. Postprostatectomy erectile dysfunction: a review. World J Mens Health 2016;34:73 88. [21] Patel HR, Ilo D, Shah N, et al. Effects of tadalafil treatment after bilateral nerve-sparing radical prostatectomy: quality of life, psychosocial outcomes, and treatment satisfaction results from a randomized, placebo-controlled phase IV study. BMC Urol 2015;15:1 10. [22] Emanu JS, Avildsent IK, Nelson CJ. Erectile dysfunction after radical prostatectomy: prevalence, medical treatments, and psychosocial interventions. Curr Opin Support Palliat Care 2016;10:102 7. [23] Anthoine E, Moret L, Regnault A, Sbille V, Hardouin JB. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcome measures. Health Quality Life Outcomes 2015;12:176.