Predictive Models. Michael W. Kattan, Ph.D. Department of Quantitative Health Sciences and Glickman Urologic and Kidney Institute

Transcription

1 Predictive Models Michael W. Kattan, Ph.D. Department of Quantitative Health Sciences and Glickman Urologic and Kidney Institute

2 Treatment for clinically localized prostate cancer Trade off: Substantial risk of immediate, long term, side effects Potential gain in life expectancy and avoidance of progression symptoms

3 Why do patients need estimates of treatment benefit and harm 1. To make a good treatment decision 2. For legal reasons informed consent 3. To understand why you are better!

4 Should you pick your patient s treatment, even if he asks you to? No. You are trying to balance quantity and quality of life for the patient without really formally assessing his preferences. Shared decision making provides other benefits. Else, Patient is at increased risk for regret.

5 Whom/how to treat? Some treatment options: 1. AS 2. ORP+LND 3. ORP+LND+NEO 4. ORP-LND 10. BRACHY 11. BRACHY+NEO 12. CRYO 13. HIFU 5. EBRT 6. EBRT+LNRT 7. EBRT+LNRT+HORM 8. LRP 9. LRP+NEO

6 Why do you tend to advocate a particular treatment? You think predicted outcomes are better 1. BCR 2. METS 3. DEATH 4. IMPOTENCE 5. INCONTINENCE 6. BOWEL DYSFUNCTION 7. LENGTH OF STAY 8. RETURN TO WORK

7 The patient needs a Metagram Treatment options Outcomes BCR AS RRP HIFU METS 3 10 Must tailor the probabilities to the individual patient.

8

9 When The Patient Wants A Prediction, What Options Does The Clinician Have? Predict based on knowledge and experience Deny ability to predict at the individual patient level Quote an overall average to all patients Assign the patient to a risk group, i.e. high, intermediate, or low Apply a model

10 Preoperative Nomogram for Prostate Cancer Recurrence Points PSA T2a T2c T3a Clinical Stage T1c T1ab T2b ? Biopsy Gleason Grade Total Points Month Rec. Free Prob Instructions for Physician: Locate the patient s PSA on the PSA axis. Draw a line straight upwards to the Points axis to determine how many points towards recurrence the patient receives for his PSA. Repeat this process for the Clinical Stage and Biopsy Gleason Sum axes, each time drawing straight upward to the Points axis. Sum the points achieved for each predictor and locate this sum on the Total Points axis. Draw a line straight down to find the patient s probability of remaining recurrence free for 60 months assuming he does not die of another cause first. Instruction to Patient: Mr. X, if we had 100 men exactly like you, we would expect between <predicted percentage from nomogram - 10%> and <predicted percentage + 10%> to remain free of their disease at 5 years following radical prostatectomy, and recurrence after 5 years is very rare. Kattan MW et al: JNCI 1998; 90: Michael W. Kattan and Peter T. Scardino

11 CaPSURE Heterogeneity within Risk Groups Preoperative Nomogram Predicted Probability Nomogram Values by Prostate Cancer Risk Group Low Intermediate High Risk Group J Urol Apr;173(4):

12 Making a nomogram Usually a regression model (Cox or logistic) Try machine learning techniques (neural nets, optimized trees like CART) Keep continuous variables continuous but relax linearity assumptions P-values for predictors don t matter No variable selection or univariable screening Bottom line is its predictive accuracy

13 Patients used in the preoperative recurrence nomogram validation USA Cleveland Clinic 1168 N LSU 583 UCLA 617 USC 1501 Europe Hamburg 1134 Rotterdam 475 Australia Sydney 754 Total 6232 Patients Graefen et al., JCO, 2002.

14 Predicted vs. Actual Freedom from Recurrence Range of Nomogram Predicted Probabilities Percent Free from Recurrence at 5 Years Predicted Actual N Graefen et al., JCO, 2002.

15 Nomogram Validation by Concordance Index (AUC) Coin Toss Perfect Discrimination Preoperative Nomogram International Validation 1. Randomly select 2 patients a. One of whom fails (reaches the event of interest) b. The other must survive longer 2. Concordance index is the proportion of these pairs in which patient who fails first also had worse nomogram prediction. Graefen et al., JCO, 2002.

16 Continuous Models vs. Staging/Grouping Systems Model Comparator CI (M vs C) Preop L/I/H Risk Groups 0.67 vs Preop + IL6/TGFβ1 L/H Risk Groups 0.84 vs Pre XRT L/I/H Risk Groups 0.76 vs Melanoma SLN+ AJCC Stage 0.69 vs Pancreatic Ca AJCC Stage 0.64 vs Gastric Ca AJCC Stage 0.77 vs Breast Ca NPI Groups 0.69 vs Sarcoma CART Groups 0.77 vs. 0.74

17 When The Patient Wants A Prediction, What Options Does The Clinician Have? Predict based on knowledge and experience Deny ability to predict at the individual patient level Quote an overall average to all patients Assign the patient to a risk group, i.e. high, intermediate, or low Apply a model

18 Urologists vs. Preoperative Nomogram 10 case descriptions from 1994 MSKCC patients presented to 17 urologists In addition to PSA, biopsy Gleason grades, and clinical stage, urologists were provided with patient age, systematic biopsy details, previous biopsy results, and PSA history. Preoperative nomogram was provided. Urologists were asked to make their own predictions of 5 year progression-free probabilities with or without use of the preoperative nomogram. Concordance indices: Nomogram = 0.67 Urologists = 0.55, p<0.05 Ross P et al., Semin Urol Oncol, 2002.

19 Nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy Vanzee K, et al., Ann Surg Oncol., 2003.

20 Breast Cancer Prediction: 17 Clinicians vs. Nomogram on 33 Patients Nomogram AUC 0.72 Sensitivity: Proportion of women with positive nodes predicted to have positive nodes Clinician AUC 0.54 Specificity: Proportion of women with negative nodes predicted to have negative nodes

21 ROC Curves Individual Clinicians and Nomogram Sensitivity Areas Nomogram Specificity

22 Biases in Human Prediction Data Acquisition Process Output availability inconsistent wishful thinking selective perception heuristics illusion of control base rate insensitive non-linear response frequency conservative illusory correlation environment data representation sources Feedback recall, overconfident, hindsight bias, chance adapted from Hogarth, 1988

23 When The Patient Wants A Prediction, What Options Does The Clinician Have? Predict based on knowledge and experience Deny ability to predict at the individual patient level Quote an overall average to all patients Assign the patient to a risk group, i.e. high, intermediate, or low Apply a model

24 PATIENTS AND METHODS mrna isolated from 79 radical prostatectomy specimen of primary prostate tumors Recurrent patients: 37 Criteria for recurrence: 3 consecutive PSA rises Non-recurrent patients: 42 Criteria for non-recurrence: undetectable PSA 60 months Median follow-up: 72 months (range ) Stephenson A, et al., Cancer, 2005.

25 NOMOGRAM VS GENE MODEL - Results of Leave-One-Out Cross-Validation Survival Functions GENE EXPRESSION MODEL Survival Functions POST-OPERATIVE NOMOGRAM Progression-Free Survival.8 GENE.6 REC REC-censored NR NR-censored.4.2 N censored Months from Prostatectomy (months) Concordance Index: Time from Prostatectomy (months) Stephenson A, et al., Cancer., 2005.

26 Paclitaxel, i.v. estramustine phosphate, and carboplatin for castrate, metastatic prostate cancer Solit D, et al., Cancer, 2003.

27 Nomograms for clinical trial design Example: CALGB 90203, preoperative therapy for patients at high risk of failure following surgery for prostate cancer Points PSA T2a T2c T3a Clinical Stage T1c T1ab T2b ? Biopsy Gleason Grade Total Points Month Rec. Free Prob < 60% Eastham et al., Urology, 2004.

28 Software to facilitate real-time predictions Software for the Palm Pilot, PocketPC, and Windows Desktop Computers Software is free from or Individual predictions should have a family of models behind them.

29

30 Conclusions Your patients need, and deserve, predictions of treatment outcomes. Formulas, such as those behind nomograms and risk calculators, are the most accurate choice for predictions because they are tailored to the patient. Nomogram predictions assist in: Patient counseling Shared decision-making Informed consent Clinical trial design and analysis Free software facilitates the computations.