SURVIVAL CURVES WITH NON-RANDOMIZED DESIGNS: HOW TO ADDRESS POTENTIAL BIAS AND INTERPRET ADJUSTED SURVIVAL CURVES Workshop W25, Wednesday, May 25, 2016 ISPOR 21 st International Meeting, Washington, DC, USA Abdalla Aly Pharmerit International Eberechukwu Onukwugha University of Maryland, Baltimore Tony Okoro Bristol-Myers Squibb Caitlyn Solem Pharmerit International INTRODUCTION TO SURVIVAL CURVES Abdalla Aly Pharmerit International 2 1
WHAT IS A SURVIVAL CURVE? No Treatment Treatment Results: In our sample, the median survival was 5 months for the untreated group and 19 months for the treated group. Interpretation: Population: In 5 months, half of the untreated patients will die. In 19 months half of the treated patients will die. Patient: If you get treated, there is a 50% chance of survival 19 months from diagnosis. If you don t get treated, there is a 50% chance of survival 5 months from diagnosis. 3 CAPTURE SPECIFIC INFORMATION RELEVANT TO THE DECISION MAKER Does it apply to me? Does it apply to the patients I treat? Does it apply to my plan members? Does it apply to those impacted by the policy? Does it apply to whomever I think it applies to? Picture sources: www.dreamstime.com; www.featurepics.com; www.thelundreport.org; blog.al.com; www.colourbox.com 4 2
MITIGATE BIAS INHERENT IN NON- RANDOMIZED STUDY DESIGNS Within the survival curves, have the authors adjusted for biases in non-randomized study designs: Selection bias Confounding bias Lead time bias Others 5 REAL-WORLD EXAMPLE: IMPACT OF SURGERY ON SURVIVAL FOR GLIOBLASTOMA Data Source: Surveillance Epidemiology and End Results (SEER) Research Database Publicly available dataset hosted by the National Cancer Institute Composite of 18 cancer registries in the SEER Research database. Collects demographic, clinical, and cause of death information on newly diagnosed cancer patients. Cohort: Patients with incident glioblastoma multiforme (malignant brain tumor) Research Question: What is the survival benefit associated with gross total resection? Key Characteristics for Adjustment: sex, race/ethnicity, age, tumor size at diagnosis 6 3
REAL-WORLD EXAMPLE: SELECTION AND CONFOUNDING BIAS Selection bias age, race, sex, tumor size Confounding bias GTR Research shows that younger patients with smaller tumors are better candidates for GTR Survival Independent of GTR receipt, older patients with larger tumors have poor prognosis Inverse Probability of Treatment Weighting Average Covariate Method Corrected Group Prognosis Method 7 THE AVERAGE COVARIATE METHOD Caitlyn Solem Pharmerit International 8 4
KEY VARIABLES IN DATASET Variable Description Values Death Whether the patient had died or not (outcome identifier). 0 = Did not die 1 = Died survtime Time from diagnosis to death or end of follow up (outcome) Continuous, measured in months surgprim Did the patient receive surgery (Primary independent variable) 0 = No 1 = Yes race_ethn Patient s race/ethnicity (Covariate) 1 = Non-Hispanic White 2 = Non-Hispanic Black 3 = Hispanic 4 = Other sex Patient s sex (Covariate) 1 = male 2 = female age Patient s age (Covariate) 1 = <18 2 = 18-44 3 = 45-64 4 = 65-74 5 = 75+ Tumor size Tumor size (Covariate) 1 = <3 cm 2 = 3-7 cm 3 = >7 cm 9 OVERVIEW OF METHOD What it does: Provides you a single estimate of survival for patients who did and did not have surgery for an average or reference patient What type of question does it answer: For a patient with characteristics A, B, C (or with the characteristics reflective of the average of the cohort), what is the survival likelihood among those who did and did not have surgery? Output: Covariate adjusted Kaplan-Meier curves 10 5
SAS CODE CONTROL LI NG FOR COVARI ATES AT MEAN VALUES proc phreg data=&dsn noprint; model &timevar*&outcom(&cnsrval)=rac_ethn age male size_c; strata &ctrlvar; baseline out=adjset survival=survival; run; Macro variable Value Purpose %dsn gbm Specify the dataset name that is in your SAS WORK library for the Cox model %ctrlvar surgprim Assign the name of stratification variable %outcom death Assign the name of event variable %cnsrval 0 Assign the value of censoring for event variable %timevar survtime Assign the name of time variable 11 OUTPUT MEAN OF CATEGORI ES 12 6
DATA BEHIND THE OUTPUT 45% MALE AND 55% FEMALE Variable race_ethn Values age 1 = <18 2 = 18-34 3 = 35-44 4 = 45-64 5 = 65-74 6 = 75+ sex Tumor size 1 = Non-Hispanic White 2 = Non-Hispanic Black 3 = Hispanic 4 = Other 1 = male 2 = female 1 = <3 cm 2 = 3-7 cm 3 = >7 cm 13 SAS CODE CONTROL LI NG FOR COVARI ATES AT REFERENCE VALUES proc phreg data=&dsn noprint; class rac_ethn (param=ref ref='1') age (param=ref ref='1') male (param=ref ref='1') size_c (param=ref ref='1'); model &timevar*&outcom(&cnsrval)=rac_ethn age male size_c/rl; strata &ctrlvar; baseline out=adjset survival=survival / nomean; run; Macro variable Value Purpose %dsn gbm Specify the dataset name that is in your SAS WORK library for the Cox model %ctrlvar surgprim Assign the name of stratification variable %outcom death Assign the name of event variable %cnsrval 0 Assign the value of censoring for event variable %timevar survtime Assign the name of time variable 14 7
DATA BEHIND THE OUTPUT NON-HISPANIC WHITE, <18 YEARS, MALE, <3CM TUMOR SIZE Variable race_ethn Values age 1 = <18 2 = 18-34 3 = 35-44 4 = 45-64 5 = 65-74 6 = 75+ sex Tumor size 1 = Non-Hispanic White 2 = Non-Hispanic Black 3 = Hispanic 4 = Other 1 = male 2 = female 1 = <3 cm 2 = 3-7 cm 3 = >7 cm 15 OUTPUT LOWER RISK GROUP: NHW, MALE, <18 YEARS, TUMOR SIZE<3 CM, 16 8
DATA BEHIND THE OUTPUT NON-HISPANIC WHITE, 75+ YEARS, MALE, >7CM TUMOR SIZE Variable race_ethn Values age 1 = <18 2 = 18-34 3 = 35-44 4 = 45-64 5 = 65-74 6 = 75+ sex Tumor size 1 = Non-Hispanic White 2 = Non-Hispanic Black 3 = Hispanic 4 = Other 1 = male 2 = female 1 = <3 cm 2 = 3-7 cm 3 = >7 cm 17 OUTPUT HIGHER RISK GROUP: NHW, MALE, 65+ YEARS, TUMOR SIZE>7 CM, 18 9
SUMMARY Ideally suited to predicting survival in a particular subgroup Physician or patient looking at likelihood, given their characteristics ACM provides hazard for a hypothetical average individual Assumes continuous covariates Mean of covariates make sense only when covariates are continuous Can result in a hard-to-interpret adjustment: 40% male and 60% female More meaningful to estimate at a particular level: females. 19 THE CORRECTED GROUP PROGNOSIS METHOD Eberechukwu Onukwugha University of Maryland, Baltimore 20 10
INTRODUCTION Bottom up group-averaging approach 1-3 No need to assume an average level of a covariate that may not apply at the individual level, e.g., 0.75 as a value for an indicator of married status. Assumes that the proportional hazards assumption is valid for included covariates 1. Nieto FJ, and J Coresh. Adjusting survival curves for confounders: A review and a new method. American Journal of Epidemiology. 1996; 143(10): 1059-1068. 2 Ghali WA et al. Comparison of 2 Methods for Calculating Adjusted Survival Curves From Proportional Hazards Models. JAMA. 2001; 286(12): 1494-1497. 3 Storer BE et al. Adjusted Estimates for Time-to-Event Endpoints. Lifetime Data Anal. 2008 Dec; 14(4): 484 495. 21 STEPS Develop survival curves using the estimated coefficients from the Cox PH model Obtain H t uc for each unique combination, uc, in the sample Obtain individual survival functions, S t uc = e H t uc Calculate a weighted average of S t uc Weight=proportional to number of individuals at the given level of UC Published example: Sample of 11,468 patients yielded 2,419 curves 1 Our example: Sample of 24,281 patients yielded 288 curves Plot survival curve based on averaged survival curve 1 Ghali WA et al. Comparison of 2 Methods for Calculating Adjusted Survival Curves From Proportional Hazards Models. JAMA. 2001; 286(12): 1494-1497. 22 11
AVERAGE COVARIATE VS CORRECTED GROUP PROGNOSIS The average covariate method (ACM) enters the mean value into one survival function to develop a curve that represents a hypothetical average individual The corrected group prognosis (CGP) method averages over separate survival functions to develop a curve that represents a (grouped) population average ACM vs. CGP Assume the following inputs into an exponential function: A = 1, 4, 2 µ=2.3 e μ = 0.1. σ3 i=1 e A i = 0.17 23 EFFECT OF INPUT VALUES One input value for the ACM and this input value is subject to influential values (e.g., prevalent condition, large hazard ratio) CGP has several input values and is robust to influential values Raw values Mean (µ) e μ A={1, 4, 2} 2.3 0.097 0.174 A={1,12,2} 5 0.007 0.167 A={1,12,14} 9 0.0001 0.123 3 i=1 ea i 24 12
ILLUSTRATION OF CALCULATION: TIME = 1 Surgery Count per combination Time S(t) S(t) x count Adjusted survival [S(t) x count)/n] Unique combination 1 0 150 1 1 1 x 150 = 150 Unique combination 2 0 50 1 0.8 0.8 x 50 = 40 Unique combination 3 0 20 1 0.75 0.75 x 20 = 15 230.5/250 = 0.922 Unique combination 4 0 30 1 0.85 0.85 x 30 = 25.5 Unique combination 1 1 150 1 0.92 0.92 x 150 = 138 Unique combination 2 1 50 1 0.97 0.97 x 50 = 48.5 Unique combination 3 1 20 1 0.95 0.95 x 20 = 19 235.5/250 = 0.942 Unique combination 4 1 30 1 1 1 x 30 = 30 25 ADJUSTED CURVES WITH CGP 26 13
DISCUSSION Adjusted curves fall within the unadjusted curves, as expected Unadjusted curves are subject to bias that works in opposite directions Unadjusted curve for no disease group may overestimate survival Unadjusted curve for disease group may underestimate survival CGP provides population-averaged hazards (as opposed to hazard for a hypothetical average individual) CGP averages actual survival curves (as opposed to averaging within the exponential function) 27 SUMMARY Ideally suited to predicting survival in a heterogeneous group of individuals. Assumes categorical covariates, including categorical variables based on recoded continuous variables. In the case of non-proportional hazards, utilize the stratified Cox PH. 28 14
THE INVERSE PROBABILITY WEIGHTING METHOD Abdalla Aly Pharmerit International 29 SELECTION BIAS IN NON- RANDOMIZED DESIGNS Observational study Randomized clinical trial GTR No GTR 30 15
IPTW CREATES A PSEUDO- POPULATION No IPTW Observational study IPTW Observational study GTR No GTR 31 RATIONALE: MIMICS RANDOMIZATION EX-POST No IPTW GTR No GTR Old 1 (25%) 3 (75%) 4 Young 3 (75%) 1 (25%) 4 Total 4 4 8 IPTW GTR No GTR Old 4 (50%) 4 (50%) 8 Young 4 (50%) 4 (50%) 8 Total 8 8 16 P (GTR old) = ¼ P (GTR young) = ¾ P (No GTR old) = ¾ P (No GTR young) = ¼ P (GTR old) = ½ P (GTR young) = ½ P (No GTR old) = ½ P (No GTR young) = ½ Unfair comparison (Tossing an unfair coin) Fair comparison (Tossing a fair coin) 32 16
STEPS Two Stage Model: First stage: Probability of receiving surgery from a logistic regression: pr GTR =1 Log = β 1 [pr GTR =1] 0 + β 1 Demog i + β 2 Clinical i + β 3 Cancer i + e i Second Stage: Survival from a weighted Cox proportional hazards model. H i t = H 0 t e β 1GTR iw + e iw In the first stage: the concordance statistic was 0.76 33 SAS CODE FIRST STAGE proc logistic data=&dsn; class &covars; /*categorical*/ model &ctrlvar (event='1') = &covars; /*categorical, continuous*/ output out=ps_data prob=ps; RUN; DATA ps_data; SET ps_data; IF &ctrlvar = 1 THEN treated_ps = ps; ELSE treated_ps =.; IF &ctrlvar = 0 THEN untreated_ps = ps; ELSE untreated_ps =.; run; proc means data=ps_data (keep=ps) noprint; var ps; output out=ps_mean mean=marg_prob; RUN; data _NULL_; set ps_mean; call symput("marg_prob",marg_prob); run; data ps_data; set ps_data; if &ctrlvar = 1 then iptw = 1/ps; else if &ctrlvar = 0 then iptw = 1/(1- ps); if &ctrlvar = 1 then siptw = &marg_prob/ps; else if &ctrlvar = 0 then siptw = (1- &marg_prob)/(1-ps); label ps = "Propensity Score" iptw = "Inverse Probability of Treatment Weight" siptw = "Stabilized Inverse Probability of Treatment Weight"; run; 34 17
SAS CODE SECOND STAGE proc phreg data=ps_data; model &timevar * &outcom (&cnsrval) = &ctrlvar freq siptw/notruncate; baseline covariates=ctrlset out=adjset survival=survival; run; 35 ADJUSTED CURVES WITH IPTW 36 18
DISCUSSION IPTW creates a pseudo-population in which randomization is imposed IPTW reweights the sample by the inverse probability of their treatment group IPTW does not average survival curves; just reweights the sample then applies the weight in a Cox model May use ACM or CGP after reweighting, if you believe residual confounding is present Can include many variables to derive the propensity score C statistic is often overlooked 37 RECAP Comparison ACM CGP IPTW Bias Confounding Confounding Selection Position Applies to Adjusted curves are shifted if subgroups; may cross if adjusted at mean Hypothetical average individual Adjusted curves usually fall within the unadjusted curves Heterogeneous population Individual-specific? Yes No No Method Covariate type Averaging within the exponential function Should be continuous for proper interpretation Averaging actual survival curves Must be categorical Covariate numbers Limited Limited Many Understandability Easy Intermediate Difficult Programming effort for beginners Easy Difficult Difficult Adjusted curves usually fall within the unadjusted curves Heterogeneous population where selection into treatment is clear Reweighted sample Weight can include both categorical and continuous covariates 38 19
TRANSLATION: WHO CARES? Tony Okoro Bristol-Myers Squibb 39 STAKEHOLDER PERSPECTIVES Industry Researchers Confirming outcomes seen in RCT critical in demonstrating product value Variety of data sources can be used to complement RCT outcomes Clinicians Access Decision Makers Regulatory Agencies Clinical trial efficacy does not automatically translate to real-world effectiveness Adjusted survival data used to help understand heterogeneous populations Desire improved outcomes to justify costs associated with treatment Increased reliance on real-world data for decision making Overall survival from RCT is the gold standard for full regulatory approval Post-marketing safety requirements mandated in some instances 40 20
CASE STUDY #1: AVERAGE COVARIATE METHOD Harrison, L. D., et al. "Comparing effectiveness with efficacy: outcomes of palliative chemotherapy for non-small-cell lung cancer in routine practice."current Oncology 22.3 (2015): 184. 41 CASE STUDY #1: AVERAGE COVARIATE METHOD Registry adjusted survival estimates similar to RCT survival data Potentially misleading survival estimates Alternative methods not feasible without more trial data Harrison, L. D., et al. "Comparing effectiveness with efficacy: outcomes of palliative chemotherapy for non-small-cell lung cancer in routine practice."current Oncology 22.3 (2015): 184. 42 21
CASE STUDY #2: INVERSE PROBABILITY WEIGHTING METHOD Langer, Corey, et al. "Comparison of survival and hospitalization rates between Medicare patients with advanced NSCLC treated with bevacizumab carboplatin paclitaxel and carboplatin paclitaxel: A retrospective cohort study." Lung Cancer 86.3 (2014): 350-357. 43 CASE STUDY #2: INVERSE PROBABILITY WEIGHTING METHOD Longer adjusted median survival with BCP compared to CP Adjusted and unadjusted results presented and consistent with RCT More reliability in results communication when baseline covariates included in adjustment Langer, Corey, et al. "Comparison of survival and hospitalization rates between Medicare patients with advanced NSCLC treated with bevacizumab carboplatin paclitaxel and carboplatin paclitaxel: A retrospective cohort study." Lung Cancer 86.3 (2014): 350-357. 44 22
CASE STUDY #3: CORRECTED GROUP PROGNOSIS METHOD Sun, Zhifu, et al. "Histologic grade is an independent prognostic factor for survival in non small cell lung cancer: An analysis of 5018 hospital-and 712 population-based cases." The Journal of thoracic and cardiovascular surgery131.5 (2006): 1014-1020. 45 CASE STUDY #3: CORRECTED GROUP PROGNOSIS METHOD Histologic grade shown to be a significant prognostic factor after adjustment Sun, Zhifu, et al. "Histologic grade is an independent prognostic factor for survival in non small cell lung cancer: An analysis of 5018 hospital-and 712 population-based cases." The Journal of thoracic and cardiovascular surgery131.5 (2006): 1014-1020. 46 23
Abdalla Aly Pharmerit International GROUP DISCUSSION Eberechukwu Onukwugha University of Maryland, Baltimore Tony Okoro Bristol-Myers Squibb Caitlyn Solem Pharmerit International 24