Trial design in the presence of non-exchangeable subpopulations

Transcription

1 Trial design in the presence of non-exchangeable subpopulations Brian P. Hobbs, PhD Cancer Biostatistics Section Head in The Taussig Cancer Institute Associate Staff, Department of Quantitative Health Sciences in The Lerner Research Institute May 2018

2 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Seminal Model of Immuno-oncology Chen and Mellman (2013). Oncology meets immunology: the cancer-immunity cycle

3 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Chen and Mellman (2013). Oncology meets immunology: the cancer-immunity cycle

4 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Chen and Mellman (2013). Oncology meets immunology: the cancer-immunity cycle

5 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Limitations of IHC immune pathology: PDL1 positivity 176 Lung cancer patients treated with resection. Samples were scored for PDL1+ positivity PD-L1+ TMA (Biopsy) Median (IQR) of % Tumor PDL1+ % Tumor PDL1+ TMA (biopsy) % Tumor PDL1+ Whole Section T1 (n=81) 18.8 ( ) 2.3 ( ) T2 (n=71) 28.1 ( ) 3.8 ( ) T3/4 (n=21) 20.7 ( ) 6.0 ( ) PD-L1+ Whole Section

6 co-first authors Motivation

7 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Current Applications of Cancer Radiomics Feature extraction Survival association Standard of care imaging? Fried IJROBP 2014 Survival association Aerts Nat Com 2014 Tang Koay, ASTRO

8 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Radiomics Signatures of Immune Environment Feature extraction Survival association Standard of care imaging Immune microenvironment? Fried IJROBP 2014 Survival association Aerts Nat Com 2014 Tang Koay, ASTRO

9 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Immune Phenotypes of NSCLC Teng MW, Ngiow SF, et al. Cancer Res (2015) Classifying Cancers Based on T-cell Infiltration and PD-L1

10 CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference Immune Phenotypes of NSCLC 4000 P=0.002 Infiltrate CD3 count %Tumor PD-L1+ Representative pathology staining CD3hi / PDL1lo CD3hi / PDL1hi CD3lo / PDL1lo CD3lo / PDL1hi CD3 PD-L1 Teng MW, Ngiow SF, et al. Cancer Res (2015) Classifying Cancers Based on T-cell Infiltration and PD-L1

11 Cancer Radiomics Imaging Models of Immune Phenotypes

12 Cancer Radiomics Radiomics signatures of Immune Phenotypes raining set: grouping, images, pathology and outcomes Radiomics cluster creation Pathology characteristics Cluster C: Low intensity (n=30) Low heterogeneity Cluster A: High intensity (n=32) Low heterogeneity Cluster D: Low intensity (n=11) High heterogeneity Cluster B: High intensity (n=41) High heterogeneity Cluster D probability %Tumor PD-L1 CD3 count 1.0 ( ) 1339 ( ) 1.7 ( ) 1728 ( ) 0.9 ( ) 2005 ( ) 1.2 ( ) 955 ( ) Cluster overall survival P=0.01 CD3 Count Log% PDL1 positive

13 Cancer Radiomics ation set: grouping, images, pathology and outcomes Radiomics signatures of Immune Phenotypes Radiomics cluster assignment Cluster C (n=40) Cluster A (n=56) Cluster D (n=38) Cluster B (n=42) Pathology characteristics %Tumor PD-L1 CD3 count 2.6 ( ) 1887 ( ) 4.0 ( ) 1650 ( ) * * 2.4 ( ) 1914 ( ) * 3.5 ( ) 1700 ( ) Cluster overall survival CD3 Count Cluster D probability Log% PDL1 positive P=0.002 P=0.001 Stage I only

14 Basket Design Designs for Precision Medicine

15 Case Studies Case Study: Vemurafenib non-melanoma basket trial Baskets Enrolled Evaluable Responders Posterior probability Pr(π > 0.15) based on response only NSCLC CRC (vemu) CRC (vemu + cetu) Bile Duct ECD or LCH ATC Bayesian Posterior Probability Pr(π > 0.15 Data) > θ, with θ fixed to control type I error at 0.10 Posterior probability Number (%) data reported Baskets in article: Enrolled Vemurafenib Evaluable inresponders Multiple Nonmelanoma Pr(π > 0.15) Cancers with BRAF V600 Mutations, NEJM (2015) based on response only 1 NSCLC (55)

16 Basket Design Basket Design Dilemma Implicit to the concept of a basket trial is exchangeable treatment effects across baskets early basket trials have been criticized [JCO Cunanan 2017] for implementing basketwise analysis strategies which failed to convey to the extent of statistical evidence for exchangeability across subtypes/baskets ignore additional sources of inter-patient heterogeneity, either observed or unobserved in the study in the presence of imbalanced enrollment, basketwise analyses fail to elucidate evidential measures of effect in small baskets conversely, pooling patients across baskets under the assumption of inter-patient exchangeability induces bias and limits the designs power for identifying favorable subtypes in the presence of heterogeneity of effect across basket labels.

17 Basket Design Bayesian Modeling to assess exchangeable effects across baskets/subtypes, is it useful?

18 Basket Design Freidlin and Korn Table 3. Empirical probabilities of rejecting the null hypothesis: 10 subgroups (no interim monitoring, 25 patients per subgroup, 10,000 replications) Design True response rate in each subgroup Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (Berry et al.; ref. 13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Abbreviation: HB, hierarchical Bayesian. not work. Therefore, relinquishing the strong control of the beginning of this article. In the first scenario, patients

19 Basket Design Table 3. Empirical probabilities of rejecting the null hypothesis: 10 subgroups (no interim monitoring, 25 patients per subgroup, 10,000 replications) Design True response rate in each subgroup Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (Berry et al.; ref. 13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Case Subgroup-specific analyses HB model 1 moderate borrowing HB model 1 strong borrowing HB model 2 (13) Abbreviation: HB, hierarchical Bayesian.

20 Basket Design Conventional Hierarchical Models are limited!

21 Biostatistics (2018) 19, 2,pp doi: /biostatistics/kxx031 Advance Access publication on July 6, 2017 Bayesian hierarchical modeling based on multisource exchangeability ALEXANDER M. KAIZER, JOSEPH S. KOOPMEINERS Division of Biostatistics, University of Minnesota, A460 Mayo Building, MMC Delaware St. SE, Minneapolis, MN 55455, USA BRIAN P. HOBBS The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd. Houston, TX 77030, USA SUMMARY Bayesian hierarchical models produce shrinkage estimators that can be used as the basis for integrating supplementary data into the analysis of a primary data source. Established approaches should be considered limited, however, because posterior estimation either requires prespecification of a shrinkage weight for each source or relies on the data to inform a single parameter, which determines the extent of influence or shrinkage from all sources, risking considerable bias or minimal borrowing. We introduce multisource exchangeability models (MEMs), a general Bayesian approach for integrating multiple, potentially nonexchangeable, supplemental data sources into the analysis of a primary data source. Our proposed modeling framework yields source-specific smoothing parameters that can be estimated in the presence of the data to facilitate a dynamic multi-resolution smoothed estimator that is asymptotically consistent while reducing the dimensionality of the prior space. When compared with competing Bayesian hierarchical modeling strategies, we demonstrate that MEMs achieve approximately 2.2 times larger median effective supplemental sample size when the supplemental data sources are exchangeable as well as a 56% reduction in bias when there is heterogeneity among the supplemental sources. We illustrate the application of MEMs using a recently completed randomized trial of very low nicotine content cigarettes, which resulted in a 30% improvement in efficiency compared with the standard analysis. Keywords: Bayesian hierarchical modeling; Heterogeneous sources of data; Multisource smoothing; Supplementary data.

22 Fig. 1: Each MEM is a combination of supplemental sources assumed exchangeable with the primary cohort in order to estimate the parameters of interest, θ p, and is contained within each box for Ω k. Within a box the solid arrows θ p and the observables, y h, represent which supplemental sources are assumed exchangeable with the primary cohort within the given MEM. Basket Design MEM Basket Design Kaizer, Koopmeiners, Hobbs (2017) Bayesian hierarchical modeling based on multi-source exchangeability. Biostatistics 26 REFERENCES Conceptual Diagram of Multi-source Exchangeability Models y 1 y 1 y p y p y 2... y H θ p Ω I Ω II θ p y H... y 2 ω I ω II q(θ p D) = K ω kq(θ p Ω k, D) k=i y 1 ω K Ω K... ω III Ω III y 1 y 2... θ p y p y p θ p... y 2 y H y H

23

24

25

26

27

28

29

30

31 MEM Methodology Case Study Analysis: Vemurafenib non-melanoma basket trial Figure 2. Prior, MAP, and PEP that result from Bayesian inference using the observed vemurafenib basket trial data

32 MEM Methodology Sequential Design based on Exchangeability Monitoring with MEM

33 Basket Design MEM Basket Design Permutation Study: Vemurafenib non-melanoma basket trial Basket pairs Exchangeability CRC.v CRC.vc CRC.vc BD CRC.v BD BD ATC BD ED.LH NSCLC BD CRC.vc ATC CRC.v ATC CRC.vc ED.LH NSCLC CRC.vc CRC.v ED.LH NSCLC CRC.v NSCLC ED.LH ED.LH ATC NSCLC ATC Enrollment Stage (n)

34 FIGURES Basket Design MEM Basket Design in Medicine Permutation Study: Vemurafenib non-melanoma basket trial Basket ATC ED.LH BD CRC.vc CRC.v NSCLC Effective Sample Size Total Enrollment Stage (n)

35 in Medicine Basket Design MEM Basket Design FIGURES Permutation Study: Vemurafenib non-melanoma basket trial Basket ATC Futility Probability 1.0 ED.LH 0.8 BD 0.6 CRC.vc 0.4 CRC.v 0.2 NSCLC Enrollment Stage (n)

36 Basket Design MEM Basket Design Freidlin and Korn example re-visited

37 Basket Design MEM Basket Design Comparing MEM to subgroup-specific analyses Scenarios in Freidlin and Korn CCR 2012 Scenarios Arm 1 Arm 2 Arm 3 Arm 4 Arm 5 Arm 6 Arm 7 Arm 8 Arm 9 Arm Scenarios in Freidlin and Korn CCR cenarios Arm 1 Arm 2 Arm 33 Arm Arm 5 Arm 0.16 Arm Arm Arm 9 Arm Frequentist Power for MEM and (subgroup-sp Frequentist Power for MEM and (subgroup-specific) analyses Frequentist Size Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 6 10% basketwise Global Null (Sc 5) 10% basketwise Single Null (Sc 1) 10% familywise Global Null (Sc 5) Frequentist Size Scenario 1 Scenario 2 Scenario (0.9) (0.9) (0.9) (0.9) (0.9) 10% basketwise Global Null (Sc 5) 10% basketwise Single Null (Sc 1) 10% familywise Global Null (Sc 5) (0.9) (0.9) ( (0.9) (0.9) (0.9) (0.9) (0.9) (0.9) (0.9) ( (0.659) (0.659) (0.659) (0.659) (0.659) (0.659) (0.659) (0

38 10% familywise Global Null (Sc 5) Global Null (Sc 5) (0.659) (0.659) (0.659) (0.659) (0.659) (0.659) Basket Design (0.659) MEM Basket (0.659) Design (0.659) (0.659) Comparing MEM to subgroup-specific analyses Fully Bayesian Evaluation Fully Bayesian Evaluation Null Scenario Alternative Scenario Null Scenario Alternative Scenario Priors Priors Prior 1 Prior Prior 2 Prior % Average 10% Average Basketwise Basketwise Type I error Type I error MEM model Priors Priors Threshold Threshold Average Average Type I Error Type I Error Average Average Power Power Prior Prior Prior Prior % Average 10% Average Familywise Familywise Type I error Type I error Prior Prior Prior Prior

39 Tumor Agnostic Biomarker Tumor Agnostic Biomarker? Is it predictive across lineage/histologies? Is it consistent/reliable across lineage/ histologies? Is there concordance across different labs/tests?

40

41 Recall Case Study A Case Study: Vemurafenib non-melanoma basket trial Baskets Enrolled Evaluable Responders Posterior probability Pr(π > 0.15) based on response only Number (%) of Prior Systemic Therapies NSCLC (55) 4 5 CRC (vemu) (70) CRC (vemu + cetu) (18) Bile Duct (63) ECD or LCH (50) 7 2 ATC (71) 1 1 Are patients with differing treatment histories statistically exchangeable as required to infer π? Pr(π > 0.15 Data) > θ No association between prior therapy reported in Table 1

42 MEM Methodology Personalized Treatment Selection Clinical, Immune Pathology, Radiomics Integrative Prognostic Model for NSCLC (n=411)

43 MEM Methodology Personalized Treatment Selection Integrative Predicted Event Probabilities

44 MEM Methodology Personalized Treatment Selection Integrative Predicted Event Probabilities

45 MEM Methodology Personalized Treatment Selection Integrative Predicted Event Probabilities

46 Precision Medicine Precision Medicine

47 Cancer Radiomics Understanding tumor/patient heterogeneity Lifestyle Characterizes the extent of benefit offered by a particular therapeutic strategy Clinical Demographics Histopathology Genomic DNA mutation DNA methylation mrna expression mirna expression Protein expression Predictive Prognostic Imaging Filtered-based Textural features Characterizes the disease extent and likelihood of recovery Ma, Stingo, Hobbs. Biometrics, (2016). Treatment Selection based on Personalized Predictive Treatment Utilities Ma, Hobbs, Stingo. Stat. Methods in Med. Res, (2017). Treatment Selection based on Personalized Predictive Failure-Time Ma, Stingo, Hobbs. submitted, (2017). Bayesian personalized treatment selection strategies that integrate predictive with prognostic determinants. Huang & Hobbs submitted (2017). Estimating mean local posterior predictive benefit for biomarker-guided treatment strategies

48 Precision Medicine Personalized Treatment Selection Bayesian partial exchangeability frameworks for prec med Ma, Stingo, Hobbs. Biometrics, (2016). Treatment Selection based on Personalized Predictive Treatment Utilities Quantifying similarities from clinical/molecular derived candidate features Characterizing pairwise partial statistical exchangeability Bayesian prediction models for treatment selection

49 Precision Medicine Personalized Treatment Selection Bayesian partial exchangeability frameworks for prec med Ma, Hobbs, Stingo. Stat. Methods in Med. Res, (2017). Treatment Selection based on Personalized Predictive Failure-Time Optimal treatment selection based on Bayesian predictive failure time Partial exchangeability based on tumor/patient characteristics, pairwise similarity Predict the probability of prolonging treatment failure

50 Precision Medicine Evidential Measures of Local Benefit Huang & Hobbs submitted (2017). Estimating mean local posterior predictive benefit for biomarker-guided treatment strategies Targeted Therapy Response Standard Therapy Response Cyclin_E2-R-C Cyclin_E2-R-C E(y A)-E(y B) Difference of Targeted and Standard Enrichment Strategy 1 Enrichment Strategy 2 Cyclin_E2-R-C Distribution of Estimated Local Posterior Predictive Benefit Denstiy Claudin-7-R-V Claudin-7-R-V Claudin-7-R-V Claudin-7-R-V Claudin-7-R-V Biomarker-guided Strategies Cyclin_E2-R-C Cyclin_E2-R-C Local Benefit

51 Acknowledgements Trainees Caimiao Wei (Pfizer), Shabnam Azadeh (FDA), Meilin Huang (Regeneron), Xiao Li (Gilead), Kate Shoemaker (Rice), Yuan Wang (Assist Prof Washington St.) Junsheng Ma (MD Anderson) Collaborators David Hong (MD Anderson) Francesco Stingo, (Univ Florence) Michele Guindani (UC Irvine) Chaan Ng (MD Anderson) Chad Tang (MD Anderson) Nan Chen (MD Anderson) Joe Koopmeiners(Minnesota) Alex Kaizer (Colorado) Michael Kane (Yale) Rick Landin (LJPC)