Estimands in clinical trials - PDF Free Download

Statistical Methodology Novartis Basel, Switzerland Estimands in clinical trials Heinz Schmidli BIAS, Parma, 28-29 Sep 2017

Disclaimer The views and opinions expressed in this presentation and on the slides are solely those of the presenter and not necessarily those of Novartis. Novartis does not guarantee the accuracy or reliability of the information provided herein. 2

Outline ICH E9(R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials Estimands and their role in clinical trials Acknowledgements o Frank Bretz, Mouna Akacha o Dan Scharfstein 3

Outline ICH E9(R1) Addendum History and Motivation Content Examples Outlook 4

ICH E9(R1) Addendum History and Motivation ICH (www.ich.org) International Council for Harmonisation of technical requirements for pharmaceuticals for human use Joint discussions on scientific and technical aspects of drug registration: Regulatory authorities (EU, US, Japan,...) Pharmaceutical industry (EFPIA, PhRMA, JPMMA,...) ICH E9 (1998) STATISTICAL PRINCIPLES FOR CLINICAL TRIALS Clinical trial design, conduct and analysis Very strong focus on intention-to-treat (ITT) principle: effect of assigning patients to test and control 5

ICH E9(R1) Addendum History and Motivation Shortcomings of ITT principle often not targeting scientific question of interest Some events after randomization ( intercurrent events ) complicate description and interpretation of treatment effects Despite what you may have heard, randomized trials are not always free of confounding and selection bias. Hernán, Hernández-Diaz, Robins (2013) Annals of Internal Medicine See also: Sheiner & Rubin (1995) Clin Pharm Ther, Sheiner (2002) Br J Clin Pharm 6

ICH E9(R1) Addendum History and Motivation Patients differ in response to treatment due to events after randomization ( intercurrent events ) Treatment switch 7

ICH E9(R1) Addendum History and Motivation How to define treatment effect in population of interest for primary variable in presence of intercurrent events? Treatment switch 8

ICH E9(R1) Addendum History and Motivation More than one treatment effect can be described and estimated, raising questions like: What is of interest for regulatory decision making? What and how to communicate to prescribers? Can these be estimated? 9

ICH E9(R1) Addendum History and Motivation US NAS report (2010) The Prevention and Treatment of Missing Data in Clinical Trials Commissioned by FDA (NAS: National Academy of Sciences) NAS report - Recommendation 1 The trial protocol should explicitly define a) the objective(s) of the trial; b) the associated primary outcome or outcomes; c) how, when, and on whom the outcome or outcomes will be measured; d) the measures of intervention effects, that is, the causal estimands of primary interest. These measures should be meaningful for all study participants, and estimable with minimal assumptions. 10

ICH E9(R1) Addendum Content ICH concept paper (2014) proposal ICH E9(R1) Addendum (2017) draft 1. Purpose and Scope 2. A Framework to Align Planning, Design, Conduct, Analysis and Interpretation 3. Estimands 4. Impact on Trial Design and Conduct 5. Impact on Trial Analysis 6. Documenting Estimands and Sensitivity Analysis 7. A Generic Example Glossary 11

ICH E9(R1) Addendum Content Structured framework to align trial objectives with analysis methods 12

ICH E9(R1) Addendum Content Estimand description A Population Patients targeted by scientific question B Variable Endpoint(s) to be obtained for each patient to address the scientific question C Intercurrent events - Specification of how to account for these to reflect the scientific question D Summary - Population-level summary for the variable which provides a basis for a comparison between treatments 13

ICH E9(R1) Addendum Content Five strategies for handling each intercurrent event discussed in ICH E9 Addendum. Treatment policy (ITT): Occurrence of intercurrent event irrelevant; Composite: Intercurrent event is component of variable; Hypothetical: Scenario is envisaged in which the intercurrent event would not happen; Principal stratum: Target population is subpopulation for which intercurrent event would not occur; While on treatment: Response to treatment prior to the occurrence of the intercurrent event is of interest. 14

ICH E9(R1) Addendum Example - Dapagliflozin Dapagliflozin case: RCT experimental vs control Diabetes patients, Change in HbA1c, rescue medication allowed Sponsor proposal: Exclude data after initiation of rescue medication, and use LOCF. However, FDA: While FDA has implicitly endorsed LOCF imputation for diabetes trials in the past, there is now more awareness in the statistical community of the limitations of this approach. Instead I have included a sensitivity analysis in which the primary HbA1c outcomes are used regardless of rescue treatment, and no statistical adjustment is made for rescue. This approach is also imperfect, but it comes closer to being a true intent-to-treat (ITT) analysis... 15

ICH E9(R1) Addendum Example - Dapagliflozin Focus on analysis/ missing data /statistics, rather than on trial objective/target of estimation Implied scientific questions of interest : Sponsor: Attempt to establish the treatment effect of the initially randomized treatments had rescue medication not been allowed; FDA: Compare treatment policies experimental plus rescue versus control plus rescue. Disagreement on estimand 16

ICH E9(R1) Addendum Example chronic heart failure EMA (2016) Guideline on clinical investigation of medicinal products for the treatment of chronic heart failure (draft) a) Heart Failure Hospitalisation (HFH) b) Terminal events: all-cause death, heart transplant, LVAD implant... recurrent HFH events may better characterise the prognosis...... analysis and interpretation are complicated by [...] terminal events...... terminal events will usually need to be addressed explicitly in the statistical analysis since certain naïve approaches to the analysis of hospitalisation rate data will not reflect the true effect...... it is strongly recommended to seek Scientific Advice, when recurrent HFH is to be used as a part of a primary endpoint [...]. 17

ICH E9(R1) Addendum Example chronic heart failure Tendency to start with selection of primary analysis among the many (complex) methods proposed, based on power and type I error However: Reverse-engineering the implied target of estimation often very difficult Further complicated by ad-hoc choices which may change the target of estimation (e.g. handling intercurrent event of non-cv-death) Communication of treatment effect can be very challenging Some approaches implicitly rely on assumptions which seem scientifically meaningless (e.g. possibilty of hospitalization after death) Causal interpretation of treatment effect sometimes not possible Better starting point: What is an appropriate estimand? 18

ICH E9(R1) Addendum Outlook Addendum is draft comments to EMA until Feb 2018! However, regulatory authorities already use Addendum framework/language: "... provide an extended discussion on appropriate estimands and how they are supposed to be estimated..."; Addendum aims to improve interaction between sponsor and regulators. So hopefully we will see Better designs by sponsors Better assessments by regulators 19

Outline Estimands and their role in clinical trials Causal estimands Strategies Design Discussion Conclusions 20

Estimands Causal estimands NAS report revisited... Recommendation 1: "... the causal estimands..." "Estimation of the primary (causal) estimand, with an appropriate estimate of uncertainty, is the main goal of a clinical trial. " Many authors of NAS report experts on causal inference Little RJ, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, Neaton JD, Rotnitzky A, Scharfstein D, Shih WJ, Siegel JP, Stern H. Term "causal" not used in ICH E9(R1) Addendum, but framework/language aligned with causal reasoning 21

Estimands Causal estimands Causal inference well established science Neyman (1923) Agricultural experiments Extended by Rubin (1974) to non-randomized studies Important contributions by: Robins, Pearl,... Not well known among biostatisticians in the pharmaceutical industry (... I think...) 22

Estimands Causal estimands RCT Experimental (E) vs Control (C) treatment Population: patients with cancer Variable: time-to-death Y [years], from time of randomization Intercurrent events ignored (...as a starting point...) Potential outcome framework* # Patient Y(E) Y(C) 1 Adam 2 1 2 Bruce 5 7 3... Y(E): how long a patient would live if randomized to E Y(C): how long a patient would live if randomized to C *Neyman (1923), Rubin (1974) 23

Estimands Causal estimands Potential outcome framework # Patient Y(E) Y(C) Y(E)-Y(C) 1 Adam 2 1 +1 2 Bruce 5 7-2 3... Y(E): how long a patient would live if randomized to E Y(C): how long a patient would live if randomized to C Population causal estimand, e.g. average(y(e) Y(C)) = average(y(e)) average(y(c)) 24

Estimands Causal estimands # Patient Y(E) Y(C) 1 Adam 2 1 2 Bruce 5 7 3... Examples of population causal estimands: average( Y(E) ) average( Y(C) ) average( Y(E) ) / average( Y(C) ) average( log{y(e)} ) - average( log{y(c)} ) Accelerated Life Time median( Y(E) ) - median( Y(C) ) prop( Y(E) > 3years) ) - prop( Y(C) > 3years) ) RMST 3 years ( Y(E) ) RMST 3 years ( Y(C) ) Restricted Mean Survival Time Prob( Y i (E) > Y j (C) )... 25

Estimands Causal estimands Is the hazard ratio a causal estimand? Short answer: No! Aalen et al. (2015) Does Cox analysis of a randomized survival study yield a causal treatment effect? Lifetime Data Anal Hernan (2010) The Hazards of Hazard Ratios. Epidemiology... another issue is the physical or substantive basis for the proportional hazards model. I think that s one of his weaknesses, [and] that accelerated life models are in many ways more appealing because of their quite direct physical interpretation... Cox, 1994 If, however, the objective is more modestly summarization for largely descriptive purposes a formulation directly in terms of hazard functions has attractions... Cox, 1997 26

Estimands Causal estimands Potential outcomes Observed # Patient Y(E) Y(C) Assignment A Y Y(E) Y(C) 1 Adam 2 1 E 2 2? 2 Bruce 5 7 C 7? 7 3... Estimation: Y(E), Y(C) independent of Assignment A for RCT s average( Y A=E) average( Y A=C) unbiased estimate of average( Y(E) ) average( Y(C) ) Testing: Strict null hypothesis: Y i (E)=Y i (C) i=1,2,... Fisher s randomization-based testing Frequentist and Bayesian approches available for inference 27

Estimands Strategies ICH E9 Addendum strategies for intercurrent events Treatment policy (ITT): Occurrence of intercurrent event irrelevant; Composite: Intercurrent event is component of variable; Hypothetical: Scenario is envisaged in which the intercurrent event would not occur; Principal stratum: Target population is subpopulation for which intercurrent event would not occur; While on treatment: Response to treatment prior to the occurrence of the intercurrent event is of interest. Illustration in RCT setting: Experimental (E) vs Control (C) treatment 28

Estimands Strategies Population: Patients with diabetes Variable: Y=Change in HbA1c from baseline to 24 weeks [mmol/mol] Intercurrent event: R=intake of rescue medication # Patient Y(E) R(E) Y(C) R(C) 1 John 2 no 5 yes 2 Brian -8 yes 0 yes Strategies a) Treatment-policy/ITT: Y(E) vs Y(C) e.g. average( Y(E) ) average( Y(C) ) 29

Estimands Strategies # Patient Y(E) R(E) Y(E,R=no) Y(C) R(C) Y(C,R=no) 1 John 2 no 2 5 yes 7 2 Brian -8 yes -1 0 yes 1 Hypothetical Y(E,R=no): if rescue medication would not have been allowed Strategies a) Treatment-policy/ITT: Y(E) vs Y(C) b) Hypothetical: Y(E,R=no) vs Y(C,R=no) Dapagliflozin case Discussion between FDA and sponsor on technical level (whether to remove data after rescue medication), rather than on strategy level 30

Estimands Strategies Population: cancer patients with weight loss Variable: Y=Change in lean body mass from baseline to 12 weeks [kg] Intercurrent event: D=death (0=no, 1=yes) # Y(E) D(E) Y(C) D(C) Z(E) Z(C) 1 3 0-2 0 3-2 2 1-3 0-100 -3 If a patient dies before 12 weeks, the value of variable Y does not exist Strategies: a) Composite: new variable Z = (1-D) x Y - D x 100 [kg] Causal estimand median( Z(E) ) - median( Z(C) ) See also: Permutt & Li (2017) Pharm Stat, Wang et al. (2017) Biometrics 32

Estimands Strategies # Y(E) D(E) Y(C) D(C) Principal Stratum 1 3 0-2 0 1 2 1-3 0 0 3 4 0 1 0 4 1 1 0 5-1 0-3 0 1 Strategies: b) Principal stratum estimand: average( Y(E) D(E)=D(C)=0 ) - average( Y(C) D(E)=D(C)=0 ) Effect in subgroup with D(E)=D(C)=0 i.e. patients which would survive regardless of treatment received See also: Frangakis & Rubin (2002) Biometrics 33

Estimands Strategies # Y(E) D(E) Y last (E) D time (E) Z(E)... 1 3 0 3 >12 3/12 2 1 2 8 2/8 Strategies: c) While alive: slope Z Causal estimand e.g. average( Z(E) ) - average( Z(C) ) 34

Estimands Design Estimand can sometimes be better estimated using non-standard trial designs run-in, enrichment, randomized withdrawal, titration,... Example: Patients with heart failure LCZ696 (test) vs enalapril (control) 1st run-in phase: all patients on control, 10513 Followed by 2nd run-in phase: all remaining patients on test, 9419 Followed by randomization of remaining patients 8442 McMurray et al. (2014) NEJM 35

Estimands Design Primary endpoint: composite of CV death & hospitalization for heart failure McMurray et al. (2014) NEJM 36

Estimands Discussion Traditional estimand (Complex) model including treatment effect parameter θ Testing, with null hypothesis H0: θ=0 Causal estimand Marginal comparison of potential outcomes: Y(E) vs Y(C) Testing, with strict null hypothesis H0: Y i (E)=Y i (C) i=1,2,... Advantages of causal estimand formulation Potential outcome framework natural for medical doctors What happens to my patient if I give him treatment E vs treatment C No statistical model needed to discuss with clinicans Easier to think about how to account for intercurrent events 37

Estimands Discussion Role of statistical models Traditional estimand is model parameter, which is not interpretable if model is not adequate Causal estimand is model-free However, models can/should be used for inference! Example Y=time-to-death Causal estimand median( Y(E) ) - median( Y(C) ) To obtain point estimates/ci/p-value/posterior distribution, one could use e.g. a Weibull model 38

Estimands Discussion Causal reasoning Provides guidance on which estimands make sense No causation without manipulation Holland and Rubin, 1986 Example hypothetical estimand Diabetes patients, change in HbA1c Intercurrent event: R=intake of rescue medication Y(E,R=no): if rescue medication would not have been allowed OK Intercurrent event: R=Discontinuation due to AE Y(E,R=no): if patients would not have had AEs? Intercurrent event: R=Death Y(E,R=no): if patients would not die??? 39

Estimands Conclusions ICH E9(R1) Addendum should lead to more transparent discussions between sponsor and regulators on trial objectives and target of estimation Causal reasoning key for appropriate choice of estimand Potential outcome framework useful for interactions between statisticians, clinicians... and patients. 40

References Akacha M, Bretz F, Ohlssen D, Rosenkranz G, Schmidli H (2017) Estimands and Their Role in Clinical Trials. Statistics in Biopharmaceutical Research, 9:3, 268-271. Akacha M, Bretz F, Ruberg S (2017) Estimands in clinical trials - broadening the perspective. Stat Med 36:5-19. Phillips A, Abellan-Andres J, Soren A, Bretz F, Fletcher C, France L, Garrett A, Harris R, Kjaer M, Keene O, Morgan D, O'Kelly M, Roger J (2017) Estimands: discussion points from the PSI estimands and sensitivity expert group. Pharm Stat. 16(1):6-11 Holzhauer B, Akacha M, Bermann G (2015) Choice of Estimand and Analysis Methods in Diabetes Trials with Rescue Medication. Pharm Stat, 14, 433 447. Leuchs A, Zinserling J, Brandt A, Wirtz D, Benda N (2015) Choosing Appropriate Estimands in Clinical Trials. Ther Inn & Reg Sci 49, 584 592. Permutt T (2016) A Taxonomy of Estimands for Regulatory Clinical Trials with Discontinuations. Stats Med, 35, 2865 2875. 41

References Imbens GW, Rubin DB (2015) Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press Hernán MA, Robins JM (2018). Causal Inference. Boca Raton: Chapman & Hall/CRC, forthcoming www.hsph.harvard.edu/miguel-hernan/causal-inference-book Little RJ, Rubin DB (2000) Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches. Annu Rev Health, 21:121-45. Pearl (2009) Causal inference in statistics: An overview. Statistics Surveys 3, 96 146. Dawid (2000) Causal inference without counterfactuals (with comments and rejoinder). J American Statistical Association, 95:407 448. 42