EU regulatory concerns about missing data: Issues of interpretation and considerations for addressing the problem. Prof. Dr.

Transcription

1 EU regulatory concerns about missing data: Issues of interpretation and considerations for addressing the problem Prof. Dr. Karl Broich

2 Disclaimer No conflicts of interest Views expressed in this presentation are the author's personal views and not necessarily the views of BfArM or EMA K Broich EU regulatory issues regarding missing data 26 September 2016 Page 2

3 Agenda Current regulatory basis Sources of missing data Missing data and trial validity Example: Simulation of a depression trial Interpretational issues Estimands Addressing the problem Sensitivity analyses Conclusions K Broich EU regulatory issues regarding missing data 26 September 2016 Page 3

4 Regulatory discussion on missing data in confirmatory Phase III studies (1) Relevant regulatory documents ICH E9 Guideline Statistical Principles for Clinical Trials (1998) EMA Guideline on Missing Data in Confirmatory Clinical Trials (2010) National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials (2010) ICH Concept Paper E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials (2014) K Broich EU regulatory issues regarding missing data 26 September 2016 Page 4

5 Regulatory discussion on missing data in confirmatory Phase III studies (2) Issues to discuss: How to avoid missing data? How to consider/address (non-)adherence? measuring efficacy assuming perfect adherence or real adherence or in those patients who tolerate treatment How to treat missing data? how to treat missing data w.r.t. adherence? and how to interpret analysis/missing data imputation K Broich EU regulatory issues regarding missing data 26 September 2016 Page 5

6 Sources of missing data in confirmatory Phase III studies Treatment discontinuation due to adverse events lack of efficacy others often leads to missing data follow-up of discontinuing patients avoids missingness Study drop-out treatment discontinuation and no follow-up Intermediate missing data usually less relevant K Broich EU regulatory issues regarding missing data 26 September 2016 Page 6

7 Missing data in CNS trials Substantial amount of drop-outs in depression drop-out may be up to 50% average drop-out rate 20 to 30% others: schizophrenia, Alzheimer Follow-up usually poor follow-up of patients that discontinue treatment lack of information on non-adherent patients but: estimation of de-facto efficacy would require follow-up of patients that discontinue treatment targeting a treatment-policy estimand treatment benefit in all subjects irrespective of treatment adherence K Broich EU regulatory issues regarding missing data 26 September 2016 Page 7

8 Regulatory concerns about missing data in confirmatory clinical trials (1) Analysis in completers only not compliant with ITT principle treatment dependent patient selection biased effect estimates and lack of type-1 error control invalid conclusions (e.g. false positive decisions ) Missing data imputation based on specific assumptions regarding (unknown) missing data requires definition of a relevant estimation target (estimand), e.g. treatment benefit if all patients adhered or treatment benefit in all patients regardless of adherence or treatment benefit attributable to the randomized treatment or treatment benefit in those who adhere to treatment K Broich EU regulatory issues regarding missing data 26 September 2016 Page 8

9 Regulatory concerns about missing data in confirmatory clinical trials (2) Missing data imputation potential concerns about underlying assumptions and resulting validity e.g. LOCF usually invalid in progressive diseases (e.g. dementia) potential concerns about target of estimation e.g. longitudinal models may target treatment benefit if all patients adhered to treatment hypothetical target that appears less relevant usually several sensitivity analyses required to show robustness of the results w.r.t. to underlying assumptions to evaluate different estimands K Broich EU regulatory issues regarding missing data 26 September 2016 Page 9

10 De-facto and de-jure estimands treatment dropout retrieved data placebo active treatment de-facto (difference in all randomized patients) de-jure (difference if all patients adhered) end of trial time K Broich EU regulatory issues regarding missing data 26 September 2016 Page 10

11 Example: Simulation of depression trials BfArM research project on missing data and non-adherence Longitudinal data (Hamilton Score) Non-adherence: Treatment discontinuation Some data were collected after treatment discontinuation Different drop-out mechanisms treatment dropout (TD) study dropout (SD) SD time TD time retrieved data from TD to SD Data generation according to a two-piece linear mixed model Leuchs et al (2014). Statistics in Medicine 33 K Broich EU regulatory issues regarding missing data 26 September 2016 Page 11

12 Bias for de-jure effect Bias for de-facto effect Example: Simulation of depression trials BfArM research project on missing data and non-adherence Bias of different analysis strategies for de-jure and de-facto estimands true de-jure effect = 2 (difference if all subjects adhered) true de-facto (treatment policy) effect = 0 (difference in all subjects) Analysis strategies 1: Multiple Imputation (Pattern- Mixture Model) 2: Joint Model of drop-out and outcome 3: Mixed Model, all data 4: Mixed Model, only data under treatment equal drop-out 30% unequal drop-out 25% und 35% Leuchs et al (2014). Statistics in Medicine % 70% 40% 100% 70% 40% Proportion of subjects followed-up K Broich EU regulatory issues regarding missing data 26 September 2016 Page 12

13 Example: Simulation of depression trials Conclusions Longitudinal Mixed Model analysis of on-treatment data targets de-jure estimand Longitudinal Mixed Model analysis of all data (off- and on-treatment) still shows relevant bias w.r.t. de-facto (treatment policy) estimand if follow-up is poor Joint model of outcome and time to drop-out behaves best but would require further investigation on robustness K Broich EU regulatory issues regarding missing data 26 September 2016 Page 13

14 Proposed procedure Be clear about the trial s objective (i.e. primary estimand) before deciding trial design and analysis Primary estimand Clinical trial design Clinical trial design Customize the design considering the primary estimand Analysis method Sensitivity analyses Analysis method Choose a primary analysis applicable for the chosen design and addressing the primary estimand Sensitivity analyses Select a number of different sensitivity analyses Leuchs et al (2015). Therapeutic Innovation & Regulatory Science 49. K Broich EU regulatory issues regarding missing data 26 September 2016 Page 14

15 Regulatory conclusions on missing data and estimands (1) Which estimand addresses best clinical relevance? Treatment policy estimand most likely targets clinical relevance for a given population Treatment effect in tolerators may be relevant for patients but require complex causal inference and assumptions for a valid conclusion (without active run-in) De-jure like estimands (if all patients adhered) are hypothetical parameters difficult to justify but: may be most sensitive for non-inferiority conclusions Many other options to be discussed e.g. composite of different estimands related to reasons for drop-out K Broich EU regulatory issues regarding missing data 26 September 2016 Page 15

16 Regulatory conclusions on missing data and estimands (2) Treatment policy estimand fails if no or only few de-facto (retrieved) data are available requiring unverifiable assumptions difference between de-facto and de-jure can hardly be substantiated without data strong de-facto conclusions require de-facto data patient follow-up after drop-out needed K Broich EU regulatory issues regarding missing data 26 September 2016 Page 16

17 Sensitivity analyses to assess the robustness of trial results! Robustness of the estimation method Robustness of the estimand Robustness with regard to generalizability of trial results Internal validity external validity K Broich EU regulatory issues regarding missing data 26 September 2016 Page 17

18 Conclusions Missing data highly relevant issue in depression trials interpretational issues related to missing data Primary estimand to be agreed upon first design and analyse accordingly Sensitivity analyses relevant to address internal validity (concerning underlying assumptions) external validity (concerning clinical relevance addressed by different estimands) Treatment policy or attributable estimand relevant for population based conclusions treatment policy estimand require follow-up of (most) patients lack of follow-up result in the need for unverifiable assumptions K Broich EU regulatory issues regarding missing data 26 September 2016 Page 18

19 References ICH Expert Working Group (1999). Statistical principles for clinical trials (ICH E9). Statistics in Medicine, 18: EMA (2010). Guideline on Missing Data in Confirmatory Clinical Trials. National Research Council of the National Academies (2010). The Prevention and Treatment of Missing Data in Clinical Trials. Washington, D.C.: National Academies Press. Mallinckrodt CH et al (2012). A structured approach to choosing estimands and estimators in longitudinal clinical trials. Pharmaceut Statist, 11: O Neill RT and Temple R (2012). The prevention and treatment of missing data in clinical trials: an FDA perspective on the importance of dealing with it. Clin Pharmacol Ther, 91: Leuchs AK, Zinserling J, Schlosser-Weber G, Berres M, Neuhäuser M, Benda N (2014). Estimation of the treatment effect in the presence of non-compliance and missing data. Statistics in Medicine, 32: ICH concept paper (2014) E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials Leuchs AK, Zinserling J, Brandt A, Wirtz D, Benda N (2015). Choosing appropriate estimands in clinical trials. Therapeutic Innovation & Regulatory Science, 49: Leuchs AK, Brandt A, Zinserling J, Benda N (2016). Disentangling estimands and the intention-totreat principle. Pharmaceut Statist (accepted for publication). K Broich EU regulatory issues regarding missing data 26 September 2016 Page 19