Evaluating Adaptive Dose Finding Designs and Methods

Transcription

1 Evaluating Adaptive Dose Finding Designs and Methods Amit Roy Bristol-Myers Squibb IIR Conference on Clinical Trial Design Princeton, NJ September 12-14, 2006

2 Acknowledgments: Adaptive Dose Finding Studies WG Alex Dmitrienko, Eli Lilly Amit Roy, BMS Beat Neuenschwander, Novartis Björn Bornkamp, U. of Dortmund Brenda Gaydos, Eli Lilly Chyi-Hung Hsu, Pfizer Frank Bretz, Novartis Frank Shen, BMS Franz König, Med. U. Vienna Greg Enas, Eli Lilly José Pinheiro, Novartis a Michael Krams, Pfizer Qing Liu, J & J Rick Sax, AstraZeneca a Tom Parke, Tessella a Leaders 2

3 Outline Background and motivation Adaptive Dose Finding PhRMA initiative: goals Simulation study to evaluate designs and methods for adaptive dose finding Simulation results Discussion 3

4 Background Pharmaceutical industry pipeline problem: decreasing number of drug approvals, despite advances in basic science FDA s Critical Path Initiative: Innovation vs. Stagnation White Paper Pharmaceutical industry (PhRMA) response: working groups (WGs) to address key drivers of poor performance Adaptive Design (AD) WG formed to foster wider usage and regulatory acceptance of adaptive designs (Gallo, Chuang-Stein, Dragalin, Gaydos, Krams and Pinheiro, 2006) Adaptive Dose Finding (ADF) WG formed to stimulate innovation in dose finding studies 4

5 Motivation Poor understanding of dose response (DR) for both efficacy and safety is pervasive in drug development Sub-optimal dose selection identified by both FDA and industry as one of root causes of late phase attrition and post-marketing problems with approved drugs Current dose finding designs and methods focus on selection of registrational study dose out of fixed, generally small number of dose levels, via pairwise hypothesis testing = inefficient Optimize patient treatment within a study, by minimizing patients exposed to ineffective treatments 5

6 What is an Adaptive Clinical Study Design? a Adaptive Clinical Study Design: A design in which data from the ongoing study is used to modify the conduct of the study Need to define objective of the adaption Adaption could involve any design element, not just dose Adaptive BY design: Adaption is a design feature, not a remedy for indadequate planning Through upfront planning is required Decision rules for adaption are prespecified a Adapted from: Adaptive Design in Exploratory Development, Brenda Gaydos, Joint Statistical Meeting,

7 Goals: ADF WG Investigate and develop designs and methods for efficiently learning about safety and efficacy dose-response = benefit/risk profile More accurate and faster decision making on dose selection and improved labeling Evaluate statistical operational characteristics of alternative designs and methods to make recommendations on their use in practice Increase awareness about this class of designs, promoting their use, when advantageous Document and publish findings of the working group 7

8 Definition and Scope: ADF Designs Flexible dose-ranging designs allowing dynamic allocation of patients and possibly variable number of dose levels based on accumulating information Intended to strike balance between need for additional DR information and increased costs and time-lines Emphasis on modeling/estimation (learning) as opposed to hypothesis testing (confirming) Investigate existing and new ADF methods via simulation Evaluate potential benefits over traditional dose-ranging designs over variety of scenarios to make recommendations on practical usefulness of ADF methods 8

9 Dose Finding Methods Fixed Doses : Conventional method based on pairwise comparisons and multiplicity adjustment (Dunnett); common approach used in dose finding studies Amit Roy and Frank Shen MCP-Mod combination of multiple comparison procedure (MCP) to identify presence of DR, and modeling to estimate target dose(s) and DR profile (Bretz, Pinheiro and Branson, 2005) José Pinheiro and Frank Bretz : novel method based on Multiple Trend Tests (Liu, 2006) Qing Liu : Bayesian Model Averaging (Hoeting, Madigan, Raftery and Volinsky, 1999) Beat Neuenschwander and Amy Racine : Nonparametric local regression fitting Björn Bornkamp and Frank Bretz 9

10 Dose Finding Methods Adaptive dose allocation : Adaptive dose allocation based on Bayesian normal dynamic linear model (Krams, Lees and Berry, 2005); allocation of patients to dose adaptively changed according to model-based optimization criteria (e.g., variance of target dose estimate) Tom Parke and Michael Krams D-opt: adaptive dose allocation based on D-optimality criterion used with sigmoid-e max model; model parameters re-estimated at interim analysis and corresponding D-optimal allocation determined for next interval Alex Dmitrienko and Chyi-Hung Hsu 10

11 Simulation study: Design and assumptions Objective: proof-of-concept + dose finding for neuropathic pain Primary endpoint: change from baseline in pain score (VAS) Key questions: is there evidence of a dose response Significance level (one-sided FWER): 0.05 Clinically relevant change in VAS: 1.3 which dose(s) should be tested in large confirmatory trials how well is the dose response (DR) curve estimated Study design scenarios: Sample sizes: 150 and 250 patients Number of doses: 5, 7, and a a (0, 2, 4, 6, 8), (0, 2, 4, 6, 8), and (0, 1,..., 8) 11

12 Dose response profiles Expected change from baseline in VAS at Week Umbrella Flat Emax Linear Dose Sigmoid Emax Logistic

13 Measuring performance Probability of identifying dose response: P r(dr) Probability of identifying clinical relevance and selecting a dose for confirmatory phase: P r(dose) Dose selection: Distribution of selected doses (rounded to nearest integer, if continuous estimate possible) 13

14 Measuring performance (contd.) Target dose interval doses that produce effect within ±10% of target effect Target dose Target interval Model actual rounded actual rounded Linear (5.67, 6.93) {6,7} Logistic (4.65, 5.35) {5} Umbrella (2.76, 3.81) {3,4} Emax (1.44, 2.95) {2,3} Sig-Emax (4.68, 5.58) {5} Probabilities of under-, over-, and correct interval estimation: P = P ( d targ < d min ), P + = P ( d targ > d min ), P = 1 (P + P + ) 14

15 Selected Simulation Results 15

16 Probability Identifying DR Flat DR N = 250 N = 250 N = 250 N = 150 N = 150 N = Significance level Type I Error is controlled at 5% by all methods 16

17 Probability of identifying DR (N = 150) Emax Sig Emax logistic umbrella linear Pr(DR) Pr(DR) generally as # doses (for fixed sample size) 17

18 Probability dose selection Flat DR N = 250 N = 150 N = 250 N = 150 N = 250 N = Pr(dose flat DR) False positive for clinically relevant effect is generally greater for 18

19 Probability dose selection (N = 150) Emax Sig Emax logistic umbrella linear Pr(dose) Most methods perfomed poorly, generally best 19

20 Probability dose in target interval Logistic DR N = 250 No dose N = 250 Under N = 250 Right N = 250 Over N = 150 No dose N = 150 Under N = 150 Right N = 150 Over Probability (%) None of the methods were entirely adequate 20

21 Probability dose in target interval Umbrella DR N = 250 No dose N = 250 Under N = 250 Right N = 250 Over N = 150 No dose N = 150 Under N = 150 Right N = 150 Over Probability (%) 21

22 Distribution of selected dose Logistic DR (N = 150) % Trials Dose selected Distribution of selected doses is wide for all methods 22

23 Distribution of selected dose Umbrella DR (N = 150) % Trials Dose selected Distribution of selected doses is wide for all methods 23

24 Sample predicted curves Logistic DR, (N = 150) Sample Median True -2-3 Predicted DR Dose Overall shape of DR was described fairly well by all methods 24

25 Sample predicted curves: Umbrella, and N = Sample Median True Predicted DR Dose Overall shape of DR was described fairly well by all methods 25

26 Conclusions Detecting DR is considerably easier than estimating it Sample sizes for DF studies, based on power to detect DR, are generally inadequate for dose selection and DR estimation None of methods had good performance in estimating dose in the correct target interval: maximum observed percentage of correct interval selection was 60% = larger N needed Adaptive dose finding methods lead to gains in power to detect DR, precision to select target dose, and to estimate DR greatest potential in the latter two had best overall performance, especially on DR estimation 26

27 Conclusions (contd.) DR estimation performance of model-based methods are superior to methods based on hypothesis testing Number of doses larger than 5 does not seem to produce significant gains, provided: overall N is fixed = trade-off between more detail about DR and less precision at each dose doses cover range of responses (minimum, intermediate, and maximum) In practice, need to balance gains associated with adaptive dose ranging designs approach against greater methodological and operational complexity 27

28 Preliminary Recommendations Adaptive, model-based dose-ranging designs should be considered, as they can lead to substantial gains in performance over traditional DF methods Sample size calculations for Phase II studies should take into account desired precision of estimated target dose and possibly also estimated DR (current methods are not adequate) When resulting sample size is not feasible, should consider selecting two or three doses for confirmatory phase to increase likelihood of including correct dose adaptive designs could be used in confirmatory phase for greater efficiency (e.g., dropping less efficient doses earlier) 28

29 Preliminary Recommendations (contd.) Proof-of-concept (PoC) and dose selection should be combined, when feasible, into one seamless trial Early stopping rules, for both efficacy and futility, should be used when feasible to allow greater efficiency in adaptive designs Bayesian methods are particularly well-suited for this purpose Trial simulations should be used to determine appropriate sample sizes, as well as for estimating operational characteristics of designs/methods under consideration Consider extent of overlap in exposure, when specifying doses to be studied 29

30 References Bretz, F., Pinheiro, J. and Branson, M. (2005). Combining multiple comparisons and modeling techniques in dose-response studies, Biometrics 61(3): Gallo, P., Chuang-Stein, C., Dragalin, V., Gaydos, B., Krams, M. and Pinheiro, J. (2006). Adaptive designs in clinical drug development: An executive summary of the phrma working group, Journal of Biopharmaceutical Statistics 16: Hoeting, J., Madigan, D., Raftery, A. and Volinsky, C. (1999). Bayesian model averaging: A tutorial, Statistical Science 14(4): Krams, M., Lees, K. R. and Berry, D. A. (2005). The past is the 30

31 future: Innovative designs in acute stroke therapy trials, Stroke 36(6): Liu, Q. (2006). Adaptive dose-response phase II trials for clinical development, Joint Statistical Meeting, Seattle, WA. 31