Randomization: Too Important to Gamble with

Size: px

Start display at page:

Download "Randomization: Too Important to Gamble with"

Wilfrid Horton
6 years ago
Views:

1 Randomization: Too Important to Gamble with A Presentation for the Delaware Chapter of the ASA Oct 18, 2012 Dennis Sweitzer, Ph.D., Principal Biostatistician Medidata Randomization Center of Excellence Optimizing Clinical Trials: Concept to Conclusion Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 1

2 Outline Randomized Controlled Trials Basics Balance Randomization methods Complete Randomization Strict Minimization Permuted Block Dynamic Allocation (Covariate-adaptive, not Response-Adaptive) Randomization Metrics Balance Predictability Loss of Power /Loss of Efficiency Secondary Imbalance: drop-outs Simulations comparing methods Confounding site & treatment effects (small sites) Overall performance Discontinuing patients Weighting stratification factors Meta-Balance Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 2

3 Why randomize anyway? Some basic principles Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 3

4 Why Gold Standard? Randomized Controlled Trial Trial: Prospective & Specific Controlled: Comparison with Control group (placebo or active) Controlled procedures Only Treatment Varies Randomization: Minimizes biases Allocation bias Selection bias Permits blinding Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 4

5 Eliminating Bias The Fact of bias? (conscious, unconscious, or instinctive) The Question of bias? Always 2 nd guessing Critics will think of unanticipated things Solution! Treat it as a game 1 statistician vs N clinicians Statistician generates a random sequence Clinicians sequential guess at each assignment Statistician wins if clinician guesses are no better than chance (NB: 75% wrong is just as bad as 75% right) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 5

6 Randomization Metrics What do we want in a randomization sequence or system? Randomness ó Unpredictable Reduce Allocation Bias (All studies) Reduce Selection Bias (All studies) Reduce placebo effects (Blinded studies) Balance ó Loss of Efficiency Maximizes statistical power Minimize Confounding Enhance Credibility (Face Validity) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 6

7 Balancing Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 7

8 Balanced Study Equal allocation between treatment arms Maximizes Statistical Power Control Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 8

9 Imbalanced Resulting in light weight results. Statistical power limited by smallest arm 36 subject simulation with Complete Randomization average loss 1 subject 10% lose 2 subject Can add 2 to compensate BUT only large imbalances have much effect on statistical power Severe Imbalances are rare in large studies Pr{worse than 60:40 split} for: n=25 <42% n=100 <4.4% n= % Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 9

10 (NB: nned Imbalance) 1:1 randomization maximizes power per patient But there are other considerations Utility: Need 100 patients on drug to monitor safety Study only requires 60 (30/arm) 2:1 randomization 100 & 50 cebo Motivation: Better enrollment if 75% chance of drug (3:1) Ethics: 85 cebo vs. 125 cebo Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 10

11 Imbalance Overall balance Only an issue for small studies Subgroup Balance Fixed size studies can have variable sized subgroups Increased risk of underpowered subgroups Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 11

12 Effective Loss of Sample Size Females Male s Con Effective Loss = Reduction of Power as Reduction in Sample Size Simulations of: 36 and 18 subjects, males as strata at 33% of population, randomized 1:1 (complete randomization) N=36 N=18 Overall Females Males Overall Females Males Effectively Lost Mean ± SD 1.0 ± ± ± ± ± ±1.3 2 pts 12% 14% 18% 23% 16% 17% 4 pts 6% 4% 5% 3% 4% 5% >=100% 0.0% 0.0% 0.4% 0.0% 0.5% 7.9% Q Median Q Imbalance (% of N) Mean ± SD 13% ±10% 16% ±12% 25% ±19% 18% ±15% 23% ±18% 35% ±28% >=50% 0.5% 1.6% 12.8% 3.1% 10.0% 27.9% Q1 6% 8% 9% 11% 9% 14% Median 11% 14% 20% 11% 20% 33% Q3 17% 22% 33% 22% 33% 50% Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 12

13 Bad Imbalance! Females Males Treatment Imbalances within factors spurious findings.. Leads to conversations like: Higher estrogen levels in patients on Treatment?? Hmm? ANCOVA showed no differences in estrogen levels due to treatment Credibility.. Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 13

14 ?"!!!!! Randomization Methods (See Animated Powerpoint Slides ) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 14

15 Randomization 4 methods Complete Randomization (classic approach) Strict Minimization Permuted Block (frequently used) Dynamic Allocation (gaining in popularity) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 15

16 Complete Randomization Every assignment Same probability for each assignment Ignore Treatment Imbalances No restrictions on treatment assignments Advantages: Simple Robust against selection & accidental bias Maximum Unpredictability Disadvantage High likelihood of imbalances (smaller samples). Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 16

17 Minimization Strict Minimization randomizes to the imbalanced arm Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 17

18 Minimization Strict Minimization rebalances the Arms BUT at a cost in predictability Random only when treatments are currently balanced Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 18

19 Permuted Block T P P? Blocks of Patients (1, 2, or 3 per treatment) Here: 2:2 Allocation T P P T T P P * (Unless Incomplete Blocks: More strata More incomplete) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 19

20 Dynamic Allocation Biases Randomization to the imbalanced arm Unpredictable Almost Balanced Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 20

21 Dynamic Allocation Complete Randomization Optimizes Unpredictability Ignores Balance Strict Minimization Optimizes Balance Ignores Predictability Dynamic Allocation 2 nd Best Probability Parameter Controls Balance vs. Predictability Tradeoff Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 21

22 Dynamic Allocation Flexibility 2 nd Best Probability= 0 Strict Minimization Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 22

23 Dynamic Allocation Flexibility 2 nd Best Probability= 0.5 Complete Randomization (for 2 treatment arms) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 23

24 Stratification Factors Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 24

25 Stratification Factors yo Males Females Over both sexes Factors Main Effects Strata 1 st Order Interactions yo >65 yo Over all Ages: Marginal Balance Marginal Balance Overall Balance Randomizing a 25 yo Male: To PLA Worsens Male balance To Worsens 18-35yo balance Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 25

26 Permuted Block Stratified Randomization Males Females Over both sexes Only balances within strata yo yo >65 yo T P P T T P T * T T P * P * * * P T * * P * * * Most strata will have incomplete blocks Imbalances accumulate at margins Over all Ages: Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 26

27 Minimization & Marginal Balance Males Females Over both sexes * Only balances on margins * Useful if too many strata, e.g.: yo # Strata > N blocksize yo >65 yo Marginal Balance * Appropriate for a main effects analysis (ie, no interactions) Marginal Balance Over all Ages: Overall Balance Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 27

28 Stratification & Dynamic Allocation Males Females Over both sexes DA: uses weighted combination of yo Overall balance Marginal balances yo Marginal Balance Strata balance Flexible >65 yo Marginal Balance Over all Ages: Overall Balance Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 28

29 Site as a Special Subgroup (Max 2 lines, 35 characters) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 29

30 Imbalance Overall balance Only an issue for small studies Subgroup Balance Fixed size studies can have variable sized subgroups Increased risk of underpowered subgroups Site as special case of subgroup Small sites Increased risk of "monotherapy at site Confounding site & treatment effects Effectively non-informative/ lost patients Actual vs Assumed distribution of site size Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 30

Enrollment per Center (Densities) Size Categories: {0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49, 50-59, 60-79, 80-99, 100-149, 150-199, 200 } Data Sample 13

31 Enrollment per Center (Densities) Size Categories: {0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49, 50-59, 60-79, 80-99, , , 200 } Data Sample 13 Studies 7.7 mo Average Enrollment period 3953 Obs.Pts 460 Listed Sites 372 Active.Sites Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 31

Enrollment per Site (#Sites) # Sites per Size Category {0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49, 50-59, 60-79, 80-99, 100-149, 150-199, 200 } Data Sample

32 Enrollment per Site (#Sites) # Sites per Size Category {0, 1, 2, 3, 4-7, 8-11, 12-15, 16-19, 20-29, 30-39, 40-49, 50-59, 60-79, 80-99, , , 200 } Data Sample 13 Studies 7.7 mo Average Enrollment period 3953 Obs.Pts 460 Listed Sites 372 Active.Sites Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 32

33 Site Enrollment Simulation Simulation based on Observations 4 mo Enrollment Period Enrollment ~ Poisson distribution µ = Obs. Pts/mo (active sites) or µ 0.5 / Enrollment period (non-active sites) Randomize using CR, PB(2:2), or DA(0.15). Confounded Pts Patients at centers with only one treatment treatment & center effects are confounded Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 33

34 Results mean ±SD (80% C.I.) Affected studies had many sites with low enrollment Studies with fewer sites (and more pts at each) were rarely affected Dynamic Allocation reduced confounding slightly more effectively than permuted block Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 34

35 Var( ) T T T 1 T zz zxxx ( ) Xz 2 Randomization Metrics Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 35

36 Randomization Metrics How do we measure badness of a randomization sequence or system? Predictability Goal: an observer can guess no better than chance Score based on Blackwell-Hodges guessing rule Easily calculated Imbalance Imbalance reduced statistical power Loss of Efficiency Measure as effective loss in number of subjects Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 36

37 Blackwell-Hodges Use Blackwell-Hodges guessing rule Directly corresponds to game interpretation Investigator always guesses the most probable treatment assignment, based on past assignments bias factor F F abs(# Correct Expected # Correct by chance alone) Measures potential for selection bias Modifications: Limits on knowledge of investigator (eg, can only know prior treatment allocation on own site) Score as percentage e.g., Score abs(% Correct 50%) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 37

38 Blackwell-Hodges Scoring (1) For treatment sequence TCCC Initial guess Expectation = ½ T Imbalance =+1 Guess C Correct TC Imbalance=0 Guess either Expectation=½ TCC Imbalance=-1 Guess T Wrong TCCC # Correct= ½ + 1+ ½ +0 =2 Score = #Correct - 2 = 2-2 = 0 Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 38

39 Blackwell-Hodges Scoring (2) For treatment sequence TCCC TCCC # Correct= ½ + 1+ ½ +0 =2 Complete Randomization Pr{ TCCC } = 1/16 Dynamic Allocation (p=0.15) Pr{ TCCC }= 0.5 *0.85 * 0.5 * 0.15 = Permuted Block (length 4) PR{ TCCC } = 0 Strict Minimization Pr{ TCCC }=0 Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 39

40 Blackwell-Hodges Scoring (3) Sequence TCCT # Correct= ½ ½ + 1 = 3 Score = 3 2 = 1 Complete Randomization Pr{TCCT}= 1/16 Strict Minimization Pr{TCCT} = ½*1*½*1 = ¼ Permuted Block Pr{TCCT} = 1/6 (NB: 6 permutations of TTCC) Dynamic Allocation (2 nd best prob.=0.15) Pr{TCCT} = 0.5 * 0.85* 0.5 * 0.85 = Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 40

41 Warning! Blackwell-Hodges Assesses potential selection bias Given known imbalance! But which imbalance(s)?? (Overall imbalance? Within strata? Within Factors?) Henceforth: only use imbalance within strata Proxy for center Assume observer only knows imbalance within his center Simple & unambiguous M Requires some caution in interpretation Local Predictability ONLY Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 41

42 Loss of Efficiency Loss of Efficiency (Atkinson, 1999) EY ( ) z X Treatment difference Var( ) T T T 1 T zz zxxx ( ) Xz 2 A constant term and k prognostic factors Inference in Covariate-Adaptive allocation Elsa Valdés Márquez & Nick Fieller EFSPI Adaptive Randomisation Meeting Brussels, 7 December 2006 Loss (for n patients and k factors; X an k design matrix) L n z T X( X Loss can be expressed as equivalent # Patients In a 100 patient study: Loss of Efficiency= 5 X) A perfectly designed study would require only 95 T 1 X T z Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 42

43 RCT vs DOE Loss L n z T X( X T X) 1 X T z (for n patients and k factors; X an k design matrix) X design matrix: n rows, 1 per pt K columns, 1 per covariate z Treatment assignments Designed Experiment (DOE): Select z and covariate values to minimize L n RCT Select only z (No control of covariates) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 43

Loss of Efficiency (Máquez & Fieller) Performance Comparison Loss of efficiency of various methods CR: Complete Randomization TV: Minimization (Taves,1974) PS:Minimization (Pocock &Simon,1975) Ds:

44 Loss of Efficiency (Máquez & Fieller) Performance Comparison Loss of efficiency of various methods CR: Complete Randomization TV: Minimization (Taves,1974) PS:Minimization (Pocock &Simon,1975) Ds: Ds-Optimum Design (Begg&Iglewicz, 1980) DA: DA-Optimum Biased Coin Design (Atkinson,1982) s L n z T X( X T Dynamic Allocation Sequentially assign Z to minimize X) 1 X T z THE BEST (without random elements) Simulated data:- 100 subjects, 5 prognostic factors 6 Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 44

45 Loss of Efficiency (Máquez & Fieller) Different factors and samples Covariate adaptive methods always more efficient than complete randomisation method with random element (PS) only efficient for larger sample sizes 1,000 group of patients 7 Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 45

46 discon&nuing)pa& %# %# %# αδϕυστ( δισχοντινυ( P P P D D D D D Randomization Performance Simulations D %# νωο Δισχ.( C C C %# 0%# 5%# 10%# 15%# Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 46

47 Simulation Set up 3 methods: Complete Randomization Permuted Block Dynamic Allocation Each simulated patient randomized w/ each method 4 Measures: Loss of Efficiency B-H Score ( Within Strata ) Overall Imbalance Relative Loss of Efficiency vs CR % Loss of Efficiency (of #pts) 6 Strata (Factors: Sex, Age) 33% or 50% Males 1:1:1, 1:1:2, 1:2:3 (Young : Middle : Old) 48 subjects Total With random 25% Dropout Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 47

48 Note on Figures Simula&on)results)as)80%)Confidence)Intervals) 25%# Plot B-H score vs 20%# DA(0),#Margin#Balance# Loss of Efficiency PB(1:1)# Poten&al)Selec&on)Bias) 15%# 10%# DA(0),#Margin#Balance# PB(1:1)# Median + 80% C.I. 10% lower & 10% higher 5%# 0%# 0# 1# 2# 3# 4# 5# 6# Loss)of)Efficiency) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 48

49 Simulation Results(1) Predictability %Imbalance Efficiency Loss DA(0.00) 22% 0.6% 0.87 DA(0.15) 16% 1.6% 1.45 DA(0.25) 13% 2.8% 1.99 DA(0.33) 8% 4.3% 2.64 DA(0.50) 4% 11.3% 4.99 CR 4% 11.4% 5.03 PB(8:8) 7% 7.1% 3.00 PB(4:4) 13% 4.9% 1.52 PB(3:3) 16% 4.2% 1.13 PB(2:2) 19% 3.5% 0.79 PB(1:1) 23% 2.6% 0.47 Averages of Metrics But for managing risk, need Worst Case 80% Confidence Intervals Both DA & PB are stratified. Simulation: 48 subjects, 2 stratification factors, 6 strata, uneven sizes (DA) Dynamic Allocation (PB) Permuted Block (CR) Completely Random DA( 2 nd Best Probability ), PB( Allocation Ratio ) Simulated subjects were randomized by all 3 methods Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 49

50 Randomizations Plotted by Metrics 25%# 20%# PB(1:1)# DA(0.00),#Wt(3:3:3)# PB(1:1), DA(0) 25%# 20%# PB(2:2)# DA(0.15),#Wt(3:3:3)# PB(2:2), DA(0.15) Poten&al)Selec&on)Bias) 15%# 10%# (Essentially Strict Minimization) Poten&al)Selec&on)Bias) 15%# 10%# 5%# 25%# 5%# 25%# PB(4:4)# DA(0.33),#Wt(3:3:3)# 0%# 20%# CR# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# # Loss)of)Efficiency) Poten&al)Selec&on)Bias) 15%# 10%# PB(4:4) DA(0.33) 0%# 20%# CR# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# Poten&al)Selec&on)Bias) 15%# 10%# PB(8:8)# DA(0.50),#Wt(3:3:3)# DA(0.5) CR PB CR Loss)of)Efficiency) PB(8:8) DA(0.5) 5%# CR 0%# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# # Loss)of)Efficiency) 5%# CR 0%# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# # Loss)of)Efficiency) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 50

51 Correlation of Metrics 0.40% Correla'ons*of*Predictability*and*Loss*of*Efficiency* 0.20% 0.00%!0.20% CR% CR% CR% CR% CR% DA(0.00)% DA(0.15)% DA(0.25)% DA(0.33)% DA(0.50)% PB(1:1)% PB(2:2)% PB(3:3)% PB(4:4)% PB(8:8)%!0.40%!0.60%!0.80%!1.00% Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 51

52 Backup scatterplots 25%# 25%# 20%# PB(4:4)# DA(0.33),#Wt(3:3:3)# CR# PB(8:8)# Poten&al)Selec&on)Bias) 20%# 15%# 10%# 5%# PB(8:8) DA(0.50),#Wt(3:3:3)# CR# DA(0.5), CR Poten&al)Selec&on)Bias) 25%# 20%# 15%# 10%# Poten&al)Selec&on)Bias) 15%# PB(3:3)# 10%# DA(0.25),#Wt(3:3:3)# CR# 5%# 0%# PB(3:3), 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# Loss)of)Efficiency) DA(0.25) 8.000# 9.000# # 0%# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# # Loss)of)Efficiency) 5%# 0%# 0.000# 1.000# 2.000# 3.000# 4.000# 5.000# 6.000# 7.000# 8.000# 9.000# # Loss)of)Efficiency) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 52

53 Simulated Comparison 25%# Predictability,vs,Loss,of,Efficiency, 20%# Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}# Dynamic#{0%,#15%,#25%,33%,#50%}# Predictability,Score, 15%# 10%# DA(0.25) PB(3:3) Complete#RandomizaGon# 1,000 simulations per case * 48 subjects each * 6 Strata, 2 factor, Variety of proportions 5%# 0%# 0.0# 1.0# 2.0# 3.0# 4.0# 5.0# 6.0# 7.0# 8.0# 9.0# Loss,of,Efficiency, Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 53

54 Simulated Comparison 25%# Predictability,vs,%,Loss,of,Efficiency, 20%# Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}# Dynamic#{0%,#15%,#25%,33%,#50%}# Predictability,Score, 15%# 10%# DA(0.25) PB(3:3) Complete#RandomizaDon# %Loss of Efficiency = Lossof Efficiency Sample Size 5%# 0%# 0%# 2%# 4%# 6%# 8%# 10%# 12%# 14%# 16%# 18%# 20%# %Loss,of,Efficiency, Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 54

25) PB(3:3) Dynamic#{0%,#15%,#25%,33%,#50%}# 5%# 0%# 0.00# 0.20# 0.40# 0.60# 0.80# 1.00# 1.20# 1.

55 Relative Loss of Efficiency 25%# Predictability,vs,Rela0ve,Loss,of,Efficiency,, 20%# Permuted#Block#{1:1,#2:2,#3:3,#4:4,#8:8}# Predictability,Score, 15%# 10%# DA(0.25) PB(3:3) Dynamic#{0%,#15%,#25%,33%,#50%}# 5%# 0%# 0.00# 0.20# 0.40# 0.60# 0.80# 1.00# 1.20# 1.40# 1.60# 1.80# 2.00# Rela0ve,Loss,of,Efficiency, Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 55

56 Local Predictability ONLY Special Topics Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 56

Dynamic Allocation Weighting 25%# Dynamic)Alloca&on)Weights) Balancing)on){Strata,)Margin,)Overall}) 25%# Dynamic)Alloca&on)Weights) versus)permuted)block,)complete)randomiza&on

DA(0),#Margin#Balance# DA(0),#Overall#Balance# CR# 5%# 5%# 0%# 0# 1# 2# 3# 4# 5# 6# 7# 8# 9# 10 Loss)of)Efficiency) DA(0) balanced only within strata DA(0) equal weighting 0%# 0.00# 1.00# 2.00# 3.

57 Dynamic Allocation Weighting 25%# Dynamic)Alloca&on)Weights) Balancing)on){Strata,)Margin,)Overall}) 25%# Dynamic)Alloca&on)Weights) versus)permuted)block,)complete)randomiza&on Poten&al)Selec&on)Bias) 20%# 15%# 10%# PB(1:1)# DA(0),#Strata#Balance# DA(0),#Margin#Balance# DA(0),#Overall#Balance# CR# Poten&al)Selec&on)Bias) 20%# 15%# 10%# PB(1:1)# DA(0),#Strata#Balance# DA(0),#Margin#Balance# DA(0),#Overall#Balance# CR# 5%# 5%# 0%# 0# 1# 2# 3# 4# 5# 6# 7# 8# 9# 10 Loss)of)Efficiency) DA(0) balanced only within strata DA(0) equal weighting 0%# 0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# 10.00# Loss)of)Efficiency) ó Approximates PB(1:1) ó Approximates PB(1:1) DA(0) balanced on margins ó Intermediate properties DA(0) balanced only overall ó Approximates CR (large N) NB: Predictability is limited to imbalance within a stratum! Local Predictability ONLY Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 57

Dynamic Allocation Weighting Poten&al)Selec&on)Bias) 25%# 20%# 15%# 10%# 5%# Dynamic)Alloca&on) Various)Weigh&ngs) DA(0),#Strata#Balance# DA(0),#Equal#WeighCng# DA(0),#Margin&Strata#

58 Dynamic Allocation Weighting Poten&al)Selec&on)Bias) 25%# 20%# 15%# 10%# 5%# Dynamic)Alloca&on) Various)Weigh&ngs) DA(0),#Strata#Balance# DA(0),#Equal#WeighCng# DA(0),#Margin&Strata# DA(0),#Unequal#WeighCng# DA(0),#Margin#Balance# DA(0),#Overall#Balance# Weighting: (Strata, Margins, Overall) DA(0) Equal Weighting (1,1,1) ó Strata Balance Dominates ó Approximates PB(1:1) DA(0) Margin & Strata (1:9:0) ó Separates from PB(1:1) DA(0) Unequal Weighting (1,6,20) DA(0) Margin Balance (0,1,0) DA(0) Overall Balance (0,0,1) ó Approx. CR 0%# 0.00# 1.00# 2.00# 3.00# 4.00# 5.00# 6.00# 7.00# 8.00# 9.00# Loss)of)Efficiency) Local Predictability ONLY Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 58

59 DA Algorithm Distance function Weighted Sum of Imbalances IMB.c;A i / D.w STUDY rimb.study.c/; A i // C.w STRATUM rimb.stratum.c/; A i // Relative Imbalance: rimb.x; A i / D Factor as Union of Strata X = C.w SITE rimb.site.c/; A i // C X.w FACTOR.v/ rimb.factor.v; c/; A i // (2) 16v6K Strata Imbalances dominate Distance function X 16j 6N X k X X k X k X X \ Aj C ı.i; j / r ˇ j.kxk C 1/ ˇˇˇˇˇ P 1 X +1 1 X k +1 Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 59

Weighting Males Females Over both sexes 18-35 yo 35-65 yo >65 yo Over all Ages: Stratified Randomization weights on

both sexes Minimization weights on margins, not strata.

Strata is balanced, the next assignment attempts to balance the margins.

60 Weighting Males Females Over both sexes yo yo >65 yo Over all Ages: Stratified Randomization weights on strata, not margins or overall Imbalances within strata tend to dominate in DA yo yo Males Females Over both sexes Minimization weights on margins, not strata. >65 yo DA can weight exclusively on margins Over all Ages: Males Females Over both sexes yo yo If a Strata is balanced, the next assignment attempts to balance the margins. >65 yo Over all Ages: Since small groups are more likely to have imbalances which reduce efficiency, balancing strata 1 st is appropriate Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 60

Hierarchical Balancing While Imbalances within strata tends to dominate in DA, if a Strata is balanced, the next assignment attempts to balance the margins Since small group

(2) If balanced within the strata, balance by age group (since age groups tend to be smaller than sex groups) (3) If balanced within age group, balance within sex group (4)

61 Hierarchical Balancing While Imbalances within strata tends to dominate in DA, if a Strata is balanced, the next assignment attempts to balance the margins Since small group imbalances tend to dominate, balancing tends to be sequential Males yo yo Females >65 yo Over all Ages: Over both sexes This example: (1) Balance within strata (2) If balanced within the strata, balance by age group (since age groups tend to be smaller than sex groups) (3) If balanced within age group, balance within sex group (4) If balanced within sex group, balance overall However: cumulative imbalances may change this order Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 61

62 ?"!!!!! Replacement Randomization Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 62

Dynamically Adapting to Dropouts 25%# Effect)of)Drop9outs)on)Permuted)Block)and)Dynamic) Alloca&on) Patients discontinue Imbalances Reduced efficiency Poten&al)Selec&on)Bias) 20%# 15%# 10%# 5%# 25%

63 Dynamically Adapting to Dropouts 25%# Effect)of)Drop9outs)on)Permuted)Block)and)Dynamic) Alloca&on) Patients discontinue Imbalances Reduced efficiency Poten&al)Selec&on)Bias) 20%# 15%# 10%# 5%# 25% DC PB(2:2)# PB(2:2)# PB(2:2),#25%DC# DA(0.15),#Eq.Wts# DA(0.15),#Eq.Wts,#25%DC# DA(0.15),#Margins# DA(0.15),#Margins,#25%#DC# CR# CR(25%DC)# Tight randomizations (PB with small blocks, DA with small 2 nd best Prob.) Lose more efficiency Loose randomizations (CR, PB with large blocks, DA with large 2 nd best Prob.) Lose less efficiency Little or no change No DC CR# 0%# 0%# 5%# 10%# 15%# 20%# 25%# %)Loss)of)Efficiency))) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 63

Dynamically Adapting to Dropouts Poten&al)Selec&on)Bias) 24%$ 22%$ 20%$ 18%$ 16%$ Effect)of)Drop9outs)&)Rerandomiza&on)) on)permuted)block)and)dynamic)alloca&on) PB(2:2)$ PB(2:2)$ PB(2:2),$25%DC$

64 Dynamically Adapting to Dropouts Poten&al)Selec&on)Bias) 24%$ 22%$ 20%$ 18%$ 16%$ Effect)of)Drop9outs)&)Rerandomiza&on)) on)permuted)block)and)dynamic)alloca&on) PB(2:2)$ PB(2:2)$ PB(2:2),$25%DC$ DA(0.15),$Eq.Wts$ DA(0.15),$Eq.Wts,$25%DC$ DA(0.15),EqWts,Adj.25%DC$ 25% DC Dynamic Allocation: Can allocate new patients to restore balance 14%$ No DC DA Adj. 12%$ 0%$ 1%$ 2%$ 3%$ 4%$ 5%$ 6%$ 7%$ 8%$ %)Loss)of)Efficiency))) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 64

65 Dynamically Adapting to Dropouts Poten&al)Selec&on)Bias) 20%# 15%# 10%# 5%# No DC Dynamic)Alloca&on:)Readjus&ng)balance)for) discon&nuing)pa&ents) DA Adj. 25% DC PB(2:2)# PB(2:2)# PB(2:2),#25%DC# DA(0.15),#Eq.Wts# DA(0.15),#Eq.Wts,#25%DC# DA(0.15),EqWts,Adj.25%DC# DA(0.15),#Margins# DA(0.15),#Margins,#25%#DC# DA(0.15),#Margins,Adj.25%Dc# CR# CR(25%DC)# CR# Tight randomizations (PB with small blocks, DA with small 2 nd best Prob.) Lose more efficiency Benefit most Loose randomizations (CR, PB with large blocks, DA with large 2 nd best Prob.) Lose less efficiency Little or no benefit 0%# 0%# 5%# 10%# 15%# 20%# 25%# %)Loss)of)Efficiency))) Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 65

66 Applications High drop-out PB, DA CR Drop-out before becoming evaluable Constrained resources (small sample size, limited drug supply,.) Crossover studies: Requires completers Evaluable ó Complete Sequence of Treatments Provisional Randomization / Randomize to ship Screening visit triggers: Randomize at screening If randomized treatment not on-site, ship blinded supplies Randomization visit: If patient eligible dispense assigned treatment If not eligible store for next eligible patient Minimizes on-site drug supply Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 66

67 Randomization Optimization Factors Equipose (less random is acceptable) Small Study Efficiency important Lower 2 nd Best Probability Large Study Are there small subgroups? All subgroups large CR is acceptable Small subgroups Need more efficiency Smaller 2 nd best Prob Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 67

68 Balancing Considerations Large Studies Studies with large subgroups Late phase studies Strong Treatment preferences Weak Blinding Subjective Endpoints Unpredictable Smaller Studies Studies with small subgroups Early phase studies Interim Analyses Equipoise Strong Blinding Objective Endpoints Many Strata / Many centers Limited blinded supplies Balanced Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 68

69 Bibliography Elsa Valdés Márquez & Nick Fieller. Inference in Covariate-Adaptive allocation. EFSPI Adaptive Randomisation Meeting, Brussels, 7 December 2006 Optimizing Clinical Trials: Concept to Conclusion 2012 Medidata Solutions, Inc. 69

Sequence balance minimisation: minimising with unequal treatment allocations

Madurasinghe Trials (2017) 18:207 DOI 10.1186/s13063-017-1942-3 METHODOLOGY Open Access Sequence balance minimisation: minimising with unequal treatment allocations Vichithranie W. Madurasinghe Abstract