Experimental Design Chusak Okascharoen, MD, PhD September 19 th, 2016 Terminology Experimental study Clinical trial Randomized controlled trial 1
PHASES OF CLINICAL TRIALS Phase I: First-time-in-man studies in normal volunteers. Phase II: Efficacy trials in very-selected patients. Phase III: Large-scale testing in a wider range of patients. Phase IV: Additional trials conducted to determine better dosing schedules, new formulations, different populations, and marketing claims (Postmarketing Trials). Experimental design 2
Generalization (external validity) Internal validity Sample Sample Randomization Population of patients with conditions Tx Cn Measure Outcome Conclusion Sample Sample Selection Population of patients with conditions Tx Cn Ascertainment Outcome Conclusion 3
Signal / Noise Trochim W. Research Methods Knowledge Base. 2006 The Trouble with Observational Studies Example: HRT and cardiovascular disease Stampfer (NEJM, 1985) Nurse s Health Study Cohort study beginning in 1976 32,317 nurses age 30-55 years without a history coronary disease Follow-up = approx. 4 years Findings (ever vs. never users): RR=0.5 (95%CI 0.3-0.8) for cardiovascular events Wilson (NEJM, 1985) Framingham Study 1,234 postmenopausal women age 50-83 years Follow-up = approx. 8 years Findings (past or current use vs. never): RR=1.8 (p<0.01) for cardiovascular events 4
Women s Health Initiative JAMA 2002;288:321-33 sponsored by NIH Design Duration Subject randomized controlled trial 8.5 years 16,608 postmenopausal women 50-79 years with intact uterus Recruitment 40 centers in the USA 1993-1998 Women s Health Initiative JAMA 2002;288:321-33 HR 95 % CI Coronary artery disease 1.29 1.02-1.63 Strokes 1.41 1.07-1.85 Pulmonary embolism 2.13 1.39-3.25 5
Experimental design Allocate treatments to patients Aim to minimize the bias between the groups being compared Aim to minimize the unexplainable variability of observations Experimental design Classified based on purposes Explanatory Confirmatory Pragmatic 6
Experimental design Classified based on settings Efficacy Can the intervention do more good than harm under ideal circumstances? Effectiveness/Pragmatic Does the intervention do more good than harm under the usual circumstances? Experimental design Classified based on comparison group Within-subject comparisons Before-after design Cross-over design Between-subjects comparisons Parallel group design Stratified parallel group design 7
before-after comparison easy to understand & explain presume that the biological variability is minimized and the likelihood of showing a response to treatment is maximized Treatment Sample: Before Sample: After measure at baseline measure after Tx Sources of Bias in before-after comparison 1. Variations of the disease through time 2. Variations of staff, equipment and environment through time 3.Regression to the mean 3. Learning effect 4. Psychological effect (placebo effect) 8
asymmetryobservations.com spectrum.troy.edu parallel group design Outcomes Population of patients With the Condition Sample Experimental group Improved Not improved Comparison group Improved Not improved 9
Flow Diagram of Randomized Controlled Trials Screening for eligibility Exclude: ineligible Obtaining informed consent Exclude: unwilling Randomization Treatment Control Follow-up Loss to follow-up Contamination Discontinuation Monitoring: Protocol, blinding Compliance, AEs Analysis Analysis PICO Population: Intervention: Comparison: Outcome: 10
Populations The study population should be defined in advance, stating unambiguous inclusion (eligibility) criteria. The selection criteria will have impact on study design and ability to generalize. Subject recruitment must be taken into account. Populations Include: Subjects who have the potential to benefit from the intervention Subjects who have high likelihood to show event of interest Subjects who are likely to comply with the study protocol Exclude: Subjects who have the potential to harm from the intervention Subjects at high risk of developing conditions which preclude the ascertainment of the event of interest 11
Study intervention Is it ethical? Is it feasible? Is the intervention well enough developed to permit evaluation? (especially surgical procedures or psychological therapies). Is there preliminary evidence that the intervention is likely to be beneficial (from observational studies), including some appreciation of the size of the likely treatment effect? Choice of control Active control Superiority Noninferiority (equivalence) Inactive control No treatment control Placebo control 12
Placebo effect A British study of 835 women who took over-thecounter analgesics for headache. The women were randomly assigned to 4 groups. Group A - unmarked placebo; Group B - placebo marked with a widely advertised brand name aspirin Group C - aspirin in a plain package; and Group D - aspirin in a widely advertised brand. Branthwaite. Br Med J (Clin Res Ed) 1981 Placebo effect The women reported their pain relief on a six point scale (from 1, worse, to +4, completely better). 2.70 for D - branded aspirin 2.48 for C - plain aspirin 2.18 for B - masked place with brand 1.78 for A - unmarked placebo; Branthwaite. Br Med J (Clin Res Ed) 1981 13
Change in disease state from t 0 to t = treatment effect + placebo effect + regression to the mean + spontaneous improvement/worsening + effect of other co-interventions/effect modifiers t 0 t = baseline measurement = follow-up measurement Outcome (endpoint) Clinical outcome Clinical events (perceivable by patient) e.g. death, stroke, or myocardial infarction Intermediate outcome Surrogate outcome e.g. serum creatinine, degree of glucose control, or patency of coronary arteries Quality of life measurement, pain score Economic evaluation e.g. cost-effectiveness Composite outcome 14
t 0 Time t Intervention Disease True clinical outcome Example: Hyperlipidemia Lipid lowering drug Mortality t 0 Time t Intervention Disease Surrogate endpoint True clinical outcome Example: Hyperlipidemia Lipid lowering drug Cholesterol Mortality DeMets DL. 2004 15
Disease Surrogate endpoint True clinical outcome (a) The surrogate is not in the causal pathway of the disease process. Intervention Disease Surrogate endpoint True clinical outcome (b) Of several causal pathways of disease, the intervention affects only the pathway mediated through the surrogate. Intervention Disease Surrogate endpoint True clinical outcome (c) The surrogate is not in the pathway of the intervention's effect or is insensitive to its effect. Intervention Disease Surrogate endpoint True clinical outcome (d) The intervention has mechanisms for action independent of the disease process. DeMets DL. 2004 High-density Lipoprotein (HDL) Cholesterol Biomarker for cardiovascular risk Biomarker (therapeutic goal) for cardiovascular drugs: There was strong inverse association between HDL levels and the risk of coronary heart disease.* *Rader D. J. Clin. Invest. 2006 16
Barter PJ. NEJM. 2007 a randomized, double-blinded trial involving 15,067 patients at high cardiovascular risk torcetrapib + atorvastatin (treatment ) or atorvastatin alone (control) median follow-up period was 550 days Results: Barter PJ. NEJM. 2007 Treatment: increase in HDL 72% and decrease in LDL of 25% > controls Death rate from cardiovascular causes in treatment group was increased by 40% > control group 17
Change in disease state from t 0 to t = treatment effect + placebo effect + regression to the mean + spontaneous improvement/worsening + effect of other confounders/effect modifiers t 0 t = baseline measurement = follow-up measurement Advantages of composite endpoint? To increase statistical precision and efficiency of a trial higher number of events To avoid a choice between several important outcomes- when several outcomes are judged to be of equal value in terms of differentiating between successful and unsuccessful interventions 18
Disadvantages of composite endpoint? Discrepancy in size of difference among outcomes make misleading interpretation Difficult for meaningful interpretation Lack of relevance to patients May be misleading Example of discrepancy among composite endpoints Comparison of irbesartan with amlodipine in the diabetic nephropathy study Ferreira-González I. BMJ 2007 19
Randomization Randomization is the assignment of subjects to treatments with a predefined probability and by chance. This implies that each individual assignment cannot be predicted based on the previous assignments, and each individual has equal chance to be assigned to treatment or control. Randomization Aim to eliminate selection bias By theory: make two groups comparable (including unknown/unmeasurable factors) Transparency Reproducibility Incorporate with concealment 20
Randomization Types: Simple randomization Blocked randomization Stratified randomization Simple randomization N=30 N=27 21
Block randomization (block of 6) N=30 Stratified randomization Stage 1&2 Stage 3&4 N=30 22
Allocation concealment The prevention of foreknowledge of treatment assignment. Inadequate concealment may cause collapse of randomization sequence. Methods: Opaque envelope Central randomization Randomization procedure Need proper calculation of sample size Tools: randomization table, software Repeatability Don t use coin flipping, ping pong picking, dictionary page no., or alternate day/week 23
24
25
26
Blinding or masking the process of withholding information about treatment allocation from those who could potentially be influenced by this information. Unblinding can systematically bias trial findings through conscious or unconscious mechanisms. BLINDING OR MASKING Problems of non-blind trials 1. Unequal co-intervention 2. Unequal ascertainment of outcomes 3. Contamination between groups 27
BLINDING OR MASKING What should we do when blinding is impossible? - implement intensive follow-up program to prevent contamination - implement strict protocol to prevent co-intervention - use objective outcome measures to prevent biased outcome assessors Screening for eligibility Exclude: ineligible Obtaining informed consent Exclude: unwilling Randomization Treatment Control Follow-up Loss to follow-up Contamination Discontinuation Monitoring: Protocol, blinding Compliance, AEs Analysis Analysis 28
Guyatt G. User s Guides to Medical Literature. 2002 Intention to treat analysis Analyzing patient outcomes based on which group into which they were randomized regardless of whether they actually received the planned intervention. Advantage: Preserve the power of randomization Maintaining important unknown factors that might influence outcome 29
Coronary Drug Project comparing clofibrate with placebo in patients with previous myocardial infarction 5-year mortality clofibrate placebo Per-protocol analysis 15% 19.4% P <0.01 Intention-to-treat analysis 18.2% 19.4% P = 0.25 Stratified parallel group design Level 1 stratification : Pt. stage I & II Level 2 stratification : Pt. stage III & IV Total Level 1 treatment factor: Drug A Pt. stage I & II treated with Drug A Pt. stage III & IV treated with Drug A All Pt. treated with Drug A Level 2 treatment factor: Placebo Pt. stage I & II treated with Placebo Pt. stage III & IV treated with Placebo All Pt. treated with Placebo Total Pt. stage I & II treated with any treatment Pt. stage III & IV treated with any treatment 30
Simberkoff. Safety of herpes zoster vaccine in the shingles prevention study: a randomized trial. Ann Intern Med. 2010 Cross-over study Period 1 Wash-out Period 2 Sequence: Tx Placebo Subjects receive Treatment (1) No treatment Subjects receive Placebo (2) Sequence: Placebo Tx Subjects receive Placebo (3) No treatment Subjects receive Treatment (4) 31
Cross over design www.cienciasinseso.com Cross-over study + better than before-after design + smaller required sample size logistical problems: i.e. repeating (painful) procedures, more visits, longer study duration increase premature discontinuation/ loss to followup period effect carry-over effect 32
Cross-over study considerations The effect of the study treatments manifests early The study treatments have reversible effects The disease/condition of interest is relatively stable The study treatments have effects that do not extend to another period Usually use in phase II dose-response studies but impractical in phase III Parallel group Factorial design Statin Placebo Carnitine Statin + Carnitine Placebo + Carnitine Placebo Statin + Placebo Placebo + Placebo 33
Parallel group Factorial design Can assess: effect of each treatment effect of combined treatment effect of interaction between treatments Parallel group Factorial design sufficient evidence that the study treatments are independent testing multiple doses of each component of a treatment combination multiple comparisons 34
SUPPORT Study Group. NEJM. May 2010 Screened Randomized Target low Oxygen Saturation Target high Oxygen Saturation CPAP Surfactant CPAP Surfactant CPAP Surfactant Low O2 Sat. Low O2 Sat. + CPAP Low O2 Sat. + Surfactant High O2 Sat. High O2 Sat. + CPAP High O2 Sat. + Surfactant SUPPORT Study Group. NEJM. May 2010 35
Q &A 36