1 研究設計理論 ( 一 ): RANDOMIZED TRIALS 醫學研究初階課程 General Medicine, CGMH 劉素旬醫師 03/12/2013
2 Learning Objectives Definition of a trial Features of trial design Elements of a trial Purposes of randomization and masking Reason for analysis by intention to treat Ethical issues Reporting guideline
3 What is a Trial? The action or process of putting something to a test or proof (Webster) An experiment designed and carried out to provide information on the merits of one treatment or procedure relative to another (Prof. Meinert C. )
4 Definitions for Clinical Trial A controlled experiment having a clinical event as an outcome measure, done in a clinical setting, and involving persons having a specific disease or health condition (Prof. Meinert) A controlled experiment to evaluate the efficacy and safety of a treatment. (Prof. Meinert)
5 Why Are Clinical Trials Important? (I) Gateway for new medical advances Gold standard for evaluation Drugs, devices, management Guideline for medical care Insurance coverage EBM
6 Why Are Clinical Trials Important? (II) Economic impact Costs of medical treatments Important segment of the economy Ethical issues Human subject experiments Best use of limited resources Access to information
7 Epidemiologic Studies Measuring Descriptive Diseases Health behaviors Environmental exposures Health-related costs Comparative Examining associations between exposures and health outcomes Group comparison Relative measures of disease
8 Alternatives to A Randomized Trial Without a comparison group Case report Case series With a comparison group Historical controls Simultaneous, nonrandomized controls Pre- post design
9 Essence of a Trial Design Experimentation (vs. observation) Exposure is under the control of the investigator Randomized Non-randomized Exposure is merely observed by the investigator in observational studies Method for allocating exposure/intervention Prospective standardized protocol Innovation or efficacy of the intervention is unknown
Source: care.diabetesjournals.org 10
Source: clincancerres.aacrjournals.org 11
12 Before The Need for A Trial Source: http://www.discoveriesmagazine.org/clinical-trials-101/
Source: http://www.discoveriesmagazine.org/clinical-trials-101/ 13
14 Phases of Clinical Trials for Evaluation of New Drugs Phase I Clinical pharmacology Optimum dose, toxicity, PK, PD (usually 10-30 healthy volunteers or patients, not controlled) Phase II Clinical investigation Safety and efficacy, sometimes controlled (usually 30-100 patients) Phase III Clinical trial Comparative efficacy, expanded and controlled usually randomized Phase IV Post-marketing trials Adverse effects or off-label use, controlled observational studies or uncontrolled post-marketing surveillance
15 Efficacy vs. effectiveness Efficacy Trial To assess the efficacy of a treatment under ideal conditions Trial setting not designed or intended to approximate everyday conditions Effectiveness Trial To assess usefulness of a test treatment Trial conditions approximate everyday practice conditions
16 Types of Controls No therapy Trial of removal of allergens from home environment to control asthma Placebo Use of estrogen vs. placebo for heart disease prevention in women Sham procedure Acupuncture vs. incorrect placement of needles Standard treatment Sequential anti-h. pylori treatment vs. standard treatment
17 Types of Trial Designs Parallel comparison Cross-over Factorial Group allocation (cluster randomized trial)
18 Parallel Randomized Trial Source: www.csp.or.jp
19 Cross-over Trial Each subject serves as his/her own control Feasible only if there is no carry-over treatment effect after the washout period Cannot exam for treatment effect on mortality If order of treatments may influence results, treatment administration can be randomized
20 Factorial Design Two rationales Potential economical way to test two treatments simultaneously, if their modes of action are independent Method to test for treatment synergy Source: Abernethy et al. Contemp Clin Trial 2006; 27(1):83-100
21 Factorial Design: Two Factors Source: Arnold et al. Simulation methods to estimate design power: an overview for applied research. 2011 BMC Med Res Methodol
22 Cluster Randomization Trial Randomization unit is a group of individuals When individual randomization and intervention is not feasible or acceptable Tracking Contamination Administrative resources Outcomes and analysis must be appropriate to group allocation design Group measures of outcome Individual measures, accounting for correlation
23 Group Allocation (Cluster Randomization) Source: Arnold et al. Simulation methods to estimate design power: an overview for applied research. 2011 BMC Med Res Methodol
24 Treatment Comparisons Depending on the alternative hypothesis Superiority trials: one treatment is better than the other Two sided: H a : A > B or A < B One sided: H a : A > B; H a : A < B Equivalence trials: two treatments are clinically equivalent (within some acceptable small margin) Two sided: H a : A B or A B Non-inferiority trials: one treatment is as effective as the other One sides: H a : A B
25 Interventions That Can Be Evaluated Treatments, including drugs and cognitive behavioral therapy Preventive measures Screening programs Vaccination Personal protective equipment Health service delivery or technology Individual vs. group intervention
26 Target Population People with a disease People at high risk for a disease/condition Primary prevention Secondary prevention General population Primary prevention Health care delivery program Community health program
27 Types of outcome Intervention Health outcomes: mortality, other clinical event rate, disease-free survival, biomarker, quality of life, cost Prevention Incidence of disease, mortality Health care delivery Health outcomes, process outcomes Community health: Indices of population health
28 Follow-up Length Short-term: often use surrogate outcome, biomarker Long-term: clinical event End of observation After a defined period of time A common close-out date (administrative censoring)
29 Scale of A Trial Single-center Multi-center International
30 Elements in the Design of Randomized Controlled Trials (RCT) 1. Selection of subjects 2. Allocation of subjects to treatment groups 3. Data collection 4. Masking 5. Data analysis
31 1.Selection of Subjects Representativeness Generalizability (external validity)
32 2.Allocation of subjects Predictable assignment (not recommended) Day of the week Birth date Order enrolled Randomization Not predictable Having no specific pattern; lack causal relationship Process in which there is associated with every legitimate outcome a probability
33 Goal of Randomization To compare otherwise similar groups by removing potential biases in the choice of treatment(s) Selection bias Factors associated with prognosis Treatment Outcome Confounding by indication Main reason why RCT is considered gold standard of study design
34 Goal of Randomization Secondary: to increase the likelihood of balanced groups in known and unknown risk factors Tertiary: to provide a probability basis for hypothesis testing, i.e. to assure statistical tests will have valid significance levels
35 Results of a Trial of BCG Vaccine Non-randomized controls No. Cases No. TB Death % Vaccinated 445 3 0.67 Controls 545 18 3.30 Randomized controls No. Cases No. TB Death % Vaccinated 556 8 1.44 Controls 528 8 1.52
36 Observational Study Intervention Arm N=1000 Control Arm N=1000 180 Total Deaths 300 Mortality: 180/1000 =18% 300/1000=30%
37 Observational Study Intervention Arm N=1000 IRHB (-) IRHB(+) N=800 N=200 Control Arm N=1000 IRHB (-) IRHB(+) N=500 N=500 80 100 50 250 180 Total Deaths 300 Mortality: 180/1000 =18% 300/1000=30% Proportions of patients with arrhythmia in the two comparison groups may differ
38 A Randomized Trial Intervention Arm N=1000 IRHB (-) IRHB(+) N=654 N=346 Control Arm N=1000 IRHB (-) IRHB(+) N=649 N=351 65 173 65 176 238 Total Deaths 241 Mortality: 238/1000 =23.8% 241/1000=24.1% P-value= 0.875 Proportions of patients with arrhythmia in the two comparison groups may differ
39 Take a break Illustration: Cathy Wilcox http://www.theage.com.au/national/education/taking-abreak-is-secret-to-success-20120815-24951.html
40 2. Allocation of Subjects Types of Randomization Schemes Simple randomization Restricted randomization Blocking Stratification Adaptive randomization
41 Simple Randomization Table of random numbers
42 Simple Randomization Scheme An Example If 2 groups are compared When digit is: 0-4 Assignment is Treatment A When digit is: 5-9 Assignment is Treatment B If digit is If 3 groups are compared Next assignment is 1-3 Treatment A 4-6 Treatment B 7-9 Treatment C 0 is ignored
43 Simple Randomization Computer randomization schemes Create lists of random numbers Generate treatment assignment Provide reproducible audit trail of treatment assignment Control and keep records of access to treatment assignment Source: www.segobit.com
44 Simple Randomization Advantages Complete independence among each assignment Completely unpredictable Numbers of patients assigned to each group should be equal in the long run Disadvantages Imbalances when sample size is not large enough Numbers of patients in each group Confounding factor Lower power Diminish credibility of results
45 Restricted Randomization Randomization with constraints to produce expected assignment ratio according to time or specified covariate(s) Permuted block or blocking Stratification
46 Blocking Choose the smallest possible block size The sum of integers defined in the treatment allocation ratio Examples: 2 for a 1:1 ratio; 3 for a 2:1 ratio Larger block sizes are multiples of the smallest one: 4, 6, 8, List all possible block sequences (permutations) Example: for a block size of 4 with a 1:1 treatment allocation ratio BAAB ABAB BBAA ABBA.
47 Blocking (cont.) Randomly choose one block with replacement until the desired number of assignments Randomly order blocks to create the assignment list Source: bmsi.ru
48 Blocking Advantages Overall balance, esp. in smaller trials Achieve balance periodically Protects against timerelated changes Good statistical power Disadvantages Can facilitate prediction of future assignment More problematic for unmasked trials or poorly masked trials Remedy#1: block size(s) remains a secret Remedy#2: use more than one block size
49 Stratification 1. Stratify by Sex 1000 Patients 600 Males 400 Females 2. Stratify by Age 360 Adults (<65 yrs) 240 Elderly ( 65 yrs) 300 Adults (<65 yrs) 100 Elderly ( 65 yrs) 3. Randomize each subgroup Treatment A 180+120+150+50 =500 Treatment B 180+120+150+50 =500
50 Stratification To ensure balance in treatment assignments within subgroups defined before randomization Subgroup should be related to outcome Essentially create separate treatment assignment schemes for each stratum
51 Practicalities of Stratification Limit to a few variables Typical ones Clinic in a multicenter trial Surgeon Stage of disease Demographic characteristics (age, gender) Too many strata may lead to overall imbalances Must use blocking to ensure that stratification is effective Strata can be varying sizes
52 Stratification and Blocking Source: Morrissey et al. 2010 Contemp Clin Trials
53 Adaptive Randomization The probability of assignment to treatment groups does not remain constant but depends on the current balance or composition of the groups (Piantadosi, 1997)
54 Covariate-Adaptive Randomization Minimization After randomizing the first patient, the treatment assignment that yields the smallest imbalance is chosen Balance marginal treatment totals across prognostic factors Can handle many more factors than stratification Allocation sequence cannot be determined in advance
55 Response-Adaptive Randomization Play the winner Change allocation ratio to favor the better treatment based on the primary outcome Need to evaluate outcome (or surrogate outcome) relatively quickly Source: Gupta et al. 2011 J Clin Epidemiol
56 Misconceptions about Randomization Haphazard procedure is the same Guarantees comparable groups Guarantees that actual treatment allocations will match allocation ratio A study without randomization is invalid
57 Randomization Does Not Assure external validity Eligibility criteria Actual patient population Real-world conditions Eliminate bias associated with data collection, treatment adherence or follow-up
58 Generalizability of Randomized Clinical Trials A.k.a external validity May not provide good estimates of the absolute effect of treatment for the general population However, relative treatment effect may be generalizable to the general population
59 3.Data Collection Treatment Compliance Outcome Prognostic profile at entry Baseline characteristics
60 4.Masking Treatment assignment Subjects Primary physicians of study subjects Data collectors (evaluators) Data analyst Outcome assessment More important if treatment assignment is not masked
61 Level of Masking Single - participant Double - participant AND investigator Triple - participant AND investigator AND monitoring committee Source: scientific-misconduct.blogspot.com
62 Masking Protects Against Bias in Reports and behavior of participants Investigator s bias Data collection Outcome assessment Concomitant treatments
63 5.Data Analysis Problems of non-compliance Unplanned cross-overs: drop-outs, drop-ins Example: National Emphysema Treatment Trial (NETT), 1998-2002 Source: JHSPH OpenCourseWare
64 5.Data Analysis Intention-totreat analysis: net effect of noncompliance will reduce the observed difference Source: jama.jamanetwork.com
65 5.Data Analysis Secondary: Per-protocol (As-treated) analysis Observational in nature
66 Compliance Analyses Coronary Drug Project 5-year Mortality No. Patients Mortality Clofibrate 1065 18.2% Placebo 2695 19.4% Source: Canner et al. 1980
67 Compliance Analyses Coronary Drug Project 5-year Mortality No. Patients Mortality Clofibrate Arm 1065 18.2% Poor complier (<80%) Good complier ( 80%) 357 24.6% 708 15.0% Placebo Arm 2695 19.4% Source: Canner et al. 1980
68 Compliance Analyses Coronary Drug Project 5-year Mortality Clofibrate group Placebo group Compliance No. Patients Mortality No. Patients Mortality Poor complier (<80%) Good complier ( 80%) 357 24.6% 882 28.2% 708 15.0% 1813 15.1% Total 1065 18.2% 2695 19.4% Source: Canner et al. 1980
69 Dealing with Non-compliance Use of pilot studies (run-in periods), however, effectiveness depends on Precision in predicting compliance Costs of recruitment Monitoring compliance Interview, pill-counting Medication bottle devices Blood/urine tests Directly observed treatment (DOT)
70 5.Data Analysis Subgroup analysis Lack of power Over-interpretation Screening questions to help decide if a subgroup analysis is justified Was the analysis a priori defined at the design phase? Is the analysis subject to bias? Is the analysis biologically plausible? Is the result of the trial significant overall?
71 5.Data Analysis: Expressing Results of Randomized Trials Efficacy Relative risk (RR)- the ratio of rates of outcomes comparing treatment and control groups RR = Rate in treatment gruop Rate in control group
72 5.Data Analysis: Expressing Results of Randomized Trials Efficacy Relative risk reduction (RRR)- the proportional reduction in rates of outcomes comparing treatment and control groups RRR = Rate in control group Rate in treatment group Rate in control group = ARR Baseline rate = 1 RR * Absolute risk reduction
73 5.Data Analysis: Expressing Results of Randomized Trials Number needed to treat (NNT) Number of patients who would have to be treated in order to prevent one event of outcome NNT = 1 ARR Number needed to harm (NNH) Number of patients who could be harmed in order to prevent one event of outcome NNH = 1 ARI * Absolute risk increase (in adverse events of therapy)
74 Ethical Issues Is it ethical to randomize? Or not to randomize? Can truly informed consent be obtained? Who should elicit consent? When can placebo be used? When can shame procedures be used? When should a trial be stopped earlier than planned?
75 Equipoise Genuine uncertainty regarding the comparative therapeutic merits of each intervention being tested Theoretical equipoise: evidence of benefits of each treatment is well balanced Clinical equipoise: no consensus within the expert clinical community Individual vs. collective equipoise
76 Reporting Guideline CONSORT for clinical trials Enrollment CONSORT 2010 Flow Diagram Assessed for eligibility (n= ) Excluded (n= ) Not meeting inclusion criteria (n= ) Declined to participate (n= ) Other reasons (n= ) Randomized (n= ) Allocated to intervention (n= ) Received allocated intervention (n= ) Did not receive allocated intervention (give reasons) (n= ) Allocation Allocated to intervention (n= ) Received allocated intervention (n= ) Did not receive allocated intervention (give reasons) (n= ) Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Follow-Up Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Analysed (n= ) Excluded from analysis (give reasons) (n= ) Analysis Analysed (n= ) Excluded from analysis (give reasons) (n= ) Source: www.consort-statement.org
Source: http://www.equator-network.org/resource-centre/library-of-health-research-reporting/ 77
78 THANK YOU FOR YOUR ATTENTION Next on 4/9/2013 研究設計理論 ( 二 ): Cohort Study