研究設計理論 ( 一 ): RANDOMIZED TRIALS. 醫學研究初階課程 General Medicine, CGMH 劉素旬醫師 03/12/2013

Transcription

1 1 研究設計理論 ( 一 ): RANDOMIZED TRIALS 醫學研究初階課程 General Medicine, CGMH 劉素旬醫師 03/12/2013

2 2 Learning Objectives Definition of a trial Features of trial design Elements of a trial Purposes of randomization and masking Reason for analysis by intention to treat Ethical issues Reporting guideline

3 3 What is a Trial? The action or process of putting something to a test or proof (Webster) An experiment designed and carried out to provide information on the merits of one treatment or procedure relative to another (Prof. Meinert C. )

4 4 Definitions for Clinical Trial A controlled experiment having a clinical event as an outcome measure, done in a clinical setting, and involving persons having a specific disease or health condition (Prof. Meinert) A controlled experiment to evaluate the efficacy and safety of a treatment. (Prof. Meinert)

5 5 Why Are Clinical Trials Important? (I) Gateway for new medical advances Gold standard for evaluation Drugs, devices, management Guideline for medical care Insurance coverage EBM

6 6 Why Are Clinical Trials Important? (II) Economic impact Costs of medical treatments Important segment of the economy Ethical issues Human subject experiments Best use of limited resources Access to information

7 7 Epidemiologic Studies Measuring Descriptive Diseases Health behaviors Environmental exposures Health-related costs Comparative Examining associations between exposures and health outcomes Group comparison Relative measures of disease

8 8 Alternatives to A Randomized Trial Without a comparison group Case report Case series With a comparison group Historical controls Simultaneous, nonrandomized controls Pre- post design

9 9 Essence of a Trial Design Experimentation (vs. observation) Exposure is under the control of the investigator Randomized Non-randomized Exposure is merely observed by the investigator in observational studies Method for allocating exposure/intervention Prospective standardized protocol Innovation or efficacy of the intervention is unknown

10 Source: care.diabetesjournals.org 10

11 Source: clincancerres.aacrjournals.org 11

12 12 Before The Need for A Trial Source:

13 Source: 13

14 14 Phases of Clinical Trials for Evaluation of New Drugs Phase I Clinical pharmacology Optimum dose, toxicity, PK, PD (usually healthy volunteers or patients, not controlled) Phase II Clinical investigation Safety and efficacy, sometimes controlled (usually patients) Phase III Clinical trial Comparative efficacy, expanded and controlled usually randomized Phase IV Post-marketing trials Adverse effects or off-label use, controlled observational studies or uncontrolled post-marketing surveillance

15 15 Efficacy vs. effectiveness Efficacy Trial To assess the efficacy of a treatment under ideal conditions Trial setting not designed or intended to approximate everyday conditions Effectiveness Trial To assess usefulness of a test treatment Trial conditions approximate everyday practice conditions

16 16 Types of Controls No therapy Trial of removal of allergens from home environment to control asthma Placebo Use of estrogen vs. placebo for heart disease prevention in women Sham procedure Acupuncture vs. incorrect placement of needles Standard treatment Sequential anti-h. pylori treatment vs. standard treatment

17 17 Types of Trial Designs Parallel comparison Cross-over Factorial Group allocation (cluster randomized trial)

18 18 Parallel Randomized Trial Source:

19 19 Cross-over Trial Each subject serves as his/her own control Feasible only if there is no carry-over treatment effect after the washout period Cannot exam for treatment effect on mortality If order of treatments may influence results, treatment administration can be randomized

20 20 Factorial Design Two rationales Potential economical way to test two treatments simultaneously, if their modes of action are independent Method to test for treatment synergy Source: Abernethy et al. Contemp Clin Trial 2006; 27(1):83-100

21 21 Factorial Design: Two Factors Source: Arnold et al. Simulation methods to estimate design power: an overview for applied research BMC Med Res Methodol

22 22 Cluster Randomization Trial Randomization unit is a group of individuals When individual randomization and intervention is not feasible or acceptable Tracking Contamination Administrative resources Outcomes and analysis must be appropriate to group allocation design Group measures of outcome Individual measures, accounting for correlation

23 23 Group Allocation (Cluster Randomization) Source: Arnold et al. Simulation methods to estimate design power: an overview for applied research BMC Med Res Methodol

24 24 Treatment Comparisons Depending on the alternative hypothesis Superiority trials: one treatment is better than the other Two sided: H a : A > B or A < B One sided: H a : A > B; H a : A < B Equivalence trials: two treatments are clinically equivalent (within some acceptable small margin) Two sided: H a : A B or A B Non-inferiority trials: one treatment is as effective as the other One sides: H a : A B

25 25 Interventions That Can Be Evaluated Treatments, including drugs and cognitive behavioral therapy Preventive measures Screening programs Vaccination Personal protective equipment Health service delivery or technology Individual vs. group intervention

26 26 Target Population People with a disease People at high risk for a disease/condition Primary prevention Secondary prevention General population Primary prevention Health care delivery program Community health program

27 27 Types of outcome Intervention Health outcomes: mortality, other clinical event rate, disease-free survival, biomarker, quality of life, cost Prevention Incidence of disease, mortality Health care delivery Health outcomes, process outcomes Community health: Indices of population health

28 28 Follow-up Length Short-term: often use surrogate outcome, biomarker Long-term: clinical event End of observation After a defined period of time A common close-out date (administrative censoring)

29 29 Scale of A Trial Single-center Multi-center International

30 30 Elements in the Design of Randomized Controlled Trials (RCT) 1. Selection of subjects 2. Allocation of subjects to treatment groups 3. Data collection 4. Masking 5. Data analysis

31 31 1.Selection of Subjects Representativeness Generalizability (external validity)

32 32 2.Allocation of subjects Predictable assignment (not recommended) Day of the week Birth date Order enrolled Randomization Not predictable Having no specific pattern; lack causal relationship Process in which there is associated with every legitimate outcome a probability

33 33 Goal of Randomization To compare otherwise similar groups by removing potential biases in the choice of treatment(s) Selection bias Factors associated with prognosis Treatment Outcome Confounding by indication Main reason why RCT is considered gold standard of study design

34 34 Goal of Randomization Secondary: to increase the likelihood of balanced groups in known and unknown risk factors Tertiary: to provide a probability basis for hypothesis testing, i.e. to assure statistical tests will have valid significance levels

35 35 Results of a Trial of BCG Vaccine Non-randomized controls No. Cases No. TB Death % Vaccinated Controls Randomized controls No. Cases No. TB Death % Vaccinated Controls

36 36 Observational Study Intervention Arm N=1000 Control Arm N= Total Deaths 300 Mortality: 180/1000 =18% 300/1000=30%

37 37 Observational Study Intervention Arm N=1000 IRHB (-) IRHB(+) N=800 N=200 Control Arm N=1000 IRHB (-) IRHB(+) N=500 N= Total Deaths 300 Mortality: 180/1000 =18% 300/1000=30% Proportions of patients with arrhythmia in the two comparison groups may differ

38 38 A Randomized Trial Intervention Arm N=1000 IRHB (-) IRHB(+) N=654 N=346 Control Arm N=1000 IRHB (-) IRHB(+) N=649 N= Total Deaths 241 Mortality: 238/1000 =23.8% 241/1000=24.1% P-value= Proportions of patients with arrhythmia in the two comparison groups may differ

39 39 Take a break Illustration: Cathy Wilcox

40 40 2. Allocation of Subjects Types of Randomization Schemes Simple randomization Restricted randomization Blocking Stratification Adaptive randomization

41 41 Simple Randomization Table of random numbers

42 42 Simple Randomization Scheme An Example If 2 groups are compared When digit is: 0-4 Assignment is Treatment A When digit is: 5-9 Assignment is Treatment B If digit is If 3 groups are compared Next assignment is 1-3 Treatment A 4-6 Treatment B 7-9 Treatment C 0 is ignored

43 43 Simple Randomization Computer randomization schemes Create lists of random numbers Generate treatment assignment Provide reproducible audit trail of treatment assignment Control and keep records of access to treatment assignment Source:

44 44 Simple Randomization Advantages Complete independence among each assignment Completely unpredictable Numbers of patients assigned to each group should be equal in the long run Disadvantages Imbalances when sample size is not large enough Numbers of patients in each group Confounding factor Lower power Diminish credibility of results

45 45 Restricted Randomization Randomization with constraints to produce expected assignment ratio according to time or specified covariate(s) Permuted block or blocking Stratification

46 46 Blocking Choose the smallest possible block size The sum of integers defined in the treatment allocation ratio Examples: 2 for a 1:1 ratio; 3 for a 2:1 ratio Larger block sizes are multiples of the smallest one: 4, 6, 8, List all possible block sequences (permutations) Example: for a block size of 4 with a 1:1 treatment allocation ratio BAAB ABAB BBAA ABBA.

47 47 Blocking (cont.) Randomly choose one block with replacement until the desired number of assignments Randomly order blocks to create the assignment list Source: bmsi.ru

48 48 Blocking Advantages Overall balance, esp. in smaller trials Achieve balance periodically Protects against timerelated changes Good statistical power Disadvantages Can facilitate prediction of future assignment More problematic for unmasked trials or poorly masked trials Remedy#1: block size(s) remains a secret Remedy#2: use more than one block size

49 49 Stratification 1. Stratify by Sex 1000 Patients 600 Males 400 Females 2. Stratify by Age 360 Adults (<65 yrs) 240 Elderly ( 65 yrs) 300 Adults (<65 yrs) 100 Elderly ( 65 yrs) 3. Randomize each subgroup Treatment A =500 Treatment B =500

50 50 Stratification To ensure balance in treatment assignments within subgroups defined before randomization Subgroup should be related to outcome Essentially create separate treatment assignment schemes for each stratum

51 51 Practicalities of Stratification Limit to a few variables Typical ones Clinic in a multicenter trial Surgeon Stage of disease Demographic characteristics (age, gender) Too many strata may lead to overall imbalances Must use blocking to ensure that stratification is effective Strata can be varying sizes

52 52 Stratification and Blocking Source: Morrissey et al Contemp Clin Trials

53 53 Adaptive Randomization The probability of assignment to treatment groups does not remain constant but depends on the current balance or composition of the groups (Piantadosi, 1997)

54 54 Covariate-Adaptive Randomization Minimization After randomizing the first patient, the treatment assignment that yields the smallest imbalance is chosen Balance marginal treatment totals across prognostic factors Can handle many more factors than stratification Allocation sequence cannot be determined in advance

55 55 Response-Adaptive Randomization Play the winner Change allocation ratio to favor the better treatment based on the primary outcome Need to evaluate outcome (or surrogate outcome) relatively quickly Source: Gupta et al J Clin Epidemiol

56 56 Misconceptions about Randomization Haphazard procedure is the same Guarantees comparable groups Guarantees that actual treatment allocations will match allocation ratio A study without randomization is invalid

57 57 Randomization Does Not Assure external validity Eligibility criteria Actual patient population Real-world conditions Eliminate bias associated with data collection, treatment adherence or follow-up

58 58 Generalizability of Randomized Clinical Trials A.k.a external validity May not provide good estimates of the absolute effect of treatment for the general population However, relative treatment effect may be generalizable to the general population

59 59 3.Data Collection Treatment Compliance Outcome Prognostic profile at entry Baseline characteristics

60 60 4.Masking Treatment assignment Subjects Primary physicians of study subjects Data collectors (evaluators) Data analyst Outcome assessment More important if treatment assignment is not masked

61 61 Level of Masking Single - participant Double - participant AND investigator Triple - participant AND investigator AND monitoring committee Source: scientific-misconduct.blogspot.com

62 62 Masking Protects Against Bias in Reports and behavior of participants Investigator s bias Data collection Outcome assessment Concomitant treatments

63 63 5.Data Analysis Problems of non-compliance Unplanned cross-overs: drop-outs, drop-ins Example: National Emphysema Treatment Trial (NETT), Source: JHSPH OpenCourseWare

64 64 5.Data Analysis Intention-totreat analysis: net effect of noncompliance will reduce the observed difference Source: jama.jamanetwork.com

65 65 5.Data Analysis Secondary: Per-protocol (As-treated) analysis Observational in nature

66 66 Compliance Analyses Coronary Drug Project 5-year Mortality No. Patients Mortality Clofibrate % Placebo % Source: Canner et al. 1980

67 67 Compliance Analyses Coronary Drug Project 5-year Mortality No. Patients Mortality Clofibrate Arm % Poor complier (<80%) Good complier ( 80%) % % Placebo Arm % Source: Canner et al. 1980

68 68 Compliance Analyses Coronary Drug Project 5-year Mortality Clofibrate group Placebo group Compliance No. Patients Mortality No. Patients Mortality Poor complier (<80%) Good complier ( 80%) % % % % Total % % Source: Canner et al. 1980

69 69 Dealing with Non-compliance Use of pilot studies (run-in periods), however, effectiveness depends on Precision in predicting compliance Costs of recruitment Monitoring compliance Interview, pill-counting Medication bottle devices Blood/urine tests Directly observed treatment (DOT)

70 70 5.Data Analysis Subgroup analysis Lack of power Over-interpretation Screening questions to help decide if a subgroup analysis is justified Was the analysis a priori defined at the design phase? Is the analysis subject to bias? Is the analysis biologically plausible? Is the result of the trial significant overall?

71 71 5.Data Analysis: Expressing Results of Randomized Trials Efficacy Relative risk (RR)- the ratio of rates of outcomes comparing treatment and control groups RR = Rate in treatment gruop Rate in control group

72 72 5.Data Analysis: Expressing Results of Randomized Trials Efficacy Relative risk reduction (RRR)- the proportional reduction in rates of outcomes comparing treatment and control groups RRR = Rate in control group Rate in treatment group Rate in control group = ARR Baseline rate = 1 RR * Absolute risk reduction

73 73 5.Data Analysis: Expressing Results of Randomized Trials Number needed to treat (NNT) Number of patients who would have to be treated in order to prevent one event of outcome NNT = 1 ARR Number needed to harm (NNH) Number of patients who could be harmed in order to prevent one event of outcome NNH = 1 ARI * Absolute risk increase (in adverse events of therapy)

74 74 Ethical Issues Is it ethical to randomize? Or not to randomize? Can truly informed consent be obtained? Who should elicit consent? When can placebo be used? When can shame procedures be used? When should a trial be stopped earlier than planned?

75 75 Equipoise Genuine uncertainty regarding the comparative therapeutic merits of each intervention being tested Theoretical equipoise: evidence of benefits of each treatment is well balanced Clinical equipoise: no consensus within the expert clinical community Individual vs. collective equipoise

76 76 Reporting Guideline CONSORT for clinical trials Enrollment CONSORT 2010 Flow Diagram Assessed for eligibility (n= ) Excluded (n= ) Not meeting inclusion criteria (n= ) Declined to participate (n= ) Other reasons (n= ) Randomized (n= ) Allocated to intervention (n= ) Received allocated intervention (n= ) Did not receive allocated intervention (give reasons) (n= ) Allocation Allocated to intervention (n= ) Received allocated intervention (n= ) Did not receive allocated intervention (give reasons) (n= ) Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Follow-Up Lost to follow-up (give reasons) (n= ) Discontinued intervention (give reasons) (n= ) Analysed (n= ) Excluded from analysis (give reasons) (n= ) Analysis Analysed (n= ) Excluded from analysis (give reasons) (n= ) Source:

77 Source: 77

78 78 THANK YOU FOR YOUR ATTENTION Next on 4/9/2013 研究設計理論 ( 二 ): Cohort Study