Randomised Controlled Trials

Transcription

1 Randomised Controlled Trials Dr John Stephenson Senior Lecturer in Biomedical/Health Statistics School of Human and Health Sciences University of Huddersfield Huddersfield, GB-HD1 3DH Staff profile:

2 Introduction who am I? I am a biomedical statistician I work collaboratively with colleagues from: UoH Schools of Human & Health Sciences and Applied Sciences Clinicians from various Health Service Trusts around Britain Medical companies and other commercial organisations Other universities in Britain, Australia and elsewhere On projects in fields such as: Tissue viability, spinal surgery, pharmaceutical sciences, clinical education, etc... I provide: Advice on quantitative study design, sample size calculations etc. Statistical methods and results sections of articles, reports and conference items

3 Where did you say you were from again? Huddersfield (pop. 130,000) 300 km N of London 50 km E of Manchester

4 Huddersfield

5 Summary Overview/types of RCT Sampling how, how many, who? Randomisation why, how? Variables predictors, endpoints Participant flow Data analysis Data presentation

6 What is an RCT? an overview A study in which participants are randomly allocated to 2 (or more) groups to test a specific drug, therapy or treatment ( intervention ) 1 or more groups (the experimental or intervention group(s)) receives treatment(s) being tested 1 group (the comparison or control group) receives either: alternative treatment ( positive control ), or placebo or nothing ( negative control )

7 What is an RCT? an overview Groups followed up to see how effective the experimental treatment was Outcomes measured at specific times Any difference in response between the groups assessed statistically

8 The first controlled trial? A report of a controlled study 2600 years ago in the Bible (Daniel 1:11-16) Daniel proposed a study to compare: a group who ate vegetables and water for ten days a group who ate royal food and wine from the table of Nebuchadnezzar the king of Babylon Those with the vegetarian diet were better nourished than those who ate royal food

9 RCTs in the modern era Statistical methods for RCTs formulated by Gossett in 1900s Further developed by Fisher in 1920s First RCT directed by Austin Bradford Hill in 1946 Demonstrated benefits of streptomycin in treatment of TB By 1970s, RCTs were commonplace

10 Hierarchy of Evidence Health services and other organisations consider RCTs to be near the top of hierarchy of evidence Mostly systematic reviews of RCTs! From Australian Institute of Family Studies,

11 Types of RCT Parallel Most common form of RCT (recent PubMed survey 1 found 78% of all RCTs to be parallel) Patient receives one of two or more treatments being compared Cross-over Every participant gets each treatment in random order Separated by wash-out period to avoid carry-over effects 1 Hopewell et al., BMJ 2010;340:c723

12 Types of RCT Pragmatic RCTs Test effectiveness in everyday practice Relatively unselected participants Flexible conditions Inform decisions about practice Explanatory RCTs (less common) Test efficacy in research setting Controlled conditions May use specially selected participants

13 Sampling for RCTs A sample should be representative of the population Sampling method (random / non-random) should be made clear Random sampling may be preferred on statistical grounds Simple random sampling Systematic sampling Cluster/multistage sampling etc. Non-random sampling much more widely used in health research Quota Convenience etc.

14 How many? Appropriate sample size is essential for an ethical trial Sample should be large enough to detect a meaningful difference between groups (if one exists) Sample too small trial may be unethical Potential risk to patients participating in study that cannot answer its research question Sample too large waste of time and resources No statistical disbenefits in general Patients could be harmed by unnecessary delay in treatment introduction

15 How many? Sample size depends on many factors: Required power of study (probability of detecting a real effect) Significance level (maximum probability of falsely claiming a real effect) Variability of observations Size of effect required to be detected

16 How many? Significance level should be as low as possible 5% significance level often specified Chance of type I error (falsely claiming a real effect) = 5% Lower significance levels require larger samples Power should be as high as possible 80% power most commonly specified Chance of type II error (failing to detect a real effect) = 20% Greater power requires larger samples

17 Estimates of sample size Higher variability of data requires larger samples Variability not usually known before start of study Can be estimated using pilot studies or the literature Detection of smaller effects requires larger samples Confidence that an observed difference not random noise Difficult to definitively ascribe smaller changes to treatment Previous research can provide estimates of effect size

18 Typical Huddersfield University sample size conversation (1) Clinician: How many patients do I need for a trial of my patent Yellow Swamp Fever antidote? Statistician: But I need to know the data variability, the expected effect of interest, significance level, study power, how many groups in the trial, relative group sizes Clinician: (sticking fingers in ears): No idea. You re the statistician. Just tell me how many I need. That s what we pay you for. Statistician:

19 Typical Huddersfield University sample size conversation (2) Clinician: I need a sample size calculation for a funding bid I m writing. I want to use messy and incomplete data to test lots of variables, whose effect on multiple outcomes measured at several time points is probably very small. Statistician: Goodness me! I think you ll need a very large sample. Give me the details and I ll just work it out. Clinician: Make sure the answer is 36. That s as many patients as I ve got. Statistician:

20 Inclusion/exclusion criteria Define which patients are to be recruited Numbers assessed for eligibility, and those excluded from trial should be recorded Exclusions comprise: refusals; those not meeting inclusion criteria

21 Inclusion/exclusion criteria Typical sample criteria for a RCT assessing quality of life in Type 2 diabetic patients with non-infected surgical wounds Inclusion criteria: Patients over 18 years Have type 2 diabetes Have a non-infected surgical wound Are able to speak and understand English Are mentally alert and able to give informed consent Exclusion criteria: Patients under 18 years old Diabetes type 1 or no diabetes Have an infected wound/ulcer or no wound Unable to speak or understand English Cannot give informed consent

22 Treatment allocation Patients allocated to treatment group on entering trial by process of random allocation (randomisation) Computer generated random numbers, random number tables, opaque sealed envelopes, dice, coins, balls in a bag etc. Allocation concealment: participants should have no prior knowledge about the intervention they will receive Allocation usually (but not always) in 1:1 ratio

23 Why do we randomise? Avoids systematic differences between groups Promotes - but does not guarantee - similarity in terms of baseline characteristics (both known and unknown) Statistical methods available to adjust for variations in baseline covariates between groups if necessary

24 Complete randomisation Every participant is randomly assigned to a treatment (e.g. control or intervention) Simple design: may not necessarily control for all variability

25 Restricted randomisation (a.k.a. block randomisation) Suppose you are planning to recruit 40 patients to a trial You prepare 40 sealed envelopes for allocation (20 control, 20 treatment) Patients randomised; commence treatment on recruitment Unfortunately, fewer patients are recruited than anticipated You realise that you will only get 24 patients in the available time You subsequently find that by chance 17 patients have been randomised to control and only 7 to treatment Imbalance could have been avoided using restricted randomisation

26 Restricted randomisation (a.k.a. block randomisation) Randomize participants into groups of equal sizes For 2-arm study, participants grouped into blocks of 2, 4, etc. Participants in each block distributed equally across treatments Keeps numbers of participants in each group similar at all times Matched pairs studies always use this form of randomisation

27 Stratified randomisation Suppose we have 100 patients (40 men and 60 women) available for 2-arm trial Assume gender is known to be a strong predictor of the outcome Gender is a source of unwanted variability (a blocking factor) We need to control for gender, by ensuring a gender balance at baseline

28 Stratified randomisation Complete randomisation: could give us (say): Control: 10 men, 40 women Treatment: 30 men, 20 women Possible confounding bias Conduct separate randomisation schemes for each gender stratified randomisation (by gender) This would give us: Control: 20 men, 30 women Treatment: 20 men, 30 women No confounding by gender

29 Cluster randomisation Consider a study testing effect on educational program administered to student nurses If applied, program must be applied across a whole school Schools randomised to receive or not receive program All students in each school have same program status Avoids practical difficulties of multiple treatments within same school School 1: Control Randomisation at school level School 2: Treatment Nurse 1: Control Nurse 2: Control Nurse 3: Control Nurse 1: Treatment Nurse 2: Treatment Nurse 3: Treatment

30 Blinding Assessment bias may arise if clinicians and/or patients aware of treatment allocation i.e. not blinded All participants (clinicians/assessors, patients) should be blinded where possible Not always possible e.g. assessment of surgical procedures Treatment may cause side-effects Clinicians/assessors and patients unaware of treatment allocation double blinded trial Clinicians/assessors only blinded single blinded trial

31 Pilot studies If time permits, a pilot study may be conducted before the main study Evaluate feasibility: time, cost, adverse events, etc. What is the compliance rate for my treatment? Have I allowed enough time for the treatment to take effect? Have I optimised the treatment parameters? Can also assess data variability to help predict sample size for main study

32 Variables Many possible variables may be recorded But analysis of comparability between groups normally restricted to those known to be strong predictors of primary outcome In many RCTs the treatment variable is the only predictor Categorical variable with (typically) 2 levels: control & intervention Large studies & effective randomisation well balanced groups no need to control for other factors Best to specify a single primary outcome a priori: any other outcomes defined to be secondary outcomes

33 Multiple outcome variables Multiple outcomes can lead to fishing expeditions Fishing the data for a favourable outcome; discarding less favourable ones! Rigorous treatment of multiple outcomes can be very complex which is maybe why it is not done properly in 1000 s of published trials!

34 Ethical issues All RCTs must be passed by an ethics committee Judge that trial does not contravene the Declaration of Helsinki Informed patient consent is required before patients entered into trial

35 Participant flow For each group, we need to report numbers of participants : Randomly assigned Receiving intended treatment Completing study protocol Analysed for primary outcome Summarised in CONSORT (Consolidated Standards of Reporting Trials) flowchart

36 CONSORT flow diagram

37 Example participant flowchart For RCTs, Methods section often includes flowchart summarising patient involvement at every stage of the study like this one for a diet trial From Truby et al. (2006), BMJ, doi: / bmj

38 Participant flow Substantial proportions of participants lost at any stage may reduce external validity of trial Those remaining may no longer be representative of those eligible Imbalance in losses between treatment groups may reduce internal validity of trial Could lead to non-random differences between treatment groups, which could influence outcome

39 Data analysis: descriptive As in other quantitative research designs, most RCTs include a descriptive summary of the data May include summary statistics in text, tables and/or graphs Both baseline and post-treatment results usually provided Participant characteristics usually summarised both by group and as a whole cohort

40 Data analysis: inferential Statistical processes used in the analysis of RCTs usually require to assess differences or ratios in group means or proportions Independent samples t-test, Mann-Whitney U test, analysis of variance (ANOVA), analysis of covariance (ANCOVA), chi-squared test, Kruskal-Wallis test, log-rank test, multivariate ANOVA All common tests available on standard statistical packages Tests for single sample studies, pre-post ( paired ) studies and repeated measures not suitable for the main comparative analysis of RCTs Single sample t-test, sign test, paired samples t-test, Wilcoxon signed ranks test, repeated measures ANOVA, Friedman test, McNemar s test

41 Reporting the analysis key statistics Group characteristics at baseline Magnitude of the treatment effect; e.g. Mean value of outcome in both groups, and the difference between them Incidence of condition in both groups, plus risk/odds ratio for treatment group Number needed to treat (NNT) may be reported if appropriate Number needed to receive new treatment to prevent 1 adverse occurrence Test statistics, p-values and confidence intervals from inferential tests

42 P-values Ubiquitous statistic that very few people seem to properly understand! Assesses the significance of the test Is any observed difference due to sampling error or a real effect? Firstly assume there is no difference between treatment groups Then the p-value is the probability that we would get the result we just got (or an even more extreme one) Greater than a given cut-off (usually 5%) findings could just be sampling error Don t claim anything! Less than the cut-off assume findings represent a real treatment effect Claim something!

43 Confidence intervals Assess the precision of the effect We derive an estimate from our sample The precise value in the population will never be known But we can derive an interval within which (broadly speaking) we are 95% confident that the true value lies Lower limit Best estimate of treatment effect Upper limit 95% confidence interval

44 Descriptive data analysis baseline summary Typical presentation of group characteristics at baseline Variables here are a mixture of interval/ratio and categorical Mean and SD used for summarising interval/ratio variables Frequencies and/or proportions/percentages used to summarise categorical variables From Bleakley et al., BMJ 2010;340:c1964 doi: /bmj.c1964

45 Options for graphical presentations (1) Line graph showing readings taken from control and treatment groups at different time points, with associated confidence intervals around points in each group Dot plot showing differences between control and treatment groups at different time points, also with associated confidence intervals Both taken from Bleakley et al., BMJ 2010;340:c1964 doi: /bmj.c1964

46 Options for graphical presentations (2) Comparison of numerical data Box plot showing medians, ranges and inter-quartile ranges readings from control and treatment groups on outcome measure From Azouz et al., Journal of Aerosol Medicine and Pulmonary Drug Delivery, 27 (3), pp

47 Options for graphical presentations (3) Comparison of categorical data Clustered bar chart: each pair of bars shows proportion of patients in each group giving labelled response Difficult to get picture of overall treatment performance From Cameron et al. (2005), Journal of Wound Care 14 (5) pp

48 Options for graphical presentations (4) Survival analysis curves compare survival over time of patients in control and treatment groups From Prentice, R. L. et al. JAMA 2006;295:

49 Subgroup analysis Subgroup analyses are widespread in published RCTs For example, males and females, or older and younger participants, analysed separately Why is this not always a good idea?

50 Sub-group analysis Smaller studies reduced study power Less likely to detect any treatment effect that may exist Multiple comparisons increased type I error rates Probability of falsely claiming a non-existent effect that does not exist Difficult to interpret results Significant result in one subgroup, but not another, is not evidence that treatment effect differs between subgroups

51 So when SHOULD we use subgroups, then? When there is a clear a priori reason to expect different treatment effects in the groups Example: effect of exercise programme on body fat % might be expected to be very different in men and women Statistical methods of control also available In multi-centre trials; may be useful to present results by centre as well as pooled results and means of data quality/consistency checking

52 Common errors in RCT data analysis (1) Use of statistical tests to compare patient characteristics in study groups at baseline Any differences can ONLY be due to sampling error so what s the point of the test? Non-significant imbalances of strong predictors may have more effect than a significant imbalance on a weak or unrelated one Multiple outcomes analysed independently No consideration of correlations between multiple outcome variables Inflated error rates

53 Common errors in RCT data analysis (2) Lots of variables, not enough data Rule of thumb: at least 10 cases per variable May need to control for covariates if groups unequal at baseline Disregarding data from large proportion of patients lost to follow-up in longitudinal RCT Unnecessarily classifying raw data into groups Loses information No account for clustering of data in cluster RCTs More advanced statistical treatment required

54 CONSORT checklist

55 CONSORT checklist

56 Conclusions RCTs the gold standard of experimental design Unpaired study; characterised by: Use of control group in addition to treatment group(s) to promote internal validity Randomisation of participants to groups (various methods available) Requires representative samples of adequate size Flow of participants through trial monitored Analysed with a range of descriptive and inferential statistical methods

57 Thank you for your attention!