Randomized Controlled Trials and Challenge Trials: Design and Criterion for Validity

Transcription

1 Zoonoses and Public Health REVIEW ARTICLE Randomized Controlled Trials and Challenge Trials: Design and Criterion for Validity J. M. Sargeant 1,2, D. F. Kelton 1,2 and A. M. O Connor 3 1 Centre for Public Health and Zoonoses, University of Guelph, Guelph, ON, Canada 2 Department of Population Medicine, Ontario Veterinary College, University of Guelph, Guelph, ON, Canada 3 Department of Veterinary Diagnostic and Production Animal Medicine, Iowa State University College of Veterinary Medicine, Ames, IA, USA Impacts Randomized controlled trials are an experimental design used to evaluate the efficacy of interventions under real world conditions; criterion for validity are described. In veterinary medicine, trials with deliberate disease induction also can be used to evaluate interventions in the species in which the intervention is intended to be used. Reporting guidelines are available for randomized controlled trials, including the REFLECT statement for livestock and food safety. Keywords: Randomized controlled trials; challenge trials; validity; study design; veterinary Correspondence: J. M. Sargeant. 103 MacNabb House, Ontario Veterinary College, University of Guelph, Guelph, ON, N1G 2W1 Canada. Tel.: +(519) ext ; fax: (519) ; sargeanj@uoguelph.ca Received for publication September 5, 2013 doi: /zph Summary This article is the third of six articles addressing systematic reviews in animal agriculture and veterinary medicine. This article provides an overview of clinical trials, both randomized controlled trials (RCTs) and challenge trials, where the disease outcome is deliberately induced by the investigator. RCTs are not the only study design used in systematic reviews, but are preferred when available as the gold standard for evaluating interventions under real-world conditions. RCTs are planned experiments, which involve diseased or at-risk study subjects and are designed to evaluate interventions (therapeutic treatments or preventive strategies, including antibiotics, vaccines, management practices, dietary changes, management changes or lifestyle changes). Key components of the RCT are the use of one or more comparison (control) groups and investigator control over intervention allocation. Important design features in RCTs include as follows: how the population is selected, approach to allocation of intervention and control group subjects, how allocation is concealed prior to enrolment of study subjects, how outcomes are defined, how allocation to group is concealed (blinding) and how withdrawals from the study are managed. Guidelines for reporting important features of RCTs have been published and are useful tools for writing, reviewing and reading reports of RCTs. Introduction Randomized controlled trials (RCTs) are the gold standard for evaluating interventions to treat or prevent adverse health events, when it is feasible and ethical to use this study design (Clancy, 2002). In a RCT, the investigator randomly allocates study units from a study population to two or more intervention groups. The efficacy of the interventions on a specified outcome is compared (Fig. 1). The term intervention is used to describe therapeutic or preventive treatments, including antimicrobials, biologics and surgical, dietary or management changes used to prevent, reduce or treat an adverse outcome. Outcomes in animal studies could include diseases, production or animal welfare indices, quality of life or carriage of animal, food safety or zoonotic pathogens. However, we will use disease or outcome as a generic term throughout this manuscript. The term study unit is used rather than study subject because interventions may be allocated at different levels: study unit refers to the level at which the intervention is Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014) 18 27

2 J. M. Sargeant et al. Randomized Controlled Trials and Challenge Trials Fig. 1. Randomized controlled trials for interventions to treat a disease condition. The study population should represent as closely as possible the population to which you wish to refer the results (i.e. the target or reference population) (Lund et al., 1994; Dohoo et al., 2010). The study population is drawn from a source population available to the researcher for inclusion in a trial. Thus, the source population also should be representative of the target population if the study results are to be generalizable back to the target population. Representativeness is important in RCTs, as they are intended to evaluate performance of an intervention under real-world conditions. Therefore, eligibility criteria for participants are of utmost importance. Particular considerations include disease definition for assessments of therapeutic interventions (e.g. clinical versus subclinical), chronicity or severity, previous or current treatments, concurrent illness and prognostic factors such as age or production stage. A common criticism of human pharmacologic trials is that eligibility criteria are so strict that patients who participate do not represent the real-world population in terms of patient characteristics and other factors such as healthcare access and adherence (Suarez-Almazor, 2002; Kingsley et al., 2005; Wolfe, 2005). In animal trials, selection of animals with homogeneous traits in some trials may not be representative of these animals in their natural environment (e.g. a colony of Beagles may or may not be representative of the broader dog population) or trials using individually housed food animals may not be representative of animal density or housing in field settings. All efforts should be made to ensure that the results of an RCT can be generalized to the real world, that is, the RCT should have external validity. Ultimately, only the consumer of the research report can determine the generalizability of a RCT to their specific situation. Additionally, internal validity (i.e. freedom from systematic error or bias) is essential; a study that is biased cannot meaningfully be generalized. allocated (allocation unit) or at which the outcome is measured (outcome unit) (O Connor et al., 2010) and may be an individual or, as is common in livestock studies, a group such as a pen, room or herd of animals. Figure 1 illustrates the design of a RCT to evaluate a therapeutic intervention. RCTs also can be used to evaluate preventive interventions when intervention groups are assigned prior to development of disease, and the incidence of disease is compared between intervention groups. Important design considerations for RCTs include the recruitment and make-up of the study population, method of allocation to intervention groups, outcome definition, blinding and withdrawals from the trial. Study Population Allocation of Study Units to Intervention Groups Randomized controlled trials are hypothesis-testing studies, and therefore, it is essential that a concurrent comparison group be used. An outcome may improve, stay the same or decline over time, with or without an intervention. Therefore, without a concurrent comparison group, it is not possible to determine the extent to which any changes in the outcome are the result of the intervention. An exception to this is the cross-over trial where individual animals serve as their own control (described later in this section). Randomization of individuals or groups to an intervention group is so fundamental to the validity of a trial that the word is included in the name, that is, randomized controlled trial. Randomization involves the use of a random process to assign study units to groups, such that each study unit has a known and usually equal probability of being allocated to each intervention group (Moher et al., 2010). Randomization may be implemented through a variety of methods, from a simple coin toss to the use of random number generators. Randomization minimizes (but does not always eliminate) differences among groups and allows probability theory to be used to express the likelihood that any differences in outcome between intervention groups are the result of chance (Schulz and Grimes, 2002). Randomization of allocation (to intervention groups) reduces selection bias in the assignment of interventions. Therefore, the comparisons will not be invalidated by the conscious or unconscious assignment of subjects with a particular trait or based on severity of disease. Randomization tends to balance the intervention groups in terms of important prognostic characteristics or confounding variables, but does not ensure that the groups will be balanced for these factors (Altman, 1985). Therefore, some 2014 Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014)

3 Randomized Controlled Trials and Challenge Trials J. M. Sargeant et al. investigators may use design features such as stratification or blocking to ensure even distribution of important features such as age, sex, weight or concurrent disease status, which may impact the outcomes of the trial, particularly when sample size is small. With block randomization, study units are divided into several blocks of equal or unequal size, and the study units are randomly allocated to intervention groups within blocks. With stratified randomization, important prognostic factors are identified, and study units are randomly assigned to treatment group within designated strata for these factors. For instance, if age was considered an essential prognostic factor, study units could be randomly assigned to intervention group separately for young animals and for old animals. Details on blocking and stratified randomization are available (Pocock, 1998; Piantadosi, 2005; Moher et al., 2010). An important consideration with randomization is the ethics of the intervention(s) administered to the comparison group(s). Control or comparison groups fall into two broad categories. Positive controls are commonly used in RCTs where there is an existing intervention and failure to provide that intervention is not ethically acceptable (Petrie and Watson, 2013). In this case, the comparison is between the new intervention and the currently best available intervention (a.k.a. positive control). Positive controls allow the investigator to comment on the efficacy of the intervention of interest relative to a current intervention in the form of as good as or better than. Negative controls are untreated, although they should receive a placebo or sham intervention to when it is possible to do so (Petrie and Watson, 2013). Placebo or sham treatments can minimize observer bias and also, in some circumstances, can allow the research to mimic the conditions of the intervention group without using the biologically active ingredient (Johnson and Besselsen, 2002). An example of this would be a vaccine trial where the sham intervention could consist of vaccinating animals with product containing the same adjuvant as the intervention, but not the antigen. While they are generally less expensive to conduct than positive control trials (as the magnitude of the difference in response between groups will generally be greater), trials using negative controls limit the investigator to make statements as to whether the intervention of interest is better than nothing. If a current therapy exists, use of a negative control may be ethically unacceptable. Equipoise, which is when no preference exists between interventions, is the ethical basis for random allocation to interventions groups (Freedman, 1987). Therefore, it is unethical to conduct a randomized trial unless there is a good chance of the null hypothesis, that is, no treatment difference. However, there is debate as to whether equipoise should exist on the basis of an individual investigator s belief ( individual equipoise ) or on the basis of the opinion of the profession ( clinical equipoise ) (Lilford and Jackson, 1995; Weijer et al., 2000; Glasser and Howard, 2006). Some alternatives to randomized allocation to intervention group include the use of historic controls or simultaneous non-random controls where there is a systematic assignment of subjects (e.g. based on day of week) (Dohoo et al., 2010). Historical controls may introduce bias into the study because many factors other than the intervention can lead to changes in the frequency or severity of disease over time. An example of the problems associated with the use of historical controls would be their use in feedlot trials for the evaluation of treatments or preventive strategies for respiratory diseases. Morbidity and mortality due to respiratory diseases can vary significantly over time due to changes in environmental conditions, pathogen presence and animal characteristics. Additionally, changes in diagnostic tests over time also could result in changes in the apparent frequency of a disease outcome over time. If historical controls are used, it is not possible to differentiate between intervention effects and time effects. Therefore, as the ability to estimate the impact of historical controls is very difficult, this approach should be avoided. Systematic allocation can introduce the potential for biases at several levels. Systematic assignment may lead to bias if the pattern of intervention assignments can be identified by the person assessing the outcome (Altman, 1991). For example, if all animals or pens with even ear tags or odd ear tags are allocated to the same group, and if the person assessing the outcome knows this sequence, the assessment of disease outcome may be affected by knowledge of the system. A different problem can arise if the individuals assigning interventions are aware of the systematic sequence and are reluctant to allocate animals perceived to be at high risk of illness into a specific intervention group. If the individuals use this information to circumvent the allocation and allocate to a different group, this can lead to selection bias. In some instances, the systematic sequence is not entirely a function of chance (e.g. if interventions are allocated based on the day of the week) (Schulz and Grimes, 2002). Systematic assignment can be strengthened if the first allocation occurs randomly, for example, using a coin toss, and when blocks are used to restart the randomization at set intervals, for example, a new coin toss at the start of each day. The use of a random method to start systematic assignment should be clearly documented in study reports. Another possible method of assigning study units to intervention groups is judgment assignment, where the investigator decides which study units receive which interventions (Pocock, 1998). Due to the possibility of selection bias, this method of intervention allocation should be avoided Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014) 18 27

4 J. M. Sargeant et al. Randomized Controlled Trials and Challenge Trials It is possible to use a subject as their own control, particularly when evaluating interventions intended to alleviate a condition rather than result in a cure (Hills and Armitage, 1979). In randomized cross-over trials, individuals (or study units) are randomly allocated to intervention groups and subsequently switched to the alternate intervention after a period of time which may include a wash-out period during which neither intervention is given. For example, a cross-over trial to determine the impact of a new intervention for chronic lameness in horses may be designed as follows: chronically lame horses are randomly assigned to receive a placebo (negative control group) or the new intervention (experimental group) for 30 days. After 30 days, all horses receive 1 week of placebo (wash-out period to reduce the likelihood of intervention effects carrying over to the subsequent intervention period). After the wash-out period, the negative control group horses are switched to receive the intervention and the experimental group horses receive the placebo for 30 days. Using each horse as its own control, between animal variability is minimized, and a smaller sample size may be used. This design is most appropriate for chronic diseases with slow progression. One limitation of this approach is that bias can occur if there are carry-over effects of the intervention beyond the wash-out period. It is important that the allocation of study subjects to the initial intervention group (negative control group or experimental group) is carried out using a formal random process. In veterinary medicine, particularly in trials involving livestock, the level at which interventions are allocated is of importance (O Connor et al., 2010). It is common for therapeutic or preventive interventions to be given at a group level in these populations. For instance, with feedlot cattle, all animals within a shipment arriving at a farm may be given, or not given, metaphylactic antibiotics based on their risk of developing respiratory disease. Preventive interventions, such as vaccines and feed additives, generally will be given (or not given) to an entire pen of animals. Therapeutic interventions may be given to individual animals based on clinical illness, or may be given to an entire pen via food or water if a threshold of percentage of animals with clinical illness is reached. Similarly, the intervention in an RCT may be allocated at different levels of an organizational structure: for example, at the farm, room, pen, individual level or part of individual (teat, quarter, eye, limb). The level at which interventions are allocated has implications to the statistical analysis of the trial and to its external validity. Even when random allocation is used, there is a possibility that the groups may still differ in one or more important characteristics (Altman, 1985). Therefore, descriptive statistics for the intervention groups based on important and potential confounders should be provided to illustrate any remaining imbalances between groups. Formal statistical comparisons of these characteristics between intervention groups are not appropriate, as randomization controls bias (systematic error), and therefore, any differences between groups are by necessity due to random error (Altman, 1985; Begg, 1990). If groups appear to be unbalanced, stratified or adjusted statistical analysis can be used. Alternatively, if covariate data are available before the treatments are applied, rerandomization to intervention group can be performed (Morgan and Rubin, 2012). However, if this technique is to be considered, it is essential that the degree of imbalance that would justify rerandomization be established a priori. Allocation Concealment To prevent bias at the recruitment stage of the RCT, the person enrolling the study units into the trial should not be aware of the allocation sequence at the time of recruitment (Schulz, 1995). This may be achieved by having the allocation schedule maintained by an individual who is not participating in the trial enrolment. Once a study subject (or, in the case of animals, an owner) has agreed to participate, this individual then informs the investigator which intervention group should be assigned (note, that to maintain blinding, the investigator should be informed of the intervention group, for example A or B, rather than the actual intervention assignment, for example experimental versus control group). Allocation concealment is rarely reported in veterinary or food safety trials (Sargeant et al., 2009a,b, 2010b). However, there is empirical evidence in the human healthcare literature that failure to report allocation concealment is associated with exaggerated intervention effects (Juni et al., 2001; Kjaergard et al., 2001). In the authors opinion, allocation concealment can be difficult to distinguish as a unique step in the some trials that involve livestock health or food safety interventions. It is easier to identify the step where allocation concealment is needed when a decision about eligibility is made on the basis of individual characteristics. For example, in a feedlot trial designed to assess a vaccine to reduce shedding of a food-borne pathogen, all cattle in the feedlot being processed are eligible and are recruited (enrolled) in the trial. The allocation to intervention group may be randomized or based on chute or pen order, and the processing crew is aware of the treatment the next animal/pen will receive. The potential for bias due to failure to conceal the allocation is likely low because the decision about eligibility has already been made. However, in other settings, allocation concealment is an easily distinguished step in the trial. For example, in a trial of oncology therapies in dogs, where a recruitment (enrolment) decision is made for each study subject, knowledge of the allocation of the next enrolled 2014 Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014)

5 Randomized Controlled Trials and Challenge Trials J. M. Sargeant et al. subject could introduce bias. For example, the person recruiting the subject may encourage or discourage an owner based on their perception of the best therapy. Clearly, in this situation, steps should be taken to ensure recruitment without knowledge of the allocation sequence. Outcome Definition The selection of study outcomes is of paramount importance in RCTs. Outcomes may represent a direct measure of the disease or a surrogate/indirect measure (e.g. morbidity/ mortality versus antibody response). If a surrogate outcome is used, there should be a clearly established relationship between the surrogate outcome and a clinically meaningful difference. Whenever possible, outcomes should be objective in nature so as to avoid biased assessment, particularly if blinding is not possible. While subjective outcomes are often relevant (e.g. clinical improvement, functional status or perceived pain, as evaluated by an owner), they should be easy to diagnose or observe, and standardized scales should be used when they are available. Standardized scales also should be used for objectively measured outcomes when they are available. Outcomes should be measured with identical rigour in the intervention and control groups. The reporting of multiple outcomes is common in controlled trials in veterinary medicine (Sargeant et al., 2006, 2009a,b, 2010b). Often, different outcomes measure different aspects of potential intervention effects, such as outcomes related to morbidity or mortality and outcomes related to quality of life or production/performance. Nonetheless, it is important to clearly identify a primary outcome of interest, as this often determines other important study features, such as calculation of sample size (Sargeant et al., 2009a,b, 2010b). The primary outcome should be the outcome of greatest relevance to end-users (Lund et al., 1994; Moher et al., 2010). Sample size calculations should be performed, and the outcome, and the clinically relevant difference in the outcome between groups, used in the calculations should be clearly described (Lund et al., 1994; Schulz and Grimes, 2005). Inadequate sample size can lead to a type II error, due to a lack of power to detect real and clinically meaningful differences (Altman and Bland, 1995). Too large a sample size results in unnecessary involvement of study subjects (Altman, 1980). Blinding Blinding refers to methods used to prevent specific individuals from knowing the intervention group to which subjects belong. Blinding may occur at several levels, and because of confusion about the terminology for individuals who should be blinded, it is preferable to refer to the tasks or functions that are blinded. Often, terms such as single, double and triple blinding are used to describe the various levels of blinding. However, there is evidence in the human healthcare literature that these terms are not consistently used (Devereaux et al., 2001, 2005). There is also evidence that these terms are poorly understood in veterinary science (Giuffrida et al., 2012). Blinding aims to ensure that animals in the intervention or control groups are not cared for, or assessed based on their intervention status, which could be potential sources of bias (Juni et al., 2001). Depending upon staffing for the trial, these tasks (allocation, caregiving, outcome assessment and data analysis) may be conducted by the same or multiple people, and therefore, describing blinding by task is clearer than using terms such as single, double or triple blinding. Although objective outcomes are less subject to bias than subjective outcomes, there is still potential for bias if intervention allocation is known. For example, in a trial evaluating the effect of an intervention on weight gain (an objectively measured outcome), knowledge of intervention status could potentially be associated with increased additional care for animals in a specific group. This could result in increased weight gain in that group beyond any effect of the intervention and thereby introduce bias into the study despite the use of an objective outcome. The person assessing the outcome should also be blinded as to which group the animal (or group of animals) is in, in addition to which intervention the animal (or group of animals) is receiving. For example, the outcome assessor should not know that the study subjects were assigned to groups based on odd/even animal identification number in addition to not knowing which intervention group contains an active agent versus a placebo. This is important because if the assessor of the outcome knows how the study subjects were assigned, they may develop a bias towards one group of animals based on experiences early in the trial. For example, if the first 5 animals requiring treatment in a feedlot trial all have odd identification numbers and the assessor knows interventions were allocated based on odd/even identification number, the outcome assessor may inadvertently increase surveillance of odd-numbered cattle after this experience. This increased scrutiny may lead to increased sensitivity and decreased specificity of disease diagnosis in the odd-numbered cattle as compared to the even numbered cattle, that is, introduce differential information bias. This bias is avoided by not allowing the person assessing the outcome to be aware of the group allocation. The task of data analysis can also be blinded to prevent differential removal of outliers or the possibility of selecting methods of analysis that favour one conclusion or another (significantly different or not). Blinding is not always possible (e.g. in subjects randomized to surgical versus medical interventions, or allocated Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014) 18 27

6 J. M. Sargeant et al. Randomized Controlled Trials and Challenge Trials to different management strategies). If blinding cannot be used, it is important to use an outcome that is measured objectively. Withdrawals from Study and Non-Compliance with Intervention A final important component of RCT study design deals with withdrawals from the study and incomplete compliance with the intervention protocol, and how these issues are managed (Moher et al., 2010). Study subjects may be lost to follow-up due to death, removal from the trial for ethical reasons, or because the owner/manager chooses not to continue. In these instances, outcome data may be incomplete or missing. For other participants, an outcome may have been obtained but the intervention was not administered exactly as per protocol. For example, deviations from the protocol may include partial or complete non-compliance with the intervention regime such as skipping treatments or administration of an incorrect (higher or lower) dosage. In some instances, study withdrawals and protocol deviations may be associated with the intervention group, for instance, if an intervention has unpleasant side effects. On the other hand, subjects may be withdrawn if they are randomized to a no-treatment control and the owner wants immediate treatment. As a result, the estimated intervention effect may be biased if some randomized participants are excluded from the analysis. If there are losses to follow-up in a trial, there are three possible approaches to the analysis: comparison of the groups exactly as randomized (i.e. intention-to-treat or ITT analysis), analysis only of individuals who complied completely with the intervention (i.e. compliers-only analysis) or analysis of individuals based on the intervention that was received, regardless whether the individual was initially randomly allocated to that intervention group (Glasser and Howard, 2006). Intention-to-treat analysis is considered the gold standard, in that this approach maintains randomization and is therefore unbiased (Lachin, 2000; Glasser and Howard, 2006). ITT analyses also address a more pragmatic and clinically relevant question, because adverse effects and lack of compliance are likely to occur in real-world use of the intervention (Lachin, 2000). Loss-to-follow-up, if it differs between intervention groups, may be indicative of issues related to the experimental intervention (i.e. severe side effects) or disease progression or emerging comorbidities requiring immediate treatment in study control subjects. Schulz and Grimes (2002) argue that in human research, loss-to-follow-up of 5% or lower is usually of little concern, whereas a loss of 20% or greater means that the reader should be concerned about the possibility of bias (and some journals refuse to publish papers with attrition of 20% or greater). However, the authors providing these guidelines also caution that losses between 5% and 20% may still be a source of bias. Challenge Trials Challenge trials are a type of trial primarily used in infectious disease studies where study subjects are deliberately exposed to ( challenged with ) the disease agent of interest. In veterinary medicine, unlike in human health care, it is possible to conduct this type of study in the species of interest (although rare examples of challenge studies do occur in human health care; for examples, see Cohen et al., 1997; Seder et al., 2013). Challenge trials may be used to evaluate both therapeutic and preventive interventions (Sargeant et al., 2010a). For therapeutic interventions, the study units are exposed to the disease challenge and then, once disease develops, randomly assigned to receive either the intervention or the control. When used to evaluate preventive interventions, study units receive either the intervention of interest or a control for a predetermined period of time, after which they are challenged with the disease agent. In some instances, negative controls (animals that are neither allocated to an intervention group nor exposed to the disease challenge) are included to validate the challenge model or to identify exposure to naturally occurring cases of the disease during the trial. It should be noted that, where a study design relies on natural exposure to the disease agent, it is important to include a negative control group to show that morbidity/mortality attributable to the agent of interest occurred in at least some of these animals. Challenge trials are often conducted using animals selected to be similar in terms of potential confounding factors such as age and may be conducted under controlled conditions, often with animals housed individually or in small groups. Because all of the study subjects develop (or are exposed to, for preventive interventions) the disease of interest, and animals can be selected to be similar to one another, challenge trials are efficient in terms of overall sample size. In general, the criteria for validity in this type of trial are the same as for RCTs. However, because sample size may be small, it is not uncommon for blocking or stratification to be used to ensure equal distribution of known confounding variables among groups. Nonetheless, a random component to intervention allocation should also be used (i.e. random allocation within blocks or strata). It is also common to use attenuated strains of the infectious disease agent in the challenge. For instance, bacterial strains may be made resistant to specific antibiotics to aid in laboratory identification and quantification. Modified disease strains, and the artificial disease challenge, may not be fully representative of natural disease exposure. The housing conditions, restricted populations and/or artificial challenge limit the external validity of the results Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014)

7 Randomized Controlled Trials and Challenge Trials J. M. Sargeant et al. Therefore, while challenge trials may provide good preliminary evidence of intervention efficacy, they are not as strong an evidentiary study design as a RCT with natural disease exposure. Deliberate disease induction trials of non-infectious diseases may also be used to evaluate interventions in the species of interest in veterinary medicine (Sargeant et al., 2010a,b ). For example, dairy cattle may be fed specific diets to induce metabolic conditions such as subacute rumen acidosis or ketosis and then randomly assigned to intervention groups. While this approach does allow the investigation of intervention efficacy in the species of interest, the artificially produced disease may not be entirely representative of the disease as it naturally occurs. Quality of Reporting Results from Randomized Controlled Trials In recent years, the quality of reporting of RCTs has been subject to much debate, particularly as the safety and effectiveness of very new and expensive pharmacologic agents continue to be contested. It is difficult to determine how a study was actually designed and conducted based on reading a study report, particularly if no mention is made in the study report of features related to bias reduction. It is possible that components essential to bias reduction were adequately implemented, but not explicitly reported in the subsequent publication. To evaluate this possibility, investigators in the human health field contacted authors of RCTs to determine whether specific trial features that were not reported were actually included in the RCT (Hills and Armitage, 1979; Devereaux et al., 2004). The investigators reported that trial features, such as the method of random allocation sequence generation, allocation concealment and blinding, were frequently used in RCTs that did not explicitly report them. Nonetheless, the reader of a research report has only the published information with which to judge the internal and external validity of the trial. CONSORT, which stands for Consolidated Standards of Reporting Trials, encompasses various initiatives developed by the CONSORT Group to alleviate the problems arising from inadequate reporting of RCTs. The main product of CONSORT is the CONSORT Statement, which is an evidence-based, minimum set of recommendations for reporting RCTs (Begg et al., 1996; Moher et al., 2001; Schulz et al., 2010). It offers a standard way for authors to prepare reports of trial findings, facilitates complete and transparent reporting, and aids readers of the trial in critical appraisal and interpretation. All CONSORT documents, including links to publications, are available free of charge on the CONSORT website (CONSORT, 2013). The most recently updated CONSORT Statement comprises a 25-item checklist of items (increased from 22 items in the 2001 version) that should be reported in a RCT and a flow diagram describing the number of participants at each stage of the trial (Schulz et al., 2010). The checklist items focus on reporting how the trial was designed, analysed and interpreted. The flow diagram displays the progress of all participants through the trial. A companion document has been published, which provides an explanation of each item in the checklist and an example of appropriate reporting of that item from the published literature (Moher et al., 2010). The CONSORT Statement has been translated into several languages and is endorsed by several hundred medical journals. The impact of CONSORT has been demonstrated in a number of ways including evidence of improvements in quality of trial reports in medical journals, evidence of website use and journal citations and an influence on related reporting guidelines and other indicators (CONSORT, 2013). An extension of the CONSORT statement for clustered trials, where groups rather than individuals are assigned to interventions, has been created (Campbell et al., 2004). There are several potential differences between human health care and veterinary medicine that may impact the use of the CONSORT statement for veterinary trials. These include nuances in the use of the phrase study participants (in veterinary trials, there are both owners/managers who make decisions on enrolment and animals who receive the interventions), the use of challenge trials, and the common housing and treatment of food animals, and some instances also companion animals, in groups. Nonetheless, CON- SORT is likely appropriate for use in small animal trials with minor modification. A modification of the CONSORT statement, the reporting guidelines for randomized controlled trials in livestock and food safety (REFLECT) statement, has been published to provide guidelines for reporting clinical trials in livestock studies addressing animal health and production or food safety outcomes (Table 1) (O Connor et al., 2010). An explanation and elaboration document for the RELECT statement also is available (Sargeant et al., 2010a). The CONSORT and REFLECT statement checklists are a valuable aid to those writing, reviewing or reading a clinical trial report. Summary Randomized controlled trials are the preferred study design to evaluate interventions under real-world conditions. In animal agriculture and veterinary medicine, trials with deliberate disease induction can be conducted in the target species (challenge trials), although this design does not provide as high a level of evidence as an RCT with a natural disease exposure. Important design features in RCTs include how the study population is selected, random allocation to intervention groups, allocation concealment, Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014) 18 27

8 J. M. Sargeant et al. Randomized Controlled Trials and Challenge Trials Table 1. Checklist of Items for the REFLECT statement: Reporting guidelines for randomized control trials in livestock and food safety (O Connor et al., 2010; Sargeant et al., 2010a) Paper Section And topic Item Descriptor of REFLECT statement item Reported on Page no. Title & abstract 1 How study units were allocated to interventions (e.g. random allocation, randomized or randomly assigned ). Clearly state whether the outcome was the result of natural exposure or was the result of a deliberate agent challenge Introduction background 2 Scientific background and explanation of rationale Methods participants 3 Eligibility criteria for owner/managers and study units at each level of the organizational structure, and the settings and locations where the data were collected Interventions 4 Precise details of the interventions intended for each group, the level at which the intervention was allocated, and how and when interventions were actually administered 4b Precise details of the agent and the challenge model, if a challenge study design was used Objectives 5 Specific objectives and hypotheses. Clearly state primary and secondary objectives (if applicable) Outcomes 6 Clearly defined primary and secondary outcome measures and the levels at which they were measured, and, when applicable, any methods used to enhance the quality of measurements (e.g. multiple observations, training of assessors) Sample size 7 How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules. Sample size considerations should include sample size determinations at each level of the organizational structure and the assumptions used to account for any nonindependence among groups or individuals within a group Randomization sequence generation Randomization allocation concealment Randomization implementation 8 Method used to generate the random allocation sequence at the relevant level of the organizational structure, including details of any restrictions (e.g. blocking, stratification) 9 Method used to implement the random allocation sequence at the relevant level of the organizational structure (e.g. numbered containers), clarifying whether the sequence was concealed until interventions were assigned. 10 Who generated the allocation sequence, who enrolled study units and who assigned study units to their groups at the relevant level of the organizational structure Blinding (masking) 11 Whether or not those administering the interventions, caregivers and those assessing the outcomes were blinded to group assignment. If done, how the success of blinding was evaluated. Provide justification for not using blinding if it was not used Statistical methods 12 Statistical methods used to compare groups for all outcome(s); clearly state the level of statistical analysis and methods used to account for the organizational structure, where applicable; methods for additional analyses, such as subgroup analyses and adjusted analyses Results study flow 13 Flow of study units through each stage for each level of the organization structure of the study (a diagram is strongly recommended). Specifically, for each group, report the numbers of study units randomly assigned, receiving intended treatment, completing the study protocol and analysed for the primary outcome. Describe protocol deviations from study as planned, together with reasons Recruitment 14 Dates defining the periods of recruitment and follow-up Baseline data 15 Baseline demographic and clinical characteristics of each group, explicitly providing information for each relevant level of the organizational structure. Data should be reported in such a way that secondary analysis, such as risk assessment, is possible Numbers analysed 16 Number of study units (denominator) in each group included in each analysis and whether the analysis was by intention-to-treat. State the results in absolute numbers when feasible (e.g. 10/20, not 50%) Outcomes and estimation 17 For each primary and secondary outcomes, a summary of results for each group, accounting for the hierarchy, and the estimated effect size and its precision (e.g. 95% confidence interval) Ancillary analyses 18 Address multiplicity by reporting any other analyses performed, including subgroup analyses and adjusted analyses, indicating those pre-specified and those exploratory Adverse events 19 All important adverse events or side effects in each intervention group Discussion interpretation 20 Interpretation of the results, taking into account study hypotheses, sources of potential bias or imprecision, and the dangers associated with multiplicity of analyses and outcomes. Where relevant, a discussion of herd immunity should be included. If applicable, a discussion of the relevance of the disease challenge should be included Generalizability 21 Generalizability (external validity) of the trial findings Overall evidence 22 General interpretation of the results in the context of current evidence Texts in bold are modifications from the original CONSORT Description (Available at: Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014)

9 Randomized Controlled Trials and Challenge Trials J. M. Sargeant et al. definition of the outcomes to evaluating the outcomes, blinding and how withdrawals from the study are managed. Guidelines for reporting important features of RCTs have been published and are useful tools for writing, reviewing and reading reports of RCTs. Acknowledgements and funding support The authors thank Annette Wilkins for assistance with this manuscript. Partial funding was obtained from the Laboratory for Foodborne Zoonoses, Public Health Agency of Canada and the Canadian Institutes of Health Research (CIHR) Institute of Population and Public Health/Public Health Agency of Canada Applied Public Health Chair. References Altman, D. G., 1980: Statistics and ethics in medical research: III How large a sample? BMJ 281, Altman, D. G., 1985: Comparability of Randomised Groups. J. R. Stat. Soc. Series D. (The Statist.) 34, Altman, D. G., 1991: Randomisation: essential for reducing bias. BMJ 302, Altman, D. G., and J. M. Bland, 1995: Absence of evidence is not evidence of absence. BMJ 311, 485. Begg, C. B., 1990: Suspended judgment. Significance tests of covariate imbalance in clinical trials. Control. Clin. Trials 11, Begg, C., M. Cho, S. Eastwood, R. Horton, D. Moher, I. Olkin, R. Pitkin, D. Rennie, K. F. Schulz, D. Simel, and D. F. Stroup, 1996: Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 276, Campbell, M. K., D. R. Elbourne, and D. G. Altman, 2004: CONSORT statement: extension to cluster randomised trials. BMJ 328, Clancy, M. J., 2002: Overview of research designs. Emerg. Med. J. 19, Cohen, S., W. J. Doyle, D. P. Skoner, B. S. Rabin, and J. M. Gwaltney Jr, 1997: Social ties and susceptibility to the common cold. JAMA 277, CONSORT, 2010: CONSORT statement. Available at: consort-statement.org/ (Accessed on 09 April 2013). Devereaux, P. J., B. J. Manns, W. A. Ghali, M. Bhandari, V. M. Montori, H. J. Schunemann, A. X. Garg, J. W. Busse, W. A. Heels-Ansdell, B. J. Manns, and G. H. Guyatt, 2001: Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA 285, Devereaux, P. J., P. T. Choi, S. El-Dika, M. Bhandari, V. M. Montori, H. J. Schunemann, A. X. Garg, J. W. Busse, D. Heels-Ansdell, W. A. Ghali, B. J. Manns, and G. H. Guyatt, 2004: An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J. Clin. Epidemiol. 57, Devereaux, P. J., M. Bhandari, V. M. Montori, B. J. Manns, W. A. Ghali and G. H. Guyatt 2005: Double blind, you are the weakest link goodbye!. Equine Vet. J. 37, Dohoo, I., W. Martin, and H. Stryhn, 2010: Veterinary Epidemiologic Research. 2nd edn. VER Inc., Charlettetown, PEI. Freedman, B., 1987: Equipoise and the ethics of clinical research. N. Engl. J. Med. 317, Giuffrida, M. A., K. A. Agnello, and D. C. Brown, 2012: Blinding terminology used in reports of randomized controlled trials involving dogs and cats. J. Am. Vet. Med. Assoc. 241, Glasser, S. P., and G. Howard, 2006: Clinical trial design issues: at least 10 things you should look for in clinical trials. J. Clin. Pharmacol. 46, Hills, M., and P. Armitage, 1979: The two-period cross-over clinical trial. Br. J. Clin. Pharmacol. 8, Johnson, P. D., and D. G. Besselsen, 2002: Experimental design and statistics in biomedical research practical aspects of experimental design in animal research. ILAR J. 43, Juni, P., D. G. Altman, and M. Egger, 2001: Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 323, Kingsley, G. H., B. Khoshaba, C. M. Smith, E. H. Choy, and D. L. Scott, 2005: Are clinical trials in rheumatoid arthritis generalizable to routine practice? A re-evaluation of trial entry criteria. Rheumatology (Oxford) 44, Kjaergard, L. L., J. Villumsen, and C. Gluud, 2001: Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann. Intern. Med. 135, Lachin, J. L., 2000: Statistical considerations in the intentto-treat principle. Control. Clin. Trials 21, Lilford, R. J., and J. Jackson, 1995: Equipoise and the ethics of randomization. J. R. Soc. Med. 88, Lund, E. M., K. M. James, and J. D. Neaton, 1994: Clinical trial design: veterinary perspectives. J. Vet. Intern. Med. 8, Moher, D., K. F. Schulz, and D. Altman, 2001: The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 285, Moher, D., S. Hopewll, K. F. Schulz, V. Montori, P. C. Gotzsche, P. J. Devereaux, D. Elbourne, M. Egger, and D. G. Altman, 2010: CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials. J. Clin. Epidemiol. 63, e1 e37. Morgan, K. L., and D. B. Rubin, 2012: Rerandomization to improve covariate balance in experiments. Ann. Stats. 40, O Connor, A. M., J. M. Sargeant, I. A. Gardner, J. S. Dickson, M. E. Torrence, C. E. Dewey, I. R. Dohoo, R. B. Evans, J. T. Gray, M. Greiner, G. Keefe, S. L. Lefebvre, P. S. Morley, A. Ramirez, W. Sischo, D. R. Smith, K. Snedeker, J. Sofos, M. P Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014) 18 27

10 J. M. Sargeant et al. Randomized Controlled Trials and Challenge Trials Ward, and R. Wills, 2010: The REFLECT Statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety by modifying the CONSORT statement. Zoonoses Public Health 57, Petrie, A., and P. Watson, 2013: Statistics for Veterinary and Animal Science. 3rd edn. Wiley-Blackwell, Oxford, UK. Piantadosi, S., 2005: Clinical Trials: A Methodological Perspective. 2nd edn. John Wiley and Sons, Inc., Hoboken, NJ. Pocock, S. J., 1998: Clinical Trials: A Practical Approach. John Wiley and Sons Ltd, Midsomer Norton, Somerset. Sargeant, J. M., M. E. Torrence, A. Rajic, A. M. O Connor, and J. Williams, 2006: Methodological quality assessment of review articles evaluating interventions to improve microbial food safety. Foodborne Pathog. Dis. 3, Sargeant, J. M., R. Elgie, J. Valcour, J. Saint-Onge, A. Thompson, P. Marcynuk, and K. Snedeker, 2009a: Methodological quality and completeness of reporting in clinical trials conducted in livestock species. Prev. Vet. Med. 91, Sargeant, J. M., J. Saint-Onge, J. Valcour, A. Thompson, R. Elgie, K. Snedeker, and P. Marcynuk, 2009b: Quality of reporting in clinical trials of preharvest food safety interventions and associations with treatment effect. Foodborne Pathog. Dis. 6, Sargeant, J. M., A. M. O Connor, I. A. Gardner, J. S. Dickson, M. E. Torrence, I. R. Dohoo, S. L. Lefebvre, P. S. Morley, A. Ramirez, and K. Snedeker, 2010a: The REFLECT statement: reporting guidelines for randomized controlled trials in livestock and food safety: explanation and elaboration. Zoonoses Public Health 57, Sargeant, J. M., A. Thompson, J. Valcour, R. Elgie, P. Saint- Onge, P. Marcynuk, and K. Snedeker, 2010b: Quality of reporting of clinical trials of dogs and cats and associations with treatment effects. J. Vet. Intern. Med. 24, Schulz, K. F., 1995: Subverting randomization in controlled trials. JAMA 274, Schulz, K. F., and D. A. Grimes, 2002: Generation of allocation sequences in randomised trials: chance, not choice. Lancet 359, Schulz, K. F., and D. A. Grimes, 2005: Sample size calculations in randomised trials: mandatory and mystical. Lancet 365, Schulz, K. F., D. G. Alman, D. Moher, and for the CONSORT group, 2010: CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomized trials. J. Clin. Epidemiol. 63, Seder, R. A., L. J. Chang, M. E. Enama, K. L. Zephir, U. N Sarwar, I. J. Gordon, L. A. Holman, E. R. James, P. F. Billingsley, A. Gunasekera, A. Richman, S. Chakravarty, A. Manoj, S. Velmurugan, M. Li, A. J. Ruben, T. Li, A. G. Eappen, R. E. Stafford, S. H. Plummer, C. S. Hendel, L. Novik, P. J. Costner, F. H. Mendoza, J. G. Saunders, M. C. Nason, J. H. Richardson, J. Murphy, S. A. Davidson, T. L. Richie, M. Sedegah, A. Sutamihardja, G. A. Fahle, K. E. Lyke, M. B. Laurens, M. Roederer, K. Tewari, J. E. Epstein, B. K. Sim, J. E. Ledgerwood, B. S. Graham, and S. L. Hoffman, and the VRC 312 Study Team. 2013: Protection against malaria by intravenous immunization with the nonreplicating sporozoite vaccine. Science 341: Suarez-Almazor, M. E., 2002: In quest of the holy grail: efficacy versus effectiveness in rheumatoid arthritis. J. Rheumatol. 29, Weijer, C., S. H. Shapiro, K. C. Glass, and M. W. Enkin, 2000: For and against - Clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial. Br. Med. J. 321, Wolfe, F., 2005: Why results of clinical trials and observational studies of anti-tnf therapy differ: methodological and interpretive issues. Ann. Rheum. Dis. 63, ii13 ii Blackwell Verlag GmbH Zoonoses and Public Health 61 (suppl. 1) (2014)