Bios 6648: Design & conduct of clinical research Section 1 - Specifying the study setting and objectives 1. Specifying the study setting and objectives Where will we end up?: (a) The treatment indication trial The scientific method 1.1 Defining the study population 1.2 Defining the study (a) Defining the interventions: What is the treatment? (b) Phase I-IV clinical trials (c) Statistical structure of the outcome space (d) 1-sided versus 2-sided 1.3 Case study (Rocket-AF trial) Bios 6648- pg 1
Where will we end up? (a) The treatment indication The FDA approves a new drug for a specific indication." The study setting/objectives must inform the intended indication (or vice versa). Components of an indication include: Treatment Disease or condition Patient population Therapeutic objective See my written notes for section 1 for further discussion Example: Exercise 1. Bios 6648- pg 2
(a) Specifying the treatment indication Notes (See also Section 1 written notes): Disease: Diseases can be defined by symptoms (COPD), causal agents (meningococcal meningitis), or treatment (MDR tuberculosis). The definition of a disease often changes with factors unrelated to the disease (e.g., new treatments or a new categorization of symptoms) Population of patients: The target patient population is similarly dynamic. Example: better diagnostic tools may detect cancer at earlier stages or make it easier to detect later-stage disease. Bios 6648- pg 3
(a) Specifying the treatment indication (con t) Treatment: The way in which a treatment is delivered may change. New formulations may allow oral instead of IV delivery, which might extend the use to other populations or other forms of the disease. Ancillary treatments (standard of care) is always changing. Desired outcome Primary clinical outcomes versus surrogate outcomes (Vioxx; mammography; colon cancer screening) Unanticipated or anticipated beneficial (sildenafil citrate) or harmful (rosiglitazone) effects. Bios 6648- pg 4
Where will we end up? (b) How will we summarize trial results? Inference upon trial (i) What is statistical inference? (ii) Four required elements (iii) Properties of estimators (iv) Interpretation of interval estimates Bios 6648- pg 5
Inference upon trial (i) Why review statistical foundations? We are discussing the study setting and objectives. As a scientific experiment, the results of a clinical trial are used to rule out (or rule in) hypotheses about treatment effects. The standards for rejecting (or accepting hypotheses) are based on statistical criteria. We need a basic understanding of statistical foundations in order to discuss the scientific setting and the role of uncertainty. Bios 6648- pg 6
Inference upon trial (i) What is statistical inference? Underlying Population θ denotes unknown center Inference about θ Sample Statistics Sample summary measure: θ^ Bios 6648- pg 7
Inference upon trial (ii) Four required elements of statistical inference We use ˆθ (observed trial result) to estimate the true underlying value θ 1. Point estimate: ˆθ is the best" estimate of θ. 2. Interval estimate: Values of θ that are consistent with the trial results. 3. Expression of uncertainty (p-value): To what degree is a particular hypothesis (the null" hypothesis) consistent with the observed trial results? 4. Decision: Based on the above measures, what decision should be reached about the use of a new therapy? Bios 6648- pg 8
Inference upon trial (ii) Four required elements of statistical inference Example: CHEST trial: Ghofrani,et.al. NEJM (2013); 369: 319-29: Riociguat for the Treatment of Chronic Thromboembolic Pulmonary Hypertension. Trial: Randomized double-blind placebo controlled trial in patients with inoperable CTEPH. Results:...By week 16, the 6-minute walk distance ( had increased by a mean of 39 m in the riociguat group, as compared with a mean decrease of 6 m in the placebo group (least-squares mean difference, 46 m; 95% confidence interval [CI], 25 to 67; P<0.001)." Bios 6648- pg 9
Inference upon trial (ii) Four required elements of statistical inference Example (con t) Setting: Primary endpoint: 16-week change in 6-minute walk distance (6MWD) Summary of outcome: mean change denoted by θ 1 (riociguat) and θ 0 (placebo) Measure of treatment effect: difference in 6mwd: θ = θ 1 θ 0. Observed effect: Inference: Observed summary outcomes: ˆθ1 = 39m; ˆθ 0 =-6m Observed treatment effect: ˆθ = ˆθ1 ˆθ 0 = 46m (by least-squares regression). Point estimate: 46m Interval estimate: 25m to 67m Uncertainty: p < 0.001 Decision: to use or not to use? Bios 6648- pg 10
Inference upon trial (iii) Properties of estimators Desirable properties of: Point estimate Unbiased and consistent: the long-run average of ˆθ is very close to θ Small variance (Uniform Minimum Variance Unbiased Estimator) Interval estimate Correct coverage probability (e.g., 95% of all 95% confidence interval include θ). As narrow as possible while maintaining the correct coverage probability. P-value Decision Correct size Decision criteria maintain the appropriate type I statistical error rate. Bios 6648- pg 11
(iv) Interpretation of an interval estimator What is the interpretation of a 95% confidence interval? Bios 6648- pg 12
(iv) Interpretation of an interval estimator What s wrong with the following picture? θ L θ^obs θ u Bios 6648- pg 13
(iv) Interpretation of an interval estimator Proper depiction of an interval estimator: θ L θ^obs θ u If θ < θ L or if θ > θ U then the observed result ˆθ = ˆθ obs would be unusual. Bios 6648- pg 14
(iv) Interpretation of an interval estimator The scientific objective is to identify the hypotheses that have (or have not) been ruled out by the trial s results. Let U(θ ref ˆθ obs ) represent a statistical measure of the consistency between the trial s result ˆθ = ˆθ obs and the hypothesis θ = θ ref. By usual frequentist criteria this measure is equal to the smaller of: P(ˆθ ˆθ obs θ = θ ref ) P(ˆθ ˆθ obs θ = θ ref ) Reject the hypothesis θ = θ ref when U(θ ref ˆθ obs ) is small; specifically when: U(θ ref ˆθ obs ) < α 2 Bios 6648- pg 15
(iv) Interpretation of an interval estimator We seek the values of θ that cannot be rejected; specifically: Find the set of θ ref such that U(θ ref ˆθ obs ) α/2 using α = 0.05. If ˆθ N (θ, V ) then the non-rejection region is given by [θ L, θ U ] where θ L = ˆθ obs 1.96 V θ U = ˆθ obs + 1.96 V (See graph above) Bios 6648- pg 16
(iv) Interpretation of an interval estimator What assumptions are necessary to assure that the above interval has the correct properties? Bios 6648- pg 17
(iv) Interpretation of an interval estimator What assumptions are necessary to assure that the above interval has the correct properties? No assumption about the distribution of the individual data elements is necessary. The estimated effect ˆθ must follow a Normal distribution: Central limit theorem assures θ is Normally distributed as long as the sample size is not too small. For interpretation as a non-rejection region, we must know the mean-variance relationship. Bios 6648- pg 18
A note on mean-variance relationships: θ L θ^obs θ u With a mean-variance relationship the confidence interval can have the correct coverage probability, but may not be a non-rejection region. Interventions often change both the mean and the variance. We will return to this issue in chapter 7. Bios 6648- pg 19
(iv) Interpretation of an interval estimator In summary... Scientific decisions must consider the magnitude of the effect (point estimate) and the hypotheses that remain viable based on the trial s results (the interval estimate). I will appeal to these considerations as I describe the scientific setting. Bios 6648- pg 20
(iv) Interpretation of an interval estimator Example (CHEST trial CI: 25m to 67m): 16-week improvement in 6MWD with riociguat was 46m better than the improvement with placebo. The variability in the results allows us to rule out (with 95% confidence) improvements that are more than 67 meters greater with riociguat or less than 25 meters greater with riociguat. Note: the following is incorrect: There is a 95% chance (or probability) that the true underlying difference is between 25 meters and 67 meters. Note: the following is sometimes accepted, but misleading: We are 95% confident that the true underlying difference is between 25 meters and 67 meters. Bios 6648- pg 21
The scientific method The scientific method is an iterative process of posing and evaluating hypotheses using carefully designed experiments. A clinical trial is an experiment and should be built on carefully-framed hypotheses: What is the treatment? What is θ (the measure of treatment effect)? What are important differences? What differences support recommending use of a new treatment? The trial must be designed to be informative relative to the hypotheses (the scientist game) Upon the range of viable hypotheses that remain is determined by the experimental results Bios 6648- pg 22
The scientific method: The scientist game Assignment: Try the scientist game: htpp://www.emersonstatistics.com/scientistgame Careful consideration of what you want to know upon trial is essential. The obvious choice is often not the best choice. The scientist game is illustrative of the scientific importance of all aspects of the design including: Specification of the treatment Selection and definition of the outcome(s) Choice of control group Definition of design hypotheses Statistical standard for evidence Choice of sample size What was your choice? Bios 6648- pg 23
The scientific method: The scientist game The scientist game illustrates how it is possible to select an entirely uninformative experiment to test a hypothesis. An explanation of game and its application to experimental design and clinical trials is given on the above website. The possible choices for your experiment and whether or not they are informative (indicated by a +) are: Possible Experiments Hypotheses A b b B a B a A letter, size, script + - - - - - - - letter, size + - - - - - - + letter, script + - - - - - + - size, script + - - - - + - - letter + - - - + - + + size + - - + - + - + script + - + - - + + - all coincidence + + + + + + + + The best choices are B, a, or A because they will reduce the number of potential hypotheses from 8 to 4. Bios 6648- pg 24
1.1 Defining the study population Defining the study population Study population: Subjects in the study are representative of the study population. Study results are intended to estimate effects in the study population Study population is defined by: Target" population: Eligibility criteria: Characteristics of individuals who will be invited to participate in the trial Exclusion criteria: Individuals who will be excluded even if they meet the eligibility criteria (usually for safety reasons). The population in which the treatment/intervention will be used. The treatment indication is for the target population. Bios 6648- pg 25
1.1 Defining the study population Defining the study population Reasons why the study population might differ from target: (a) Ethical: Cannot force participation Sometimes you must exclude vulnerable populations"; e.g.: (b) Practical Prisoners Pregnant women Children These populations might be studied after efficacy is established in the study population. E.g., restriction to major regional centers ** Beware of excessive restriction (c) Scientific: Compliant (exclude subjects who are likely to be non-compliant). Restrict to subjects who will be able to complete follow-up visits. Exclude co-morbid conditions Require basic level of health(?) Bios 6648- pg 26
1.1 Defining the study population Defining the study population Reasons why the study population might differ from target: (d) Special considerations in placebo-controlled trials: Restrict to patients who can (ethically) receive placebo (Sometimes this restriction is used even if the new drug would be offered to the patients who are excluded.) But remember: If it is ethical to conduct a placebo controlled trial, then it is unethical not to... (Lloyd Fisher)" Bios 6648- pg 27
1.1 Defining the study population Defining the study population Selected examples for illustration OCEANS trial (Carol AghajanianC., et.al. J Clin Oncol (2012); 30:2039-2045: A Randomized, Double-Blind, Placebo-Controlled Phase III Trial of Chemotherapy With or Without Bevacizumab in Patients With Platinum-Sensitive Recurrent Epithelial Ovarian, Primary Peritoneal, or Fallopian Tube Cancer Problems with gastro-intestinal perforation in early trials Restricting eligibility to platinum-sensitive recurrent cancers appears to have eliminated those problems. See Eligibility Criteria in paper (bottom of first column page 2040). CHEST trial (see first exercise) Review JK notes on chapter 12. Bios 6648- pg 28
End: Section 1.1 Bios 6648- pg 29