Study Designs & Methodologies (w/ /M Measures of Association) What is the most useful and appropriate study design? Larry W. Segars, PharmD, DrPH, FCCP, FACE, BCPS, RPh Associate Dean College of Biosciences Associate Professor of Pharmacology College of Medicine None Conflicts of Interest What is the most useful and appropriate study design? Learning Objectives Study Designs Describe, differentiate, and be able to recognize the structure, methods and designs of the common types of observational & interventional studies. Delineate, differentiate, and be able to recognize the common types of perspectives utilized by observational and interventional studies. Describe the purpose of Randomization and how one can confirm its success, and also describe and differentiate between the 3 common forms of randomization. Delineate and differentiate the common methods of handling lost to follow up/drop outs associated with studies. Delineate, calculate, and interpret common terms associated with comparing groups based on measures of association for disease (or outcome) occurrence. Increas ing Evidence Interventional Pre-Clinical * Phase 1 Phase 2 Phase 3 Phase 4 Observational Cases * (Reports/Series) Ecological * Cross-Sectional Case-ControlControl Cohort Increas ing Evidence 1
2 KEY QUESTIONS to Selecting the Correct Study Design 1. Is researcher FORCING GROUP ALLOCATION? a. Yes Interventional (sample size/duration/focus) b. No Observational ( outcome / exposure ) ) Case Control Control Study Disease/Outcome NO Exposed Non-Exposed Population Exposed Disease/Outcome YES Non-Exposed CASE CONTROL CONTROL design useful when studying a rare disease Commonly generates the Odds Ratio (OR) as measure of association 2 KEY QUESTIONS to Selecting the Correct Study Design 1. Is researcher FORCING GROUP ALLOCATION? a. Yes Interventional (sample size/duration/focus) b. No Observational ( outcome / exposure ) ) 2. For Observational studies, how were groups ORGANIZED? Considering OUTCOME vs. EXPOSURE, Look for what is KNOWN vs. being SOUGHT a. By Outcome Status Case Control/ Control/Nested Case ControlControl b. By Exposure Status Cohort c. By Common Factor Cross Sectional or Cohort Population Cohort Study Exposure -NO Exposure - YES Diseased/Outcome Non-Diseased/ No Outcome Diseased/Outcome Non-Diseased/ No Outcome COHORT design useful when studying a rare exposure Commonly generates the Risk Ratio (RR) as measure of association Studies can be : PROSPECTIVE outcome is NOT yet known at start of study RETROSPECTIVE outcome IS already known at start of study AMBIDIRECTIONAL (both perspectives in same study) First looking retrospectively, then looking prospectively for additional outcomes Observational Studies: Cohort Study (a.k.a. Longitudinal & Follow up Study) DEFINITION vs. STUDY METHODOLOGY 1. StudyMethodology: Group organization based on EXPOSURE status (presence/absence) 2. Definition: A group with something in common Birth / Inception / Exposure / etc 2
NHANES www.cdc.gov/nchs/nhanes.htm www.channing.harvard.edu/nhs or www.nhs3.org www.framinghamheartstudy.org Observational Studies Cont d: Nested Case ControlControl A Case ControlControl study derived out of a completed Cohort study The diseased study subjects determined from Cohort study becoming Cases of a different, subsequent study»includes new subjects without disease, called Controls, who may or may not have necessarily come from the original Cohort study Interventional Studies: Pre Clinical An investigation conducted in non humans or in vitro Phase 1 (new treatment (medication/device) Small N (20 80), healthy volunteers, first time use in humans to assess short term safety, toxicity and pharmacokinetics (A D M E) Short duration (e.g., a couple of days to a few weeks) Phase 2 (new treatment (medication/device), indication or population) Larger N (100 300), commonly utilize patients with condition of interest, used to expand on purpose of Phase 1 study (short term safety) but also to begin assessing efficacy in a diseased population Short to Medium duration (e.g., a few weeks to a few months) Likely to have narrow inclusion criteria Observational Studies Cont d: Cross Sectional Study (a.k.a.; Prevalence study) Purpose: Examines relationships between disease AND exposure (simultaneously) among individuals in a defined study population at a point in time A snap shot in time (across the entire study population) Patient selection: inclusion in large study population Think of a cross sectional study when the information gathered represents what is going on with disease AND exposures (at the same time) a cross an entire (large) study population Interventional Studies Cont d: Phase 3 (new treatment (medication/device), indication or population) Larger N (500 3,000), used in patients with the condition of interest to continue determination of short to intermediate safety, with primary purpose to assess efficacy (broader inclusion criteria; more clinically relevant (yet still possible to limit generalizability to full population) Longer duration (e.g., a few to numerous months; may up to 1 3 years)» Superiority & Non Inferiority perspectives utilized Phase 4 (post marketing/post FDA approval) Longer term effects (risks & benefits) in a large population of diseased patients (expanded use population (age, ethnic)) Registries, Survey s (or via FDA s MedWatch program) 3
Example Board Questions Researchers randomly select 12,980 patients in the San Antonio Military Multi Service Market who were being treated with a statin medication and 45,997 patients in this same Market who were not treated with a statin. The researchers wish to investigate a possible association between treatment with a stain and the incidenceof various musculoskeletal and neoplastic diseases between statin users and non users. Which of the following study types would be most appropriate? A. Cohort study B. Phase 3 clinical trial C. Case Control study D. Case Series E. Phase 4 clinical trial Researchers randomly select 100 patients with newly diagnosed oropharyngeal cancer and 200 patients without this cancer, and wish to investigate a possible association between HPV infection and oropharyngeal cancer. Which of the following study types would be most appropriate? A. Cohort study B. Case Control study C. Phase 2 study D. Case Series E. Phase 3 study Researchers randomly select 12,980 patients in the San Antonio Military Multi Service Market who were being treated with a statin medication and 45,997 patients in this same Market who were not treated with a statin. The researchers wish to investigate a possible association between treatment with a stain and the incidenceof various musculoskeletal and neoplastic diseases between statin users and non users. Which of the following study types would be most appropriate? A. Cohort study B. Phase 3 clinical trial C. Case Control study D. Case Series E. Phase 4 clinical trial Am J Med Sci 2013;345:343 48. 48. Researchers randomly select 100 patients with newly diagnosed oropharyngeal cancer and 200 patients without this cancer, and wish to investigate a possible association between HPV infection and oropharyngeal cancer. Which of the following study types would be most appropriate? A. Cohort study B. Case Control study C. Phase 2 study D. Case Series E. Phase 3 study NEJM 2007;356:1944 56. 56. Researchers desire to evaluate the safety and efficacy of treating gout with allopurinol versus febuxostat. Researchers plan to randomize patients with confirmed gout to treatment with either febuxostat 10 mg/d or allopurinol 100 mg/d initially for 12 days, and then permit dose titration, as clinically deemed appropriate, up to 40 mg/d and 200 mg/d, respectively, over a total of 8 weeks. Which of the following study designs would be most appropriate? A. Case Control B. Phase 1 D. Cross sectional E. Phase 3 4
Researchers desire to evaluate the safety and efficacy of treating gout with allopurinol versus febuxostat. Researchers plan to randomize patients with confirmed gout to treatment with either febuxostat 10 mg/d or allopurinol 100 mg/d initially for 12 days, and then permit dose titration, as clinically deemed appropriate, up to 40 mg/d and 200 mg/d, respectively, over a total of 8 weeks. Which of the following study designs would be most appropriate? A. Case Control B. Phase 1 D. Cross sectional E. Phase 3 J Clin Rheumatol 2011;17:S13 S18. S18. Researchers desired to compare the efficacy and tolerability of a fixed combination of timolol maleate 0.5%/brimonidine tartrate 0.2% versus a fixed combination of timolol maleate 0.5%/dorzolamide 2% over 8 weeks in patients with elevated IOP. Researchers utilized an open labeled, randomized format in 5 centers across 2 countries. Which of the following study designs would be most appropriate? A. Phase 2 B. Case Control D. Phase 4 E. Cross sectional Researchers utilized the National Ambulatory Medical Care Survey to evaluate the use of HMG CoA reductase inhibitors in U.S. diabetics. The researchers used survey data from 2002 2004 to evaluate differences in patient characteristics between those U.S. diabetics treated with a statin medication versus those not treated with these medications, while controlling for potential confounders. Which of the following study designs most appropriately represents the utilized study design? A. Case Control B. Phase 3 D. Cross sectional E. Phase 2 Researchers desired to compare the efficacy and tolerability of a fixed combination of timolol maleate 0.5%/brimonidine tartrate 0.2% versus a fixed combination of timolol maleate 0.5%/dorzolamide 2% over 8 weeks in patients with elevated IOP. Researchers utilized an open labeled, randomized format in 5 centers across 2 countries. Which of the following study designs would be most appropriate? A. Phase 2 B. Case Control D. Phase 4 E. Cross sectional J Glaucoma 2008;17:674 9. Researchers utilized the National Ambulatory Medical Care Survey to evaluate the use of HMG CoA reductase inhibitors in U.S. diabetics. The researchers used survey data from 2002 2004 to evaluate differences in patient characteristics between those U.S. diabetics treated with a statin medication versus those not treated with these medications, while controlling for potential confounders. Which of the following study designs most appropriately represents the utilized study design? A. Case Control B. Phase 3 D. Cross sectional E. Phase 2 Clin Therapeutics 2008;30:2159 66. Group Allocation Procedures: Non Random e.g., Alternating subjects based on fixed criteria (odd/even days; every other patient; etc, etc ) Random (most common utilized) Subjects have equal probability of being assigned to each of the predefined intervention groups Random number table or Computer program random number generator Randomization Purpose: to make all groups as equal as possible; based on known (and potentially unknown) confounders Attempts to reduces systematic difference (bias) between groups which could impact results/outcomes Table 1 customarily used to show group baseline characteristics Equal ness of groups not guaranteed! 5
Common Types of Randomization Processes: Blocked (Permuted Blocked) Ensures balance within each intervention group Stratified Subjects separated and balanced by known confounding variables Examples: Gender, Age group, Disease severity, etc Can also pre select levels to be balanced within each prognostic factor (confounder)»very useful when confounders are known and sample size is not very large Managing Drop Outs/Lost Outs/Lost to Follow Up: Include Them Intent to Treat (most conservative) Use last known assessment for remaining (missed) study evaluations, OR Exclude Them Per Protocol (a.k.a., Efficacy Analysis) Must meet pre established level of compliance/participation to be included Formats of Interventional Studies: Parallel Subjects exclusively managed in originally randomized group No switching of groups Study Strengths/Weaknesses (Broad) Ability to prove Causation Validity (Accuracy/Precision/Consistency) Measurements / Assessments / Classifications Applicability (External Validity) Inclusion / Exclusion criteria Others (Ethics / Cost / Time requirement) Formats of Interventional Studies: Cross Over (a.k.a., Self Control) Subjects switched to different interventions at one or more points after initial randomization Between & Within Group comparisons possible Differentiate Wash out vs Lead in ; Run in Measures of Association Interpreting RATIO s Risk (RR), Odds (OR), Hazard (HR): =1.0 = no difference (no increase/no decrease) in risk/odds/ratio >1.0 = Increased Ratio (RR/OR/HR) +1.00001 to +1.99 = use decimal value (converted to %) for interpretation If RR = 1.53, then a 53% increased (greater) risk in comparator group»increased because ratio value is ABOVE 1.0 2.0 = use statement of times control for interpretation +2.0 to If OR = 6.18, then comparator group is 6.18 times greater odds»greater because ratio value is ABOVE 1.0 <1.0 = Decreased Ratio (RR/OR/HR) 0.00001 to 0.99 = subtract decimal value from 1 (answer converted to %) for interpretation If HR = 0.73, then a 27% lower probability of the hazard outcome»lower because ratio value is BELOW 1.0 6
Measures of Association When Interpreting Ratio s [RR/OR/HR], look for : RR = 0.78 1. Group Comparison Orientation a. Ramipril vs. Placebo / Placebo vs. Ramipril 2. Direction of words a. Increased / Decreased 3. Magnitude a. 1.8 is 80% or 1.8 times / 0.8 is 20% or 0.8 times When looking at the CI for Ratio s (RR/OR/HR), if both values are on the SAME SIDE of 1.0, it is always statistically significant! Example Board Questions Measures of Association Absolute Risk Reduction (ARR) [a.k.a. Attributable Risk (AR)] Simple Absolute Difference (subtraction) in Risks AR defines the risk difference in the outcome among Exposed that can be attributed attributed to the actual exposure Ramipril = 14% & Placebo = 17.8% ARR = 3.8% A retrospective cohort study was conducted in 100 subjects living in a small village in Ethiopia. The study evaluated the frequency of developing intestinal infectious disease associated with exposure to contaminated pond water. The study determined the risk ratio (RR) to be 2.54. Which of the following is the most appropriate interpretation of this finding? Additional Representations Number Needed to Treat (NNT) 1 Absolute Risk Reduction (1/ARR) From ramipril example: 1 0.038 = 26.32; [27 Patients] Interpretation: Number of (whole) patients needed to be treated to experience the studied event outcome Outcome event can be a Beneficial or Harmful event A retrospective cohort study was conducted in 100 subjects living in a small village in Ethiopia. The study evaluated the frequency of developing intestinal infectious disease associated with exposure to contaminated pond water. The study determined the risk ratio (RR) to be 2.54. Which of the following is the most appropriate interpretation of this finding? A. Compared to subjects not exposed to the pond water, subjects exposed to the contaminated pond water were 2.54 times more likely to develop intestinal disease. B. Compared to subjects not exposed to the pond water, subjects exposed to the contaminated pond water were 54% more likely to develop intestinal disease. C. Compared to subjects exposed to the contaminated pond water, subjects not exposed to the pond water were 62.5% less likely to develop intestinal disease. D. Compared to subjects exposed to the contaminated pond water, subjects not exposed to the pond water were 2.54 times more likely to develop intestinal disease. E. Compared to subjects not exposed to the pond water, subjects exposed to the contaminated pond water were 200% more likely to develop intestinal disease. 7
A retrospective cohort study was conducted in 100 subjects living in a small village in Ethiopia. The study evaluated the frequency of developing intestinal infectious disease associated with exposure to contaminated pond water. The study determined the risk ratio (RR) to be 2.54. Which of the following is the most appropriate interpretation of this finding? A. Compared to subjects not exposed to the pond water, subjects exposed to the contaminated pond water were 2.54 times more likely to develop intestinal disease. B. Compared to subjects not exposed to the pond water, subjects exposed to the contaminated pond water were 54% more likely to develop intestinal disease. C. Compared to subjects exposed to the contaminated pond water, subjects not exposed to the pond water were 62.5% less likely to develop intestinal disease. D. Compared to subjects exposed to the contaminated pond water, subjects not exposed to the pond water were 2.54 times more likely to develop intestinal disease. E. Compared to subjects not exposed to the pond water, subjects exposed to the contaminated pond water were 200% more likely to develop intestinal disease. A clinical trial was conducted to determine the impact of dual treatment for hypertensive emergencies compared to single agent therapy. Based on the study findings, the risk of death from hypertensive emergency in the dualtreatment group was 13% compared to 27% in the single agent treatment group. Based on these findings, which of the following values most closely represents the number needed to treat (NNT) and its proper interpretation? A. A total of 7 patients would have to be treated with dual agent agent therapy to reduce one death following a hypertensive emergency B. A total of 14 patients would have to be treated with a single agent to reduce one death following a hypertensive emergency C. A total of 13 patients out of 100 (13%) would have to be treated with dual agent therapy to reduce one death from a hypertensive emergency D. If a total of 14 patients were treated with dual agent therapy for a hypertensive emergency, it would lead the a single death E. A total of 7 patients out of 100 (7%) would die from hypertensive emergency if treated with dual agent therapy A clinical trial was conducted to determine the impact of dual treatment for hypertensive emergencies compared to single agent therapy. Based on the study findings, the risk of death from hypertensive emergency in the dualtreatment group was 13% compared to 27% in the single agent treatment group. Based on these findings, which of the following values most closely represents the number needed to treat (NNT) and its proper interpretation? SUMMARY Appropriate study design selection based on study question, purpose, and group allocation format Every study has its unique strengths & weaknesses Risk difference (ARR) used to easily determine NNT RR and OR are simply ratio s of 2 risks ik or odds and interpreted exactly the same (as is HR) Calculation & Interpretation of (OR/RR) is useful A clinical trial was conducted to determine the impact of dual treatment for hypertensive emergencies compared to single agent therapy. Based on the study findings, the risk of death from hypertensive emergency in the dualtreatment group was 13% compared to 27% in the single agent treatment group. Based on these findings, which of the following values most closely represents the number needed to treat (NNT) and its proper interpretation? A. A total of 7 patients would have to be treated with dual agent agent therapy to reduce one death following a hypertensive emergency B. A total of 14 patients would have to be treated with a single agent to reduce one death following a hypertensive emergency C. A total of 13 patients out of 100 (13%) would have to be treated with dual agent therapy to reduce one death from a hypertensive emergency D. If a total of 14 patients were treated with dual agent therapy for a hypertensive emergency, it would lead the a single death E. A total of 7 patients out of 100 (7%) would die from hypertensive emergency if treated with dual agent therapy QUESTIONS? 8