Purpose. Study Designs. Objectives. Observational Studies. Analytic Studies

Transcription

1 Purpose Study Designs H.S. Teitelbaum, DO, PhD, MPH, FAOCOPM AOCOPM Annual Meeting Introduce notions of study design Clarify common terminology used with description and interpretation of information collected via selected study designs 9/16/2012 HST AOCOPM 2 Objectives Present frameworks for common study designs which populate the medical literature Present the advantages and disadvantages of selected study designs Present a hierarchy of study designs Descriptive Case Report Case Series Observational Cohort CaseControl Analytical Experimental Randomized NonRandomized Cross Sectional 9/16/2012 HST AOCOPM 3 9/16/2012 HST AOCOPM 4 Analytic Studies Can the existence and strength of possible associations be established between an exposure and outcome? Observational Studies Attempts to make valid comparisons between exposure and outcome BUT does not really control either exposure or treatment OBSERVE groups maybe two or more which differ in exposure to a suspected cause of disease and then compare results or outcome of disease after a suitable period of time. 9/16/2012 HST AOCOPM 5 9/16/2012 HST AOCOPM 6 A1

2 Experimental In a RANDOMIZED study, the researcher ABSOLUTELY controls who gets into which group. This is always considered the gold standard. A NONRandomized is what I am also calling a type of cohort study. There are two groups but you do not RANDOMLY assign people to the groups, they appear naturally. (Section A and Section B of a factory). Validity is the key Internal Validity CAN I BELIEVE the data? External Validity Can the results be GENERALIZED to places other than to where the study was done or to populations that were not studied? OF THESE Premium is always on internal validity 9/16/2012 HST AOCOPM 7 9/16/2012 HST AOCOPM 8 Confounding Variables This is the bane of medical research Is there a variable LURKING around that will CONFUSE THE RESULTS hence threaten internal validity Example Drinking Alcohol will increase the probability of an cardiovascular event. ETOH MI SMOKING 9/16/2012 HST AOCOPM 9 9/16/2012 HST AOCOPM 10 Criteria for CONFOUNDING Independently related to the cause Independently related to the outcome CANNOT be a consequent of the independent variable Questions to determine which design is used 1 Q1: What is the aim of the study? To simply describe a population descriptive To quantify the relationship between factors analytic Q2: If analytic, was the intervention randomly allocated? Yes RCT No Observational Q3: If observational when was the outcome determined? Sometime after the exposure or intervention Cohort At the same time as the exposure or intervention cross sectional study or survey Before the exposure was determined? Casecontrol 1 9/16/2012 HST AOCOPM 11 9/16/2012 HST AOCOPM 12 A2

3 Prospective Study or Randomized Control Trial Pool of People in whom we are interested Drop Outcome Outcome Outcome Outcome Outcome Outcome Present Future PAST Present 9/16/2012 HST AOCOPM 13 9/16/2012 HST AOCOPM 14 Advantages and Disadvantages of Various Designs Randomized Control Trials Advantages unbiased distribution of confounders Blinding (masking) more likely Stronger statistical analysis Disadvantages Expensive both in time and money Volunteer bias Ethically problematical at times 9/16/2012 HST AOCOPM 15 9/16/2012 HST AOCOPM 16 Continued Crossover Advantages All subjects serve as own controls smaller sample is needed All subjects receive treatment Statistical testing is strong Masking can be maintained Disadvantages All subjects receive placebo or alternative treatment at some point Washout period may be lengthy or unknown Cannot be used for treatments with permanent effects Continued CrossSectional Advantages Cheap and simple Ethically safe Disadvantages Establishes association (at best) NOT CAUSALITY Recall bias susceptibility Confounders may be unequally distributed No control over number of people who will participate 9/16/2012 HST AOCOPM 17 9/16/2012 HST AOCOPM 18 A3

4 Advantages and disadvantages of cohort and case control studies Cohort studies Case control studies No Yes since starting Suited for rare diseases with a set of cases Suited for rare exposures Yes since starting with exposure status No Allows for studying several exposures Difficult but examples exists (Framingham study) Yes Allows for studying several outcomes Yes No Disease status easy to ascertain Sometimes difficult Easier since starting point of the study status easier to ascertain Yes since starting point of the study. Sometimes difficult. Except for retrospective cohorts Information biases. Allows computation of risk and rates Yes No Allows computation of effect Computationof riskratio and rate ratio Estimationof riskratio, rate ratio from odds ratio Allows studying natural history of disease Yes More difficult Easier to show that cause precedes effect. Temporality between cause and effect difficult to establish Based on existing data sources Difficult Yes but access to information sometimes difficult Easiness to find a reference group Usually not difficult to identify an unexposed No population Major potential biases when selecting a control group Sample size Large Small Cost Elevated Smaller except if retrospective cohorts Time required Long, sometimes very long except if retrospective Shorter cohorts Follow up Difficult, loss to follow up No follow up Logistics Heavy Easier Many staff, large data sets Long duration Concept Easy to understand Difficult to understand particularly if case cohort or density case control study Ethical issues Major if studying risk factors. None since outcome already happened Interruption of study if exposure shown to be harmful. Need for intermediate analysis. 9/16/2012 HST AOCOPM 19 Quick and Dirty Randomized Control Trial Short induction period, multiple outcomes, good control for confounding. Best for study the effect of an intervention Crossover Best control for confounder. Best if there is no danger of sensitizing the person to the effects of a drug or intervention. Person is own control. Cohort Best to study the effect of predictive risk factors on an outcome. Good for common diseases and multiple outcomes Casecontrol Good for ethical control and rare diseases. Best for rare diseases Crosssectional Best for quantifying the prevalence of a disease or risk factor and cooccurrence of factors 9/16/2012 HST AOCOPM 20 Comments on RCT 2 Unnecessary when the effect is dramatic and the likelihood of confounders is small Inappropriate when dealing with infrequent outcomes or interventions designed to prevent rare events. Impossible when ethical issues are a major concern Inadequate when setting or patients are so atypical that the results are difficult to generalize (addresses external validity). 2 Overview of research designs M J Clancy Emerg Med J2002;19: doi: /emj Terminology Bias Systematic deviation from the truth. Anything which systematically increases or decreases the probability of the outcome. Selection Bias This occurs when the study subjects differ systematically from the population with the same condition Followup Bias Those patients who remain in a study may differ from those lost both in terms of personal characteristics and outcome status. Those lost may have died or not wished to be followed up because of the treatment 9/16/2012 HST AOCOPM 21 9/16/2012 HST AOCOPM 22 Terminology Confounding A third variable or process which confuses the relationship between two variables. Variables that cause/prevent an outcome from occurring outside of or along with the variable being studied. These variables render it difficult or impossible to distinguish the relationship between the variable and outcome being studied. Blinding or Masking (Single, Double or Triple) Single: When a participant does not know which group they are assigned to (experimental or control) Double: When neither the participant nor the investigator know to which group the participant belongs Triple: Neither the participant, investigator or data analyst know the group to which the participant is assigned (experimental or control) Population Bias/Volunteer Bias A sample may be skewed by those who are selected or selfselected into a study. If only certain portions of a population are considered in the selection process, the results of a study may have poor validity. Volunteers may be more motivated and thus tend to overestimate the effect of the intervention. Terminology Selection bias Selection bias occurs when study subjects are selected or become part of the study as a result of a third, unmeasured variable which is associated with both the exposure and outcome of interest. One group will have a higher probability of reaching an outcome and it is on that basis that they were included in the study. Surveillance bias One group is studied more closely or with greater care than a comparison group. This can be eliminated with masking the investigator as to group membership of the participant. 9/16/2012 HST AOCOPM 23 9/16/2012 HST AOCOPM 24 A4

5 Classic Examples of Types of Studies CaseControl Aspirin and Reye s syndrome in children Oral contraceptives and reduced risk of ovarian/endometrial cancer Cohort Study Framingham Heart Study Identification of major risk factors for heart disease Original cohort (1948) 5,209 residents of Framingham, MA. Offspring cohort (1971) 5,124 children spouses Framingham III: 3,500 grandchildren (ongoing) Classic Examples Randomized Trials Women s Health Initiative Clinical trial component: 68,131 postmenopausal women Multiple interventions: Dietary, hormone therapy, calcium/vitamin D Physician s Health Study PHS1 22,071 male physicians Assess benefits and risks of aspirin and beta carotene PHS2 14,642 male physicians Multiple interventions: Vitamin C, vitamin E, beta carotene, multivitamin 9/16/2012 HST AOCOPM 25 9/16/2012 HST AOCOPM 26 Q1: To investigate the relationship between egg consumption and heart disease, a group of patients admitted to hospital with myocardial infarction were questioned about their egg consumption. A group of age and sex matched patients admitted to a fracture clinic were also questioned about their egg consumption using an identical protocol. What design does this study illustrate? A. Casecontrol C. Randomized Control D. Cross Sectional 9/16/2012 HST AOCOPM 27 9/16/2012 HST AOCOPM 28 Q2: To investigate the relationship between certain solvents and cancer, all employees at a factory were questioned about their exposure to an industrial solvent, and the amount and length of exposure measured. These subjects were regularly monitored, and after 10 years a copy of the death certificate for all those who had died was obtained. What design does this scenario represent? A. CaseControl C. Randomized Control Q3: A study was conducted to see if practicing occupational medicine physicians were able to learn better by either attending a AOA sponsored conference or an athome webbased instructional webinar. The physicians were given a choice of which format they wanted to use. A multiple choice examination (pretestposttest format) was given to see which group acquired the most information. Which would be the preferred design? A. Casecontrol C. Randomized Control Trial D. Crosssectional 9/16/2012 HST AOCOPM 29 9/16/2012 HST AOCOPM 30 A5

6 Q4: One was interested in studying the efficacy of smoking on relieving the symptoms of low back pain in highway construction workers. Which design would be preferred? a. Casecontrol b. Cohort c. Randomized control trial PostTest Q1: A survey was conducted of all nurses employed at a particular hospital. Among other questions, the questionnaire asked about the certification of the nurse and whether she was satisfied with her career prospects. This illustrates which type of study design? A. Crosssectional C. CaseControl D. Randomized Control Trial 9/16/2012 HST AOCOPM 31 9/16/2012 HST AOCOPM 32 Post Test Q2. To evaluate a new protocol for treating low back pain, patients with lower back pain were randomly allocated to either the new protocol or to conventional occupational therapy. After 3 months they were questioned about their back pain, and observed lifting a weight by independent monitors. This illustrates which type of study design? A. Case study C. CaseControl D. Randomized Clinical Trial Post Test Q3: A new triage system has been set up at Dr. Young s Occupational Medicine Office. To evaluate it the waiting times of patients were measured for 6 months and compared with the waiting times at Dr. Scott s office located 5 miles away. The scope of practice and severity of illness of the patients are comparable. This illustrates which type of study design? A. Case Study C. Randomized Control D. Crosssectional 9/16/2012 HST AOCOPM 33 9/16/2012 HST AOCOPM 34 PostTest Q4. In studying the effect of information retention on a group of experienced physicians attending a medical conference, it was found that those on the right side of the room achieved a higher post test score than people on the left side of the room. What might one consider a reasonable explanation for the difference in scores? A. A confounding variable heretofore unknown B. Nature of the material C. Height of the instructor Post Test Q5: An investigator wishes to study the effects of an a medication on low back pain on patients who are attending a physical therapy clinic. He does not know if the medication he administers to the patient is the active medication or a placebo. The patient does not know if that which they take is active or placebo. This is an example of which type of masking? A. Single masking B. Double masking C. Triple making 9/16/2012 HST AOCOPM 35 9/16/2012 HST AOCOPM 36 A6

7 Answers Pretest 1. A 2. B 3. B 4. A Posttest 1. A 2. D 3. B 4. A 5. B 9/16/2012 HST AOCOPM 37 A7