Research Design & Protocol Development Paul J Kim, DPM, MS, FACFAS Associate Professor Director of Research
Definitions Randomization Sampling Allocation Variables Independent Variable Treatment, intervention, or experimental activity that is manipulated or varied by the researcher to create an effect on the dependent variable Dependent Variable Response, behavior, or outcome that is predicted and measured in research; change in the dependent variable are presumed to be caused by the independent variable
Definitions Groups Experimental Control Validity Level of control and bias Blinding Single: subject or examiner Double: both subject and examiner Sites Single center Multi-center Direction Prospective Retrospective
Definitions Survival (survivorship) Data Medical research that determines if an event occurs and the time to occurrence Registries Sample analysis Intention-to-treat (ITT) All subjects are compared in the treatment groups to which they were originally randomly assigned, regardless of any treatment that they subsequently received Preserves the strength of randomization Provides a pragmatic estimate of the effect of a treatment strategy rather than just the specific efficacy of the treatment itself Per protocol Excludes subjects who were not fully compliant with the study protocol
Levels of Clinical Evidence Systematic Review of Experimental Studies (well designed randomized controlled trials) Meta-analysis of Experimental (RCT) and Quasi-Experimental Studies Integrative Reviews of Experimental (RCT) and Quasi-Experimental Single Experimental Study (RCT) Single Quasi-Experimental Study Meta-Analysis of Correlational Studies Integrative Reviews of Correlational and Descriptive Studies Qualitative Research Metasynthesis and Metasummaries Single Corrlelational Studies Single Qualitative or Descriptive Study Expert Opinions
Research Design Quantitative Formal, objective, systematic study process to describe and test relationships and to examine causeand-effect interactions among variables E.g. test the effects of a drug vs placebo Qualitative Systematic, interactive, subjective approach used to describe life experiences and their meaning E.g. Addresses theoretical, philosophical, historical questions
Descriptive Explore and describe phenomena in reallife situations Generation of new knowledge about concepts or topics about which there is limited or no research
Correlational Examines linear relationships between 2 or more variables and determines the type of relationship (negative or positive) and strength of that relationship
Experimental Examine cause-and-effect relationships between independent and dependent variables under highly controlled conditions E.g. laboratory or animal studies Randomization into treatment or control Control can be Placebo No treatment Usual/Standard treatment
Experimental Experimental posttest-only comparison group designs Dependent variable cannot be measured before the treatment
Other Experimental Factorial Design 2 or more characteristics, treatments, or events are independently varied within a single study Examines multiple causality E.g. 2x2, 4x4 etc Randomized Blocking Design A blocking variable introduced to a two group pretest posttest or the two group post test Nested Design When you want to consider the effect of variables that are found only at some levels of the independent variables being studied Crossover or Counterbalanced Designs More than one treatment administered sequentially Comparison are made of the effects of the different treatments on the same subject Randomized Controlled Trials
Quasi-Experimental Examines cause-and-effect relationships among selected independent and dependent variables E.g. effect of a treatment intervention on patient outcome A Pre-experimental Design is a research design with many weaknesses and numerous threats to validity
Preexperimental One group post test only Can not make causal inferences No comparisons can be made
Preexperimental Posttest only with a comparison group (not really quasi-experimental but is considered preexperimental) Nonequivalent comparison group used hence direct comparisons may not be valid
Preexperimental One-Group Pretest-Posttest No comparison group Multiple treats to validity
Quasi-Experimental Design Pretest and Posttest with a comparison group Nonequivalent comparison group
Quasi-Experimental Design Pretest and Posttest with 2 comparison treatments New treatment option compared with: No treatment group or Placebo treatment group or Usual treatment group
Quasi-Experimental Design Pretest and Posttest with a removed treatment A Pretest is done then an intervention is implemented, then it is removed Multiple measures are taken
Quasi-Experimental Design Pretest and Posttest with a reversed treatment 2 groups exposed to one of the treatments Test difference between the two treatments
Quasi-Experimental Design Interrupted Time-Series Designs Treatment applied at some point in the observations 4 types Simple interrupted time-series design Interrupted time-series design with comparison group Interrupted time-series designs with multiple treatment replications
Research Proposal/Protocol Development
Definitions Proposal Argument of relevance to achieve funding Detailed outline of research project Protocol Step-by-Step Project Execution Template for IRB Submission
Steps to the Research Process 1. Formulate a research question and/or state purpose 2. Review relevant literature 3. Develop framework 4. Formulate objectives, questions, or hypothesis 5. Define variables 6. Identify limitations 7. Select research design 8. Define the population and sample
Steps to the Research Process 9. Select method of measurement 10.Develop plan for data collection and analysis 11.Write Research proposal and/or protocol 12.Attempt to receive funding if necessary 13.Implement the research plan 14.Pilot study? 15.Interpret research outcomes 16.Communicate findings
Components of a Proposal/Protocol Protocol information page Introduction Background Rationale Specific Aims and Objectives Study design Subject Selection Inclusion/Exclusion criteria Treatment/Intervention Study Procedures (schedule of activities) Efficacy/Safety assessments Efficacy/Safety assessments Adverse events reporting Statistical Methods/Data Analysis (endpoints; sample size) Quality control Data Handling Ethics (IRB etc) Regulatory Requirements Strengths and Limitations References Appendix Budget Timeline
Conclusions Have a plan for the proposed project Research design is critical prior to executing the project