Non-Experimental Approaches to Research

Transcription

1 Non-Experimental Approaches to Research Why is this Important? The classic true experiment is one where the experimenter has control over all aspects of the situation. Groups of appropriately selected subjects are assigned at random to experimental conditions. Sometimes this isn't possible. Can we put you in such conditions? Let's say you wanted to test memory after some traumatic event. How about the details of what occurred during a nuclear reactor mishap in your community? In a True Experiment - we would select subjects at random and then randomly assign them to a control group (live in a calm environment - Group 1) or an experimental group (we arrange you to live next to a nuclear incident). Consider our illustrations. Would this work? Here are the conditions. Group 1 is the control group. We wish to thank the Physics Department for helping in producing the conditions for Group 2.

2 Now what would our subjects think about this? I bet this wouldn't work. It is obvious we can't put people in situations that are dangerous. It is also obvious that we cannot always manipulate all the IVs. What are you going to do? Let's see some solutions! QUASI-EXPERIMENTS Quasi-experimental designs do NOT allow the researcher to control the assignment of subjects to conditions! Factorial Designs with One Nonmanipulated Variable Another example, might be a study of gender issues - you can't make someone a male or female for purposes of the study. In our study, above, the EXPLOSION vs. Control would be the nonmanipulated variable as we would test accident survivors vs. a suitable matched control group. " Time Series and Interrupted Time Series The researcher makes several observations of behavior over time prior to and then immediately after introduction of an IV. The basic idea is: Observation1 O2 O3 O4 X(treatment) O5 O6 O7 O8 This is the classic design from Cook and Campbell. NUANCES YOU NEED TO ESTABLISH A STABLE BASELINE BEFORE THE TREATMENT IS INTRODUCED. AN INTERRUPTED TIME SERIES IS SOMETIMES DEFINED AS HAVING A TREATMENT WHICH IS A NATURAL EVENT AS COMPARED TO ONE INTRODUCED BY THE EXPERIMENTER.

3 Here's an example from a study analyzing the effect of a new gun law on crime. Note the base rate for several years before the law took effect and the decrease in crime after its passage. # Multiple Time Series This design is used to attempt to rule out possible alternative interpretations. Maybe in Lott and Mustard's study, some other event decreased crime. You might then compare their location against an area that did not pass the same law. The results might look like the smaller graph above. The control group doesn't show the change across time. SINGLE SUBJECT DESIGNS! Advantages and Disadvantages Observation of a single subject has a noble tradition in Psychology. Some of our most basic principles in areas of Perception and memory were discovered using it. It is still an incredibly useful technique in these fields. It is also a mainstay of research in Clinical Psychology where one focuses on the progress of a client. One major advantage is the focus on individual performance. A classic example is in the field of learning. If you plotted correct performance on a memory task for a group of subjects, one might conclude that learning is a gradual process. Or is it because a grouped graph sums over a number of subjects learning the item in an all or none fashion but over different trials. Look for this conundrum in the graphic below. Another advantage related to clinical work is that you don't leave individuals untreated in control groups. Single subject designs also allow flexibility in design. If a treatment is not working - it is easy to change. What are some of the disadvantages? : Some effects are small and can't really be seen in one subject.

4 Sometimes you can't try out different variables on the same subject and you must use between-subject designs. Correct Responses Grouped Data Learning Curve An Individual's Learning Curve " Techniques commonly used are: Time Series Interrupted Time Series Multiple Time Series Useful techniques of testing include: Withdrawal of Treatment Designs (ABA - baseline, treatment, withdraw treatment) ABAB - as above but with treatment resumed after withdrawal Interaction Designs A-B-A-B--BC-B-BC. The purpose is to see if C has an effect in addition to B. NATURALISTIC OBSERVATION Observation is our most primitive form of learning but sometimes it can be quite elegant. Let's take a look! Do men and women carry their books differently (in their arms or by their sides)? Why can't you just look out the window at campus (Jenni & Jenni, 1976, Science)?! Concerns

5 Reactive Behavior - Do the observed act differently? Selective Perception - Do the observers see what they want to see? Data Analysis - Do you need a behavioral taxonomy to describe all the behaviors you need to record? " Participant or Nonparticipant You can't really dress up as a fire hydrant and stand outside of church to observe the behavior of churchgoers. Do you have to join the church? Then, do you become part of the process and influence it? However, we do recommend this procedure for observing canine behavior. META-ANALYSIS! Combines effect sizes or significance testings from many different studies " Based usually on translating study results into a comparative z-score mode # Concerns are: You may be combining studies of unequal quality The studies used different and perhaps not comparable methods Not all relevant studies give you the relevant statistics or information Studies that had negative results may be in the trash or file drawer somewhere and never published. SURVEY METHODS (see Workshop #16) Bottom Line The reason you use a true experimental design is that you want to have the best chance you can to determine causality. But sometimes you can't conduct a true experiment. This doesn't mean that no research or useful information can be gathered. Several techniques exist that are useful, such as: % Quasi-experimental designs % Time Series % Single Subject Designs % Survey Research % Meta-analysis These techniques do not involve direct manipulation of all postulate causal factors and this must give you caution as you report your conclusions or recommend policy decisions. However, used judiciously they are useful and can sometimes suggestion true experimental designs.