EXPERIMENTAL DESIGN Page 1 of 11. relationships between certain events in the environment and the occurrence of particular

Size: px

Start display at page:

Download "EXPERIMENTAL DESIGN Page 1 of 11. relationships between certain events in the environment and the occurrence of particular"

Sandra O’Neal’
6 years ago
Views:

1 EXPERIMENTAL DESIGN Page 1 of 11 I. Introduction to Experimentation 1. The experiment is the primary means by which we are able to establish cause-effect relationships between certain events in the environment and the occurrence of particular forms of behavior. 2. The basic notion is simple: At least two groups of subjects are treated exactly alike in all ways except one -- the differential treatment. 3. Any differences observed in the behavior of the two groups of subjects is then attributed to, or said to by caused by the difference in the specific treatment conditions. We will consider this simple notion in some detail. II. The Anatomy of an Experiment 1. We can describe the function of an experiment most clearly by covering each of its component parts in turn. 2. Few experiments are conducted out of the blue without any rationale. Experiments are based on previous observations, the results of other experiments, or deductions from theory. 3. Researchers set out to discover something or to "prove" that a particular line of reasoning is correct or incorrect.

2 EXPERIMENTAL DESIGN Page 2 of In every research report, an introductory section presents background research to the problem, the reasoning leading up to the reported experiment, and finally, the research hypothesis -- a succinct statement of the purpose of the experiment. III. Treatment Conditions 1. An experiment consists of two or more treatment conditions. The actual treatment conditions in an experiment are usually determined by the manipulations specified in the research hypothesis. 2. A research hypothesis may state, for example, that variations in the amount of a particular drug will have differential effects on behavior, or that different kinds of drugs will have differential effects on behavior. 3. "Variations in amount," in the first example, and "different kinds of drugs," in the second example, are descriptive phrases referring to the treatment conditions included in the two experiments designed to test the hypotheses. 4. However, these phrases are not detailed enough to define the treatment conditions sufficiently. The researcher would have to provide more information -- specifying the dosage variations in the first example, naming the particular drugs to be administered in the second -- to clearly delineate the different treatment conditions in these two experiments. 5. A number of different terms are used to refer to the treatment conditions included in an

3 EXPERIMENTAL DESIGN Page 3 of 11 experiment. Most commonly, the treatment conditions are called, collectively, the independent variable. A. In the first of our examples, variations in amount constitute the independent variable of the experiment, while in the second, variations in drug type constitute the independent variable. B. The independent variable comprises the range of treatment conditions under the control of the experimenter. The term under control means manipulated by or varied by the experimenter. C. For this reason, the independent variable is also known as the manipulated variable or the treatment variable. IV. The Response Measure 1. The behavior observed by the experimenter in conjunction with the manipulation of the independent variable is called the response measure. 2. Any behavior capable of being measured can be a response measure, for example, the speed of completing a task, the number of errors made, or even ratings on a scale. 3. The response measure is often referred to as the dependent variable. 4. The response measure should measure the behavior it is supposed to measure and must

4 EXPERIMENTAL DESIGN Page 4 of 11 be stable or reliable. 5. Researchers often "solve" the problem of choosing among response measures simply by including several response measures in an experiment. They follow this procedure in order to preserve some of the richness and complexity of the behavior being studied and to ascertain whether different aspects of the behavior are affected differently by the administration of the treatment conditions. 6. In experiments in which two or more dependent variables are to be recorded and analyzed, results can be examined as deriving from separate experiments, one for each of the response measures. V. Method, Design, and Subjects 1. The specification of the research hypothesis and the identification of the independent and dependent variables provides only a skeletal outline of an experiment. 2. Many specific details must be worked out before experimentation can actually begin. 3. The actual design of the experiment is a topic which we will have a great deal to say. In a statistics book, experimental design usually refers to a general plan for conducting an experiment. 4. Experimental designs can differ in a number of ways, for example, in the number of independent variables to be manipulated.

5 EXPERIMENTAL DESIGN Page 5 of The most common design for an experiment in which two or more independent variable are manipulated is called the factorial design. A. For example, Forebrain ablation in Siamese Fighting Fish. The two independent variables employed were ablation and stimulus (conspecific and mirror). B. Schatcher and Singer cognitive view of emotion. Three independent variables drug, informed, situational factor. 6. Most experiments in the contemporary literature of psychological research employ factorial designs. 7. Before we study the more complex factorial designs, we will focus on the single variable experiment, because is it simpler to understand and it serves as an important building block for the more complicated factorial design. 8. A second major way in which experimental designs can differ is in the method of assigning subjects the different treatment conditions. 9. In the simplest procedure, and the one we consider first, an independent group of subjects is assigned to each of the treatment conditions. 10. At the other extreme is a design in which each subject serves in all the treatment conditions.

6 EXPERIMENTAL DESIGN Page 6 of In between are designs in which these features are combined, with the subjects serving in some but not all of the treatment conditions. 12. A final design consideration concerns the subjects included in the study. Generally, two questions arise at this point. A. First: What sort of subjects do we wish to study? Should we test animals or humans? Any particular sort of animal or human? Do we want subjects with a particular past history? Do we want individuals at a particular stage of development? B. The second question concerns the number of subjects needed in an experiment. The answer to this question is so simple it seems trite. C. An experimenter needs as many subjects as are necessary to provide a relatively sensitive test of the research hypothesis. One way to increase the sensitivity in an experiment is to increase the number of subjects. D. Due to the nature of most psychological research, by and large the minimum number of subjects in an experiment is five subjects per condition, although the gain in sensitivity with ten or more subjects is considerable. VI. Classification of Independent Variables 1. There are two major classifications of independent variables: the nature and the source of the manipulation.

7 EXPERIMENTAL DESIGN Page 7 of The Nature of the Manipulation. One common classification scheme concerns the nature of the manipulation. A variable in which the treatment levels differ in degree or in amount as measured by either a physical or a psychological scale is called a quantitative independent variable. 3. A qualitative independent variable, on the other hand, is a variable in which the levels differ in kind rather than in amount. 4. A quantitative variable can be any manipulation that represents variations in the amount of the independent variable. A. The number of hours of food deprivation for rats in a maze-learning experiment, the degree of background noise in a signal-detection task, the amount of money given as incentive in a problem solving task are all examples of quantitative independent variables. B. In an experiment that has a quantitative independent variable, the variable is usually tested on several levels, or conditions, covering a relatively large range of variation. C. Qualitative independent variables, as opposed to quantitative ones, are easy to recognize, since the specific treatment conditions chosen to define the independent variable cannot be ordered meaningfully on a quantitative scale. Types of drugs, variations in the instructions given to subjects, and differences in teaching methods are examples of qualitative variables.

8 EXPERIMENTAL DESIGN Page 8 of The Source of the Manipulation. A second classification, which cuts across the qualitative-quantitative distinction, specifies the source of the treatment manipulation. A. One source of manipulation is the task; variations in some characteristic of the task itself constitute a task variable. Mazes differing in complexity and different types of conceptual problems are examples of task variables. B. A second type of independent variable is an environmental variable. In this case, the nature of the task is held constant, but some aspect of the experimental environment is manipulated. Obvious examples include the temperature, humidity, illumination, and other conditions prevailing in the testing situation. 6. A final source of manipulation is the subject. There are basically two types of subject manipulation; temporary subject variables and permanent subject variables. A. The first consists of some sort of temporary psychological or physiological intervention. Giving differential instructions to human subjects is the typical way of manipulating subjects' set, or view, of the task. B. Drugs are commonly used to induce differential physiological states in subjects. C. Internal changes produced in the subjects by such manipulations are temporary in the sense that the subjects will quickly return to their "normal" selves after the experiment.

9 EXPERIMENTAL DESIGN Page 9 of 11 D. The second is a different sort of manipulation, involving more or less permanent characteristics of subjects. E. The effect of intelligence on problem solving and the effect of biological sex on speed of learning are examples of this class of manipulation. F. Such variables are variously referred to as subject variables, classification variables, organismic variables, and individual-difference variables. G. In the context of an experiment, classification variables are manipulated through the selection of subjects on the dimension to be studied. H. Manipulation of this sort -- involving classification variables -- does not constitute an experiment as we have defined it, however, since the "administration of the experimental treatments" is obviously not under the control of the experimenter. J. In an experiment, the independent variable is the only feature of the situation that is allowed to vary systematically from condition to condition. It is this characteristic of an experiment that permits the researcher to infer that a particular manipulation caused systematic differences in behavior observed among the different groups. VII. The Confounding of Variables 1. The strength of the experimental method depends on our ability to guarantee that only the manipulated variable is permitted to vary systematically from condition to condition.

10 EXPERIMENTAL DESIGN Page 10 of In cases where a second independent variable is unwittingly permitted to vary along with the intended one, we say that the two independent variables are confounded. 3. If the confounding variable influences the behavior under study, we will not be able to distinguish the effects of the manipulated variable from the effects of the other. Thus, the confounding of variables usually ruins an experiment. 4. As an example, suppose we want to compare the effectiveness of three different methods of teaching arithmetic to elementary school children. For the experiment, we train a different teacher in each method, making sure that in all three methods exactly the same material is covered. 5. The teachers then instruct their classes for two weeks using the method in which they were trained, and the performance of the students is assessed by a test made up by the experimenter. 6. The methods of teaching arithmetic constitute the independent variable, and the scores on the proficiency test constitute the dependent variable. 7. How would you evaluate this experiment? We hope that you can see the rather serious flaws built into this example. A. Although the manipulated independent variable consists of the differences in the teaching methods, several other variables were at work, thus confounding the variable under study.

11 EXPERIMENTAL DESIGN Page 11 of 11 B. One such variable is the ability of the students in the different classes. C. Another confounding variable might consist of differences in the abilities of the three teachers. The teachers probably differ in teaching effectiveness. D. The obvious first step toward preventing confounding in your own research is to be constantly aware of the problem. You should examine critically all phases of your planned research to discover possible flaws. E. One way of doing this is to try to list all factors that might affect the behavior under study and to make sure that these are all controlled in some manner in the design of the experiment.

Statistics Mathematics 243

Statistics Mathematics 243 Michael Stob February 2, 2005 These notes are supplementary material for Mathematics 243 and are not intended to stand alone. They should be used in conjunction with the textbook