Research Methods Gleitman et al. (2011), Chapter 1

Size: px

Start display at page:

Download "Research Methods Gleitman et al. (2011), Chapter 1"

Grace Ray
5 years ago
Views:

1 Research Methods Gleitman et al. (2011), Chapter 1 Mike D Zmura Department of Cognitive Sciences, UCI Psych 9A / Psy Beh 11A January 14, 2014 T. M. D'Zmura 1

2 Research Methods One of the dullest subjects and also one of the most important! People who do not use appropriate research methods in psychological research run a high risk of generating BS. An example: NY Times: Journal s Paper on ESP Expected to Prompt Outrage Daryl Bem, Cornell: precognition for erotic photos. What is precognition? Ability to predict the future. Try Philip K. Dick s short story Minority Report, written in T. M. D'Zmura 2

3 A software program randomly posted a picture behind one curtain or the other but only after the participant made a choice. Still, the participants beat chance, by 53 percent to 50 percent, at least when the photos being posted were erotic ones. They did not do better than chance on negative or neutral photos. The effect size is rather small (53% vs. 50% guessing rate). However, if there are many many human subjects with each running many trials, such an effect can reach statistical significance as determined by standard methods of inferential statistics. If you keep on running experiments of this sort, you will get something that looks like a result JUST BY CHANCE. Such research belongs more properly in the Journal of Irreproducible Results There is no result if it cannot be replicated. T. M. D'Zmura 3

4 Basic principle of statistics: The more observations that you make, the more likely it is that you will observe patterns that couldn t possibly occur by chance even though they occur just by chance! Example. Texas Hold Em in the movie Casino Royale. How likely is it that that the player two seats over to the right has a full house, the player one seat over has a better full house, and you have a straight flush? Jillions-to-one. It makes for a good movie but: this is very unlikely. Yet it is true: the more you play Texas Hold Em the more chance you have of observing such an unlikely sequence. If you keep on running ESP experiments, you are bound to get one that looks as though it is producing positive results (even though it is just by chance). Appropriate research methods are needed to interpret such events. T. M. D'Zmura 4

5 Observation and observed variables. Induction on examples. Testable hypothesis. Scientific method: test a theory by performing an experiment. There are many theories out there, based on observation, which do not lead to hypotheses that can be easily tested by experiment. From Physics: Cosmology At this time at least, we are unable to directly manipulate key variables concerning an original big bang, universe expansion, black hole behavior, etc. T. M. D'Zmura 5

6 Observation and observed variables. Induction on examples Testable hypothesis. Scientific method: test a theory by performing an experiment. There are many theories out there, based on observation, which do not lead to hypotheses that can be easily tested by experiment. From Economics: Sunspots Cause Economic Cycles T. M. D'Zmura 6

7 Observations of economic variables (employment, gross domestic product, etc.) suggest that these change cyclically (the business cycle ). Downturns appear to occur maybe once every eight-12 years. T. M. D'Zmura 7

8 William Stanley Jevons in 1878 suggested that the 11- year sunspot cycle lies at the root of the business cycle. While there may be a correlation between these two variables, there is no reason to think that there is a causal relationship. There is no reason to think that increased numbers of sunspots cause economic downturns. Color_Small.jpg T. M. D'Zmura 8

9 Observation and observed variables. Induction on examples Testable hypothesis. Scientific method: test a theory by performing an experiment. There are many theories out there, based on observation, which do not lead to hypotheses that can be easily tested by experiment. From Psychology: Darwinian Evolution Has Played a Key Role in Determining Human Behavior T. M. D'Zmura 9

10 Experiments Independent variable(s): these are directly manipulated by the experimenter Dependent variable(s): these are measured Control variable(s): these are independent variables that are manipulated in an experiment with the aim of ruling out explanations for the results other than those that bear directly on the tested hypothesis. Subject variable(s): these are characteristics of the (human) subjects who participate in an experiment. Because they are not directly manipulated by the experimenter, they are not true independent variables. Quasi-experiment: an experiment in which only the effects of subject variables are tested. T. M. D'Zmura 10

11 Descriptive Statistics Mean (average value) Median (middle value) Mode (most frequent value) Variance and its square root: standard deviation (measure of variability about the mean) T. M. D'Zmura 11

12 Inferential Statistics Compare human female heights to human male heights Compare the difference between the average height for males and the average height for female to the spread (variability) in the values found for males and females. If the difference between the two averages is large compared to the variation in the measurements, then the difference is deemed significant. Otherwise, it is not significant. One s ability to ascertain that a small difference is statistically significant increases as the number of measurements is increased: statistical power. T. M. D'Zmura 12

13 Between-Subjects Design Levels of an independent variable are represented by different groups of subjects. Example: clinical drug-testing. Random assignment to experimental and control groups of subjects. Within-Subjects Design Effects of varying the level of an independent variable are examined for single subjects. Example: visual psychophysics ability to detect differences in the colors of lights Mixed Design 1.There are at least two independent variables. 2.There is at least one independent variable, the effects of which are determined between subjects (different groups of people) 3.There is at least one (other) independent variable, the effects of which are studied for single subjects (in all groups). T. M. D'Zmura 13

14 Correlation Try the java applet at: T. M. D'Zmura 14

15 Correlation Is Not Causation For example, high sunspot activity seems correlated with economic downturn. Does that mean that high sunspot activity causes economic downturn? NO. T. M. D'Zmura 15

16 Third or Hidden Variable Taking Latin in high school Getting good grades in college Having high academic aptitude and motivation Getting good grades in college Getting good grades in college Going to an excellent high school that caters to college-bound students High school offers Latin High school provides good college prep Taking Latin in high school Getting good grades in college T. M. D'Zmura 16

17 Validity Internal validity refers to the logic and execution of the experiment with respect to the hypotheses that it was designed to test. External validity refers to whether the result (perhaps found in a laboratory) has external relevance to clinical treatment, to practical applications, to other fields of study, etc. Reliability Often assessed using statistical tests, reliability is the consistency of a set of measurements. For instance, if a panel of judges in the Olympics all return the same score for an athlete s performance, then we are more comfortable in thinking that the average score reflects performance well. If there are inconsistencies in scoring then we start wondering about the judges (inter-rater reliability). For instance, if take the SAT test twice and get about the same score both times (hopefully with minor improvement!), then the score is said to be more reliable (test-retest reliability). T. M. D'Zmura 17

18 T. M. D'Zmura 18

The Role and Importance of Research

The Role and Importance of Research What Research Is and Isn t A Model of Scientific Inquiry Different Types of Research Experimental Research What Method to Use When Applied and Basic Research Increasing