9 research designs likely for PSYC 2100

Transcription

1 9 research designs likely for PSYC ) 1 factor, 2 levels, 1 group (one group gets both treatment levels) related samples t-test (compare means of 2 levels only) 2) 1 factor, 2 levels, 2 groups (one group for each treatment level) independent measures t-test (compare means of 2 levels only) 3) 1 factor, 3+ levels, 1 group (one group gets all treatment levels) repeated measures ANOVA (compare means of any number of levels) 4) 1 factor, 3+ levels, 3+ groups (one group for each treatment level) independent measures ANOVA (compare means of any number of levels) 5) 2 factors, 2+ levels, 1 group repeated measures factorial ANOVA (2 factors require ANOVA even if factors have only 2 levels) 6) 2 factors, 2+ levels, 4+ groups, independent measures factorial ANOVA (2 factors w/ minimum 2 levels each, so minimum 4 possible combinations, each group gets one combination) 7) 2 factors, 2+ levels, 2+ groups mixed factorial ANOVA (one independent measures factor, one repeated measures factor; number of groups = number of levels in independent measures factor) 8) correlational design correlation coefficient (r) 9) count / frequency / proportion in categories chi-square (χ 2 ) "Factors" are Independent Variables; note that in (5), (6), (7), and (9), there may be more than one IV present in the same experiment. "Levels" are the values of the IV or factor. For example, the IV "Treatment Group" could have two levels, "experimental" and "control"; other IVs may have three or more levels. "Groups" refers to how many sets of participants are given the different IV levels. In some experiments, one group will get all the treatment conditions, so each participant gets all conditions and gives a response under each condition; in others, there may be a different group of participants for each treatment condition, so each participant will get only one condition and give only one response. These can also be combined, as in (7). The DV plays no role in determining the experimental design. The design describes the number and combination of IV treatment conditions and which groups they are given to, and the DV is just the measurement of the participants' responses under those conditions. BONUS: a 10 th design possible for PSYC 2100WQ research projects: regression uses correlations to create an equation that predicts DV scores based on what the IV scores are; "simple regression" means one IV is used to predict the DV, while "multiple regression" means more than one IV is used to predict the DV. These are usually performed as "linear" regression, which applies to cases where the DV is a continuous variable, like a score on an exam.

2 SCIENCE naturalistic explanation all explanation is of nature, in terms of nature allows no entities that have powers beyond other entities, that are not subject to determinism, that are uncaused or otherwise unique our view of nature: scientific materialism only matter in motion exists view of nature resulting from 17th century science SCIENCE AS METHOD systematic empirical observation guided by theory to reveal something about world theory is set of testable propositions has implications for observation organizes past observations guides future observations focuses on solvable problems publicly observable data replication by others using method info peer review in journals

3 DETERMINISM Is Science the Only True Way of Knowing the World? NO Sometimes, we are all guided by authority figures. Sometimes, we just use common sense to get around in the world Sometimes, we accept truths on the basis of belief or faith alone BUT, science is based on direct observation and empirical testing.

4 The science of Psychology seeks to 1. Describe behavior 2. Predict behavior 3. Understand behavior 4. Change behavior How does it accomplish these aims? --by using the Scientific Method From Zechmeister et al. text Goals of Science Describe Predict Understand Control Scientific Method world of concepts real world Theory (inductive thinking) Observation (deductive thinking) Revision Prediction Verification

5 Where do Methods & Statistics Fit? world of concepts Theory (inductive thinking) (deductive thinking) Revision Prediction real world Observation Verification Methods Correlational Experimental Statistics Hypothesis Testing Steps of the Scientific Method 1. Develop a research question 2. Generate a research hypothesis 3. Form operational definitions 4. Choose a research design 5. Evaluate the ethics 6. Collect data 7. Analyze data and form conclusions 8. Report research results These steps are used in both basic and applied research

6 Cyclical Process Report the Results Define the Question Analyze the Data Design the Study Collect the Data Some Terminology experiment vs. correlational study IV vs. DV descriptive vs. inferential statistics sample vs. population statistic vs. parameter H 0 vs. H 1 (or H a ) (hypotheses) Type I vs. Type II error constructs and operational definitions reliability and validity continuous vs. categorical variables scales of measurement

7 Experiment -- involves random assignment of participants and control over the research situation to minimize the influence of other variables and reveal the causal effect of the manipulation. Correlational Study -- examines direction and strength of relationship between variables; no cause implied. Independent variable -- the one manipulated by the experimenter (cause). Dependent variable -- the one measured by the experimenter (effect). Descriptive Statistics -- statistics and methods for organizing and summarizing data. Inferential Statistics -- techniques to permit inferences or generalizations from samples to the populations from which they were drawn. Statistic is to sample as Parameter is to population. Null Hypothesis Significance Testing ask whether observed relationships in sample reflect true population relationships, or mere natural sampling variability null hypothesis H 0 : default description of data relationships in population - can it be rejected on basis of sample? alternative hypothesis H 1 (or H a ): any data relationship in population other than what H 0 specifies Type I error - conclude H 0 false when it's true Type II error - conclude H 0 true when it's false "significance" - conventionally, "p<.05": less than 5% probability of observing this data (or data more extreme) if H 0 is true, which leads us to reject H 0

8 two major problems in psychological research measurement problem: relation between constructs and operational definitions is not as tight as in other natural sciences, making construct validity an important issue noise problem: inherent variability among individuals, and within individuals from occasion to occasion, makes it impossible to attain exact group equivalence or replication and obscures effects of independent variables of interest; makes internal validity issues especially important (e.g., random assignment, ruling out confounds, etc.) Reliability The consistency or repeatability of a measure The degree to which a measure would give you the same result over and over, assuming the phenomenon being measured is not changing Cannot be calculated, only estimated [Based on true score theory of measurement (Trochim pp )]

9 three types of validity (there are many others) construct validity (addresses measurement problem) - relation between constructs and operational definitions; consider exams, SATs, behavioral vs MRI measures of cognitive processing; includes "face validity" or how good the measure SEEMS to reflect the construct on the surface internal validity (addresses noise problem, among others) - use of random assignment and other aspects of experimental method to ensure legitimate conclusions external validity (concerned with applying experiment's conclusions to real world) - use of random selection of participants so they represent the population accurately; includes "ecological validity" or similarity of processes in lab setting to the real world processes being investigated Construct Validity Construct validity is the approximate truth of the conclusion that your operationalization accurately reflects its constructs. Central questions to ask are: Is your operationalization an accurate translation of the construct? Does your program/treatment accurately reflect what you intended? Does your sample accurately represent your idea of the population of interest? Are you measuring what you intended to measure?

10 Internal Validity The approximate truth about inferences regarding cause-effect (causal) relationships (Cook & Campbell,1979) The primary consideration in establishing cause and effect Key question: Can observed changes (effect) be attributed to the program or intervention (cause) and not some other possible (alternative) cause? Only relevant to the specific study in question (i.e., is not concerned with generalizability) Random Selection and Random Assignment Random selection is how you draw the sample of people for your study from a population impacts external validity. Helps insure that the sample is representative of the population (and hence, findings are more generalizable) Random assignment is how you assign the sample to different groups or treatments in your study impacts internal validity. Helps insure that groups are comparable at the beginning of the study

11 Reliability and Validity types of research design: correlational vs. experimental correlational design typically examines how 2 variables go together in a single group no casuality implied because no control is assumed, and confounds and spurious or coincidental relationships are probably present

12 types of research design: correlational vs. experimental experimental design typically compares mean DV scores of 2 or more groups intent is to change one thing between the groups and then attribute group differences on the dependent variable to the difference in treatments (independent variable) "change ONE thing" (manipulation) implies "keep everything else the same" (control) when random assignment and other appropriate controls are in place, the manipulation of the IV allows causal conclusions to be drawn when participants are not randomly assigned to treatments, the method is only superficially experimental and is called "quasi-experimental" experimental control physical control (for environmental variables, not participant variables): temperature, lighting conditions, time of day, noise levels control by experimental design...

13 experimental control: control by experimental design hold constant (for environmental variables, some subject variables): temperature, lighting; age, sex; not really IQ (even if measurement were accurate, you wouldn't choose only people with IQ = 126); definitely not anxiety or authoritarianism or depression matching (for environmental variables and explicitly measured participant variables): have corresponding subjects (e.g., similar IQ) in each treatment group so groups are equal on average (equated at individual or group level); groups may still differ on unsuspected variables random assignment (for all variables): all characteristics, known or unknown, are randomly spread across all groups so they're the same on average nuisance variability (nuisance variables): factors affecting scores on the DV other than the factor you're interested in unsystematic nuisance variability doesn't affect one group more than another or bias scores or correlations to be higher or lower - just adds to variability (noise) you're trying to see through systematic nuisance variability does affect one group more than another or bias scores or correlations to be higher or lower - confound: don't know which factor to attribute DV differences to random assignment converts systematic nuisance variability into unsystematic by distributing it randomly among all groups

14 Scales of Measurement nominal: assign labels to categories ordinal: assign order to categories interval: ordinal, and includes equal distances; no real zero ratio: interval, and includes an absolute zero Scales of Measurement nominal: assign labels to categories ex: car color, sex, religion, ethnicity ordinal: assign order to categories ex: reading grade level; exam finishing order interval: ordinal, and includes equal distances; no real zero ex: Fahrenheit temperature; IQ, SAT (?) ratio: interval, and includes an absolute zero ex: Kelvin temperature; height; reaction time MUST HAVE interval or ratio level measurement to calculate mean, standard deviation, etc

15 Central Tendency mean: sum of scores divided by number of scores use for: interval and ratio scale data, whenever possible median: half of all scores fall above, half below; if even number of scores, median is mean of middle two scores : : 11 use for: extreme values; skewed distribution; undetermined values ( never finished ); open-ended categories ( 5 or more ); ordinal scale data ( half read above 8 th grade level, half below ) mode: most frequent score; may be more than one ( bimodal ) : : 7 & 15 use for: nominal scale data; describing shape and informally when more than one peak even if not the same (lower peak and higher peak) Statistics quiz scores for a section of n = 8 students.

16 Examples of different shapes for distributions. Amount of time to complete puzzle.

17 Median cost of a new, single-family home by region. A bar graph showing the distribution of personality types in a sample of college students. Because personality type is a discrete variable measured on a nominal scale, the graph is drawn with space between the bars.

18 A grouped frequency distribution table showing the data from Example 2.3. The original scores range from a high of X = 94 to a low of X = 53. This range has been divided into 9 intervals with each interval exactly 5 points wide. The frequency column (f) lists the number of individuals with scores in each of the class intervals. An example of a frequency distribution histogram. The same set of data is presented in a frequency distribution table and in a histogram.

19 A frequency distribution histogram showing the heights for a sample of n = 20 adults. An example of a frequency distribution histogram for grouped data. The same set of data is presented in a grouped frequency distribution table and in a histogram.

20 The population distribution of IQ scores: an example of a normal distribution. Measures of central tendency for skewed distributions.

21 The statistical model for defining abnormal behavior. The distribution of behavior scores for the entire population is divided into three sections. Those individuals with average scores are defined as normal, and individuals who show extreme deviation from average are defined as abnormal. Frequency distribution for a population of N = 16 scores. The first quartile is Q1 = 4.5. The third quartile is Q3 = 8.0. The interquartile range is 3.5 points. Note that the third quartile (Q3) divides the two boxes at X = 8 exactly in half, so that a total of 4 boxes are above Q3 and 12 boxes are below it.

22 Population distributions of adult heights and adult weights.

23 The graphic representation of a population with a mean of µ = 40 and a standard deviation of σ = 4. The population of adult heights forms a normal distribution. If you select a sample from this population, you are most likely to obtain individuals who are near average in height. As a result, the scores n the sample will be less variable (spread out) than the scores in the population.