META-ANALYSIS. Topic #11

Transcription

1 ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 16C LECTURE NOTES [11/09/16] META-ANALYSIS PAGE 1 Topic #11 META-ANALYSIS Meta-analysis can be escribe as a set of statistical methos for quantitatively aggregating the results of several primary stuies to arrive at an overall summary statement or conclusion about the relationship between specifie variables. Effects of violent vieo games on a number of aggression-relate epenent variables

2 ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 16C LECTURE NOTES [11/09/16] META-ANALYSIS PAGE 2 Calculating the Effect Size Statistics ( or r)! In meta-analysis, cumulating the effects across stuies requires that outcomes from all stuies be converte to a common metric. Two of the most common effect size statistics or metrics are an r. For convenience, we shall focus on.! The statistic provies a measure of the strength of a treatment or inepenent variable (e.g., ifferent training methos). The effect size statistic,, is the stanarize ifference between two means. Thus, in experimental esigns, it represents the observe ifference between the experimental an the control group in stanar eviation units. A positive value inicates that the experimental group performe better than the control group on the epenent variable. Conversely, a negative value inicates that the control group performe better than the experimental group, an a zero value inicates no ifference between the groups. Thus, a of.50 represents half a stanarize ifference between means.! As shown in Formula 1, the statistic is calculate as the ifference between the means of the experimental (M E ) an control groups (M C ) ivie by a measure of the variation. M E M S W C (1)! The measure of variation use in the above formula is S W, which is the poole, within-group stanar eviation.! For stuies that report actual means an stanar eviations for the experimental an control groups, effect sizes can be calculate irectly using these statistics. For stuies that report other statistics (e.g., correlations, t statistics, or univariate two-group F statistics), the appropriate conversion formulas can be use to convert them to s.

3 ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 16C LECTURE NOTES [11/09/16] META-ANALYSIS PAGE 3 Cumulating Effect Sizes Across Stuies! Mean sample-weighte effect sizes () can be calculate using Formula 2 below: ini N T (2) where is the mean effect size; i is the effect size for each stuy; N i is the sample size for each stuy; an N T is the total sample size across all stuies. Sample weighting assigns stuies with larger sample sizes more weight an reuces the effect of sampling error since sampling error generally ecreases as the sample size increases. P EXAMPLE Stuy Ne Me SDe Nc Mc SDc Ntot Sw n Ne & Nc = sample sizes of experimental an control groups, respectively Me & Mc = mean of experimental an control groups, respectively SDe & SDc = stanar eviations of experimental an control groups, respectively Ntot = total sample for specifie stuy ini N T

4 ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 16C LECTURE NOTES [11/09/16] META-ANALYSIS PAGE 4 Avantages of meta-analysis 1. Controls for sampling error by assigning more weight to stuies with larger samples an therefore, obtains more stable effect size estimates. 2. Focuses on the magnitue of effects instea of the statistical significance of effects; significance tests have recently been critique as being a major stumbling block to scientific progress in psychology, an the social sciences in general. 3. Uses a common metric to aggregate effect sizes across stuies. Therefore, meta-analysis summarizes IV/DV effects across multiple stuies to reach a population-level overall conclusion about specifie effects. 4. Ability to summarize large volumes of literature. 5. Can be use to resolve conflicts between two or more boies of literature by comparing effect sizes across them. 6. Ability to investigate relationships not investigate in original primary stuies. 7. Investigate an fin trens too subtle to ientify with narrative reviews. 8. More stanarize an relatively more objective. Disavantages of meta-analysis 1. Garbage in-garbage out. 2. Apples an oranges comparisons. 3. Number of primary stuies available. 4. Selection of primary stuies for inclusion. 5. File rawer problem. 6. Jugement calls.

5 ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 16C LECTURE NOTES [11/09/16] META-ANALYSIS PAGE 5 NOTE. The following is just for your information. You are NOT expecte to know this sequence of steps for the purposes of the exams. Typical sequence of steps involve in implementing a metasanalysis 1. Topic selection efining the research omain 2. Specifying the inclusion criteria 3. Searching for an locating relevant stuies 4. Selecting the final set of stuies 5. Extracting ata an coing stuy characteristics 6. Deciing to keep separate or aggregate multiple ata points (correlations or effect sizes) from the same article inepenence an noninepenence of ata points 7. Testing for an etecting outliers 8. Data analysis calculating mean effect sizes, variability, an correcting for artifacts 9. Deciing to search for moerators 10. Selecting an testing for potential moerators 11. Interpreting results an making conclusions