Day 11: Measures of Association and ANOVA

Transcription

1 Day 11: Measures of Association and ANOVA Daniel J. Mallinson School of Public Affairs Penn State Harrisburg PADM-HADM 503 Mallinson Day 11 November 2, / 45

2 Road map Measures of Association Types of Measures Examples Analysis of Variance (ANOVA) Mallinson Day 11 November 2, / 45

3 General Notes Remember the three following questions: 1 Is the relationship between the variables significant? Conduct a significance test 2 How strong is the relationship? Use a measure of association 3 What is the nature of the relationship between the variables? Interpret outputs of your analyses: charts, tables, mathematical formulas Mallinson Day 11 November 2, / 45

4 General Notes Mallinson Day 11 November 2, / 45

5 General Notes Significance Tests for Measures of Association: SPSS displays results of statistical tests for measures of association Shows if the calculated measures an association that appears by chance or is real Ignore them! - Pointless to discuss, does not replace t-tests, ANOVA, or chi-square Mallinson Day 11 November 2, / 45

6 Choosing a Measure of Association Need to select appropriate test based on level of measurement of IV(s) and DV Need to consider the measure s sensitivity (more on this later) Researcher should be familiar with the chosen statistic Mallinson Day 11 November 2, / 45

7 Choosing a Measure of Association Asymmetric or symmetric? Asymmetric Measures Preferred when you know which variable is the IV and which is the DV Symmetric Measures Choose when you do not know which is IV and which is DV or when a symmetric measure is not available. Choose asymmetric measures when they are available! Mallinson Day 11 November 2, / 45

8 Choosing a Measure of Association Several choices for calculating measures of association: Proportional reduction of error (PRE) measures Chi-square based measures Correlational measures Specific measures for ANOVA See Table 13.8 in the textbook. Mallinson Day 11 November 2, / 45

9 How to Interpret Levels of Association Perfect positive relationship between variables: +1.0 Perfect negative relationship between variables: -1.0 No relationship between variables = 0 In general: The closer to 0, the weaker the relationship The closer to ±1, the stronger the relationship Mallinson Day 11 November 2, / 45

10 How to Interpret Levels of Association There is no universal scale to determine if a relationship is strong or weak Guidelines exist for some measures See Table 13.8 in textbook Mallinson Day 11 November 2, / 45

11 Types of Measures of Association Proportional reduction of error (PRE) measures Chi-square based measures Correlational measures Specific measures for ANOVA Mallinson Day 11 November 2, / 45

12 Types of Measures of Association Level of Measurement Measure of Assoc Type Symmetric Nominal Lambda PRE Both Phi Coefficient χ 2 Symmetric Coef. of Contingency χ 2 Symmetric Cramer s V χ 2 Symmetric Ordinal Gamma PRE Symmetric Tau-b (square) PRE Symmetric Tau-c (rectangle) PRE Symmetric Somers d PRE Asymmetric Spearman s Rho (ρ) Correlation N/A Interval Pearson s r Correlation N/A Eta and Eta 2 ANOVA N/A Bold measures are most likely candidates for use Rule of thumb: report several if available, note the differences Mallinson Day 11 November 2, / 45

13 Types of Measures of Association 1. PRE Indicate how much knowing the IV decreases errors in estimating the values of the DV Conservative measure: Yield lower values than chi-square based measures, thus less likely to overestimate the strength of association Lambda sometimes underestimates the strength of a relationship, can yield 0 even when significance test shows a significant relationship. Cramer s V preferred to Lambda Report both in tables, talk about Cramer s V in interpretation Mallinson Day 11 November 2, / 45

14 Types of Measures of Association 2. Measures Based on Chi-Square Difficult to interpret, not intuitive Cramer s V is most relevant of the three Mallinson Day 11 November 2, / 45

15 Types of Measures of Association 3. Correlation-Based Measures Spearman s ρ is a relatively old measure Kendall s Tau-b is usually prefered over Spearman s ρ when IVs and DVs are ordinal Mallinson Day 11 November 2, / 45

16 Types of Measures of Association 4. ANOVA Eta and Eta 2 Used to measure strength of relationship in one-way ANOVA Eta 2 interpreted as proportion of variance explained in DV by the IV Similar to R 2 in multiple regression Mallinson Day 11 November 2, / 45

17 Examples of Measures of Association Example 1: IV and DV Nominal Data file: gssnet.sav Research Question: Are men or women more likely to use ? To answer, we use data from General Social Survey dataset Variables: Respondent s sex (sex) and Use (use ). There are two categories of the DV (yes and no) Mallinson Day 11 November 2, / 45

18 Examples of Measures of Association Steps: 1. State hypotheses Research hypothesis: There is a difference between mean and women in their usages of Null hypothesis: There is not difference. 2. Select and alpha level α = 0.05 Mallinson Day 11 November 2, / 45

19 Examples of Measures of Association Steps: 3. Compute test of statistical significance Chi-square 4. Make a decision If p <.05, there is a significant relationship 5. Interpret strength of the relationship If there is a significant relationship, interpret Lambda and Cramer s V Mallinson Day 11 November 2, / 45

20 Examples of Measures of Association In SPSS: Descriptive Statistics Cross Tabs Select a column variable (IV) and a row variable (DV) Click Statistics and select Chi-square, also select Phi and Cramer s V and Lambda under Nominal Click Cells and select observed counts, expected counts, and column percentages Mallinson Day 11 November 2, / 45

21 Examples of Measures of Association Results: There is a difference between men and women Mallinson Day 11 November 2, / 45

22 Examples of Measures of Association Results: The difference is significant Mallinson Day 11 November 2, / 45

23 Examples of Measures of Association Results: Lambda is erroneous, so interpret Cramer s V; difference is significant Mallinson Day 11 November 2, / 45

24 Examples of Measures of Association Example 1 Interpretation There is a significant relationship between sex and usage The relationship is very weak Men are more likely to use messages We can reject our null hypothesis We can be confidence of this conclusion 95% Mallinson Day 11 November 2, / 45

25 Examples of Measures of Association Example 4: Independent and Dependent Variables Scale Data file: country.sav RQ: Does the availability of doctors in a country make any difference in the female life expectancy in that country? Variables in the dataset: doctors per 10,000 people (docs) and female life expectancy (lifeexpf) Mallinson Day 11 November 2, / 45

26 Examples of Measures of Association Steps: This time we will not follow the steps from earlier examples No hypothesis, for example The purpose is to show how Pearson s r is calculated and to show a visual association between the variables (scatterplot) We need to conduct a regression analysis to establish the causal relations between the variables Mallinson Day 11 November 2, / 45

27 Examples of Measures of Association In SPSS: Correlation Bivariate Select the two variables: doctors per 10,000 people and female life expectancy Also select Pearson under Correlation Coefficients Mallinson Day 11 November 2, / 45

28 Examples of Measures of Association In SPSS: For a visual illustration (scatterplot) Graphs Legacy Dialog Scatter/Dot Simple Scatter Define Enter doctors per 10,000 people as the X axis and female life expectancy as the Y axis Mallinson Day 11 November 2, / 45

29 Examples of Measures of Association Results: Positive association between the variables. How strong? Mallinson Day 11 November 2, / 45

30 Examples of Measures of Association Results: Pearson s r is fairly strong. We will leave further interpretation to our discussion of regression. Mallinson Day 11 November 2, / 45

31 Analysis of Variance (ANOVA) ANOVA is similar to a t-test The IV is nominal, the DV is scale ANOVA is used when the IV has more than two groups Makes overall comparisons among the groups of the IV Mallinson Day 11 November 2, / 45

32 Analysis of Variance (ANOVA) Also makes comparisons between the pairs of groups Can be used with two groups, but produces identical results to t-test Can calculate the strength of the statistical relationship between IV and DV (measure of association) Mallinson Day 11 November 2, / 45

33 Analysis of Variance (ANOVA) Need to keep the assumptions of ANOVA in mind: 1 DV must be scale-measured 2 Variances among groups of the IV should be equal 3 Each group normally distributed within itself 4 Groups should be independent of each other (no pre-post designs) Mallinson Day 11 November 2, / 45

34 Analysis of Variance (ANOVA) Steps in conducting an ANOVA test: 1 Plot an error bar chart to visual inspect the differences among groups 2 Describe group characteristics (mean values for each group) 3 Interpret the ANOVA table for overall differences among the groups 4 If the F-test is significant, then run the Levene s test (homogeneity of variance test), to determine the kind of post-hoc test you should use 5 Run the appropriate post-hot test (for pairwise group comparisons) Mallinson Day 11 November 2, / 45

35 Analysis of Variance (ANOVA) An SPSS example: Dataset: gssft.sav IV: Highest degree (degree) DV: Number of hours worked last week (hrs1) Mallinson Day 11 November 2, / 45

36 Analysis of Variance (ANOVA) Step 1. Generating an error bar graph in SPSS: Graphs Legacy Dialog Error Bar Simple Select Summaries for groups of cases Define Select variables (IV to category axis and DV to variable ) Accept confidence interval for mean under Bar Represents Mallinson Day 11 November 2, / 45

37 Analysis of Variance (ANOVA) Bars show 95% confidence intervals Bars do not represent variances, but because standard errors are used to calculate them they are approximations of variances Arithmetic means shown in middle Mallinson Day 11 November 2, / 45

38 Analysis of Variance (ANOVA) Step 2. Describe the group characteristics and Step 3. Run the ANOVA test Analyze Compare means One-way ANOVA Assign your DV to Dependent List and your IV to Factor Under Options, select Descriptive Mallinson Day 11 November 2, / 45

39 Analysis of Variance (ANOVA) Mallinson Day 11 November 2, / 45

40 Analysis of Variance (ANOVA) Step 4. If the test is significant, run homogeneity of variance test (Levene Test) Analyze Compare means One-way ANOVA Under Options, select Homogeneity of variance test If the test is not significant, equal variances must be assumed Mallinson Day 11 November 2, / 45

41 Analysis of Variance (ANOVA) Step 4: Run the appropriate post-hot test Equal Variances The Bonferroni procedure is usually recommended for multiple comparisons when the variances of samples are roughly equal Unequal Variances Use Dunnett T3 or Tamhane Mallinson Day 11 November 2, / 45

42 Analysis of Variance (ANOVA) Step 4: Run the appropriate post-hot test Can you use a series of t-tests, instead of using the pair-wise comparisons in ANOVA? Statisticians tell us this will create a multiple comparison problem (i.e., increased risk of rejecting the null when it is true Type I Error) O Sullivan et al. say the opposite - Ignore them! Mallinson Day 11 November 2, / 45

43 Analysis of Variance (ANOVA) Step 4: Run the appropriate post-hot test in SPSS Analyze Compare means One-way ANOVA Under Post-Hoc tests, select either an equal variance (Bonferroni) or an unequal variance (Tamhane or Dunnet T3) test Mallinson Day 11 November 2, / 45

44 Analysis of Variance (ANOVA) Look at values under Sig. Those less than.05 indicate pair of groups that are different from each other In this example, only Graduate and High school categories are significantly different from each other Mallinson Day 11 November 2, / 45

45 Lab/Homework Problem 1 Using the gssft.sav dataset, choose another ordinal variable that you believe is associated with general happiness. Lay out of the four steps of significance testing (hypotheses, alpha, test, decision). Make sure you choose and defend your chosen measure of association and correctly interpret your results. Problem 2 Now, using the same dataset, choose a scale variable that you believe is associated with general happiness. Again, lay out all four of the steps of significance testing. Make sure you choose the correct post hoc test based on the equal variances test. Interpret your results. Mallinson Day 11 November 2, / 45

46 Appendix Slides Additional Measures of Association examples Mallinson Day 11 November 2, / 45

47 Examples of Measures of Association Example 2: Independent and Dependent Variables Ordinal (Rectangular Table) Data file: gssnet.sav RQ: Does more education made you happier? Variables in the dataset: respondent s highest degree (degree) and general happiness (happy) Mallinson Day 11 November 2, / 45

48 Examples of Measures of Association Steps: 1. State hypotheses Research hypothesis: There is a positive relationship between education level and general happiness. This is a directional hypothesis. Null hypothesis: There is no relationship. 2. Select and alpha level α = 0.05 Mallinson Day 11 November 2, / 45

49 Examples of Measures of Association Steps: 3. Compute test of statistical significance Chi-square 4. Make a decision If p <.05, there is a significant relationship 5. Interpret strength of the relationship If there is a significant relationship, interpret Somers d, Kendall s tau-c, and gamma Mallinson Day 11 November 2, / 45

50 Examples of Measures of Association In SPSS: Descriptive Statistics Cross Tabs Select a column variable (IV) and a row variable (DV) Click Statistics and select Chi-square, also select Somers d, Gamma, and Kendall s tau-c under Ordinal Click Cells and select observed counts and column percentages (no expected counts this time) Mallinson Day 11 November 2, / 45

51 Examples of Measures of Association Results: Education seems to make a difference in happiness Mallinson Day 11 November 2, / 45

52 Examples of Measures of Association Results: The relationship is significant Mallinson Day 11 November 2, / 45

53 Examples of Measures of Association Results: Somers d and Kendall s tau-c agree; gamma exaggerates; prefer Somers d as it is directional Mallinson Day 11 November 2, / 45

54 Examples of Measures of Association Example 2 Interpretation There is a significant relationship between educational degree and general happiness The relationship is weak The relationship is positive (as education increases, so does general happiness) We can reject our null hypothesis We can be confidence of this conclusion 95% Mallinson Day 11 November 2, / 45

55 Examples of Measures of Association Example 3: Independent and Dependent Variables Ordinal (Square Table) Data file: gssnet.sav RQ: Does happiness in marriage you happier in general? Variables in the dataset: happiness of marriage (hapmar) and general happiness (happy) Mallinson Day 11 November 2, / 45

56 Examples of Measures of Association Steps: 1. State hypotheses Research hypothesis: There is a positive relationship between happiness in marriage and general happiness. This is a directional hypothesis. Null hypothesis: There is no relationship. 2. Select and alpha level α = 0.05 Mallinson Day 11 November 2, / 45

57 Examples of Measures of Association Steps: 3. Compute test of statistical significance Chi-square 4. Make a decision If p <.05, there is a significant relationship 5. Interpret strength of the relationship If there is a significant relationship, interpret Somers d, Kendall s tau-b, gamma, and Spearman s Rho Mallinson Day 11 November 2, / 45

58 Examples of Measures of Association In SPSS: Descriptive Statistics Cross Tabs Select a column variable (IV) and a row variable (DV) Click Statistics and select Chi-square, also select Somers d, Gamma, and Kendall s tau-c under Ordinal Click Cells and select observed counts and column percentages (no expected counts this time) Mallinson Day 11 November 2, / 45

59 Examples of Measures of Association In SPSS: To compute Spearman s Rho, you will need to go through another path: Correlation Bivariate Select two variables: Happiness of marriage and general happiness Also select Spearman under Correlation Coefficients Mallinson Day 11 November 2, / 45

60 Examples of Measures of Association Results: There seems to be a relationship Mallinson Day 11 November 2, / 45

61 Examples of Measures of Association Results: The relationship is significant Mallinson Day 11 November 2, / 45

62 Examples of Measures of Association Results: Mallinson Day 11 November 2, / 45

63 Examples of Measures of Association Results: Somers d, tau-b, and Spearman are all similar. Shows a moderately strong relationship. Mallinson Day 11 November 2, / 45

64 Examples of Measures of Association Example 3 Interpretation There is a significant relationship between happiness in marriage and general happiness The relationship is moderately strong The relationship is positive (as happiness in marriage increases, so does general happiness) We can reject our null hypothesis We can be confidence of this conclusion 95% Mallinson Day 11 November 2, / 45