SUMMATED RATING SCALES AND LEVELS OF MEASUREMENT

Transcription

1 Measurement, Scaling, and Dimensional Analysis Summer 07 Bill Jacoby SUMMATED RATING SCALES AND LEVELS OF MEASUREMENT Assume that we are interested in measuring public attitudes toward government spending. A public opinion survey (the 99 CPS National Election Study) asked a national sample of 50 respondents whether federal spending should be increased, kept the same, or decreased in each of several policy areas. Specifically, respondents were asked about spending on: Welfare; AIDS research; programs to help Blacks; assistance to the countries of the former Soviet Union; and child care. For each of these program areas, the numeric response codes were as follows:. Decrease spending. Keep spending at current levels. Increase spending In order to measure attitudes toward government spending, we will construct a summated rating scale from these responses. Actually, we will try two versions of the summated rating scale: The first uses only twoof the items (spending on AIDS research, and on programs to help Blacks), while the second scale incorporates the information from all five of the items. This dataset is very unusual in that it also contains the true, error-free values of the respondents' overall attitudes toward government spending. In any real research situation, this latter information is unavailable (since the true, underlying dimension is unobservable, by definition). Nevertheless, the true scores on the dimension of interest are available here, so we can use this information to construct empirical item response functions (also known as trace lines or item characteristic curves) for each of the program-specific spending items. These are shown in Figures through 5. Note that these trace lines all exhibit monotonic relationships with the underlying dimension; hence, we are justified in our assumption that the program-specific spending items constitute ordinal measurement of the unobserved attitude. Next, Figures 6 and 7 show the trace lines for the two-item and five-item summated rating scales, respectively. Note that these trace lines are almost perfectly linear. Therefore, they indicate that we have achieved interval-level measurement of attitudes toward spending. Also, note that the linear pattern is more clearly visible in Figure 7 than in Figure 6. This is an empirical demonstration of the principle that longer scales have higher reliability. Figures 8 and 9 reiterate this principle, by showing the residual-fit spreadplots that are obtained when the two-item and five-item scales are each regressed on the underlying dimension. Finally, this handout contains some information on the intercorrelations of the scales, the underlying dimension, and the individual items, in order to assess the scale reliabilities.

2 Page Figure Figure Spending on Welfare Spending on AIDS Research Figure Figure 4 Spending on Programs to Help Blacks Spending on Programs to Help the Former USSR Figure 5 Spending on Child Care

3 Page Figure 6.0 Two-Item Scale of Spending Attitudes Figure 7.0 Five-Item Scale of Spending Attitudes

4 Page 4 Figure 8: Residual-fit spreadplot for the linear relationship between the two-item summated rating scale, and the true, underlying, attitude Fitted Values minus Mean Residuals Quantiles of the Distribution Figure 9: Residual-fit spreadplot for the linear relationship between the five-item summated rating scale, and the true, underlying, attitude Fitted Values minus Mean Residuals Quantiles of the Distribution

5 Page 5 The correlation between the two-item scale (composed of responses to spending on AIDS research and on programs to help Blacks) and the underlying dimension is The correlation between the five-item scale (composed of responses to spending on welfare, AIDS research, programs to help Blacks, assistance to the countries of the former Soviet Union, and child care) and the underlying dimension is Correlation Matrix for Individual Items and Underlying Dimension Welfare AIDS Research Help Blacks Assist Former USSR Child Care Welfare.000 AIDS Research Help Blacks Assist Former USSR Child Care True Dimension