RESEARCH NOTE. University of Michigan, USA

Transcription

1 International Journal of Public Opinion Research Vol. 22 No. 4 ß The Author Published by Oxford University Press on behalf of The World Association for Public Opinion Research. All rights reserved. doi: /ijpor/edq037 Advance Access publication 25 October 2010 RESEARCH NOTE The Relation Between Unit Nonresponse and Item Nonresponse: A Response Continuum Perspective Ting Yan 1 and Richard Curtin 2 1 NORC at the University of Chicago, USA and 2 Survey Research Center, University of Michigan, USA Nonresponse is a significant problem for survey research. The phenomena encompass nonresponse at both the unit and the item level. Unit nonresponse refers to the complete absence of an interview from a sampled household whereas item nonresponse refers to the absence of answers to specific questions in the interview after the sampled household agrees to participate in the survey. Traditionally, survey research has treated unit and item nonresponse as two separate problems with different impacts on data quality, different statistical treatments and adjustments, and different underlying causes (Beatty & Herrmann, 2002; Groves, 1989; Groves, Cialdini & Couper, 1992; Groves, Presser, & Dipko, 2004; Gorves, Singer, & Corning, 2000). Unit nonresponse is usually considered to pose a much greater threat to survey research than item nonresponse. Quantitatively, unit nonresponse is usually much larger than item nonresponse and there are a wide variety of statistical techniques that have been developed to address unit nonresponse. Item nonresponse poses additional threat to data quality because it reduces sample size if only completed cases are used in an analysis. Imputation techniques can be employed to impute for missing data to avoid the shrinkage of sample size. However, it is still problematic for surveys when item nonresponse produces nonignorable missing data. Nonignorable missing data happen when the missing data pattern is correlated with the values of the variable of interest (Little & Rubin, 1987). Of course, unit nonresponse can also create nonignorable nonresponse if the missing pattern in sample persons is related to the values of the variable of interest. The survey literature has recorded a persistent and well-documented rise over the past few decades in unit nonresponse rates for household surveys (Atrostic, Bates, Burt, & Silberstein, 2001; Curtin, Presser, & Singer, 2005; de Heer, 1999; de Leeuw All correspondence concerning this article should be addressed to Ting Yan, NORC at the University of Chicago, 55 East Monroe Street, Chicago, IL, 60603, USA. yan-ting@norc.org

2 536 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH & de Heer, 2002; Djerf, 2004; Hox & de Leeuw, 1994). But the trend of item nonresponse over time is less well documented. Furthermore, the literature on the interconnection between unit and item nonresponse is sparse. The empirical evidence seems to unanimously point to a positive relation between respondents propensity to not participate in a survey and the amount of item nonresponse. For instance, respondents who initially refused but were later successfully converted had higher item missing data rates than initial cooperators (Mason, Lesser, & Traugott, 2002; Stinchcombe, Jone, & Sheatsley, 1981). Respondents who expressed disinterest initially at survey introduction and who were coded to be reluctant to take part by interviewers were also more likely to produce missing data than those who did not show disinterest (Burton, Laurie, & Moon, 1999; Campanelli, Sturgis, & Moon, 1996; Couper, 1997). In addition, respondents who had a higher item missing data in an earlier wave tended to refuse to grant a subsequent interview (Loosveldt, Pickery, & Billiet, 2002). However, the theoretical aspect of the interconnection between unit and item nonresponse is less well developed. The traditional view is that the causes and the underlying processes are different for unit and item nonresponse; Groves and Couper (1998) believe that the influences towards item nonresponse are quite different from those of the initial acceptance of the interview (p. 22). However, this view does not seem to be supported by the observed positive relation between unit and item nonresponse propensity. As a competing alternative view, we posit a response continuum model to link unit and item nonresponse (also see a similar model linking item nonresponse and unit nonresponse illustrated in Burton et al., 1999). This model places respondents on a response continuum based on their propensity to respond to a survey request and to answer survey questions. Theoretically speaking, respondents with a zero propensity to take part in a survey are placed on the left and those who are the most cooperative and will participate with certainty in a survey and answer all questions occupy the other end of the continuum. As we move from left to right on the continuum, respondents propensity to participate in a survey increase and so does their propensity to answer survey questions, as shown in Figure 1. Other than at the extremes, the figure is not meant to indicate the model implies the same quantitative probability for unit and item nonresponse. The figure is only meant to indicate differences in relative probabilities. We also acknowledge that a sample person s propensity to respond to a survey request and to answer a survey question is conditional on survey conditions. Any change in survey design features such as the mode of administration, interviewer assignment, and interviewer behavior will change a sample person s propensity to take part in a survey and/or to answer a survey question, moving him/her to a different location on our response continuum. Figure 1 The response continuum model Zero propensity to respond to interview & questions Low relative propensity to respond to interview & questions High relative propensity to respond to interview & questions Certain propensity to respond to interview & questions

3 RESEARCH NOTE 537 The unique thing about the response continuum model is its focus on common antecedents shared by unit and item nonresponse. Survey practitioners have long recognized the connection between unit and item nonresponse in their judgments about whether a respondent answered a sufficient number of questions to reach the minimum requirements to qualify as a completed or partial interview based on American Association for Public Opinion Research guidelines. These judgments represent choosing a point on the response continuum. At the individual level, the response continuum model predicts a positive relation between a sample member s propensity to respond to a survey request and his/her propensity to answer survey questions. This individual-level prediction manifests itself in two different forms. On the one hand, according to the response continuum model, sample persons who have a low propensity to answer survey questions also have a low propensity to cooperate with a survey request. Research in the context of longitudinal surveys has found evidence that sample persons who are more likely to provide missing data in an earlier wave of a longitudinal survey are less likely to participate in the next wave (Loosveldt et al., 2002; Yan & Curtin, 2007). On the other hand, the model predicts that respondents who are more likely to not participate in a survey are also more likely to provide missing data. This relationship is often observed in cross-sectional surveys (Campanelli et al., 1996; Couper, 1997; Mason et al., 2002; Stinchcombe et al., 1981). Both manifestations of the positive relation between individual respondent s propensity to respond to surveys and propensity to answer survey questions are important for survey researchers. For instance, respondents with high levels of item nonresponse in one wave of a longitudinal survey can be given special treatments such as higher incentives or special advance letters so as to increase their chance to take part in the next wave. Similarly, sample persons who require more effort to agree to the survey request could be assigned interviewers trained to reduce their propensity to provide missing data. The response continuum model can, thus, be used in responsive designs to help survey practitioners to make informed survey design decisions. In addition to individual level relationship, the response continuum model can also generate predictions at the aggregate level (i.e., at the survey level). In terms of a cross-sectional survey, the response continuum model predicts a trade-off between unit and item nonresponse rates. Surveys with lower unit nonresponse rates, which presumably include more people with a low propensity to respond, will have higher item nonresponse rates. Thus, at the survey level, there exists a negative relation between unit and item nonresponse rates. This negative relation has great practical interest. One concern that survey practitioners have about expending effort to reduce unit nonresponse rates is that people with a low propensity to participate in surveys may transfer their resistance into the question answer process by expending less effort on their answers. It is conceivable, for example, that a sample person who is bothered by an interviewer s persistent callback decides to do the survey but to answer as few questions as possible. A common concern among survey practitioners is whether bringing these people into the respondent pool would reduce data quality. With limited resources, the survey practitioners have to stop the effort to recruit difficult sample persons at some point. The decision of when to stop should take into

4 538 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH consideration this trade-off between unit and item nonresponse. To our best knowledge, there are few studies looking into the relation between unit and item nonresponse rates. This article has two main goals. The first aim of the article is to examine the trend of item nonresponse over 20 years. Specifically, we investigate the trend of item nonresponse in face of increasing unit nonresponse experienced by most surveys in the recent decades. Second, we examine the relation between unit and item nonresponse rates by using the response continuum model. We first test the response continuum model with cross-sectional respondent-level data. We then expand the model in two ways by testing the aggregate level hypotheses of the negative relation between unit and item nonresponse rates and by examining the trend of this relation over time. The Data Twenty years of data from the Surveys of Consumers (SCA), conducted by the University of Michigan, are used to examine the relationship between trends in unit and item nonresponse. Data were collected from 1986 to 2005, when the wording of the core set of questions asked in the SCA remained unchanged. Utilizing this database has advantages and disadvantages for this examination. The main disadvantage is that all the analysis is solely focused on the prevalence of missing data on questions about economic expectations. Clearly, the findings may not generalize to other attitudes or other behavioral measures. The advantages of using these data are that the sample design and survey methodology has been constant. The survey design factors are an important source of differences in item nonresponse, such as the mode of the interview, the complexity of the questions, and so forth. More importantly, holding these factors constant allows the analysis to get a net effect of unit nonresponse on item nonresponse. The SCA started out as an area-probability in-home survey in the mid-1940s, and was converted to a random-digit dial-telephone survey in the mid-1970s. The survey is conducted monthly, and is based on a rotating-panel design. The analysis is restricted to the newly drawn representative samples. This restriction is consistent with published work on the survey (e.g., Curtin, Presser, & Singer, 2000, 2005; Singer, Van Hoewyk, & Maher, 2000). The new cases were selected until 1993 using Mitofsky Waksberg procedures (Waksberg, 1978; for random digit dialing sampling methods, see Wolter, Chowdhury, & Kelly, 2009) and since then using list-assisted procedures. From each household, one respondent has been randomly selected from among all household residents aged 18 years or older. About 300 new interviews are now conducted each month. Except for the constraint imposed by the month-long interviewing period, no limit is placed on the number of calls, and attempts are made to convert virtually all initial refusals. For information on the historical trend of unit response rates, see Curtin et al. (2000, 2005). The core part of the SCA assesses respondents current and future financial situation, current and future economic conditions, and expectations for inflation, unemployment, interest rate, and attitudes toward buying condition. These questions

5 RESEARCH NOTE 539 are components to the Index of Consumer Sentiment and are asked at the beginning of each monthly survey; the wordings of these core questions (displayed in Appendix A) have remained unchanged for the past 20 years. Prevalence of Item Nonresponse We first present results on the prevalence of item nonresponse and its trend over the past 20 years before discussing the interconnection between unit and item nonresponse rates. We calculated an index of item nonresponse based on responses to 16 of the core questions asked each month over the 20-year period. The index represents the average number of times the respondents refused to answer or answered don t know to these 16 items. This index is constructed in the same way as used in Singer et al. (2000) except that their index was based on 17 questions; we removed one question from our index since it was not asked over the entire 20-year period. The only non-attitudinal component item to our index is the income question, which has a reputation for incurring higher item nonresponse. The Pearson bivariate correlation of the item nonresponse index with and without the income question is.94. Excluding the income question from the item nonresponse index does not change the conclusions reported below. Thus, we kept the income question in our item nonresponse index to be consistent with published literature (i.e., Singer et al., 2000). The item nonresponse percentage for each of the 16 items in the index is displayed in Appendix B. We also analyzed don t know and refused responses separately. The conclusions were unchanged whether don t know and refused responses were combined or separated. Therefore, we combined the two categories in the results reported below. The index of item nonresponse measures the extent of missing values at the respondent level; that is, the percent of times a respondent offered a missing value (rather than a substantive value) or the percent of missing values a respondent provided. Over the 20-year period, from June 1986 to December 2005, the overall mean value on the index of item nonresponse is 0.044, which means that, on average, respondents provided missing values 4.4% of the time, or to 0.7 survey questions (i.e., ¼ 0.704). Figure 2 shows the time trend in the yearly average index values of item nonresponse (i.e., the top line in the figure corresponding to the left vertical axis), which ranges from 2.4 to 5.5%. (The trend presented in Figure 2 is based on unweighted data. The weighted item nonresponse rates were also examined to determine the effect of differential nonresponse rates across population subgroups over the 20-year period. There was no significant difference between the weighted and the unweighted trends of item nonresponse. As a result, the rest of analyses are based on the unweighted data.) Perhaps the most remarkable aspect of the data in item nonresponse is how much variation was recorded to an identical set of questions over the past few decades. For most of the period, the variations were unsystematic, with a trend line indicating insignificant variations until the past several years. Presumably the variations were due to changes in the motivations of respondents, as expressed in unit as well as item nonresponse rates.

6 540 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH Figure 2 Item nonresponse rates and unit nonresponse rates, % 60% 5% 50% 4% 40% 3% 30% 2% 20% 1% 10% 0% % Item Nonresponse Rates Unit Nonresponse Rates Another remarkable feature is the sharp decline in item nonresponse in the recent years. Our examination of the causes for the decline started with the trend line for unit nonresponse rates, which is also shown in Figure 2 (the bottom line corresponding to the right vertical axis). As also shown in Curtin et al. (2005), a gradual increase to no change from 1979 to 1996 was recorded in the SCA unit nonresponse rates and a sharper increase after To better understand the trend in item nonresponse in light of this increasing trend of unit nonresponse, we fitted separate trend lines for the two subperiods: and (see Figure 3). Indeed, the trend line fitted from 1986 to 1996 suggests there is no significant change in item nonresponse for those 10 years when unit nonresponse rates showed no change as well; the trend coefficient is (SE ¼ 0.041). By contrast, the data from 1997 to 2005 indicate a significant decline (trend coefficient ¼ 0.302, SE¼ 0.079), falling from 4.8% in 1997 to just 2.4% in2005. The recent decline is outside of the range recorded in the prior 20 years to a considerable degree, with new lows in item nonresponse recorded in each of the past 3 years. The trend data seemed to suggest that item nonresponse rates moved with the unit nonresponse rates. When there was no significant trend change in unit nonresponse rates, item nonresponse rates did not show a significant trend change either. However, item nonresponse rates showed a significant drop in trend during the same time period when unit nonresponse rates increased. We test this relationship between unit and item nonresponse rates formally in the next section.

7 RESEARCH NOTE 541 Figure 3 Item nonresponse rates, % 5% 4% 3% 2% y = x y = x % 0% Relation Between Unit and Item Nonresponse at the Individual Level The response continuum model predicts a positive relation between a respondent s propensity to be a unit nonresponder and to produce missing data. To test the hypothesis that respondents who had a lower propensity to respond provided more missing data, an ordinary least square regression was run on the combined cross-sectional data with the index of item nonresponse as the dependent variable. Included as key independent variables are two proxy indicators of a low propensity to agree to the survey request the number of call attempts a respondent took to complete the interview and an indicator of whether the respondent initially refused the survey request. Even though we are not always able to make inference about noncontacts from respondents requiring a higher number of calls and refusal from those who initially refused the survey request, the number of calls and the refusal status are measures of the difficulty to contact and recruit a sample person. Respondents requiring a higher number of call attempts and refusal conversion efforts place them toward the left end of the response continuum and they are expected to have more missing data than those who, for instance, agreed to the survey request at the first call and who did not need refusal conversion efforts. We also entered into the regression model demographic characteristics (such as age, gender, education, race, marital status, and income) to control for any idiosyncratic effects. Table 1 shows the results from the regression model at the respondent level, drawing on monthly surveys of 20 years. Consistent with the response continuum model, respondents who needed more calls to complete the interview had significantly more item nonresponse, holding various demographic characteristics constant. On average, one additional call produces an increase of about one hundredth percentage point in the index of item nonresponse. Replicating the results of Mason and colleagues (Mason et al., 2002) and Stinchcombe and colleagues (Stinchcombe et al., 1981), respondents who initially refused the survey request also had higher item nonresponse. Initial refusers had, on average, 0.6 percentage points more item nonresponse than cooperators. The regression further confirmed that older respondents,

8 542 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH Table 1 Predicting Item Nonresponse Rates: Individual Level Regression Individual level model Parameter estimate SE Initial refuser Number of call attempts Time index Age Male Married White Income above median High school or less education Intercept R Note: All estimates are significant at p <.05. females, respondents who were not married, non-white, less wealthy, and had high school or less education had more missing data, consistent with prior work on item nonresponse (Bell, 1984; Craig & McCaan, 1978; de Leeuw, 2001; Ferber, 1966; Riphahn & Serfling, 2005). Since January 2000, SCA began sending a $5 prepaid incentive to all potential respondents whose mailing address was located (see Curtin et al., 2005). Given the sharp decline in item nonresponse in the recent years, there was a concern that the decline was in part driven by the inclusion of respondents who received a prepaid incentive. In other words, receiving an incentive might affect the amount of item nonresponse a respondent provided. To examine this effect of incentive receiving, we compared those respondents who received a prepaid incentive with those who were not sent an incentive in terms of item nonresponse. We were only able to identify, for each person, whether they were sent an incentive for four monthly surveys (November 2003, December 2003, January 2004, and February 2004). Data from these four surveys were used to examine the effects of incentives on item nonresponse. Respondents who were not sent a prepaid incentive tended to have more missing data (M ¼ 3.26) than those who received an incentive (M ¼ 2.92). However, the difference was not statistically significant, F(1, 1180) ¼ 0.88, ns. In addition, we entered the incentive receiving status to the regression model in Table 1. The status variable was not significantly related to the item nonresponse index (regression coefficient ¼ 0.355, SE¼ 0.426). Furthermore, it did not interact with the number of calls and the initial refuser indicator variable either (results not shown). These analyses seem to suggest that the use of incentives cannot account for the sharp decline in item nonresponse in the recent years. To test the individual-level hypothesis that respondents who had higher item nonresponse at wave one were more likely to not participate in the reinterview, we modeled the probability of not responding to the reinterview as a function of the item

9 Table 2 Predicting Response to Reinterview Using Item Nonresponse Index Individual level model Parameter estimate SE Odds ratio in log scale Item nonresponse index Initial refuser Number of call attempts Time index Age Male Married White Income above median High school or less education Intercept Max-Rescaled R Note: Bolded estimates are significant at p <.05. RESEARCH NOTE 543 nonresponse index at the first interview. (Besides the item nonresponse index, the same independent variables in the ordinary least square regression in Table 1 were also entered.) As shown in Table 2, the logistic regression results confirmed that, holding everything else constant, respondents who had more missing data (a higher value at the item nonresponse index) were more likely to not respond to the reinterview (regression coefficient in log scale ¼ 0.020, SE¼ 0.001); in other words, holding everything else constant, when item nonresponse index was increased by 1 unit, the odds of not responding to the reinterview was increased by 1.02 (see also Yan & Curtin, 2007). Both individual level hypotheses of the response continuum model are supported. Relation Between Unit and Item Nonresponse at the Aggregate Level In the earlier section, we showed that the item nonresponse index moved with the trend of the unit nonresponse. The response continuum model hypothesizes that, at the aggregate level, surveys with lower unit nonresponse rates are expected to have higher item nonresponse rates. This is presumably because surveys with lower unit nonresponse rates include more respondents with a lower propensity to participate in the survey. To test for this survey-level hypothesis, we used the survey-level data; that is, each observation in the analysis is a monthly survey and the variables are monthly survey-level attributes. Twenty years SCA s data included 231 monthly surveys, yielding 231 pairs of unit and item nonresponse rates (monthly average of the index of item nonresponse). We first took a simple correlation of unit nonresponse rates for monthly surveys and the monthly average of item nonresponse index values. The data indicate that

10 544 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH Table 3 Predicting Item Nonresponse Rates: Aggregate Level Regression Aggregate level model Parameter estimate SE Unit nonresponse rates Percent of initial refusers Average number of calls Mean age Percent of male respondents Percent of married respondents Percent of White respondents Percent of respondents with income above median Percent of respondents with high school or less education Intercept R Note: Bolded estimates are significant at p <.05. unit nonresponse rates have risen but item nonresponse has fallen; coefficient of is statistically significant (p <.0001), suggesting that there is a significant negative correlation between the unit and item nonresponse rates and supporting the aggregate level hypothesis predicted by the response continuum model. To obtain a net effect of unit nonresponse rates on item nonresponse rates, we regressed the monthly average of item nonresponse index values on monthly unit nonresponse rates, the percentage of initial refusers in the monthly sample, and the average number of call attempts as the independent variables. To control for the possible effects of differential sample composition of each monthly survey, we included in the model as control variables the mean age, the percentage of males, the percentage of whites, the percentage of married, the percentage of respondents having an income higher than the median, and the percentage of respondents who had high school or less (the same set of variables is also used in weighting). An AR(1) autoregressive model was fit to correct for autocorrelated errors resulting from the use of time-series data. The regression results are displayed in the Table 3. As shown in the Table 3, after controlling for sample composition, the unit nonresponse rates still have a significant negative effect on item nonresponse rates. Reducing the unit nonresponse rates by 1 percentage point (or improving unit response rates by 1 percentage point) increases item nonresponse by almost 0.06 percentage points. The net effect might be small for 1 unit change in unit nonresponse rates, but the accumulative effects are more substantial. The unit nonresponse rates have increased from about 29.3% in 1986 to 55.2% in 2005, which translates into a reduction of item nonresponse by 1.53 percentage points; roughly corresponding to the difference in item nonresponse at 1986 (4.3%) and 2005 (2.4%). Both the significant negative regression coefficient from the aggregate level regression model and the significant negative Pearson correlation coefficient provide one

11 RESEARCH NOTE 545 more support to the response continuum model. We next examine how this negative relation between unit and item nonresponse changes over time. Trend of Aggregate-Level Relation Between Unit and Item Nonresponse Because of the increase of SCA unit nonresponse rates at 1996, we calculated the bivariate correlation between unit and item nonresponse separately for and For the first 10 years from 1986 to 1996, the correlation between unit and item nonresponse is essentially zero (r ¼.090, p ¼.317). By contrast, the correlation between the two aspects of nonresponse is.742 (p <.001) from 1997 to The set of correlations confirmed the trend we observed in the early section. When unit nonresponse rates did not change, there was no change in item nonresponse rates. However, the increase in unit nonresponse rates was accompanied by the fall in item nonresponse rates, supporting the response continuum model. We ran the same regression model in Table 3, but we ran it twice; one on monthly surveys before 1997 and the other since Both models contain the same set of independent variables and control variables. The dependent variables are the monthly average of the index of item nonresponse, as in Table 3. Table 4 displays the regression results. Holding sample compositions constant, unit nonresponse rates are not significantly related to item nonresponse rates for monthly surveys conducted before However, unit nonresponse rates are negatively related with item nonresponse for surveys conducted in 1997 or later; the negative coefficient is significant at the p ¼.01 level. The results of unit nonresponse rates supported the aggregate-level hypothesis derived from the response continuum model. Discussion This article provided some rare good news about nonresponse rates: item-nonresponse rates have significantly declined during recent years. The bad news was that it was partly due to increases in unit nonresponse rates. The data confirmed a significant connection between unit and item nonresponse at both the individual and the aggregate level. Respondents with a lower propensity to participate in surveys provided more missing data and surveys with lower unit nonresponse rates (or higher response rates) were associated with higher item nonresponse rates. These findings support the response continuum model. An examination of the response continuum model only at the aggregate level may involve an ecological fallacy. The concerns would be valid if the relationship between unit and item nonresponse at the aggregate level was used to make inferences about the likelihood of an individual to participate in the survey and to answer survey questions. In contrast, this analysis started by estimating the response continuum model at the individual level and then the individual-level information was aggregated to study the relationship between unit and item nonresponse at the survey level. This bottom-up analysis does not run the risk of an ecological fallacy. In addition, the fact

12 546 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH Table 4 Predicting Item Nonresponse Rates at the Aggregate Level: Before 1996 versus 1996 or Later (n ¼ 127) (n ¼ 104) Parameter estimate SE Parameter estimate Unit nonresponse rates Percentage of initial refusers in the sample Average number of call attempts Mean age Percent of male respondents Percent of married respondents Percent of White respondents Percent of respondents with income above median Percent of respondents with high school or less education Intercept R Note: Bolded estimates are significant at p <.05. SE that both individual- and aggregate-level analyses come to the same conclusion demonstrates that the aggregate-level analyses are not an example of ecological fallacy. This article showed a sharper decline in item nonrepsonse rates after 2001 than before Respondent characteristics, interviewer characteristics, question characteristics, and survey-design features all contribute to item nonrepsonse. The study protocol and wording of survey questions that contribute to the item nonresponse rate remained unchanged for the 20 years examined. Conversations with SCA project managers confirmed that training protocol has been consistent throughout the time period presented in this article. The decline in item nonresponse rates, then, is more likely to result from changes in respondent-level characteristics, both in terms of measured variables (e.g., race, education, sex, income) and unmeasured variables (e.g., sensitivity thresholds, motivation to respond, general cooperativeness) that are related to item nonresponse. The response continuum model points toward common factors that are partly responsible for the increase of unit nonresponse rates and the decrease of item nonresponse rates observed in the past 5 years. For instance, there is evidence suggesting that privacy issues have grown in importance. Studies have shown that the percentage of the U.S. population that owns an answering machines rose from the 13% in 1985 to around 67% in 2000 (Tuckel & Feinberg, 1991; Tuckel & O Neil, 1995, 2001). Roth, Montaquila, and Brick (2001) also reported the national estimate of the prevalence of owning a call-screening device (including answering machines and

13 RESEARCH NOTE 547 calling IDs) was around 81% in These changes, together with the increasing refusal rates, reflect an increasing willingness of respondents to assert their privacy rights and not participate in surveys as well as making it harder to contact respondents. Such a change in social norms suggests that cooperating respondents have been increasingly drawn from the right side of the response continuum. As a result, respondents retained in the interviewed pools are more cooperative in nature, resulting in lower item nonresponse rates at the cost of higher unit nonresponse rates. To be sure, our findings may not generalize to other survey modes (such as Web surveys) or to other measures of attitudes or behaviors. But the relatively large decline in item nonresponse rates for measures of economic attitudes and expectations we observed reflects a trend uncontaminated by changes in survey design, question wording, or interview methodology. Our findings have other implications for survey research. The significant negative relationship between unit and item nonresponse rates (even after controlling for respondent characteristics) represents a classic trade-off. To properly evaluate effect on data quality, the impact of shifts in unit and item nonresponse needs to be jointly assessed. While the impact of response rates on data quality has been addressed in the literature, the impact of item nonresponse rates on data quality has not received as much attention and there is even less attention on the joint impact of unit and item nonresponse rates. Furthermore, the survey practitioners are still exploring the best ways to balance the two aspects of nonresponse. Thus, it is extremely important for future research to study the joint impact of both unit and item nonresponse on data quality in order to gain a more complete understanding of potential nonresponse errors. We believe, however, that it is important for survey practitioners to keep this trade-off in mind and avoid naive attempts to decrease one dimension of nonresponse rate at the price of increasing the other. The confirmation of the response continuum model has implications for survey researchers and practitioners. The model can inform design decisions (such as when to stop the data collection) and guide research on nonresponse adjustments. We showed that the response continuum model was able to capture accurately the relationship between unit and item nonresponse at the survey level and the relationship between respondents propensity to take part in a survey and their propensity to answer survey questions at the individual level. The response continuum model indicates that respondents who have high missing-data rates are similar in some respects to those who refused to participate in the survey. Future research should take into account of these findings and consider using the response continuum model in responsive design and for nonresponse adjustment and imputation. Finally, the findings offer one more basic insight about nonresponse. Item-nonresponse rates have varied considerably in the past and can be expected to vary in the future. Any single survey may document an unusually high or an unusually low incidence of item missing data. These fluctuations suggest that more dynamic theories are needed to adequately understand changes in item nonresponse.

14 548 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH Appendix A Question Wording of the 16 Core Questions of Surveys of Consumers Used in Analysis 1. (PAGO) We are interested in how people are getting along financially these days. Would you say that you (and your family living there) are better off or worse off financially than you were a year ago? 2. (PEXP) Now looking ahead do you think that a year from now you (and your family living there) will be better off financially, or worse off, or just about the same as now? 3. (BUS12) Now turning to business conditions in the country as a whole do you think that during the next 12 months we ll have good times financially, or bad times, or what? 4. (BAGO) Would you say that at the present time business conditions are better or worse than they were a year ago? 5. (NEWS1) During the last few months, have you heard of any favorable or unfavorable changes in business conditions? 6. (BEXP) And how about a year from now, do you expect that in the country as a whole business conditions will be better, or worse than they are at present, or just about the same? 7. (GOVT) As to the economic policy of the government I mean steps taken to fight inflation or unemployment would you say the government is doing a good job, only fair, or a poor job? 8. (UNEMP) How about people out of work during the coming 12 months do you think that there will be more unemployment than now, about the same, or less? 9. (RATEX) No one can say for sure, but what do you think will happen to interest rates for borrowing money during the next 12 months will they go up, stay the same, or go down? 10. (PX1Q1) During the next 12 months, do you think that prices in general will go up, or go down, or stay where they are now? 11. (RINC) During the next year or two, do you expect that your (family) income will go up more than prices will go up, about the same, or less than prices will go up? 12. (INEXP1) During the next 12 months, do you expect your (family) income to be higher or lower than during the past year? 13. (HOM) Generally speaking, do you think now is a good time or a bad time to buy a house? 14. (DUR) About the big things people buy for their homes such as furniture, a refrigerator, stove, television, and things like that. Generally speaking, do you think now is a good or a bad time for people to buy major household items? 15. (CAR) Speaking now of the automobile market do you think the next 12 months or so will be a good time or a bad time to buy a car?

15 16. (INCOME) To get a picture of people s financial situation we need to know the general range of income of all people we interview. Now, thinking about (your/ your family s) total income from all sources (including your job), how much did (you/your family) receive in (filling in time)? Appendix B Item Nonresponse Rates for the 16 Component Questions RESEARCH NOTE 549 Mean Item Nonresponse Percentage Over 20 years (%) 1. PAGO PEXP BUS BAGO NEWS BEXP GOVT UNEMP RATEX PX1Q RINC INEXQ HOM DUR CAR INCOME Item nonresponse index (excluding income) 3.27 Item nonresponse index (including income) 4.40 References Atrostic, B. K., Bates, N., Burt, G., & Silberstein., A. (2001). Nonresponse in U.S. government household surveys: Consistent measures, recent trends. new insights. Journal of Official Statistics, 17, Beatty, P., & Herrmann, D. (2002). To answer or not to answer: Decision processes related to survey item nonresponse. In R. Groves, D. Dillman, J. Eltinge & R. Little (Eds.), Survey nonresponse (pp ). New York, NY: John Wiley & Sons. Burton, J., Laurie, H., & Moon, N. (1999). Don t ask me nothin about nothin, I just might tell you the truth. The interaction between unit non-response and item non-response. Paper presented at the 1999 International Conference on Survey Nonresponse, Portland, OR.

16 550 INTERNATIONAL JOURNAL OF PUBLIC OPINION RESEARCH Campanelli, P., Sturgis, P., & Moon, N. (1996). Exploring the impact of survey introduction. In American Statistical Association Proceedings: Section on survey research methods (pp ). Alexandria, VA: American Statistical Association. Couper, M. (1997). Survey introduction and data quality. Public Opinion Quarterly, 61, Curtin, R., Presser, S., & Singer, E. (2000). The effects of response rate changes on the index of consumer sentiment. Public Opinion Quarterly, 64, Curtin, R., Presser, S., & Singer, E. (2005). Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly, 69, Djerf, K. (2004). Nonresponse in time: A time series analysis of the Finnish labor force survey. Journal of Official Statistics, 20, de Heer, W. (1999). International response trends: Results of an international survey. Journal of Official Statistics, 15, de Leeuw, E. (2001). Reducing missing data in surveys: An overview of methods. Quality and Quantity, 35, de Leeuw, E., & de Heer, W. (2002). Trends in household survey nonresponse: A longitudinal and international Comparison. In R. Groves, D. Dillman, J. Eltinge & R. Little (Eds.), Survey nonresponse (pp ). New York, NY: John Wiley & Sons. Groves, R. (1989). Survey errors and survey cost. New York: John Wiley & Sons. Groves, R., Cialdini, R., & Couper, M. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, Groves, R., & Couper, M. (1998). Nonresponse in Household Surveys. John Wiley. Groves, R., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68, Groves, R., Singer, E., & Corning, A. (2000). Leverage-salience theory of survey participation: Description and illustration. Public Opinion Quarterly, 64, Hox, J., & de Leeuw, E. (1994). A comparison of nonresponse in mail, telephone, and face-to-face surveys: Applying multilevel modeling to meta-analysis. Quality and Quantity, 28, Little, R., & Rubin, D. (1987). Statistical analysis with missing data. New York, NY: John Wiley & Sons. Loosveldt, G., Pickery, J., & Billiet, J. (2002). Item nonresponse as a predictor of unit nonresponse in a panel survey. Journal of Official Statistics, 18, Mason, R., Lesser, V., & Traugott, M. (2002). Effect of item nonresponse on nonresponse error and inference. In R. Groves, D. Dillman, J. Eltinge & R. Little (Eds.), Survey nonresponse (pp ). New York, NY: John Wiley & Sons. Roth, S., Montaquila, J., & Brick, J. (2001). Effects of telephone technologies and call screening devices on sampling, weighting and cooperation in a random digit dialing (RDD) survey. In American Statistical Association Proceedings: Section on survey research methods. Alexandria, VA: American Statistical Association. Singer, E., Van Hoewyk, J., & Maher, M. (2000). Experiments with incentives in telephone surveys. Public Opinion Quarterly, 64, Stinchcombe, A., Jones, C., & Sheatsley, P. (1981). Nonresponse bias for attitude questions. Public Opinion Quarterly, 45,

17 RESEARCH NOTE 551 Sudman, S., Bradburn, N., & Schwarz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass Publishers. Tourangeau, R., Rips, L., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press. Tuckel, P. S., & Feinberg, B. M. (1991). The answering machine poses many questions for telephone survey researchers. Public Opinion Quarterly, 55, Tuckel, P., & O Neill, H. (1995). A profile of telephone answering machine owners and screeners. In American Statistical Association Proceedings: Section on survey research methods (pp ). Alexandria, VA: American Statistical Association. Tuckel, P., & O Neill, H. (2001). The vanishing respondent in telephone surveys. In American Statistical Association Proceedings: Section on survey research methods. Alexandria, VA: American Statistical Association. Waksberg, J. (1978). Sampling methods for random digit dialing. Journal of the American Statistical Association, 73, Wolter, K., Chowdhury, S., & Kelly, J. (2009). Design, conduct, and analysis of random-digit dialing surveys. In D. Pfeffermann & C. R. Rao (Eds.), Handbook of statistics Sample surveys: Design, methods and applications, Volume 29A (pp ). The Netherlands: North-Holland. Yan, T., & Curtin, R. (2007). Responsive designs in longitudinal surveys. Paper presented at the 2007 American Association for Public Opinion Research, Orange County, CA. Biographical Notes Dr Ting Yan received her Ph.D. in Survey Methodology from the Joint Program in Survey Methodology (JPSM), University of Maryland. After her Ph.D., she completed a two-year post-doctoral training at the Institute for Social Research, University of Michigan. She is currently working at NORC at the University of Chicago as a Senior Survey Methodologist. Dr Yan has published articles on a wide variety of methodological issues. Her main research interests include measurement errors, understanding of nonresponse errors and the link between measurement and nonresponse errors, and web survey methodology. Dr Richard Curtin is a Research Associate Professor and the Director of the University of Michigan s Surveys of Consumers and the Panel Study of Entrepreneurial Dynamics.