Attrition in longitudinal survey data: Evidence from the Indonesia Family Life Survey

Transcription

1 Attrition in longitudinal survey data: Evidence from the Indonesia Family Life Survey Elizabeth Frankenberg Duke University Bondan Sikoki SurveyMETER John Strauss University of Southern California Cecep Sumantri SurveyMETER Wayan Suriastini SurveyMETER Duncan Thomas Duke University Firman Witoelar SurveyMETER and The World Bank December 2009 Paper prepared for Conference on Survey Design and Measurement in Development Economics held at The World Bank, December 10-11, Financial support from the National Institutes on Aging, the National Institute for Child Health and Human Development and Fogarty International Center are gratefully acknowledged.

2 1. INTRODUCTION Over the last quarter century, there has been spectacular growth in the quantity and quality of socio-economic surveys conducted across the globe. Longitudinal surveys, which have been on the leading edge of this movement, have proved to be extremely powerful resources for research in economics and the social sciences. However, attrition in many important longitudinal surveys is high. Whereas non-response in higher income societies like the United States is largely due to refusal, in most developing countries, refusal rates in household surveys are extremely. In those contexts, attrition is primarily driven by migration of respondents. Many longitudinal surveys in developing countries are designed to only interview respondents who do not move away from the location at baseline. Others follow respondents who move within the vicinity of that location. These surveys are not only designed to have higher attrition than studies that follow migrants but the attrition will be selected on the same characteristics that are associated with geographic mobility. Empirical studies that have explored the implications of attrition for understanding behavior of individuals has not reached a consensus. One line of research argues that attrition can be treated as random and is ignorable at least in the (broad class of) models examined in this work. (Alderman et al, 2001; Falaris, 2003). Others have argued that because attrition is intimately connected to geographic mobility, it is not ignorable in many of the same models of behavior. (Thomas, Frankenberg and Smith, 2001; Rosenzweig, 2003). The Indonesia Family Life Survey (IFLS) is ideally suited to shed new light on this issue. Since the first follow-up interview, IFLS has sought to minimize attrition and track respondents who move. In each of the three follow-up surveys, 19 out of every 20 target households have been recontacted. This is not because of low rates of mobility. Of the respondents interviewed in the fourth wave in 2007, nearly two-thirds of those interviewed in the baseline in 1993 had moved away from the home in which they were interviewed at baseline. Moreover, the four waves of the survey span a period of rapid economic growth that was interrupted by dramatic economic and political upheavals 1

3 in the late 1990s at the time of the Asian Crisis and the 2004 Indian Ocean tsunami which was one of the largest natural disasters in recorded history. These events have been accompanied by rates of geographic mobility that are elevated above an already substantial baseline. The next section describes the protocols used in IFLS to maximize recontact across the survey waves. This is followed by a discussion of the levels of attrition in the waves and contrast the recontact rates with what they would have been had we not followed movers. We further distinguish movers by, roughly speaking, the distance they move from the baseline location. We present evidence that not only does failure to track movers result in higher rates of attrition but that the selectivity of those who attrit is linked to the distance they move from the baseline location. We establish that this selection is not only complicated but also depends on characteristics that are unlikely to be observed at baseline in most socio-economic and demographic surveys. The final section exploits a Survey of Surveyors that we conducted in the first follow-up in 1997 and describes the survey and surveyor characteristics that are predictive of successful follow-up in subsequent waves. These results suggest additional strategies that might be adopted to reduce attrition in the field as well as potential instruments that might assist in reducing the impact of attrition in models of behavior. 2. FOLLOW UP IN IFLS A key feature of successful follow-up and re-interviewing of respondents is providing interviewers and trackers with detailed information on a wide range of family and individual attributes. This process necessarily begins with the first wave of data collection. In IFLS1 (and in subsequent waves) we collected information for future re-contact efforts during the household interview. This information included the current location and questions on whether a respondent thought they would be moving, and if so where (down to the village level). We also obtained the name, address, land phone and cell phone information for a family member or friend who could provide updated information about the whereabouts of the household if they moved. 2

4 At the start of each post-baseline wave, we assemble all potentially relevant information and generate household-specific recontact sheets for the field teams. These sheets contain the information described above, as well as individual household members answers to other questions routinely asked during the interview, such address information on their schools and workplaces, and the names of other non-coresident family members. In the field interviewing teams begin by returning to the dwelling where the household was last found. If the entire household had moved the interviewers sought informants who could provide new information on the household s location. A tracking form was used to record information on the name and address of a new local informant in the origin area; the new address and telephone of the household; names, addresses and phone numbers of additional household members no longer with the main household; the name and address of a non-household informant in the new location; and work places of members in the new location. Although not all of this information was easily obtainable, when it was, locating households that had moved since the last interview was much easier. If the dwelling was still inhabited by someone from the target household but one or more individuals from the household had moved and were targets for individual tracking (defined according to protocols specified below and included in the field teams recontact information) the interviewers collected an individual tracking form. This form recorded names, addresses and phone numbers of local informants; the address and telephone of the mover respondent, if known; the names and address information of up to 2 informants in the destination area; the address where the respondent works in the destination area, the names and addresses of school, if any, where the respondent attends in the destination area, and finally a map in the destination area. Armed with information on either household or individual movers, the teams made a choice as to whether the case should be handled as local (destinations within 45 minutes of the origin area by public transportation) or longer distance tracking. Teams typically searched for local tracking cases while the teams were in the enumeration area. For longer distance tracking cases, the forms, 3

5 after electronic data entry in the field, were uploaded onto a centrally maintained website and reassigned by staff at central headquarters to the field team whose route would take them closest to the new location. This was a dynamic and complex process. The tracking database was updated as new information on movers was collected. In some cases, conflicting information about the location of a respondent was collected from different people and, in many cases, by different teams. These cases were resolved by electronic communication among the interviewers and staff in headquarters. When location information was updated, the cases were re-assigned if necessary and the updated information was sent electronically to the team in the field. At the end of the field period field teams were collapsed to a smaller number of tracking teams (composed of the best interviewers) to continue long distance tracking. The tracking phase typically lasted a month or two, until the cases that remained incomplete were judged unlikely to be found in that survey wave. 3. RESULTS {Introduction to be completed} Completion rates in each wave of IFLS A central design issue for any household survey is the question of whom within the household to interview and, in longitudinal surveys, whom to follow if any household member moves. In IFLS, when all the members of a household have changed location, we track that household unless the available information indicates that the entire household has moved outside the IFLS provinces. The rules for interviewing and tracking individual members within IFLS households vary across the different waves of the IFLS. In Table 1 we summarize information on sample size and recontact rates for each survey wave. IFLS1 was conducted in The total number of individuals living in the selected IFLS households was 33,081. For cost reasons IFLS1 implemented a within-household sampling scheme, which involved individual interviews with the household head and his/her spouse, two randomly 4

6 selected children of the head and spouse age 0 to 14, an individual age 50 or older and his/her spouse, randomly selected from remaining members and, for a randomly selected 25% of the households, an individual age 15 to 49 and his/her spouse, randomly selected from remaining members. This scheme led to the designation of 22,588 people as eligible for individual interviews, among which 97.5% granted interviews and 2.5% refused. The first follow up wave took place in In IFLS2 the goal was to relocate and reinterview all IFLS1 households and the 33,081 respondents in those households, as well as new household members (see Frankenberg and Thomas, 2000, for a detailed description). If one or more individuals had moved out of the household by 1997, the movers were tracked proved they were target respondents. Target respondents were movers who were main respondents in 1993 (defined as having provided an individual interview or being born before 1968). The new households in which movers were found are split-off households. Once a household was found, the rules for interviewing household members differed for origin and split-off households. In origin households the goal was to interview all members. In splitoff households only target respondents, their spouses, and any of their biological children living in the household were to be interviewed (this reduced the number of new respondents with only a tenuous connection to IFLS1 household members). Results for IFLS2 are displayed in columns 3 and 4 of Table 1. Of the 33,081 respondents, 854 died between IFLS1 and IFLS2. Of respondents who were eligible to be tracked and interviewed, 89% were interviewed and about 1% refused (the remainder were not interviewed). The sampling approach in IFLS3 was to re-contact all original IFLS1 respondents who were alive the last time their household was contacted, plus target respondents, their spouses and children that were in split-off households in IFLS2 or IFLS2+. Likewise individuals who moved out of their IFLS households were followed. The rules for following individuals who moved out of an IFLS household were expanded in IFLS3. In addition to those defined as target respondents in IFLS2, IFLS3 added four classes of respondents. They are individuals born since 1993 in original IFLS1 5

7 households, individuals born after 1988 and resident in an original IFLS1 household in 2007, IFLS1 household members born between 1968 and 1988 and interviewed in 1997 and a 20% random subsample of 1993 household members born between 1968 and 1988 if they were not interviewed in The motivation behind expanding the group of individuals who would be tracked beyond the group followed in 1997 was to be able to follow small children in panel households and a subset of young adults, born between 1968 and This strategy was designed to keep the sample, once weighted, closely representative of the original 1993 population in the 13 IFLS provinces. Rules for interviewing individual household members were expanded slightly in IFLS3. In origin IFLS1 households everyone was to be interviewed. In split-off households, all IFLS1 household members, their spouses and biological children, were to be interviewed, but not others. However when the tracking was local, all household members were interviewed. Results for IFLS3 are displayed in columns 5 and 6. Among a pool of 37,631 individuals eligible for the survey, 32,218 were eligible to be tracked and interviewed. Of these, 90.5% were interviewed and 0.8% refused. These numbers are comparable to IFLS2. IFLS4 used the almost the same re-contact and interview protocols as IFLS3. Interviewing rules were also kept the same as 2000 except that households that were locally tracked were treated the same as other tracked households and the IFLS respondent, spouse and children were eligible for interview. Results for IFLS4 are displayed in Columns 7 and 8. In IFLS4 85.6% of those eligible to be tracked and reinterviewed (n=31,995) were reinterviewed and 14.3% were not found. Follow-up in 2007 wave of IFLS Our analysis of the factors that are associated with attrition focuses on those respondents who were eligible to be tracked in In addition, we restrict attention to respondents who were age 15 and older at the time of the baseline survey in 1993 because those respondents were each interviewed by an enumerator and completed an extensive battery of questions. 6

8 As shown in Table 2, there are slightly over 16,500 such respondents of whom 88% were interviewed in the 2007 survey resulting in an attrition rate for this sample of 12% after 14 years. Recall that in Table 1, there are nearly 32,000 respondents who were eligible to be tracked in The 15,500 not included in Table 2 were born after 1978 (and thus were not age 15 or older at baseline). About half of those respondents were born after the baseline survey was completed. As in many other surveys in developing countries, refusals are very uncommon in IFLS. Refusals account for only 1% of the respondents in this sample. We had no contact with half the rest of the people who were not interviewed. However, we were able to find out some information about the other half of respondents who were not interviewed since at least one member of the 1993 household was interviewed. A limited amount of proxy information was collected from those respondents about the respondents we did not found. On-going research is exploring the quality of this proxy information. For research that uses that proxy information, the attrition rate would be almost halved to less than 7%. Many longitudinal surveys are not designed to follow movers. One indicator of the costs of this approach is apparent from row 2a of the table Among those respondents who were eligible to be tracked, 70% were located in the same place (desa, which is a village or neighborhood in a town) that they were interviewed in Had we not followed people who moved, attrition would have been over 30% -- which is more than two and half times the actual level of attrition. Some of the people interviewed in the same desa had moved within that desa; we do not attempt to distinguish those people from those who were re-interviewed in the same house. In the absence of good address information, it is difficult to know whether a person is living in exactly the same housing structure or on the same plot of land in each survey wave. Further, it is not the case that all these people who were interviewed in the same desa had never moved. Many of them had moved to a different desa between the waves and returned to the original desa. The 18% of target respondents who had moved from the desa of residence at baseline, were tracked to their new desa and interviewed in the new location. Of those, 4% had moved outside the 7

9 desa but within the kecamatan (or sub-district) which one might interpret as relatively local movers. Another 5% had moved outside the kecamatan but remained within the kabupaten (district or regency). A further 5% moved outside the kabupaten but remained in the province and 4% had moved out of the province. We think of the last three destinations as being longer-distances moves. 1 Selectivity of non-respondents and movers Table 3 draws comparisons between those who were interviewed in 2007 and those who were not. The comparisons are made several different dimensions all of which are measured at baseline and so are known for every eligible respondent including those who were not interviewed in The average number of years of completed schooling of respondents, as of 1993, is reported in the first column Those who were interviewed had, on average, almost 6 years of schooling (row 1 of Panel A), those who were not interviewed had completed almost 8 years of schooling (row 2) on average. The difference, 2 years of schooling, is significant (row 3). Standard errors that take into account clustering of the sample are reported below the means and the difference in Panel A. Relative to those not interviewed in 2007, respondents who were interviewed in 2007 had less well-educated fathers and mothers. The level of household per capita expenditure (PCE) was significantly lower at baseline for those interviewed than those not interviewed. This is not because those who were interviewed came from larger households: in fact there is no difference in average household size between the two groups. Those interviewed in 2007 were significantly more likely to be working in 1993 but, among those who were working, the earnings of those who were interviewed were significantly lower than those who were not interviewed in About 40% of those who were interviewed were living in a household that owned a farm business but only a quarter of those who were not interviewed lived in a 1 To put these geographic distinctions into perspective, in the 15 provinces included in IFLS, there are 228 kabupaten, 3,323 kecamatans and 42,824 desas. According to the 2003 Potensi Desa Survey, a survey of leaders of each desa in the country, the median number of people living in a desa is slightly under 3,000, the median population of a kecamatan is nearly 45,000 people and the median population of a kabupatens is almost 650,000 people. 8

10 household with a farm business. Attrition is lower among those who were working and those who owned a farm business. However, owning a non-farm business is not associated with attrition slightly over one-third of all respondents lived in a household with a non-farm business and there is no difference between those who were interviewed and those not interviewed. Respondents who were married at baseline are more likely to be re-interviewed as are people who had not moved prior to age 12 (which was before the baseline survey). There are large and significant differences in the socio-economic characteristics, work choices, farm ownership, demographic characteristics and migration experience between those who were interviewed in 2007 and those who were lost to follow-up. To explore these differences in more depth, panel B of the table distinguishes those interviewed by the location in which they were found in 2007, following the same geographic groups used in Table 2. Those who were not interviewed also follow the distinctions adopted in Table 2. The last row of the table reports the average for those respondents who were not interviewed and no one from their 1993 household was interviewed. The average education of this group is 8 years. The remaining rows report the difference between this average and the average for the group in the row heading. For example, people who were interviewed in the same desa in 1993 and 2007 had, on average, 2.5 years less schooling than those for whom there was no contact. This difference is significant. Respondents who were interviewed in the same kecamatan (but a different desa) have about 1.7 years more schooling than those who were interviewed in the same desa and 0.8 years less schooling than those with whom there was no contact. These differences are significant. However, none of the differences between the respondents in each the other outcome classes and those with whom there was no contact is significant. Thus the 2 year schooling gap between those who were interviewed and those not interviewed is, to a large extent, driven by differences between respondents who were re-interviewed within the original kecamatan and all other respondents. Put another way, in terms of schooling levels at baseline, it is not possible to distinguish longer distance movers from respondents who were not interviewed. (Refusals have a year more 9

11 schooling then those for whom no information was collected but the difference is not significant. This is, in part, because a very small fraction of respondents refuse to participate in the survey.) As with own education, respondents who were interviewed in the original desa have parents who have significantly less schooling than respondents who were not interviewed. Parents of refusals are better educated and the difference is significant for maternal education. There is an additional important difference: respondents who moved outside the original kecamatan have significantly better educated parents. These differences might reflect the fact that males and younger respondents are more likely to move. Appendix Table 1 displays the same differences separately for males and females after adjusting for age (with 5 year cohort indicator variables). The key result in the table persists: there are large gaps in education and parental education of respondents who are interviewed in the same kecamatan relative to those who were not interviewed there. The patterns for females, after adjusting for age, are very similar to the patterns in Table 3. While males interviewed outside the original kecamatan are better educated than those interviewed in the original desa, they are not as well educated as those not interviewed. The same applies to the education of their parents. The fourth column of Table 3 reports differences in the (logarithm) of PCE. Respondents who were not interviewed had significantly higher levels of PCE at baseline with those who were interviewed in the same desa in 2007 having the lowest level of PCE at baseline. PCE rises with distance from the original location and respondents who moved outside the original kabupaten had levels of PCE that were about halfway between those interviewed in the original desa and those for whom no information was collected in Whereas refusals tend to be better educated, they did not have higher PCE at baseline. However, respondents who provide proxy information on those not found come from households that were poorer at baseline. Differences in PCE may not reflect differences in resources available to households but differences in household composition. In principle, one might like to adjust for these differences but it is far from clear how that should be done. Rather, the fifth column presents differences in 10

12 household size across the classes of interview outcomes. Panel A indicated there are no differences between those interviewed and those not interviewed. This dichotomy masks substantial heterogeneity. The people for whom no information is collected were living in the smallest households at baseline. Respondents interviewed in the same desa in 2007 were living in households about 2/3 of a person larger and all other respondents who were interviewed were living in households with about one additional member at baseline. Those who refused to participate were living in the largest households at baseline. None of the differences among these groups is significant: the key result is that it is the smallest and highest PCE households that are least likely to have been re-interviewed in The probability the respondent was working at baseline and, conditional on working, the logarithm of monthly earnings are in the next two columns. Respondents interviewed in the original desa were more likely to be working at baseline and they had lower monthly earnings at that time. People re-interviewed in the same kabupaten also had lower earnings. Respondents who were living in households that owned a farm business at baseline are significantly more likely to be re-interviewed. While this is true for everyone who is interviewed in 2007 and for respondents for whom only proxy information was collected, the gap declines with distance from the original desa. This is not because entrepreneurs or the self-employed are more likely to be interviewed owning a business is unrelated to interview outcome. Owning land likely anchors at least some members of the original household and even movers keep contact with that family so that they are easier to track and re-interview. Marriage is much like owning a farm business. People who were married at baseline are significantly more likely to be interviewed in The gap is greatest among those interviewed in the original desa and declines with distance from the desa. Finally, mobility at an early age is a powerful predictor of the interview outcome in Over 80% of respondents who were not re-interviewed had moved prior to age 12 (and before the baseline). Of those interviewed in the original desa, 55% had moved by age 12. The differential 11

13 between those interviewed and those not interviewed declines with distance from the original desa. Respondents for whom proxy information is collected are also much less likely to have moved before age 12 than any of the other target respondents other than those re-interviewed in the original desa. While these people moved away, presumably the low rates of mobility when they were young reflects greater geographic stability among other household members and it is one of those people who provided proxy information on the respondent. In sum, the table paints a picture of not just selection among respondents who attrited from the survey but also selection among those who were interviewed in different locations in the followup. Overall, based on characteristics measured at baseline, those with the most human capital, highest earnings, highest household PCE, those who were single and had moved at an early age are the most likely to attrit from the survey. Those who were the least human capital, lowest earnings, lowest PCE, were married, lived in a household with a farm business and had not moved at any early age are the most likely to be interviewed in the same desa in both baseline and the resurvey. Movers who were interviewed fall between these extremes. In some cases, movers look (in terms of these dimensions of socio-demographic characteristics at baseline) more like those not found and in other cases they look more like the people interviewed in the same desa. In general, the longer distance the move from the original desa, the greater the distance in socio-economic and demographic characteristics from respondents who were interviewed in the same desa in the baseline and follow-up. Conversely, longer distance movers are closer to those who were not interviewed in terms of baseline characteristics. These differences are summarized in Appendix Table 2 which presents multinomial logit models that include as covariates all the characteristics included in Table 3 along with indicator variables for gender, birth cohort and province of interview at baseline. The univariate results in Table 3 carry through to the multivariate models except that paternal education, work status and ln(earnings) in 1993 are not significant predictors of interview outcomes in Clearly, had IFLS not been designed to follow movers, not only would the 12

14 attrition rate have been higher (as shown in Table 2) but attrition would have been even more selected than it is, at least as measured by these baseline characteristics. Unobserved differences between movers and stayers It has been argued that if the differences between respondents who are interviewed and those who attrit can be fully captured by characteristics that are observed at baseline, then controlling these characteristics in empirical models of behavior will take into account the differences between those who attrit and those who do not. It is not possible to interpret the estimated coefficients on these characteristics since they will reflect the combined impact of attrition and whatever association they have with the outcome of interest. As a practical matter, many of the characteristics that are included in models of behavior are likely to be correlated with attrition and so this methods of adjusting for attrition does not seem very appealing. If the differences between those who attrit and those who do not are not observed at baseline, then, in general, the regression adjustment strategy will not be a valid approach for taking attrition into account attrition. To explore this concern, we examine changes in the relationship between (the logarithm of) monthly earnings and education over time for respondents who were interviewed in the same desa in both the baseline and follow-up and respondents who had moved. The estimated coefficients on education are reported in Table 4. The first row of the table displays results from the baseline survey for respondents who were, in 2007, interviewed in the same desa (in the first column, whom we refer to as stayers ) and in a different desa (in the second column, movers ). The difference in the estimates is reported in the third column. At baseline, people who did not subsequently move, had higher returns to their education than those who subsequently moved. The second row reports the same model using the 2007 follow-up survey. By then, the tables had turned, and for the same respondents, movers had higher returns on their education than stayers and this difference is also significant. Growth in earnings is reported in the third row. Among people interviewed in the original desa, income growth 13

15 is unrelated to education. Among movers, income growth is 3% higher for each year of education. This difference is also significant. We conclude that not only do returns to education measured at baseline differ for movers and stayers but changes in those returns after the baseline also differ. This likely reflects differences between these respondents that are not measured in the baseline survey (at least with any of the characteristics that we have included in the models). This may reflect differences between movers and stayers that we do not measure at baseline or it may reflect innovations in the lives of those who move. Moreover, those innovations that occurred after the baseline may affect the probability an individual moves from the origin desa and does not return. Given the patterns reported above, we conjecture that the differences between movers and stayers are lower bound estimates of the differences between those not found and respondents interviewed in the original desa. An important implication of these results is that it may not be prudent to draw conclusions about the extent of selectivity among those who attrit based on comparisons of baseline characteristics of respondents who are followed up with those who are not. Our results indicate that in the case of IFLS, there are characteristics that affect attrition that are not observed at baseline. Survey characteristics and subsequent interview outcomes Successful completion of an interview in a survey is the outcome of a complex interaction between enumerators, people who know the respondent and can provide information that leads to contacting the respondent and ultimately communication with the respondent. Thus far, we have focused on the characteristics of respondents that are predictive of interview outcomes. We turn next to the characteristics of the survey and enumerators. Many of the factors that we believe are extremely important for success in the field are not amenable to empirical investigation in the context of a single longitudinal survey because the factors are either very difficult to measure or are effectively fixed across the waves of the survey. These 14

16 factors include, for example, the design of the survey including the design and implementation of protocols for tracking and follow-up, the quality of leadership and oversight in the survey team, the support provided to enumerators to help them complete their work, development and deployment of electronic tools that aid the field workers locate and interview respondents. Other factors that are hard to measure include building rapport with respondents and others in the community, the enthusiasm, commitment and work ethic of the survey team. The extent to which survey characteristics that we have measured are predictive of interview outcomes in subsequent surveys is explored in Table 5. We use survey characteristics measured in the first follow-up of IFLS respondents in 1997 because one of our innovations in that wave was a Survey of Surveyors which collected a rich array of markers of human capital that are potentially related to success in the field. The Survey of Surveyors was completed by each enumerator when the fieldwork was completed. It was not conducted in We examine whether survey characteristics in 1997 are predictive of interview outcomes ten years later in the 2007 survey. Respondents who were not interviewed in 1997 are excluded from the analyses. There are at least two mechanisms through which a really good enumerator will have an influence on successfully interviewing the same respondent in subsequent survey waves. First, the best enumerator develops good rapport with each respondent, conveys a sense of empathy towards the respondent and builds trust with the respondent. This rapport should facilitate future contacts with the same respondent including reducing the probability the respondent will refuse to participate in subsequent rounds. Second, the quality of information that is collected is likely to be higher as the quality of the enumerator increases. It is difficult to overstate the importance of detailed and accurate information about potential future locations and people to contact in order to re-locate the respondent. Table 5a reports results of three different multivariate regression models that include our indicators of survey characteristics as covariates. The first model in the table is a logistic regression with whether the respondent was interviewed in 2007 as the outcome. Odds ratios in column A 15

17 indicate the odds that a person was interviewed relative to not being interviewed for each of the covariates. The second model is a multinomial logit with four outcomes: not being interviewed (the excluded category), interviewed in the same desa, interviewed in the same kecamatan (a local mover ), interviewed elsewhere (a longer distance mover). Risk ratios are reported in columns B, with the risks being relative to the excluded category of not interviewed. The third model is a multinomial logit with the same eight outcomes used in Table 2. The risk ratios are relative to the excluded category of no information collected on the respondent. Markers of the human capital of the enumerators are included in the first block of covariates. Prior survey experience is a powerful predictor of success in subsequent rounds. Respondents who were interviewed by enumerators who had had prior survey experience were 66% more likely to be interviewed than respondents interviewed by an enumerator who had never worked on a survey before. Looking across the models, it is apparent that the benefits of experienced enumerators apply to those who were interviewed in the same location in 2007, to local movers and to longer distance movers (other than those who left the province). Although it is not significant, a respondent is half as likely to refuse to participate in 2007 if the enumerator was experienced. All enumerators were paid the same amount in IFLS. In the Survey of Surveyors, we asked them about their income in their prior job (for those who had worked before). Higher income in the prior job is associated with low rates of successful interviews in subsequent surveys. Looking across the models, this is driven almost entirely by significantly elevated probabilities of refusals among respondents interviewed by an enumerator who had a higher paying job before IFLS (column C1). About three-quarters of the enumerators had a bachelor s degree prior to working on IFLS. Around 60% of the enumerators planned to return to school. Whereas a bachelor s degree is not predictive of subsequent interview success, respondents interviewed by an enumerator who planned to return to school were less likely to be interviewed in 2007 because they (and everyone else in their household) was less likely to be found in This suggests that the quality of the information 16

18 on re-contact was poorer among those enumerators who saw IFLS as a temporary job between episodes at school rather than a longer-term commitment. As part of the Survey of Surveyors, each enumerator completed a 30 items test that covered high school and college mathematics skills. According to this marker, enumerators with more human capital are more successful. Respondents who were interviewed by enumerators who scored in the top quartile on the test were more likely to be interviewed in The next set of interviewer characteristics are intended to capture non cognitive skills or personality traits. We asked about six traits: assertive, talkative, shy, organized, systematic and careful. Each enumerator was asked to think about the statement I am [trait] and rate him or herself on a ten point scale ranging from strongly disagree (1) to strongly agree (10). Only one of these traits emerges as a significant predictor of subsequent interview success: enumerators who consider themselves to be more organized perform better. Enumerators worked in teams of eight interviewers. In the Survey of Surveyors, we asked each enumerator to identify the two best interviewers in the team, excluding the enumerator him or herself. Some interviewers were never nominated as one of the top two on their team. One interviewer was nominated by 80% of teammates. However, these ratings have no impact on interview success in The last set of survey characteristics included in the model pertain to the characteristics of the respondent and interactions between the respondent and enumerator. First, at the end of every interview, the enumerator rated his or her impression of the accuracy of the respondent s answers. We created an indicator variable for the 8% of respondents whose accuracy was thought to be good. This is another powerful predictor of subsequent interview success, particularly for respondents who were interviewed outside the original desa in These respondents are also much less likely to refuse in subsequent surveys (although the effect is not statistically significant). IFLS is a complex instrument that involves administering an interview to every adult household member. The average adult interview lasts about an hour. Respondents who were 17

19 interviews were longer in 1997 are more likely to be re-interviewed in There is no evidence that length of interview is correlated with the probability of refusal in subsequent waves. There are many potential mechanisms that might explain this association. For example, it is possible that older respondents take longer because, for example, they have more events to recall in each of the domains of their life histories they are asked about. We will test this hypothesis below. Some of the individual interviews are conducted by proxy because, after multiple attempts, it is not possible to make an appointment with the respondent for the interview. Those people are no less likely to be interviewed in subsequent rounds although, if they are not interviewed, they are far more likely to have information about them provided by another household member than to disappear entirely from the survey. The evidence in Table 5a indicates that the success of a survey and success of tracking depends on the characteristics of the enumerators and their interactions with the respondents. The results suggest that enumerators who have experience, good mathematical skills and are organized tend to perform better (in terms of subsequent successful interviews of their respondents). Enumerators who perceive themselves as having high value of time (because they were better paid before the survey) and those who see the work is temporary are less successful. In addition to identifying the characteristics of enumerators that are associated with lower attrition in later surveys, the enumerator characteristics are potentially useful instruments for taking attrition into account in models that seek to explain respondent behavior. These might include, for example, models of education attainment, labor market success or migration decisions. While attrition is related to enumerator characteristics, there are no reasons to suppose that the behaviors of respondents are related to those characteristics. As a first step in this direction, Table 5b repeats the analyses in Table 5a but includes as controls the characteristics described in Table 3. They are reported in the second half of Table 5b. To summarize the results, the benefits of survey experience, performing well in mathematics and considering oneself to be organized disappear. However, the deleterious effects of planning to 18

20 return to school and earning more prior to the survey are little affected. These results suggest that there is a complex interaction between the enumerators and respondents: we are in the process of exploring interactions among enumerator and respondent characteristics. Table 3 provides a simple way to draw comparisons between respondents interviewed in different locations and also between them and those not interviewed. Those comparisons are univariate. The lower panel of Table 5b presents the multivariate analogue. The main difference between Tables 3 and 5b is that, controlling other characteristics, father s education is not associated with attrition or location of interview. Survey characteristics and behavioral models {To be completed} 4. CONCLUSIONS {To be completed} 19

21 REFERENCES 20

22 Table 1 : IFLS Completion rates: Individual respondents # % # % # % # % 01. Eligible for survey 33,081 37,631 39, Died between the waves , Eligible for survey alive at survey date 33,081 32,227 36,841 37,377 (row 01-row 02) 04. Assessed 26, % 35, % 31, % (%=row 04/row 03) 05. Eligible to be tracked and interviewed 22,588 23,049 32,218 32,757 of whom 06. Interviewed 22, % 20, % 29, % 27, % 07. Refused % % % % 08. Not interviewed 2, % 2, % 4, % 09. New entrants in this wave - 5,404 3,146 12, Total sample interviewed this wave 31,948 38,417 44,097 (row 04+row 09) 11. Total potential sample for next wave 33,081 37,631 39,987 49,479 (row 03+row 09)

23 Table 2: 2007 interview outcomes for respondents who were eligible to be tracked in 2007 and age at least 15 in 1993 # resp % # resp % 1. Eligible to be tracked 16, Interviewed 14,485 88% Location of interview a. same desa as , % b. same kecamatan % as 1993 c. same kabupaten % as 1993 d. same province % as 1993 e. different province % 3. Not interviewed 2,024 12% Reason not interviewed a. refused % b. other HH mem % interviewed c. no contact w/ % any HH member

24 Table 3: Characteristics of respondents at 1993 baseline and interview outcomes in 2007 survey wave Characteristics of respondents measured at baseline in 1993 Education Father's Mother's ln(per cap Household % working ln(monthly % HHs own % HHs own % married % moved (years) education education expenditure) size for pay earnings) farm bus non farm bus 0.00 by age 12 [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] PANEL A: Interviewed in 2007? 1. Interviewed [0.13] [0.09] [0.08] [0.03] [0.05] [0.81] [0.05] [2.09] [1.08] [0.40] [1.41] 2. Not interviewed [0.20] [0.19] [0.18] [0.05] [0.12] [1.57] [0.08] [2.43] [1.77] [1.75] [1.95] 3. Difference (Ivwd - not ivwd) [0.17] [0.17] [0.16] [0.04] [0.12] [1.59] [0.08] [2.01] [1.68] [1.69] [1.96] PANEL B: Outcome in 2007 relative to no information about respondent Interviewed in 2007 in 1. same desa as [0.29] [0.26] [0.24] [0.06] [0.18] [2.00] [0.10] [3.02] [2.74] [2.32] [2.38] 2. same kecamatan as 1993 [0.32] [0.30] [0.27] [0.07] [0.22] [2.78] [0.14] [3.63] [3.41] [2.77] [3.17] 3. same kabupaten as 1993 [0.34] [0.32] [0.29] [0.08] [0.22] [2.80] [0.14] [3.46] [3.52] [2.74] [3.19] 4. same province as 1993 [0.32] [0.31] [0.28] [0.07] [0.21] [2.99] [0.15] [3.45] [3.51] [2.88] [3.07] 5. different province [0.31] [0.31] [0.27] [0.07] [0.22] [3.27] [0.14] [3.57] [3.69] [3.00] [3.12] Not interviewed in refused [0.53] [0.46] [0.48] [0.15] [0.41] [4.39] [0.19] [5.97] [7.61] [3.80] [5.33] 7. other HH mem interviewed [0.29] [0.32] [0.29] [0.07] [0.18] [3.20] [0.14] [3.03] [3.20] [3.30] [3.67] Base outcome 8. no contact w/ any HH member [0.30] [0.26] [0.25] [0.07] [0.18] [1.97] [0.10] [3.00] [2.64] [2.32] [2.16] Notes: [Standard errors] of means take into account clustering of sample.

25 Table 4: Earnings and education of movers and stayers in 1993, 2007 and change in earnings over this period Models estimated separately for respondents who were interviewed in same desa in 1993 and 2007 and respondents who had moved from the 1993 desa by Location of interview in 2007 Dependent variable In Desa Out of Desa Difference ln(monthly earnings) Measured in [0.6] [0.6] [0.8] Measured in [0.5] [0.6] [0.7] Change in ln (monthly earnings) ( ) [0.7] [1.2] [1.4] Notes: Dependent variable is ln(monthly earnings) measured in 1993 and 2007 for the same respondents. Estimate of association with years of completed schooling at baseline are reported in the table. Models also control age, gender and location of residence in [Standard errors] take into account clustering.

26 Table 5a: Interviewer characteristics and interview outcomes in 2007 Outcome in 2007 Interviewed Location in 2007 relative to 1993 Ivw member Interviewer location relative to 1993: Same Different in 2007 Same Local Long dist Refuse of 1993 HH Desa Kecamatan Kabupaten Province Province Reference outcome: Not ivwd Not interviewed No information obtained on respondent [A1] [B1] [B2] [B3] [C1] [C2] [C3] [C4] [C5] [C6] [C7] (1) interviewer has survey experience [4.46] [4.61] [2.36] [2.37] [1.45] [0.93] [3.05] [2.02] [2.22] [2.00] [0.47] first job [1.09] [1.36] [0.69] [0.22] [0.51] [0.08] [0.81] [0.62] [0.11] [0.00] [1.23] ln(income in last job) (if had job) [2.77] [2.47] [2.42] [3.22] [2.08] [1.79] [0.02] [0.58] [0.42] [0.28] [0.78] has bachelor's degree [0.46] [0.53] [0.49] [0.46] [0.62] [1.16] [1.04] [0.15] [0.89] [2.08] [0.29] plans to return to school [4.52] [4.72] [2.90] [2.34] [2.34] [4.27] [5.77] [4.81] [4.17] [4.80] [2.22] (1) interviewer math score bottom quartile [0.73] [0.72] [0.29] [0.59] [0.30] [1.25] [0.33] [0.38] [1.80] [0.35] [0.97] top quartile [2.05] [2.25] [1.52] [0.03] [0.23] [0.41] [1.81] [1.47] [1.54] [0.41] [0.85] Interviewer assessment Assertive [0.91] [1.32] [0.41] [0.73] [0.75] [0.94] [1.20] [0.55] [1.16] [0.22] [1.95] Talkative [1.10] [1.12] [0.40] [1.21] [0.39] [0.71] [0.71] [0.33] [0.76] [0.20] [1.63] Shy [0.20] [0.12] [0.22] [0.52] [0.26] [2.12] [1.43] [1.18] [1.58] [2.67] [0.66] Organized [2.26] [2.94] [1.00] [0.57] [0.72] [2.85] [3.79] [1.96] [0.95] [2.44] [1.23] Systematic [1.59] [1.47] [0.08] [2.34] [0.99] [0.63] [1.08] [0.09] [0.71] [1.87] [1.56] Careful [1.47] [1.62] [0.75] [0.67] [0.73] [3.09] [2.82] [2.09] [2.11] [2.26] [1.45] By other team members [0.16] [0.12] [0.62] [0.24] [0.20] [1.11] [0.62] [0.04] [0.68] [0.67] [1.39] Respondent accurate [2.11] [1.71] [3.13] [2.96] [1.54] [0.28] [0.78] [1.84] [2.12] [1.56] [0.22] Length of interview [4.27] [4.52] [1.98] [2.37] [0.13] [0.08] [3.59] [1.85] [2.66] [0.77] [1.12] Proxy respondent [0.29] [0.47] [0.01] [0.34] [0.08] [2.47] [1.29] [1.25] [1.55] [0.61] [2.06] Odds ratios from logistic regression (in model A) and risk ratios (relative to reference outcome) from multinomial logistic regressions (in models B and C). [t statistics] in parentheses take into account clustering of survey. Interviewer characteristics measured in Sample includes only those interviewed in 1997.