Proxies vs. Self Reports: Data Quality Regarding Potential Duplicate Living Situations

Transcription

1 Proxies vs. Self Reports: Data Quality Regarding Potential Duplicate Living Situations Tiffany R. King, Sarah Cook, RTI International; Jennifer Hunter Childs, US Census Bureau AAPOR 2012 RTI International is a trade name of Research Triangle Institute.

2 Use of proxy reporting in research Methodological concerns The best information about a sampled participant must come from that person Effect on data quality Quality of proxy responses Few studies have concretely demonstrated better measurement quality in self reports over proxy reports Necessity Cost containment - rising cost of data collection Response rates (Moore, 1988; Chestnut, 2005) 2

3 The focus of this paper. We specifically examine quality of living situation information provided by the proxy vs. targeted duplicate respondent in the Qualitative Interviewing with Suspected Duplicates and Cognitive Testing of the Targeted Coverage Follow-up (TCFU) Interview study 3

4 The Cognitive Study Aimed to assess the efficacy of the TCFU questionnaire Did it uncover duplicate addresses across a broad range of living situations? Aimed to collect in depth information on different living situations What situations are individuals more likely to be listed at more than one address? 226 Cognitive interviews were conducted 130 respondents were the targeted duplicates 96 respondents were proxy respondents 4

5 Methods We compare data quality (proxy vs. self-report duplicate) across various types of information provided in the interview: Revealing (verification) of duplicate address Completeness of dates and addresses provided Rates of DK & Refusals Privacy concerns Interviewer observations 5

6 Methods (cont.) Who were proxies? An adult household member (1) When the duplicate respondent was unavailable or a child (2) Had to be an adult listed on the original census form for the household Limitations Small-scale site test Not a random sample 6

7 Verifying the duplicate address Self-report were slightly more successful than proxy interviews in verifying the duplicate address Table 1. Case Outcome by Self-Report (Nonproxy) versus Proxy Respondent Self-Report Case Outcome (NonProxy) Proxy Verified/Resolved Duplicate Situation* 91 (70%) 65(67.7%) Unresolved Situation 19 (14.6%) 19 (19.8%) False Match or Suspected False Match 20 (15.4%) 12 (12.5%) 7 Total *Included is one case of a Housing Unit Mix-Up, in which the people were duplicated unknowingly. It is included in the Resolved category because through the interviewing procedures, we were able to resolve this situation

8 Completeness of dates and addresses More often the actual duplicate respondent (self-report) was able to provide complete addresses during the interview about their living situations. Table 2. Completeness of Addresses Provided by Proxy versus Self-Report Only Complete Addresses Only Partial Addresses Both Partial and Complete Addresses No Addresses Provided Total Cognitive Interviews Proxy 20 (21%) 31 (32%) 25 (26%) 20 (21%) 96 Self-Report 36 (28%) 16 (12%) 30 (23%) 48 (37%) 130 8

9 Completeness of dates and addresses More duplicate respondents provided only complete dates throughout the interview than proxy respondents. Table 3. Completeness of Dates Provided by Proxy versus Self- Report Respondent Only Complete Dates Only Incomplete Dates Both Complete and Incomplete Dates No Dates Provided Proxy 22 (23%) 6 (6%) 7 (7%) 61(64%) Self-Report 45 (35%) 5 (4%) 13 (10%) 67(51%) 9

10 Rates of Don t Know and Refusals All of the don t know responses came from proxy respondents. Table 4. Don t Know to Gate Questions by Proxy versus Self-Report Number of "Don't Know" Responses Proxy Self-Report Stayed anywhere else, even for just one night or on the weekend 4 0 Stayed with any other people, such as friends 2 0 Stay anywhere for financial problems 1 0 Stayed with a brother or sister 1 0 Stayed anywhere else, even for just one night or on the weekend 1 0 Stayed at a seasonal or vacation home 1 0 Spent even one night on base housing 1 0 Spent even one night in a military academy 1 0 Spent even one night in any of the following places 1 0 TOTAL 13 (100%) 0 (0%) 10

11 Rates of Don t Know and Refusals The majority of refusals were from Self-Report (nonproxy) respondents. Table 5. Number of Refusals to TCFU Questions by Proxy or Self-Report (N=14 [1] ) TCFU Question [1] Where N is the number of participants who refused at least once. Proxy Self-Report (NonProxy) Names of the people who usually live there 4 8 What was the address of that place? 2 3 Was [dup] there on April 1 st, 2010? 2 Stay anywhere else 1 Did [dup] move? 1 Total 6 (29%) 15 (71%) 11

12 Other observations Privacy Concerns 27% (26/96) proxy versus 22% (29/130) self-report (nonproxy) respondents explicitly expressed privacy concerns Interviewer Observations Privacy concerns when duplicate was a minor child Privacy concerns when duplicate was in a socially undesirable situation (jail, shelter etc.,) 12

13 Conclusions Conclusions Overall self-report (nonproxy) respondents provided more complete information than proxy respondents Asking either proxy respondents about the suspected duplicate or self-report (nonproxy) respondents about other households members with whom they stayed sometimes caused privacy concerns Key Asking about OTHER PEOPLE was sometimes sensitive 13

14 Future Research Examine these data items in a production setting Look at item completeness and refusal rates by proxy status with the goal of determining: Do these same findings hold up in a nationally representative sample? 14

15 More Information Tiffany R. King Research Survey Methodologist Sarah Cook Survey Methodologist Jennifer Hunter Childs Research Psychologist 15