Does it Matter How You Ask? Question Wording and Males' Reporting of Contraceptive Use at Last Sex Jennifer Yarger University of Michigan Sarah Brauner-Otto Mississippi State University Joyce Abma National Center for Health Statistics NSFG Survey Methodology Working Papers, No. 10-010 Revised June 2011
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 2 ABSTRACT This paper reports results from a unique experiment conducted in the 2002 National Survey of Family Growth (NSFG) to gauge the effect of question format on men s reports of contraceptive use at last sexual intercourse. We examine whether receiving separate questions about the respondent s own and his partner s contraceptive use, as opposed to one combined question about either partner s use, is related to higher reports of using any contraceptive methods, using specific methods, and the reports of female methods in addition to male methods. We found that reports of any contraceptive use at last sex and use of the most common methods, condom and the pill, were stable across question formats. However, we find significantly higher reports of withdrawal and injection or hormonal implant among men who received the separate-question format. We also find that length of time since last sex conditions the effect of question format on men s reports, particularly their reports of both male and female method use.
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 3 INTRODUCTION Researchers have increasingly acknowledged the importance of including men in studies of contraceptive behavior (Edwards 1994; Forste 2002; Gillmore et al. 2003; Landry and Camelo 1994). However, studies that ask men to report on contraceptive methods may suffer from greater reporting error than similar studies of women. Men often lack direct knowledge of their partners contraceptive use, relying instead on being told or seeing physical evidence of a method, such as a pill pack; research has found that proxy reporting or reporting on another s behavior is more susceptive to measurement error (Sudman, Bradburn, and Schwarz 1996). Survey researchers may be able to reduce measurement error in men s reports of contraceptive use with careful selection of question wording and format. Current largescale surveys have varied in how they ask questions about contraceptive use. For example, the National Longitudinal Study of Adolescent Health (Add Health), the Youth Risk Behavior Survey (YRBS), and the National Survey of Family Growth (NSFG) asked a single question about the contraceptive methods a couple uses, while the National Survey of Adolescent Males (NSAM) asked separate questions about the respondent s and his partner s method. Studies have shown that variation in question format and wording may contribute to differences in estimates of method use (Catania et al. 1990; Santelli et al. 2000). This paper reports findings from an experiment on question format to help us understand whether and, if so, how these differences in question wording and format influence men s reports of contraceptive use. Evidence from survey research suggests that a separate-question format may help address the concern that men misreport their female partner s method use. For example, asking multiple questions and focusing the second question on less memorable information aids recall (Cannell and Fowler 1965; Cannell, Marquis, and Laurent 1977; Groves et al. 2004; Sudman and Bradburn 1982). Thus, a separate, specific question about female methods might help men recall the methods their partner used. In addition, motivating respondents to be complete in their responses can be critical for producing accurate information (Groves et al. 2004). Asking a separate question about their
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 4 partners method use may encourage men to provide a more thorough report of the contraceptives used at last sex. The NSFG, the largest national study of contraceptive use, provides an important new measurement resource for understanding the impact of question wording and format on men s contraceptive reporting. In 2002, the NSFG included a split-sample experiment to allow analysis of how a single- or separate-question format affects men s contraceptive reporting. Some male respondents were randomly chosen to receive a single, combined question, while others received separate questions about their own and their partner s contraceptive use. This paper analyzes the results of the experiment, examining how men s reports of contraceptive use vary between the single- and separate-question groups. The lack of an objective truth against which to measure estimates of contraceptive use precludes evaluation of which question format yields the more accurate report. Rather, we investigate the degree to which men s reporting of contraceptive use is sensitive to question format. We hypothesize that the separate-question format will be positively associated with reports of any method use, each specific method use, and using female methods rather than only male methods or no method. We also test for interactions between question format and the amount of time passed since the respondent s last sexual experience. Specific details about events tend to be more difficult to recall with time (Rubin and Wetzel 1996). Thus, we expect that men whose last sexual experience occurred further in the past will be more sensitive to the question format. DATA AND METHODS We use data from Cycle 6, the 2002 round of the NSFG, which included 4,928 completed interviews of men 15-44 years of age and an overall male response rate of 78%. For a detailed discussion of sampling procedures and study design see Groves et al. (2005) and Lepkowski et al. (2006). We analyze data from the 4,109 men who reported having had sexual intercourse with opposite-sex partners.
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 5 Measure of Question Format Respondents were randomly assigned to receive either the combined-question or separate-question format. For each format, Table 1 shows the question wording and categories of contraceptives, which were printed on cards shown to the respondents. We used a dichotomous measure for question format, coded 1 for the separate-question format (70%, by design) and 0 otherwise. Table 2 shows the weighted mean of this and all other measures used in these analyses. Measures of Contraceptive Method Use We created a series of dichotomous measures of contraceptive method(s) used at last sex. The first measure is coded 1 if the respondent reported using any method and 0 otherwise. In all, 71% of men aged 15 to 44 who had ever had sex used a contraceptive method at last sex. We created three dichotomous measures for male-specific methods: (1) condoms; (2) withdrawal; (3) vasectomy or male sterilization. Each measure is coded 1 if the respondent reported using that specific method and 0 otherwise. Condoms were the most commonly reported male method at 32%. We also created five dichotomous measures for each of the female-specific methods: (1) pill; (2) tubal ligation or female sterilization; (3) injection or hormonal implant; (4) spermicidal foam/jelly/cream/film/suppository; (5) rhythm or safe period. 1 Each method is coded 1 if the respondent reported his partner used that method and 0 otherwise. The pill was the most commonly reported female method used, with 26% of men reporting their partners used it. Only 1% of men reported using a spermicidal foam etc. or the rhythm method, which prevents us from including these measures in the multivariate analyses. 1 We do not investigate reports of using something else, which was the actual wording of the response category presented to respondents. Respondents were not asked to specify the other method, so we lack sufficient information to warrant analysis.
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 6 Finally, because our theory predicts that asking men separately about their partner s contraceptive use will elicit more reports of female methods in particular, we create a categorical measure summarizing reports of male- and female-specific method use. The responses are coded into four categories: (1) no method; (2) male method only; (3) female method only; (4) both male and female methods. Measures of Time since Last Sexual Experience We created a dichotomous variable equal to 1 if the time since the respondent s last sexual encounter was at least one month and 0 if it occurred more recently. We also investigated a continuous measure of time since last sex. However, since the majority of men had sex within the same month as the interview, the dichotomous variable best operationalizes the concept and the respondents ability to recall details about contraception. Controls All analyses control for three key sociodemographic characteristics. Self-reported race/ethnicity is included as a categorical variable with four categories: 1) non-hispanic white (comparison group); 2) non-hispanic black, 3) Hispanic black; 4) non-hispanic other race. Age at the time of the interview is included as a continuous variable. Marital status is included as a dichotomous indicator equal to 1 if the respondent was married or cohabiting with his last sexual partner and 0 otherwise. Analytic Strategy First, we estimate separate logistic regression models to investigate whether question format is related to reports of any contraceptive method use and specific method use. Next, we estimate multinomial logistic regression models to investigate whether question format influences the likelihood of men reporting male, female, or both male and female methods. For both approaches we test for interactions between question format and time since the last sexual experience. In each model, we account for the clustering resulting from the complex survey design.
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 7 RESULTS Table 3 presents the results from logistic regressions of the relationship between question format and reports of any contraceptive use and the use of each specific method. The table presents beta coefficients with standard errors in parentheses. Model 1 indicates that question format does not significantly affect men s probability of reporting any contraceptive use overall. However, of the specific methods, question format is positively related to reporting withdrawal (Model 3) and injections or hormonal implants (Model 7). Men who received the separate-question format had 0.38 higher log odds of reporting withdrawal than men who received the combined question. Similarly, men who received the separate-question format had 0.40 higher log odds of reporting that their partners had injections or hormonal implants. We also tested for interactions between question format and time since the last sexual experience (results not shown). This interaction effect was positive and marginally significant for reporting the pill; the effect of the separate-question format on reporting pill use was stronger for men whose last sexual experience occurred at least a month prior to the interview. This finding suggests that the separate-question format may exert a stronger influence in situations when contraceptive method use is more difficult to recall. Table 4 presents the results from multinomial regression models of the relationship between question format and reports of a female method in combination with a male method. The only significant effect of question format in these models was for the comparison of reporting both a male and female method versus no method. Model 3 shows that men who received the separate-question format were more likely to report using both a male and female method than they were to report not using a method. Counter to our hypothesis, question format was not significantly related to reporting both a male and female method as opposed to only a male method. However, it is important to note that this table shows us results on average, across all men. Table 5 shows the multinomial regression models with interactions by time since last sex. Here we see that the effect of question format on men s reports of their partners method use varies based on time since last sex. As predicted, question format is more influential for men whose last sex occurred further in the past. Furthermore, the interaction effect is strongest when comparing using both a male and female method to using a male method only.
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 8 DISCUSSION This study utilized a survey methodology experiment on question format embedded in the NSFG to examine how manipulating question format affects men s reports of contraceptive use. The findings are particularly relevant for designing future studies of contraceptive use and analyzing existing contraceptive use data. In addition, the findings may be important for studies of other behaviors that are commonly conceptualized at the couple level and asked about in proxy interviews, such as domestic violence. We found that reports of any contraceptive use at last sex and use of the most common methods, condom and the pill, were stable across question formats. The stability of reports of condom use across question format is an encouraging indication of data quality for this method, which is essential for men s roles in preventing pregnancy and STIs. On the other hand, we found significantly higher reports of withdrawal and injection or hormonal implant among men who received the separate-question format. Reporting withdrawal may be sensitive to question format because withdrawal is often used as a secondary method in combination with a female method, and when asked about male and female methods together, men may report the primary method only. Reporting injection or hormonal implant may be sensitive to question format because the methods offer no physical evidence and, therefore, men may need more assistance recalling that their partner has used the method. Thus, researchers who are particularly interested in capturing men s reports of withdrawal or injection/hormonal implant should consider using the separate-question format. Furthermore, we initially found that the separate-question format was positively associated with reporting both male and female method use but only compared to reporting no methods. However, by examining interactions between question format and time since the last sexual experience, we learned that subgroup differences were diluting the effect of question format. In particular, the effect of the separate-question format on reporting male and female method use compared to only male methods was stronger among men whose last sexual experience was at least one month before the interview. This finding suggests that the separate-question format may help improve data quality by
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 9 facilitating recall and that survey designers and analysts should attempt to limit questions or analysis to recent events when possible. Although care should be exercised when interpreting the relationship between the question format and methods reported, there are reasons to believe that the basic tenets of survey design are operating to enhance data quality by simplifying questions, aiding recall, and encouraging men to report their partners method use. The two-question format was retained in the NSFG male questionnaire subsequent to 2002, and the singlequestion format discontinued. The two question format retains comparability with prior male surveys, and maximizes simplicity of the recall and response task. Thus although the results for data quality are not definitive, the benefits of keeping the two-question format outweighed those for the single-question format. One concern is that the separate-question format may encourage men to over report contraceptive use due to social desirability bias. Earlier analysis of the NSFG s preliminary data for this experiment showed that the difference between young men and women s reports of condom use is greater for the separate-question format than the combined-question format (Abma, Porter, and Sonenstein 2003). More of this type of analysis, and perhaps analysis of couples reports, may help illuminate to what degree social desirability plays a role in sensitivity to question format.
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 10 REFERENCES Abma, J. L. Porter, and F. Sonenstein. 2003. Men s Reports of Contraceptive Use: The Effect of Question Variation and the Extent of Uncertainty. Paper presented at the Annual Meeting of the Population Association of America (Minneapolis, MN). Cannell, C. and F. Fowler. 1965. Comparison of Hospitalization Reporting in Three Survey Procedures. Vital and Health Statistics, Series 2, No. 8. Washington, DC: Govt Printing Office. Cannell, C., K. Marquis, and A. Laurent. 1977. A Summary of Studies. Vital and Health Statistics, Series 2, No. 69. Washington, DC: Government Printing Office. Catania, J., D. Gibson, D. Chitwood, and T. Coates. 1990. Methodological Problems in AIDS Behavioral Research: Influences on Measurement Error and Participation Bias in Studies of Sexual Behavior. Psychological Bulletin 108:339-362. Edwards, S. 1994. The Role of Men in Contraceptive Decision-Making: Current Knowledge and Future Implications. Family Planning Perspectives 26:77-82. Forste, R. 2002. Where Are All the Men? A Conceptual Analysis of the Role of Men in Family Formation. Journal of Family Issues 23:579-600. Gillmore, M., S. Stielstra, B. Huang, S. Baker, B. Beadnell, and M. Morrison. 2003. Heterosexually Active Men's Beliefs about Methods for Preventing Sexually Transmitted Diseases. Perspectives on Sexual and Reproductive Health 35:121-129. Groves, R., G. Benson, W. Mosher, J. Rosenbaum, P. Granda, W. Axinn, J. Lepkowski, and A. Chandra. 2005. Plan and Operation of the 2002 National Survey of Family Growth. Vital and Health Statistics, Series 1, No. 42. Hyattsville, MD: National Center for Health Statistics. Groves, R., F. Fowler, M. Couper, J. Lepkowski, E. Singer, and R. Tourangeau, R. 2004. Survey Methodology. Hoboken, NJ: John Wiley and Sons, Inc. Landry, D. and T. Camelo. 1994. Young Unmarried Men and Women Discuss Men's Role in Contraceptive Practice. Family Planning Perspectives 26:222-227. Lepkowski, J., W. Mosher, K. Davis, R. Groves, J. van Hoewyk, and J. Willem. 2006. National Survey of Family Growth, Cycle 6: Sample Design, Weighting, Imputation, and Variance Estimation. Vital and Health Statistics, Series 2, No. 142. Hyattsville, MD: National Center for Health Statistics. Rubin, D. and A. Wetzel. 1996. One Hundred Years of Forgetting: A Quantitative Description of Retention. Psychological Review 103:734-760. Santelli, J., L. Lindberg, J. Abma, C. McNeely, and M. Resnick. 2000. Adolescent Sexual Behavior: Estimates and Trends from Four Nationally Representative Surveys. Family Planning Perspectives 32:156-165, 194. Sudman, S. and N. Bradburn. 1982. Asking Questions. San Francisco: Jossey-Bass. Sudman, S., N. Bradburn, and N. Schwarz. 1996. Thinking about Answers: The Application of Cognitive Processes to Survey Methodology. San Francisco: Jossey-Bass.
Question Wording and Males' Reporting of Contraceptive Use at Last Sex 11 Table 1. Questions in Experiment Question Format Wording Combined-question 1a) Please look at this card. That last time you had sexual intercourse with (partner), did you or she use any methods to prevent pregnancy or sexually transmitted disease? (Yes/No) 1b) (If yes): Please look at this card. That last time, what methods did you and she Condom or rubber 1 Withdrawal or pulling out 2 Vasectomy or male sterilization 3 Pill 4 Tubal ligation ( tubes tied ) or female sterilization 5 Injection (Depo-provera) 6 Spermicidal foam/jelly/cream/film/suppository 7 Hormonal implant (Norplant) 8 Rhythm or safe period 9 Something else 10 Separate question 1a) Please look at this card. That last time you had sexual intercourse with (partner), did you yourself use any methods to prevent pregnancy or sexually transmitted disease? (Yes/No) 1b) (If yes): Please look at this card. That last time, what methods did you use? Condom or rubber 1 Withdrawal or pulling out 2 Vasectomy or male sterilization 3 Something else 10 2a) Please look at this card. That last time you had sexual intercourse with (partner), did she use any methods to prevent pregnancy or sexually transmitted disease? (Yes/No) 2b) (If yes): Please look at this card. That last time, what methods did she use? Pill 4 Tubal ligation ( tubes tied ) or female sterilization 5 Injection (Depo-provera) 6 Spermicidal foam/jelly/cream/film/suppository 7 Hormonal implant (Norplant) 8 Rhythm or safe period 9 Something else 10
Question Wording and Males' Reporting of Contraceptive Use at Last Sex Table 2. Weighted Descriptive Statistics Mean Characteristic (N=4,109) Survey Experiment Separate question format 0.71 Contraceptive Method Use at Last Sex Used any contraception 0.71 Used condom or rubber 0.32 Used withdrawal or pulling out 0.07 Used vasectomy or male sterilization 0.05 Used pill 0.26 Used tubal ligation or female sterilization 0.08 Used injection or hormonal implant 0.03 Used spermicidal foam/jelly/cream/film/suppository 0.01 Used rhythm or safe period 0.01 Contraceptive use by gender-specific type a No method 0.30 Male method only 0.31 Female method only 0.27 Male and female methods 0.12 Time Since Last Sex Last sex at least one month before interview 0.33 Controls Age at interview 31.33 Race/ethnicity Hispanic 0.17 Non-Hispanic white 0.65 Non-Hispanic black 0.12 Non-Hispanic other 0.06 Married or cohabiting with partner 0.65 Note: Means are based on weighted data. a N=4,032
Question Wording and Males' Reporting of Contraceptive Use at Last Sex Table 3. Logistic Regression Coefficients for the Effects of Question Format on Any Method Use and Specific Method Use (N=4,109) Any Injection/ contraceptive Tubal Hormonal method Condom Withdrawal Vasectomy Pill ligation implant 1 2 3 4 5 6 7 Separate question format 0.16-0.02 0.38* -0.09 0.12-0.08 0.40+ (0.13) (0.11) (0.18) (0.29) (0.14) (0.22) (0.24) Last sex at least one month before interview -0.43** 0.45*** -0.58** -0.87* -0.26** -0.79** -0.25 (0.14) (0.09) (0.19) (0.41) (0.11) (0.26) (0.32) Age at interview -0.04*** -0.06*** -0.05*** 0.16*** -0.06*** 0.11*** -0.07*** (0.01) (0.01) (0.01) (0.02) (0.01) (0.01) (0.02) Race/ethnicity a Hispanic -0.45** 0.09-0.02-1.23** -0.73*** 0.25 0.60* (0.14) (0.11) (0.18) (0.43) (0.14) (0.29) (0.28) Non-Hispanic black -0.48** 0.47*** -0.46* -1.75*** -0.65*** 0.36+ 0.15 (0.16) (0.13) (0.24) (0.47) (0.14) (0.22) (0.28) Non-Hispanic other 0.11 0.71** -0.14-1.19+ -0.85*** 1.44*** -0.39 (0.26) (0.23) (0.39) (0.87) (0.24) (0.41) (0.51) Married or cohabiting with last sexual partner -1.36*** -1.81*** -0.33* 1.45*** -0.37** 0.76** 0.74** (0.16) (0.11) (0.17) (0.36) (0.12) (0.25) (0.26) Constant 3.21*** 1.83*** -0.85** -9.24*** 1.10*** -6.65*** -2.07*** Note: Standard errors in parentheses. a Reference group is non-hispanic white. + p<.10; * p<.05; **p<0.01; *** p<.001 (one-sided tests) (0.26) (0.22) (0.32) (1.01) (0.24) (0.52) (0.51)
Question Wording and Males' Reporting of Contraceptive Use at Last Sex Table 4. Multinomial Logistic Regression Coefficients for the Effects of Question Format on Type of Contraceptive Method Used (N=4,032) Male only vs. none Female only vs. none Male and female vs. none Female only vs. male only Male and female vs. male only Male and female vs. female only 1 2 3 4 5 6 Separate question format 0.12 0.14 0.29* 0.02 0.17 0.15 (0.15) (0.17) (0.16) (0.16) (0.17) (0.20) Last sex at least one month before interview -0.19-0.64*** -0.48** -0.45*** -0.29** 0.16 (0.15) (0.17) (0.16) (0.14) (0.10) (0.15) Age at interview -0.03*** -0.02** -0.08*** 0.01-0.05*** -0.06*** (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) Race/ethnicity a Hispanic -0.32* -0.51** -0.71*** -0.19-0.39** -0.20 (0.15) (0.19) (0.16) (0.17) (0.15) (0.19) Non-Hispanic black -0.27+ -0.65** -0.46* -0.38* -0.19 0.19 (0.17) (0.22) (0.20) (0.19) (0.18) (0.22) Non-Hispanic other 0.23-0.03 0.33-0.25 0.10 0.35 (0.26) (0.34) (0.42) (0.27) (0.35) (0.40) Married or cohabiting with last sexual partner -1.68*** -0.59*** -2.13*** 1.09*** -0.44** -1.53*** (0.16) (0.17) (0.21) (0.12) (0.15) (0.15) Constant 2.35*** 1.40*** 3.01*** -0.95** 0.66* 1.61*** Note: Standard errors in parentheses. a Reference group is non-hispanic white. + p<.10; * p<.05; **p<0.01; *** p<.001 (one-sided tests) (0.27) (0.36) (0.33) (0.33) (0.29) (0.38)
Question Wording and Males' Reporting of Contraceptive Use at Last Sex Table 5. Multinomial Logistic Regression Coefficients for the Effects of Question Format on Contraceptive Method Use, Including Interactions by Time Since Last Sex (N=4,032) Male only vs. none Female only vs. none Male and female vs. none Female only vs. male only Male and female vs. male only Male and female vs. female only 1 2 3 4 5 6 Separate question format 0.20 0.15 0.10-0.05-0.09-0.04 (0.19) (0.19) (0.22) (0.18) (0.24) (0.24) Last sex at least one month before interview -0.06-0.62** -0.86*** -0.56** -0.80** -0.24 (0.21) (0.24) (0.25) (0.23) (0.26) (0.26) Last sex at least one month before interview Separate question format -0.19-0.04 0.50+ 0.15 0.68* 0.53+ (0.25) (0.31) (0.32) (0.28) (0.32) (0.34) Age at interview -0.03*** -0.02** -0.08*** 0.01-0.05*** -0.06*** (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) Race/ethnicity a Hispanic -0.32* -0.51** -0.71*** -0.19-0.39** -0.20 (0.15) (0.19) (0.16) (0.17) (0.15) (0.19) Non-Hispanic black -0.28+ -0.65** -0.45* -0.38* -0.18 0.20 (0.18) (0.22) (0.20) (0.18) (0.18) (0.22) Non-Hispanic other 0.23-0.02 0.34-0.25 0.11 0.36 (0.26) (0.34) (0.43) (0.27) (0.35) (0.41) Married or cohabiting with last sexual partner -1.68*** -0.59*** -2.13*** 1.09*** -0.45** -1.53*** (0.16) (0.17) (0.21) (0.12) (0.14) (0.15) Constant 2.30*** 1.40*** 3.13*** -0.90** 0.83** 1.73*** Note: Standard errors in parentheses. a Reference group is non-hispanic white. + p<.10; * p<.05; **p<0.01; *** p<.001 (one-sided tests) (0.28) (0.36) (0.35) (0.34) (0.32) (0.39)