Comparison of Two Instruments for Quantifying Intake of Vitamin and Mineral Supplements: A Brief Questionnaire versus Three 24-Hour Recalls

Transcription

1 American Journal of Epidemiology Copyright 2002 by the Johns Hopkins Bloomberg School of Public Health All rights reserved Vol. 156, No. 7 Printed in U.S.A. DOI: /aje/kwf097 Comparison of Two Instruments for Quantifying Intake of Vitamin and Mineral Supplements: A Brief Questionnaire versus Three 24-Hour Recalls Suzanne P. Murphy 1, Lynne R. Wilkens 1, Jean H. Hankin 1, Janet A. Foote 1, Kristine R. Monroe 2, Brian E. Henderson 2, and Laurence N. Kolonel 1 1 Cancer Etiology Program, Cancer Research Center of Hawaii, University of Hawaii, Honolulu, HI. 2 Department of Preventive Medicine, School of Medicine, University of Southern California, Los Angeles, CA. Received for publication November 30, 2001; accepted for publication May 24, Although methods of collecting food intake data have been studied intensively, there have been fewer investigations into the collection of supplement intake data. Use of eight types of vitamin and mineral supplements was reported between 1994 and 1997 by 2,377 subjects participating in a calibration substudy of the Hawaii-Los Angeles Multiethnic Cohort Study. Subjects gave information on supplement use as part of a dietary questionnaire (administered twice) and during three 24-hour dietary recalls. Multivitamins were the most commonly used supplements (55% of the subjects), followed by vitamin C (40%), vitamin E (33%), and calcium (29%). Vitamin A, β-carotene, selenium, and iron supplements were each used by fewer than 10% of the subjects. Weighted kappa statistics for agreement between the recalls and the questionnaire across six categories of frequency of use ranged from 0.74 for vitamin E to 0.16 for vitamin A and were generally higher for frequently used supplements. The reproducibility of questionnaire responses at two time points varied from 0.64 to In comparison with three recalls, a brief questionnaire can accurately and reproducibly capture data on supplement use for frequently consumed products, but it may perform less well for products used less often or more intermittently. data collection; dietary supplements; epidemiologic methods; ethnic groups; nutrition surveys; questionnaires; vitamins In 1997, it was estimated that consumers would spend $11.8 billion that year on dietary supplements, with an expected growth rate of percent over the ensuing 3 years (1). National survey data confirm widespread use of these supplements; in the Third National Health and Nutrition Examination Survey ( ), approximately 40 percent of Americans aged 2 months or older took a dietary supplement during the month prior to interview (2). Use may be even higher in certain population groups; for example, 81 percent of women at risk for breast cancer recurrence reported taking a dietary supplement at least once in a 4-day period (3). When a substantial proportion of a study population is taking dietary supplements, total intake of vitamins and minerals of interest cannot be determined unless supplement use, as well as food consumption, is measured. If supplement use is not considered in nutritional analyses, observed associations between diet and health may be attenuated or even misleading. Although nutritional epidemiologists have urged that accurate data on supplement use be collected (4, 5), there is a paucity of information on both the validity and the reliability of current collection methods. The baseline questionnaire for the Hawaii-Los Angeles Multiethnic Cohort Study contained several questions about the use of vitamin and mineral supplements. Because of the large number of participants, information on brand names was not requested. However, a subsample of participants in a calibration study also completed three 24-hour recalls which included detailed information on supplements consumed and a second questionnaire. In this paper, we present the results of a validation study comparing reported usage from the two instruments and the results of a reproducibility study comparing reported usage over a 2- to 4-year period. Reprint requests to Dr. Suzanne P. Murphy, Cancer Research Center of Hawaii, 1236 Lauhala Street, Suite 407, Honolulu, HI ( suzanne@crch.hawaii.edu). 669

2 670 Murphy et al. MATERIALS AND METHODS Subjects The Hawaii-Los Angeles Multiethnic Cohort Study recruited more than 215,000 adults aged years (6). The cohort was designed to include males and females from five US ethnic groups: African-American, Japanese-American, Latino, Native Hawaiian, and Caucasian. Subjects completed an extensive dietary history questionnaire by mail between 1993 and A random sample of approximately 260 subjects from each sex-ethnicity group was selected to participate in a calibration substudy (7). In addition to the initial dietary questionnaire, these subjects completed three 24-hour recalls at 1-month intervals between 1994 and 1997 and completed a second dietary questionnaire approximately 4 6 weeks later. The analytical sample for the current analyses included subjects who completed both questionnaires and all three recalls (n = 2,377). The study proposal and informed consent procedures were approved by human subjects review boards at the University of Hawaii and the University of Southern California. Questionnaires The dietary questionnaires were self-administered and included questions about the use of eight vitamin and mineral supplements. Subjects were asked to indicate whether any of the following supplements had been used at least weekly during the past year: multivitamins or multivitamins with minerals; vitamin A; vitamin C; vitamin E; β- carotene; calcium; selenium; and iron. If a supplement had been used, subjects were asked to indicate one of five categories of use: 1 3 tablets per week, 4 6 tablets per week, one tablet per day, two tablets per day, or three or more tablets per day. Use of a supplement less often than once a week was defined as never use. For all categories but the multivitamin category, subjects were also asked to indicate the approximate dosage per tablet, choosing from several dose ranges. Nutrient levels in a multivitamin were calculated as composites of levels in the two multivitamin brands most frequently reported in the dietary recalls (Centrum Silver and Centrum Hi Potency (Wyeth Consumer Healthcare, Madison, New Jersey) were reported by 33 percent of persons who gave a brand name). Previous analyses had shown that this default choice minimized errors in intake estimates (8). If a subject did not know the nutrient level of a particular vitamin or mineral supplement consumed, the amount in the lowest amount category was assumed. Recalls Three 24-hour recalls were administered by registered dietitians during telephone interviews. Days of the week were randomly assigned for the interviews in order to obtain a balance of all seven days. Recalls were completed only on days that subjects considered typical of their usual intake. Subjects were asked whether they had used any dietary supplements on the day of the recall. If they had, the type of supplement and the brand name, place of purchase, dosage, number of tablets per dose, and number of tablets taken were recorded. Supplement intakes were then quantified using a supplement composition table compiled and maintained by the Cancer Research Center of Hawaii. Default values based on the most commonly consumed product in the same category were used when no matching composition data could be located (approximately 4 percent of the recall days). To match the six categories of frequency of supplement use that appeared on the questionnaire, we defined approximately equivalent categories for the 3 days of recall: never = not reported on any of the days; 1 3 tablets per week = one tablet reported across all three recalls; 4 6 tablets per week = two tablets reported on the recalls; one tablet per day = 3 4 tablets reported on the recalls; two tablets per day = 5 7 tablets reported on the recalls; and three or more tablets per day = eight or more tablets reported on the recalls. Several assumptions were made when categorizing vitamin and mineral supplement data from the recalls to match the eight types of supplements on the questionnaire. Multiminerals, cod liver oil, and all herbal products were excluded. On the basis of preliminary descriptive analyses, calcium supplements that included vitamin D or magnesium were categorized as calcium supplements, and three products that contained vitamin E and selenium were considered vitamin E supplements. Calculation of amounts consumed For the questionnaire data, frequency of use was multiplied by the dosage reported (or assumed, for multivitamin use) to obtain an estimate of the amount consumed daily for each supplement. For the recall data, the daily amount consumed was the average across the 3 days of intake for each product. For both instruments, the total amount consumed per day was then calculated as the sum of the nutrient levels of multivitamins, if reported, plus the nutrient levels of all single vitamin or mineral supplements reported. For the questionnaire, amounts of the B vitamins and zinc reflect only those amounts contained in multivitamins, because intake of these nutrients in other forms was not assessed. To correspond with current recommendations, vitamins A and E in supplements were converted from International Units to equivalent weight measures (9, 10). Statistical analyses Agreement between the two questionnaires and between the second questionnaire and the recalls was measured in three ways: frequency of use in two categories (ever use, never use); frequency of use in six categories (ranging from never to three or more per day); and nutrient amount. Kappa coefficients were calculated to correct for chance agreement between frequency-of-use categories (11). Weighted kappa coefficients ( κ w ) were used to weight by the level of agreement when frequency of use was divided into six categories. The weighting factors for disagreement were proportional to the square of the distance between the cells in the 6 6 table. Confidence intervals were computed for κ w assuming normality. The homogeneity of kappa statistics between age, sex, and ethnic subgroups was tested using a χ 2 statistic (11).

3 Two Instruments for Quantifying Supplement Intakes 671 TABLE 1. Percentages of respondents using vitamin and mineral supplements according to three dietary recalls and a questionnaire administered at two time points (n = 2,377), Hawaii-Los Angeles Multiethnic Cohort Substudy, Supplement Questionnaire 1 Questionnaire 2 Three 1-day recalls Multivitamin supplement Vitamin A Vitamin C Vitamin E β-carotene Calcium Selenium Iron Any of the above supplements Because the large number of zero values precluded the use of even nonparametric statistical tests, agreement in nutrient amounts was determined only for persons who reported use of a supplement on both instruments. Before these comparisons were conducted, outlier values were excluded, reducing the sample size by less than 1 percent for each nutrient. Pearson s correlation coefficients were calculated for the log values of these continuous variables. RESULTS Supplement use As table 1 shows, approximately two thirds of the subjects reported using one or more dietary supplements. The percentage increased slightly between the first and second questionnaires (from 65.2 percent to 70.7 percent) and, as expected, was somewhat lower for the recalls (62.1 percent). Multivitamins were the most commonly used of the eight supplements examined, followed by vitamin C and vitamin E. Intake of infrequently used supplements, such as vitamin A, β-carotene, selenium, and iron, was substantially underreported on the recalls, because the recall covered only 3 days of use. Most supplement users reported daily use. Over 70 percent of the users of multivitamins, vitamins C and E, and calcium reported taking at least three tablets over the 3 days of the recalls. However, selenium supplements and iron supplements were reported daily on the recalls by only half of the users. Agreement for frequency of use Agreement between the 24-hour recalls and the second questionnaire for frequency of use is shown in table 2. When data were dichotomized into users versus nonusers (never use vs. ever use), agreement was very good (κ > 0.6) for the most commonly used supplements (multivitamins, vitamin C, and vitamin E) and moderately good for the remaining supplements, with the exception of vitamin A. Agreement was similar when use across six frequency-of-use categories was evaluated. The proportion within one category was high ( ). Agreement between the two instruments was relatively constant across gender and age categories (table 3). When differences among the five ethnic groups were examined, average agreement was somewhat lower for African-American and Latino subjects than for Japanese-American, Hawaiian, and Caucasian subjects, and for all nutrients except β-carotene there was a statistically significant lack of homogeneity across the ethnic groups. When the ethnic groups were further divided into males and females, these trends remained (data not shown). Questionnaire reproducibility The reproducibility of the frequency-of-use responses across the two questionnaires is shown in table 2. The average time between the first and second questionnaires was 2.4 years, so a low kappa statistic was likely to reflect changes in the use of the supplement. The nutrient with the greatest consistency across this time period was vitamin C (κ = 0.64), followed by vitamin E, calcium, and multivitamins. Consistency was lowest for vitamin A. To examine whether consistency of use varied with time between administrations of the questionnaire, we divided the elapsed time into tertiles. Weighted kappas declined slightly as the time between administrations increased, but the differences were not large, nor were they consistent across nutrients (data not shown). Averaged across the eight types of supplements, kappa was 0.54 when the elapsed time was less than 704 days (lowest tertile) and 0.48 when the time was greater than 1,016 days (highest tertile). Agreement for supplement amount For subjects who reported use of a supplement on both the recalls and the questionnaire, the amounts consumed are presented in table 4. Among the seven nutrients with dosage amounts from the questionnaire, mean intake on the questionnaire tended to be greater than mean intake from the recalls (or, for β-carotene, approximately equal). As expected, intakes of most of the B vitamins were higher on the recall reports, because use of single supplements of these vitamins was not captured on the questionnaire.

4 672 Murphy et al. TABLE 2. Agreement on frequency of use (never/ever and across six categories*) of vitamin and mineral supplements according to dietary recalls versus the second questionnaire and according to the first questionnaire versus the second questionnaire (n = 2,377), Hawaii-Los Angeles Multiethnic Cohort Substudy, Supplement Kappa for never use vs. ever use 95% CI Weighted kappa for six categories of use 95% CI Proportion within one category Dietary recalls vs. second questionnaire Multivitamin supplements , , Vitamin A , , Vitamin C , , Vitamin E , , β-carotene , , Calcium , , Selenium , , Iron , , First questionnaire vs. second questionnaire Multivitamin supplements , , Vitamin A , , Vitamin C , , Vitamin E , , β-carotene , , Calcium , , Selenium , , Iron , , * Frequency-of-use categories for the questionnaire were: none (<1 tablet per week), 1 3 tablets per week, 4 6 tablets per week, one tablet per day, two tablets per day, and three or more tablets per day. Corresponding categories for the dietary recalls were: none reported, one tablet in 3 days, two tablets in 3 days, 3 4 tablets in 3 days, 5 7 tablets in 3 days, and eight or more tablets in 3 days. CI, confidence interval. Correlations between the amounts reported on the two instruments are also shown in table 4. Correlations were poor ( 0.35) for the nutrients without specific questions on dosage from the questionnaire. Even for supplements that were specifically incorporated into the questionnaire, correlations were low if the supplement was not consumed frequently (vitamin A, selenium, and iron). However, intake of β-carotene was somewhat better correlated between the two instruments (correlation = 0.43), even though this supplement was not frequently reported. In addition, we examined the effect of using a multivitamin default based on a weighted composite of 18 brands, rather than only two brands, and found very similar correlations (data not shown). DISCUSSION Short-term measures (such as 3 days of dietary recalls) may not accurately capture usual use of dietary supplements. Supplements that are taken less often than daily could be missed entirely, as might supplement use from an earlier time period. Thus, it is not surprising that the percentage of participants who used a supplement was lower according to the recalls than according to the questionnaires (table 1). If recall data had been collected for more days, it is likely that usages would have been more similar, particularly for infrequently consumed supplements. Use estimates between the two instruments were most similar for multivitamins, and multivitamins were also the most frequently used type of supplement. The short questionnaire also had limitations, including the need to assume a default value for the composition of multivitamins. The dosage of a supplement was likely to be measured more accurately with the dietary recall method, because participants were asked to give the actual name of the supplement used and, in some cases, to read the name from the label to the interviewer. However, although there was substantial variation in nutrient levels among the multivitamin and mineral products commonly reported during the recalls, correlations with the questionnaire were similar when two different default options were examined. Likewise, Patterson et al. (12) examined the effect of imputing values for three nutrients in multivitamins (vitamin E, folic acid, and iron) and found good correlations with actual intake levels for vitamin E (0.84) and folic acid (0.61), although the correlation for iron (0.29) was poor. In addition, many participants appeared to report complexes of nutrients as single supplements on the questionnaire (e.g., calcium and vitamin D complex was often

5 Two Instruments for Quantifying Supplement Intakes 673 TABLE 3. Agreement on frequency of use of vitamin and mineral supplements according to dietary recalls versus the second questionnaire (weighted kappa value across six categories), by gender, age, and ethnic group, Hawaii-Los Angeles Multiethnic Cohort Substudy, Gender Subgroup No. of subjects Multivitamin supplement * p value comparing the homogeneity of kappa statistics across subgroups. Supplement Vitamin A Vitamin C Vitamin E β-carotene Calcium Selenium Iron Men 1, Women 1, p value* Age group (years) p value* Ethnicity Japanese American Hawaiian Caucasian African-American Latino p value* < <0.01 < <0.01 <0.01 Average reported as a calcium supplement). As a result, amounts of other nutrients in complexes were not included in the calculation of total intake. It is also possible that dosages reported on the questionnaire were less accurate because there were only 3 5 dose categories available, and the lowest category may have been higher than the subject s usual dose. Absolute differences in dosage levels may not be important for many epidemiologic studies, as long as intakes are ranked similarly by the two instruments. However, correlations between the nutrient intake levels were low (<0.4) for three nutrients (vitamin A, selenium, and iron) (table 4), implying that intakes would be ranked differently. With correlations of approximately 0.7 for vitamins C and E, rankings appeared more similar, though absolute nutrient intake levels from the questionnaire were higher. When intake of a nutrient from single supplements is not captured on the questionnaire, it is likely that total intake from supplements will be underestimated (see lower half of table 4). Surprisingly, this was not always the case; intakes of both folate and zinc were higher using the questionnaire data, perhaps because supplemental intake of these nutrients is primarily from multivitamins. Correlations were very poor, however, implying that individual intakes were not ranked similarly by the two instruments. For the remaining B vitamins, the average intake was lower on the questionnaire, but correlations were somewhat higher than those for zinc and folate (ranging from 0.29 to 0.35). Thus, because the questionnaire lacked questions about use of these nutrients as single supplements, nutrient intakes that are based only on multivitamin and mineral supplements may not be useful in analyses of disease outcomes; associations are likely to be severely attenuated. Because more than 2 years elapsed between the two administrations of the questionnaire, this study could not measure true instrument reliability (i.e., the repeatability of responses, assuming no change in supplement intakes). As a result, the agreement between the two responses shown in table 2 is a measure of consistency of supplement use, because change in use would be expected over a period of years. The kappa statistics shown in table 2 for multivitamins, calcium, and vitamins C and E indicate relatively high long-term stability in the use of these supplements. Kappa values were lower for iron and vitamin A, which could reflect changes over time in use of these supplements, as might be expected if iron is taken therapeutically. Agreement tended to decline slightly as the time between administrations increased, supporting the hypothesis that agreement would have been higher if the elapsed time had been shorter (months rather than years). One of the few studies to report on the accuracy of supplement-use data in the United States was conducted by Patterson et al. (5). These authors compared the results of a detailed in-person interview with a short questionnaire similar to the one used in our study. On average, they found no evidence of bias (either underreporting or overreporting) on the questionnaire. Correlations between intakes from the two instruments were remarkably similar to ours, with the highest agreement being observed for vitamins C and E. Unlike dietary recalls, this interview method attempted to capture usual usage. However, it still did not provide a

6 674 Murphy et al. TABLE 4. Agreement on vitamin and mineral supplement use according to dietary recalls versus the second questionnaire (amount consumed per day for persons reporting use on both instruments*), Hawaii-Los Angeles Multiethnic Cohort Substudy, Nutrient No. of subjects Mean intake from second questionnaire * All values include nutrients contained in multivitamins. Based on log-transformed values. Numbers in parentheses, standard deviation. Nutrient values for the questionnaire represent intake from multivitamins only. Daily nutrient intake from supplements Mean intake from dietary recalls Pearson correlation (r) 95% confidence interval Nutrients with questionnaire items Vitamin A (µg of retinol) 985 2,494 (1,817) 1,723 (1,609) , 0.27 Vitamin C (mg) 1, (1,273) 493 (623) , 0.70 Vitamin E (mg) 1, (174) 113 (129) , 0.74 β-carotene (µg) 90 10,240 (7,942) 10,773 (8,694) , 0.58 Calcium (mg) 1, (562) 424 (409) , 0.63 Selenium (µg) (44.3) 33.2 (46.0) , 0.43 Iron (mg) (26.7) 16.4 (16.7) , 0.25 Nutrients without questionnaire items Folate (µg) (238) 315 (163) , 0.17 Niacin (mg) (16.2) 36.4 (47.1) , 0.37 Riboflavin (mg) (1.4) 7.2 (14.3) , 0.39 Thiamin (mg) (1.2) 8.1 (16.3) , 0.40 Vitamin B 6 (mg) (2.0) 8.7 (16.7) , 0.40 Vitamin B 12 (µg) (12.6) 22.9 (39.5) , 0.34 Zinc (mg) (12.1) 15.9 (13.7) , 0.20 completely accurate measure, because it was necessary for participants to provide a single estimate of use (times per week) over the past year, even though usage may have varied over time. In addition, although current supplement labels were examined, other brands that had been used during the past year may not have been captured. In another study, Patterson et al. (13) examined differences between long-term intake (over the past 10 years) and current intake of three dietary supplements and found substantial disagreement, particularly in dosage levels. We also found that reproducibility of intake was low for some nutrients, particularly those that were not frequently reported. Thus, it appears that individuals change their frequency of use of supplements as well as the dosages. Ishihara et al. (14) compared the ability of a questionnaire versus multiple dietary recalls (for four 7-day periods) to classify participants in Japan as users or nonusers of supplements. Having up to 28 days of recalls should allow more accurate classification of users than having only 3 days as in the current study. However, the kappa value calculated from Ishihara et al. s data for use of any supplement was 0.66, which is slightly lower than our kappa of 0.69 for use of multivitamins. If the true measure of interest is long-term exposure, administration of daily or even periodic dietary recalls or records of supplement use is not practical for most studies. However, these analyses indicate that asking more questions on the type of multivitamin used, as well as giving more detailed instructions regarding the classification of multinutrient formulations, would increase the accuracy of questionnaire intake estimates. For nutrients of particular interest in a study, specific questions regarding use of single nutrient supplements should be included. Neither instrument in our study could be considered a gold standard measure of dietary supplement intake. Data from the recalls covered too short a time period to represent usual supplement use, while data from the questionnaire covered a longer time period but did not reflect specific product use. Although our study yielded information about the types of measurement error incurred with the use of questionnaires, more research is needed. A correct assessment of associations between dietary intakes and disease outcomes requires that we better understand how to collect accurate data on supplement use and dosages. Substantial effort has been devoted to understanding the error structure of dietary intake data. A similar effort is needed for supplement intake data. ACKNOWLEDGMENTS This study was supported by grant R01 CA from the National Cancer Institute. The authors thank Lucy Liu Shen for programming assistance.

7 Two Instruments for Quantifying Supplement Intakes 675 REFERENCES 1. Nesheim MC. What is the research base for the use of dietary supplements? Public Health Nutr 1999;2: Ervin RB, Wright JD, Kennedy-Stephenson J. Use of dietary supplements in the United States, (Vital and health statistics, series 11, no. 244). Hyattsville, MD: National Center for Health Statistics, (DHHS publication no. (PHS) ). 3. Newman V, Rock CL, Faerber S, et al. Dietary supplement use by women at risk for breast cancer recurrence. J Am Diet Assoc 1998;98: Block G, Sinha S, Gridley G. Collection of dietary-supplement data and implications for analysis. Am J Clin Nutr 1994; 59(suppl):232S 9S. 5. Patterson RE, Kristal AR, Levy L, et al. Validity of methods used to assess vitamin and mineral supplement use. Am J Epidemiol 1998;148: Kolonel LN, Henderson BE, Hankin JH, et al. A multiethnic cohort in Hawaii and Los Angeles: baseline characteristics. Am J Epidemiol 2000;151: Stram DO, Hankin JH, Wilkens LR, et al. Calibration of the dietary questionnaire for a multiethnic cohort in Hawaii and Los Angeles. Am J Epidemiol 2000;151: Foote JA, Murphy SP, Wilkens LR, et al. Selection of multivitamin default composition to minimize error. (Abstract). Proc Am Assoc Cancer Res 2002;43: National Research Council. Recommended Dietary Allowances. Washington, DC: National Academy of Sciences, Institute of Medicine. Dietary reference intakes for vitamin C, vitamin E, selenium, and carotenoids. Washington, DC: National Academy Press, Fleiss JL. Statistical methods for rates and proportions. New York, NY: John Wiley and Sons, Inc, Patterson RE, Levy L, Tinker LF, et al. Evaluation of a simplified vitamin supplement inventory developed for the Women s Health Initiative. Public Health Nutr 1999;2: Patterson RE, Neuhouser ML, White E, et al. Measurement error from assessing use of vitamin supplements at one point in time. Epidemiology 1998;9: Ishihara J, Sobue T, Yamamoto S, et al. Validity and reproducibility of a self-administered questionnaire to determine dietary supplement users among Japanese. Eur J Clin Nutr 2001;55: