Effect of Screening Out Implausible Energy Intake Reports on Relationships between Diet and BMI

Transcription

1 Diet and Physical Activity Effect of Screening Out Implausible Energy Intake Reports on Relationships between Diet and BMI Terry T.-K. Huang, Susan B. Roberts, Nancy C. Howarth, and Megan A. McCrory Abstract HUANG, TERRY T.-K., SUSAN B. ROBERTS, NANCY C. HOWARTH, AND MEGAN A. MCCRORY. Effect of screening out implausible energy intake reports on relationships between diet and BMI. Obes Res. 2005;13: Objective: We present an updated method for identifying physiologically implausible dietary reports by comparing reported energy intake (rei) with predicted energy requirements (per), and we examine the impact of excluding these reports. Research Methods and Procedures: Adult data from the Continuing Survey of Food Intakes by Individuals 1994 to 1996 were used. per was calculated from the dietary reference intake equations. Within-subject variations and errors in rei [coefficient of variation (CV) 23%] over 2 days (d), per (CV 11%), and measured total energy expenditure (mtee; doubly labeled water, CV 8.2%) were propagated, where 1 SD CV 2 rei/d CV 2 per CV 2 mtee 22%. Thus, a report was identified as implausible if rei was not within 78% to 122% of per. Multiple cut-offs between 1 and 2 SD were tested. Results: %rei/per 81% in the total sample (n 6499) and progressively increased to 95% in the 1 SD sample (n 2685). The 1 to 1.4 SD samples yielded rei-weight associations closest to the theoretical relationship (mtee to weight). Weak or spurious diet BMI associations were present in the total sample; 1 to 1.4 SD samples showed the strongest set of associations and provided the maximum n while maintaining biological plausibility. Received for review September 2, Accepted in final form April 15, The costs of publication of this article were defrayed, in part, by the payment of page charges. This article must, therefore, be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact. Jean Mayer USDA Human Nutrition Research Center on Aging and Gerald J. and Dorothy R. Friedman School of Nutrition Science and Policy, Tufts University, Boston, Massachusetts. Address correspondence to Megan A. McCrory, School of Nutrition and Exercise Science, Bastyr University, Juanita Dr. NE, Kenmore, WA mmccrory@bastyr.edu Copyright 2005 NAASO Discussion: Our methodology can be applied to different data sets to evaluate the impact of implausible reis on health outcomes. Implausible reis reduce the overall validity of a sample, and not excluding them may lead to inappropriate conclusions about potential dietary causes of health outcomes such as obesity. Key words: energy intake, underreporting, energy requirement, eating patterns, adults Introduction Physiologically implausible energy intake reporting, particularly underreporting, is widespread among dietary data sets, with mean reported energy intakes (reis) 1 10% to 50% lower than actual mean energy intake being common (1 3). In general, the magnitude of underreporting increases with increasing body mass (4). This is problematic because it gives the appearance that overweight and obese individuals consume less than or the same amount of energy as their normal weight counterparts (2,5,6), an observation that is inconsistent with doubly labeled water studies showing increasing total energy expenditure (TEE) in individuals with increasing weight (1,2) and BMI (7,8). In addition, concurrent with implausible reis is the increased likelihood of altered proportions of nutrients such as fat, sugar, protein, and/or fiber (9 16). Associations between nutrient intakes and body weight status derived using data sets inclusive of implausible reis are, therefore, also likely to be altered (11,12,17 22), which could impact not only the interpretation of results, but also inferences drawn about potential dietary contributors to obesity. Although progress in methodology has led to improvements in the quality of dietary data collected (e.g., multiple 1 Nonstandard abbreviations: rei, reported energy intake; TEE, total energy expenditure; mtee, measured TEE; BMR, basal metabolic rate; PAL, physical activity level; CSFII, Continuing Survey of Food Intake by Individuals; per, predicted energy requirement; DRI, Dietary Reference Intake; CV rei, intraindividual variation in energy intake reporting; CV per, the error in the equations for per; CV mtee, measurement error and day-to-day biological variation in TEE; FAFH, food away from home. OBESITY RESEARCH Vol. 13 No. 7 July

2 pass 24-hour recall) (23,24), implausible reporting still exists. Therefore, there has been growing interest in developing methods to contend with implausible reis during analysis. One common strategy is to exclude outlying or extreme values based on data distribution or subjective assessment. Alternatively, Goldberg et al. (25) has suggested determining the validity of rei by comparing it with TEE when both are expressed as a multiple of basal metabolic rate (BMR) and using 95% confidence limits ( 2 SD) for a statistical comparison of rei/bmr with physical activity level (PAL, or TEE/BMR). With this method, a standard value for PAL is used if it is not known. Recently, we proposed another option (17) to screen for implausible reports by comparing rei directly with TEE (either predicted or measured) using cut-offs for their agreement based on error propagation calculations. In preliminary analysis of data from participants 21 to 45 years of age in the Continuing Survey of Food Intake by Individuals (CSFII) 1994 to 1996 (17), we found that implausible reis obscured relationships between diet and BMI and also confirmed previous suggestions (21) that a cut-off less than 2 SD may be needed to identify a sample with sufficient biological plausibility. However, that analysis was limited by the small sample used to generate the prediction equations of TEE (17). In this paper, we have extended our previous analysis to CSFII 1994 to 1996 participants 20 to 90 years of age and improved our method by using the Institute of Medicine s recently published prediction equations for TEE [or predicted energy requirement (per)] based on 700 measurements by doubly labeled water (7). We also calculated several cut-offs for the agreement between rei and per and tested which ones yielded the closest association with body weight as expected biologically. We assessed associations of rei and select eating patterns with BMI using the different cut-offs and, finally, examined socio-demographic characteristics associated with implausible reporting. Research Methods and Procedures Subjects Nationally representative data of noninstitutionalized adults (20 to 90 years of age) from the USDA CSFII 1994 to 1996 were used (26). The CSFII 1994 to 1996 included two nonconsecutive 24-hour dietary recalls. Individuals were included for analysis if they completed both 24-hour dietary recalls, were not on prescription diets, did not experience food insecurity, provided both weight and height data, did not fail to report the time of consumption for any eating occasion, and, in the case of women, were not pregnant or lactating. This resulted in a total n 6499 before screening out implausible reporters. Standardization of Eating Occasions Subjects reported the type of eating occasion at which each food was consumed: breakfast, brunch, lunch, dinner, supper, or snack. However, this self-report resulted in nonstandardized coding of eating occasions and multiple same meals consumed in the same day. Therefore, meals were standardized to include not more than one each of breakfast, lunch, and dinner, but allowing for multiple snacks, as follows. If two or more meals of the same eating occasion were reported within 59 minutes of each other, they were considered one meal and combined, using the average of the consumption times. Otherwise, the occasion with the largest energy content was coded as a meal and the others coded as snacks. Brunch was coded as breakfast if it occurred before 11:00 AM and lunch if it occurred between 11:00 AM and 4:00 PM. In some cases, lunch was missing, but dinner and supper were both reported (some individuals named the current conventional concept of lunch in the United States, i.e., the midday meal, dinner). If any dinner and any supper occurred more than 1 hour apart, and dinner had lower energy than supper, dinner was coded as lunch and supper coded as dinner. If more than one dinner was reported, a second dinner could not be recoded to lunch unless the first dinner was already recoded to lunch, to preserve the temporal order of meals. Finally, daily totals of nutrient values were calculated across all meals and snacks. Screening for Potentially Implausible Reports To identify potentially implausible reis, we calculated the 1 SD cut-off (and multiples of 1 SD in subsequent steps) for rei as a percentage of per (i.e., rei/per 100) specific to sex, age [20 to 30, 31 to 50, 51 to 70, and 70 years per Dietary Reference Intake (DRI) categories], and weight status (BMI 25 kg/m 2 vs. BMI 25 kg/m 2 ). Energy requirements were predicted using the most recent DRI equations from the Institute of Medicine (7). These equations were developed from a meta-analysis of methodologically sound studies using doubly labeled water as the criterion measure of TEE (pooled n 767 adults) (7). In the absence of actual, measured TEE, these prediction equations served as the best proxy. Predictors of TEE in these equations included age, weight, height, and PAL (sedentary, low active, moderately active, highly active). Because of the lack of an objective, validated physical activity measure in the CSFII 1994 to 1996, we assigned all subjects to the low active category of PAL (i.e., between 1.4 and 1.59). This assignment was made based on national data in which over 60% of U.S. adults are considered inactive or not regularly active (i.e., 20 minutes of moderate exercise 3 d/wk) (27). A PAL of 1.6 represents daily TEE equivalent to an individual of 70 kg walking 4.4 miles at 2 to 4 mph (7). Note that PAL is minimally dependent on weight status because both TEE and BMR increase in overweight indi OBESITY RESEARCH Vol. 13 No. 7 July 2005

3 Table 1. 1 SD cut-offs of rei as a percentage of per to determine plausible reporting in CSFII Stratum n CV rei CV per CV mtee 1 SD cut-off BMI 25 kg/m 2 Men 20 to 30 years % 31 to 50 years % 51 to 70 years % 70 years % Women 20 to 30 years % 31 to 50 years % 51 to 70 years % 70 years % BMI 25 kg/m 2 Men 20 to 30 years % 31 to 50 years % 51 to 70 years % 70 years % Women 20 to 30 years % 31 to 50 years % 51 to 70 years % 70 years % Weight, sex, and age strata for the cut-offs were based on age categories in the DRIs. Because of the similarity of the 1 SD cut-offs across groups, an average of 22% was used for all subsequent analyses. Plausibility range, or acceptable rei as a percentage of per, was, in this case, 78% to 122%. Cut-offs of rei at 33% and 44% of per were used to define the 1.5 SD and 2 SD samples and resulted in plausibility ranges of 67% and 133% and 56% and 144%, respectively. SD, standard deviation. viduals (7,8) and that PAL is used only to select the equation for per rather than as a reference for the development of any cut-offs of reis. Different equations for per based on sex and weight class (overweight vs. normal weight) were available and used. Recall that in weight-stable conditions and when rei is representative of usual intake, rei should equal per (i.e., rei 100% of per), because energy balance is a function of energy intake being equal to energy expenditure (28). In addition, based on the concept of error propagation (17,18,25), we were able to account for intraindividual variation in energy intake reporting (CV rei ) over the number of days (d) of intake, the error in the equations for per (CV per, which encompasses the errors of the parameters in those equations, including PAL), and measurement error and day-to-day biological variation in TEE (CV mtee )in estimating what the cut-offs should be for rei as a percentage of per (%rei/per). Values for each of these and the 1 SD cut-offs by the different sex-age-bmi strata are shown in Table 1. In the case of CSFII 1994 to 1996, there were 2 days of rei, with CV rei ranging from 20% to 28% (Table 1). The error in per by the Institute of Medicine s recently published DRI equations was calculated by dividing the SD of the residuals in those equations by the mean TEE within each sex-age-bmi stratum (CV per ranged from 8% to 15%) (7). The measurement error and day-to-day biological variation in TEE measured by doubly labeled water (used to develop DRI prediction equations) has been estimated to have a CV mtee of 8.2%, on average (29). Therefore, the 1 SD cut-offs for %rei/per were calculated as follows: 1SD CV 2 rei/d CV 2 per CV 2 mtee Using the 1 SD cut-offs, a report was excluded if %rei/per was outside the 1 SD range. Because the cutoffs were found to be similar across strata (20% to 24%, Table 1), an average of 22% was used for subsequent analyses. Hence, in this case, any given report would be excluded if the rei was 78% or 122% of the per. OBESITY RESEARCH Vol. 13 No. 7 July

4 From the 1 SD cut-off range, we calculated the 1.5 SD and 2 SD cut-offs as 33% and 44% for %rei/per, respectively. Using the 1.5 SD cut-off, a report would be excluded if the rei was 67% or 133% of the per. Using the 2 SD cut-off, a report would be excluded if the rei was 56% or 144% of the per. Note that the skewed distribution of rei in the population provided the basis for estimating the variance of rei on the log scale as approximately its squared coefficient of variation on the original scale. However, in setting the symmetrical interval around %rei/per on the original scale, we chose not to exponentiate the calculated values because, under the null hypothesis of no reporting bias, the SDs on both the log and original scales are approximately equal. In addition, exponentiation in this case would tend to slightly narrow the cut-off for underreporters and slightly widen the cut-off for overreporters. Because underreporting is a much more severe problem in the population than overreporting, we tried to maximize the sample size by not exponentiating the cut-offs. The same reasoning applied to the variance of per; however, the distribution of per in the total sample was much less skewed than rei. Our approach also simplifies the application of our method for other future studies. Recall that rei/per outside these cut-off ranges may include both under/overreporting and under/overeating (i.e., rei outside of representative range of long-term energy intake needed to maintain body weight). However, we could not separate individuals having unusual energy intake from true implausible reporters using these data. Thus, for the purpose of this paper, the term underreporting will be used to describe %rei/per below the minimum cut-off and the term overreporting will be used to describe %rei/per above the maximum cut-off. Data Analysis The 1 SD sample consisted of 41% of the total reports (n 2685 adults: 1512 men and 1173 women), the 1.5 SD sample consisted of 62% of the total reports (n 4039 adults: 2215 men and 1824 women), and the 2 SD sample consisted of 79% of the total reports (n 5142 adults: 2763 men and 2379 women). Two demonstrations of the effect of our method were undertaken in the total vs. each of the samples resulting from the three cut-off levels. First, we examined %rei/per by sex, age, and weight status. It was expected that mean %rei/per would approach 100% in a plausible sample (increasingly so with a more stringent cut-off) but be much less than 100% in the total sample. Second, using regression analysis, we compared the relationship between rei and weight using the CSFII data with the relationship between TEE (measured by doubly labeled water) and weight using the DRI data (7). This was chosen because rei and TEE are direct functions of weight. During weight stability and when rei is valid, the association of rei with weight should equal that of TEE with weight. The comparison of these two relationships was performed in the total, 2 SD, 1.5 SD, and 1 SD samples. We expected that the exclusion of a sufficiently high proportion of implausible reports would yield a biological relationship between rei and weight similar to what would be physiologically expected. If the sample was biologically plausible, the regression slope of rei on weight in CSFII should not be significantly different from the slope of TEE on weight in the DRI. Note that these demonstrations were not employed to validate or prove our method, because validation should be done in an independent sample; rather, they were used to confirm previously known and physiologically expected associations (i.e., between rei and weight) in the reduced samples. Statistical correlations were taken to reflect natural biological correlations. To illustrate the application of the method and to determine the influence of implausible rei on relationships between dietary variables and BMI, we conducted regression analyses in the total, 2 SD, 1.5 SD, and 1 SD samples to examine the differing associations of daily rei (megajoules), total daily eating frequency and frequency of snacks, average portion consumed (grams) and energy content (megajoules) of meals and snacks, energy density (kilojoules per gram) of meals and snacks, percent energy from fat, carbohydrate, and protein, and percentage energy from food away from home (FAFH; any food obtained outside of home, even if consumed at home) with BMI. Each dietary variable was examined in a separate model. Two-day averages of dietary information were used for all analyses. BMI was calculated using self-reported weight and height. In descriptive statistics, overweight was defined as 25 kg/m 2 BMI 30 kg/m 2 and obesity as BMI 30 kg/m 2. Age (continuous), sex, ethnicity (white vs. nonwhite), household income expressed as percentage above poverty level (0% to 130%, 131% to 350%, and 350% of poverty threshold), education (some college or not), urbanicity (urban, suburban, rural area), geographic region (Northeast, Midwest, South, West), current smoking (yes/no), self-report of at least one major illness (yes/no for diabetes, hypertension, heart disease, cancer, hypercholesterolemia, and/or stroke), and television viewing hours per day (as a proxy of inactivity) were covaried. Moreover, using the 1 SD sample (the minimum recommended cut-off based on propagation of error variances) as a reference, we examined characteristics associated with underreporting (rei/per 78%), plausible reporting (78% rei/per 122%), and overreporting (rei/per 122%). ANOVAs were performed to study potential differences across the three categories of rei in age, sex, ethnicity, household poverty level, education, urbanicity, geographic region, current smoking, presence of major illness, television viewing hours per day, and BMI. Categorical independent variables were dummy-coded OBESITY RESEARCH Vol. 13 No. 7 July 2005

5 Finally, based on the methodological test results from the 2 SD, 1.5 SD, and 1 SD cut-offs, we performed post hoc analyses to identify a more precise cut-off between 1 and 2 SD that would maintain the biological plausibility while maximizing the sample size. We performed regression analyses using this new cut-off to compare the biological relationships between eating variables and BMI to those observed in the 2, 1.5, and 1 SD samples. SAS v.8.2 (Cary, NC) was used to calculate coefficients of variation and determine cut-offs for implausible rei, to calculate means of the variation of rei and rei as a percentage of per, and to perform regression analyses of rei on per, rei on weight, and TEE on weight, because these were all sample-specific statistics. All linear regressions and ANOVAs conducted to examine sample characteristics, associations of dietary variables with BMI, and characteristics associated with reporting plausibility status were performed using SUDAAN v.8 (Research Triangle Institute, Research Triangle Park, NC) because these statistics were to be generalizable to the population. SUDAAN was used to incorporate the CSFII sample design (stratified, multistage area probability sample) for variance estimation. Failure to account for sample design is known to underestimate SEs of parameters, therefore increasing the risk of rejecting true null hypotheses. All significance testing had an set at Results Demographic and dietary characteristics of the subjects are presented in the total sample vs. the 1 SD sample in Table 2. The 1 SD sample results from the most stringent cut-off; therefore, it is presented here to contrast with the total sample. By observation, mean differences in sociodemographic characteristics were not substantial between the total sample and the 1 SD sample. Mean rei was higher in the 1 SD sample compared with that in the total sample. Meal and snack frequency, energy, and portion consumed were higher in the 1 SD sample, suggesting underreporting of entire meals and/or snacks, as well as amounts per meal and snack, in the total sample. Energy densities were also slightly higher in the 1 SD sample, because of higher percentage energy from fat and lower percentage energy from protein and carbohydrates. Values for the 2 SDand 1.5 SD samples were between those of the total sample and the 1 SD sample (data not shown). Statistical and Biological Effects of the Method Reported EI as a percentage of per is shown by sex, age, and weight status in the total, 2 SD, 1.5 SD, and 1 SD samples (Figure 1). Low levels of %rei/per were evident in the total sample, particularly among women, and older and overweight subgroups. Among individuals in the total sample, %rei/per ranged from 9% to 312% (interindividual SD 29%). With progressively smaller cut-off ranges (i.e., moving from 2 SDto 1.5 SD to 1 SD), %rei/per progressively approached 100% in all subgroups. The interindividual SDs of %rei/per were 20% in the 2 SD sample, 18% in the 1.5 SD sample, and 12% in the 1 SD sample. To examine the biological plausibility of the samples obtained using the different cut-offs, we compared the regression slope of rei on body weight in the CSFII data with that of TEE on body weight using the DRI data (7) (R , p 0.001). As shown in Figure 2, the regression slope of rei on weight using the CSFII data was substantially smaller than the slope of TEE on weight using the DRI data (slopes 0.05 vs. 0.09, p 0.001), indicating that, in the total sample, the biological relationship between rei and weight did not approximate the theoretical relationship. In both the 2 SDand 1.5 SD samples, while the slopes of rei on weight in the CSFII more closely approximated the slope of TEE on weight than in the total sample, they remained significantly different from the theoretical relationship (slope 0.07 vs. 0.09, p in the 2 SD sample; slope 0.08 vs. 0.09, p 0.03 in the 1.5 SD sample). On the other hand, in the 1 SD sample, the slope of rei on weight using the CSFII data was not significantly different from the slope of TEE on weight using the DRI data (slope 0.09, p 0.37). Thus, a cut-off 1 but 1.5 SD seemed to be necessary to yield a biologically plausible sample, based on our criteria. Application of the Method: Associations of Eating Patterns with BMI Table 3 shows the regression results of eating patterns in relation to BMI in the total sample and the results generated using the different cut-off levels. Total Sample In the total sample, daily rei, portion consumed and energy content per meal and snack, and percentage energy from fat, protein, and FAFH were positively associated with BMI. Percent energy from carbohydrate was negatively associated with BMI. Total eating frequency per day, snack frequency per day, and energy density of meals and snacks were not significantly associated with BMI. Samples: 2 SD, 1.5 SD, and 1 SD In each of these samples, daily rei, portion consumed and energy content per meal and snack, and percentage energy from fat and FAFH were positively and strongly associated with BMI. Percent energy from carbohydrate was negatively associated with BMI. As shown in Table 3, as the cut-off range narrowed, the effect sizes of these relationships became much greater, and for many variables, increasingly stronger associations were found as the cut-off range narrowed. Contrary to results in the total sample, total OBESITY RESEARCH Vol. 13 No. 7 July

6 Table 2. Characteristics of subjects in the total vs. 1 SD samples Total sample (n 6499) 1 SD sample (n 2685) Characteristic Mean or percentage SE Mean or percentage SE Demographics Age Percent female Percent nonwhite Percent 130% household poverty level Television viewing (h/d) Body mass BMI (kg/m 2 ) Percent overweight (25 kg/m 2 BMI 30 kg/m 2 ) Percent obese (BMI 30 kg/m 2 ) Diet rei (MJ/d) per (MJ/d) rei/per (%)* Total eating frequency/d Snack frequency/d Meal portion size (g) Snack portion size (g) rei per meal (MJ) rei per snack (MJ) Meal energy density (kj/g) Snack energy density (kj/g) Percent energy from fat Percent energy from carbohydrate Percent energy from protein Percent energy from FAFH SD cut-off provides the most severe exclusion of implausible reports and is shown here to provide maximal contrast with the total sample. SE values are weighted for sampling design. * Percentage and SE are sample-specific, not weighted for sampling probability. eating frequency per day and snack frequency per day were significantly and positively related to BMI in the 2 SD, 1.5 SD, and 1 SD samples. Note that the regression coefficients for total eating frequency per day and snack frequency per day were negative and positive, respectively, although not statistically significant, in the total sample, but both became positive and significant after the exclusion of reports outside the cut-off ranges. Furthermore, percent energy from protein became more negatively (although not significantly) related to BMI with the use of increasingly narrower cut-off ranges. Significant improvements in the effect sizes of dietary variables were seen in the 1.5 SD and 1 SD samples compared with the 2 SD sample. Compared with the 1.5 SD sample, the 1 SD sample yielded even stronger associations for variables directly related to energy intake. Moreover, meal energy density was positively related to BMI in the 1 SD sample but not in the 1.5 SD or the 2 SD sample. Characteristics Associated with Reporting Status Because of the substantial reporting bias in the total sample, we performed ANOVAs using the 1 SD sample 1210 OBESITY RESEARCH Vol. 13 No. 7 July 2005

7 have been diagnosed with a major illness and had lower BMI compared with the plausible reporters. Figure 1: rei as a percentage of per by (A) sex, (B) age, and (C) weight status in the total vs. 2, 1.5, 1.4, and 1 SD samples. rei as a percentage of per approaches 100% in all strata with progressively narrower cut-off ranges. as a plausibility reference (i.e., minimum recommended cut-off and maximum proportion of plausible reis) to examine potential differences in an array of socio-demographic characteristics among underreporters (rei/per 78%, 51% of total sample), plausible reporters (78% rei/per 122%, 41% of total sample), and overreporters (rei/per 122%, 8% of total sample), as shown in Table 4. Comparing underreporters with plausible reporters, underreporters were more likely to be older, female, nonwhite, poor, to have attained only high school education or less, to be from the Midwest or southern United States, to have been diagnosed with a major illness, and to be obese. Comparing overreporters with plausible reporters, overreporters were younger, more likely to be male, non-white, and current smokers. Also, overreporters were less likely to Post Hoc Analyses of Cut-off Values > 1 and < 1.5 SD Based on the observation that the 1 SD cut-off yielded the best biological plausibility compared with the wider cut-offs of 1.5 and 2 SD and that the 1 SD necessitated the exclusion of a large number of subjects, we conducted post hoc analyses of different cut-offs, in increments of 0.05 SD, between 1 and 1.45 SD. These analyses were aimed at finding the largest sample with biological plausibility to minimize the exclusion of valuable data. Results showed that at 1.4 SD (n 3755, 58% of total sample; reports excluded if rei was 69.2% or 130.8% of per; mean SD of rei as a percentage of per 92 16%, Figure 1), the slope between rei and weight was not significantly different from the theoretical relationship between TEE and weight (rei Weight, R , Figure 2). These findings suggest that, while the 1 SD remains the minimum recommended cut-off that provides the highest proportion of potentially plausible reports, in this particular dataset, a cut-off of up to 1.4 SD may render adequate biological plausibility while significantly increasing the usable sample size for the study of diet and BMI relationships. In addition, results of 1.4 SD showed the same set of associations between dietary variables and BMI as found in the 1 SD sample, including the observation of a positive relationship between meal energy density and BMI, which was not found in either the 1.5 or 2 SD sample. Parameter coefficients ranged between those in the 1 and 1.5 SD samples (Table 3). Comparisons of group characteristics by reporting status using the 1.4 SD sample as the plausible reference yielded similar results as using the 1 SD sample. However, because of the larger sample size in the 1.4 SD sample relative to the 1 SD sample, underreporters were less likely to be from the western United States (p 0.02) and more likely to be overweight (p 0.002) and obese than plausible reporters. Discussion Given the widespread problem of implausible dietary energy reports and their impact on the study of relationships between diet and BMI, methods to improve the quality of existing dietary data sets are urgently needed. In this paper, we presented a simple method to identify implausible reports, where rei is directly compared with per, and per is derived from easily measured variables of age, weight, height, and sex. Using this method, we analyzed dietary data from food-secure, nonpregnant, nonlactating adult participants 20 to 90 years of age in the CSFII dataset. In the OBESITY RESEARCH Vol. 13 No. 7 July

8 Figure 2: Regressions of rei on weight in the CSFII data and doubly labeled water measured TEE (mtee) on weight in the DRI data (7) in the (A) total sample vs. the (B) 2 SD, (C) 1.5 SD, (D) 1.4 SD, and (E) 1 SD samples. During weight stability, rei mtee; therefore, the relationship between rei and weight should be the same as the physiologically expected relationship between mtee and weight. Increasing approximation of the two slopes in the reduced samples, but not the total sample, indicates biological validation for the method used to exclude implausible reports. Only in the 1 to 1.4 SD samples, however, are the two slopes not significantly different from each other (p 0.05), suggesting that narrower cut-offs may be needed to provide plausible samples to study biological relationships between diet and obesity.

9 Table 3. Regression of BMI on rei and select eating patterns in the total, 2, 1.5, 1.4, and 1 SD samples Variable Total sample (n 6499) 2 SD sample (n 5142) 1.5 SD sample (n 4039) 1.4 SD sample (n 3755) 1 SD sample (n 2685) rei/d (MJ) * Total eating frequency/d * Snack frequency/d Meal portion size (g) Snack portion size (g) rei per meal (MJ) rei per snack (MJ) * Meal energy density (kj/g) * Snack energy density (kj/g) Percent energy from fat Percent energy from carbohydrate Percent energy from protein Percent energy from FAFH Values are ß SE. Each eating pattern variable was tested in separate models. All models adjusted for sex, age, ethnicity (white vs. nonwhite), percent above household poverty threshold, education (high school or less vs. college and above), urbanicity (urban, suburban, rural), geographic region (Northeast, Midwest, South, West), current smoking (yes/no), diagnosis of major illness (yes/no), and daily television viewing hours. * p 0.05, p 0.01, p OBESITY RESEARCH Vol. 13 No. 7 July

10 Table 4. Comparison of characteristics between implausible reporters of energy intake and plausible reporters in the 1 SD sample Reporting status Variable Group A underreporters (n 3290) Group B plausible reporters (n 2685) Group C overreporters (n 524) p < (A vs. B) p < (C vs. B) Age (years) Percent female Percent nonwhite Percent poverty NS* Percent high school or less NS Percent urban NS NS Region (%) Northeast NS NS Midwest NS South NS West NS NS Percent current smoker NS Percent diagnosed with illness Television viewing (h/d) NS NS BMI (kg/m 2 ) Percent overweight (25 kg/m 2 BMI 30 kg/m 2 ) NS NS Percent obese (BMI 30 kg/m 2 ) NS Values are weighted means SE unless indicated as proportion. * Not significant because of large variation among overreporters. Percent poverty, 0% to 130% of household poverty threshold; NS, not significant; underreporters, rei/per 78%; plausible reporters, 78% rei/per 122% (i.e., 1 SD sample); overreporters, rei/per 122% OBESITY RESEARCH Vol. 13 No. 7 July 2005

11 samples identified as plausible after testing different cut-off levels, mean rei closely approximated mean per (%rei/ per approached 100%) across sex, age, and weight status, and expected relationships between rei and body weight (based on the relationship between TEE and body weight) were found. Of critical importance was our finding that not excluding implausible reports resulted in weak, nonsignificant, or misleading associations between diet and BMI. Considering these results, we hope that this method will become more widely used and that future analysis of other data sets with and without exclusion of implausible reports will considerably forward the understanding of the potential dietary causes of obesity in different populations. We found a high degree of implausible reporting when using the 1 SD cut-off, the most stringent level based on error propagation calculations. Despite retaining only 41% of the total sample after using this cut-off, excluding implausible reports resulted in a dataset of much higher quality with a higher proportion of physiologically plausible reis compared with the total sample. However, to avoid the exclusion of such a large number of subjects, we also conducted post hoc analysis to determine whether a larger cut-off that also offered acceptable plausibility of the retained dataset could be identified. This occurred in the 1.4 SD sample, in which 58% of the total sample was retained. In this sample, mean %rei/per remained high (92% compared with 95% in the 1 SD sample), and the plausibility of slope of the relationship between rei and body weight was maintained. It is important to recognize that the high degree of implausible reporting in the particular dataset we used in this analysis is probably not unique, and we hope that future investigators will consider applying our method to other data sets to identify implausible reports and determine their influence on relationships between a variety of dietary and health parameters. One major disadvantage of our method is in the large number of participants who could potentially be excluded from the data set of analysis. However, in the absence of more precise dietary assessment technology, it is indeed a trade-off that researchers must confront in choosing to perform their analysis in a larger sample with a higher proportion of implausible dietary reports or in a smaller sample with a lower proportion of such reports. In the former case, the risk would be that conclusions could be misleading or spurious, as shown in this paper, whereas in the latter case, the risk would be that conclusions could be less generalizable to the entire population. Compared with other conventional methods for dealing with implausible reports, the 95% confidence intervals recommended by the Goldberg approach may be too wide to screen out the majority of implausible reports. Alternatively, energy intake could be adjusted statistically when studying dietary factors. However, Schatzkin et al. (21), from the Observing Protein and Energy Nutrition Study, have shown that because the attenuation factor for energy intake values is so substantial, even for multiple administrations of food frequency questionnaires or 24-hour recalls, the use of energy adjustment to detect important but moderate dietary effects remains questionable. Using a range of cut-off levels, we showed that implausible reis impacted nearly all of the relationships between dietary factors and BMI that we examined. For example, we found nonsignificant associations of total eating frequency per day (negative) and snack frequency per day (positive) with BMI in the total sample. However, when implausible reports were excluded, both total eating frequency per day and snack frequency per day were positively and significantly related to BMI, leading to very different conclusions about potential dietary factors associated with obesity. Our study corroborates the previous suggestion of Bellisle et al. (30) that a significant relationship between decreased eating frequency or snack consumption and increased body mass may be artifacts of biased dietary recall, in which entire eating occasions may be omitted from dietary reports (31,32). We also found that daily energy intake, portion and energy consumed per meal and snack, and percentage energy from fat and FAFH were positively associated with BMI in all samples, but the magnitude of these relationships became stronger with smaller cut-offs. In addition, a negative but not significant relationship was found between meal energy density and BMI in the total sample. However, a positive and significant relationship was shown once the majority of implausible reports were excluded ( 1 to 1.4 SD samples), a finding that is more probable based on previous experimental studies (33). Certain characteristics were associated with implausible reporting in our study. Our findings support numerous previous studies showing that overweight individuals are more prone to underreporting (1 3,11,31). Obesity is more prevalent among socio-economically disadvantaged and minority individuals and is associated with a wide array of chronic diseases. Our findings also corroborate another study in Norway that showed that a smaller proportion of overreporters were obese (34). Results need to be interpreted with caution, however, because underreporters may include undereaters and overreporters may include overeaters, factors that could not be quantified in these data. We calculated the cut-offs for rei as a percentage of per by sex, age, and weight status based on the strata in the DRI (7). In mixed-age samples, however, average values could be used instead of sex-age-bmi-specific values. For example, an average of 11% (averaged from Table 1) could be used instead for CV per for ease of computation. Also, based on our findings, the suggestion of Black (18) of 23% for CV rei is reasonable in large samples. However, CV rei may OBESITY RESEARCH Vol. 13 No. 7 July

12 differ in small samples or samples with substantially different characteristics. In such cases, sample-specific CV rei should be calculated. A few limitations are present in this study. Our method needs to be validated in independent samples in the future, where dietary analysis is performed and compared in both total and reduced data sets. Because this paper was aimed to show only our method, we also plan in future studies to formally compare the different methods available to deal with implausible reports. While most of the eating pattern variables studied in this paper were related to energy intake, therefore justifying the use of energy intake as our deciding measure for the identification of implausible dietary reports, we acknowledge the need to study other approaches if the variables of interest are not directly related to energy. For instance, Subar et al. (3) have shown that underreporters of protein and energy intake do not always overlap. It is also true that choosing the level of cut-off would pose some difficulty for future researchers because of the need to avoid picking a cut-off that suits the desired conclusion. This is why, by definition of our method, one cannot go below 1 SD, and a multiple of the 1 SD cut-off (if it is desired) should be chosen based on a previously accepted biological association, not an association the researchers wish to achieve. Furthermore, because of constraints within the data set, we did not have a validated and individualized measure of physical activity. Instead, the equations for low-active individuals were used for all subjects to calculate per. While this is adequate for most U.S. adults (7,27), in very active individuals, per would be underestimated, having the effect of overestimating %rei/per, thus tending to retain these individuals in the plausible sample. Because no objective physical activity measure was available, we relied on television viewing hours per day as a proxy of inactivity level in the regression analyses between eating patterns and BMI; however, this may not be the mirror construct of physical activity (35). When valid PAL is available, it can be used to select appropriate equations of per, but because of the lack of precision in many methods for measuring PAL, this may not necessarily be superior to assigning an average PAL to everyone. Finally, weight and height were self-reported. The underestimation of weight could lead to the underestimation of per; the overestimation of height could lead to the overestimation of per. In conclusion, physiologically implausible reporting, particularly underreporting, of energy intake is widespread; therefore, it is necessary to develop methods for enhancing the quality of existing dietary data sets. There is no simple way to deal with reporting error, but our method offers an alternative that may be simpler and more individualized than existing methods. Of key importance, we showed that different conclusions might be drawn with and without the inclusion of implausible reports. We expect that the application of our method to different data sets may help forward knowledge about relationships between dietary factors and obesity and may even be useful to the broader study of relationships between diet and health. Acknowledgments This study was supported, in part, by USDA/ERS/ FANRP Grant 43-3AEM Contents of this publication do not necessarily reflect the views or policies of the U.S. Department of Agriculture. References 1. Schoeller DA. Limitations in the assessment of dietary energy intake by self-report. Metabolism. 1995;44: Schoeller DA, Bandini LG, Dietz WH. Inaccuracies in selfreported intake identified by comparison with the doubly labelled water method. Can J Physiol Pharmacol. 1990;68: Subar AF, Kipnis V, Troiano RP, et al. Using intake biomarkers to evaluate the extent of dietary misreporting in a large sample of adults: the OPEN study. Am J Epidemiol. 2003;158: Livingstone MB, Black AE. Markers of the validity of reported energy intake. J Nutr 2003;133(Suppl 3):895S 920S. 5. Ohlson MA, Harper LJ. Longitudinal studies of food intake and weight of women from ages 18 to 56 years. J Am Diet Assoc. 1976;69: Romieu I, Willett WC, Stampfer MJ, et al. Energy intake and other determinants of relative weight. Am J Clin Nutr. 1988;47: Institute of Medicine. Dietary Reference Intakes for Energy, Carbohydrate, Fiber, Fat, Fatty Acids, Cholesterol, Protein, and Amino Acids, Part I. Washington, DC: National Academy Sciences; Das SK, Saltzman E, McCrory MA, et al. Energy expenditure is very high in extremely obese women. J Nutr. 2004; 134: Goris AH, Westerterp-Plantenga MS, Westerterp KR. Undereating and underrecording of habitual food intake in obese men: selective underreporting of fat intake. Am J Clin Nutr. 2000;71: Samaras K, Kelly PJ, Campbell LV. Dietary underreporting is prevalent in middle-aged British women and is not related to adiposity (percentage body fat). Int J Obes Relat Metab Disord. 1999;23: Voss S, Kroke A, Klipstein-Grobusch K, Boeing H. Is macronutrient composition of dietary intake data affected by underreporting? Results from the EPIC-Potsdam Study. European Prospective Investigation into Cancer and Nutrition. Eur J Clin Nutr. 1998;52: Pomerleau J, Ostbye T, Bright-See E. Potential underreporting of energy intake in the Ontario Health Survey and its relationship with nutrient and food intakes. Eur J Epidemiol. 1999;15: Lafay L, Basdevant A, Charles MA, et al. Determinants and nature of dietary underreporting in a free-living population: the Fleurbaix Laventie Ville Sante (FLVS) Study. Int J Obes Relat Metab Disord. 1997;21: OBESITY RESEARCH Vol. 13 No. 7 July 2005

13 14. Briefel RR, Sempos CT, McDowell MA, Chien S, Alaimo K. Dietary methods research in the third National Health and Nutrition Examination Survey: underreporting of energy intake. Am J Clin Nutr 1997;65:1203S 9S. 15. Bingham SA, Cassidy A, Cole TJ, et al. Validation of weighed records and other methods of dietary assessment using the 24 h urine nitrogen technique and other biological markers. Br J Nutr. 1995;73: Kipnis V, Subar AF, Midthune D, et al. Structure of dietary measurement error: results of the OPEN biomarker study. Am J Epidemiol. 2003;158: McCrory MA, Hajduk CL, Roberts SB. Procedures for screening out inaccurate reports of dietary energy intake. Public Health Nutr. 2002;5: Black AE. Critical evaluation of energy intake using the Goldberg cut-off for energy intake:basal metabolic rate. A practical guide to its calculation, use and limitations. Int J Obes Relat Metab Disord. 2000;24: Rosell MS, Hellenius ML, de Faire UH, Johansson GK. Associations between diet and the metabolic syndrome vary with the validity of dietary intake data. Am J Clin Nutr. 2003;78: Macdiarmid JI, Vail A, Cade JE, Blundell JE. The sugarfat relationship revisited: differences in consumption between men and women of varying BMI. Int J Obes Relat Metab Disord. 1998;22: Schatzkin A, Kipnis V, Carroll RJ, et al. A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarkerbased Observing Protein and Energy Nutrition (OPEN) study. Int J Epidemiol. 2003;32: Huang TT, Howarth NC, Lin BH, Roberts SB, McCrory MA. Energy intake and meal portions: associations with BMI percentile in U.S. children. Obes Res. 2004;12: Conway JM, Ingwersen LA, Moshfegh AJ. Accuracy of dietary recall using the USDA five-step multiple-pass method in men: an observational validation study. J Amer Diet Assoc. 2004;104: Conway JM, Ingwersen LA, Vinyard BT, Moshfegh AJ. Effectiveness of the US Department of Agriculture 5-step multiple-pass method in assessing food intake in obese and nonobese women. Am J Clin Nutr. 2003;77: Goldberg GR, Black AE, Jebb SA, et al. Critical evaluation of energy intake data using fundamental principles of energy physiology: 1. Derivation of cut-off limits to identify underrecording. Eur J Clin Nutr. 1991;45: USDA ARS Food Surveys Research Group. Data and Documentation for the , and 1998 Continuing Surveys of Food Intake by Individuals (CSFII)-Diet and Health Knowledge Survey. Washington, DC: National Technical Information Service; U.S. Department of Health and Human Services. Physical Activity and Health: A Report of the Surgeon General. Washington, DC: U.S. Department of Health and Human Services; Hervey GR. Regulation of energy balance. Nature. 1969;222: Black AE, Cole TJ. Within- and between-subject variation in energy expenditure measured by the doubly-labelled water technique: implications for validating reported dietary energy intake. Eur J Clin Nutr. 2000;54: Bellisle F, McDevitt R, Prentice AM. Meal frequency and energy balance. Br J Nutr 1997;77(Suppl 1):S Ma Y, Bertone ER, Stanek EJ III, et al. Association between eating patterns and obesity in a free-living US adult population. Am J Epidemiol 2003;158: Kant AK, Schatzkin A, Graubard BI, Ballard-Barbash R. Frequency of eating occasions and weight change in the NHANES I Epidemiologic Follow-up Study. Int J Obes Relat Metab Disord. 1995;19: Yao M, Roberts SB. Dietary energy density and weight regulation. Nutr Rev. 2001;59: Johansson L, Solvoll K, Bjorneboe GE, Drevon CA. Underand overreporting of energy intake related to weight status and lifestyle in a nationwide sample. Am J Clin Nutr. 1998;68: Gordon-Larsen P, McMurray RG, Popkin BM. Adolescent physical activity and inactivity vary by ethnicity: the National Longitudinal Study of Adolescent Health. J Pediatr. 1999; 135: OBESITY RESEARCH Vol. 13 No. 7 July