Published in: European Journal of Clinical Nutrition

Transcription

1 An evaluation of the sensitivity and specificity of energy e xpenditure measured by heart rate and the Goldberg cut-off for energy intake: basal metabolic rate for identifying mis-reporting of energy intake by adults and children: a retrospective analysis. Livingstone, M. B. E., Robson, P. J., Black, A. E., Coward, W. A., Wallace, J. M. W., McKinley, M.,... McKenna, P. G. (2003). An evaluation of the sensitivity and specificity of energy e xpenditure measured by heart rate and the Goldberg cut-off for energy intake: basal metabolic rate for identifying mis-reporting of energy intake by adults and children: a retrospective analysis., 57, Published in: Queen's University Belfast - Research Portal: Link to publication record in Queen's University Belfast Research Portal General rights Copyright for the publications made accessible via the Queen's University Belfast Research Portal is retained by the author(s) and / or other copyright owners and it is a condition of accessing these publications that users recognise and abide by the legal requirements associated with these rights. Take down policy The Research Portal is Queen's institutional repository that provides access to Queen's research output. Every effort has been made to ensure that content in the Research Portal does not infringe any person's rights, or applicable UK laws. If you discover content in the Research Portal that you believe breaches copyright or violates any law, please contact openaccess@qub.ac.uk. Download date:15. Feb. 2017

2 ORIGINAL COMMUNICATION (2003) 57, ß 2003 Nature Publishing Group All rights reserved /03 $ An evaluation of the sensitivity and specificity of energy expenditure measured by heart rate and the Goldberg cut-off for energy intake: basal metabolic rate for identifying mis-reporting of energy intake by adults and children: a retrospective analysis MBE Livingstone 1 *, PJ Robson 1, AE Black 2, WA Coward 3, JMW Wallace 1, MC McKinley 1, JJ Strain 1 and PG McKenna 1 1 Northern Ireland Centre for Diet and Health, University of Ulster, Coleraine, Co Londonderry, Northern Ireland, UK; 2 9 Birch Close, Cambridge, UK; and 3 MRC Human Nutrition Research, Elsie Widdowson Laboratory, Cambridge, UK Objective: To identify adults and children as under- (UR), acceptable (AR), or over-reporters (OR) of energy intake (EI) using energy expenditure measured by doubly labelled water (DLW) (EE DLW ), and to use this as a reference to determine the sensitivity and specificity of (i) EE measured by heart rate (EE HR ), and (ii) the Goldberg cut-off technique for classifying subjects into the same categories. Design: Retrospective analysis of a dataset comprising concurrent measurements of EE DLW,EE HR, basal metabolic rate (BMR), and EI by weighed record (EI WR ) on 14 adults and 36 children. EI by diet history (EI DH ) was also measured in the children only. EI WR :EE DLW provided the reference definition of subjects as UR, AR or OR. Three strategies for classifying mis-reporters based on EE HR and Goldberg cut-offs were then explored. Sensitivity and specificity were calculated respectively as the proportion of UR and non-ur correctly identified. Results: Approximately 80% of all subjects were AR. For EI WR and EI DH respectively, the sensitivity of EE HR was 0.50 and 1.00, and specificity was 0.98 and Although designating subjects as having low, medium or high activity levels (EE HR :BMR meas ) and calculating cut-offs based on appropriate WHO physical activity level PALs did not change sensitivity, specificity dropped to 0.98 (EI WR ) and 0.97 (EI DH ). Cut-offs based on a PAL of 1.55 reduced sensitivity to 0.33 (EI WR ) and 0.00 (EI DH ), but specificity remained unchanged. The sensitivity of all cut-offs based on physical activity level (PALs) for EI WR was 0.50 (adults) and 0.25 (children). Conclusions: If the precision of EE HR was improved, it may be useful for identifying mis-reporters of EI. (2003) 57, doi: =sj.ejcn Keywords: adults; children; validity; heart-rate; energy; intake; dietary surveys *Correspondence: Dr MBE Livingstone, Northern Ireland Centre for Diet and Health, University of Ulster, Coleraine, Co Londonderry, Northern Ireland, UK. mbe.livingstone@ulst.ac.uk Guarantor: Dr MBE Livingstone. Contributors: MBEL was the principal author and was responsible for executing the original adult and child cohorts. PJR participated in the conceptualisation, data analysis, drafting and critical review of the manuscript. AEB formulated the specific hypothesis and critically reviewed the manuscript. WAC was responsible for over-seeing the original doubly labelled water analysis and critically reviewed the manuscript. JMWW, McMcK, JJS and PGMcK participated in conceptualisation and helped with the drafting and critical review of the manuscript. Received 18 January 2002; revised 2 May 2002; accepted 17 June 2002 Introduction The independent validation of energy intake (EI) data by doubly labelled water (DLW) measurements of energy expenditure (EE) has conclusively demonstrated that systematic under-reporting in dietary surveys, due to under-eating and=or under-recording, is pervasive and is evident across the entire range of energy expenditures (Schoeller, 1990; Black et al, 1993; Black, 1997, 1999). However, while DLW measurements of EE have been paramount in highlighting the existence of bias in dietary surveys, the cost of the technique precludes its routine use in the screening of EI data, particularly in large epidemiological surveys. Reported EI can also be evaluated against presumed energy requirements (Goldberg et al, 1991; Black, 2000a).

3 456 This procedure, known as the Goldberg cut-off technique, has demonstrated a widespread tendency to underestimation in large national dietary surveys from several countries as well as in many smaller surveys (Heywood et al, 1993; Fogelholm et al, 1996; Briefel et al, 1997; Lafay et al, 1997; Price et al, 1997; Pryer et al, 1997; Rothenberg et al, 1997; Voss et al, 1998). However, this technique was devised to evaluate the overall bias towards under-reporting at the group level. Although its use has been extended to identify under-reporting at the individual level, the cut-off is limited by low sensitivity, as it only identifies about 50% of underreporters (Black, 2000b). Furthermore, it can make no distinction between varying degrees of mis-reporting. If the complex nature of biased reporting is to be unravelled, and its effects minimized in the analysis of dietary data, it is vital that individuals who have provided data of poor validity are identified. To improve sensitivity in the detection of mis-reporting in dietary surveys, some attempt should be made to establish values for EE, against which reported EI can be evaluated. To date, five studies have compared reported EI directly with values for EE derived from a variety of techniques other than EE DLW (Johnson et al, 1994; de Vries et al, 1994; Körtzinger et al, 1997; Charlton & Lambert, 1999; Goris & Westerterp, 1999). Unfortunately, the sources of error and bias of these techniques, when used as validation tools for screening EI data, are unknown. If, as appears likely, researchers increasingly turn to these more feasible and cost-effective ways of measuring EE in dietary surveys, it is vital that their sensitivity and specificity for detecting dietary mis-reporting is established at the outset. The present study examines data from two studies in adults (Livingstone et al, 1990a, b) and children (Livingstone et al, 1992a, b) in which EE was measured concurrently by both DLW and HR monitoring. In the same subjects, EI was measured by 7 day weighed record (WR; adults and children) and diet history (DH; children only). The present analysis uses the direct comparison of EI and DLW measurements of EE to classify individuals as acceptable, under- or overreporters (the reference definition). The sensitivity and specificity of two different strategies for placing individuals into the same categories were then explored: (i) EE measured by HR monitoring; and (ii) Goldberg cut-offs based on PALs for differing intensities of physical activity. Subject and methods The dataset The dataset comprised individual data from four previously published studies of 14 free-living adults (Livingstone et al, 1990a, b) and 36 children aged 7, 9, 12, and 15 y (Livingstone et al, 1992a, b). The adult subjects were a sub-sample from a large randomly selected community dietary survey, and the children were recruited through schools that were selected to represent the range of socio-economic status. The age distribution and physical characteristics of the subjects are presented in Table 1. These studies were approved by the Queen s University Ethical Committee (Livingstone et al, 1990a, b) and by the Ethical Committee of the University of Ulster (Livingstone et al, 1992a, b). Energy intake Weighed dietary record. All subjects kept a WR for 7 consecutive days of all individual items of food and fluid consumed, together with the weights of leftovers. Parents of the 7- and 9-y-old children reported the food intakes of their Table 1 Description of the subjects Age (y) Height (m) Weight (kg) BMI (Kg=m 2 ) Age group n Mean (SD) Mean (SD) Mean (SD) Mean (SD) 7y Total (0.3) 1.23 (0.07) 24.5 (5.0) 16.0 (1.7) Males (0.3) 1.26 (0.07) 25.4 (6.6) 15.8 (2.2) Females (0.3) 1.20 (0.05) 23.5 (2.5) 16.3 (1.2) 9y Total (0.2) 1.35 (0.07) 31.6 (7.2) 17.2 (2.4) Males (0.2) 1.35 (0.09) 30.2 (9.4) 16.1 (2.5) Females (0.5) 1.34 (0.05) 33.4 (3.8) 18.4 (1.0) 12 y Total (0.3) 1.55 (0.08) 44.4 (5.8) 18.5 (1.6) Males (0.3) 1.52 (0.09) 43.8 (7.3) 18.8 (2.0) Females (0.4) 1.58 (0.08) 45.1 (4.7) 18.2 (1.1) 15 y Total (0.3) 1.65 (0.12) 53.0 (9.6) 19.8 (4.4) Males (0.4) 1.72 (0.09) 50.7 (6.4) 17.2 (2.3) Females (0.4) 1.57 (0.11) 55.4 (13.2) 22.4 (4.7) Adults Total ( (0.10) 73.5 (16.6) 24.6 (3.8) Males (6.4) 1.78 (0.07) 81.9 (13.8) 26.0 (3.8) Females (6.9) 1.62 (0.06) 58.3 (8.4) 22.2 (2.6) BMI body mass index

4 children while subjects aged 12 and 15 y took a greater responsibility for measurement of their own food intake in co-operation with their parents. Subjects were issued with dietary scales, a log book for recording foods and fluids eaten at home, or prepared at home for eating elsewhere, a pocket notebook for recording foods and fluids obtained and eaten away from home, and written instructions which included examples of completed forms. On the day before recording started, each subject was given a detailed explanation and demonstration of the cumulative weighing technique. Subjects were instructed to record brand names of foods and to provide a complete description of methods of food preparation and cooking and recipes for composite dishes. The records were used to adjust for losses during cooking. For foods and fluids eaten away from home, a description of the food, place of purchase and price were requested. Subjects were visited on a least four occasions during the weighing period and on the day after completion of weighing to monitor progress and to check log books for completeness and accuracy. The intake of metabolizable energy was calculated from food tables using a computerized data base. Diet history. The usual food intake of the children only was also assessed by DH interview conducted with the child and=or parent. Potential carryover memory effects between reported EI by diet history (EI DH ) and weighed record (EI WR ) were minimized by conducting the DH interviews either 2 4 weeks before or after the completion of the WR. To eliminate inter-interviewer bias and effects, all interviews were conducted by one investigator. Information was obtained on the usual meal and snack pattern of the subject, including place of consumption, usual foods consumed during the week and on weekends, and detailed descriptions of these foods including methods of preparation and portion size. Amounts of foods and fluids consumed were estimated by means of photographs of known portion weights of foods supplemented with the use of common household cups, glasses and dishes. School meal intakes were assessed by obtaining a typical week s menu with weights from the school meals staff and asking the child to indicate how much of these meals they usually ate. Self-service school cafeteria meals were assessed by asking the child which foods he or she usually selected and how much they normally ate. Typical portion sizes were subsequently obtained from the supervisory staff. Energy intakes were calculated as described above. Details of each dietary methodology are fully documented in the source references (Livingstone et al, 1990b, 1992b). Total energy expenditure Doubly labelled water method. Total EE by the DLW method was measured over 15 days in the adults and over days in the children (Livingstone et al, 1990a, b, 1992a, b). The DLW technique, together with methods of calculation, validation studies and estimates of potential errors, has been fully described elsewhere (International Dietary Energy Consultative Group, 1990; Speakman, 1997). After collection of a predose urine sample, each subject was dosed orally with 0.05 g 2 H 2 O and 0.15 g (adults) or g (children) H 2 18 O=kg body weight. Aliquot samples of urine were collected 5 h postdose and each day at a known time for 15 days in the adults and days in the children. Isotope analysis of postdose urine samples was made in duplicate, corrected for background amounts and carbon dioxide production rate calculated by using the multipoint method (Coward, 1988). The mean respiratory quotient required to estimate EE from carbon dioxide production was calculated from food quotients measured by WR in the adults and DH in the children (Black et al, 1986). Proporation of error analysis yielded an average standard error for the estimates of EE DLW of 6 2.6% (adults) and % (children). Heart-rate monitoring. In all subjects, total EE by the Flex- HR method was measured over two to four separate days concurrently with the assessment of EE DLW. The procedures are described in detail in the source references (Livingstone et al, 1990a, 1992a). Briefly, the estimation of EE HR involved measurements of basal metabolic rate (BMR), resting metabolic rate (RMR), individually determined heart rate (HR) oxygen consumption (VO 2 ) regression lines, and minute-byminute daytime HR recordings in free-living conditions. To determine the individual HR VO 2 regression line, five calibration points were obtained by simultaneous measurement of VO 2 and recording of HR under standardized conditions. Calibration points were obtained for the following activities carried out in sequence: supine, sitting quietly, standing quietly (resting activities) and two exercise activities. HR was monitored for two to four separate days in the freeliving situation with a cardiofrequency meter (Sport Tester PE3000; Polar Electro, Kempele, Finland). Daytime HR was recorded at 1 min intervals up to a maximum recording time of 16 h, at which time stored information was retrieved and the memory reprogrammed. Twenty-four-hour EE from HR was calculated as follows. A FLEX HR was calculated as the mean of the highest HR for the resting activities and the lowest HR of the exercise activities. RMR was calculated as the mean of the VO 2 for the resting activities. EE for the 16 h of HR recording was determined as follows. For periods of the daytime when HR fell below FLEX HR, EE was calculated as RMR. For the remainder of the time, when HR was above FLEX HR, EE was derived from the minute-by-minute recorded HR by reference to the subject s regression line for the VO 2 corresponding to the HR. Twenty-four-hour EE was computed by summing the estimated EE by HR monitoring and EE at night. The latter was assumed to be equal to the measured BMR. Basal metabolic rate. BMR was measured (BMR meas ) by indirect calorimetry using a ventilated hood apparatus (Datex Metabolic Monitor; Datex Instrumentation Corpora- 457

5 458 tion, Helsinki, Finland). In the adult subjects, a classical BMR was measured in which subjects spent the previous night in a metabolic laboratory and BMR was measured at 1 min intervals for min under standardised conditions immediately upon waking and with minimal physical disturbance. Children were brought to the laboratory early in the morning in a fasted state and allowed to rest quietly for 20 min before BMR was measured. BMR was also estimated (BMR est ) for each subject from the appropriate Schofield equation based on height and weight (Schofield et al, 1985). Identifying dietary data of poor validity: reference definition Subjects were identified as acceptable reporters (AR), underreporters (UR), or over-reporters (OR) from their ratio of EI WR :EE DLW (adults and children) and EI DH :EE DLW (children only), according to whether the individual s EI:EE ratio was within, below or above the 95% confidence limits of agreement between the two measurements. The 95% confidence limits of agreement between EI WR or EI DH and EE DLW were calculated as 95%CL ¼2 ððcv 2 wei =dþþðcv weeþ 2 Þ where d is the number of days of diet assessment and CV wei and CV wee are the pooled mean coefficients of variation in EI (by WR or DH) and EE DLW, respectively. The CV wei in EI WR for the adults, for the children and for the total group was 27, 23 and 24%, respectively. For the purpose of this analysis, the EI data measured by DH were treated as 7 day records and assumed to have a pooled mean CV wei of 23% (Bingham, 1987; Nelson et al, 1989). The coefficient of variation (CV wee ) for EE DLW was taken as 8.2% (Black & Cole, 2000). This equation defined AR by WR as having an EI WR :EE DLW ratio within the range (adults), (children) and (total group). UR and OR were defined as having an EI WR :EE DLW ratio less than or greater than the minimum and maximum values of the appropriate ranges, respectively. AR by DH (children only) were defined as having an EI DH :EE DLW ratio in the range , UR as < 0.76 and OR as > Sensitivity and specificity The classification into ar, ur and or, according to the ratios defined above, provided the baseline or reference definition. Subjects were then classified as acceptable reporters (ar), under-reporters (ur), and over-reporters (or) according to three strategies described below. 1.55BMR. The principles of the Goldberg cut-off and the statistical derivation of the equation have been described in detail elsewhere (Goldberg et al, 1991; Black, 2000a, b). The revised factors to be used in the equation proposed by Black (2000a, b) were applied in calculating the cut-offs. 3 Each subject s value for EE HR was expressed as the PAL (EE HR :BMR meas ) and subjects were allocated to low, medium and high levels of activity as defined by the appropriate age sex physical activity level (PAL) values from the WHO recommended energy requirements (FAO=WHO=UNU, 1985). Upper and lower cut-offs were calculated for these individual PAL values using the Goldberg equation for n ¼ 1. The sensitivity of each of these three strategies for detecting UR was calculated as the proportion of UR correctly identified, while the specificity was calculated as the proportion of non-ur correctly identified. The percentage of all subjects mis-classified by each strategy was also calculated. Results The plots of EI WR (adults and children) and EI DH (children only) against EE DLW are shown in Figures 1 and 2. The dotted lines which indicate the 95% confidence limits of agreement for EI WR :EE DLW ( 25%) and EI DH :EE DLW ( 24%) provide the reference definition of AR, UR and OR by the respective dietary assessment methods. Approximately 80% of both adults and children provided acceptable reports of their EI WR, while under-reporting (14% adults; 11% children) was more prevalent than over-reporting. Comparison of mis-reporting by WR and DH (children only) shows that over-reporting was more prevalent by DH (17% DH vs 6% WR), while under-reporting was more prevalent by WR (11% WR vs 3% DH) (Table 2). Table 3 shows the sensitivity and specificity of the first strategy for evaluating reported EI, namely a direct compar- 1 Cut-offs for ar, ur and or were calculated from the 95% confidence limits of agreement between EI WR or EI DH and EE HR. 2 Subjects were classified according to their EI:BMR est and EI:BMR meas ratio and the upper and lower cut-offs were calculated by the Goldberg equation for n ¼ 1, assuming an energy requirement for a sedentary lifestyle of Figure 1 Energy intake assessed by seven day weighed record (WR EI) against energy expenditure measured by doubly labelled water (DLW EE) for the total group (n ¼ 50). Dotted lines indicate upper and lower 95% confidence limits of agreement between energy intake and energy expenditure.

6 Figure 2 Energy intake assessed by diet history (DH EI) against energy expenditure measured by doubly labelled water (DLW EE) for the children only (n ¼ 36). Dotted lines indicate upper and lower 95% confidence limits of agreement between energy intake and energy expenditure. ison of EI with EE measured by HR monitoring. It shows the number of AR, UR and OR, as defined by EI WR :EE DLW or EI DH :EE DLW, falling into each category of acceptable, underand over-reporters (designated by, ar, ur and or) as assessed by EI WR :EE HR or EI DH :EE HR. The numbers in brackets are subjects who have been mis-classified. The major impact of the wider 95% confidence limits of agreement for EI WR :EE HR, compared with EI WR :EE DLW, was to reduce the apparent extent of under-reporting by 50% in both adults and children. However, the specificity of EE HR for identifying non-ur by WR was good, with only one adult subject and none of the children being misclassified as a ur rather than an ar. On the other hand, the sensitivity and specificity of EE HR for identifying both UR and non-ur by DH (children only) were excellent. When cut-offs based on a blanket PAL of 1.55 were applied to the EI WR data, specificity also remained high (1.00 adults; 0.97 children) for cut-offs based on BMR est and was perfect (1.00) when based on BMR meas (Table 4). The sensitivity of this strategy in the adult subjects was similar to that of EE HR, with only 50% of subjects who under-reported by WR being correctly identified. However, sensitivity of the cut-offs was particularly poor when applied to the EI reports of the children. Only 25% of the children who under-reported by WR and none of the UR by DH were correctly identified using this strategy. In the third strategy, the subjects were assigned to low, medium and high activity levels according to their own 459 Table 2...ofEI WR or DH and EE DLW Weighed dietary record (WR) Diet history (DH) Children Adults Total Children Reporting category n % n % n % n % Acceptable reporters (AR) Over-reporters (OR) Under-reporters (UR) Total The numbers of subjects classified as acceptable-, over-, and under-reporters (AR, OR, UR) by the direct comparison of EI WR or DH and EE DLW Table 3 The numbers of UR, AR and OR as defined by the reference of EI WR or DH :EE DLW and as classified by EI WR or DH :EE HR Reference clasisication by EI:EE DLW UR AR OR Classification by EI:EE HR Lower cut-off Upper cut-off ur ar ur ar or ar or Mis-classified (%) Sensitivity Specificity EI WR :EE HR Total group (n ¼ 50) (3) (1) 40 (1) (1) Adults (n ¼ 14) (1) (1) Children (n ¼ 36) (2) 0 29 (1) (1) EI DH :EE HR Children (n ¼ 36) (1) UR and ur, under-reporters; AR and ar, acceptable reporters; OR and or, over-reporters. EI, energy intake; EE, energy expenditure, DLW, doubly labelled water; HR, heart rate monitoring; WR, weighed record; DH, diet history. Numbers in brackets are mis-classified subjects.

7 460 Table 4 The numbers of UR, AR and OR as defined by the reference EI WR or DH :EE DLW and as classified by EI:BMR using a blanket PAL of 1.55 and either a measured or estimated BMR to calculate the Goldberg cut-off Reference clasisication by EI:EE DLW UR AR OR Classification by EI:BMR Lower cut-off Upper cut-off ur ar ur ar or ar or Mis-classified (%) Sensitivity Specificity EI WR :BMR est Total group (n ¼ 50) (4) (1) Adults (n ¼ 14) (1) Children (n ¼ 36) (3) (1) EI WR :BMR meas Total group (n ¼ 50) (4) 0 41 (1) (1) Adults (n ¼ 14) (1) Children (n ¼ 36) (3) 0 29 (1) EI DH :BMR est Children (n ¼ 36) (1) 0 27 (2) EI DH :BMR meas Children (n ¼ 36) (1) 27 (2) (1) UR and ur, under-reporters; AR and ar, acceptable reporters; OR and or, over-reporters. EI, energy intake; EE, energy expenditure, DLW, doubly labelled water; BMR, basal metabolic rate; WR, weighed record; DH, diet history. Numbers in brackets are mis-classified subjects. measured PAL (EE HR :BMR meas ). Table 5 presents the WHO PAL values for the three activity levels for each age sex group and the boundaries selected for assigning subjects to each level (the midpoints between the WHO PAL values). Table 5 Activity levels and PAL ranges for low, medium and high activity levels for each age sex group Subjects PAL a Activity level PAL range b Adult males 1.55 Low < Medium Heavy > 1.94 Adult females 1.56 Low < Medium Heavy > , 9- and 12-y-old males 1.54 Low < Medium Heavy > , 9- and 12-y-old females 1.48 Low < Medium Heavy > y-old males 1.6 Low < Medium Heavy > y-old females 1.46 Low < Medium Heavy > 1.76 a PAL (physical activity level) values taken from FAO=WHO=UNU (1985). b Boundaries calculated as the mid-points of FAO=WHO=UNU (1985) values. Table 6 shows the effect of applying the upper and lower cutoffs based on these PAL values to scrutinize the EI data. Only data based on BMR est are presented, as the sensitivity and specificity were not improved by calculations based on BMR meas. Relative to the use of cut-offs based on a single measured PAL of 1.55, the application of cut-offs based on the PALs of the subjects did not alter the proportion of adults or children who were mis-classified. However, they were fully sensitive to the extent of under-reporting by WR in the adult subjects, albeit there was some loss of specificity. In contrast, while the cut-offs based on activity levels offered no improvement in sensitivity (0.25) when applied to the EI WR of the children, they were fully sensitive to correctly identifying UR by DH. Discussion At the outset it is acknowledged that the small study sample used in this analysis precludes any definitive conclusions concerning the efficacy of EE HR as an independent validity check on mis-reporting of EI data. Unfortunately, the expense and other problems of conducting such validation studies will inevitably constrain the numbers that can be studied, and hence, the generality of the conclusions for dietary surveys. The merit of this analysis lies in drawing attention to the possible pitfalls that could arise when applying validation techniques of unknown sensitivity and specificity for detecting mis-reporting of EI. With this caveat in mind, the results of this retrospective study on randomly selected subjects demonstrated that EE HR

8 Table 6 The numbers of UR, AR and OR as defined by the gold standard EI WR or DH :EE DLW and as classified by the cut-off for EI:BMR est using values for low, medium and high activity (calculated from EE HR :BMR meas ) as the reference PAL 461 Clasisication by EI:EE DLW UR AR OR Classification by EI:BMR est ur ar ur ar or ar or Mis-classified (%) Sensitivity Specificity EI WR :BMR est Total group (n ¼ 50) 3 (3) (1) 41 0 (1) (1) Adults (n ¼ 14) 2 0 (1) Children (n ¼ 36) 1 (3) (1) (1) EI DH :BMR est Children (n ¼ 36) 1 0 (1) 27 (1) (1) (5) UR and ur, under-reporters; AR and ar, acceptable reporters; OR and or, over-reporters. EI, energy intake; EE, energy expenditure, DLW, doubly labelled water; BMR, basal metabolic rate; PAL, physical activity level; WR, weighed record; DH, diet history. Numbers in brackets are mis-classified subjects. had only half the sensitivity of EE DLW for detecting UR by WR, although its specificity remained largely uncompromised. Relative to EE DLW, EE HR was equally effective for detecting UR and non-ur by the DH method (children only). The loss of sensitivity for screening EI WR can be largely attributed to the higher within-subject coefficient of variation in the measurements of EE by HR monitoring (13.3, 17.6, 11.2% for the total group, adults and children respectively) relative to DLW, which was taken as 8.2% (Black & Cole, 2000). In the original studies (Livingstone et al, 1990a, 1992a), EE HR and EE DLW showed good agreement at the group level, but individual estimates of EE HR ranged from 722 to þ 52% of corresponding EE DLW values. The reasons for these discrepancies have been discussed in detail in the source references (Livingstone et al, 1990a, 1992a) and include limited sampling periods for the estimation of HR EE, inappropriate Flex-HR definition and=or unrepresentative calibration data. In particular, it is most unlikely that 2 4 days of EE HR, as was measured in these subjects, would provide a representative estimate of habitual EE HR. Undoubtedly, the CV wee would be reduced and the sensitivity of the method enhanced, if the precision of HR monitoring could be improved. Inevitably, more lengthy periods of EE HR assessment would have practical and resource implications for researchers who intend to exploit this methodology as a validation tool. However, failure to extend the assessment periods would mean that EE HR would offer little advantage over the original Goldberg equation based on a blanket PAL of Another constraint associated with estimating EE HR is the requirement for individual calibration of HR vs VO 2, necessitating appropriate metabolic facilities for measuring respiratory gas exchange. Thus, assessment of EE HR for screening EI in large dietary surveys may simply not be a viable option. However, with the proviso that representative measures of EE HR must be obtained to maximize sensitivity, the method could prove an extremely effective validation tool in small studies where individual data have much greater influence on the results and conclusions (Black et al, 1997). An alternative application of HR monitoring, more suited to larger population samples, is to dispense with the assessment of EE HR in favour of simply defining patterns of physical activity as low, medium or high. In default, subjects in this analysis were classified into their respective activity levels based on a comparison of their measured EE HR :BMRmeas ratio with WHO PALs since this is a better measure of true activity level as measured by HR. Overall, this strategy proved as sensitive for screening EI WR data as that based on individual measurements of EE HR and more sensitive than that based on a blanket PAL of Nevertheless, it is apparent that cut-offs based on activity levels are of differential sensitivity, given that they identified all of the adult UR but 75% of the children who under-reported by WR went undetected. Clearly, the efficacy of this strategy hinges on being able to select a suitable PAL value for each activity level. Even then, differences in sensitivity will emerge depending on the extent to which a selected PAL at any one level is close to the true value. For example, the actual mean EE HR :BMR meas of the adult subjects assigned to each activity level were 1.42, 1.68 and 2.30 for the men and 1.56, 1.65 and 1.78 for women while the corresponding WHO PAL values used to derive the cut-offs were 1.55, 1.78 and 2.10 (men), and 1.56, 1.64 and 1.80 (women). Fortuitously, these cut-offs proved to be fully sensitive in the adult group but the penalty was a reduced specificity. Even if subjects can be correctly classified into their respective activity levels, the application of a single cut-off to encompass the range of EE within a given activity level will vary the probability between subjects that they will be (mis)classified as a UR, AR or OR. Consequently, while the cut-offs based on the physical activity levels clearly have more merit than those

9 462 based on a blanket cut-off of 1.55, and should be used in preference, it is with the caveat that they are most unlikely to fulfill the criterion of full sensitivity as shown in this small sample of adults. The differential sensitivity of cut-offs is particularly well exemplified in the dietary reports of the children. Most attempts at identifying mis-reporting of EI by children have applied the same strategies as those used to scrutinize the EI data of adults. Intuitively this seems logical, but as the present analysis has shown, there are inherent problems in doing so. Of the strategies explored, EE HR was the most sensitive and specific indicator of mis-reporting by WR in children and it is reasonable to assume that the detection of UR would improve if due cognisance was taken of the need to obtain more representative estimates of EE HR. The use of cut-offs based on a blanket PAL of 1.55 and physical activity levels was much more problematic because application of each of these strategies was only sensitive to 25% of UR by EI WR, even though their specificity was perfect. It is perhaps not surprising that cut-offs based on a blanket PAL of 1.55 were so insensitive for evaluating the EI WR in a paediatric population. A PAL of 1.55 was originally devised to evaluate the overall bias to under-reporting in a sedentary adult population, although in theory the cut-off values for a sample size of n ¼ 1 can be used to identify UR at the individual level (Goldberg et al, 1991; Black, 2000a, b). Consequently, its application for identifying individual mis-reporters in paediatric groups cannot be advocated. Of equal concern is that application of age-sex specific cut-offs based on three levels of physical activity did not improve the sensitivity of the technique, suggesting that the WHO recommended PAL levels for these age-sex-activity groups may be inappropriate for this purpose (Torun et al, 1996). In the present state of knowledge, therefore, all cut-offs based on assumed PAL levels used for screening the EI data of children should be applied with caution and with full acknowledgement of the pitfalls involved. There is some evidence from these data to suggest that EE HR and cut-offs based on physical activity levels may be more sensitive in detecting UR by EI DH. In marked contrast, a cut-off based on a PAL of 1.55 was totally insensitive to EI DH underreporting. However, EI DH was assessed only in the children and, as this analysis shows, the issue with these data was of over-reporting, not under-reporting. Therefore, no conclusions about the efficacy of these strategies for detecting UR by DH in this, or any other group, could be justified at present. The final issue is whether sensitivity and specificity of cutoffs based on assumed PAL values would be improved if BMR was not estimated but measured directly. The data presented in Table 4 show no differences in the sensitivities based on BMR est or BMR meas, although the detection of non-ur was marginally improved. Therefore, any advantages of BMR meas, relative to the resource implications of making such measurements, do not justify inclusion of BMR meas in large epidemiological studies. However, BMR meas in small-scale studies could help to avoid misclassification of subjects (Black et al, 1997). The identification of mis-reporters of EI at the individual level must underpin any attempt to assess the causes and consequences of mis-reporting in dietary surveys. This is a formidable challenge which can only be addressed by direct comparison of EI with EE, or in default, by using cut-offs for EI:BMR based on the physical activity level of individual subjects. However, as this analysis has shown, currently available techniques such as EE HR may be able to identify only gross bias. The implications are salutary. Before any technique for estimating EE comes into common use for validating EI, its sensitivity and specificity for detecting misreporters needs to be carefully evaluated, not just assumed. Otherwise, indiscriminate use of instruments of unknown sensitivity and specificity will only generate more confusion in an already confused area. Acknowledgements We thank the subjects who took part in these studies. References Bingham S (1987): The dietary assessment of individuals; methods, accuracy, new techniques and recommendations. Nutr. Abstr. Rev. 57, Black AE, Prentice AM, Goldberg GR, Jebb SA, Bingham SA, Livingstone MBE and Coward WA (1993): Measurements of total energy expenditure provide insights into the validity of dietary measurements of energy intake. J. Am. Diet. Assoc. 93, Black AE (1997): Under-reporting of energy intake at all levels of energy expenditure: evidence from doubly labelled water studies. Proc. Nutr. Soc. 56, 121A. Black AE (1999): Small eaters or under-reporters? In Progress in Obesity Research, ed. G Ailhaud & B Guy-Grand pp Paris: John Libbey. Black AE (2000a): Critical evaluation of energy intake using the Goldberg cut-off for energy intake: basal metabolic rate. A practical guide to its calculation, use and limitations. Int. J. Obes. Relat. Metab. Disord. 24, Black AE (2000b): The sensitivity and specificity of the Goldberg cutoff for EI:BMR for identifying diet reports of poor validity. Eur. J. Clin. Nutr. 54, Black AE & Cole TJ (2000): Within- and between-subject variation in energy expenditure measured by doubly-labelled water technique: implications for validating reported dietary energy intake. Eur. J. Clin. Nutr. 54, Black AE, Prentice AM and Coward WA (1986): Use of food quotients to predict respiratory quotients for the doubly-labelled water method of measuring energy expenditure. Hum. Nutr. Clin. Nutr. 40C, Black AE, Bingham SA, Johansson G & Coward WA (1997): Validation of dietary intakes of protein and energy against 24 hour urinary N and DLW energy expenditure in middle aged women and retired men: comparison with validation against presumed energy requirements. Eur. J. Clin. Nutr. 51, Briefel RR, Sempos CT, McDowell MA, Chien S & Alaimo K (1997): Dietary methods research in the third National Health and Nutrition Examination Survey: under-reporting of energy intake. Am. J. Clin. Nutr. 65, S1203 S1209. Charlton KE & Lambert EV (1999): Validation of a food frequency questionnaire in older South Africans. S.A. Med. J. 89, Coward WA (1988): The doubly-labelled-water ( 2 H 2 18 O) method: principles and practice. Proc. Nutr. Soc. 47,

10 de Vries JH, Zock PL, Mensink RP, Katan MB (1994): Underestimation of energy intake by 3-d records compared with energy intake to maintain body weight in 269 nonobese adults. Am. J. Clin. Nutr. 60, FAO=WHO=UNU (1985): University. Report of a joint expert consultation. Energy and protein requirements. WHO Technical Report Series no. 724 Geneva: WHO. Fogelholm M, Männisto S, Vartiainen E & Pietinen P (1996): Determinants of energy balance and overweight in Finland 1982 and Int. J. Obes. Relat. Metab. Disord. 20, Goldberg GR, Black AE, Jebb SA, Cole TJ, Murgatroyd PR, Coward WA & Prentice AM (1991): Critical evaluation of energy intake data using fundamental principles of energy physiology. 1. Derivation of cut-off values to identify under-recording. Eur. J. Clin. Nutr. 45, Goris AHC & Westerterp KR (1999): Underreporting of habitual food intake is explained by undereating in highly motivated lean women. J. Nutr. 129, Heywood P, Harvey PJW & Marks GC (1993): An evaluation of energy intake in the 1983 Australian National Dietary Survey of Adults. Eur. J. Clin. Nutr. 47, International Dietary Energy Consultative Group (IDECG) (1990). The doubly-labelled water method for measuring energy expenditure: technical recommendations for human applications, ed. AM Prentice. Vienna: IAEA=IDECG NAHRES-4. Johnson RK, Goran MI & Poehlman ET (1994): Correlates of overand underreporting of energy intake in healthy older men and women. Am. J. Clin. Nutr. 59, Körtzinger I, Bierwag A, Mast M & Müller MJ (1997): Measurements of energy intake using a 7-day dietary record and a diet history in non-obese subjects. Ann. Nutr. Metab. 41, Lafay L, Basdevant A, Charles M-A, Vray M, Balkau B, Borys J-M, Eschwege E & Romon M (1997): Determinants and nature of dietary underreporting in a free-living population: the Fleurbaix Laventie Ville-Santé (FLVS) Study. Int. J. Obes. Relat. Metab. Disord. 27, Livingstone MBE, Prentice AM, Coward WA, Ceesay SM, Strain JJ, Mckenna PG, Nevin GB, Barker ME & Hickey RJ (1990a): Simultaneous measurement of free-living energy expenditure by the doubly labeled water method and heart-rate monitoring. Am. J. Clin. Nutr. 52, Livingstone MBE, Prentice AM, Strain JJ, Coward JJ, Black AE, Barker ME, McKenna PG & Whitehead RG (1990b): Accuracy of weighed dietary records in studies of diet and health. Br. Med. J. 300, Livingstone MBE, Coward WA, Prentice AM, Davies PSW, Strain JJ, McKenna PG, Mahoney CA, White JA, Stewart CM & Kerr MJ (1992a): Daily energy expenditure in free-living children: comparison of heart-rate monitoring with the doubly labeled water ( 2 H 2 18 O) method. Am. J. Clin. Nutr. 56, Livingstone MBE, Prentice AM, Coward WA, Strain JJ, Black AE, Davies PSW, Stewart CM, McKenna PG & Whitehead RG (1992b): Validation of estimates of energy intake and diet history in children and adolescents. Am. J. Clin. Nutr. 56, Nelson M, Black AE, Morris JA & Cole TJ (1989): Between- and within-subject variation in nutrient intake from infancy to old age: estimating the number of days required to rank dietary intake with desired precision. Am. J. Clin. Nutr. 50, Price GM, Paul AA, Cole TJ & Wadsworth MEJ (1997): Characteristics of the low-energy reporters in a longitudinal national dietary survey. Br. J. Nutr. 77, Pryer J, Vrijheid M, Nichols R, Kiggins M & Elliott P (1997): Who are the Low energy reporters in the dietary and nutritional survey of British adults? Int. J. Epidemiol. 26, Rothenberg E, Bosaeus I & Steen B (1997): Evaluation of energy intake estimated by a diet history in three free-living 70 year old populations in Gothenburg, Sweden. Eur. J. Clin. Nutr. 51, Schoeller DA (1990): How accurate is self-reported dietary energy intake? Nutr. Rev. 48, Schofield WN, Schofield C & James WPT (1985): Basal metabolic rate. Hum. Nutr. Clin. Nutr. 39C(Suppl 1), S1 S96. Speakman JR (1997): Doubly Labelled Water: Theory and Practice. London: Chapman and Hall. Torun B, Davies PSW, Livingstone MBE, Paolisso M, Sackett, & Spurr GB (1996): Energy requirements and dietary energy recommendations for children and adolescents 1 to 18 years. Eur. J. Clin. Nutr. 50(Suppl 1), S37 S81. Voss S, Kroke A, Klipstein-Grobusch K, Boeing H (1997): Obesity as a major determinant of underreporting in a self-administered food frequency questionnaire: results from the EPIC-Potsdam Study. Z. Ernahrungswiss. 36,