Author's response to reviews Title: Reliability and Relative Validity of a Food Frequency Questionnaire to Assess Food Group Intakes in New Zealand Adolescents Authors: Jyh Eiin Wong (jyh.wong@otago.ac.nz) Winsome Parnell (winsome.parnell@otago.ac.nz) Katherine E Black (katherine.black@otago.ac.nz) Paula ML Skidmore (paula.skidmore@otago.ac.nz) Version: 2 Date: 3 August 2012 Author's response to reviews: see over
Dear Editor, We would like to thank the reviewers for their valuable and constructive comments, which have improved the clarity of the manuscript. We have revised the manuscript accordingly. Please see are our point-by-point response to the comments below. Reviewer 1 1. The authors acknowledge the sample size is small (52 for test-retest reliability and 41 for relative validity) and clearly state this in their abstract. However, they do not state the recommended sample size. Something along the lines of the following could be included: A sample size of at least 50 is desirable [Cade 2004], and ideally between 100 and 200 participants [Willett 1998]. Cade, J.E., et al., Food-frequency questionnaires: a review of their design, validation and utilisation. Nutrition Research Reviews, 2004. 17: p. 5-22. Willett, W. and E. Lenart, Reproducibility and validity of food-frequency questionnaires, in Nutritional Epidemiology, W. Willett, Editor. 1998, Oxford University Press: Oxford. A sentence has now been added to our Discussion (page 13, line 19-22) to describe the recommended sample size. 2. Abstract: If the word limit permits, the statistical analyses for testing reliability and relative validity would be a useful inclusion in the abstract (ie. as on page 9, 2nd para, 1 st sentence for validity, 3rd sentence for reliability). This would help clarify the results section of the abstract. A key conclusion (as stated on page 14) is the useful of the FFQ in assessing dietary patterns. Again, if word limits permit, it would be good to state this in the conclusions in the abstract, specifically the suitability of the FFQ for identifying dietary patterns at the population level. We have amended the abstract as suggested by the Reviewer (page 2, line 13-15 and page 3, line1). 3. The authors may also like to consider giving their tool a name, to enable it to be easily identified from other questionnaires. We thank the Reviewer for this suggestion. We have named the FFQ the New Zealand Adolescent FFQ (NZAFFQ), and referred to this throughout the manuscript. 1
4. The FFQs upon which the study FFQ is based are for slightly younger age groups. The Health Behaviour in School-aged Children (HBSC) is designed for 11 to 14 years. The Children s Dietary Questionnaire (CDQ) is designed for 4 to 16 year olds. However, there is no reason the design of these FFQs is not suitable for the older age groups, but I am wondering why other FFQs developed for older age groups were not the basis for the design of this one since the study population is 14 to 18 years. It would be good to include this justification. Our literature search showed that although some FFQs exist for use in older adolescents aged above 16 years [1-4], they contain portion size questions and/or extensive food lists, which are not relevant to the New Zealand context. We used the HBSC FFQ and CDQ as the basic construct of our FFQ as these two nonquantitative FFQs covered different important aspects (i.e. variety and intake frequency) of an adolescent s diet and have been used to derive index-based dietary patterns [5-7] (see page 6, line 22-24). 5. Page 6, 3rd paragraph, 1st sentence: the reference for the Veerecken paper (24) should be moved to the middle of the sentence, following HBSC FFQ as it relates to this FFQ and not foods relevant to the New Zealand adolescent population. This correction has been made. We thank the Reviewer for pointing this out. 6. Reliability testing: FFQ completed twice within a two-week period. On page 8, the authors note that This time interval was chosen to minimize the variation in food intake responses due to true changes over time. However, this short timeframe is likely to enhance the test-retest reliability results. It would have been ideal to leave a little more time between FFQs (for example 1 month as other studies Cullen et al. Relative reliability and validity of the Block Kids Questionnaire among youth aged 10 to 17 years. J Am Diet Assoc. 2008 May; 108(5):862-6. We agree that two-week interval chosen may have enhanced our test-retest reliability results. Our decision to assess test-retest reliability at two-week interval was informed by a New Zealand FFQ repeatability study on 130 children and young adolescents [8]. Metcalf and colleagues found no significant differences in the reported food servings between two FFQs when administered 2-weeks apart (please see page 8, line 31-32). We have also added additional information to the discussion (page 11, line 18-20) to acknowledge that this may have an impact on our estimates of validity. 7. Relative validity testing: I feel it would be beneficial to clarify throughout the paper that relative validity was assessed using FFQ1 data. In the Relative Validity paragraph on page 10, 2
it would be good to reiterate that the FFQ data was from FFQ1 only (as stated on page 9m 2nd para, 1st sentence). As well as adding this clarification to Table 2. We have clarified (Page 9, line 16) that the first administration of the FFQ was used for the relative validity analysis. We now refer to the first administration of the FFQ as FFQtime1 throughout the manuscript and in Table 2. 8. The FFQ data used for relative validity has consequences for the interpretation of the validity results. FFQ1 is likely to underestimate relative validity (as mentioned on page 12, 1st para). We acknowledge that the use of the first administration of the FFQ could potentially lead to an underestimation of validity, compared to using the second administration of the FFQ (see page 12, line 21-23). However, in overall estimations of validity were essentially the same if the second administration of the FFQ was used for validity analyses (Please see comments to Reviewer 2 for further information). Reviewer 2 1. Methods, Data and statistical analysis: The quantity or gram of food consumption in the 4DFR seems not to be considered in the analysis for relative validity and only the intake frequency is taken into account. If so, it should be more clearly mentioned here. Additionally, what if the intake quantity derived from the 4DFR is included in the analysis? Although a nonquantitative FFQ is used, such analysis may attenuate the effect of intake in a very small portion, of which biological significance may be negligible. As our FFQ was designed to specifically rank participants by frequency of intake, and not for estimating actual absolute intakes, the frequency responses were used as the unit of comparison between the two methods. We have added an additional sentence in the Methods section (page 9, line 11-13) to clarify this. The FFQ does not contain any information on portion size therefore validation estimates obtained using actual amounts of foods would not be appropriate. An average portion size for each food item (question) in the FFQ would have to be determined, which would introduce additional measurement error. 2. Results and Table 2: In assessing the relative validity, this reviewer recommends the authors to compare the median and inter-quartile range of food intake frequency between the FFQ and the 4DFR. This will clarify the under- or over-reporting in the FFQ. Our choice of statistical method was governed by the intended application of the FFQ, which is to rank participants by food group intakes. We agree with the reviewer that examination of absolute agreement (i.e. comparison of means or medians) is 3
appropriate for use in some validation studies, i.e. specifically for studies where estimates of actual intake are the primary aim of the study. Our statistical approach was undertaken after seeking advice from a statistician and nutritional epidemiologist whose expertise is in FFQ validation. As the FFQ is not designed to provide actual estimates of intake, comparisons of under and over-reporting in terms of actual intake are not needed. 3. Discussion, page 12, lines 2 9: In the assessment of relative validity, the authors adopted the first FFQ before the 4DFR to eliminate learning effects from completion of a dietary record. How the validity is altered if the second FFQ after the 4DFR is used? This analysis would provide an informative finding because the true validity may exist between the validity estimated from the first FFQ and that from the second one. We acknowledge that there are alternative ways to approach validity analyses. When comparing the relative validity using the second FFQ, some food groups showed slightly higher correlation coefficients, however the overall correlation coefficients were the same (median SCC=0.40 for first FFQ vs. 0.41 for second FFQ). Although using the first FFQ is likely to underestimate validity, the use of the first administration of an FFQ is in keeping with international expert recommendations for validation studies [9]. The use the first FFQ represents a more conservative approach to avoid overestimation of validity. 4. Abstract, Methods, line 1: The phrase "a non-quantitative" may not be familiar to readers. Please add a more specific description such as "without portion sizes". We have added the recommended specific description without portion size in the abstract (page 2, line 9). We have also taken the opportunity to include the description without collection of portion size information in our Introduction (page 5, line 9-10) to clarify this term. 5. Results, Sample: Although the issue of low compliance rate is appropriately addressed in the Discussion, comparing characteristics (e.g., age, gender) between those who completed the study and those who did not may be useful to assess the extent of bias due to the dropouts. Following the Reviewer s suggestion, we compared the demographic characteristics of those who completed the study with those excluded due to incomplete data. While more males than females did not complete all parts of the study (χ 2 =7.6404, P=0.006), there were no differences in demographics and anthropometric measures between study completers and non-completers. We have added additional information to the results text (page 10, line 1-4). 4
6. Results, Sample, line 5: The sum of the numbers of male participants (28) and female ones (25) does not equal the total number (52) for the reliability study. The above typographical mistake has been rectified. Thank you for pointing this out. 5
References 1. Araujo MC, Yokoo EM, Pereira RA: Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc 2010, 110:1170-1177. 2. Cullen KW, Watson K, Zakeri I: Relative reliability and validity of the Block Kids Questionnaire among youth aged 10 to 17 years. J Am Diet Assoc 2008, 108:862-866. 3. Rockett HR, Breitenbach M, Frazier AL, Witschi J, Wolf AM, Field AE, Colditz GA: Validation of a youth/adolescent food frequency questionnaire. Prev Med 1997, 26:808-816. 4. Xia W, Sun C, Zhang L, Zhang X, Wang J, Wang H, Wu L: Reproducibility and relative validity of a food frequency questionnaire developed for female adolescents in Suihua, North China. PLoS ONE 2011, 6:e19656. 5. Magarey A, Golley R, Spurrier N, Goodwin E, Ong F: Reliability and validity of the Children's Dietary Questionnaire; a new tool to measure children's dietary patterns. Int J Pediatr Obes 2009, 4:257-265. 6. Vereecken CA, Rossi S, Giacchi MV, Maes L: Comparison of a short foodfrequency questionnaire and derived indices with a seven-day diet record in Belgian and Italian children. Int J Public Health 2008, 53:297-305. 7. Yannakoulia M, Karayiannis D, Terzidou M, Kokkevi A, Sidossis LS: Nutrition-related habits of Greek adolescents. Eur J Clin Nutr 2004, 58:580-586. 8. Metcalf PA, Scragg RK, Sharpe S, Fitzgerald ED, Schaaf D, Watts C: Shortterm repeatability of a food frequency questionnaire in New Zealand children aged 1-14 y. Eur J Clin Nutr 2003, 57:1498-1503. 9. Cade J, Thompson R, Burley V, Warm D: Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr 2002, 5:567-587. 6