Example of a quality control protocol for a food consumption survey among infants, toddlers and children based on the PANCAKE project

Transcription

1 Example of a quality control protocol for a food consumption survey among infants, toddlers and children based on the PANCAKE project 1

2 Contents Contents Scope Objective Description of quality controls Organization of the survey Sampling and data analyses Interview/data entry Data cleaning of food consumption data Data interpretations of food consumption data Anthropometry General characteristics Inconsistencies between food consumption data of 2-3 days, general questionnaire and general information Scoring indicators of quality control

3 1. Scope This document describes the standard operating procedures for the quality control procedures in the pan European food consumption survey among infants, toddlers and children and their lactating mothers. It is to be used as a generic document applicable for all participating countries. For each participating country, these procedures should be applied and if necessary translated in their own specific organization. 2. Objective The objective of this SOP is to standardise and harmonise the quality controls within the Pan- European food consumption survey among infants, toddlers and children and their lactating mothers in order to ensure that the quality of the data is high and the results across the participating countries are comparable. In addition, this SOP helps to get insight in the quality of the surveys in the different European countries. This document describes generally the specific aspects on which quality controls should be conducted by each participating organization. Some quality controls are indicated as less essential, although these controls affect the quality of the results and will affect the interpretation. This document may serve the participating partners for defining country specific quality controls in the different steps of the survey: during the sampling and recruitment of the study population, during the data collection to collect reliable and detailed data, during the data entry, in the data check and cleaning phase, and the data handling phase. Insight in the quality of the results is essential for the interpretation and comparison of the results. Therefore, for each quality aspect an indicator is given. 3. Description of quality controls In this chapter the guidelines for the different quality controls are given. As quality controls should be conducted in several steps of the survey, also these quality controls shall be conducted by several persons, depending on who is performing that specific step of the survey. In table 1 an overview of the quality controls by each step in the survey is shown, including the most likely responsible persons (this might however differ according to the local situation) and the indicator for the interpretation of the quality. All proposed quality controls are discussed in the following paragraphs. 3

4 Table 1 Overview of the quality controls Quality control Aim quality control Most likely performing organization/function Organization survey Indicator of quality + =high quality =lower quality Essential List tasks, competences, responsibilities of all personnel Insight in / clear assignments of tasks, competences, responsibilities List of personnel for conducting the survey Organization who is organizing the survey (project leader) Overview of tasks, competences, responsibilities available? + = yes; = no Equip qualified personnel Qualified personnel for conducting the survey Organization who is organizing the surveyproject leader Personnel qualified? + = yes; = no Organize communication between different bodies/members Clear communication Organization who is organizing the survey Communication organized: + = yes; = no Sampling / data analyses Select potential participants following sampling protocol High representativeness Organization conducting sampling Guidelines sampling protocol followed? + = yes; = no Monitor response across target groups High representativeness Organization who is conducting the fieldwork Data collection monitored and if necessary adapted on this aspect during study + = yes; - = no. Indicate nonresponse rate Low non-response and high representativeness Organization conducting data analyses % of indicate % of non contactable and % of non eligible and % of refusing individuals by agegender stratum Get insight in Representativeness (age, gender, district, and ses) High representativeness of study population Organization conducting data analyses Qualitative indication of representativeness + = high; = low, namely not for 4

5 Quality control Aim quality control Most likely performing organization/function Indicator of quality + =high quality =lower quality Essential Weigh results for non-representative distributions across socio-demographic groups* High representativeness of study population Organization who is conducting the data analyses In case of unequal distributions, Weighing factor is used: + = yes or not necessary; = no Get insight in distribution of Record day Equal distribution across days of the week Organization conducting the data analyses Days equally distributed + = 14,3 % for each day (between 10,0-18,0%) - = not equally distributed Get Insight in distribution of Season* Equal distribution across seasons; Organization conducting the data analyses Days equally distributed + = 25% (between 20-30%) - = not equally distributed Weigh for inequalities in day /season Representative results Organization who is doing the data-analyses If necessary, weighing factor is used + = yes, or not necessary - = no. Get insight in time period between record days in case of 2 * 1 day diary Independent days Organization conducting the data analyses % of records with more than 14 days between both record days Interviewers Train national coordinators on the dietary tool* Qualified trainers EFSA or delegated organization National coordinators trained on the dietary tool? + = yes; - = no Train interviewers Qualified personnel for conducting the interview and doing data entry. Organization who is conducting the fieldwork Interviewers trained according to training protocol? + = yes; - = no 5

6 Quality control Aim quality control Most likely performing organization/function Indicator of quality + =high quality =lower quality Essential Keep training program for interviewers data entry staff up to date Up to date training program Organization who is conducting the fieldwork Training program kept up to date? + = yes or not relevant; - = no No Monitor performance of interviewers on recruitment Insight in performance of interviewers and if necessary adapt Organization conducting the fieldwork Indicators mentioned in the protocol ( A b- 1) No Monitor performance of interviewers on progress in fieldwork Insight in performance of interviewers and if necessary adapt Organization conducting the fieldwork Indicators mentioned in the protocol ( A b- 2) No Monitor performance of interviewers on quality of fieldwork Insight in performance of interviewers and if necessary adapt Organization conducting the fieldwork Indication of type of performed quality controls and conclusion of its results Data entry Perform standard quality checks available in dietary software during data entry High quality of obtained 24 hr food consumption data Interviewer Standard Quality checks of the dietary software performed? + = yes; - = no Data cleaning food consumption data Check the outliers High quality of obtained food consumption data Data cleaners (dieticians/data managers) Outliers checked? + = yes; - = no No Solve all notes in the food consumption data High quality of food consumption data Data cleaners (dieticians/data managers) % of checked notes No Solve other errors following the dietary software manual High quality of food consumption data Data cleaners (dieticians/data managers) Checks from dietary software manual performed? No + yes; - = no 6

7 Data interpretations of food consumption data Evaluate the energy intake on poor validity (at group level) Insight in possible influence of misreporting on the conclusions Organization conducting the data analyses and dissemination Comparison of energy ratio with PAL Estimate % of high and low energy reporters (at individual level) Insight in prevalence of high and low energy reporters Organization who is doing the data-analyses and dissemination % of high and low energy reporters No General data Limit the number of missings and inconsistencies High quality of general data Organization conducting the data analyses Data entry forms developed for pan- European survey is used Estimate the number of missings and inconsistencies High quality of general data Organization who is doing the data-analyses Number of missings and inconsistencies per total participants No Anthropometry Follow protocol for anthropometric values High quality of anthropometric data Organization the fieldwork conducting Is the standard protocol for anthropometric values used? + = yes; - = no Quality control of devices High quality of anthropometric data Organization the fieldwork conducting Are the devices calibrated each 6 months? No + = yes; - = no General Check on completeness of the data High quality of the data Data managers % of the participants in final study population with incomplete data No Entry part of general data twice Double data entry for sub sample? Interviewer % of double data entry No 7

8 3.1 Organization of the survey In order to ensure the quality of the organization of the survey, tasks, competences and responsibilities of all personnel involved should be clear and the personnel should be qualified for conducting the task. In addition, the communication should be well organized. These quality issues are conducted by the organization that is responsible for the survey. - Make a list of the tasks, competences and responsibilities for the personnel. The availability of this list is the indicator of this quality control. This list should be available before the start of the data collection and the training. - Equip qualified personnel for each step of the survey. An example for the qualifications of an interviewer is given in annex A. To harmonise the qualification across the European countries, qualifications for each type of personnel should be worked out. The qualification requirements should be available before assignment of tasks to available personnel and before recruitment of personnel. A list of all personnel involved and whether the persons are qualified for the tasks is the indicator for this quality control. If a person is not or not entirely qualified the discrepancy should be described. - Make an overview of the communication between the different body s members of the organization. 3.2 Sampling and data analyses Representativeness The purpose is to obtain a representative study population in each participating country of the pan-european survey. In addition, it is important to obtain insight in the representativeness of the study population for the target population; response rates are an important aspect for this. Representativeness is important for the interpretation of the national results and for the comparison of the national results across countries. During the fieldwork, the figures on refusals, non-contactables, non-eligible persons, completed materials, etc are reported by the interviewers by specific forms to the study secretary. (See for more details in the sampling and administration protocol.) Based on these figures the performance of the interviewers in this regard can be monitored and if necessary, the interviewers get more instructions in order to improve the results on this quality aspect. This monitoring during the survey is especially relevant for long-term surveys. The overall figures give insight in the representativeness of the whole study, which is of importance for the interpretation of the results. In case of unequal distributions of relevant characteristics of the study population and the target population, these aspects can be taken into account in the data analyses. The following activities should be conducted by the organization who is conducting the sampling and the data analyses: - Follow the guidelines in the sampling protocol for the selection of the potential participants. Report any deviance of these guidelines if this should occur. 8

9 - To improve the response rate from long-term surveys, the non-response rates in each subgroup should be monitored during the study by the organization that is performing the fieldwork. If necessary, the sampling can be adapted to obtain the required numbers and/or the recruitment procedure can be changed; if applicable for specific subgroups only. An indicator for this quality control is the final number of participant and the response rate for each age-sex stratum. - Compare the characteristics of the target population on age-sex and socio-demographic factors (district, and a factor for socioeconomic status) and those selected from the national register or national census data to examine the representativeness of the study population. If possible also for other main life style factors, like obesity, physical activity and diet. If non-response questions are allowed, the comparison between the respondents and the non-respondents for which this information is available should be made. The indicator for the quality of the representativeness is a qualitative indicator. In case of + the study population is assumed to be representative. In case of -, it is not representative for the factors mentioned above. - If necessary, adjust in the analyses with a weighing factor for eventual unequal distributions across the socio-demographic groups. Record days/interview days The purpose is to obtain a representative data collection across the seasons of the year and days of the week. NB in the pilot in PANCAKE season will not be taken into account. If the sampling protocol is followed then each interviewer has to contact the participants on predefined days. At the end of the study figures about distribution of the record days and the seasons give insight in the representativeness of the data, which is important for the interpretation of the results. During the fieldwork, the day of the week and season can be monitored in order to evaluate and improve the data collection on this issue. The following activities should be conducted by the organization who is conducting the fieldwork and the data analyses: - Monitor and adjust if necessary the distribution of the across seasons and days of the week of the dietary records during the study. All seasons and days should be equally available (approximately 14% each day and 25% each season). The cut-off for each season are arbitrary chosen as the first record/recall day in the period: Winter: 1 th December 29 th Februari Spring: 1 th March 31 th May Summer: 1 th June 31 th August Autumn: 1 th September 30 th November. - The indicators of this quality control is the distribution of the days of the week and the distribution across seasons. The target for each day is 14% with a range between 10 and 18%. The target for each season is 25% with a range between 20 and 30%. If differences occur from the targets, the indicators of this quality control would be negative. 9

10 - If necessary, adjust in the analyses with a weighing factor for eventual unequal distributions for day of the week or season. The use of this weighing factor is the indicator. - Get insight in the time period between the record days in case of method of 2x 24hr records. Ideally, the record days should be independently of each other. It is assumed that the period between two record days should be at least 2 weeks. The proportion of participants with more than 2 weeks is the indicator of this quality control. 3.3 Interview/data entry Competence of interviewers The interviewer should be trained to have sufficient knowledge of data collection software, other study tools, food consumption habits, and the study protocol (see annex 1 for an example of recommended qualifications). To harmonize and standardize the national training for a pan-european food consumption survey, it is advised to organize first an international training of the trainers. This could be done by the overall coordinating centre, or be delegated to an organization with a lot of expertise on this aspect. Each country will organize the training of its local interviewers. The following quality aspects should be taken into account: - Train national coordinators on the dietary software in a harmonized way. - Train interviewers specifically for all tasks involved in the study. The topics of the training period are given in the interview protocol (annexes). Whether the interviewers are trained is the indicator for this quality aspect. A testing phase is recommended before starting the field work to make sure that instructions are well understood and if necessary adjustments can be made. - In a pan-european survey, this training program should be kept up to date in terms of evolution of the local markets and features of the dietary software. Whether this training program is actual, is the indicator for this quality aspect. Monitor and report on quality of interviewers The performance of the interviewers should be monitored and evaluated during and at the end of the study. There are several possibilities to monitor these effects: Monitor the performance of the dieticians with regard to the recruitment and if necessary give feed back in order to improve the performance. The protocol for this is given in annex B-1. The indicators from this protocol are the indicator for this quality control. Monitor the performance of the dieticians with regard to the progress on the fieldwork and if necessary give feed back in order to improve the performance. The protocol for this is given in annex B-2. The indicators from this protocol are the indicator for this quality control. Monitor the performance of the dieticians with regard to the quality of the fieldwork and if necessary give feed back in order to improve the performance. Examples of possible monitoring activities are: 10

11 o o o o o o o Perform fake interviews. Each interviewer will contact a fake participant. This person will report predefined food consumption. During the interview, the fake participants will evaluate the interview on several aspects. To contact a predefined number of participants that completed the survey to check if all parts of the survey were carried out. To recontact a predefined number of participants to check whether all parts of the survey were carried out. For example for each active dietician, 1 individual having participated to the 2*1dFR, 1 individual having participated to the 3dFR and 1 individual having not participated (refusal, non-eligible or non-contactable) will be recontacted. Those persons will be recontacted by phone (it is asked to the dieticians to record the phone number of each person during their contact attempts (Registration form for contact attempts). Verification involves calling the respondent by telephone and asking 2-3 questions from the questionnaire and some questions on the process. Double data entry of the 3dFR by different data entry operators. Audio record and evaluate for each interviewer a couple of interviews. Monitor number of errors made using the dietary software by interviewers, based on the data controls during data cleaning. Monitor interviewer effects in energy and other dietary components. These possible monitoring activities are not harmonised or standardised. Based on these monitoring activities a conclusion on the performance of the interviewers is given. This is the indicator of this quality control. Controls during data entry Different quality controls are made on the data collected or entered at the different stages of the data entry procedure by the interviewer within the dietary software program (for each interview). - Check on out of range values, empty food consumption occasions or easily forgotten foods. Controls on out-of-range values (e.g. date of birth, weight, height, portion sizes), controls on empty food consumption occasions or controls on easily forgotten foods attached to a given food item will pop up in the dietary software screens. These controls are performed by the interviewers/data entry operators. If possible, these items are corrected or a note of these errors is made. - Check missing quantities. Before ending the data entry, a screen summarizing the foods/recipes with missing (or overflow) quantities is displayed. These items concern all the foods/recipes reported for which the UNKNOWN quantification method has been used. These aspects are checked by the interviewer/data entrier. If possible these aspects are corrected a note of these errors is made. 11

12 - Check roughly energy and macronutrient intakes. A final control is then made at the nutrient level. This last quality control of the dietary interview compares the estimated energy and macronutrient intakes with standard requirements calculated for the age and sex of the subject. The DIETARY INTERVIEW SUMMARY AND FINAL CHECKS screen displays general information on the subject (sex, age, weight, et cetera) and the calculated energy requirement (Figure 1). The estimated energy and macronutrients (in grams and percentage of the energy intake) are reported at the top of the table. If energy or one of the macronutrient values (proteins, fats, carbohydrates, alcohol) estimated from the 24-hour food diary is out of the expected ranges, a warning TOO LOW or TOO HIGH is displayed. The complete list of the items sorted by groups/sub-groups, and their corresponding "as consumed" weights are reported in four main categories (breakfast, lunch, dinner and other) to check whether an error can explain out of range intake value. If so these errors are corrected. Figure 1 Example of a screen of a dietary software on Dietary interview summary and final checks step 3.4 Data cleaning of food consumption data A number of checks are not performed during the data collection/data gathering, but are performed in a later phase. The following aspects are checked and if necessary corrected in order to improve the quality of the data. 12

13 - Solving all notes. A dietician should check all notes. In the new version of EPIC-Soft the status of these notes are logged. Whether all notes are checked and solved if possible can be an indicator of this quality control. - Outliers. Check the food consumption data on outliers in food group consumption by age and gender groups. This should be done at food group level. An example of such procedure is available at RIVM. - Unknown quantities. All unknown quantities that remain after the recompute step within EPIC-Soft should be dealt with during the data cleaning. It is recommended to log these updates in order to be consistent. The principle here is to fill in a best estimate of the quantity either by using information from the respondent him/herself or if this is not applicable a mean quantity from the group. Furthermore, there is a specific check (available in the ES-manual) to identify and correct some errors in the collected data: Check heaped spoons. Within EPIC-Soft, it is possible to estimate quantities of some foods using heaped spoons. When this option is chosen for a thin liquid food item, this is not correct. These errors can be corrected during the data-cleaning phase. 3.5 Data interpretations of food consumption data Low and high-energy reporters For the interpretation of the quality of the survey data, it is necessary to consider the possible influence of misreporting on the conclusions drawn, and to get insight in the contribution of diet reports with poor validity. This includes for example diet records in which food items consciously or unconsciously are omitted. But also diet records of dieting persons. In addition, over-reporting /over-eating have been shown to give some systematic bias. At the group level the Goldberg cut-off can be used to determine the probable degree of overall bias to reported energy intake in the subgroups of the survey. With its limitations is can also be applied at the individual level, however this method fails to identify underreporters among those with high-energy requirements. The protocol for estimating the degree of low and high-energy reporters is as follows: - Each food item should be linked to food composition data. - Calculate for each individual the reported energy intake by foods in KJ/day (EI rep ). In case of the pilot in PANCAKE the information on food composition is based on the row available information on macronutrients within EPIC-SOFT. - For children a part of the energy will be used for the growth metabolism. Therefore from the reported energy intake (EI rep) the accretion factor (AF) should be subtracted. This part of the energy intake can be used for the energy expenditure (EE). The accretion factor is given in Table 2, based on Black Table 2 Accretion costs in MJ/day 13

14 Age Accretion factor / boys Accretion factor / girls 0-2 months 0,5 0,5 3-5 months 0,3 0, months 0,1 0, ,05 0, ,06 0, ,12 0, ,11 0,07 - As the absolute energy requirements of individuals vary with age, sex and body size and the basal metabolic rate (BMR) also varies with these factors, energy requirements are often stated as multiples of BMR. This expression of energy requirements ((EI rep AF):BMR) is called PAL (physical activity level). o (eq 1) EE:BMR = PAL - At the group level we assume that the energy expenditure is equal to the energy intake minus the accretion. This means that the equation can be rewritten as: o (eq 2 ) (EI rep AF):BMR = PAL - The Basal Metabolic Rate (BMR) for each individual can be estimated using the formula of Schofield (Schofield 1985). o (eq 3) BMR = a + b * body weight. See table 3 for the values of a and b. Table 3. Calculation of BMR in MJ/d BMR = a + b * body weight a b Boys 0 to 2 years - 0,13 0, to 9 years + 2,11 0, to 17 years + 2,75 0,0740 Girls/women 0 to 2 years - 0,13 0, to 9 years + 2,03 0, to 17 years + 2,90 0, to 29 years + 2,08 0, to 59 years + 3,47 0,

15 - Estimate for each individual the energy ratio (eq 2= ratio of the (reported energy intake accretions) / BMR). - Compare the mean ratio for each age-sex stratum with the expected PAL values for the specific population. Ideally, for each subpopulation appropriate values should be available. In order to compare these values in the pan-european survey, the following PAL-values will be used (based on Black 1996 and Torun) 1 Table 4. Pal values Black 2000 Torun 1996 Boys 1 to 6 years to 12 years Girls/women 1 to 6 years to 12 years to 17 years to 29 years to 39 years As there are errors in the measurement of all elements of this equation, confidence intervals of the ratio should be estimated. See for details in manuscript from Black. (Black, 2000). To conclude whether a mean value is acceptable the following should be satisfied: o o o EI-AF:BMR > PAL * exp [SD min * (S/100): n] and EI-AF:BMR < PAL * exp [SD max * (S/100): n] In which S = ((CV 2 w: : d )+ CV 2 wb + CV 2 tp) If this is the case, the quality indicator is positive. - Ideally, for each subgroup the appropriate values will be used. However, these should be available. In order to standardise between the European countries, the following values are suggested. o Between subject variation in physical activity (CV tp )= 15% o Within subject daily variation in energy intake (CV w) )= 23% o Variation in metabolic rate (CV bw) ) =8.5% 1 Torun B, Davies PSW, Livingstone MBE, e.a. Energy requirements and dietary energy recommendations for children and adolescents 1 to 18 years old. Eur J Clin Nutr 1996; 50 (suppl 1): S37-81.): 1 15

16 - Estimate the percentage of subjects with a lower energy ratio as expected using Goldberg cut-off levels 2 specific for the number or record days. These are the low-energy reporters. However, this number will not identify under-reporters among those with high-energy requirements. - Estimate the percentage of subjects with a higher energy ratio as expected using Goldberg cut-off levels 1 specific for the number of record days. These are the high-energy reporters. The numbers of high and low energy reporters are the indicators for this quality control. 3.6 Anthropometry For height and weight measurements, the standard protocol for the assessment of anthropometric values and quality check of the devices (see protocol M) should be followed. 3.7 General characteristics Within the data entry forms developed for the pan-european survey among toddlers, children and breastfeeding women, several measures are taken in order to improve the data quality: Most attributes are mandatory. If a field is left empty an error message will pop-up at the moment the records is stored. E.g. All attributes with limited set of allowable values are implemented in the way that the user can choose the appropriate value from a list. Values that are not on the list are not accepted by the tool. 2 Goldberg GR, Black AE, Jebb SA, Cole TJ, Murgatroyd PR, Coward WA, Prentice AM. Critical evaluation of energy intake data using fundamental principles of energy physiology: 1. Derivation of cut-off limits to identify under-recording. Eur J Clin Nutr Dec;45(12):

17 All internal consistencies are checked during data entry. E.g. it is not possible to enter data on supplement usage (FPQ Question 7) if at Question 6 Did your child use vitamin tablets, the answer is No (value 1) Whether these data entry forms are used is an indicator of these quality controls. - Check in the final data whether the mandatory variables are missing. The number of missings in the mandatory variables divided by the number of participants is an indicator of this quality control. - Check the final data on internal consistencies (same controls as used in the data entry forms). The number of internal inconsistencies divided by the number of participants is the indicator of this quality control. 3.8 Inconsistencies between food consumption data of 2-3 days, general questionnaire and general information - Check on completeness of all data for all subjects. Compare list of subjects in the different data files. Subjects with incomplete data, such as only one day, or no general characteristics will be excluded. - Check on inconsistencies on birth dates, gender. If possible delete incorrect information. - For some aspects, inconsistencies can occur. For instance age, weight or height can differ during the datacollection. The preferred values should be indicated. - % of subjects with inconsistent data in the final database the indicator for this quality aspect. 4. Scoring indicators of quality control The information about all these indicators of quality control can be summarized in a scoring list per survey. This scoring list can be found in annex c of this protocol. 17

18 Annex A Example of personnel qualifications o Recommended qualifications for interviewer (example) BSc Nutrition or Dietetics good knowledge of foods motivation communicating skills good computer skills accurate having the possibility to use own car with navigation available on Monday to Saturday. flexible and persistent available for minimal 2 home visits a week 18

19 Annex B-1. monitoring performance of the dieticians on recruitment. In evaluating the performance of the dieticians, two aspects can be distinguished: (a) are the individuals contacted according to the guidelines, (b) is the dietician able to convince the individuals to participate in the survey and to make them complete the diaries successfully. The following indicators that can be constructed from the Registration form for contact attempts (RF) and from the Individual participation form (IF) will be used to evaluate the performance of a dietician. Completed individuals I 1 : the proportion of individuals who participated (RF) (2*1dFR and 3dFR) I 2 : the proportion of individuals who completed participation (IF) (2*1dFR and 3dFR) I 3a : the proportion of individuals who completed participation and on correct days (IF) (2*1dFR and 3dFR) I 3b : the proportion of individuals who completed participation and on correct days (IF), and with 2 weeks interval for completion of the diaries (in case of 2*1dFR). Not participating individuals I 4 : the proportion of refusals (RF) I 5 : the proportion of non-contactable individuals (RF) I 6 : the proportion of non-eligible individuals (RF) Stand by individuals I 7 : the proportion of stand-by individuals (RF) For these calculations, the number of activated individuals that should be contacted by the dietician will be used as the denominator except I 2 and I 3a and I 3b for which the number of individuals with a final status is used (= number of individual forms received by the secretariat). For each dietician values for these indicators can be calculated. For I 4 and I 5 extremes can be detected by using tools like deciles, box plots, etc. In addition, the following indicators will be calculated to allow us to evaluate the performance of each dietician separately. I 8 : number of individuals classified as non-contactable, while the procedure was not properly applied. I 9 : number of individuals who are not contacted at the end of the first two weekly interval. This information will also be presented as a proportion: the proportion of the activated individuals for a specific period, which were not contacted within two weeks. I 10 : (binary) Indicator whether or not one third of the quota is reached half way the pilot study. The first time that the guidelines are violated, so when I 8 = 1 or I 9 =1, a letter will be sent to the dietician to inform about the violation and to ask to correct the mistake. In case both indicators I 8 and I 9 equal one, or when one of the indicators is larger than 1, the central secretariat will contact the dietician by phone to find out what is going on. 19

20 Halfway the study, all the dieticians with I 10 = 0 will be contacted by phone to motivate them to speed up the fieldwork so that at the end of pilot study the quota can be reached. The results should be presented bi-weekly in table. In the table, the ratio refusal/ (refusals + responders) and completed / target will also be displayed. It is foreseen to recontact a sub-sample of all individuals who were invited to participate to the survey (eligible individuals contactable or not and participating or not in the survey). For each active dietician, 1 individual having participated to the 2*1dFR, 1 individual having participated to the 3dFR and 1 individual having not participated (refusal, non eligible or non contactable) will be recontacted. Those persons will be recontacted by phone (it is asked to the dieticians to record the phone number of each person during their contact attempts (Registration form for contact attempts). Verification involves calling the respondent by telephone and asking 2-3 questions from the questionnaire and some questions on the process. 20

21 Annex B-2. monitoring performance of the dieticians on the planning of the fieldwork To evaluate the quality of the interviewers on the planning of the fieldwork we will look at transgressions regarding the contact procedures: The number of individuals who are not yet contacted at the end of the first two weekly interval, and the proportion with respect to the number of activated individuals. The number of individuals classified as non-contactable, but for which the procedure of contacting was not correctly applied, and the proportion with respect to the total number of non-contactable individuals. The distribution of the record days for 2*1dFR and 3dFR. These tables can be reconstructed from the individual participation forms. The performance of the dieticians is also reflected in the progress of the fieldwork. Therefore indicators, similar to the ones used to evaluate the performance of the dieticians are used to evaluate the progress of the fieldwork. Every week, the following indicators will be presented, based on all data collected since the start of the survey. Completed individuals I 1 : the proportion of individuals who participated (RF) (2*1dFR and 3dFR) I 2 : the proportion of individuals who completed participation (IF) (2*1dFR and 3dFR) I 3a : the proportion of individuals completed participation and on correct days (IF) (2*1dFR and 3dFR) I 3b : the proportion of individuals completed participation and on correct days (IF), and with 2-8 weeks interval for completion of the diaries (in case of 2*1dFR). Not participating individuals I 4 : the proportion of refusals (RF) I 5 : the proportion of non-contactable individuals (RF) I 6 : the proportion of non-eligible individuals (RF) Stand by individuals I 7 : the proportion of stand-by individuals (RF) The results should be presented weekly in a table (if possible). In the table, the ratio refusal/ (refusals + responders) and completed / target will also be displayed. 21

22 Annex C. Scoring list Quality. Quality control Indicator of quality Result List tasks, competences, responsibilities of all personnel Overview of tasks, competences, responsibilities available? + = yes; = no Equip qualified personnel Personnel qualified? + = yes; = no Organize communication between different bodies/members Select potential participants following sampling protocol Communication organized: + = yes; = no Guidelines sampling protocol followed? + = yes; = no Monitor response across target groups Data collection monitored and if necessary adapted on this aspect during study + = yes; - = no. Indicate non-response rate % of indicate % of non contactable and % of non eligible and % of refusing individuals by age-gender stratum Get insight in Representativeness (age, gender, district, and ses) Qualitative indication of representativeness + = high; = low, namely not for Weigh results for nonrepresentative distributions across socio-demographic groups* Get insight in distribution of Record day In case of unequal distributions, Weighing factor is used: + = yes or not necessary; = no Days equally distributed + = 14,3 % for each day (between 10,0-18,0%) - = not equally distributed Get Insight in distribution of Season Days equally distributed + = 25% (between 20-30%) 22

23 Quality control Indicator of quality Result - = not equally distributed Weigh for inequalities in day /season If necessary, weighing factor is used + = yes, or not necessary - = no. Get insight in time period between record days in case of 2 * 1 day diary Train national coordinators on the dietary software to be used % of records with more than 14 days between both record days National coordinators trained on the dietary software? + = yes; - = no Train interviewers Interviewers trained according to training protocol? + = yes; - = no Keep training program for interviewers data entry staff up to date Monitor performance of interviewers on recruitment Monitor performance of interviewers on progress in fieldwork Monitor performance of interviewers on quality of fieldwork Training program kept up to date? + = yes or not relevant; - = no Indicators mentioned in the protocol ( A b-1) Indicators mentioned in the protocol ( A b-2) Indication of type of performed quality controls and conclusion of its results Data entry Perform standard quality checks available in EPIC-Soft during data entry Check the outliers Standard Quality checks in Epicsoft performed? + = yes; - = no Outliers checked? + = yes; - = no Solve all notes* Solve other errors following the software manual % of checked notes Checks from software manual performed? + yes; - = no Evaluate the energy intake on poor validity (at group level)** Comparison of energy ratio with PAL 23

24 Quality control Indicator of quality Result Estimate % of high and low energy reporters** % of high and low energy reporters General data Limit the number of missings and inconsistencies Estimate the number of missings and inconsistencies Data entry forms developed for pan-european survey is used Number of missings and inconsistencies per total participants Anthropometry Follow protocol for anthropometric values Is the standard protocol for anthropometric values used? + = yes; - = no Quality control of devices* Are the devices calibrated each 6 months? + = yes; - = no Check on completeness of the data % of the participants in final study population with incomplete data Entry part of general data twice* % of double data entry 24