Data collection for the identification of emerging risks related to food and feed 1

Transcription

1 EFSA Journal 2011;9(8):EN-185 TECHNICAL REPORT OF EFSA Data collection for the identification of emerging risks related to food and feed 1 ABSTRACT European Food Safety Authority 2, 3 European Food Safety Authority (EFSA), Parma, Italy This report presents the results of an investigation performed by the working group on data collection for the identification of emerging risks related to food and feed (DACO WG). The WG proposed a procedure to identify, assess, rank and prioritize data sources - a two-step process based on the National Intelligence Model and the Dataquest approaches. This procedure consists of i) an initial classification of the identified data sources, followed by a monitoring period of the pre-selected data sources and ii) a consecutive and more detailed quality assessment of the relevant data sources for ranking and prioritization purposes. For this assessment, the WG defined text descriptors and quality parameters (i.e. link with indicators, data type, geographic and period coverage, language, edition, timeliness, accessibility, clarity and comparability). Then, as a pilot study, based on expert knowledge, the WG proposed data sources in the chemical, biological and nutritional areas, including research projects in the food area from the EU framework programme (n=188). These data sources were linked to eleven priority indicators (i.e. the ESCO indicators) and qualitatively assessed and profiled. These results allowed making conclusions and recommendations on a strategy for emerging risk identification (ERI). Such a strategy should encompass the selection of priority areas and the identification of appropriate indicators and associated data sources. Prospective studies, for example by running various risk scenarios, may help in defining such priorities. Close collaboration with relevant research projects, experts and stakeholders is a major condition for a successful ERI. The WG considered that monitoring a large number of data sources is inevitable and therefore recommended EFSA utilises software tools to support the collection and processing of this huge amount of information. Specific automated filtering processes may be designed that enable the selection of the most relevant signals for further analysis by experts. European Food Safety Authority, 2011 KEY WORDS Emerging risk, food, feed, data collection, data source, indicator, signal On request from EFSA, Question No EFSA-Q , issued on 24 th August 2011 Correspondence: emrisk@efsa.europa.eu Acknowledgement: EFSA wishes to thank the members of the data collection working group for the identification of emerging risks related to food and feed (DACO): Wayne Anderson, Jacqueline Castenmiller, Ines Golly, Gro Skøien Johannessen, Bruno Le Bizec, Alberto Mantovani, Hans Marvin (Chair), Philip Randles, Chris Rodgers, Anne-Sophie Roy, David Spurgeon (Rapporteur) and Mieke Uyttendaele for the preparatory work on this scientific output and the hearing experts: Victor Del Rio Vilas, Terry Donohoe and Jonathan Rushton, and EFSA staff: Agnès Rortais for the support provided to this scientific output. European Food Safety Authority; Data collection for the identification of emerging risks related to food and feed. EFSA Journal 2011; 9(8):EN-185. [52 pp.] doi: /j.efsa.2011.en-185. Available online: European Food Safety Authority, 2011

2 SUMMARY The European Food Safety Agency (EFSA) has started to develop an effective and transparent approach to identify emerging risks in the food and feed chain. This approach relies on three fundamental steps, namely 1) data collection 2) data analysis, signal detection and filtering, and 3) exchange of information. The establishment of a structured data collection system has been identified by the EFSA scientific cooperation working group (ESCO WG) on Emerging Risks as an important next step in the implementation of a fully operational capacity for emerging risk identification (ERI). To support this development, EFSA s Emerging Risks unit (EMRISK) initiated a working group on data collection for the identification of emerging risks related to food and feed (DACO WG). The WG consisted of a multi-disciplinary team of 15 experts having a broad expertise in food safety, including ERI. The main tasks of this WG were to support EMRISK in defining a list of priority sources of information and suitable strategies and tools to gather relevant signals indicative of emerging risks. The DACO WG recognized that potential emerging food and feed safety hazards or risks and related data sources are numerous. Therefore, to achieve a maximal and sustainable contribution to the development of a system to identify (any) emerging food and feed safety risks, the members of the DACO WG, in consultation with EMRISK, decided to focus on approaches to identify and assess data sources related to emerging food and feed safety risks. It was further decided to build on the results from previous activities of EFSA in this field and to use the eleven priority indicators identified by the ESCO WG. Hence, the DACO WG aimed to determine practical approaches for the identification and quality assessment of data sources, as well as conduct an initial characterisation of these approaches. To achieve its objectives the DACO WG identified the need for a two pronged approach. The first aspect of the work of the DACO WG focused on the identification of generic approaches that could be used for the assessment of present and future data sources. The second part focused on the use of expert judgement to compile an initial list of data sources, including those related to on-going research projects under EU Framework Programs (FP) 6 and 7. By applying a data source characterisation technique to the initial list of potential data sources, an initial critique of the value of both the assessment approaches and data sources was achieved. The assessment of the various data sources complied by the WG highlighted the need for a closer collaboration with relevant research projects, experts and stakeholders. Among the 165 EU FP projects screened by the WG, 145 could be tentatively linked to the eleven ESCO indicators by screening titles and abstracts, but only 23 projects were further selected when specific knowledge of the projects was applied. In addition to the FP projects, the WG proposed data sources related to the biological, chemical and nutritional ESCO indicators as a case-study. The majority of these data sources were indicator-specific (n=126) while a relatively still high proportion of these data were linked to more than one indicator (n=62). The description and qualitative assessment of these data sources showed that the ones related to the ESCO indicators 5 and 10, and to a lesser extent to the indicators 2, 4 and 9, were not well represented which pinpointed areas where additional sources could be gathered. The assessment of the FP projects also highlighted the lack of coverage for the ESCO indicator 5 (i.e. emergence of increased resistance to antimicrobials and plant protection products), which may be related to the lack of expertise within the WG in this scientific domain. Overall, the descriptors used for the assessment of the data sources showed that some data types were over- (e.g. research, national and international official data) or under- (e.g. stakeholders, consumption patterns, trade and national scientific committees) represented, that both the geographic and period coverage were relatively good (i.e. 50% of the data sources had a world-wide coverage and 80% were collated over periods greater than one year), that most data sources were accessible (13% were accessible at financial cost and 2% were not accessible), predominantly described in English (2% used alternative languages), contained processed data (84%) of sufficient or mixed comparability (76%), but that metadata was not often available (44% with unavailable metadata). Finally, the assessment of the timeliness of the selected sources to the ESCO indicators showed that indicators with a timeline of less than a week were the most abundant. EFSA Journal 2011;9(8):EN-185 2

3 Based on the activities performed by the DACO WG the following main conclusions and recommendations were made: to select the most appropriate data sources, a two-stage quality assessment should be performed. This includes (i) an initial classification of new data sources based on a modified National Intelligence Model (NIM) from the UK Food Standards Agency (FSA), and (ii) a detailed quality assessment of that source as its usefulness and significance to EMRISK becomes better understood. The detailed quality assessment stage should be based upon the Dataquest approach which is potentially quantifiable and can therefore also be used for ranking and prioritisation. The selection and quality assessment of data sources should be part of a predefined strategy of ERI. It should encompass the selection of priority areas and the identification of appropriate indicators and associated data sources. It was recognized that prospective studies, for example, by running various risk scenarios, may help in defining such priorities. Since emerging risk has the implicit character of being unexpected, proactive identification implies that numerous indicators and data sources should be monitored and evaluated simultaneously. To manage such an approach, the DACO WG recommends implementing specific software to collect and process this huge amount of information. A specific automated filtering process may be designed that enables the selection of the most relevant signals for further analysis by an expert and/or expert team. EFSA Journal 2011;9(8):EN-185 3

4 TABLE OF CONTENTS Abstract... 1 Summary... 2 Table of contents... 4 Background as provided by EFSA... 5 Terms of reference as provided by EFSA... 5 Evaluation Introduction Introduction to the area of emerging risks Introduction to signals and indicators Introduction to data sources Objectives of the DACO working group Materials and Methods Establishment and composition of the working group Identification of data sources and linking to the eleven ESCO indicators (expert judgement) Data sources from the EFSA colloquium on emerging risks Approaches to assess the quality of data sources The FSA approach based on the National Intelligence Model (NIM) The Dataquest project approach The Multi Criteria Decision Analysis (MCDA) approach Results and Discussion Identification of signals and data sources linked to the eleven ESCO indicators Linking EU Framework Programme projects to the ESCO indicators Approaches and criteria proposed by the DACO working group for the assessment of data sources Proposed approaches for the assessment of data sources Qualitative profiling of data sources Overview Descriptor parameters Quality parameters Conclusions and recommendations References Appendices A. Data sources and signals proposed by the DACO working group linked to the eleven ESCO indicators and organised by type as defined at the EFSA colloquium (EFSA, 2011c) B. FP7 projects in the food area linked to the ESCO indicators (EFSA, 2009) C. FP6 projects in the food area linked to the ESCO indicators (EFSA, 2009) D. Definition of suggested metadata and quality parameters required for each data source E. Characterisation of the data sources identified by the DACO working group Glossary and abbreviations EFSA Journal 2011;9(8):EN-185 4

5 BACKGROUND AS PROVIDED BY EFSA According to EFSA s Founding Regulation 4, the Authority is required to undertake action to identify and characterise emerging risks in the field of food and feed safety. The Emerging Risks unit (EMRISK) contributes to this mission by supporting the development of a capacity for the collection and analysis of data/information from different types of sources with a view to identifying future trends and developments in areas directly or indirectly related to food and feed safety. EFSA has started to execute its programme to develop an effective and transparent approach to identify emerging risks (ERs) (EFSA/SC/Tender/01/2004, 2006a). This consists of an operational definition of emerging risks and an overall strategy for the collection, analysis and evaluation of the relevant data and information (EFSA 2006b; EFSA 2007; EFSA 2009). The general procedure for emerging risk identification (ERI) that has previously been identified, relies on three fundamental steps, namely 1) data collection 2) data analysis, signal detection and filtering, and 3) exchange of information (EFSA, 2010). The further development of a structured data collection system has been identified as a crucial next step in the development of a fully operational capacity for emerging risk identification. The implementation of such a system will have to take into account the extreme variability in the type and quality of the available information, the vast number of sources of potential interest, and the resources and expertise available in EFSA or through external experts. Thus, it was proposed to establish a working group on data collection (DACO) for the Identification of Emerging Risks related to Food and Feed. TERMS OF REFERENCE AS PROVIDED BY EFSA Within the framework of the general procedure for emerging risk identification, the main task of the working group (WG) was to support EMRISK in the establishment of a system for the collection of data considered relevant for the identification of emerging risks. Specifically, the tasks of the WG were to: Identify the type and quality of data needed (e.g. structured and unstructured, quantitative/qualitative, surveillance data, expert opinion, recent research findings, etc.); Define and list priority sources of information considered relevant for the type and quality of data needed; Indicate most suitable methods and strategies for data collection (e.g. consultation/survey with experts, call for data, ad hoc studies, IT tools, inventories, etc.) and their strengths and weaknesses, taking into account the limited resources available to EFSA; Provide recommendations for the implementation of the data collection framework defined. Timeline It was foreseen to establish the WG and to have the first meeting in April Expected deliverables By May 2011, the WG would be responsible for issuing a technical report on: The identification and characterization of the type and quality of data needed, and a priority list of sources of information to be deployed; Strategies to retrieve the different types of information identified. Publication of the report 4 Article 23(f) Reg. 178/2002/EC EFSA Journal 2011;9(8):EN-185 5

6 It was proposed to publish the report as soon as finalized and approved. EVALUATION 1. INTRODUCTION In the process of the collection of data sources for the identification of emerging risks, the terms emerging risks, indicators and signals need to be clearly defined. These terms were described by EFSA and have been used throughout this report: Emerging risk: an emerging risk to human, animal and/or plant health is understood as a risk resulting from a newly identified hazard to which a significant exposure may occur or from an unexpected new or increased significant exposure and/or susceptibility to a known hazard. An assessment of emerging risk is characterized by the early detection of facts related to that risk derived from research and/or from monitoring programmes or episodic observations (EFSA, 2007). Indicator: the evidence for the identification of an emerging risk should preferably be in the form of an indicator (measurement and/or observation) and of a trend over time and space. It may point to a specific emerging risk in different ways, either directly or indirectly related to the food chain. It should be reliable, sensitive, and quantifiable and should provide the information on the nature of the hazard (agent/process involved) and the source of the risk (EFSA, 2007). Signal: is identified as a temporal or spatial trend of an indicator value (EFSA, 2007). Drivers: the DACO working group identified drivers as issues which shape the development of emerging risks to food and feed safety. One important characteristic of drivers is that they may act as modifiers of effect on the onset of emerging risks, namely they can either amplify or attenuate the magnitude or frequency of risks arising from various sources (EFSA 2011a). Drivers can be classified in categories such as those represented by the analysis PESTLE standing for Political, Economic, Social, Technological, Legislative and Environmental analysis. The DACO working group has interpreted the term Data Source as meaning information of a range of types of data sources (published peer reviewed and other literature, databases, websites, newsfeeds, opinions, blogs, etc.) that provide information and intelligence that can support the identification of an emerging risk. The basis of an ER system was described by EFSA s definitions on emerging risks (EFSA, 2007) followed by publications on overall strategy and the framework of ERI, including data collection, data analysis and evaluation and information sharing, such as networking (EFSA, 2009, 2010) Introduction to the area of emerging risks The global dimensions and increased complexity of food and feed supply chains nowadays make them more vulnerable to the occurrence of food and feed safety hazards in products meant for human consumption. It is apparent that food and feed safety risks can be reduced if food and feed safety hazards are identified at an early stage. Marvin et al. (2009) have recently evaluated a number of worldwide early warning systems and classified them in three categories, namely: i) Reactive systems which are endpoint- or hazard-focused and which are designed to monitor the food and feed safety hazard or human-, animal- and plant diseases (e.g. Enter-Net, EPPO, GIDEON, GOARN, INFOSAN, OIE, PulseNet International, ProMED, RASFF, WAHIS), EFSA Journal 2011;9(8):EN-185 6

7 ii) iii) Proactive predictive systems for known hazards, such as those systems developed for prediction of mycotoxin contamination of wheat and maize (e.g. DONcast) or broadlyorientated information collecting systems such as GIEWS and EMM, and Holistic 5 systems, which also take into account developments outside the food and feed supply chain that may eventually lead to the emergence of risks within this chain (e.g. EPICA, Futures studies, ERDSS). The vast majority of early warning systems are reactive systems that collect, analyze and interpret data from running hazard or disease monitoring and surveillance programmes. A typical example of a hazards-based system is the European Rapid Alert System for Food and Feed (RASFF). RASFF is a system that enables the exchange of information on unsafe foods and feeds between EU Member States and the European Commission (EC) and has been in place since RASFF s current legal basis is laid in the article 50 of the Commission Regulation No 178/ and more recently implemented in Commission Regulation No 16/ Various types of food and feed safety hazards are reported to RASFF, including chemical, microbial and physical hazards, and this information is used by its members to take proper preventive measures, if appropriate. Weekly summaries and annual reports from RASFF are published on the EC s website ( RASFF, in common with other reactive systems, provide information on food and feed safety problems after they have occurred, making all interventions, by definition, corrective. It is evident that systems that detect signals of potential food and feed safety risks at an earlier stage, ideally before they have developed (often referred to as emerging risks systems) are preferable, since these would allow authorities to take proactive preventive measures. Development of emerging risk systems falls within the competence of the European Food Safety Authority (EFSA) 6, which has set up a dedicated Emerging Risks unit, EMRISK, supporting the development of an operational capacity to identify emerging risks, the first instance, in food and feed. To be able to identify emerging food and feed safety risks before they occur or at an early stage of development may be best achieved using a holistic 8, interdisciplinary approach. Such an approach is based on the collection, analysis and evaluation of signals directly and indirectly related to a hazard collected from both within and beyond the food and feed supply chain. Indeed, analysis of recent food and feed incidents has revealed that a more detailed analysis of the significance of such signals (i.e. changes in indicators) would have placed relevant authorities in a better position to identify these past food and feed safety problems at an earlier stage (Hagenaars et al., 2006; Kleter et al., 2009). For example, early identification of the uncertainties concerning the potential safety of nanotechnology products has allowed authorities to appraise the suitability of existing risk analysis procedures for nanotechnologies of relevance to the food and feed sectors and, where appropriate, to commission research to quantify better the potential for exposure and health effects Introduction to signals and indicators A successful identification of emerging risks depends on the collection of data and information from sources related to one or more indicators that, once processed, should deliver a signal of an emerging risk. Therefore, it is important to define and list priority indicators considered relevant for the 6 Regulation (EC) No 178/2002 of the European Parliament and of the Council of 28 January 2002 laying down the general principles and requirements of food law, establishing the European Food Safety Authority and laying down procedures in matters of food safety (OJ L p1-24). 7 Regulation (EC) No 16/2011 of the European Parliament and the Council of 10 January 2011 laying down implementing measures for the Rapid alert system for food and feed (Text with EEA-relevance) Official Journal L 6/7, , p Definition of holistic : Explanations that attempt to explain complex phenomena in terms of the properties of the system as a whole from Henderson s Dictionary of Biological Terms (Ed: E. Lawrence, 11 th Edition, Longman Group Ltd., Harlow, 693 pp). EFSA Journal 2011;9(8):EN-185 7

8 identification of emerging risks. Based on the EFSA definitions for emerging risks, indicators and signals (EFSA, 2007), the EFSA scientific cooperation working group (ESCO WG) has identified eleven priority indicators using selected criteria such as relevance for EFSA, influential sectors, proximity to the food and feed chain, measurability and interpretability and potential impacts on public health (EFSA, 2009). The DACO WG used this priority list as a basis to develop a system for the collection of data considered relevant for the identification of emerging risks. The eleven priority indicators as defined by the ESCO WG are: 1. New research data indicating toxic properties of substances possibly occurring in food or feed, 2. Unexpected detection in food or feed of a potentially toxic/radioactive chemical, 3. Emergence of new zoonotic and other foodborne pathogens, 4. Emergence of a new or exotic biological agent pathogenic to animals, 5. Emergence of increased resistance to antimicrobials and plant protection products, 6. Emergence of a new or exotic biological agent pathogenic to plants/food/feed crops, 7. Increased virulence of known pathogens (including plant pathogens), 8. Unexpected evidence of increased exposure of specified human (sub)populations to particular hazardous chemical/biological/radioactive contaminants and other agents in the food/feed chain, 9. Unexpected evidence of increased exposure of specified animal (sub)populations to particular hazardous chemical/biological/radioactive contaminants and other agents through food, 10. Unexpected increased susceptibility of (sub)populations to known contaminants and other hazardous substances not regulated in the food/feed chain, 11. Dietary changes or imbalances caused by modification of food consumption habits Introduction to data sources Food and feed supply chains are susceptible to the effect of a large number of influencing factors (e.g. drivers) that may give rise to the development of new food and feed safety problems or risks. Many drivers have been identified that influence agricultural systems such as transport, food processing, retailing, market demand and economics (Noteborn and Ooms, 2005; EFSA, 2011a). However, prediction of their effects on food and feed safety is not without challenges and will be characterised by different degrees of uncertainty. Hence, changes in indicators (e.g. signals) arising from data sources that are connected to these factors will need to be interpreted for use in emerging risks identification. A better characterised relation between an emerging risk and a hazard is available in the eleven prioritised indicators for the chemical, biological and nutritional hazards that have been selected by the ESCO WG on Emerging Risks (EFSA, 2009). This inventory (see section 1.2) included (limited) information on signals and data sources. A preliminary list of the main data and information sources related to the eleven priority indicators was identified. These data sources were clustered into four types, namely: i) soft including media and grey literature, ii) regulatory including data from rapid alert systems and compulsory reporting or monitoring of foodborne illness, EFSA Journal 2011;9(8):EN-185 8

9 iii) iv) scientific including published papers, proceedings, research findings and documented reports, and expert judgement. It is clear that one indicator may encompass many signals and also that many types of data sources can provide information on one signal. Furthermore, the data sources may vary considerably in quality and their usefulness for the identification of emerging risks. In a recent EFSA colloquium (EFSA, 2011b), key issues related to the identification of emerging risks in the food and feed chain were debated by a number of international experts working in different sectors. The colloquium participants distinguished (without ranking) eight types of data sources based on their anticipated differences in quality or uncertainty (EFSA, 2011c) as follow: 1. National and international official data, e.g. regulatory surveillance data, various data from food safety agencies or ministries of public health and agriculture: meat hygiene data (import/export inspections), food monitoring data, health surveillance data, and animal health surveillance data, 2. Data from various EU agencies; there is a need to ensure linkages between food-, human-, animal-, plant health- and environmental agencies, 3. Data from consumption patterns (national nutrition surveys), consumer behaviour surveys, information on food consumption in susceptible populations, 4. Research data including conference proceedings, preliminary results, scientific literature, research work at regional, national, EU and international levels, 5. Trade data and market research (at regional, national, EU and international levels), 6. Data from the media (newspapers, internet), 7. Data from the EFSA scientific experts and national scientific committees and/or research coordinators, 8. Data from stakeholders including industry, primary production and non-governmental organisations, and consumers Objectives of the DACO working group The main objective of the DACO WG, as given in the terms of references, was to support EFSA, particularly the EMRISK unit, in the establishment of a system for the collection of data considered relevant for the identification of (re-)emerging risks. Crucial for the success of an ER system are: i) The identification, characterisation and prioritization of relevant data sources that provide information on a signal for an indicator, and ii) The identification and characterisation of efficient methods for collecting and analysing this information in a structured manner that highlight trends in the patterns and the extent of a potential risk (EFSA, 2007). The first point falls into the remit of the DACO WG and will be reported here, whereas the second point will be addressed by the newly established EFSA WG on methodology (M ), although the results of the DACO WG will be a useful starting point for the methodology WG. EFSA Journal 2011;9(8):EN-185 9

10 Potential emerging food and feed safety hazards, risks and related data sources are numerous. To achieve a maximal and sustainable contribution to the development of a system to identify (any) emerging food and feed safety risks, the members of the DACO WG, in consultation with EMRISK, decided to focus on the approaches to identify and assess data sources related to emerging food and feed safety risks. It was decided to build on the results obtained by the ESCO WG (EFSA, 2009) and to use the eleven priority indicators identified by the ESCO WG for this purpose. Additionally, the outcome of the ESFA 15 th scientific colloquium on emerging risks in food: from identification to communication (EFSA, 2011b) was also considered in the appraisal. The DACO WG aimed to determine practical approaches for the identification and quality assessment of data sources, as well as conduct an initial characterisation of these methods. The DACO WG did not, however, seek to make an inventory and produce an exhaustive list of data sources as part of this exercise. The DACO WG concluded that a temporal monitoring of the signal(s) from these data sources should be part of the quality assessment in order to determine the usefulness of the data sources. However, given the time available and the large number of relevant data sources identified, such an exercise could not be performed. By focusing on methods and building on previous exercises in this field, the WG endeavoured to provide EFSA with tools and approaches to extend and improve their newly established operational system for ERI. 2. MATERIALS AND METHODS Convening of the DACO WG began by identifying the range of expertise that needed to be included to ensure that the group had the skills available to meet the objectives of the task. The first meeting of the DACO WG identified the need for a two-pronged approach. One aspect would focus on the identification of approaches that could be used for data source assessment. It was agreed that the identified approaches should be generic, meaning those that could be applied to a range of present and future data sources. As a result, three methods with suitable characteristics were identified. The second aspect of the work focused on the use of expert judgement to compile an initial list of data sources, including those related to ongoing research projects under EU Framework Programmes (FP) 6 and 7. By applying a data source characterisation technique to the initial list of potential data sources, an initial critique of the value of both the assessment approaches and data sources could be achieved. This approach is summarized in Figure Establishment and composition of the working group To support EFSA in the establishment of a system for the collection of data considered relevant for the identification of emerging risks it was important to involve experts with knowledge of food and feed safety and any possible hazards. Experts were sought having a broad expertise, including indirectly some of the topics mentioned above. It was aimed to cover as many research fields as possible that were linked (directly and indirectly) with food and feed safety. In this way, the DACO WG was composed of 15 experts having the expertise identified in Table 1. Although food and feed safety risks may also be driven by factors, such as human behaviour and demographic trends (Kleter and Marvin, 2009; EFSA, 2011a) and such factors should be taken into consideration when developing emerging risks systems, it was not deemed necessary in this case to appoint a specific expert on these topics for the exercise that was undertaken. EFSA Journal 2011;9(8):EN

11 Figure 1: Data sources collection and assessment process conducted by EMRISK and the DACO- WG. EFSA Journal 2011;9(8):EN

12 Table 1: Description of the scientific expertise available in the DACO WG. Scientific expertise Agricultural economics, animal and plant health, biochemical pharmacology, biochemistry, biotechnology, botanicals, drug, feed and food additive safety, chemical residue monitoring, economics of animal health, ecology, ecotoxicology, epidemiology, fish and shellfish microbiology and diseases, food hygiene and analysis, food safety, human health, human nutrition, livestock production, microbiology, nanosafety, novel foods/ingredients, nutrition medicine, pharmacology, toxicology Systems expertise Early warning systems, emerging risks systems, exposure assessment, horizon scanning, quality systems, risk assessment, risk analysis, The selection of experts was performed by EMRISK in consultation with the DACO WG chair, in accordance with EFSA standard operating procedures. Care was taken to cover all possible hazard fields and to include experience in risk assessment and emerging risk analysis. Many experts were involved in international ongoing projects on risk assessment related to emerging risks identification Identification of data sources and linking to the eleven ESCO indicators (expert judgement) To develop a good basis on emerging risks and to cover the DACO mandate, relevant background documents were provided and several discussion sessions were organised on issues and definitions related to emerging risks (such as indicators, signals, data sources, etc.). Subsequently, the experts were asked, based on their expert judgement, to formulate signals related to one or more ESCO indicator(s) and to provide data sources that could deliver information for a particular signal. Development of the list of signals and data sources was performed following an iterative process. One specific type of data source that may provide signals for emerging risks is represented by research projects (national and international), including projects funded within the EU Framework Programmes (FP). The finding of new antibiotic resistances, new diseases, new exposure routes or other changes with potential to impact on the food and feed chain within a research project may lead to the establishment of the potential for or actual identification or development of a new emerging hazard or risk. It was therefore felt to be important to identify at their outset those EU FP projects (and other research programmes or individual research projects) that may deliver such information. Both the ESCO and DACO WGs recognised this potential and therefore the DACO WG performed a scoping exercise to investigate how this potential could be exploited. The DACO WG performed an exercise to link a limited number of EU FP projects (74 FP6 and 91 FP7; 165 in total) to the eleven ESCO indicators for which they may have been able to provide information or signals. Only research projects from FP 6 and 7 from Calls 3 and 4 in Activity 2 - Food area were analysed by the DACO WG experts in order to assess their potential to deliver information on the eleven ESCO indicators. This activity was carried out using the title and project abstract overview and therefore should be seen as a limited scoping exercise rather than one designed to develop a comprehensive inventory Data sources from the EFSA colloquium on emerging risks On October 2010, EFSA organised a scientific colloquium on emerging risks in food: from identification to communication (EFSA, 2011b). The scope of this colloquium was to bring together international experts including DACO WG members for an open scientific debate on the key issues EFSA Journal 2011;9(8):EN

13 related to the identification of emerging risks in the food and feed areas, aiming at providing practical inputs for the development of the EFSA s methodological framework for ERI. Furthermore, the results of discussion group 2 (EFSA, 2011c), which was chaired by a DACO WG member, was presented to the other WG members at a subsequent meeting. During the ensuing DACO WG discussions the results of the colloquium were considered, especially in relation to the usefulness of the data types for a first grouping/classification of the DACO data sources (see section Results & Discussion). Eight types of data sources for the identification of emerging risks were compiled at the colloquium (see section 1.3) stressing the fact that the quality of data and data gaps will always need to be considered. It was noted that data and data collection should be preferably harmonised (to the extent possible), timely and categorised for optimal use. It was mentioned that unstructured data, e.g. media and internet discussion rooms, can be a useful trigger to further investigations. Furthermore it was expressed that building trust between partners is essential in the exchange of information. Finally linking of indicators and data sources was established to be crucial but challenging as the idea is to use and to add value to data not generated for the purpose of identifying emerging issues Approaches to assess the quality of data sources It is inevitable that a large number of data sources will be identified that deliver some kind of information about the development or occurrence of an emerging risk and that large variations will exist in the quality of the data provided. To gauge the value of a data source, it is important to have knowledge about its quality. The WG identified three different approaches to achieve such an assessment, namely: i) The approach from the UK Food Standards Agency (FSA) which is based on the National Intelligence Model (NIM), ii) iii) The Dataquest model approach, The multi-criteria decision analysis (MCDA) approach. These approaches were considered for their applicability to assess the quality of data sources as identified by the WG. In the following section a brief description is given for the approaches identified The FSA approach based on the National Intelligence Model (NIM) The National Intelligence Model (NIM) (ACPO, 2005a, b) is the business model for police enforcement in the UK. It provides an intelligence-led proactive approach by identifying linkages between items of related data to determine patterns and trends in crime. The model assesses the credibility of any particular data source using a matrix approach which evaluates the reliability, legitimacy and accessibility of that source. The principles of the NIM have been adapted by the DACO WG to develop a rapid method for the initial classification of data sources. This suggested model assesses each data source by focusing on its relevance, validity and credibility using the matrix outlined in Table 2. The model is intended to classify a source rather than rank or prioritise it against other potential sources. This characteristic means that the model can be applied rapidly within any assessment, which makes the model particularly useful for synthesizing results across multiple and new sources. By using this matrix, it is possible to highlight the key attributes of a particular source, thereby identifying its potential suitability for providing signals relating to the selected indicators. The ability to generate such signals can then be monitored by assessing, for example, the number of food safety incidents occurring per month against a baseline of expectation generated from analysing historical data. A signal from this source would therefore be characterised by the exceedence of statistically determined thresholds set around the baseline. Techniques for initial assessment will vary in EFSA Journal 2011;9(8):EN

14 accordance with the type of data source being evaluated and can also include the identification and assessment of unexpected events or trends. EFSA Journal 2011;9(8):EN

15 Table 2: Data Source Classification Matrix for the FSA method based on the NIM and adapted for DACO. Relevance To food and feed safety Subject matter known to have direct relevance to food/feed safety To consumer media/perception of food/feed safety A B C D E Subject matter believed Subject matter believed Subject matter not to be relevant to not to be relevant to relevant to food/feed food/feed safety food/feed safety safety Subject matter known be perceived by the consumer/ media as a food safety concern Subject matter could potentially be perceived by the consumer/media as a food safety concern Indirectly relevant to selected indicator Subject matter unlikely to be perceived by the consumer/media as a food safety concern Subject matter not perceived by the consumer/media as a food safety concern Unknown relevance to food/feed safety Unknown consumer/ media reaction to subject matter To selected indicators Directly relevant to selected indicator Possible relevance to selected indicator No relevance to selected indicator Unknown relevance to selected indicator Number of similar data Unique source 1 similar source 2-5 similar sources Multiple sources Unknown number of sources sources Time between data Instantly available < 1 wk 1 wk - 1 mo 1-12 mo Random/unspecified availability and the event described by the source Accuracy Precise Approximate Estimated Imprecise Accuracy of source unknown Validity Selection bias Known to be Believed to be Believed to be biased Known to be biased Unknown bias in source unbiased unbiased data Measurement bias Known to be Believed to be Believed to be biased Known to be biased Unknown bias in source unbiased unbiased data Sampling bias Known to be Believed to be Believed to be biased Known to be biased Unknown bias in source unbiased unbiased data Independent validation Source validated by Source validated by Source validated by Source not validated Validation status trusted 3 rd party unknown 3 rd party untrusted 3 rd party unknown Reliability Trusted source - Source believed to be Source believed to be Untrusted source - Reliability of source known to be reliable reliable unreliable known to be unreliable unknown Credibility Factual Source data known to Source data believed to Source data believed to Source data known to be Untested source be true be factual be false false or malicious Accessibility Fully accessible Accessible on request Restricted Not accessible Accessibility status unknown EFSA Journal 2011;9(8):EN

16 The Dataquest project approach Data sources on emerging risks The identification of emerging risks, requires supporting data which should be compatible with existing standards in order to enable cross-comparison and analysis. The Dublin Core metadata element set (also described under ISO Standard 15836:2009) is an agreed procedure for providing simple and generic resource descriptions (Dublin Core Metadata Initiative, 2010), whereas the European Statistical Service Dimensions for quality (Eurostat, 2005) provides an adaptable code of practice for a series of principles with potential application to more quantitative approaches for data source characterization. Data sets have to be of sufficient quality and precision and well described to enable uncertainty in any subsequent risk assessment to be measured. In addition, it is important that data is correct, well documented and reliable, which can only be achieved by using harmonised terminology and criteria/parameters. Such an approach helps to support the management of information across a common platform and is a way of applying benchmarking to a potentially complex situation. A determination of the quality of data sources should additionally consider data class parameters and specific criteria. This approach could help to identify any poor quality data and any key areas where data is currently missing that could help to support associated identification of emerging risks and relevant signals for the selected indicators. This approach is detailed in the Dataquest report (Rodgers et al., 2011), from which the DACO WG used the quality assessment approach to conduct an initial (non-exhaustive) source assessment. According to the Dataquest approach, once identified, each data source should be subjected to a quality assessment. The main parameters considered are included in sections using a basic (text) descriptor and another on quality (see Table 3). This approach used specific parameters related to the Dublin Core standards for metadata (Dublin Core Metadata Initiative, 2010) and the European Statistical Service Dimensions for quality (Eurostat, 2005). Table 3: Basic descriptors considered for data sources assessment (based on Dublin Core criteria and the European Statistical Service Dimensions for quality). Section 1 Name of the resource and identifier Title, Relevance (for the ESCO indicators as mentioned in section 1.2), Creator, Subject, Description, Publisher, Contributor, Date, Type, Format, Identifier, Source, Links/Relation, Coverage (geographic and period), Language Section 2 Quality parameters Edition, Timeliness, Accessibility, Clarity, Comparability The Multi Criteria Decision Analysis (MCDA) approach The suggested framework for the evaluation of databases used to provide signals for indicators of emerging risks to food and feed is a clear example of a multi-attribute-decision/preference problem. In this case, a number of databases of interest are assessed against a set of common criteria accepted by all stakeholders. This allows a standardized and comparable characterization of the databases. Multicriteria-decision-analysis (MCDA) techniques are well suited to deal with these types of problems. MCDA approaches are most commonly used to support strategic decision making in organizations when multiple competing alternatives are possible and where evidence to support decisions is limited. EFSA Journal 2011;9(8):EN

17 MCDA techniques provide a consistent framework to support decisions under these conditions by facilitating: i) The identification of fundamental objectives or criteria, such as those that are crucial to the decision problem (usefulness of databases to inform emerging risks to food and feed in our case) and that allow the consistent characterization of alternatives, and, ii) The construction of a value hierarchy by breaking the objectives down into quantifiable attributes. Montibeller and Franco (2007) provide a clear and comprehensive description of the methodology. Del Rio Vilas et al. (2009) show the application of MCDA to emerging risks that could affect Great Britain s animal health status. Walshe and Burgman (2010) describe a systematic framework for the assessment and management of risks posed by emerging disease in which MCDA contributed the evaluation of consequences and preferences associated with competing disease control alternatives. 3. RESULTS AND DISCUSSION 3.1. Identification of signals and data sources linked to the eleven ESCO indicators During the data source identification phase, the aim of the DACO WG was to generate an unbiased (although admittedly non-exhaustive) list of potential data sources. At this stage, source exclusion was not considered, since this would ultimately occur based on the outcome of the quality assessment. Based on their expert judgement, the DACO WG members identified a large number of data sources (in many cases with associated signals) that could be linked to the current eleven ESCO indicators. In total, 188 potential data sources were identified (see Appendix A). This list included many indicatorspecific data sources (i.e. 126) but data sources that were linked to more than one indicator were also identified (i.e. 62) (see Appendix A). For the eleven ESCO indicators, potentially relevant data sources were identified in all cases, albeit the number per indicator varied significantly (e.g. indicators 5 and 10 had the lowest number: 9 and 14 data sources, respectively and indicators 8, 11 and 6 had the highest numbers: 56, 48 and 44 data sources, respectively). Many data sources were proposed by more than one DACO WG member, irrespective of their expertise. It was clear that if more experts were involved, the number of data sources would increase and a more balanced distribution of the data sources across the indicators may be achieved. Whether this will improve the identification of an emerging risk related to the indicators though remains to be seen. Alternatively, those indicators with significantly fewer linked data sources, as shown by the DACO WG exercise, may genuinely be less well covered (e.g. indicator 5 - Emergence of increased resistance to antimicrobials and plant protection products). Consequently, a quality assessment of the data sources and temporary monitoring of the signal from the particular data source will be necessary in order to establish the true value of any data sources used for this purpose Linking EU Framework Programme projects to the ESCO indicators To appraise a method to identify relevant projects within a research portfolio, the DACO WG performed an exercise to link a limited number of EU FP6 and FP7 projects to the eleven ESCO indicators. In total, 74 FP6 and 91 FP7 projects were evaluated (see Appendices B and C). Generally, the DACO WG members analysed the title and abstract of a project for their potential to provide information to one of the ESCO indicators. In this way, most EU FP projects could be tentatively linked (i.e. 70 out of the 74 FP6 and 75 out of the 91 FP7), although the value of these links was in most cases uncertain as title and abstract alone did not always provide enough detail to judge the expected outcome of the EFSA Journal 2011;9(8):EN