Use of Microbial Risk Assessment in Decision-Making

Transcription

1 Use of Microbial Risk Assessment in Decision-Making Slide show on: /firstmicrobial.htm Note the 2 l s! David Vose Consultancy Les Lèches Dordogne France David Vose's secretary David Vose Introduction Applying CODEX guidelines in reality Difficulties Other ways of thinking Experience with microbial modelling Some survey results The Dutch experience Some US experience Modelling challenges Comparison of some complete models Reviewing a model in context Slide 2

2 Codex Alimentarius Commission FAO/WHO (1995) Microbial risk assessment is a scientifically-based process consisting of four steps: 1. Hazard Identification The identification of known or potential health affects associated with a particular agent; 2. Exposure Assessment The qualitative and/or quantitative evaluation of the degree of intake likely to occur; 3. Hazard Characterization The quantitative and/or qualitative evaluation of the nature of the adverse effects associated with biological, chemical and physical agents that may be present in food For biological agents a dose-response assessment should be performed if the data is available; 4. Risk Characterization Integration of Hazard Identification, Hazard Characterization and Exposure Assessment into an estimation of the adverse effects likely to occur in a given population, including attendant uncertainties. Slide 3 OIE experience OIE produced guidelines for animal import risk assessments (for the management of disease spread) Now in its second edition Guidelines were offered as a way to help member (including developing) countries understand how to perform a r.a. First Ed. guidelines were used too literally, both by analysts and lawyers, and found to be often impractical or irrelevant to the risk question Lesson: keep guidelines non-specific, encourage understanding rather than prescribing a formulaic approach Popular interpretation of CODEX guidelines suffer similarly Slide 4

3 Society for Risk Analysis Principles for Risk Analysis (1) Risk analysis uses observations about what we know to make predictions about what we don t know. Risk analysis is a fundamentally science-based process that strives to reflect the realities of Nature in order to provide useful information for decisions about managing risks. Risk analysis seeks to inform, not to dictate, the complex and difficult choices among possible measures to mitigate risks... Slide 5 Society for Risk Analysis Principles for Risk Analysis (2) Risk analysis seeks to integrate knowledge about the fundamental physical, biological, social, cultural, and economic processes that determine human, environmental, and technological responses to a diverse set of circumstances. Because decisions about risks are usually needed when knowledge is incomplete, risk analysts rely on informed judgment and on models reflecting plausible interpretations of the realities of Nature. We do this with a commitment to assess and disclose the basis of our judgments and the uncertainties in our knowledge. Slide 6

4 Current modelling Microbial QRA is a developing science We re making a lot of progress, but it is still in infancy Mostly producing farm-to-fork Models the whole system but very poorly Not designed to model any decision question well Often relies on poor data, surrogates, and guesses Almost never is a decision question posed beforehand Assessors have probably over-sold QRA s usefulness Managers have expected too much Slide 7 F2F Achilles Heels Very little data available, system being modelled is hugely complex! Uncertainty, variability, inter-individual variablility Take too long to complete, too easy to make mistakes F2F considers only pathogen on the food source E.g. not E.coli produced during life of animal, appearing in water, vegetables, farmers exposure Predictive microbiology still unreliable Broth data doesn t translate well to food (usually overestimate, but some data Tamplin, USDA shows lag period can be shorter, e.g. E.coli in ground beef, Listeria in processed hams) Models often not based on physical/biological ideas, so we don t learn Attenuation may not be death, and ignores reactivation of bacteria D-R models inadequate Don t describe variability observed P(ill dose, infected) = P(ill infected)? Feeding trial data don t match epi data can hugely underestimate the risk Little cost-benefit analysis effort made Including actions affecting several risk issues Requires enormous resources impractical for many countries Slide 8

5 Dutch observations on past QRA Havelaar, Jansen (2002) The lessons learnt from risk analysis experiences: 1. Risk management has not always been an integral part of risk analysis so far; 2. Risk managers should be trained to understand risk assessment, and risk assessors should be trained to explain their work; 3. Available data are often of limited use for risk assessment and communication of data needs between risk assessors, food scientists and risk managers is a critical issue; 4. The risk manager questions usually require rapid results, whereas (farm-to-fork) risk assessment projects require several years to complete. Solving this conflict requires open communication; 5. Uncertainty is often large. Slide 9 Our survey Internet based, voluntary participation, 39 valid responses 60% 50% 40% 30% 20% 10% Do you consider that risk analysis has brought about the following improvement to government decision-making? Very much Much Some Little Very little 0% Fairness Rationality Consistency Transparency Slide 10

6 What factors jeopardise the value of an assessment? Always 45% Usually 40% 35% 30% 25% 50:50 Seldom Never 20% 15% 10% 5% 0% Insufficient human resources to complete the assessment Insufficient time to complete the assessment Insufficient data to support the risk assessment Insufficient in-house expertise in the ares Insufficient general scientific knowledge of the area Slide 11 What factors jeopardise the appropriate implementation of a risk management decisions? Always 60% 50% 40% Usually 50:50 Seldom Never 30% 20% 10% 0% Politics Other issues take precedence Legal restrictions Insufficient resources to implement the action Risk assessment too complicated Risk assessment not accepted as valid Insufficient time Slide 12

7 70% 60% 50% 40% 30% 20% 10% Do decision makers: Always Usually 50:50 Seldom Never 0% 1) Understand and use the results of a risk assessment 2) Encourage involvement of stakeholders and external expertise? 3) Encourage 4) Put a lot of emphasis 5) Involve risk 6) Involve stakeholders 7) Expect too much of a 8) Expect too little of a assessors to explain on receiving comments assessors in planning after completion of the risk assessment? risk assessment? w hat may and may not at the planning stage of how to communicate assessment of the be possible to achieve? a risk assessment the risk assessment appropriate risk results? management action to take? 70% 60% 50% Do decision makers: Always Usually 50:50 Seldom Never 40% 30% 20% 10% 0% 9) Assign sufficient resources and time to complete the risk assessment? 10) Encourage assessors 11) Require the 12) Make the results 13) Encourage assessors 14) Involve risk assessors to suggest alternative assessment to be simplified available only if it suits their to produce an assessment in the decision-making? approaches to the for ease of understanding, purposes? to support a predetermined assessment? at the expense of technical position? accuracy? 15) Allow /expect the risk assessors to make the decision? Slide 13 Completion times of some farm-to-fork QRAs Harvard BSE Final report CVM Campy Final report FSIS E. Coli Draft report FDA Listeria Being revised USDA Vibrio Draft report US FSIS SE Final report Jan-96 Jan-97 Jan-98 Jan-99 Jan-00 Jan-01 Jan-02 Slide 14

8 Salmonella dose-response Epi and feeding trial comparison S. All S. typhirurium Salmonella enteritidis spp.: only: Mean mean outbreak attack rates (plus 90% confidence intervals) 100% International outbreak attack rate 80% 60% 40% 20% Normal; Normal food Normal; Fatty food Susceptible; Normal Food Susceptible; Fatty food All participants Naive participants 0% 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08 1.E+09 1.E+10 Dose Review by Amir Fazil in FAO/WHO (2001) D-R mathematical models review by Haas (2002) Slide 15 USDA-FSIS-FDA Salmonella Enteritidis Although the goal was to make the model comprehensive, it has some important limitations. It is a static model and does not incorporate possible changes in SE over time as either host, environment or agent factor change. For many variables, data were limited or nonexistent. Some obvious sources of contamination, such as food handlers, restaurant environment, or other possible sites of contamination on or in the egg (such as the yolk), were not included. And, as complex as the model is, it still represents a simplistic view of the entire farm-to-table continuum. Finally, the model does not yet separate our uncertainty from the inherent variability of the system. Much more work is needed to address this, and all other, limitations. Slide 16

9 USDA-FSIS-FDA Salmonella Enteritidis Original model impetus was to evaluate effect of refrigeration temp from laying to retail on food safety Empirically must have little affect since it only deals with a few days in the life of an egg No cost-benefit attached Now being redone to focus on level of performance required for shell, and liquid egg pasteurisation i.e. much more decision focused Slide 17 FDA Listeria risk assessment No specific decision questions attached Attempted to look at relative importance of a large list of Listeria-carrying foods Given the data available, perhaps the only method possible to estimate which food types contribute the greatest risk So a good QRA application Slide 18

10 Remedies: focusing on decisions Consider what is known about the risk problem, and data available immediately or within acceptable time frame Use epi data as much as possible Collect more epi data (e.g. Japan, Denmark) Consider what analysis could be done with this knowledge i.e. a risk-based reasoned argument for evaluating particular actions Estimate the possible magnitude of benefit for a risk action Note that it may not be possible to evaluate all actions Perform a cost-benefit analysis on these actions Perform a Value of Information analysis Determines whether it is worth collecting more data before making a decision Consider strategy to validate whether predicted improvement occurs Train data producers to supply maximally useful data E.g. microbiologists taken more than one cfu from a plate More inter-agency unity E.g. Farm (APHIS) Slaughter (FSIS) Retail (FDA) Slide 19 Make it as simple as possible: example Risk: Human illness from SE in eggs A shell-egg selection system proposed that will reduce by 30% the number of contaminated eggs going to market Currently, 30,000 people a year suffer from SE from eggs What will be the reduction in cases if the new system is implemented? Reduction in cases = 30%*30,000 = 9,000 people/year No need for models of D-R, bacterial growth, handling, etc. Vulnerability to assumptions smaller than from using F2F model Slide 20

11 An example of the way forward Havelaar, Jansen 2002 Campylobacter Risk Management and Assessment Dutch proposal The main objectives of the project are to advise on the effectiveness and efficiency of measures aimed at reducing campylobacteriosis in the Dutch population. The two key questions are: 1. What are the most important routes (quantifiable?)? 2. Which (sets of) measures can be taken to reduce the exposure to Campylobacter, what is their expected efficiency and societal support? Slide 21 The way forward cont. The target of the assessment is not limited to estimating the possible reduction in disease incidence but to evaluate both costs and benefits of possible interventions and to access their acceptance by stakeholders. Interventions with low social support will require more effort to uphold, which increases their costs and reduces their efficacy. Slide 22

12 Danish Vet Service Salmonella QRA A Bayesian Approach to Quantify the Contribution of Animal-food Sources to Human Salmonellosis - Hald, Vose, Koupeev (2002) Eggs Travel Outbreak Pork Imported poultry 97.5% percentile Mean 2.5% percentile Imported pork Broilers Imported beef Turkeys Ducks Beef Unknown source ,000 1,200 1,400 Estimated number of cases Estimated number of cases of human salmonellosis in Denmark in 1999 according to source Model ranks food sources by risk. Easily updateable with each year s data. Bayesian update improves estimate and checks validity of assumptions. Slide 23 Section 1 Campylobacter culture confirmed cases observable in US population Section 2 Total number of Campylobacter infections in year in US population Fluoroquinolone-resistant Campylobacte risk assessment Section 3 Number of those with Fluoroquinolone-resistance from chickens and administered Fluoroquinolone Section 4 Number of Fluoroquinolone resistant Campylobacter contaminated chicken carcasses consumed in year Section 5 Using the model to manage risk. Measuring the level of risk. Controlling the risk. Model: Contaminated carcasses after slaughter plant * probability = affected people Slide 24

13 Broiler house Transport Slaughterhouse model Example of Farm-to-Fork model Slaughter house Hanging Scalding Defeathering Evisceration Washing Chilling Export Chicken parts Whole chickens Campylobacter in poultry Draft report 2001 Further Processing Chilled Frozen Import Institute of Food Safety and Toxicology Division of Microbiological Safety Danish Veterinay and Food Administration Behaves the same way as CVM model if prevalence is reduced Catering Cross contamination Heat treatment Retail Consumer Cross contamination Heat treatment Dose response Consumer model Risk estimation Slide 25 AHI model Chicken Fluoroquinolone resistant Campylobacter in poultry Microbial Load by Season (cfu) Harvest P(Resistant Inf.) Transport Factor (increase cfu) Transport AHI / Cox 2000 A dynamic simulation model (i.e. follow the path of a random chicken) Processing Factor (decrease cfu) Processing Used same data as CVM where available. Strong potential because it reduces model complexity (at the expense of simulation time)and easy to follow. Storage and Prep. Factors (decrease or eliminate cfu) Storage and Preparation Rate by Demographic Group Difficulty is availability of data to make model parameters meaningful. Most model parameters in current version represent nothing physical ( factors ), so don t enlighten us as to what actions to take. P(Infection Dose) P(Illness Infection) P(Treatment Ill) Consumption Human Illness Human D-R and consumer handling difficulties remain. P(FQ Treatment) FQ and Resistant Excess Slide 26 Illness Days

14 Reviewing a risk assessment Risk assessment should be decision focused It is not appropriate to review a risk assessment independently from the question(s) the assessment is addressing Eg because a point is moot if the decision is insensitive to the argument It uses science but is not itself scientific research So we have to go with the best we ve got Slide 27 Finally - risk assessors should gain hands-on experience to ensure their models reflect the real world Slide 28

15 References Haas, C.N., (2002), Conditional Dose-Response Relationships for Microorganisms: Development and Applications. Risk Analysis 22 (3): Havelaar, H. and J. Jansen, (2002), Practical Experience in the Netherlands with quantitative microbiological risk assessment and its use in food safety policy. Draft paper, RIVM, Bilthoven, The Netherlands. Hope, B.K., et al., (2002), An overview of the Salmonella Enteritidis Risk Assessment for Shell Eggs and Egg Products. Risk Analysis 22 (3): Joint FAO/WHO Expert Consultation on the Application of Risk Analysis to Food Standards Issues (Joint FAO/WHO, 1995). Joint FAO/WHO Expert Consultation on Risk Assessment of Microbiological Hazards in Foods: Risk characterization of Salmonella spp. in eggs and broiler chickens and Listeria monocytogenes in ready-to-eat foods. (2001), FAO headquarters, Rome. Teunis, P.F.M. and A.H.Havelaar, (2001), The Beta-Poisson Dose-Response Model Is Not a Single-Hit Model. Risk Analysis 20 (4): Slide 29