The Animal Health Quadrilateral Epiteam International collaboration on Foot-and- Mouth Disease simulation modelling for emergency preparedness.

Size: px

Start display at page:

Download "The Animal Health Quadrilateral Epiteam International collaboration on Foot-and- Mouth Disease simulation modelling for emergency preparedness."

Godwin Potter
5 years ago
Views:

1 Appendix 10 The Animal Health Quadrilateral Epiteam International collaboration on Foot-and- Mouth Disease simulation modelling for emergency preparedness. Dubé Caroline 1,*, Garner G 2, Sanson R 3, Harvey N 4, Stevenson M 5, Wilesmith J 6, Griffin J 7, Estrada C 8, Van Halderen A 9 1: Canadian Food Inspection Agency, 59 Camelot, Ottawa, Ontario, K1A 0Y9. 2: Department of Agriculture Fisheries and Forestry Australia, GPO Box 858, Canberra, ACT : AgriQuality Limited, Batchelar Centre, Tennent Drive, Palmerston North, New Zealand. 4: University of Guelph, Ontario, Canada, N1G 2W1 5: IVABS, Massey University, Private Bag , Palmerston North, New Zealand 6: Department for Environment, Food and Rural Affairs, 1A Page Street, London, SW1P 4PQ. 7: Department of Agriculture and Food, Agriculture House, Kildare Street, Dublin 2. 8: United States Department of Agriculture, 4700 River Road, Unit 41, Riverdale, MD, : Biosecurity New Zealand, Private Bag 2526, Wellington, New Zealand Abstract: Introduction: In March 2005, the Quadrilateral (QUADs) countries (Australia, Canada, New Zealand, United States) held a workshop on foot-and-mouth disease (FMD) modelling and policy in Canberra, Australia. The objectives of the workshop were to present policy-makers with the models developed or under development and review the current status of FMD policy development in these countries. A significant outcome of this meeting was the creation of an EpiTeam, a small technical group that includes epidemiologists from the QUADs countries, Ireland and the UK. Following this workshop, the EpiTeam developed a work program, with a formal FMD simulation model comparison as a priority. This paper presents results of the modelling workshop, of the model comparison and the objectives of the EpiTeam. Materials and Methods: Three spatial simulation models, used for FMD policy development in the QUADS countries were compared: AusSpread (Australia), InterSpread Plus (New Zealand) and the North American Animal Disease Spread Model (NAADSM, North America). The formal model comparison included: (1) an evaluation of written model descriptions, (2) eleven scenarios testing spread mechanisms, and control measures. Descriptive statistics of selected outputs were statistically compared. Kaplan-Meier survival curves were used to compare temporal spread. Minimum convex hulls and kappa index of agreement were used to compare the spatial spread in each scenario. Results: Despite the different approaches used in model building, the three models produced similar results in most scenarios. All models were improved as a result of this comparison: programming errors and assessment of impact of certain programming decisions were described. Discussion: Code verification and validation are critical steps in model development. We believe that a formal comparison of different models should be a mandatory step towards gaining model credibility. Furthermore, guidelines for proper model development, testing and use should be developed and promoted to the World Organization for Animal Health (OIE). 60

2 Introduction: Models are a representation of reality that can help test hypotheses and answer questions. Simulation models have been developed to evaluate the consequences of foot-and-mouth disease (FMD) and the effect of various control strategies. Disease models were used during the 2001 FMD UK outbreak to guide and support decisions. Significant recommendations were made following the UK experience on the proper uses of disease models. In addition, many countries have developed simulation models to prepare against such outbreaks. Models need to be used carefully by people with knowledge of underlying assumptions and limitations of the models. Proper representation of results to policy-makers and decision-makers is critical to the success or failure of the uses of these models. In March 2005, The Quadrilateral (QUAD) countries (Australia, Canada, New Zealand and the United States), through the Animal Health Emergency Management Group, held a workshop on the role of modelling to support decision-making in a disease emergency (with specific emphasis on FMD) in Canberra, Australia. The aim of the workshop was to present policy-makers with the models developed or under development in the QUADs countries and review the current status of FMD policy in the various countries. It is believed that modellers and policy-makers should work together to develop the most efficient methods of controls for FMD outbreaks. The Workshop was intended to identify actions/activities to promote better understanding of the role of modelling in policy development, and opportunities for collaboration by QUADs countries. Key outcomes of the meeting were the recognition of the need to: 1) Build trust in models by proactively engaging with all stakeholders and organisations that will use or make use of disease simulation and economic impact models. 2) Use a range of epidemiological tools (including models) to provide decision-makers with useful insights: a. in planning and preparing for exotic disease events b. in managing and debriefing exotic disease events 3) Collaborate to share information, approaches and undertake joint validation studies. 4) Use economic analysis in decision-making. Outputs from epidemiological models can be inputs to these economic analyses. To meet these key outcomes, an action plan was developed, to be implemented over the next few years, involving both the Quadrilateral Emergency Management Working Group (QEMWG) and a proposed new subgroup, the EpiTeam. Essential to achieving these outcomes was the formation of the EpiTeam, in May 2005, as a subgroup of QEMWG, with membership expanded to include Ireland and the United Kingdom, who are signatories to the International Animal Health Emergency Reserve (IAHER). The purpose of the EpiTeam is to: provide technical (epidemiology and modelling) support to the QEMWG and the QUADs Chief Veterinary Officers (CVOs) regarding policy development to manage significant animal health events; develop tools that will guide decision-makers on the control and management of significant animal health events; exchange epidemiological and modelling expertise between QUADs countries in peacetime and also during significant animal health events; work cooperatively with other groups with similar objectives; improve the understanding of the modelling approaches adopted by member countries and the country specific issues for modelling. Following the 2005 workshop, the EpiTeam developed a work program. Top priority tasks are: complete a model comparison of respective countries FMD models develop banks of FMD outbreak scenarios to support policy development share methodologies for data collection and analyses extend modelling methods to other diseases such as avian influenza 61

3 develop decision-support tools for the initial stages of an outbreak develop guidelines for proper uses of simulation models both before and during animal health emergencies. This paper presents the preliminary results of the QUADs model comparison project. Materials and Methods: Simulation models: Three models are currently being used by EpiTeam members to support policy development for FMD response within the QUADS: AusSpread in Australia (Garner and Beckett, 2005) InterSpread Plus in New Zealand (Stevenson et al, 2006) and the North American Animal Disease Spread Model (NAADSM) in Canada and the United States, an enhanced version of Spreadmodel (Schoenbaum and Disney, 2003). All models are stochastic, spatial state-transition simulation models that use daily time steps and the farm as the basic unit of interest. The model descriptions of the most recent, complete version of these models were obtained for this comparison. Comparative study: population dataset, scenarios and parameters: A fictional dataset that included 6000 units (3960 cattle units, 2040 swine units) located in the middle of the Atlantic Ocean was uploaded in each model. As all three models compared are stochastic (take chance and randomness into consideration) a number of runs were completed for each scenario in order to get a representative distribution of possible outcomes. Following discussions, 40 runs per scenario were considered appropriate, based on previous studies conducted with InterSpread Plus (Wilesmith et al, 2003). There were three phases to this project: (1) comparison of conceptual models through a formal review of model descriptions, (2) comparison of six scenarios assessing spread mechanisms and (3) comparison of five scenarios assessing control measures. Model descriptions were provided to the project lead at the University of Guelph at the start of the project and from there, a series of parameters for the first scenario were provided to the modelling groups. Results of the scenarios were entered in Microsoft Excel spreadsheets and sent by to the project lead. Upon successful completion of exchange of results of the first scenarios, parameters for the remaining scenarios of each phase were communicated by . Conference calls and communications provided the venue for discussion on issues relating to parameters and results. Initially, a set of six scenarios compared the spread mechanisms in their most basic form - FMD could spread among cattle premises only, and no control measures were used. Spread pathways included direct contacts (DC - movement of livestock among farms), indirect contact (IC- sharing of equipment, movement of people and fomites among farms), and airborne spread (AB). Parameter values were specified, including the duration of disease states (latent, infectious and immune) as triangular distributions, a rate of contact, distance distribution for these contacts and a probability of transmission if a contact occurred (for direct and indirect contacts), and direction, probability of infection by distance from an infected premises and a maximum distance for airborne spread. Farm-to-farm spread was simulated with constant infectivity of infected premises assumed in all scenarios. The third phases of the comparison included scenarios assessing control measures: surveillance, movement controls, vaccination using different sizes of ring buffers and various slaughter policies including destruction of detected infected premises (IPs) and farms within various distances of IPs, contact and ring premises. Analysis of results: The total numbers of new IPs were recorded at the end of all iterations for each scenario and a one-way analysis of variance (ANOVA) used to compare the model predictions. In scenarios assessing control measures, the total numbers of destroyed, detected and vaccinated premises and the numbers of days until the end of the outbreak were also compared using ANOVA. To assess temporal spread, Kaplan-Meier survival curves comparing the numbers of farms infected by each model for each scenario were computed. To assess the predicted sizes of outbreak areas, minimum convex hulls were drawn around all IP locations for each iteration and the area of each minimum convex hull calculated. A one-way ANOVA was used to compare outbreak sizes predicted by the three models. For all comparisons, a P-value <0.05 was considered statistically significant. 62

4 Results: The comparison of the conceptual models highlighted the amount of complexity that some of the simulation models could handle in terms of modelling the contact structure among farms in the population and in modelling airborne transmission, disease control zones and intra-unit spread. In general though, the more complex models could be adapted to represent simpler scenarios while the simpler models could not readily represent more complex scenarios. The scenarios used in this comparison project were relatively simple, therefore all models were able to be adapted to represent the parameters required. In the second phase of the project, there were there were small but statistically significant differences in the number of IPs generated by the three models (Table 1) in all scenarios except Scenario 5. In four out of six scenarios, the NAADSM had the lowest mean predicted number of IPs. The models also differed (statistically) in the temporal spread in Scenarios 1 and 3 (Figure 1). In all scenarios but Scenario 6, the size of the outbreak area was statistically different among the three models (Figure 2). Scenario 5 had the same disease and contact parameters as in Scenario 4, but was allowed to run for a longer period, to allow immune herds to become susceptible and generate a second wave of infection. AusSpread had the lowest mean predicted number of infected premises in four out of the five Phase 3 scenarios. There were statistically significant differences in the number of IPs in Scenarios 7, 9 and 10 (Table 2). There was no difference in temporal spread in all scenarios compared in this phase. The predicted sizes of the outbreak areas differed in Scenarios 7, 9 and 10 (Figure 3). Discussion: All three models are based on similar objectives and methodological approaches (all being spatial simulation models). However, given the differences in the way various model components have been implemented, it is not surprising that they gave slightly different results. Although there were statistically significant differences among the number of IPs and temporal and spatial spread predicted by the three models over some of the scenarios, these differences were generally small and from a practical perspective, the outputs were quite similar. Differences in the size of outbreak areas were attributed to variations in the process of recipient premises selection among the three models, although in most scenarios those differences were relatively small. The outbreak areas in Scenario 6 where the outbreak spread to all cattle premises did not differ. This was the only scenario where realistic disease and spread parameters were used. In all other scenarios, somewhat unrealistic parameters were used to clearly observe the impact of the spread mechanism being tested. The model comparison project has had an additional positive outcome in that it has led the modellers to take an in-depth look at the way core functions are implemented in their models. For example, whether values selected from distributions were rounded or not had a significant impact on the results. For Scenario 1, AusSpread was initially run in its default mode, which allows infectivity of a herd to increase with time reflecting within-herd spread of FMD. This resulted in significantly fewer new infections than when infectivity was kept constant (assuming a 100% prevalence from the first day of infection), showing that this is an important issue to consider. Future studies will allow for variable within-herd prevalence. The impact of delays in state-transitions occurring in the models was observed in the first scenario of Phase 2. By default, AusSpread and InterSpread Plus allowed the transition from susceptible herd to latent herd (following successful exposure) to occur on the day infection occurred while the transition to latent occurred on the following day in NAADSM. This meant that NAADSM produced significantly smaller outbreaks than the other two models until this issue was identified and accounted for by AusSpread and InterSpread Plus. AusSpread recorded a lower number of IPs in Scenario 10, in which the infection spread to the west edge of the simulated study area. When infection is active near the edge of the study area, AusSpread can generate contacts that go out of the area, and are recorded as such. The other two models do not simulate out-of-area contacts, and thus will tend to concentrate new infections along the edge of the study area, leading to an increased number of IPs. Some extra experimentation with this scenario (running for fewer days, so that the infection does not reach the edge) suggests that this "edge effect" is responsible for the differing results. Future comparisons will include further testing of this effect. 63

5 Programming decisions relating to the order in which events occurred in a simulation day had important impacts on the results. For example, some models allowed spread to occur before detection in a simulation day whilst another model had detection occur prior to spread. This produced more spread in the model that had spread occur prior to detection in a day. It was suggested that models should allow these events to occur in a random order during a simulation day to mimic the fact that in a real outbreak, detection could occur at any time during a day. For policy makers, it is reassuring that despite the different approaches used in the models, they produced similar outcomes and it can be concluded that despite the small differences, decisions based on the results from any of the three models would not differ. This comparison study was an extremely valuable exercise as it forced the modellers to consider all the assumptions made in the model-building process, including the quantification of the impact of certain programming decisions. In addition, minor bugs were identified in all models and therefore all models have been improved as a result. Further discussion on different assumptions and further comparison scenarios will take place among the modelling groups to test other features of the models with more complex scenarios. Conclusions: The model comparison has proven a very valuable exercise in gaining insight into each country s models and in testing and quantifying some of the assumptions included in these models. Through this project, the EpiTeam will develop guidelines on proper model testing and validation to ensure that FMD models used for policy development and in some cases, as predictive tools during outbreaks, are based on proper biological principles and are computationally sound. These guidelines should be proposed to the World Organization for Animal Health (OIE) as a template by which other modelling groups may assess their models. This is the first time an international team of government epidemiologists have formally agreed to collaborate on developing and validating tools for use in animal health emergency preparedness and response. These tools should enhance decision-making by the respective governments if faced with FMD. In addition, the team will provide resources and expertise that can be exchanged during these events. References: Garner, M.G. and Beckett, S.D. (2005) Modelling the spread of foot-and-mouth disease in Australia. Australian Veterinary Journal 83, Schoenbaum, M.A., Disney W.T. (2003) Modeling alternative mitigation strategies for a hypothetical outbreak of Foot-and-Mouth Disease in the United States. Preventive Veterinary Medicine 58, Stevenson, M.A., Sanson, R.L., Stern, M.W., O Leary, B.D., Mackereth, G., Sujau, M., Moles-Benfell, N., Morris, R.S. (2006) Interspread Plus: a spatial and stochastic simulation model of disease in animal populations (submitted). Wilesmith, J.W., Morris, R.S., McIntyre, L., Stevenson, M.A., Stern, M.W., O'Leary, B.D., Sujau, M. (2003) Evaluation of livestock standstill policies applied in conjunction with animal movement distance restrictions for foot-and-mouth disease in Great Britain. URL: 64

6 Table 1. Mean number of infected units by simulation model in the first 6 scenarios compared in the Quads model comparison project. Scenario AusSpread Interspread Plus NAADSM Mean St dev Mean St dev Mean St dev 1 Direct contact Airborne spread Direct contact / airborne spread Direct and indirect contact Direct and indirect contact short duration 6 Direct and indirect contact long duration Table 2. Mean number of infected units by simulation model in the last 5 scenarios compared in the Quads model comparison project. Scenario AusSpread Interspread Plus NAADSM Mean St dev Mean St dev Mean St dev 7 Detection and movement controls 8 Vaccination small rings Vaccination large rings Destruction small rings Destruction large rings

7 Scenario 1 Scenario 3 Figure 1. Kaplan-Meier survival curve showing the proportion of farms that were free of disease as a function of time, using the median number of predicted infections per day (computed over 40 iterations) as the outcome in Scenarios 1 and 3 of Phase 2 of the QUADs model comparison project. 66

8 Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5 Figure 2. Box and whisker plots showing distributions of the sizes of the predicted outbreak areas (over 40 iterations) for the Australian, New Zealand, and North American models in scenarios where a statistically significant difference was observed. Area expressed as 1000 sq km units. 67

9 Scenario 7 Scenario 9 Scenario 10 Figure 3. Box and whisker plots showing distributions of predicted outbreak areas (over 40 iterations) for the Australian, New Zealand, and North American models in the scenarios showing a statistically significant difference. Area expressed as 1000 sq km units. 68

Evaluating vaccination for foot-and-mouth disease control an international study

Evaluating vaccination for foot-and-mouth disease control an international study Garner MG, Gauntlett FA, Sanson RL, Stevenson MA, Forde-Folle K, Roche SE, Birch C, Owen K, Dube C, Rooney J, Corso B, Cook