Analyzing the Impact of Modeling Choices and Assumptions in Compartmental Epidemiological Models

Transcription

1 Analyzing the Impact of Moeling Choices an Assumptions in Compartmental Epiemiological Moels Journal Title XX(X):1 11 c The Author(s) 2016 Reprints an permission: sagepub.co.uk/journalspermissions.nav DOI: /ToBeAssigne Özgür Özmen 2, James J. Nutaro 1, Laura L. Pullum 1, an Arvin Ramanathan 2 Abstract Computational isease sprea moels can be broaly classifie into ifferential equation-base (EBM) an agentbase moels (ABM). We examine these moels in the context of illuminating their hien assumptions an the impact these may have on the moel outcomes. Drawing relevant conclusions about usability of a moel requires reliable information regaring its moeling strategy an its associate assumptions. Hence, we aim to provie clear guielines on evelopment of these moels an elineate important moeling choices that causes the ifferences between the moel outputs. In this stuy, we present a quantitative analysis of how the choice of moel trajectories an temporal resolution (continuous versus iscrete-event moels), coupling between agents (instantaneous versus elaye interactions), an progress of patients from one stage of isease to the other affect the overall outcomes of moeling isease sprea. Our stuy reveals that the magnitue an velocity of the simulate epiemic epens critically on the selection of moeling principles, various assumptions of isease process, an the choice of time avance. In orer to inform public health officials an improve reproucibility, these initial ecisions of moelers shoul be carefully consiere an recore when builing an ocumenting an agent-base moel. Keywors Susceptible-Infecte-Recovere (SIR), Epiemiology, Agent-base, Event-base, Equation-base moels 1 Introuction Computational moels of isease sprea processes can be broaly classifie into two types: (i) ifferential equationbase moels (EBM) an (ii) agent-base moels (ABM). For influenza-like illnesses (ILI) an other infectious iseases, computational moels are critical to public health officials for unerstaning the effects of isease sprea amist ense urban populations, for preicting the socioeconomic effects of major epiemics, an for forecasting the effects of ifferent intervention strategies 29,12,6. While both EBMs an ABMs are wiely use in practice, moelers often neglect to examine the impact of ifferent moeling choices an assumptions ecie in the early stages of moel evelopment. EBMs moel the population by segmenting it into ifferent compartments, such as susceptible (S), infecte (I), an recovere (R), with each compartment representing a stage in the progression of the isease 12. Aitional compartments can be efine base on moel requirements. Transitions between each of the compartments capture the overall ynamics of the epiemic, showing how the population as a whole gets infecte an eventually recovers. EBMs capture aggregate behaviors over the whole population. Iniviuals an interactions between iniviuals are omitte from EBMs, an so these moels can be use with relatively little computational power an relatively small amounts of input. Therefore EBMs can capture largescale (incluing country-wie/ continent-wie panemic) isease sprea with a relatively small effort. On the other han, ABMs inclue iniviual level information to capture how people interact an participate in ifferent activities (work, school, exercise, etc.) 2,28. Although the term agents in epiemiology is use to refer to infectious agents, in this stuy we use ABMs to escribe moels that incorporate iniviual entities (or agents that can be autonomous, heterogeneous an/or aaptive) an are evelope by emphasizing the progression of the iniviual (an his/her behavior) over the course of the isease. While iniviuals within a community unergo a similar isease progression as in EBMs, the outcomes from the ABMs are largely etermine by the interaction between these iniviuals an the ata that characterize the interactions. 1 Computational Sciences an Engineering Division, Oak Rige National Laboratory, USA 2 Health Data Sciences Institute, Oak Rige National Laboratory, USA This manuscript has been authore by UT-Battelle, LLC uner Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The Unite States Government retains an the publisher, by accepting the article for publication, acknowleges that the Unite States Government retains a non-exclusive, pai-up, irrevocable, worl-wie license to publish or reprouce the publishe form of this manuscript, or allow others to o so, for Unite States Government purposes. The Department of Energy will provie public access to these results of feerally sponsore research in accorance with the DOE Public Access Plan ( Corresponing author: Laura L. Pullum, P.O. Box 2008 Oak Rige, TN USA Telephone: pullumll@ornl.gov Prepare using sagej.cls [Version: 2013/07/26 v1.00]

2 2 Journal Title XX(X) Previous research on comparing epiemiological moels has largely focuse on the comparison of ifferent moels an their outcomes without consiering the unerlying assumptions an implementations specific to these moels. Rahmana an Sterman 24 examine the choice between EBMs an ABMs from the perspective of heterogeneity in iniviual populations an the network topology of interactions, incluing fully connecte, ranom, smallworl, scale-free, an lattice networks. The authors foun that the EBM an ABM implementation base on the ifferent network moels iffere significantly in terms of their outcomes incluing iffusion spee (the amount of time that passes until the peak loa is observe), peak loa on health services (the maximum infectious proportion of the population observe over time), an total isease buren (cumulative number of people who ie from the isease at the en of the epiemic). Further, the authors speculate that the ifferences in the outcomes coul be a consequence of the ifferences in network topologies, heterogeneity (amongst iniviuals in the population), an iscretization of stochastic interactions between agents. However, they also observe similar ifferences for homogenous population. In their homogenous population scenario, the ifferences in the output trajectories can originate from stochasticity or moeling assumptions examine in our work. Unless the moel assumptions are explicitly ocumente, we can not etermine how much stochasticity or moeling assumptions contribute to their conclusions. Ajelli an co-authors compare an agent-base moel that is base on census ata of Italy an a structure meta-population moel name GLEaM 3 that was esigne to preict spatio-temporal aspects of global epiemics by incorporating International Air Transport Association (IATA) ata 1. Despite the similarity of results regaring the isease propagation at ifferent locations, this comparison i not provie etails on unerlying moeling principles an assumptions in the two istinct moels. Jaffry an coauthors qualitatively compare the results from an EBM an ABM an note that the ifferences in the results were a consequence of the sampling variability 10. Similarly, Macal claime that isagreements between agent-base an system ynamics moels of epiemiology are cause by stochasticity 15. Kenney an co-workers applie a variety of valiation techniques on one EBM an one ABM an aresse the importance of knowing the abstraction (i.e., conceptual moel) an implementation ecisions when comparing moels 13. Cross-valiation techniques 25, sensitivity analyses 18, an other statistical techniques 9 have been use to analyze EBMs an ABMs. Although unerstaning how istinct moels can have similar behaviors base on their input parameter values is valuable, it is ifficult, in general, to istinguish whether the results are a consequence of the calibration process or moeling assumptions an the specific implementation use in the moels. In this stuy, we present our analyses of the effect of moeling approach between ABMs an EBMs using 1918 flu panemic parameters. Our goal is to emonstrate our results via realistic scenario. The 1918 flu, referre to as the Spanish flu, was an exceptionally ealy an wiesprea panemic which resulte in over 50 million eaths worlwie 27. With nearly a thir of the worl population infecte, an an increase case fatality over other types of the flu, this panemic represents a wiely stuie phenomenon within moern infectious isease epiemiology. Our analysis examines three aspects of Susceptible- Infecte-Recovere (SIR) 14 moels; these are (i) the choice of an iniviual-base (ABM) or population-base (EBM) approach; (ii) the unerlying assumptions in moeling the isease process; an (iii) temporal resolution. We assess the effects of the aforementione moeling choices on the behavior of large-scale epiemiological simulations for a realistic scenario. For this purpose, we evelop a family of moels that are closely relate to the basic SIR. Each moel highlights a specific choice of EBM versus ABM an the accompanying mathematical assumptions concerning the isease process within an iniviual, interactions between iniviuals, an temporal resolution with which the moel evolves. Using these moels, we make an explicit stuy of how the unerlying mathematical structures of these moels can result in ifferent outcomes in spite of their share conceptual unerpinnings; specifically, we examine: I. The choice of moel trajectories an their attenant temporal resolution. This may be continuous or iscrete-event, which both have systems evolve in time over the real numbers, or iscrete-time, which has systems evolve in time over the natural numbers. II. The nature of the couplings between agents, which may be instantaneous functions (e.g., as occurs in EBM moels an in some iscrete-time an iscreteevent moels) or a elay function (e.g., as occurs in many iscrete-time moels). III. The mathematical process of progressing through the SIR compartments, which may be governe by a linear process (i.e., as in most EBMs) or with a non-linear elay (i.e., as in most agent base moels). In a majority of scenarios consiere here, the choice of ABM (i.e., iniviual-base) versus EBM (i.e., populationbase) moel has no impact on the aggregate (i.e., population level) ynamics of an epiemic when the moels agree on the mathematical processes for isease sprea (II-III above) an temporal resolution (I above). This suggests that the question of agent-base versus equation-base moel is of practical interest only when parameters are at certain limits or some particular use of the moel epens on a quantity mae visible by its agent-base version (e.g., the network of interactions between iniviual agents). However, ifferences between moels in any of the above traits can cause substantive ifferences in outcomes. 2 Methos In this section, we escribe the input parameters an moeling ecisions in etail. To facilitate a meaningful comparison of moels constructe with ifferent tools an using ifferent moeling approaches, we impose the following conitions on all of the moels: (i) Initial conitions: If two or more moels are being compare, then Prepare using sagej.cls

3 Özmen et al. 3 their initial populations are ientical, (ii) Outputting: Output from the ifferent moels show the state of the population at the same instants of time, an (iii) Parameterization: For iscrete-time moels, time-relate parameter values are scale in accorance with the step size. Initial parameters an their erive values are iscusse in the following section. 2.1 Parameters for the 1918 Influenza Panemic Previous stuies of the 1918 flu panemic (e.g., 26 ) provie estimates of the isease sprea parameters, an these parameters are use in the current stuy. The parameters were kept consistent across the six EBMs an ABMs use herein an were specific to the isease an population within the Unite States uring the 1918 panemic. A summary of the ifferent parameters an their values are shown in Table 1. Note that we use only a small number of parameters in this stuy in orer to balance the complexity of the moels versus the actual observations of the 1918 panemic. In orer to make the moels computationally feasible, we scale the population of Unite States in 1918 from 103 million to about 500,000. Note that this is an assumption that is practice in the former analysis 22,26 an the finings are scale to reflect the actual population. Previous stuies estimate the number of Decease within the Unite States to be between 550,000 an 675,000 uring 1918 influenza outbreak 11,27. This number can be use to estimate pm (i.e., the probability of eath given a person has contracte the flu) base on the estimate number of infecte people which correspons to approximately 50% of the population 11 at that time. However, these estimates are likely lower than the actual numbers ue to uner-reporting of incients an long term fatality rates. Moreover, the methos use to collect the ata can be biase reflecting impacts of emographic an geographic factors. Regarless, the estimates use in this stuy are selecte from the parameter ranges given in previous stuies. An important parameter in epiemiology is the R 0 value. This value is the average number of seconary cases cause by a primary infectious iniviual 12. Assuming a fully Susceptible population, when the basic parameter notations an values (shown in Table 1) are use, the R 0 value is calculate by R 0 = [((1 pm) tr + pm tm) re] pt. (1) Accoring to Chowell an co-workers 5, the R 0 of 1918 Influenza panemic varie between 1.5 to 2.0 epening on which wave of the panemic is consiere. The 1918 Influenza occurre in ifferent seasons with istinct peaks observe for each wave of the panemic. This stuy uses the value of R 0 1.8, which is within this range. 2.2 Equation-base Disease Sprea Moels In the previous section, we explaine the choice of parameters use in all the moels evelope. In this section, we escribe the iniviual moels. In SIR moels, the population is ivie into three compartments: Susceptible (S), Infectious (I) an Recovere (R). The ynamics of the isease sprea process are capture by the interactions between these compartments, which are moele as orinary ifferential equations (ODE). In the following sections, we escribe variations of the equation-base moels that mimic the SIR ynamics Simple SIR moel To account for the large number of eaths cause by the 1918 panemic, we moify the traitional SIR moel to inclue an aitional compartment, for the Decease population (represente as D within the moel). The R population is still maintaine to represent immune iniviuals who were sick an have recovere. Together with R an D, we create an aggregate compartment, terme the Remove (Re) population. Figure 1 represents the compartments use in the moels. Figure 1. SIR Moel Compartments. Using those state variables an parameters, the ynamics of this SIR moel is escribe by the set of Equations 2. ( pm t Re = tm + 1 pm ) I tr t D = pm t Re Re (2) t R = (1 pm) t S S = re pt I t t I = t Re t S S + I + R The expression for (/t)re in Equation (2) makes a significant simplification that assumes the Inf ectious population is homogeneous. At each instant of time, the moel estimates a fraction (1/tr) of the Inf ectious population will recover regarless of the ay of infection for those population members. To unerstan how this simplification affects the moel s evolution, suppose that Re(t = 0) = 0, I(t = 0) = 0, an that everyone recovers (i.e., pm = 0). By fixing (/t)i = K, we may ignore the Susceptible population S an assume that the number of infecte people increases at a constant rate K. We may rewrite the moel as: ( ) 1 t Re = I tr t I = K (3) Prepare using sagej.cls

4 4 Journal Title XX(X) Table 1. Parameters an their escriptions Parameter Parameter Description Reference Values pm Probability of eath given a person has contacte the isease tr Time to recover given that the infecte person will not ie 5,8,27 3 ays tm Time to ie given that the person with the isease will ie 27,7 1 ay pt Probability of transmission to another person after contact re Number of people encountere by an iniviual in a ay 5 4 people/ay NI Number of people that are initially infectious * 1,000 T P Total number of people in the population 500,000 * Assume values Scale values an the Recovere population at time t is therefore Re(t) = K 2tr t2 (4) However, if each infecte iniviual recovers in tr ays then we shoul expect the Recovere population at time t to equal the infecte population at time t tr. That is, we shoul expect Re(t) = I(t tr) = K (t tr) if tr < t 0 otherwise Hence the Recovere population preicte by the SIR moel an the Recovere population that woul actually be generate by the isease process are not the same. To correct this error, we evelop a slightly more elaborate moel that is escribe in the following section. (5) T I = T D = V i=1 M i=1 I i D i S S = re pt (T I + T D) t S + R + T I + T D t R = I V (6) t D = D M t I i = t D i = t S (1 pm) I i if i = 1 (I i 1 I i ) otherwise t S pm D i if i = 1 (D i 1 D i ) otherwise. The introuction of the ifferent substates overcomes the simplification in the EBM-SIR escribe previously; but it can run into the issue of iviing the population into large numbers of compartments as t 0. To overcome this shortcoming, we introuce an SIR moel that has a elay in the following section SIR Moel with Substates The simplifying assumptions in the previously escribe SIR moel can be countere by introucing substates I 1, I 2,..., I V with V = tr/ t an D 1, D 2,..., D M with M = tm/ t where t is the resolution in time with which the progress of the isease is moele. These substates represent the number of people in their first perio of infection, secon perio of infection, etc. Figure 2 represents the logic behin the ivision of the moel compartments. Assuming that T I is the total Infectious population that will recover an T D is the total Infectious population that will ie, the EBM moel incorporating these substates note above can be escribe by the set of ODEs SIR Moel with Delay In this moel, the substates are replace by a elaye ifferential equation that is the limit of the substate moel as t 0. In this moel, two iscontinuities appear, namely: (i) an event where there is a suen jump at tm of the Decease population (i.e., to account for the eaths of the initially infecte persons) an (ii) an event where there is a suen jump at tr of the Recovere population. To overcome these iscontinuities we incorporate two iscrete events. For the first event that occurs at t = tm, we (i) make Decease jump from zero to Infectious(t = 0) pm an (ii) remove Inf ectious(t = 0) pm (ecease persons) from the Inf ectious population. For the secon event that occurs at t = tr, we (i) cause Recovere to switch from zero to Infectious(0) (1 pm) an (ii) remove the Recovere from the infectious pool. The equations for this elaye moel with its two iscrete events are Prepare using sagej.cls

5 Özmen et al. 5 Figure 2. SIR Moel with Substates. t R = (1 pm) I(t tr) if t > tr 0 otherwise t D = pm I(t tm) if t > tm 0 otherwise S S = re pt I t S + I + R ( t I = t S + t R + ) t D (7) Figure 3. Agent Processes in the Event-base Moel. The SIR moel with elay represents an aitional refinement of the moels presente above. We next escribe the construction of three agent-base moels reflecting the same processes of isease sprea explaine above. 2.3 Agent-base Disease Sprea Moels Agent-base moels (ABMs) use etaile representations of iniviual entities (i.e., humans) to moel micro-level interactions among iniviuals. However, there are ifferent approaches to avancing time in the moel (event-base vs. time-steppe) an hanling the interactions between iniviuals in ABMs. Therefore, in the following sections, three variations of agent-base implementations for the moifie SIR ynamics are introuce Event-base SIR Moel Our first ABM is a iscrete-event moel, an this moel is built by ecomposing the population of the elaye SIR moel into its iniviual agents. In this ABM, the progress of the isease is moele explicitly for each agent an transmission of the isease occurs through aily contact between iniviual agents. Each agent is escribe by the couple Discrete EVent System specification 30 (DEVS) shown in Fig. 3. A DEVS moel is comprise of atomic an couple moels. An atomic moel is a state transition system as efine in 30, which has a set of states S, a set of input X, an set of output Y. An output function λ : S Y efines the output prouce by a moel when in a given state, an these output occur after an interval given by the time avance function ta : S R + }. The next state of the moel is given by its internal state transition function δ int : S S. It is also possible that the atomic moel will receive an input x after some elapse time e prior to its time avance expiring. In this case, the moel generates no output an its next state is given by the external state transition function δ ext : (s, e) : 0 e ta(s), s S} X S. When an input an expiration of the time avance coincie, then the next state is given by δ conf : S X S. Couple moels escribe how the input an output of atomic moels an other couple moels are interconnecte. Because DEVS is close uner coupling, each couple moel may be reuce to an equivalent atomic moel, an this forms the basis for hierarchical moel construction. A etaile introuction to the DEVS formalism an its simulation proceure can be foun in 30,20. The DEVS moel efine here has three atomic moels: the exposure process, the isease process, an the infectious process. The exposure process initiates isease in the agent upon receipt of an infect event from some other infectious agent. Upon receipt of its first infect event, the exposure process forwars this event to the isease process an the infectious process. The exposure process then becomes inert, thereby making the agent immune to infection through new contact with other, infecte agents. The isease process moels the progression of the isease through its stages, an etermines the outcome for that agent. A sick agent transmits the isease to other agents through its infectious process. The infectious process sens infect events to other agents at a rate etermine by the number of expecte encounters per ay. An infect output generate by the infectious process of agent i becomes an infect input for an agent j i selecte at ranom from the set of agents that are alive at the time of the output. The mathematical formulation of these three processes is given below. Exposure process: The exposure process begins a perio of infection for the agent. The exposure process has the single state variable stage that may take the values 0, 1, an 2. The initial value is stage = 1 if the agent begins with the isease Prepare using sagej.cls

6 6 Journal Title XX(X) an stage = 0 otherwise. If stage = 2, then the agent has alreay been infecte. The ynamics of the exposure process are given by δ int (stage) = 2 δ ext (stage, e, x) = 1 if stage = 0 stage otherwise δ con (stage, x) = 2 (8) λ(stage) = infect 0 if stage = 1 ta(stage) = otherwise Disease process: The isease process carries the agent through the stages of the isease. The isease process has a state variable stage that may take the values Decease (D), Recovere (R), Dying (Dy), Recovering (R ), an Susceptible (S). The initial state is stage = Susceptible. A uniformly istribute ranom variable u with range [0, 1] is use to etermine the outcome of the isease for this agent. The ynamics are δ int (stage) = D R δ ext (stage, e, x) = if stage = Dy otherwise Dy if u pm R otherwise δ con (stage, x) = δ int (stage) (9) λ(stage) = infect tm if stage = Dy ta(stage) = tr if stage = R otherwise Infectious process: The infectious process is responsible for causing other agents to be expose to this agent while the agent is infectious. The infectious process has a boolean value state variable inf ectious; a ranom variable u uniformly istribute in [0, 1] is use to etermine if a particular encounter will lea to infection of the encountere agent; an an exponentially istribute ranom variable µ with mean 1/re is use to etermine the time between encounters. The initial value of infectious is false. The ynamics of the infectious process are given by δ int (infectious) = infectious δ ext (infectious, e, x) = infectious δ con (infectious, x) = infectious (10) infect if u < pt λ(inf ectious) = Φ otherwise µ if infectious ta(inf ectious) = otherwise A Time-steppe SIR Moel with Two Forms The moels escribe above incrementally refine the implementation of the isease sprea process. All of the above moels have two features in common: (i) interactions happen instantaneously an (ii) the state of the moel can change at any instant of the real-value simulation clock. The moels escribe in this section eliminate these two features by using a iscrete-time, rather than continuous time, approach to moel construction. Discrete-time (i.e., time-steppe) moels are a popular type of agent-base moel, an this approach is the basis of most moeling tools for agent-base systems. Our time-steppe ABM is implemente using the Repast framework 16, an it eparts from the above moels (Equation-base moels an Event-base SIR moel) in two specific ways. First, this agent-base moel may elay interactions between agents: one variant of the moel uses instantaneous interactions an the other oes not. Secon, this moel has a iscrete time base an the states of its agents change at fixe instants t 0, t 1, t 2,... that are separate by the moel s time increment h (i.e., t n+1 t n = h). At each step of the simulation, the agents are activate in an orer that is generate at ranom. In Repast, this proceure for activating agents is calle scheuler scramble 17. Upon activation, an Infecte agent raws the number of agents to contact for that time step from a Poisson istribution with mean re. The Poisson istribution is use for two reasons. First, it is necessary to raw the number of people to encounter from a iscrete istribution: an agent cannot encounter part of a person. Secon, inter-encounter times in the event-base moel are rawn from an exponential istribution, an the Poisson istribution is its natural counterpart. Two ifferent approaches, corresponing to two ifferent forms of the time-steppe moel, are use to calculate a new state for the agent an the agents it contacts: Form-1: Asynchronous interaction: In this approach to upating the agents states, if Agent A infects Agent B at time t, then the state of Agent B is upate immeiately. Hence, there can be two cases; (i) Agent B was activate before Agent A, so Agent B waits until t + 1 to infect other agents, or (ii) Agent B is activate after Agent A, so Agent B starts passing along the infection at time t. Form-2: Synchronous interaction: In this process, if Agent A infects Agent B at time t, Agent B becomes infecte at time t + 1 regarless of the orer of agent activation. Moreover, agents observe the state of other agents as the state that existe before agent activation began. For example, if Agent A infects Agent B an the next agent to be activate is Agent C, then Agent C perceives Agent B as susceptible, not infecte. This process allows an infecte agent to pick an alreay infecte agent to pass the infection along. 2.4 Obtaining comparable outputs from EBM an ABM For ABMs escribe here, the state variable stage of the isease process moel is use to construct the quantities Decease, Recovere, Infectious, an Susceptible. For ABMs, the quantities Decease an Recovere in the respective EBMs correspon to the count of agents with stage = D an stage = R, respectively. The quantity Susceptible in EBMs correspons to the count of agents with stage = S in ABMs. The quantity Infectious in the Prepare using sagej.cls

7 Özmen et al. 7 EBMs correspons to the sum of the number of agents with stage = Dy an stage = R in the ABMs. In the time-steppe ABMs, similar to the event-base ABMs, the numbers of agents that match the corresponing state are counte at the en of each time tick. The only exception is the quantity Decease; Decease is just the number of agents that are remove from the environment in the time-steppe moel. 3 Results In this section, we start with comparisons of point estimators. Subsequently, we focus on the evolution of outputs over time. The ifferences between the moel outputs are elineate in both quantitative an qualitative manners. 3.1 Peak loa, iffusion spee, an isease buren Three metrics in particular are important for policy makers when consiering how to respon to an outbreak of influenza. These metrics are peak loa on health services, efine here as the greatest number of infectious population observe over time, iffusion spee, efine as the amount of time that passes from the initial outbreak until the peak loa is observe, an total isease buren efine as the cumulative number of people who ie from the isease at the en of the epiemic. Table 2 compares these metrics across the six moels with parameters for the 1918 flu epiemic as previously escribe. At the 95% confience level, the EBM-Delaye an eventbase ABM are inistinguishable. The iffusion spee an total buren of the EBM-Delaye moel is within the 95% confience interval for the iffusion spee an total buren of the event-base ABM. The peak loa for the event-base ABM is slightly higher than that of the EBM-elaye moel, but only by 79 persons out of 500,000 people in relation to the lower boun on the 95% confience interval. The ifference in the peak loa measures from the two moels is just 0.07% of the 114,480 (out of 500,000 as the initial population; Table 1) of the EBM-Delaye moel, implying that these moels iffer very little with respect to this metric. It is also interesting to note that the other moels iffer significantly with respect to the peak loa values, with the EBM-SIR moel estimating the lowest peak loa compare to the other moels. This is also expecte since the EBM- SIR moel incorporates a significant simplification in the recovery/eath process as aforementione. However, one can observe that the comparisons with other moels are not the same. While there is significant agreement between the Event-base moel an the Timesteppe-Asynch moel, the Time-steppe-Synch moel iffers consierably with respect to the iffusion spee. As such, the Event-base an Time-steppe-Asynch moels suggest that the initial rise in the number of sick people is similar in both cases, implying similar isease propagation. Whereas for the Time-steppe-Synch moel, the isease spreas more slowly reaching a lower peak loa. The variation of iffusion spee is also true for comparisons between the Time-steppe moels an EBM-Delaye moel an comparisons between the three EBMs. The total buren measures an equilibrium state of the moel. Specifically, for the three ifferent EBMs, the total buren measures the overall number of ecease people an for the ABMs, inicates the maximum number of ecease agents when the moel becomes quiescent. Table 2 suggests that the equilibrium states for the six types of moels are very similar, in spite of not sharing any common isease progression trajectories. The two other observables, namely the peak loa an iffusion spee measure transient aspects (see the next section) of the moels, an therefore epen on measurements of the state variables when the erivatives are far from zero (for the EBMs) or the agents are active (for the ABMs). Therefore, we can conclue that the transient ynamics for these six moels are quite ifferent with the exception of the Event-base an EBM-Delaye moels. For these two moels, we shoul expect similar transient behaviors because the two moels agree with respect to peak loa an iffusion spee. 3.2 Transient behaviors To probe the changes in the transient behaviors, we performe aitional analyses on the moels. Figure 4 shows trajectories for each of the six moels with parameters for the 1918 flu epiemic. This figure shows the (S) Susceptible, (I) Infectious, (R) Recovere, an (D) Decease population as the proportion of the total population at the en of each simulate ay. The equation-base moels are eterministic an so the trajectories shown are the moel s only possible output for the given parameters an initial conitions. The agent-base moels are stochastic, an the trajectories shown in Figure 4 represent the mean values an the 95% confience interval to inicate typical evolution of the moels. There is a general agreement in terms of the overall shapes of trajectories generate by each moel. While the trajectories closely resemble one another in terms of the final outcomes (as observe previously in terms of the equilibrium measures), there are significant ifferences regaring their transient ynamics. In Figure 4 (S), it is quite clear that EBM-SIR (orange line) shows S ecreasing much more slowly than o the other moels. Similarly, in Figure 4 (I), (R), an (D), the isease propagation in the EBM-SIR is slower. This iscrepancy between the EBM-SIR an the other moels is anticipate because of the simplification in the EBM-SIR moel. The subsequent corrections to the EBM-SIR moel, first by aing substates an then aing elays, brings the trajectory of the EBM-SIR moel very close to that of the Event-base ABM. It is much more ifficult to explain the ifferences between the EBMs an ABMs. The substantial ifference between the Time-steppe-Synch an -Asynch ABMs is particularly noteworthy. The choice of a synchronous versus asynchronous upating strategy has a large effect on the moels trajectories. The synchronous strategy spreas the isease more slowly, an we hypothesize this observation is ue to the elay between the exposure of an agent to the isease an that agent becoming infectious. This notion is reinforce by the trajectories of the Time-steppe ABM moel that uses an asynchronous upating strategy, for which such a elay oes not exist. Inee, the Timesteppe-Asynch moel much more closely approximates Prepare using sagej.cls

8 8 Journal Title XX(X) Table 2. Peak Loa, Diffusion Spee, an Total Buren of Different Moels Moels EBM-SIR EBM-Substates EBM-Delaye Event-base Time-steppe-Asynch Time-steppe-Synch 1918 Influenza Data Peak Loa (± 95%CI) 55, , , ,800.7(± 241.7) 95,749.0(± 196.7) 87,343.9(± 165.9) - Diffusion Spee (± 95%CI) (± 0.2) 13.1(± 0.1) 18.0(± 0.0) - Total Buren (± 95%CI) 3, , , ,653.0(± 19.4) 3,657.2(± 22.1) 3,662.2(± 23.2) 2,750-3,375 EBMs of this stuy are eterministic, hence o not have variability Influenza ata value is foun by scaling the real-worl estimates own base on the population size use in experiments. CIs assume Normality. Figure 4. Evolution of Main Compartments over Time. (S) Susceptible, (I) Infectious, (R) Recovere, an (D) Decease the trajectories of the EBM-Delaye an the event-base ABM moel. We teste the aforementione hypothesis with simulations of both Time-steppe moels using successively smaller time steps. The results are presente in Figure 5 (A) an (B) for asynchronous upate an in Figure 5 (C) an (D) for synchronous upate moels. Specifically in Figure 5 (A) an (C), it is clear that the iscrepancies between curves graually iminish as the time steps are successively reuce (h = 1.0, 0.5, 0.2, 0.1, an 0.05 ays). This iminishing size of the time step has two effects: first, it causes the iscretization of time in both of the Timesteppe moels to better approximate the continuous time use in the EBMs an the event-base moel; secon, it causes the one-step elay between exposure an infection in the synchronous moel to more closely approximate the instantaneous infection that occurs in the other moels. These two effects are both significant in etermining the outcome of the moel. As the time step is reuce, the trajectories of the Time-steppe moels resemble one Prepare using sagej.cls another more closely, approaching the trajectories of the EBM-Delaye moel, an the trajectories of the Event-base moel. This suggests that the selection of a time base, either continuous or iscrete, has important consequences for a moel s ynamic behavior. Moreover, if the time base is iscrete, then the size of the time step also has a substantive effect on the moel s behavior. Hence, the problem of choosing a representation for time is an important, but often overlooke, aspect of builing an using agent-base moels, most of which use a iscrete time base an rely on the moeler s intuition to choose a time step. The choice of an Asynch or Synch upating strategy has important ramifications. The behavior of both Timesteppe moels are quite istinct when their time step is one ay. This ifference only iminishes as the time step becomes smaller an, consequently, the ifferences between the upate strategies become less significant. This section highlights the importance of choosing when an how agents interact within a moel (see Discussion).

9 O zmen et al. 9 Figure 5. The Comparisons of Different Time Step Decisions for Time-steppe Moels. (A) Infectious - Time-steppe-Asynch, (B) Decease - Time-steppe-Asynch, (C) Infectious - Time-steppe-Synch, an (D) Decease Time-steppe-Synch. The lines represent the mean values. 3.3 Sensitivity to small encounter rates The number of opportunities for a sick person to encounter healthy persons is a major factor in the rate an extent of an epiemic. In our moels, this is capture by the encounter rate. The encounter rate, re, is one of the epiemic relate parameters that has a high impact on the R0 value. Small values of re are of particular interest for comparing ABMs an EBMs. If re is small enough, the EBM will always inicate that the epiemic falters an ABMs might generate highly variable outcomes. The cause for this is clearly apparent in the term for (/t)i in the SIR moel presente in Eqn. 2. Near the start of the epiemic, we may take S/(S + I + R) 1 an so write (/t)i as pm 1 pm I= + (re pt) I (11) t tm tr from which it follows that I grows if, an only if, pm 1 pm + (re pt) > 0 tm tr (12) an rearranging this expression shows that growth requires 1 pm 1 pm re > + (13) pt tm tr For the parameters given in Table 1, the above expression inicates that re > 2.27 is require for growth in the EBMSIR. Note that a similar argument hols for both EBMSubstates an EBM-Delaye. Prepare using sagej.cls Although the absolute lower boun for the isease to propagate within the ABMs is 0, note that with re 2.27 the isease in an ABM is most likely to sprea to a very small number of persons before the epiemic ies out. Hence, the infectious population will grow slightly before shrinking to zero in the event-base moel. This expecte behavior is ifferent from the behavior of the EBM-Delaye where the number of infectious people is strictly iminishing. Moreover, with the ABMs there is a small but efinite chance that the infectious population will become quite large an the isease will sprea very rapily. This is a scenario that cannot be capture by the EBMs. Figure 6. EBM-Delaye vs. Event-base Moel - MAPE between the Cumulative Numbers of Decease.

10 10 Journal Title XX(X) Inee, numerical experiments show that the event-base ABM an EBM-Delaye moels agree for relatively large values of re. However, for simulations with re 2.27 the agreement between EBM an ABM is much less pronounce 23 even though isease propagation is mitigate in both moels. This fact is illustrate in Figure 6 which shows the mean absolute percentage error (MAPE) for the number of Decease when comparing the EBM-Delaye moel an event-base ABM. The MAPE is efine by M = 1 T T t=1 O t E t O t (14) where M is the MAPE, t is the number of ays that the isease sprea continues, an O t an E t are the cumulative number of Decease at ay t for EBM-Delaye an Eventbase moels, respectively. In Figure 6 it can be seen that the level of isagreement increases significantly when the re value falls below 2.27, an outcome that was anticipate by the above analysis. 4 Discussion The primary goal of our stuy is to unerstan the extent to which three moeling choices, namely (i) the choice of moel trajectories an temporal resolution (continuous versus iscrete-event moels), (ii) coupling between agents (instantaneous versus elaye interactions) an (iii) progress of patients from one stage of isease to the other, affect the overall isease progress an temporal ynamics/outcomes from these epiemiological moels. Regaring ifferential equation moels, error an uncertainty cause by numerical calculations (relate to (i)) are well-stuie 21. Although, there are stuies that aime to generate the exact same behavior from system ynamics moels an their agentbase counterparts 4,19, there are no clear guielines on specific choices in moel evelopment processes involve. To eluciate the effect of these choices, we introuce six variations of isease sprea moels that mimic the moifie SIR ynamics. First, we iscusse the specific implementation routes taken for each moel. The six moels evelope here capture key features within alreay establishe influenza moels. By using moel parameter values erive in previous stuies for 1918 Influenza, we generate a realistic test case for comparing moel outcomes. We showe that both ABMs an EBMs reach similar equilibrium states, however their transient ynamics may iffer significantly base on the moeling principles chosen. In particular, within time-steppe moels, the ynamic behavior has a strong epenence on the choice of time-step an type of interaction between the agents (i.e., synchronous or asynchronous). However, these strong epenencies are rarely consiere when constructing agent-base moels, for which an a hoc selection of the time-step an interaction moel is most typical. Our results suggest that an a hoc selection is not appropriate in the general case. Inee, if a strong justification exists for a specific choice of time-step or type of interaction, either on theoretical grouns or on the basis of available ata, then this justification is a crucial part of the moel s unerlying theory. If this selection is a hoc, then the time-step an type of interaction shoul be subjecte to sensitivity stuies like those applie to the moel s other parameters. We emonstrate that while ABMs an EBMs can agree for a wie range of scenarios, this agreement can breakown in cases where the isaggregate population in the ABM enables ynamics that are not realizable within EBMs that agrees with other stuies 15,24. An example of this was emonstrate for small values of re in our influenza moels. When re is sufficiently small, an so encounters between agents are relatively infrequent, the EBM imposes an irreversible ecline in the number of infectious persons. Conversely, the ABM permits small rises in the number of infections uring the course of a long term ecline. An experimental comparison of MAPEs reveals that this effect causes a noticeable ifference in the moel trajectories at re 2.3 when using parameters from the 1918 influenza panemic. Our moel evelopment process also reveals why the previous works 15,24 were not able to generate system ynamics results of SIR ynamics for homogenous populations an imprecisely conclue that stochastic variability cause the iscrepancy among the trajectories. Because when a continuous istribution (i.e., exponential) is use for tr in the agent-base moel an the time-step truncates samples of this istribution (iscretize), the mean of the istribution is ifferent in these moels. Our results suggest that if one nees to reprouce the results of an EBM by a time-steppe ABM, then iscrete counterparts of continuous istributions shoul be use an all timeepenent variables shoul be efine in terms of the timestep. Even then, for relatively large time-steps, we shoul not expect the time-steppe moel to agree with continuous time moels. Aitionally, a small number of initial infectious people (etermine as 1 in 15 ) an lower R 0 values contribute to the isagreement between EBMs an stochastic ABMs. This work aime to elineate some of the funamental, but often overlooke, assumptions that are ecie in the earliest stages of moel evelopment, an to ientify the impact of these assumptions on the behavior of the resulting moel. These initial ecisions shoul be explicitly an carefully consiere an recore when builing an ocumenting an agent-base moel. The results of our stuy suggest that, if agreement between an ABM an EBM is esire (e.g., for the purpose of moel verification), then it is important for the ABM to use continuous time an instantaneous interactions. These help the ABM to better mimic the continuous evolution of the EBM. Perhaps surprisingly, this is true espite the iscrete states of the ABM. For iscretetime moels, the moels shoul be carefully constructe with iscrete istributions. However, we cannot give specific guielines for selecting an appropriate time-step ue to its omain an problem efinition epenence. But our results suggest that this is an important topic an a venue for future research. Acknowlegements The authors woul like to thank the Defense Threat Reuction Agency (DTRA) for the support fune uner the interagency agreement with Department of Energy (DOE) for DOE proposal number 2216-V as authorize by DOE contract number DE-AC05-00OR The contents Prepare using sagej.cls

11 Özmen et al. 11 of this publication are the responsibility of the authors an o not necessarily represent the official views of DTRA. References 1. Ajelli, M., B. Gonçalves, D. Balcan, V. Colizza, H. Hu, J. J. Ramasco, S. Merler, an A. Vespignani (2010). Comparing largescale computational approaches to epiemic moeling: Agentbase versus structure metapopulation moels. BMC Infectious Diseases 10(1), Auchincloss, A. H. an A. V. Diez Roux (2008). A new tool for epiemiology: The usefulness of ynamic-agent moels in unerstaning place effects on health. American Journal of Epiemiology 168(1), Balcan, D., B. Gonçalves, H. Hu, J. J. Ramasco, V. Colizza, an A. Vespignani (2010). Moeling the spatial sprea of infectious iseases: The GLobal} epiemic an mobility computational moel. Journal of Computational Science 1(3), Borshchev, A. an A. Filippov (2004). From system ynamics an iscrete event to practical agent base moeling: reasons, techniques, tools. In Proceeings of the 22n international conference of the system ynamics society, Volume 22. Citeseer. 5. Chowell, G., C. E. Ammon, N. W. Hengartner, an J. M. Hyman (2006, July). Transmission ynamics of the great influenza panemic of 1918 in Geneva, Switzerlan: Assessing the effects of hypothetical interventions. Journal of Theoretical Biology 241(2), Chowell, G., M. Miller, an C. Vibou (2008). Seasonal influenza in the unite states, france, an australia: transmission an prospects for control. Epiemiology an infection 136(06), Crosby, A. W. (2003, July). America s Forgotten Panemic. The Influenza of Cambrige University Press. 8. Gani, R., H. Hughes, D. Fleming, T. Griffin, J. Melock, an S. Leach (2005). Potential impact of antiviral rug use uring influenza panemic. Emerging infectious iseases 11(9), Hyer, A., D. L. Buckerige, an B. Leung (2013, June). Preictive Valiation of an Influenza Sprea Moel. PLoS ONE 8(6), e Jaffry, S. W. an J. Treur (2008). Agent-Base an Population- Base Simulation: A Comparative Case Stuy for Epiemics. In Proc. of the 22th European Conference on Moelling an Simulation, ECMS, pp Citeseer. 11. Johnson, N. P. an J. Mueller (2002). Upating the accounts: global mortality of the Spanish influenza panemic. Bulletin of the History of Meicine 76(1), Keeling, M. an P. Rohani (2008). Moeling Infectious Diseases in Humans an Animals. Princeton University Press. 13. Kenney, R. C., X. Xiang, T. F. Cosimano, L. A. Arthurs, P. A. Maurice, G. R. Maey, an S. E. Cabaniss (2006). Verification an valiation of agent-base an equation-base simulations: a comparison. SIMULATION SERIES 38(2), Kermack, W. O. an A. G. McKenrick (1991). Contributions to the mathematical theory of epiemics-i. Bulletin of Mathematical Biology 53(1 2), Macal, C. M. (2010). To agent-base simulation from system ynamics. In Simulation Conference (WSC), Proceeings of the 2010 Winter, pp IEEE. 16. North, M., T. Howe, N. Collier, an J. Vos (2007). A eclarative moel assembly infrastructure for verification an valiation. In Avancing social simulation: The first worl congress, pp Springer. 17. North, M. J. an C. M. Macal (2011). Prouct esign patterns for agent-base moeling. In Simulation Conference (WSC), Proceeings of the 2011 Winter, pp Nsoesie, E. O., R. J. Beckman, an M. V. Marathe (2012, October). Sensitivity Analysis of an Iniviual-Base Moel for Simulation of Influenza Epiemics. PLoS ONE 7(10), e Nutaro, J., O. ozmen, an J. Schryver (2014). Disaggregation an refinement of system ynamics moels via agent-base moeling. In Proceeings of the 2014 Summer Simulation Multiconference, pp. 11. Society for Computer Simulation International. 20. Nutaro, J. J. (2010). Builing Software for Simulation: Theory an Algorithms, with Applications in C++. Wiley. 21. Oberkampf, W. L., S. M. DeLan, B. M. Rutherfor, K. V. Diegert, an K. F. Alvin (2002). Error an uncertainty in moeling an simulation. Reliability Engineering & System Safety 75(3), Pullum, L. L. an O. Ozmen (2012). Early results from metamorphic testing of epiemiological moels. In BioMeical Computing (BioMeCom), 2012 ASE/IEEE International Conference On, pp IEEE. 23. Pullum, L. L., A. Ramanathan, S. Sukumar, J. Nutaro, O. Ozmen, C. Stee, an X. Cui (2013). Verification an Valiation of Agent-Base Disease Sprea Moels. Technical report, Oak Rige National Laboratory. 24. Rahmana, H. an J. Sterman (2008, May). Heterogeneity an Network Structure in the Dynamics of Diffusion: Comparing Agent-Base an Differential Equation Moels. Management Science 54(5), Skvortsov, A. T., R. B. Connell, P. Dawson, an R. Gailis (2007, October). Epiemic Moelling: Valiation of Agentbase Simulation by Using Simple Mathematical Moels. In Proceeings of the Congress on Moelling an Simulation, MODSIM, pp Sukumar, S. R. an J. J. Nutaro (2012). Agent-base vs. equation-base epiemiological moels: A moel selection case stuy. In BioMeical Computing (BioMeCom), 2012 ASE/IEEE International Conference on, pp IEEE. 27. Taubenberger, J. K. an D. M. Morens (2006) Influenza: the mother of all panemics. Rev Biome 17, Van Wave, T. W., F. D. Scutchfiel, an P. A. Honoré (2010). Recent avances in public health systems research in the unite states*. Annual Review of Public Health 31(1), PMID: Wearing, H. J., P. Rohani, an M. J. Keeling (2005, 07). Appropriate moels for the management of infectious iseases. PLoS Me 2(7), e Zeigler, B. P., H. Praehofer, an T. G. Kim (2000). Theory of Moeling an Simulation, Secon Eition. Acaemic Press. Prepare using sagej.cls