Bayesian Networks Improve Causal Environmental Assessments for Evidence-Based Policy

Size: px
Start display at page:

Download "Bayesian Networks Improve Causal Environmental Assessments for Evidence-Based Policy"

Transcription

1 pubs.acs.org/est Bayesian Networks Improve Causal Environmental Assessments for Evidence-Based Policy John F. Carriger,*, Mace G. Barron, and Michael C. Newman Oak Ridge Institute for Science and Education, U.S. Environmental Protection Agency, Office of Research and Development, National Health and Environmental Effects Research Laboratory, Gulf Ecology Division, 1 Sabine Island Drive, Gulf Breeze, Florida 32561, United States U.S. Environmental Protection Agency, Office of Research and Development, National Health and Environmental Effects Research Laboratory, Gulf Ecology Division, 1 Sabine Island Drive, Gulf Breeze, Florida 32561, United States College of William & Mary, Virginia Institute of Marine Science, P.O. Box 1346, Route 1208 Greate Road, Gloucester Point, Virginia 23062, United States ABSTRACT: Rule-based weight of evidence approaches to ecological risk assessment may not account for uncertainties and generally lack probabilistic integration of lines of evidence. Bayesian networks allow causal inferences to be made from evidence by including causal knowledge about the problem, using this knowledge with probabilistic calculus to combine multiple lines of evidence, and minimizing biases in predicting or diagnosing causal relationships. Too often, sources of uncertainty in conventional weight of evidence approaches are ignored that can be accounted for with Bayesian networks. Specifying and propagating uncertainties improve the ability of models to incorporate strength of the evidence in the risk management phase of an assessment. Probabilistic inference from a Bayesian network allows evaluation of changes in uncertainty for variables from the evidence. The network structure and probabilistic framework of a Bayesian approach provide advantages over qualitative approaches in weight of evidence for capturing the impacts of multiple sources of quantifiable uncertainty on predictions of ecological risk. Bayesian networks can facilitate the development of evidence-based policy under conditions of uncertainty by incorporating analytical inaccuracies or the implications of imperfect information, structuring and communicating causal issues through qualitative directed graph formulations, and quantitatively comparing the causal power of multiple stressors on valued ecological resources. These aspects are demonstrated through hypothetical problem scenarios that explore some major benefits of using Bayesian networks for reasoning and making inferences in evidence-based policy. INTRODUCTION The term weight of evidence strongly connotes the significance of evidence applied to judgments and decisions. Its meaning is underscored by the general notion that any major inference or decision with important personal or societal impacts should rely on a requisite assessment of the best information available. The role of weight of evidence and evidence synthesis can be seen in diverse fields such as criminal and civil law, healthcare, policy, business, economics, and science. It can be reasonably assumed that weight of evidence will have a universal role in the future of civil societies. The field of evidence synthesis has advanced the understanding of how lines of evidence can be collected to make informed decisions so much so that they routinely impact the development of weight of evidence in contemporary policymaking. 1 Since 2003, one nongovernmental organization, the Collaboration for Environmental Evidence, has expanded the implementation and dissemination of environmental evidence synthesis for a variety of important policy issues such as those linking biodiversity and poverty. 2 4 Evidence weighing is the motivation behind collection and analysis of information in ecological risk assessment. Numerous lines of evidence and causally related hypotheses are the norm in environmental protection and restoration. Environmental policy decisions use evidence with varying degrees of formality to establish where resources should be invested before a decision is made, whether results are being achieved during implementation, and the successes or failure after decision implementation. As an example of the first situation, the World Bank developed tools to help predict impacts of funding decisions on coastal receptors and social-biophysical endpoints. 5,6 As an example of the second situation, the U.S. Fish and Wildlife Service utilized an adaptive management framework for establishing waterfowl harvesting targets that iteratively incorporated population monitoring data to reduce uncertainties in predictive capabilities. 7 For the third situation, Received: June 29, 2016 Revised: September 27, 2016 Accepted: November 14, 2016 Published: December 8, American Chemical Society 13195

2 evidence-based procedures were used by the Australian Landscape Logic project to examine the success of past environmental management investments on social and ecological endpoints and to use this information for future interventions. 8 One common approach used in these examples is Bayesian inference. The World Bank describes a Bayesian network (BN) approach to lending decisions that can be implemented across a variety of possible coastal impacts. 6 Additionally, the U.S. Fish and Wildlife Service utilized Bayes theorem to compare multiple models to observed population trends and establish model weightings based on predictive accuracy for subsequent harvesting decisions. 7 Moreover, in Landscape Logic, BNs were one of the modeling approaches used for synthesizing evidence from multidisciplinary research endeavors and for examining the effectiveness of different interventions on the environment. 8 Other useful examples of Bayesian applications to complex environmental assessment problems include but are not limited to Amstrup et al., 9 Bayliss et al., 10 and Pollino et al. 11 This article highlights opportunities for using Bayesian tools to combine evidence during ecological risk assessment. This work demonstrates how probabilistic and causal reasoning about complicated problems are accommodated in BNs and discusses how a causal BN approach would benefit future evidence synthesis in environmental management and risk assessment by improving inferences under high uncertainty. First, the mathematical properties of Bayes theorem for updating beliefs about a hypothesis based on evidence are introduced. The discussion is then broadened to environmental measurements and how Bayes theorem can be utilized to incorporate the explanatory potential of an experiment and its implications for a riskbased hypothesis. A methodology is introduced that examines accuracy of multiple observational methods and confidence in a hypothesis derived from observational inference. Lastly, inclusion of interventional inference is discussed for testing causal assumptions and the causal power of interacting stressors in a complex system. The conclusion describes the implications of the inferential properties of BNs and several remaining issues. Introduction to Bayesian Networks. Bayesian statistics provide a framework for updating what is known from available information. This process can be translated to weight of evidence approaches in a straightforward manner. The diachronic Bayes theorem can be expressed as P( HE ) = P( H) PEH ( ) PE ( ) The posterior probability (P(H E)) calculated by Bayes theorem represents what is known about how likely a hypothesis (H) is to be true given the observed evidence (E). The prior probability, or P(H), is what is known about how likely a hypothesis is to be true before the evidence is observed. The additional terms weight the prior beliefs in the hypothesis by the degree it is maintained by the evidence with P(E H) being the likelihood function for the evidence given the hypothesis and P(E) being the marginal probability of the evidence, which is constant for all hypotheses. The P(E H) is normalized by P(E) in the likelihood ratio so that the posterior probability, P(H E), is equal to or less than 1. Both frequentist and Bayesian approaches to statistics make use of the likelihood function, but the likelihood can only provide a measure of the strength of the evidence itself given a particular hypothesis is true. Bayes theorem provides a valuable means of using evidence to evaluate the strength of a hypothesis through a probabilistic framework. Bayesian reasoning is coherent with ecological risk assessment where vested parties establish risk-based hypotheses and use past and future evidence to update beliefs about the hypotheses. It also provides a useful alternative to commonly practiced simple scoring procedures in ecological risk assessment that are difficult to integrate and have been implicated in major failures of risk management for other fields. 12 Bayesian networks are probabilistic graphical models representing the joint probability of a distribution. Bayes theorem can become cumbersome to use with numerous hypotheses supported by different lines of evidence, but BNs are more useful for visualizing and providing solutions for such problems. 13 Their structure consists of a series of nodes (variables) connected by arcs in a directed fashion. For all nodes in a BN that are directly connected with arcs, conditional probabilities are used to represent the probability of each possible event in a child node given each possible event in the parent nodes. A simple BN is illustrated with two nodes and the calculations used for the conditional probability table (Figure 1). These conditional probabilities express uncertainty Figure 1. Probability (expressed as %) of a pesticide being present in a water sample given a positive monitoring result. (a) Bayesian network with conditional probability table (above Test Result node) built using background evidence provided in Rizak and Hrudey; 20 (b) probability the pesticide is present given a positive result. Gray nodes have hard evidence placed into a state of the variable indicating that it has been observed (100% chance of being true). Tan nodes do not have hard evidence placed in any variable state. After inputting evidence to a network, posterior probabilities are calculated in an omnidirectional fashion so inferences can be made in or against the direction of the arcs depending on the structure of the network and any other hard or soft evidence in the network. in the relationships and are themselves conditioned on background knowledge which would change probability assignments as the knowledge base increases. 14 A fully developed model with all probabilities will initially provide prior probabilities for each variable based on the background information for the problem. Unlike other analytical approaches, the BN variables can be updated to posterior probabilities with findings on any node in an omnidirectional fashion. 15 As a consequence, three types of inferences that lead to posterior probabilities can be made with BNs. The first is predictive inference (forward direction- from parent to child node). The second is diagnostic inference (backward direction- from child to parent node), and the final is mixed inference (forward and backward). As will be 13196

3 discussed, it is also possible to frame a BN so as to separate observational from interventional inferences. The former is particularly useful for examining what can be inferred from monitoring data and environmental observations, and the latter is useful for examining the causal strengths of relationships between variables. Several features of BNs make them valuable for complex weight of evidence problems. One is their graphical building block or connective properties for problems with complex causal chains, such as, social-ecological knowledge representations. 16 Their graphical feature is a powerful representation, reasoning, and communication tool. The assumptions of a BN such as those related to conditional independence (see Pearl 17 for details on assumptions) facilitate tractability in complex model construction and development. The second is the robust capability for characterizing the problem in a probabilistic context. Each linkage between nodes is established through a conditional probability table. The implications of the table are that contrasting claims can be considered about the relationships between different variables, and these claims are reflected in the probabilities for all possible scenarios. Thus, counterfactual possibilities are naturally integrated into the BN process for each element or object within a model so that analysts can piece together the uncertainties for a range of possibilities in a given context. A related advantage of the probability tables is the ease in which data and procedures from domain experts can be used for quantifying the relationships. The information used to build the conditional probabilities includes many sources of information already used in ecological risk assessment problems including environmental measurements, experimental evidence, model simulations, analogous literature studies, and/or expert opinion. The third involves the inferential possibilities with a compiled network. Posterior probabilities are propagated throughout the model and can be utilized for posing observational and interventional inferences among the causal connections. Probabilistic inference with BNs is useful for examining multiple scenarios of interest and the sensitivity of the relationships among sets of causally related variables. The first step in deriving BNs is to ensure that the most useful structure is being developed to model and make inferences about the situation. In practical applications, BNs can be quite complex. Neil et al. 18 identified model fragments or idioms that fit many problems and assist in constructing coherent models. Later work by Fenton et al. 19 specified idioms for legal reasoning. These idioms were developed to understand how crime scene evidence impacts the plausibility of a perpetrator committing a crime when combined with additional causal factors that could influence a hypothesis of guilt or innocence, such as motive and opportunity. Two of the idioms from Neil et al. 18 will be highlighted here due to their potential in weight of evidence risk assessments. The first is the measurement idiom which represents the measurement inaccuracies in observations. The second is the cause-consequence idiom which represents a causal connection between two variables. The latter is the most common idiom for knowledge representation with BNs. 18 Building multilayered networks tracing root causes to ultimate effects fulfills a role for model application in many problems. These two idioms alone can broadly represent many key factors for model-building in evidence synthesis. Problem Scenario: Determining Whether a Detection Supports Presence of a Stressor. The value of the measurement idiom in risk assessment can be illustrated by extension of a survey of 352 Australian professionals. 20 In the original survey, 42% of those surveyed were involved in water quality monitoring and most had more than a decade of experience. Thirty percent worked at water utilities. Each was presented with the following pesticide detection scenario and asked to estimate their level of confidence that the pesticide was truly present in the tap water sample. [Hypothesis scenario] Monitoring evidence for a(n) [Australian] city has indicated that in treated drinking water, a pesticide, say atrazine, is truly present above the recognized standard methods detection limit once in every 1000 water samples from consumers taps. 95% of tests will be positive for detection when the contaminant is truly present above the detection limit, and 98% of tests will be negative for detection when the contaminant is truly not present above the detection limit. Q. With these characteristics, given a positive result (detection) on the analytical test for the pesticide in the drinking water system, how likely do you think the positive result is true? Almost certain Very Unlikely (95 to 100%) (5 to 20%) Very likely Extremely unlikely (80 to 95%) (0 to 5%) More likely than not Do not know (50 to 80%) Less likely than not (20 to 50%) Most respondents chose almost certain or very likely (80 to 100%) when, as will be shown shortly, the correct response was extremely unlikely (0 to 5%). In addition, from an open-ended follow-up question, approximately 15% of the respondents from both the water professional and academic groups indicated they would take what Rizak and Hrudey 20 called an alarmist response to the monitoring result including taking the water treatment plant offline, initiating public warnings and advisories, or shifting to another source of water. Between 2009 and 2014, one of us (MCN) surveyed environmental students and professionals in southern China (2009, 50 students in the advanced course, Eutrophication and Environmental Risk Assessment at Xiamen University), central China (2010, 26 graduate students in a practical environmental statistics course at Huazhong Normal University), Spain (2010, 174 attendees of SETAC Seville International Meeting), southern India (2011, 57 attendees of India Erudite Program lecture at Cochin University of Science and Technology), and the USA (2014, 37 attendees of invited lecture at the University of Georgia s Savannah River Ecology Laboratory). Metaanalysis (random model) of the results indicated that 10% of the responders chose the correct answer, and most (90%) chose almost certain or very likely. Water quality experts and environmental science students and professionals surveyed from around the world consistently misinterpreted simple measurement data. Using percentages as measures of belief, their answers were off by roughly 90%/2.5% or 36-fold. Several extremely common heuristic mistakes likely contributed: inverse probability error, 21 base-rate fallacy, 22,23 representativeness (if their probabilities were representative of the power of the test), 24 and perhaps, the Ellsberg Paradox. 25 It would be unsound to apply some analytical method to measure the concentration of a pollutant if that method was biased by 36-fold and a better method was available. A sampling 13197

4 of educated environmental scientists provided consistently inaccurate judgments of this magnitude in the above survey due to pervasive cognitive errors. It would seem reasonable to use a better available tool for such judgments, that is, BNs. The measurement idiom is illustrated in this example (Figures 1a and 1b). A prior BN is used as a starting point before evidence is entered into the network regarding a positive test result for a pesticide (Figure 1a). From the prior information available, there is a 1 in 1,000 chance (0.10% probability) of the pesticide being truly present and a ( ) 100 or 2.09% chance that any measurement of the water for a pesticide will give a positive result. There is also a ( ) 100 or 97.9% chance of a negative test. From this prior BN, a scenario where a positive result is found, and what may be inferred about the pesticide being present given this positive test result, can be examined (Figure 1b). The probability that the pesticide is present given a positive detection (4.54%) was demonstrated in Chart 1 of Rizak and Hrudey 20 where it was referred to as the test positive predictive value. Newman 26 expanded the example in Rizak and Hrudey 20 to one where additional samples are subjected to a more precise analysis and combined with the output from the initial test. The initial BN that tested a single sample is updated to include this new evidence (Figure 2). The prior probability that the pesticide is present or absent has not changed as expected, but an additional node is added to test the implications from a second test of the water supply (Figure 2a). Entering the evidence of two positive test results provides the answer given in Newman 26 (Figure 2b). The analyst is now 70% confident that the pesticide is in the water supply. (An urn standard may be used to assist in interpreting these probabilities as described in Aven and Reniers.) 27 A third sample taken would increase her confidence to greater than 99% (Figure 2c). In practice, these and similar BNs are not only useful for examining the power of the evidence in disconfirming a hypothesis but also for examining and developing a measurement process ahead of time that will provide reliable information for the purposes it is being developed for. This notion is expanded in the next section to more complicated problem examples. Problem Scenario: Inferences on Bird Deaths at a Wind Farm. A crucial question in weighing evidence is how to tell if observations match reality. Retrodictive assessments attempt to understand environmental injury from human or natural activities through evidence of damage. A BN for representing the number of birds killed at a wind farm can be depicted for retrodictive assessments with evidence (Figure 3). Figure 3. A measurement accuracy model fragment for estimating bird mortality based on recovered carcasses and a specific sampling strategy. The breakout table from the Found carcasses node is the underlying conditional probability table. The numbers found at the bottom of the nodes below the probability distributions refer to the mean ± the standard deviation (e.g., 7.5 ± 1.4) for continuous variables. Figure 2. Network for calculating whether a pesticide is present in a water sample given multiple positive monitoring results: (a) Bayesian network built using background evidence provided in Rizak and Hrudey 20 and Newman; 26 (b) probability the pesticide is present given two positive results; (c) probability the pesticide is present given three positive results. In this example, a wind farm is developing a measurement system for identifying the number of birds killed during operations. The method relies upon collecting and counting the number of carcasses found at ground level. To test the accuracy of this procedure, calibration exercises are initiated similar to ones described in Bispo et al. 28 The assumptions are that sampling is not continuous and carcasses can be unidentifiable due to removal by scavengers or decomposition and searcher inefficiencies. For carcass removal, tests are used to examine how many carcasses remain in an area and at what time period they were removed. For search efficiencies, bird carcasses are placed in different areas around the wind farm and a trial monitoring estimates how many of these dead birds are found. This information is then combined with other influential factors related to the search characteristics and used to estimate the number of birds that would be detected given the number that were killed. Estimating bird deaths is an active area of research 13198

5 that will continue to develop in the future, but multiple measurement functions have been deployed with various assumptions. 29,30 The networks and probabilities constructed in this and the next scenarios are for example purposes and to illustrate the inferential properties of BNs. The study results discussed above are used to create a BN (Figure 3) and populate the probabilities including the probability of detection given that sampling is more or less intensive which could influence the detectability of the carcasses. The latter node called Sampling strategy is necessary to specify the uncertainty in the evidence from the measurement process and how this influences the determination of bird mortalities. 31 The conditional probability table for the Found carcasses node contains the output from the functions built on the sampling information collected from the calibration exercises. Reading across the table gives each probability of obtaining a certain number of carcasses given the actual number of birds that died and the sampling scheme used for monitoring the birds. For example, a 97% chance is given of finding 0 to 5 birds with an intensive sampling strategy when 0 to 5 birds are actually killed. There is also a slight chance of false positives where birds may die within the area due to other causes. This can also be seen in the next row that provides the probabilities of finding different carcasses if 5 to 10 birds die and the sampling strategy is intensive. For this scenario, there is a 47% chance of finding fewer birds (0 to 5) and a 51% chance of finding 5 to 10 carcasses. There is a 1% chance each of finding 10 to 20 and greater than or equal to 20 dead birds. From the quantitative information in the conditional probability table and the network structure, observational inference is used as done in the previous example to update probabilities of the range of birds that have been killed (Figure 3). This time, the prior possibilities for bird mortality (0 to 20 bird deaths) were all weighted equally, indicating complete ignorance outside of what is found during the measurement process. From an observation of 5 to 10 birds killed and an intermittent sampling strategy, the likelihood that more birds were killed than observed is high. Though clear evidence is provided of bird deaths, the observed number of birds does not lead us to conclude that a similar number were actually killed. The idioms discussed thus far have a risk hypothesis as a root node, that is, a node without parents. This allows users to establish an unconditional prior belief on the risk hypothesis. 19 More commonly, prior conditions such as exposure events will influence an assessment endpoint in a risk problem and thus the prior probability of a negative outcome. The causal influences of several supporting hypotheses for estimating the cumulative number of bird deaths may be usefully examined with a BN (Figure 4). In addition, several other measurement processes can assist in determining how many birds were killed Figure 4. Bayesian network for measuring the cumulative number of bird mortalities at a hypothetical wind farm. Gray nodes are observable nodes with evidence placed into them. Tan nodes are unobservable problem variables

6 at the wind farm. The first provides video evidence of any bird collisions with wind rotors. The second provides radar observations for determining the number of birds at risk from rotor operation. Measurement accuracies are established for each of these processes and corresponding hypotheses pertaining to birds in the vicinity of the wind farm and birds colliding with the rotors. Other variables that are useful for estimating mortalities are included such as the season of operation and the presence of wind gusts. An important aspect of this example is how the problem is structured using both a cause-consequence idiom for the unobservable and background variables of interest and the measurement idiom for the uncertainties in the measurement process for each hypothesis. Influential factors for determining the number of birds killed are naturally included, leading to improved reasoning with the measurement data. Without including these causal factors, a user could reason incorrectly by assuming that no bird deaths occurred and therefore risk was low, whereas the explanation might have been that fewer birds were in the vicinity due to some seasonal influence. Moreover, the BN can be used to better consider where in the inferential process a measurement is and the correct hypothesis to attribute it to. The cause-consequence and measurement idioms can be used to piece together the hypotheses and evidence sources in a manner that is useful for aiding judgments in many situations. 19 Using BNs for developing risk-based monitoring programs provides considerable opportunities for improved monitoring practices and inferences from observations. QUALITATIVE AND QUANTITATIVE SUPPORT: BAYESIAN NETWORKS IN CAUSAL ASSESSMENTS Comprehensive risk assessment procedures should use the best available tools and evidence for assessing causality. The problems with establishing a formal causal relationship between variables is well-known in science and statistics. Practical and logically sound insight needs to be brought to bear on any causal problem, especially ones dealing with environmental issues where high uncertainty is the norm. Spurious correlations often arise due to measurement deficiencies, lack of expertise for reasoning with the available information, and/or incorrectly weighted prior beliefs. On a structural level, misperceptions can occur when observations for two variables are tightly associated due to indirect causal factors. Causality is a central theme in science just as in evidencebased policy making. Statistical inference is often built on an assumed causal structure 32 just as management interventions are built on a causally related system of varying degrees of explication. Evidence-based policy seeks causal explanations of what might have happened in the past or will happen in the future. These explanations are stochastic in that any answer is uncertain. 33 Scientific judgments that attribute causality often are made deliberately after repeated evidence shows a strong causal association between two variables under conditions that would provide proof of a causal theory and withstand scrutiny. The information used to make these determinations can come from both observational and interventional studies such as randomized experiments. Environmental management decision making uses causal knowledge to design interventions and weight the impacts of multiple stressors on receptors; causal understanding of a system and the impacts of interventions are a key component of the management process. Similar to judicial decisions, evidence-based policy making in environmental management is a time sensitive activity that cannot always wait for the best available science to be completed for defining causal relationships. 34 This does not mean that causality should be ignored or implicitly assumed in complex policy problems. Quite the opposite, it should often be made explicit with the best available information, key uncertainties, and structural causal assumptions. Probabilistic reasoning is well-established for combining uncertainties and is the foundation of statistics and risk assessment. Probabilistic modeling alone is insufficient for causal analysis unless augmented by system knowledge. 35 For example, probabilistic modeling will not differentiate between whether an ecosystem outcome was caused by a stressor or a stressor was caused by an ecosystem outcome beyond the conditional strength of the relationship. 35 Both of these structures can properly encode the joint probability of the two variables and allow for appropriate inferences. Accounting for a causal relationship requires evidence for both a causal mechanism for an effect, on a probabilistic level, and the strength of an association. 36 The representation of causal interactions including possible outcomes, known confounding factors, and relationships between unobserved problem variables and measurements are key building blocks for useful causal inferences. Causal BNs can formalize causal possibilities as a graphical BN that makes these assumptions apparent for invalidation. 35 Graphical models not only can communicate the potential issues in causality but also can examine the evidence for causality through improved measurements to test the potential relationships. But proof of causality is often elusive, making plausibility a more useful criterion. 37 Furthermore, causal operations are most usefully measured probabilistically (e.g., malaria infections do not always lead to organ failures) and are more likely to provide evidence of changes in probabilities for effects and not on definitive occurrences if manipulated in experimental or observational contexts. Problem Scenario: Causal Explanations from Correlated Data - Fish Population Reductions and Contaminant Concentrations. Direct and indirect effects are critical issues in environmental management problems. 38 In a simple matter of two variables being associated with one another, numerous possibilities arise in an open environmental system that might explain the causal mechanism for the associations. This complexity can be compounded because observational data might not capture the temporal sequences of occurrences at a resolution for determining the plausible causal mechanisms. Formal structures like BNs provide greater clarity in situations with ambiguous causality by revealing potential causal mechanisms. This example (adapted from Neapolitan) 39 illustrates several instances of the cause-consequence idiom and causal possibilities for two variables associated with one another on a temporal and spatial scale. Assume that a pesticide (A) and a fish population (B) have an inverse linear relationship in an aquatic system where higher pesticides are associated with low fish abundance and low pesticides are associated with high fish abundance. A team of analysts has come up with four different possible nonmutually exclusive explanations that could plausibly explain this association but requires further analysis for resolution. One causal relationship that explains this association is A B where A is a cause of B (i.e., the pesticide is reducing fish through mortality). However, one may induce a different causal possibility: that B is causing A or A B. In this belief, the fish might bioconcentrate the pesticide when present. A common issue is a third variable (C) that is causing both B and A (A C B). Hydrodynamics (C) could be dictating 13200

7 the inverse relationship between the presence of the fish (habitat quality) and the presence of the pesticide (transport). A final scenario could be a common effect variable (D) which in different states, such as present or absent, causes A and B to be associated (A D B). Crustacean prey abundance (D) might be low from direct pesticide effects, causing fish to not aggregate in the area. All, some, or none of these factors might be true, and monitoring could be improved to better capture the temporal dynamics. A cohesive directed causal model might be developed out of these causal arguments and further investigated, or the implications of several candidate structures could be examined if structural possibilities remain unspecifiable by one cohesive model. 40,37 Either way, the directed graph representation makes the causal issues clear from the structural assumptions, can enhance communication and clear thinking about the problem when causes and effects are an issue, and can maintain analytical focus on the implications from the structural issues. Problem Scenario: Forward Causal Reasoning on Risks to a Recreational Fish Population. Even when a useful causal modeling structure has been constructed, hypotheses are often difficult to resolve about the causal influence between problem variables. An example causal BN was developed for assessing the abundance of a recreational fish population (Figure 5). The structure of the network uses the Figure 5. Causal model with prior probabilities for investing relevant risk factors to breeding adult trout abundance in an aquatic system. cause-consequence format to study the impacts of biological and chemicals stressors on a fish population. For many BN applications, the conditional probabilities can be estimated with existing data using expressions for each of the relationships similar to how a Monte Carlo analysis is developed. Another common approach relies on nonparametric maximum likelihood or Bayesian methods to establish conditional probabilities from a case file of occurrences. 41 The questions of interest in this scenario pertain to the relative influence of parasitic lampreys and polluted sediment on adult fish abundance. For simplicity, the variables are placed in three discrete levels of low, medium, or high, and the model is assumed to be spatially and temporally coherent for all relationships. The implications of using observational inference for assessing causal questions will now be demonstrated (Figure 6). An observation of high Parasitic lamprey abundance is input to the model. This observation directly changes the distribution of Figure 6. Posterior probabilities after observing high parasitic lamprey abundance. Breeding adult trout abundance as well as the parent variable (Larval lamprey abundance) which propagates the influence of this observation to Breeding adult trout abundance. Thus, the inferences work in an omnidirectional fashion so that both the probability of Larval lamprey abundance and Polluted sediment extent has been updated from this observation which also influences the probability of Breeding adult trout abundance outside of the causal connection between Parasitic lamprey abundance and Breeding adult trout abundance. The causal power of parasitic lampreys over the adult trout population is obscured and can be misinterpreted when relying on observational inference in this scenario. 42 The question of interest is on the causal influence and not the implications from observation, so acausal, backdoor pathways are problematic in this scenario. Intervening on a variable will now be differentiated from observing a variable, and a formal measure of causal power described in Korb et al. 42 and Korb et al. 43 will be adapted. Manipulations, or system interventions, occur when a real or simulated action is taken to change part of a system. It is an active event distinct from passive ones like observation inputs. Though the idea may seem similar, the implications are vastly different. 44 To remove the influence of the noncausal pathway, an intervention node is added to Parasitic lamprey abundance (Figure 7). Intervening on Parasitic lamprey abundance cuts the parent variables away from the node, thus conducting graphical surgery on the model. Cutting the parents of a variable after a full intervention removes any backdoor, acausal pathway from the parents to children or descendants of the variable in the model. Noncausal pathways no longer influence any subsequent analysis. As described in Korb et al., 43 the first state of the Intervention node enforces the causal implications from the prior frequencies of Parasitic lamprey abundance (i.e., the original distribution). The second is a soft intervention to construct a uniform distribution which is akin to constructing random experimental assignments but simulated within the model. The last state labeled None nullifies the impact of the intervention node on the network and allows use of the original, unmutilated model. For the remaining discussion, the interventions will establish a uniform distribution as a consistent basis for contrasting the two stressors of concern. Intervening with a uniform distribution on the Parasitic lamprey abundance is illustrated (Figure 7). As can be seen when comparing 13201

8 Figure 7. Bayesian network for breeding adult trout abundance after a uniform distribution is placed on parasitic lamprey abundance by intervening on the variable. The backdoor, acausal pathway through larval lamprey abundance is now cut by this intervention. Table 1. Causal Information (CI) Questions and Their Corresponding Calculations 43a question type question example equation c and e What is the causal power of high parasitic lamprey abundance over low breeding adult trout abundance? P* ( ec ) CI( c, e) = P* ( ec )log P* () e P* ( ec ) C and e What is the causal power of parasitic lamprey abundance over low breeding adult trout abundance? CI( C, e) = P* ( c) P* ( ec )log P* () e P* ( ec ) c and E What is the causal power of high parasitic lamprey abundance over breeding adult trout abundance? CI( c, E) = P* ( ec )log P* () e P* ( ec ) C and E What is the causal power of parasitic lamprey abundance over breeding adult trout abundance? CI( C, E) = P* ( c) P* ( ec )log P* () e a Lower case c and e refer to a state of a cause and effect variable, respectively. Upper case C and E refer to the complete distribution of the cause and effect variable. P* refers to the manipulated probability distribution that is constructed from an overwhelming simulated intervention on the causal variable in question. Base 2 is used for logarithm calculations. The distribution of C can be of several types after intervention including an original, uniform, or maximizing distribution. 43 c C e E c C, e E Figure 6 with Figure 7, the intervention changes the distribution of the causal variable without changing the prior distribution on the causal node s parent. The prior distribution for the parent nodes, Larval lamprey abundance and the ancestor node Polluted sediment extent, is unchanged due to the graph surgery of the intervention. The impacts of the intervention on the causal relationship can now be considered without concern from a backdoor pathway. The causal power on the effect variable is calculated using mutual information statistics between the causal variable (Parasitic lamprey abundance) and the effect variable (Breeding adult trout abundance). Causal power is measured in bits of causal information gained, or uncertainty that is reduced, for an effect from knowledge of the cause. 43 Causal power uses an asymmetric adaption of mutual information calculations that take into account the directed nature of causal relationships. 43 Using mutual information statistics in Netica s Sensitivity to Findings feature, 45 the causal power between Parasitic lamprey abundance and Breeding adult trout abundance is calculated to be in the intervened graph (Figure 7). Similarly a uniform stochastic intervention on Polluted sediment extent is specified, and causal power is calculated between polluted sediment extent and breeding adult trout abundance. The causal power is This value is almost an order of magnitude higher indicating the greater causal sensitivity for Breeding adult trout abundance to changes in Polluted sediment extent over Parasitic lamprey abundance. Even though the polluted sediment has an indirect influence (i.e., polluted sediment does not directly cause loss of fish but only through mortality of its prey and lamprey abundance), the Polluted sediment extent has a greater causal power to determine adult trout abundance. The qualitative and quantitative reasons for this were already encoded within the model but were made apparent with these causal power calculations. The questions posed in the previous paragraph can be refined to provide more information than the overall strength of the relationships between distributions. This can help to drill down into what is really desired from a causal analysis. For example, the overall causal strength of the relationship between specific sections of the distributions or events is usually important to consider. To answer these questions, additional causal power questions may be asked for examining the implications of different stressors on a receptor of concern (Table 1). 43 In risk assessment, the potential influence of a high exposure scenario is often utilized to include worst-case possibilities. For this system 13202

9 model, the intent is to examine the causal power of high polluted sediment extent over low adult fish abundance. From the equation (c and e), the causal power is calculated to be The positive value means this is a promoting factor for low fish abundance. 42 If the same is done with high parasitic lamprey abundance, the causal power is indicating an inverse or preventative causal relationship between high lampreys and low adult fish. Examining the additional questions and the probabilistic relationship between these variables in this causal framework provides further insights into how each of these potential stressors relates to an ecological state or variable. This can be viewed as a sensitivity analysis of risk factors that is purely based on the uncertainties in the cause-effect relationships. The credence given to these values depends on the capabilities for capturing causal events of interest in the system that the model represents. 42 DISCUSSION Bayesian inference in general and BNs in particular provide powerful approaches for evidence-based policy in environmental management, but their potential capabilities are only now beginning to be explored. Although simple examples of how BNs can help reason with evidence were provided, as the complexity of the evidence base increases and numerous hypotheses and uncertainties need to be considered, BNs can integrate the information for improved understanding of the endpoints and causal factors of interest. In policy making where trade-offs have to be carefully considered, the qualitative and quantitative properties of BNs can enhance causal understanding of the uncertainties regarding the weight of the evidence for hypotheses of interest. This can provide an explicit basis for data gathering and monitoring efforts that improve the evidence base even when data gaps are large. The measurement idiom directly translates the impact of evidence on a hypothesis through Bayes theorem. The accuracy and validity of the evidence can then be propagated throughout the model providing greater understanding of the power of a test or observation. The capabilities of a measurement process are quantified based on the strength of the evidence in resolving a hypothesis. Misuse of false signals or patterns can be avoided along with the potential for a mismatch between the value of a measurement and the perception of its value. The measurement idiom for BNs alone has broad importance for weight of evidence assessments and subsequent application to policy. Causality is a central component of evidence-based policymaking and causal BNs increase applied relevance. As noted in Bollen and Pearl, 37 statistical science has largely ignored pioneers in graphical statistical models such as path analysis. 40 There still is a disconnect between modeling efforts used to establish causal inferences and the reasoning from the output of these models that largely follows the difficulties in statistics with establishing causality from evidence. Bayesian networks provide insights into the causal aspects being investigated in an evidence-based problem including the assumptions and quantitative implications. Besides examining the causal possibilities in a problem structurally, BNs can examine the strength of the associations probabilistically to support causal inferences and reasoning in a policymaking situation. Evidence-based questions are often questions regarding causal relationships. 46 For addressing these questions and the attendant uncertainties, a BN framework is beneficial for interpreting the causal implications from the evidence Several guidance documents on BN development and interpretation for environmental problems can help facilitate implementation and broader understanding of network model applications in environmental assessments Uusitalo 50 provides an overview of BN usage in environmental applications including advantages and disadvantages. In ecological risk assessments, the BN might help incorporate the qualitative knowledge used in conceptual model development and better align it with quantitative risk estimates. In this way, causal understanding can be more explicitly tested as knowledge is iteratively developed and incorporated. They can also be a central reasoning tool for adaptive management applications 48 where the causal knowledge and beliefs regarding management interventions are captured and updated with new information as management learning progresses. In this process, influence diagrams can extend the BN by incorporating information on how decisions interact with the system model and uncertain desired and undesired outcomes The utility nodes in influence diagrams contain a scale for quantifying the costs and benefits of decision outcomes and, in some cases, the relative or absolute value of evidence to a decision. Evidence-based policy for environmental management is moving in directions that will strain conventional weight of evidence analysis practices in ecological risk assessment. Although there is no single framework that can match the scope of every problem, arbitrary or overly simplistic rules for combining and weighing evidence can inhibit insights and create difficulties in data interpretation or discerning the validity of multiple diverging viewpoints. Moving beyond the exploratory phases of environmental management requires greater consideration and explication of causality. Evidence-based assessments would benefit from incorporating Bayesian approaches to weighing evidence even under conditions of high uncertainty and incomplete evidence. The transparency, tractability, mathematical rigor, and focus on the causal aspects of a problem make BNs an advantageous analytic structure in the evidencebased practitioner s toolbox. AUTHOR INFORMATION Corresponding Author *Phone: carriger.john@epa.gov. Notes The authors declare no competing financial interest. ACKNOWLEDGMENTS This article is dedicated to Bonnie Carriger. This research was supported in part by an appointment to the ORISE participant research program through an interagency agreement between the U.S. Environmental Protection Agency and the U.S. Department of Energy. The views expressed in this article are those of the authors and do not necessarily reflect the views or policies of the U.S. Environmental Protection Agency. M.C. Newman was the A. Marshall Acuff Jr. Professor of Marine Science during the tenure of this study. REFERENCES (1) Nature Editorial. Look after the pennies: Government decisions about where to spend and where to cut should be based on evidence, not ideology. Nature 2013, 496, 269. (2) CEE. Guidelines for Systematic Reviews in Environmental Management, Version 4.2; Collaboration for Environmental Evidence, Bangor University: Bangor, UK, 2013.

Outline. What s inside this paper? My expectation. Software Defect Prediction. Traditional Method. What s inside this paper?

Outline. What s inside this paper? My expectation. Software Defect Prediction. Traditional Method. What s inside this paper? Outline A Critique of Software Defect Prediction Models Norman E. Fenton Dongfeng Zhu What s inside this paper? What kind of new technique was developed in this paper? Research area of this technique?

More information

DECISION ANALYSIS WITH BAYESIAN NETWORKS

DECISION ANALYSIS WITH BAYESIAN NETWORKS RISK ASSESSMENT AND DECISION ANALYSIS WITH BAYESIAN NETWORKS NORMAN FENTON MARTIN NEIL CRC Press Taylor & Francis Croup Boca Raton London NewYork CRC Press is an imprint of the Taylor Si Francis an Croup,

More information

Chapter 02 Developing and Evaluating Theories of Behavior

Chapter 02 Developing and Evaluating Theories of Behavior Chapter 02 Developing and Evaluating Theories of Behavior Multiple Choice Questions 1. A theory is a(n): A. plausible or scientifically acceptable, well-substantiated explanation of some aspect of the

More information

January 2, Overview

January 2, Overview American Statistical Association Position on Statistical Statements for Forensic Evidence Presented under the guidance of the ASA Forensic Science Advisory Committee * January 2, 2019 Overview The American

More information

How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis?

How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis? How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis? Richards J. Heuer, Jr. Version 1.2, October 16, 2005 This document is from a collection of works by Richards J. Heuer, Jr.

More information

A Case Study: Two-sample categorical data

A Case Study: Two-sample categorical data A Case Study: Two-sample categorical data Patrick Breheny January 31 Patrick Breheny BST 701: Bayesian Modeling in Biostatistics 1/43 Introduction Model specification Continuous vs. mixture priors Choice

More information

Supplement 2. Use of Directed Acyclic Graphs (DAGs)

Supplement 2. Use of Directed Acyclic Graphs (DAGs) Supplement 2. Use of Directed Acyclic Graphs (DAGs) Abstract This supplement describes how counterfactual theory is used to define causal effects and the conditions in which observed data can be used to

More information

MS&E 226: Small Data

MS&E 226: Small Data MS&E 226: Small Data Lecture 10: Introduction to inference (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 17 What is inference? 2 / 17 Where did our data come from? Recall our sample is: Y, the vector

More information

Analysis of complex patterns of evidence in legal cases: Wigmore charts vs. Bayesian networks

Analysis of complex patterns of evidence in legal cases: Wigmore charts vs. Bayesian networks Analysis of complex patterns of evidence in legal cases: Wigmore charts vs. Bayesian networks V. Leucari October 2005 Typical features of the evidence arising from legal cases are its complex structure

More information

A Brief Introduction to Bayesian Statistics

A Brief Introduction to Bayesian Statistics A Brief Introduction to Statistics David Kaplan Department of Educational Psychology Methods for Social Policy Research and, Washington, DC 2017 1 / 37 The Reverend Thomas Bayes, 1701 1761 2 / 37 Pierre-Simon

More information

Lecture 9 Internal Validity

Lecture 9 Internal Validity Lecture 9 Internal Validity Objectives Internal Validity Threats to Internal Validity Causality Bayesian Networks Internal validity The extent to which the hypothesized relationship between 2 or more variables

More information

The Regression-Discontinuity Design

The Regression-Discontinuity Design Page 1 of 10 Home» Design» Quasi-Experimental Design» The Regression-Discontinuity Design The regression-discontinuity design. What a terrible name! In everyday language both parts of the term have connotations

More information

Insight Assessment Measuring Thinking Worldwide

Insight Assessment Measuring Thinking Worldwide California Critical Thinking Skills Test (CCTST). The CCTST measures the reasoning skills human beings use in the process of reflectively deciding what to believe or what to do. Skill/Attribute Name SE

More information

Expert judgements in risk and reliability analysis

Expert judgements in risk and reliability analysis Expert judgements in risk and reliability analysis Jan-Erik Holmberg M2-E2117 Risk analysis, April 17, 2016 Aalto university Contents Definitions Applications Bayesian approach Examples 2 Background Objective,

More information

INTERVIEWS II: THEORIES AND TECHNIQUES 5. CLINICAL APPROACH TO INTERVIEWING PART 1

INTERVIEWS II: THEORIES AND TECHNIQUES 5. CLINICAL APPROACH TO INTERVIEWING PART 1 INTERVIEWS II: THEORIES AND TECHNIQUES 5. CLINICAL APPROACH TO INTERVIEWING PART 1 5.1 Clinical Interviews: Background Information The clinical interview is a technique pioneered by Jean Piaget, in 1975,

More information

Lecture 3: Bayesian Networks 1

Lecture 3: Bayesian Networks 1 Lecture 3: Bayesian Networks 1 Jingpeng Li 1 Content Reminder from previous lecture: Bayes theorem Bayesian networks Why are they currently interesting? Detailed example from medical diagnostics Bayesian

More information

Improving statistical estimates used in the courtroom. Precis. Bayes Theorem. Professor Norman Fenton. Queen Mary University of London and Agena Ltd

Improving statistical estimates used in the courtroom. Precis. Bayes Theorem. Professor Norman Fenton. Queen Mary University of London and Agena Ltd Improving statistical estimates used in the courtroom Professor Norman Fenton Queen Mary University of London and Agena Ltd Address: Queen Mary University of London School of Electronic Engineering and

More information

6. A theory that has been substantially verified is sometimes called a a. law. b. model.

6. A theory that has been substantially verified is sometimes called a a. law. b. model. Chapter 2 Multiple Choice Questions 1. A theory is a(n) a. a plausible or scientifically acceptable, well-substantiated explanation of some aspect of the natural world. b. a well-substantiated explanation

More information

Logistic Regression and Bayesian Approaches in Modeling Acceptance of Male Circumcision in Pune, India

Logistic Regression and Bayesian Approaches in Modeling Acceptance of Male Circumcision in Pune, India 20th International Congress on Modelling and Simulation, Adelaide, Australia, 1 6 December 2013 www.mssanz.org.au/modsim2013 Logistic Regression and Bayesian Approaches in Modeling Acceptance of Male Circumcision

More information

Live WebEx meeting agenda

Live WebEx meeting agenda 10:00am 10:30am Using OpenMeta[Analyst] to extract quantitative data from published literature Live WebEx meeting agenda August 25, 10:00am-12:00pm ET 10:30am 11:20am Lecture (this will be recorded) 11:20am

More information

Funnelling Used to describe a process of narrowing down of focus within a literature review. So, the writer begins with a broad discussion providing b

Funnelling Used to describe a process of narrowing down of focus within a literature review. So, the writer begins with a broad discussion providing b Accidental sampling A lesser-used term for convenience sampling. Action research An approach that challenges the traditional conception of the researcher as separate from the real world. It is associated

More information

High-level Vision. Bernd Neumann Slides for the course in WS 2004/05. Faculty of Informatics Hamburg University Germany

High-level Vision. Bernd Neumann Slides for the course in WS 2004/05. Faculty of Informatics Hamburg University Germany High-level Vision Bernd Neumann Slides for the course in WS 2004/05 Faculty of Informatics Hamburg University Germany neumann@informatik.uni-hamburg.de http://kogs-www.informatik.uni-hamburg.de 1 Contents

More information

Evaluation Models STUDIES OF DIAGNOSTIC EFFICIENCY

Evaluation Models STUDIES OF DIAGNOSTIC EFFICIENCY 2. Evaluation Model 2 Evaluation Models To understand the strengths and weaknesses of evaluation, one must keep in mind its fundamental purpose: to inform those who make decisions. The inferences drawn

More information

Hypothesis-Driven Research

Hypothesis-Driven Research Hypothesis-Driven Research Research types Descriptive science: observe, describe and categorize the facts Discovery science: measure variables to decide general patterns based on inductive reasoning Hypothesis-driven

More information

Modelling Experimental Interventions: Results and Challenges

Modelling Experimental Interventions: Results and Challenges Concrete Causation Munich 2010 Modelling Experimental Interventions: Results and Challenges Jan-Willem Romeijn Faculty of Philosophy University of Groningen Contents 1 Overview 3 2 Interventions and confirmation

More information

You must answer question 1.

You must answer question 1. Research Methods and Statistics Specialty Area Exam October 28, 2015 Part I: Statistics Committee: Richard Williams (Chair), Elizabeth McClintock, Sarah Mustillo You must answer question 1. 1. Suppose

More information

PLANNING THE RESEARCH PROJECT

PLANNING THE RESEARCH PROJECT Van Der Velde / Guide to Business Research Methods First Proof 6.11.2003 4:53pm page 1 Part I PLANNING THE RESEARCH PROJECT Van Der Velde / Guide to Business Research Methods First Proof 6.11.2003 4:53pm

More information

Bradford Hill Criteria for Causal Inference Based on a presentation at the 2015 ANZEA Conference

Bradford Hill Criteria for Causal Inference Based on a presentation at the 2015 ANZEA Conference Bradford Hill Criteria for Causal Inference Based on a presentation at the 2015 ANZEA Conference Julian King Director, Julian King & Associates a member of the Kinnect Group www.julianking.co.nz www.kinnect.co.nz

More information

Introduction to Computational Neuroscience

Introduction to Computational Neuroscience Introduction to Computational Neuroscience Lecture 5: Data analysis II Lesson Title 1 Introduction 2 Structure and Function of the NS 3 Windows to the Brain 4 Data analysis 5 Data analysis II 6 Single

More information

5/20/2014. Leaving Andy Clark's safe shores: Scaling Predictive Processing to higher cognition. ANC sysmposium 19 May 2014

5/20/2014. Leaving Andy Clark's safe shores: Scaling Predictive Processing to higher cognition. ANC sysmposium 19 May 2014 ANC sysmposium 19 May 2014 Lorentz workshop on HPI Leaving Andy Clark's safe shores: Scaling Predictive Processing to higher cognition Johan Kwisthout, Maria Otworowska, Harold Bekkering, Iris van Rooij

More information

Potential Use of a Causal Bayesian Network to Support Both Clinical and Pathophysiology Tutoring in an Intelligent Tutoring System for Anemias

Potential Use of a Causal Bayesian Network to Support Both Clinical and Pathophysiology Tutoring in an Intelligent Tutoring System for Anemias Harvard-MIT Division of Health Sciences and Technology HST.947: Medical Artificial Intelligence Prof. Peter Szolovits Prof. Lucila Ohno-Machado Potential Use of a Causal Bayesian Network to Support Both

More information

Analysis A step in the research process that involves describing and then making inferences based on a set of data.

Analysis A step in the research process that involves describing and then making inferences based on a set of data. 1 Appendix 1:. Definitions of important terms. Additionality The difference between the value of an outcome after the implementation of a policy, and its value in a counterfactual scenario in which the

More information

Answers to end of chapter questions

Answers to end of chapter questions Answers to end of chapter questions Chapter 1 What are the three most important characteristics of QCA as a method of data analysis? QCA is (1) systematic, (2) flexible, and (3) it reduces data. What are

More information

UNLOCKING VALUE WITH DATA SCIENCE BAYES APPROACH: MAKING DATA WORK HARDER

UNLOCKING VALUE WITH DATA SCIENCE BAYES APPROACH: MAKING DATA WORK HARDER UNLOCKING VALUE WITH DATA SCIENCE BAYES APPROACH: MAKING DATA WORK HARDER 2016 DELIVERING VALUE WITH DATA SCIENCE BAYES APPROACH - MAKING DATA WORK HARDER The Ipsos MORI Data Science team increasingly

More information

Representing Variable Source Credibility in Intelligence Analysis with Bayesian Networks

Representing Variable Source Credibility in Intelligence Analysis with Bayesian Networks Edith Cowan University Research Online Australian Security and Intelligence Conference Conferences, Symposia and Campus Events 2012 Representing Variable Source Credibility in Intelligence Analysis with

More information

MTAT Bayesian Networks. Introductory Lecture. Sven Laur University of Tartu

MTAT Bayesian Networks. Introductory Lecture. Sven Laur University of Tartu MTAT.05.113 Bayesian Networks Introductory Lecture Sven Laur University of Tartu Motivation Probability calculus can be viewed as an extension of classical logic. We use many imprecise and heuristic rules

More information

C/S/E/L :2008. On Analytical Rigor A Study of How Professional Intelligence Analysts Assess Rigor. innovations at the intersection of people,

C/S/E/L :2008. On Analytical Rigor A Study of How Professional Intelligence Analysts Assess Rigor. innovations at the intersection of people, C/S/E/L :2008 innovations at the intersection of people, technology, and work. On Analytical Rigor A Study of How Professional Intelligence Analysts Assess Rigor Daniel J. Zelik The Ohio State University

More information

Definitions of Nature of Science and Scientific Inquiry that Guide Project ICAN: A Cheat Sheet

Definitions of Nature of Science and Scientific Inquiry that Guide Project ICAN: A Cheat Sheet Definitions of Nature of Science and Scientific Inquiry that Guide Project ICAN: A Cheat Sheet What is the NOS? The phrase nature of science typically refers to the values and assumptions inherent to scientific

More information

Model calibration and Bayesian methods for probabilistic projections

Model calibration and Bayesian methods for probabilistic projections ETH Zurich Reto Knutti Model calibration and Bayesian methods for probabilistic projections Reto Knutti, IAC ETH Toy model Model: obs = linear trend + noise(variance, spectrum) 1) Short term predictability,

More information

Cognitive domain: Comprehension Answer location: Elements of Empiricism Question type: MC

Cognitive domain: Comprehension Answer location: Elements of Empiricism Question type: MC Chapter 2 1. Knowledge that is evaluative, value laden, and concerned with prescribing what ought to be is known as knowledge. *a. Normative b. Nonnormative c. Probabilistic d. Nonprobabilistic. 2. Most

More information

Learning Deterministic Causal Networks from Observational Data

Learning Deterministic Causal Networks from Observational Data Carnegie Mellon University Research Showcase @ CMU Department of Psychology Dietrich College of Humanities and Social Sciences 8-22 Learning Deterministic Causal Networks from Observational Data Ben Deverett

More information

What is analytical sociology? And is it the future of sociology?

What is analytical sociology? And is it the future of sociology? What is analytical sociology? And is it the future of sociology? Twan Huijsmans Sociology Abstract During the last few decades a new approach in sociology has been developed, analytical sociology (AS).

More information

GUIDELINE COMPARATORS & COMPARISONS:

GUIDELINE COMPARATORS & COMPARISONS: GUIDELINE COMPARATORS & COMPARISONS: Direct and indirect comparisons Adapted version (2015) based on COMPARATORS & COMPARISONS: Direct and indirect comparisons - February 2013 The primary objective of

More information

ANIMAL HEALTH SURVEILLANCE

ANIMAL HEALTH SURVEILLANCE 291 Annex 33a CHAPTER 1.4. ANIMAL HEALTH SURVEILLANCE Article 1.4.1. Introduction and objectives 1) In general, surveillance is aimed at demonstrating the absence of infection or infestation, determining

More information

Inferencing in Artificial Intelligence and Computational Linguistics

Inferencing in Artificial Intelligence and Computational Linguistics Inferencing in Artificial Intelligence and Computational Linguistics (http://www.dfki.de/~horacek/infer-ai-cl.html) no classes on 28.5., 18.6., 25.6. 2-3 extra lectures will be scheduled Helmut Horacek

More information

August 29, Introduction and Overview

August 29, Introduction and Overview August 29, 2018 Introduction and Overview Why are we here? Haavelmo(1944): to become master of the happenings of real life. Theoretical models are necessary tools in our attempts to understand and explain

More information

ISPOR Task Force Report: ITC & NMA Study Questionnaire

ISPOR Task Force Report: ITC & NMA Study Questionnaire INDIRECT TREATMENT COMPARISON / NETWORK META-ANALYSIS STUDY QUESTIONNAIRE TO ASSESS RELEVANCE AND CREDIBILITY TO INFORM HEALTHCARE DECISION-MAKING: AN ISPOR-AMCP-NPC GOOD PRACTICE TASK FORCE REPORT DRAFT

More information

School of Population and Public Health SPPH 503 Epidemiologic methods II January to April 2019

School of Population and Public Health SPPH 503 Epidemiologic methods II January to April 2019 School of Population and Public Health SPPH 503 Epidemiologic methods II January to April 2019 Time: Tuesday, 1330 1630 Location: School of Population and Public Health, UBC Course description Students

More information

Analysis of Competing Hypotheses using Subjective Logic (ACH-SL)

Analysis of Competing Hypotheses using Subjective Logic (ACH-SL) Analysis of Competing Hypotheses using Subjective Logic (ACH-SL) Simon Pope and Audun Jøsang, DSTC, CRC for Enterprise Distributed Systems Technology ICCRTS, 13-16 June 2005 June 22, 2005 DSTC 2005. All

More information

An Interactive Modeling Environment for Systems Biology of Aging

An Interactive Modeling Environment for Systems Biology of Aging An Interactive Modeling Environment for Systems Biology of Aging Pat Langley Computer Science and Engineering Arizona State University Tempe, Arizona Thanks to D. Bidaye, J. Difzac, J. Furber, S. Kim,

More information

Unit 1 Exploring and Understanding Data

Unit 1 Exploring and Understanding Data Unit 1 Exploring and Understanding Data Area Principle Bar Chart Boxplot Conditional Distribution Dotplot Empirical Rule Five Number Summary Frequency Distribution Frequency Polygon Histogram Interquartile

More information

DEVELOPING THE RESEARCH FRAMEWORK Dr. Noly M. Mascariñas

DEVELOPING THE RESEARCH FRAMEWORK Dr. Noly M. Mascariñas DEVELOPING THE RESEARCH FRAMEWORK Dr. Noly M. Mascariñas Director, BU-CHED Zonal Research Center Bicol University Research and Development Center Legazpi City Research Proposal Preparation Seminar-Writeshop

More information

Data and Statistics 101: Key Concepts in the Collection, Analysis, and Application of Child Welfare Data

Data and Statistics 101: Key Concepts in the Collection, Analysis, and Application of Child Welfare Data TECHNICAL REPORT Data and Statistics 101: Key Concepts in the Collection, Analysis, and Application of Child Welfare Data CONTENTS Executive Summary...1 Introduction...2 Overview of Data Analysis Concepts...2

More information

Probabilistic Reasoning with Bayesian Networks and BayesiaLab

Probabilistic Reasoning with Bayesian Networks and BayesiaLab The presentation will start at: 13:00:00 The current time is: 13:00:42 Central Standard Time, UTC-6 Probabilistic Reasoning with Bayesian Networks and BayesiaLab Introduction Your Hosts Today Stefan Conrady

More information

Investigative Biology (Advanced Higher)

Investigative Biology (Advanced Higher) Investigative Biology (Advanced Higher) The Mandatory Course key areas are from the Course Assessment Specification. Activities in the Suggested learning activities are not mandatory. This offers examples

More information

Introduction to Research Methods

Introduction to Research Methods Introduction to Research Methods Updated August 08, 2016 1 The Three Types of Psychology Research Psychology research can usually be classified as one of three major types: 1. Causal Research When most

More information

Quantitative and qualitative research

Quantitative and qualitative research Quantitative and qualitative research Quantitative and qualitative research 1 Qualitative vs quantitative 2 Overview of qualitative research 3 Qualitative research techniques 3 Overview of quantitative

More information

Modeling and Environmental Science: In Conclusion

Modeling and Environmental Science: In Conclusion Modeling and Environmental Science: In Conclusion Environmental Science It sounds like a modern idea, but if you view it broadly, it s a very old idea: Our ancestors survival depended on their knowledge

More information

Simpson s Paradox and the implications for medical trials

Simpson s Paradox and the implications for medical trials Simpson s Paradox and the implications for medical trials Norman Fenton, Martin Neil and Anthony Constantinou 31 August 2015 Abstract This paper describes Simpson s paradox, and explains its serious implications

More information

Book Note: In Doubt: The Psychology Of The Criminal Justice Process, by Dan Simon

Book Note: In Doubt: The Psychology Of The Criminal Justice Process, by Dan Simon Osgoode Hall Law Journal Volume 51, Issue 2 (Winter 2014) Article 13 Book Note: In Doubt: The Psychology Of The Criminal Justice Process, by Dan Simon Jennifer McKee Follow this and additional works at:

More information

Bayesian and Frequentist Approaches

Bayesian and Frequentist Approaches Bayesian and Frequentist Approaches G. Jogesh Babu Penn State University http://sites.stat.psu.edu/ babu http://astrostatistics.psu.edu All models are wrong But some are useful George E. P. Box (son-in-law

More information

Mixed Methods Study Design

Mixed Methods Study Design 1 Mixed Methods Study Design Kurt C. Stange, MD, PhD Professor of Family Medicine, Epidemiology & Biostatistics, Oncology and Sociology Case Western Reserve University 1. Approaches 1, 2 a. Qualitative

More information

Discussion Meeting for MCP-Mod Qualification Opinion Request. Novartis 10 July 2013 EMA, London, UK

Discussion Meeting for MCP-Mod Qualification Opinion Request. Novartis 10 July 2013 EMA, London, UK Discussion Meeting for MCP-Mod Qualification Opinion Request Novartis 10 July 2013 EMA, London, UK Attendees Face to face: Dr. Frank Bretz Global Statistical Methodology Head, Novartis Dr. Björn Bornkamp

More information

Understanding Science Conceptual Framework

Understanding Science Conceptual Framework 1 Understanding Science Conceptual Framework This list of conceptual understandings regarding the nature and process of science are aligned across grade levels to help instructors identify age-appropriate

More information

Citation for published version (APA): Ebbes, P. (2004). Latent instrumental variables: a new approach to solve for endogeneity s.n.

Citation for published version (APA): Ebbes, P. (2004). Latent instrumental variables: a new approach to solve for endogeneity s.n. University of Groningen Latent instrumental variables Ebbes, P. IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document

More information

Doing After Seeing. Seeing vs. Doing in Causal Bayes Nets

Doing After Seeing. Seeing vs. Doing in Causal Bayes Nets Doing After Seeing Björn Meder (bmeder@uni-goettingen.de) York Hagmayer (york.hagmayer@bio.uni-goettingen.de) Michael R. Waldmann (michael.waldmann@bio.uni-goettingen.de) Department of Psychology, University

More information

EXECUTIVE SUMMARY 9. Executive Summary

EXECUTIVE SUMMARY 9. Executive Summary EXECUTIVE SUMMARY 9 Executive Summary Education affects people s lives in ways that go far beyond what can be measured by labour market earnings and economic growth. Important as they are, these social

More information

Assignment 4: True or Quasi-Experiment

Assignment 4: True or Quasi-Experiment Assignment 4: True or Quasi-Experiment Objectives: After completing this assignment, you will be able to Evaluate when you must use an experiment to answer a research question Develop statistical hypotheses

More information

HOW IS HAIR GEL QUANTIFIED?

HOW IS HAIR GEL QUANTIFIED? HOW IS HAIR GEL QUANTIFIED? MARK A. PITT Department of Psychology, Ohio State University, 1835 Neil Avenue, Columbus, Ohio, 43210, USA JAY I. MYUNG Department of Psychology, Ohio State University, 1835

More information

CISC453 Winter Probabilistic Reasoning Part B: AIMA3e Ch

CISC453 Winter Probabilistic Reasoning Part B: AIMA3e Ch CISC453 Winter 2010 Probabilistic Reasoning Part B: AIMA3e Ch 14.5-14.8 Overview 2 a roundup of approaches from AIMA3e 14.5-14.8 14.5 a survey of approximate methods alternatives to the direct computing

More information

Pythia WEB ENABLED TIMED INFLUENCE NET MODELING TOOL SAL. Lee W. Wagenhals Alexander H. Levis

Pythia WEB ENABLED TIMED INFLUENCE NET MODELING TOOL SAL. Lee W. Wagenhals Alexander H. Levis Pythia WEB ENABLED TIMED INFLUENCE NET MODELING TOOL Lee W. Wagenhals Alexander H. Levis ,@gmu.edu Adversary Behavioral Modeling Maxwell AFB, Montgomery AL March 8-9, 2007 1 Outline Pythia

More information

Systems Theory: Should Information Researchers Even Care?

Systems Theory: Should Information Researchers Even Care? Association for Information Systems AIS Electronic Library (AISeL) SAIS 2016 Proceedings Southern (SAIS) 2016 Systems Theory: Should Information Researchers Even Care? Kane J. Smith Virginia Commonwealth

More information

Underlying Theory & Basic Issues

Underlying Theory & Basic Issues Underlying Theory & Basic Issues Dewayne E Perry ENS 623 Perry@ece.utexas.edu 1 All Too True 2 Validity In software engineering, we worry about various issues: E-Type systems: Usefulness is it doing what

More information

Relationships Between the High Impact Indicators and Other Indicators

Relationships Between the High Impact Indicators and Other Indicators Relationships Between the High Impact Indicators and Other Indicators The High Impact Indicators are a list of key skills assessed on the GED test that, if emphasized in instruction, can help instructors

More information

Competency Rubric Bank for the Sciences (CRBS)

Competency Rubric Bank for the Sciences (CRBS) Competency Rubric Bank for the Sciences (CRBS) Content Knowledge 1 Content Knowledge: Accuracy of scientific understanding Higher Order Cognitive Skills (HOCS) 3 Analysis: Clarity of Research Question

More information

SEMINAR ON SERVICE MARKETING

SEMINAR ON SERVICE MARKETING SEMINAR ON SERVICE MARKETING Tracy Mary - Nancy LOGO John O. Summers Indiana University Guidelines for Conducting Research and Publishing in Marketing: From Conceptualization through the Review Process

More information

DRAFT (Final) Concept Paper On choosing appropriate estimands and defining sensitivity analyses in confirmatory clinical trials

DRAFT (Final) Concept Paper On choosing appropriate estimands and defining sensitivity analyses in confirmatory clinical trials DRAFT (Final) Concept Paper On choosing appropriate estimands and defining sensitivity analyses in confirmatory clinical trials EFSPI Comments Page General Priority (H/M/L) Comment The concept to develop

More information

IE 5203 Decision Analysis Lab I Probabilistic Modeling, Inference and Decision Making with Netica

IE 5203 Decision Analysis Lab I Probabilistic Modeling, Inference and Decision Making with Netica IE 5203 Decision Analysis Lab I Probabilistic Modeling, Inference and Decision Making with Netica Overview of Netica Software Netica Application is a comprehensive tool for working with Bayesian networks

More information

Theory and Methods Question Bank

Theory and Methods Question Bank Theory and Methods Question Bank Theory and Methods is examined in both the AS and the A Level. The AS questions focus mostly on research methods and at A Level include sociological debates, perspectives

More information

Chapter 3 Tools for Practical Theorizing: Theoretical Maps and Ecosystem Maps

Chapter 3 Tools for Practical Theorizing: Theoretical Maps and Ecosystem Maps Chapter 3 Tools for Practical Theorizing: Theoretical Maps and Ecosystem Maps Chapter Outline I. Introduction A. Understanding theoretical languages requires universal translators 1. Theoretical maps identify

More information

ELICITING EXPERT KNOWLEDGE IN SUPPORT OF PLANNING AND ADAPTIVE MANAGEMENT

ELICITING EXPERT KNOWLEDGE IN SUPPORT OF PLANNING AND ADAPTIVE MANAGEMENT ELICITING EXPERT KNOWLEDGE IN SUPPORT OF PLANNING AND ADAPTIVE MANAGEMENT C. Ashton Drew Research Coordinator North Carolina Fish & Wildlife Cooperative Research Unit Outline Expert Knowledge Applications

More information

BIOLOGY. The range and suitability of the work submitted

BIOLOGY. The range and suitability of the work submitted Overall grade boundaries BIOLOGY Grade: E D C B A Mark range: 0-7 8-15 16-22 23-28 29-36 The range and suitability of the work submitted In this session essays were submitted in a wide range of appropriate

More information

Remarks on Bayesian Control Charts

Remarks on Bayesian Control Charts Remarks on Bayesian Control Charts Amir Ahmadi-Javid * and Mohsen Ebadi Department of Industrial Engineering, Amirkabir University of Technology, Tehran, Iran * Corresponding author; email address: ahmadi_javid@aut.ac.ir

More information

Bayes Linear Statistics. Theory and Methods

Bayes Linear Statistics. Theory and Methods Bayes Linear Statistics Theory and Methods Michael Goldstein and David Wooff Durham University, UK BICENTENNI AL BICENTENNIAL Contents r Preface xvii 1 The Bayes linear approach 1 1.1 Combining beliefs

More information

One Stop Shop For Educator

One Stop Shop For Educator One Stop Shop For Educator The following instructional plan is part of a GaDOE collection of Unit Frameworks, Performance Tasks, examples of Student Work, and Teacher Commentary. Many more GaDOE approved

More information

Measuring impact. William Parienté UC Louvain J PAL Europe. povertyactionlab.org

Measuring impact. William Parienté UC Louvain J PAL Europe. povertyactionlab.org Measuring impact William Parienté UC Louvain J PAL Europe povertyactionlab.org Course overview 1. What is evaluation? 2. Measuring impact 3. Why randomize? 4. How to randomize 5. Sampling and Sample Size

More information

Identifying a Computer Forensics Expert: A Study to Measure the Characteristics of Forensic Computer Examiners

Identifying a Computer Forensics Expert: A Study to Measure the Characteristics of Forensic Computer Examiners Journal of Digital Forensics, Security and Law Volume 5 Number 1 Article 1 2010 Identifying a Computer Forensics Expert: A Study to Measure the Characteristics of Forensic Computer Examiners Gregory H.

More information

THE INDIRECT EFFECT IN MULTIPLE MEDIATORS MODEL BY STRUCTURAL EQUATION MODELING ABSTRACT

THE INDIRECT EFFECT IN MULTIPLE MEDIATORS MODEL BY STRUCTURAL EQUATION MODELING ABSTRACT European Journal of Business, Economics and Accountancy Vol. 4, No. 3, 016 ISSN 056-6018 THE INDIRECT EFFECT IN MULTIPLE MEDIATORS MODEL BY STRUCTURAL EQUATION MODELING Li-Ju Chen Department of Business

More information

Hierarchy of Statistical Goals

Hierarchy of Statistical Goals Hierarchy of Statistical Goals Ideal goal of scientific study: Deterministic results Determine the exact value of a ment or population parameter Prediction: What will the value of a future observation

More information

Student Performance Q&A:

Student Performance Q&A: Student Performance Q&A: 2009 AP Statistics Free-Response Questions The following comments on the 2009 free-response questions for AP Statistics were written by the Chief Reader, Christine Franklin of

More information

THEORY OF CHANGE FOR FUNDERS

THEORY OF CHANGE FOR FUNDERS THEORY OF CHANGE FOR FUNDERS Planning to make a difference Dawn Plimmer and Angela Kail December 2014 CONTENTS Contents... 2 Introduction... 3 What is a theory of change for funders?... 3 This report...

More information

Introduction to Bayesian Analysis 1

Introduction to Bayesian Analysis 1 Biostats VHM 801/802 Courses Fall 2005, Atlantic Veterinary College, PEI Henrik Stryhn Introduction to Bayesian Analysis 1 Little known outside the statistical science, there exist two different approaches

More information

Bayesian (Belief) Network Models,

Bayesian (Belief) Network Models, Bayesian (Belief) Network Models, 2/10/03 & 2/12/03 Outline of This Lecture 1. Overview of the model 2. Bayes Probability and Rules of Inference Conditional Probabilities Priors and posteriors Joint distributions

More information

Lec 02: Estimation & Hypothesis Testing in Animal Ecology

Lec 02: Estimation & Hypothesis Testing in Animal Ecology Lec 02: Estimation & Hypothesis Testing in Animal Ecology Parameter Estimation from Samples Samples We typically observe systems incompletely, i.e., we sample according to a designed protocol. We then

More information

Glossary of Research Terms Compiled by Dr Emma Rowden and David Litting (UTS Library)

Glossary of Research Terms Compiled by Dr Emma Rowden and David Litting (UTS Library) Glossary of Research Terms Compiled by Dr Emma Rowden and David Litting (UTS Library) Applied Research Applied research refers to the use of social science inquiry methods to solve concrete and practical

More information

Evaluating the Causal Role of Unobserved Variables

Evaluating the Causal Role of Unobserved Variables Evaluating the Causal Role of Unobserved Variables Christian C. Luhmann (christian.luhmann@vanderbilt.edu) Department of Psychology, Vanderbilt University 301 Wilson Hall, Nashville, TN 37203 USA Woo-kyoung

More information

Artificial Intelligence Programming Probability

Artificial Intelligence Programming Probability Artificial Intelligence Programming Probability Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science University of San Francisco p.1/25 17-0: Uncertainty

More information

The Objects of Social Sciences: Idea, Action, and Outcome [From Ontology to Epistemology and Methodology]

The Objects of Social Sciences: Idea, Action, and Outcome [From Ontology to Epistemology and Methodology] The Objects of Social Sciences: Idea, Action, and Outcome [From Ontology to Epistemology and Methodology] Shiping Tang Fudan University Shanghai 2013-05-07/Beijing, 2013-09-09 Copyright@ Shiping Tang Outline

More information

W e l e a d

W e l e a d http://www.ramayah.com 1 2 Developing a Robust Research Framework T. Ramayah School of Management Universiti Sains Malaysia ramayah@usm.my Variables in Research Moderator Independent Mediator Dependent

More information