Machine Learning in Dose-Response Assessment:

Transcription

1 Machine Learning in Dose-Response Assessment: Translating Science to Decisions Jacqueline MacDonald Gibson, Ph.D. RTI University Scholar Associate Professor, Department of Environmental Sciences and Engineering University of North Carolina Chapel Hill

2 Longstanding U.S. Environmental Policy Requires Cost-Benefit Analysis In deciding whether and how to regulate, agencies should assess all costs and benefits of available regulatory alternatives, including the alternative of not regulating. [A]gencies should select those approaches that maximize net benefits. Executive Order (William J. Clinton, 1993) Federal Register Vol. 58, No. 190 Monday, October 4, 1993 Title 3 The President Presidential Documents Executive Order of September 30, 1993 Regulatory Planning and Review The American people deserve a regulatory system that works for them, not against them: a regulatory system that protects and improves their health, safety, environment, and well-being and improves the performance of the economy without imposing unacceptable or unreasonable costs on society; regulatory policies that recognize that the private sector and private markets are the best engine for economic growth; regulatory approaches that respect the role of State, local, and tribal governments; and regulations that are effective, consistent, sensible, and understandable. We do not have such a regulatory system today. 2

3 Dose-Response Assessment Is Cornerstone of Benefits Analysis Dose-response assessment translates environmental quality change to health effect: disease cases = ) pollution where = change ) = dose response function 3

4 Outline Dose-response assessment Currently used dose-response functions Limitations Machine-learned Bayesian networks as an alternative Case study: Arsenic dose-response assessment Introduction: health risks and case study data source Methods: machine-learned dose-response network Results: comparing the traditional to the new approach Policy implications 4

5 Current dose-response assessment methods 5

6 Different Approaches Are Used for Cancer, Non-Cancer Effects Cancers: Probability of cancer =!(D) = < > where D = dose All other illnesses:! D % if D reference dose =$ 2 if D > reference dose < = slope factor SOURCE: NIH, ToxTutor, REFERENCE DOSE (RfD) 6

7 Policy Analysts Use Look-Up Tables to Find Slope Factors, Reference Doses Example: cancer from arsenic Noncancer risk from arsenic Cancer probability = 1.5 mg As kg body weight day <=>? Risk = 1 if <=>? > 1 B CD EF mg As kg body weight day 7

8 Limitations of Current Approachs Cancer, noncancer methods differ Noncancer benefits not quantifiable Dose>RfD implies people may be at risk Dose-response functions not customizable to the individual Nonlinearities not accounted for Interactions (e.g., gene, environment) not considered 8

9 21 st century approach: machinelearned Bayesian networks 9

10 Bayesian Network Has Two Parts 1. Directed acyclic graph - Nodes=variables of interest - Edges=relationships 10

11 Bayesian Network Has Two Parts 1. Directed acyclic graph - Nodes=variables of interest - Edges=relationships 2. Joint probability distribution over the variables - Conditional probability tables 11

12 Bayes Theorem Used to Update Nodes with Evidence! "# "$ =!("$ "#)!("#)!("$) 12

13 Bayes Theorem Used to Update Nodes with Evidence! "# "$ =!("$ "#)!("#)!("$) All Bayesian methods are not equivalent. 13

14 Originated in Artificial Intelligence Early AI challenge: compact representation of data - Need 2 5-1=31 parameters to represent joint distribution 14

15 Originated in Artificial Intelligence Early AI challenge: compact representation of data - Need 2 5-1=31 parameters to represent joint distribution Compact representation via conditional independencies - 17 parameters instead of 31 15

16 First Solution Algorithms in Late 1980s 16

17 Hypotheses Machine-learned Bayesian networks can provide a unified framework for cancer, noncancer benefits enable quantification of health benefits improve accuracy of predicted benefits, compared to current methods support analysis of health benefits to support policy decisions 17

18 Case study demonstration Diabetes risks from arsenic in drinking water 18

19 Arsenic Has Many Health Effects High doses long known to cause blackfoot disease Established associations with bladder, lung cancers Emerging evidence of association with diabetes 19

20 Current US Standard for Arsenic in Drinking Water Considers Only Cancer Although arsenic causes numerous health effects, bladder cancer is the only endpoint for which an Agencyapproved metric for evaluating arsenic related risk currently exists. Proposed Arsenic in Drinking Water Rule Regulatory Impact Analysis U.S. Environmental Protection Agency, June

21 Cost-Benefit Analysis Considered Only Bladder Cancer Net Benefits (Bladder Cancers Prevented) Proposed Drinking Water Standard (µg/liter) $(538.9) $ (287.4) $(111.2) $(31.8) Benefit/Cost Ratio Proposed standard Final standard 21

22 Cohort Data from Mexico Provide Opportunity to Show New Approach 1,050 adults > 18 years old: without diabetes with diabetes Study area: Chihuahua, Mexico 22

23 Cohort Data from Mexico Provide Opportunity to Show New Approach 1,050 adults > 18 years old: without diabetes with diabetes Study area: Chihuahua, Mexico Variables in data set: - Arsenic in drinking water - Arsenic and metabolites in urine - Water source - Diet - Smoking - Anthropometry: BMI, waist size - Age, gender, education, ethnicity 23

24 High Arsenic Exposure in Study Area Frequency U.S. Standard Mexican Standard Arsenic Concentration (micrograms/liter) 24

25 Methods 25

26 Learned Bayesian Network from the Chihuahua Data Used BayesiaLab software: 1. Search equivalence classes of possible networks 2. Keep nodes within five links of diabetes 3. Eliminate nodes not significantly related to diabetes 4. Re-run using augmented naïve Bayes algorithm 5. Test via five-fold cross-validation 26

27 Compared Predictive Accuracy to Three Traditional Approaches 1. Reference dose method (noncancer) # if [%&] )*+ P(diabetes) = ", if [%&] > )*+ 2. Slope-factor method Use EPA s Benchmark Dose Software to determine slope factor 3. Logistic regression ln diabe : = ;, Dose ± ; >? > + 27

28 To Compare Accuracy, Used Receiver- Operating Characteristic (ROC) Curves True Positive Rate Random Guess False Positive Rate Diagnostic accuracy = area under curve (1=perfect) 28

29 Trade-off between specificity and sensitivity Random Guess True Positive Rate ROC Curves, continued False Positive Rate Diagnostic accuracy = area under curve (1=perfect) 29

30 Conducted Hypothetical Policy Analysis Frequency U.S. Standard Mexican Standard Arsenic Concentration (micrograms/liter) Treat all drinking water to < 25 µg/liter - Generic population - Vulnerable population: age>50, metabolic risk factors 30

31 Conducted Hypothetical Policy Analysis Frequency U.S. Standard Mexican Standard Frequency U.S. Standard Mexican Standard Arsenic Concentration (micrograms/liter) Arsenic Concentration (micrograms/liter) Treat all drinking water to < 25 µg/liter - Generic population - Vulnerable population: age>50, metabolic risk factors 31

32 Results 32

33 Learned Network Shows Multiple Connections Predictor variables influence each other 33

34 Reference Dose Method Has No Predictive Accuracy True Positive Rate Hazard quotient model, AUC= False Positive Rate 34

35 Slope-Factor Method Also Is Highly Inaccurate True Positive Rate Hazard quotient model, AUC=0.53 Slope factor model, AUC= False Positive Rate 35

36 Logistic Regression Has Predictive Power True Positive Rate Hazard quotient model, AUC=0.53 Slope factor model, AUC=0.55 Logistic model, AUC= False Positive Rate 36

37 Bayesian Network Has Highest Accuracy True Positive Rate Hazard quotient model, AUC=0.53 Slope factor model, AUC=0.55 Logistic model, AUC=0.79 Bayesian network model, AUC= False Positive Rate 37

38 Conventional Risk Analysis Overlooks Noncancer Benefits of Intervention Scenario: Arsenic < 25 µg/l in all water Cancer benefit: Cases = slope factor / = ; / <9;9 = 2 38

39 Conventional Risk Analysis Overlooks Noncancer Benefits of Intervention Scenario: Arsenic < 25 µg/l in all water Cancer benefit = 2 cancers prevented Cases = slope factor / = ; / <9;9 = 2 Non-cancer benefit = 0 39

40 Bayesian Network Can Estimate Diabetes Risk Reduction Benefits 40

41 Bayesian Network Can Estimate Diabetes Risk Reduction Benefits 41

42 Bayesian Network Can Estimate Diabetes Risk Reduction Benefits Cases = '( ) '( +,-./0.,-+1, = 23) 24) =10 42

43 Network Can Also Estimate Effects on Vulnerable Populations 43

44 Network Can Also Estimate Effects on Vulnerable Populations Age>55 High arsenic methylation during metabolism 44

45 Use-Friendly, Interactive Interfaces Can Facilitate Network use 45

46 Summary: 21 st -Century Dose-Response Assessment with Bayesian Networks Current methods: Are inconsistent for cancer, other illnesses May not be accurate Overlook complex interactions (e.g., genetics, environment) Cannot represent nonlinear relationships Bayesian networks could provide a new approach. User-friendly versions are possible. 46

47 Acknowledgements: Chihuahua Data Set University of North Carolina Chapel Hill Michelle A. Mendez (UNC, Department of Nutrition) Rebecca C. Fry (UNC, Department of Environmental Sciences & Engineering) Miroslav Stýblo (UNC, Department of Nutrition) Maria Ishida Daniela Gutiérrez-Torres R. Jesse Saunders Zuzana Drobná John Buse Dana Loomis Joseph Zabinski Universidad Autonoma De Chihuahua Blanca Sánchez-Ramírez Colegio de Médicos Cirujanos y Homeópatas del Estado de Chihuahua Damián Viniegra Morales Francisco A. Baeza Terrazas Facultad de Medicina, Universidad Juárez del Estado de Durango Gonzalo García-Vargas Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional Luz Del Razo 47

48 Funding Sources NIEHS Training Grant UNC Nutrition and Obesity Research Center UNC Center for Environmental Health and Susceptibility UNC Royster Society of Fellows 48