Introduction to Bayesian Networks

Transcription

1 Introduction to Bayesian Networks Jun Zhu, Ph. D. Department of Genomics and Genetic Sciences Icahn Institute of Genomics and Multiscale Biology Icahn Medical School at Mount Sinai New York,

2 Outline 1. What are Bayesian networks? 2. What are usages of Bayesian networks? 3. What is a naïve Bayes net? 4. How to train a Bayesian network? 5. How to construct a Bayesian network? 6. What is different in biology system for BN? 7. What is Dynamic BN?

3 What are Bayesian networks? A Bayesian network is an expert system that captures all existing knowledge; They are also called belief networks, Bayesian belief networks, causal probabilistic networks; A Bayesian network consists of a directed acyclic graph (a set of nodes and directed edges connecting nodes)--dag A set of conditional probability tables (for discrete data) or probability density functions (for continuous data)

4 A Bayesian network A B p( C A, B) p(d B) C D E p(e B) F DAG p(f C) Conditional probability tables

5 A tree is a Bayesian network B C D E F A

6 A Bayesian network is not a tree A B C D E F

7 Conventional Notations p( A) p( Ai pa( Ai)) i A {A 1,A 2,,A n} are nodes. p( A) is the joint probability of nodes A. pa( A ) are parent nodesof A. i i

8 A diverging structure A out-degree =4 B C D E

9 A converging structure A B C in-degree =3 D

10 Why a DAG is required? p( A) p( Ai pa( Ai)) i It is guaranteed that there is a node Aj in a DAG that has no child. p( A) p( A \ { A }) p( A A \ { A }) j p( A \ { A }) p( A pa( A )) j j j ( p( Ai pa( Ai )))* p( Aj pa( Aj)) i j j j

11 : usages Bayesian networks can be used to predict outcomes or diagnose causal effects (if structures are known) Bayesian networks can be used to discover causal relationships (if structures are not known)

12 : an example A burglar alarm system burglar Earthquake Alarm Radio Internet Phonecall

13 : a classifier What is a naïve Bayes net A B C D E p( A,B,C,D,E)= p(b A)p(C A)p(D A)p(E A) p( A B, C, D) p( A,B,C,D,E) p( B,C,D,E)

14 How to train a Bayesian network A A=a1 A=a2 A=a3 B=b B=b B=b B C A=a1 A=a2 A=a3 C=c C=c C=c

15 How to construct a Bayesian network? Enumerating possible structures A A A A B C B C B C B C A A A B C B C B C

16 How to construct a Bayesian network? Enumerating all possible structures is impossible N N, N is thenumber of nodes

17 How to construct a Bayesian network? Heuristic approach Pa 1 Pa 2 Pa n x i a b c Pa 1 Pa 2 Pa n X Pa 1 Pa 2 Pa n Pa n+1 Pa 1 Pa 2 Pa j Pa n x i x i x i

18 How to construct a Bayesian network? Heuristic approach A B A B C D D Parameters to estimate=3x3x3 Parameters to estimate=3x3x3x3

19 How to construct a Bayesian network? Heuristic approach p( M D) p( D M ) p( M ) pd ( ) 2ln( p( M D)) BIC 2ln( D Mˆ ) k ln( n) n k : number of samples : number of parameters to estimate

20 How to construct a Bayesian network? averaging Zhu et al., PLoS CompBio, 2007 Zhu et al., Nature Genetics, 2008

21 How to construct a Bayesian network? Enforcing DAG after averaging 1. Calculate shortest distance 2. Identify loops 3. Remove the weakest link in a loop 4. Go to step 1 Zhu et al., PLoS CompBio, 2007

22 How to construct a Bayesian network? Upper limit on in-degree Pa 1 Pa 2 Pa n x i Parameters to estimate= 3 n 1

23 Continuous vs discrete models Discrete model is faster, easier to capture high order interactions Any discretization lost information

24 Missing information A X A X A B C B C B C

25 Feedbacks in biological systems dg dt dc dt n g n n c g c v( g, c) ( g, c) Chen et al., Nature, 2008

26 Feedbacks in biological systems Changing parameters in activation results in negative correlation.

27 Feedbacks in biological systems Changing parameters in inhibition results in positive correlation.

28 Biological network is context specific Bayesian network is just a snapshot under a specific condition

29 Dynamic Bayesian Bayesian network? Dynamic Bayesian network Granger causality Zhu et al., PLoS CompBio, 2010

30 Granger s causality test y y n, t n n, t 1 n, t y y x n, t n n, t 1 n n, t 1 n, t Zhu et al., PLoS CompBio, 2010

31 Dynamic Bayesian network 1 T i i T t t t 1 i p( X,, X ) p( X Pa( X )) i t t 1 t Pa( X ) X X Zhu et al., PLoS CompBio, 2010

32 Dynamic Bayesian network: Assumptions 1. Sampling time is faster than reaction time 2. Sampling time is the exact same as the reaction time 3. Sample time is slower than reaction time Zhu et al., PLoS CompBio, 2010

33 Aknowledgements Sage Bionetworks Stephen Friend et al. Mount Sinai Eric Schadt Bin Zhang Zhidong Tu Decode Valur Emilsson UCLA Jake Lusis Xia Yang, et al U Washington Roger Baumgarner Berkerley Rachel Brem Princeton Lenoid Kruglyak Harvard Jun Liu U Wisconsin Alan Attie Mark Keller, et al Merck Qiuwei Xu Ethan Xu Theretha Zhang Fred Hutchingson Paddison lab MD Anderson Hanash lab Mount Sinai Powell lab Oh Lab Casaccia

34 Aknowledgements Icahn Institute of Genomics and Multiscale Biology, Icahn Medical School at Mount Sinai Canary Foundation Prostate Cancer Foundation NIH NCI