Nonstandard Machine Learning Algorithms for Microarray Data Mining. Byoung-Tak Zhang

Transcription

1 Nonstandard Machne Learnng Algorthms for Mcroarray Data Mnng Byoung-Tak Zhang Center for Bonformaton Technology (CBIT) & Bontellgence Laboratory School of Computer Scence and Engneerng Seoul Natonal Unversty Outlne! Mcroarray Bonformatcs! Probablstc Graphcal Models for Gene Expresson Analyss! Gene-Drug Dependency Analyss wth Bayesan Networks! Gene and Tssue Clusterng wth Latent Varable Models! Summary and Future Work 2

2 Mcroarray Bonformatcs Molecular Bology: Central Dogma 4

3 Reductonstc and Synthetc Approaches n Bology Bologcal System (Organsm) Reductonstc Approach (Experments) Synthetc Approach (Bonformatcs) Buldng Blocks (Genes/Molecules) 5 Topcs n Bonformatcs Sequence analyss 4 Sequence algnment 4 Structure and functon predcton 4 Gene fndng Structure analyss 4 Proten structure comparson 4 Proten structure predcton 4 RNA structure modelng Expresson analyss 4 Gene expresson analyss 4 Gene clusterng Pathway analyss 4 Metabolc pathway 4 Regulatory networks 6

4 Applcatons of DNA Mcroarrays! Gene dscovery! Analyss of gene regulaton! Dsease dagnoss! Drug dscovery: pharmacogenomcs! Toxcologcal research: toxcogenomcs A Comparatve Hybrdzaton Experment Image analyss 8

5 Image Analyss! Scanned mages " probe ntenstes " numercal values for hgher-level analyss Array target segmentaton Background ntensty extracton Target detecton Rato analyss Target ntensty extracton 9 Data Preparaton for Data Mnng Sample 1 Gene 2 Sample 1 Sample 2 Sample Image analyss Sample k Sample n <Mcroarray mage samples> <Numercal data for data mnng> 10

6 Gene Expresson Data Mnng Data preprocessng: - Normalzaton - Dscretzaton Mnng - Classfcaton - Clusterng - Gene selecton Learnng: - Greedy search - EM algorthm - Regulaton analyss 11 Why Data Mnng?! Tradtonal analyss One gene n one experment Small data sets Smple analytcal methods! Hgh-throughput genomc analyss Smultaneous measurements of thousands of gene expresson levels " massve data sets Statstcal methods Machne learnng approach 12

7 An Example of Data Mnng: Clusterng 13 Analyss of DNA Mcroarray Data Prevous Work! Characterstcs of data Analyss of expresson rato based on each sample Analyss of tme-varant data! Clusterng Self-organzng maps [Golub et al., 1999] Sngular value decomposton [Orly Alter et al., 2000]! Classfcaton Support vector machnes [Brown et al., 2000]! Gene dentfcaton Informaton theory [Stefane et al., 2000]! Gene modelng Bayesan networks [Fredman et.al., 2000] 14

8 Probablstc Graphcal Models for Gene Expresson Analyss An Introductory Example! A Bayesan network classfer for acute leukemas [Hwang et al. 2001] Zyxn Zyxn Zyxn Leukema ALL or AML Leukema LTC4S C-myb MB-1 <Network structure> LTC4S C-myb MB-1 Leukotrene C4 synthase (LTC4S) gene C-myb gene extracted from Human (c-myb) gene, complete prmary cds, and fve complete alternatvely splced cds MB-1 gene n P( X) = = P( X 1 Pa ) 16

9 Probablstc Graphcal Models! The jont probablty dstrbuton over X ={X 1, X 2,,X n } Chan rule P(X) = = P( X X,..., X! Condtonal ndependence X Y Z P ( X Y, Z) = P( X Z)! Effcent representaton of the jont probablty dstrbuton usng condtonal ndependences encoded by the graph (network) structure P(X) = = n = n = 1 P( X P( X X {X 1,,X -1 }-V(X ) V(X ) n 1 1 1) X,..., X V ( X )) ) 1 Analyss Procedure Gene Expresson Data Gene A Gene C Gene B Gene D Target <Learned Graphcal Models> - Classfcaton - Dependency analyss - Clusterng Dscretzaton and Selecton Learnng Graphcal Models Gene A Gene D Gene B Gene C Target Selected genes and the target varable 18

10 Why Probablstc Graphcal Models?! The jont probablty dstrbuton " all the knowledge about the (bologcal) system.! Generatve " modelng data sources! Effcent probablstc nference " predcton! The network structure " an nsght nto the ntrcate relatonshps between components (.e., genes) of the bologcal system " dependency analyss! Robust to the nose and error n gene expresson data 19 Classes of Graphcal Models Graphcal Models Undrected Drected - Boltzmann Machnes - Markov Random Felds - Bayesan Networks - Latent Varable Models - Hdden Markov Models -Generatve Topographc Mappng -Non-negatve Matrx Factorzaton 20

11 Gene-Drug Dependency Analyss wth Bayesan Networks NCI Drug Dscovery Program NCI 60 cell lnes data set 22

12 NCI 60 Cell Lnes Data Set! 60 human cancer cell lnes Colorectal, renal, ovaran, breast, prostate, lung, and central nervous system orgn cancers, as well as leukemas and melanomas.! Drug actvty patterns (Database A) Sulphorhodamne B assay " changes n total cellular proten after 48 hours of drug treatment.! Indvdual targets (matrx T ) Analyss of molecular characterstcs other than mrna expressons.! Ths study focuses on Senstvty to therapy #" not molecular consequences of therapy 23 Gene-Drug Dependency Analyss Usng Bayesan Networks! To dscover Gene-gene expresson dependency Gene expresson-drug actvty dependency Drug-drug actvty dependency Drug A Drug B Gene A Drug C Local probablty dstrbuton Gene B 24

13 Bayesan Networks: the Qualtatve Part! The drected-acyclc graph (DAG) structure Condtonal ndependences between varables " dependency analyss Gene E Gene A Gene B Gene C Gene D <Condtonal ndependences> - Gene B and Gene D are ndependent gven Gene A. - Gene B asserts dependency between Gene A and Gene E. - Gene A and Gene C are ndependent gven Gene B. <The DAG structure> 25 Bayesan Networks: thequanttatvepart! The jont probablty dstrbuton over all the varables n a Bayesan network. Gene E n P( X) = = P( X 1 Pa ) Gene B Gene A Gene D Local probablty dstrbuton for X Θ = ( θ,..., θ ) ~ parameter for P( X q r : : j # # 1 of of q P( θ ) = Dr( θ α,..., α ) j j1 states for X jr confguratons for Pa Pa ) Gene C P( A, B, C, D, E) = P( A) P( E A) P( B A, E) P( D A, E, B) P( C A, E, B, D) = P( A) P( E) P( B A, E) P( D A) P( C B) 26

14 Applcatons of Bayesan Networks! Classfcaton Bayesan network classfers Probablstc nference " predcton " cancer classfcaton based on gene expresson! Dependency analyss Bayesan network structure " dependency between components " putatve causal relatonshps 2 Dscretzaon of Gene Expresson Levels! The local probablty dstrbuton for each varable Multnomal dstrbuton wth Drchlet pror. p( θ S Γ(1) = 1, S h : j h ) = Dr( θ α, α,..., α ) j1 the network structure j! Dscretzaton of gene expresson levels and drug actvtes j 2 All values " 0 and 1 accordng to the mean values across the tranng samples. jr r Γ( αj α ) jr = r θ Γ( α ) k 1 k 1 jk = = Γ( x + 1) = xγ( x) α k jk 1 28

15 Selecton of Genes and Drugs for Analyss Known drugs Mssng values elmnaton human genes and ESTs - 82 drugs Learnng Bayesan networks wth 648 nodes 29 Learn the Bayesan Network from Data! Two approaches to Bayesan network learnng Dependency analyss-based approach Optmzaton-based approach! Network score: ftness of the network to tranng data D.! Search the best-scorng network structure! Scorng metrc for the network MDL (mnmum descrpton length) score BD (Bayesan Drchlet) score 30

16 BD (Bayesan Drchlet) Score! Assumptons Multnomal sample Parameter ndependence and modularty Drchlet pror p( θ S j ) = c Complete data h r αjk 1 k = θ 1 jk! BD score s calculated as follows: Pror probablty Suffcent statstcs calculated from D p( D, S) = = p( S) p( D S) p( S) n q Γ( αj ) r Γ( αjk + N k Γ( α + N ) ( α ) = 1 j= 1 = 1 j j Γ jk jk ). S: the network structure, D: the tranng data 31 Search Strategy! Fndng the best network structure " NP-hard problem. Exponental tme complexty. In general, greedy search or smulated annealng s used.! General greedy search algorthm s not applcable to the constructon of large-scale Bayesan networks wth hundreds of nodes. Tme and space complexty! Reduce the search space Sparse canddate algorthm by [Fredman et al. 1999] Local to global search algorthm! Explot the local search for the Markov blanket of each node to reduce the global search space. 32

17 Local to Global Search Algorthm F_a= 6 D/XX// 6 D_/Z_ZZX]/EXZX_/_`c\/cc/E 9 6 D/`^a`XY]/`cZ_/^cZ5 B Score( B, D) = Score( X Pa ( X ), D). La= D/EXZX_/_`c\/cc/E I``a 5/_Z]/`_c_ 6I`X]/Xc/a= 3/ EX/ `_/ X_/ E _ :5/ ]/ `c/ X/ XcZXY]/ Z 5/ X/ / E Z_ E Z_ / \2/ `/ X_ZX/JXc\`/Y]X_\/`/ Z 3/ `c/ X/ / Z 5/ E Z_ 5/ ]Xc_/ Z/ ]`X]/ cc/ X_/ c^z_/ / JXc\`/ Y]X_\/`/ Z 5/EI _ Z 25/c`^/Z/]`X]/cc 3/Jc/X]]//]`X]/_`c\/cc/ Z 5/EI _ Z 25/ Z 2/Z_`/X/]`YX]/_`c\/ cc/ _ X]]/]Z2 6]`YX]/Xc/a= 3/Z_//EXZX_/_`c\/cc/E _ _ 5/Z/^XZ^Z/`cE _ 5/2/X_/ cxz_/x]]/_`_6]z//z_/ _ 33 Learnng Curve BD score Learnng takes about 8 mnutes. # of greedy search steps 34

18 Drug-Drug Dependency! The pyramdne analogues Aphdcoln-glycnate Floxurdne Cyclocytdne Cytarabne DNA synthess nhbtor! Shows smlar actvty pattern across 60 cancer cell lnes! Clustered together n [Scherf et al. 2000]. 35 Expermental Results: Drug-Drug Dependency! Drug-drug actvty correlatons - Three drugs Aphdcolnglycnate, Floxurdne, and Cytarabne drectly depend on each other. - Cyclocytdne drectly depends on Cytarabne and vce versa. Confrmaton <Part of the learned Bayesan network structure> 36

19 Gene-Drug Dependency! Gene ASNS Certan malgnant cells, ncludng many acute lymphoblastc leukemas (ALL) lack asparagne synthetase (ASNS) " exogenous L-asparagne [Scherf et al. 2000]. Senstvty of the drug L-asparagne Expresson levels of the gene ASNS Hgh negatve correlaton 3 Expermental Results: Gene-Drug Dependency! Gene expresson-drug actvty correlatons ` Confrmaton Dscovery - Thenegatvecorrelaton between ASNS and Lasparagne s medated by two other genes. - The relatonshps revealed by the Bayesan network s putatve andshouldbeverfedby bologcal experments. " exploratory analyss <Part of the learned Bayesan network structure> 38

20 Gene and Tssue Clusterng wth Latent Varable Models CAMDA-2000 Data Set! Gene expresson data for cancer predcton Tranng data: 38 leukema samples (2 ALL, 11 AML) Test data: 34 leukema samples (20 ALL, 14 AML) Datasets contan measurements correspondng to ALL and AML samples from bone marrow and perpheral blood. 40

21 Cluster Analyss Usng Non-negatve Matrx Factorzaton! Method Usng NMF for class clusterng and predcton of gene expresson data from acute leukema patents! NMF (non-negatve matrx factorzaton) G WH r ( G) µ ( WH) µ = WaH aµ a= 1 G =/gene expresson data matrx W =/bass matrx (prototypes) H =/encodng matrx (n low dmenson) G, W, H 0 µ a aµ! NMF as a latent varable model h 1 h 2 W g 1 g 2 < g >= Wh g n h r 41 Clusterng Gene Expresson Data H 1 H 2 G W(?) H(?) W,129 genes,129 genes 38 samples encodng g 1 g 2 g 3 g 4 g, samples 2factors! Factors can capture the correlatons between the genes usng the values of expresson level.! Cluster tranng samples nto 2 groups by NMF Assgn each sample to the factor (class) whch has hgher encodng value. 42

22 Learnng Procedure Input : Gene expresson data matrx, G (n m) Output : base matrx W (n k), encodng matrx H (k m) n: data sze, m: number of genes, k: number of latent varables Objectve functon : Procedure 1. Intalze W, H wth random numbers. W H j j 0, 0 j W j = 1 F = n m [ G µ log( WH ) µ ( WH ) µ ] = 1 µ = 1 2. Update W, H teratvely untl max_teraton or some crteron s met. a j µ W a W a H aµ µ ( WH ) µ W W a W ja G H aµ = H aµ W a G µ ( WH ) µ 43 Learnng Curve 44

23 Expermental Results: Cancer Clusterng Accuracy: 0 ~1 error for the tranng data set 45 Dagnoss Usng NMF h 1 h 2 W g W,129 genes h(?),129 genes g 1 g 2 g 3 g 4 g,129 2factors! For each test sample g, estmate the encodng vector h that best approxmates the sample. W s the bass matrx computed durng tranng (fxed). As n tranng, assgn each sample to the factor (class) whch has the hghest encodng value.! Accuracy: 1~2 error(s) for the test data set 46

24 Clusterng Usng Generatve Topographc Mappng! GTM: a nonlnear, parametrc mappng y(x;w) from a latent space to a data space. Grd t 3 yx>w2 x 2 t 2 x 1 t 1 <Latent space> <Data space> 4 Learnng Generatve Topographc Mappng! Learnng algorthm!generate the grd of latent ponts.!generate the grd of latent functon centers.!compute the matrx of bass functon actvatons Φ.!Intalze weghts W n Y = ΦW and the nose varance β.!compute n,k = t n Φ k W 2 = t n y k (x,w) 2 for each n, k!repeat 4 - Compute the responsblty matrx R usng and β. [E-Step] 4 Compute G=R T R 4 - Update W by Φ T GΦW= Φ T RT [M-step] 4 - Compute = t ΦW Update β!untl convergence 48

25 GTM: Vsualzaton! Posteror dstrbuton n latent space gven a data pont t: X(t) ~! For a whole set of data: for each t, plot n the latent space Posteror mode: Posteror mean: 49 GTM: Clusterng Experment! Gene Selecton Select about 50 genes out of,129 based on the three test scores of cancer dagnoss.! Correlaton metrc (smlar as t-test)! Wlcoxon test scores (a nonparametrc t-test)! Medan test scores (a nonparametrc t-test)! Clusterng & Vsualzaton After learnng a model, genes are plotted n the latent space. Wth the mappng n the latent space, clusters can be dentfed. 50

26 Lst of Genes Selected 51 GTM: Learnng Curve 52

27 GTM: Clusterng Result Genes wth hgh expresson levels n case of ALL (large P-metrc value) Genes wth hgh expresson levels n case of AML (negatve large P-mertc value) 53 Expermental Results: Clusters Found by GTM! Three cell cycle-regulated clusters found by GTM [Shn et al. 2000] Cluster center No. of tran Data/ no. n cluster Correct no. / test data Overall mean expresson levels (Cln/b) of known genes S/G2 5 / 1 / 2 ( ) S ( ) 5 / 5 5 / 5 (100%) ( ) M/G1 c1 c2 c3 ( ) ( ) ( ) 13 / / 2 / 2 1 / 6 0 / 6 0 / 6 ( ) G2/M c1 c2 ( ) ( ) 10 / 5 / 3 0 / 5 3 / 5 (80%) ( ) G1 c1 c2 ( ) ( ) 35 / 18 / 10 / 16 (62%) 0/16 ( ) 54

28 Expermental Results: Comparson wth Other Methods! Comparson of prototype expresson levels No. of selected genes Mean expresson levels by GTM No. of selected genes by Spellman Mean expresson levels by Spellman S/G2 92 ( ) 121 ( ) S 25 ( ) 1 ( ) M/G1 c1 c2 c ( ) ( ) ( ) 113 ( ) G2/M c1 c ( ) ( ) 195 ( ) G1 c1 c (total = 50) ( ) ( ) 300 (total = 800) ( ) 55 Concluson

29 Summary and Future Work! A class of nonstandard learnng algorthms,.e. probablstc graphcal models, s presented for mcroarray data analyss: Bayesan networks Non-negatve matrx factorzaton Generatve topographc mappng! Probablstc graphcal models are useful for bologcal data mnng: Comprehensblty Generatve Soft nference (clusterng) Dependency analyss! Future work ncludes: Constructon of larger-scale networks Acceleraton of learnng processes Handlng data sparseness and mssng data problems 5 References! [Chang et al. 2001] Chang, J.-H., Hwang, K.-B., Zhang, B.-T., Analyss of gene expresson profles and drug actvty patterns for the molecular pharmacology of cancer, In Proceedngs of CAMDA 01, 2001 (to appear).! [Fredman et al. 1999] Fredman, N., Nachman, I., and Pe er, D., Learnng Bayesan network structure from massve datasets: the sparse canddate algorthm, In Proceedngs of UAI 99, 1999.! [Hwang et al. 2001] Hwang, K.-B., Cho, D.-Y., Park, S.-W., Km, S.-D., and Zhang, B.-T., Applyng machne learnng technques to analyss of gene expresson data: cancer dagnoss, Methods of Mcroarray Data Analyss, , 2001.! [Scherf et al. 2000] Scherf, U. et al., A gene expresson database for the molecular pharmacology of cancer, Nature Genetcs, 24: , 2000.! [Shn et al. 2000] Shn, H.-J., Chang, J.-H., Yang, J.-S., Zhang, B.-T., and Augh, S.-J., Probablstc models for clusterng cell cycle-regulated genes n the Yeast, In Proceedngs of CAMDA 00,

30 Acknowledgements CAMDA-2000 & CAMDA-2001! Srk-June Augh (Molecular Bology)! Jeong-Ho Chang (Helmholtz Machnes, NMF, PLSA)! Dong-Yeon Cho (Bayesan Evoluton, Neural Trees)! Kyu-Baek Hwang (Bayesan Networks)! Sung-Dong Km (Herarchcal Clusterng)! Sang-Wook Park (RBF Networks)! Hyung-Joo Shn (Probablstc LSA)! Jn-San Yang (Statstcs, GTM, SOM)! Ho-Jean Chung (Statstcal Learnng)! Seung-Woo Chung (ICA, PCA)! Jae-Hong Eom (Hdden Markov Models)! Tae-Jn Jeong (Renforcement Learnng)! Je-Gun Joung (Evoluton, Neural Trees)! Chul-Joo Kang (Molecular Bology)! Jun-Shk Km (Statstcal Physcs)! Sun Km (Evolutonary Algorthms)! Yoo-Hwan Km (ICA, Naïve Bayes, Boostng)! In-Hee Lee (Evolutonary Computaton)! Jae-Won Lee (Renforcement Learnng)! Jong-Woo Lee (Latent Varable Models)! S-Eun Lee (Bayesan Evoluton)! Seung-Joon Lee (Renforcement Learnng)! Jong-Yoon Lm (SOM, Graphcal Models)! Hyun-Koo Moon (HMM, Naïve Bayes)! Jang-Mn O (Support Vector Machnes, Boostng)! Seong-Bae Park (Decson Trees)! Hyung-Joo Shn (Probablstc LSA)! Soo-Yong Shn (Evoluton, Helmholtz Machnes) 59 More nformaton at