Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics

Transcription

1 Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics Karsten Borgwardt February 21 to March 4, 2011 Machine Learning & Computational Biology Research Group MPIs Tübingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1

2 Clustering in bioinformatics Microarrays Clustering is a widely used tool in microarray analysis Class discovery is an important problem in microarray studies for two reasons: either the classes are completely unknown beforehand or it is unknown whether a known class contains interesting subclasses Karsten Borgwardt: Data Mining in Bioinformatics, Page 2

3 Clustering in bioinformatics Examples Classes unknown: Does a disease affect gene expression in a particular tissue? Does gene expression differ between two groups in a particular condition? Subclasses unknown: Are there subtypes of a disease? Is there even a hierarchy of subclasses within one disease? Karsten Borgwardt: Data Mining in Bioinformatics, Page 3

4 Clustering in bioinformatics Popularity Clustering tools are available in the large microarray database NCBI Gene Expression Omnibus (GEO) pubmed hits for microarray clustering Recent editorial of OUP Bioinformatics Karsten Borgwardt: Data Mining in Bioinformatics, Page 4

5 Distance metrics Euclidean distance Euclidean distance of gene x and y of n samples or sample x and y of n genes: d xy = n (x i y i ) 2 (1) Pearson s Correlation i=1 Pearson Correlation of gene x and y of n samples or sample x and y of n genes, where x is the mean of x and is ȳ the mean of y: n i=1 r xy = (x i x)(y i ȳ) n i=1 (x i x) 2 n i=1 (y (2) i ȳ) 2 Karsten Borgwardt: Data Mining in Bioinformatics, Page 5

6 Distance metrics Un-centered correlation coefficient Un-centered correlation coefficient of gene x and y of n samples or sample x and y of n genes: r u xy = n i=1 x2 i n i=1 x iy i n i=1 y2 i (3) Karsten Borgwardt: Data Mining in Bioinformatics, Page 6

7 Clustering algorithms Hierarchical Clustering Single linkage: The linking distance is the minimum distance between two clusters. Complete linkage: The linking distance is the maximum distance between two clusters. Average linkage/upgma (The linking distance is the average of all pair-wise distances between members of the two clusters. Since all genes and samples carry equal weight, the linkage is an Unweighted Pair Group Method with Arithmetic Means (UPGMA)) Flat Clustering k-means (k from 2 to 15, 3 runs) k-median (k-medoid) Karsten Borgwardt: Data Mining in Bioinformatics, Page 7

8 The two-sample problem Interpretation of clusters Clustering introduces structure into microarray datasets But is there a statistical or biomedical meaning of these classes? Biomedical meaning has to be established in experiments Statistical meaning can be measured using statistical tests, by a so-called two-sample test A two-sample tests decides whether two samples were drawn from the same probability distribution or not Karsten Borgwardt: Data Mining in Bioinformatics, Page 8

9 The two-sample problem Data diversity Molecular biology produces a wealth of information The problem is that these data are generated on different platforms and by different protocols under different levels of noise Hence data from different labs show different scales different ranges different distributions Main problem: Joint data analysis may detect differences in distributions, not biological phenomena! Karsten Borgwardt: Data Mining in Bioinformatics, Page 9

10 The two-sample problem The two-sample problem Given two samples X and Y. Were they generated by the same distribution? Previous approaches two-sample tests exist for univariate and multivariate data Karsten Borgwardt: Data Mining in Bioinformatics, Page 10

11 The two-sample problem t-test A test of the null hypothesis that the means of two normally distributed populations are equal unpaired/independent (versus paired) For equal sample sizes and equal variances, the t statistic to test whether the means are different can be calculated as follows: t = x ȳ σ xy 2 n σ 2 where σ xy = x +σy 2 2. The degrees of freedom for this test is 2n 2 where n is the size of each sample. (4) Karsten Borgwardt: Data Mining in Bioinformatics, Page 11

12 The two-sample problem New challenges in bioinformatics high-dimensional structured (strings and graphs) low sample size Novel distribution test: (MMD) Maximum Mean Discrepancy Karsten Borgwardt: Data Mining in Bioinformatics, Page 12

13 MMD key idea Karsten Borgwardt: Data Mining in Bioinformatics, Page 13

14 MMD key idea Key Idea Theorem Theorem Avoid density estimator, use means in feature spaces Maximum Mean Discrepancy (Fortet and Mourier, 1953) D(p, q, F) := sup E p [f(x)] E q [f(y)] f F D(p, q, F) = 0 iff p = q, when F = C 0 (X). Follows directly, e.g. from Dudley, D(p, q, F) = 0 iff p = q, when F = {f f H 1} provided that H is a universal RKHS. (follows via Steinwart, 2001, Smola et al., 2006). Karsten Borgwardt: Data Mining in Bioinformatics, Page 14

15 MMD statistic Goal: Estimate D(p, q, F) E p,p k(x, x ) 2E p,q k(x, y) + E q,q k(y, y ) U-Statistic: Empirical estimate D(X, Y, F) k(x i, x j ) k(x i, y j ) k(y i, x j ) + k(y i, y j ) 1 m(m 1) i j Theorem D(X, Y, F) is an unbiased estimator of D(p, q, F). Test Estimate σ 2 from data. Reject null hypothesis that p = q if D(X, Y, F) exceeds acceptance threshold. Karsten Borgwardt: Data Mining in Bioinformatics, Page 15

16 Attractive for bioinformatics MMD two-sample test in terms of kernels Computationally attractive search infinite space of functions by evaluating one expression no optimization problem has to be solved All thanks to kernels! Karsten Borgwardt: Data Mining in Bioinformatics, Page 16

17 Attractive for bioinformatics Wide applicability for one- and higher-dimensional vectorial data, but also for structured data! two-sample problems can now be tackled on strings: protein and DNA sequences graphs: molecules, protein interaction networks time series: time series of microarray data and sets, trees,... Karsten Borgwardt: Data Mining in Bioinformatics, Page 17

18 Cross-platform comparability Data Task microarray data from two breast cancer studies one on cdna platform (Gruvberger et al., 2001) other on oligonucleotide microarray platform (West et al., 2001) Can MMD help to find out if two sets of observations were generated by the same study (both from Gruvberger or both from West)? different studies (one Gruvberger, one West)? Karsten Borgwardt: Data Mining in Bioinformatics, Page 18

19 Cross-platform comparability Experiment sample size each: 25 dimension of each datapoint 2,116 significance level: α = times: 1 sample from Gruvberger, 1 from West 100 times: both from Gruvberger or both from West report percentage of correct decisions compare to t-test, Friedman-Rafsky Wald-Wolfowitz and Smirnov Karsten Borgwardt: Data Mining in Bioinformatics, Page 19

20 Cross-platform comparability Karsten Borgwardt: Data Mining in Bioinformatics, Page 20

21 Kernel-based statistical test novel statistical test for two-sample problem: easy to implement non-parametric first for structured data best on high-dimensional data quadratic runtime w.r.t. the number of data points impressive accuracy in our experiments kernel method for two-sample problem: all kernels recently defined in molecular biology can be re-used for data integration applicable to vectors, strings, sets, trees, graphs and time series Karsten Borgwardt: Data Mining in Bioinformatics, Page 21

22 Biclustering Clustering in two dimensions alternative names: co-clustering, two-mode clustering A bicluster is a subset of genes that show similar activity patterns under a subset of conditions. Clustering in 2 dimensions Cluster patients and conditions Earliest work by Hartigan, 1972: Divide a matrix into submatrices with minimum variance. Most interesting cases are NP-complete. Many extensions in bioinformatics (e.g. Cheng and Church, 2002) Karsten Borgwardt: Data Mining in Bioinformatics, Page 22

23 References and further reading References [1] Gretton, Borgwardt, Rasch, Schölkopf, Smola: A kernel method for the two-sample problem. NIPS 2006 Karsten Borgwardt: Data Mining in Bioinformatics, Page 23

24 The end See you tomorrow! Next topic: Feature Selection in Bioinformatics Karsten Borgwardt: Data Mining in Bioinformatics, Page 24