Classification of Microarray Gene Expression Data

Transcription

1 Classification of Microarray Gene Expression Data Geoff McLachlan Department of Mathematics & Institute for Molecular Bioscience University of Queensland

2 Institute for Molecular Bioscience, University of Queensland

3 A wide range of supervised and unsupervised learning methods have been considered to better organize data, be it to infer coordinated patterns of gene expression, to discover molecular signatures of disease subtypes, or to derive various predictions. Statistical Methods for Gene Expression: Microarrays and Proteomics

4

5 by C. Tilstone Nature 424, , DNA microarrays have given geneticists and molecular biologists access to more data than ever before. But do these researchers have the statistical knowhow to cope? Branching out: cluster analysis can group samples that show similar patterns of gene expression.

6 MICROARRAY DATA REPRESENTED by a p n matrix x x x 1 n contains the gene expressions for the p genes of the th tissue sample ( = 1,, n). p = No. of genes ( ) n = No. of tissue samples ( ) STANDARD STATISTICAL METHODOLOGY APPROPRIATE FOR n >> p HERE p >> n

7

8 Two Groups in Two Dimensions. All cluster information would be lost by collapsing to the first principal component. The principal ellipses of the two groups are shown as solid curves.

9 bioarray News (2, no. 35, 2002) Arrays Hold Promise for Cancer Diagnostics Oncologists would like to use arrays to predict whether or not a cancer is going to spread in the body, how likely it will respond to a certain type of treatment, and how long the patient will probably survive. It would be useful if the gene expression signatures could distinguish between subtypes of tumours that standard methods, such as histological pathology from a biopsy, fail to discriminate, and that require different treatments.

10 van t Veer & De Jong (2002, Nature Medicine 8) The microarray way to tailored cancer treatment In principle, gene activities that determine the biological behaviour of a tumour are more likely to reflect its aggressiveness than general parameters such as tumour size and age of the patient. (indistinguishable disease states in diffuse large B-cell lymphoma unravelled by microarray expression profiles Shipp et al., 2002, Nature Med. 8)

11 by C. M. Schubert Nature Medicine 9, 9, The Netherlands Cancer Institute in Amsterdam is to become the first institution in the world to use microarray techniques for the routine prognostic screening of cancer patients. Aiming for a June 2003 start date, the center will use a panoply of 70 genes to assess the tumor profile of breast cancer patients and to determine which women will receive aduvant treatment after surgery.

12 Microarrays also to be used in the prediction of breast cancer by Mike West (Duke University) and the Koo Foundation Sun Yat-Sen Cancer Centre, Taipei Huang et al. (2003, The Lancet, Gene expression predictors of breast cancer).

13 CLASSIFICATION OF TISSUES SUPERVISED CLASSIFICATION (DISCRIMINANT ANALYSIS) We OBSERVE the CLASS LABELS y 1,, y n where y = i if th tissue sample comes from the ith class (i=1,,g). AIM: TO CONSTRUCT A CLASSIFIER C(x) FOR PREDICTING THE UNKNOWN CLASS LABEL y OF A TISSUE SAMPLE x. e.g. g = 2 classes G 1 -DISEASE-FREE G 2 -METASTASES

14

15 LINEAR CLASSIFIER FORM C x 0 β T x β 0 for the production of the group label y of a future entity with feature vector x. β 1 x 1 β p x p

16 FISHER S LINEAR DISCRIMINANT FUNCTION C y ) ( ) ( 2 1 ) ( x x S x x x x S β T and x 1 x 2 covariance matrix found from the training data where and S are the sample means and pooled sample

17 SUPPORT VECTOR CLASSIFIER Vapnik (1995) C x subect to min β, β where β 0 and β are obtained as follows: 0 0, y C( x ) β β x 1 n 1 ( 1,, n) β p x p 1,, n relate to the slack variables separable case

18 n y x β 1 ˆ ˆ with non-zero ˆ only for those observations for which the constraints are exactly met (the support vectors) ˆ, ˆ ˆ ˆ ) ( n n T y y C x x x x x

19 Support Vector Machine (SVM) REPLACE x x h ˆ ), ( ˆ ˆ ) ( ), ( ˆ ) ( n n K h h C x x x x x where the kernel function ) ( ), ( ), ( x x x x h h K is the inner product in the transformed feature space. by

20 HASTIE et al. (2001, Chapter 12) The Lagrange (primal function) is (1) ) (1 ) ( n n n P C y L x β which we maximize w.r.t. β, β 0, and ξ. Setting the respective derivatives to zero, we get )., 1, ( 0 0, 0, (4) )., 1, ( (3) (2) 1 1 n n y y n n x β with and

21 (5) k T k k n n k n D y y L x x We maximize (5) subect to n y 1 0. and 0 In addition to (2) to (4), the constraints include., 1, for (8) 0 ) (1 ) ( (7) 0 (6) 0 ) (1 ) ( n C y C y x x Together these equations (2) to (8) uniquely characterize the solution to the primal and dual problem. By substituting (2) to (4) into (1), we obtain the Lagrangian dual function

22 Leo Breiman (2001) Statistical modeling: the two cultures (with discussion). Statistical Science 16, Discussants include Brad Efron and David Cox

23 Selection bias in gene extraction on the basis of microarray gene-expression data Ambroise and McLachlan Proceedings of the National Academy of Sciences Vol. 99, Issue 10, , May 14,

24 GUYON, WESTON, BARNHILL & VAPNIK 2002 Machine Learning COLON Data (Alon et al., 1999) LEUKAEMIA Data (Golub et al., 1999)

25 Since p>>n, consideration given to selection of suitable genes SVM: FORWARD or BACKWARD (in terms of magnitude of weight β i ) RECURSIVE FEATURE ELIMINATION (RFE) FISHER: FORWARD ONLY (in terms of CVE)

26 LEUKAEMIA DATA: Only 2 genes are needed to obtain a zero CVE (cross-validated error rate) COLON DATA: Using only 4 genes, CVE is 2%

27 The success of the RFE indicates that RFE has a built in regularization mechanism that we do not understand yet that prevents overfitting the training data in its selection of gene subsets.

28 Figure 1: Error rates of the SVM rule with RFE procedure averaged over 50 random splits of colon tissue samples

29 Figure 2: Error rates of the SVM rule with RFE procedure averaged over 50 random splits of leukemia tissue samples

30 Figure 3: Error rates of Fisher s rule with stepwise forward selection procedure using all the colon data

31 Figure 4: Error rates of Fisher s rule with stepwise forward selection procedure using all the leukemia data

32 Figure 5: Error rates of the SVM rule averaged over 20 noninformative samples generated by random permutations of the class labels of the colon tumor tissues

33 G 1 G 2 C(x) (x 1, x 2, x 3,, x n ) C(x).

34 From the original data set, remove x 1 to give the reduced set (x 2, x 3,, x n ) Then form the classifier C (1) (x ) from this reduced set. Use C (1) (x 1 ) to allocate x 1 to either G 1 or G 2.

35 Repeat this process for the second data point, x 2. So that this point is assigned to either G 1 or G 2 on the basis of the classifier C (2) (x 2 ). And so on up to x n.

36 Figure 1: Error rates of the SVM rule with RFE procedure averaged over 50 random splits of colon tissue samples

37 ADDITIONAL REFERENCES Selection bias ignored: XIONG et al. (2001, Molecular Genetics and Metabolism) XIONG et al. (2001, Genome Research) ZHANG et al. (2001, PNAS) Aware of selection bias: SPANG et al. (2001, Silico Biology) WEST et al. (2001, PNAS) NGUYEN and ROCKE (2002)

38 BOOTSTRAP APPROACH Efron s (1983, JASA).632 estimator B AE.632 B1 where B1 is the bootstrap when rule is applied to a point not in the training sample. A Monte Carlo estimate of B1 is where E B1 K k 1 I k n 1 Q k E K n k 1 with and I k I Q k k R k * if x kth bootstrap sample otherwise if * R k misallocates x otherwise

39 Toussaint & Sharpe (1975) proposed the ERROR RATE ESTIMATOR A (w) (1 - w)ae wcv2e where w 0.5 McLachlan (1977) proposed w=w o where w o is chosen to minimize asymptotic bias of A(w) in the case of two homoscedastic normal groups. Value of w 0 was found to range between 0.6 n and 0.7, depending on the values of 1,, and. p n 2

40 .632+ estimate of Efron & Tibshirani (1997, JASA) B. 632 (1 - w)ae wb1 where.632 w r B1 AE r AE g i1 (1 pi qi ) (relative overfitting rate) (estimate of no information error rate) If r = 0, w =.632, and so B.632+ = B.632 r = 1, w = 1, and so B.632+ = B1

41 One concern is the heterogeneity of the tumours themselves, which consist of a mixture of normal and malignant cells, with blood vessels in between. Even if one pulled out some cancer cells from a tumour, there is no guarantee that those are the cells that are going to metastasize, ust because tumours are heterogeneous. What we really need are expression profiles from hundreds or thousands of tumours linked to relevant, and appropriate, clinical data. John Quackenbush

42 UNSUPERVISED CLASSIFICATION (CLUSTER ANALYSIS) INFER CLASS LABELS y 1,, y n of x 1,, x n Initially, hierarchical distance-based methods of cluster analysis were used to cluster the tissues and the genes Eisen, Spellman, Brown, & Botstein (1998, PNAS)

43 Hierarchical (agglomerative) clustering algorithms are largely heuristically motivated and there exist a number of unresolved issues associated with their use, including how to determine the number of clusters. in the absence of a well-grounded statistical model, it seems difficult to define what is meant by a good clustering algorithm or the right number of clusters. (Yeung et al., 2001, Model-Based Clustering and Data Transformations for Gene Expression Data, Bioinformatics 17)

44 Attention is now turning towards a model-based approach to the analysis of microarray data For example: Broet, Richarson, and Radvanyi (2002). Bayesian hierarchical model for identifying changes in gene expression from microarray experiments. Journal of Computational Biology 9 Ghosh and Chinnaiyan (2002). Mixture modelling of gene expression data from microarray experiments. Bioinformatics 18 Liu, Zhang, Palumbo, and Lawrence (2003). Bayesian clustering with variable and transformation selection. In Bayesian Statistics 7 Pan, Lin, and Le, 2002, Model-based cluster analysis of microarray gene expression data. Genome Biology 3 Yeung et al., 2001, Model based clustering and data transformations for gene expression data, Bioinformatics 17

45 The notion of a cluster is not easy to define. There is a very large literature devoted to clustering when there is a metric known in advance; e.g. k-means. Usually, there is no a priori metric (or equivalently a user-defined distance matrix) for a cluster analysis. That is, the difficulty is that the shape of the clusters is not known until the clusters have been identified, and the clusters cannot be effectively identified unless the shapes are known.

46 In this case, one attractive feature of adopting mixture models with elliptically symmetric components such as the normal or t densities, is that the implied clustering is invariant under affine transformations of the data (that is, under operations relating to changes in location, scale, and rotation of the data). Thus the clustering process does not depend on irrelevant factors such as the units of measurement or the orientation of the clusters in space.

47 x

48 MIXTURE OF g NORMAL COMPONENTS ) ; ( ) ; ( ) ( 1 g g g 1 1 f Σ, µ x Σ, µ x x ) ( ) ( µ x µ x T EUCLIDEAN DISTANCE ) ( ) ( ) ( log 2 µ x Σ µ x Σ µ, x; 1 T where constant constant ) ( ) ( ) ( log 2 µ x Σ µ x Σ µ, x; 1 T MAHALANOBIS DISTANCE where

49 MIXTURE OF g NORMAL COMPONENTS f ( x) ( x; µ 1, Σ1) ( x; µ 1 g g g, Σ ) k-means σ 2 1 g SPHERICAL CLUSTERS

50 Equal spherical covariance matrices

51 Figure 6: Plot of Crab Data

52 Figure 7: Contours of the fitted component densities on the 2 nd & 3 rd variates for the blue crab data set.

53 With a mixture model-based approach to clustering, an observation is assigned outright to the ith cluster if its density in the ith component of the mixture distribution (weighted by the prior probability of that component) is greater than in the other (g-1) components. f ( x) 1 ( x; µ 1, Σ1) i ( x; µ i, ( x; µ, Σ ) g g g Σ i )

54 Finite Mixture Models..

55 It was the publication of the seminal paper of Dempster, Laird, and Rubin (1977) on the EM algorithm that greatly stimulated interest in the use of finite mixture distributions to model heterogeneous data. McLachlan and Krishnan (1997, Wiley)

56 If need be, the normal mixture model can be made less sensitive to outlying observations by using t component densities. With this t mixture model-based approach, the normal distribution for each component in the mixture is embedded in a wider class of elliptically symmetric distributions with an additional parameter called the degrees of freedom.

57 The advantage of the t mixture model is that, although the number of outliers needed for breakdown is almost the same as with the normal mixture model, the outliers have to be much larger.

58

59 Clustering of genes on basis of tissues genes not independent Clustering of tissues on basis of genes - latter is a nonstandard problem in cluster analysis (n << p)

60 McLachlan, Peel, Adams, and Basford (1999)

61 EMMIX for Windows

62 PROVIDES A MODEL-BASED APPROACH TO CLUSTERING McLachlan, Bean, and Peel, 2002, A Mixture Model- Based Approach to the Clustering of Microarray Expression Data, Bioinformatics 18,

63

64 n=62 (40 tumours; 22 normals) tissue samples of p=2,000 genes in a 2, matrix.

65

66

67 t

68 t

69 t

70

71

72 In this process, the genes are being treated anonymously. May wish to incorporate existing biological information on the function of genes into the selection procedure. Lottaz and Spang (2003, Proceedings of 54 th Meeting of the ISI) They structure the feature space by using a functional grid provided by the Gene Ontology annotations.

73

74

75 Grouping for Colon Data

76

77

78

79 Grouping for Colon Data

80 A normal mixture model without restrictions on the component-covariance matrices may be viewed as too general for many situations in practice, in particular, with high dimensional data. One approach for reducing the number of parameters is to work in a lower dimensional space by adopting mixtures of factor analyzers (Ghahramani & Hinton, 1997).

81 g f x i1 i x i i i B B i T i D i i g B i p q D i

82 Testing for the number of components, g, in a mixture is an important but very difficult problem which has not been completely resolved.

83 A mixture density with g components might be empirically indistinguishable from one with either fewer than g components or more than g components. It is therefore sensible in practice to approach the question of the number of components in a mixture model in terms of an assessment of the smallest number of components in the mixture compatible with the data.

84 An obvious way of approaching the problem of testing for the smallest value of the number of components in a mixture model is to use the LRTS, -2log. Suppose we wish to test the null hypothesis, H 0 : g g 0 versus H : 1 g g 1 for some g 1 >g 0.

85 We let denote the MLE of Ψ i calculated under H i, (i=0,1). Then the evidence against H 0 will be strong if is sufficiently small, or equivalently, if -2log is sufficiently large, where Ψˆ 2log 2{log L( Ψˆ ) log L( Ψˆ 1 0 )}

86 McLachlan (1987) proposed a resampling approach to the assessment of the P-value of the LRTS in testing H : g g v H : g g 1 for a specified value of g 0.

87 The Bayesian information criterion (BIC) of Schwarz (1978) is given by L d n as the penalized log likelihood to be maximized in model selection, including the present situation for the number of components g in a mixture model.

88

89

90

91 Grouping for Leukemia Data

92

93

94

95 Breast cancer data set in van t Veer et al. (van t Veer et al., 2002, Gene Expression Profiling Predicts Clinical Outcome Of Breast Cancer, Nature 415) These data were the result of microarray experiments on three patient groups with different classes of breast cancer tumours. The overall goal was to identify a set of genes that could distinguish between the different tumour groups based upon the gene expression information for these groups.

96 The chips are down; Diagnosing breast cancer (Gene chips have shown that there are two sorts of breast cancer)

97 Nature News feature (Ball)

98 Colour-coded: this plot of gene-expression data shows breast tumours falling into two groups

99 Microarray data from 98 patients with primary breast cancers with p = 24,881 genes 44 from good prognosis group (remained metastasis free after a period of more than 5 years) 34 from poor prognosis group (developed distant metastases within 5 years) 20 with hereditary form of cancer (18 with BRAC1; 2 with BRAC2)

100 Pre-processing filter of van t Veer et al. only genes with both: P-value less than 0.01; and at least a two-fold difference in more than 5 out of the 98 tissues for the genes were retained. This reduces the data set to 4869 genes.

101 Heat Map Displaying the Reduced Set of 4,869 Genes on the 98 Breast Cancer Tumours

102 Unsupervised Classification Analysis Using EMMIX-GENE Steps used in the application of EMMIX-GENE: 1. Select the most relevant genes from this filtered set of 4,869 genes. The set of retained genes is thus reduced to 1, Cluster these 1,867 genes into forty groups. The maority of gene groups produced were reasonably cohesive and distinct. 3. Using these forty group means, cluster the tissue samples into two and three components using a mixture of factor analyzers model with q = 4 factors.

103 Insert heat map of 1867 genes Heat Map of Top 1867 Genes

104

105

106

107 i m i U i i m i U i i m i U i i m i U i where i = group number m i = number in group i U i = -2 log λ i

108 Heat Map of Genes in Group G1

109 Heat Map of Genes in Group G2

110 Heat Map of Genes in Group G3

111 1. A change in gene expression is apparent between the sporadic (first 78 tissue samples) and hereditary (last 20 tissue samples) tumours. 2. The final two tissue samples (the two BRCA2 tumours) show consistent patterns of expression. This expression is different from that exhibited by the set of BRCA1 tumours. 3. The problem of trying to distinguish between the two classes, patients who were disease-free after 5 years 1 and those with metastases within 5 years 2, is not straightforward on the basis of the gene expressions.

112 Selection of Relevant Genes We compared the genes selected by EMMIX- GENE with those genes retained in the original study by van t Veer et al. (2002). van t Veer et al. used an agglomerative hierarchical algorithm to organise the genes into dominant genes groups. Two of these groups were highlighted in their paper, with their genes corresponding to biologically significant features.

113 Cluster A Cluster B Identification of van t Veer et al. containing genes co-regulated with the ER-a gene (ESR1) containing co-regulated genes that are the molecular reflection of extensive lymphocytic infiltrate, and comprise a set of genes expressed in T and B cells Number of genes Number of matches with genes retained by select-gene We can see that of the 80 genes identified by van t Veer et al., only 47 are retained by the select-genes step of the EMMIX-GENE algorithm.

114 Comparing Clusters from Hierarchical Algorithm with those from EMMIX-GENE Algorithm Cluster A Cluster Index (EMMIX- GENE) Number of Genes Matched Pe rce ntage Matched (%) Cluster B Subsets of these 47 genes appeared inside several of the 40 groups produced by the cluster-genes step of EMMIX-GENE.

115 Genes Retained by EMMIX-GENE Appearing in Cluster A (vertical blue lines indicate the three groups of tumours)

116 Genes Reected by EMMIX-GENE Appearing in Cluster A

117 Genes Retained by EMMIX-GENE Appearing in Cluster B

118 Genes Reected by EMMIX-GENE Appearing in Cluster B

119 Assessing the Number of Tissue Groups To assess the number of components g to be used in the normal mixture the likelihood ratio statistic was adopted, and the resampling approach used to assess the P-value. By proceeding sequentially, testing the null hypothesis H 0 : g = g 0 versus the alternative hypothesis H 1 : g = g 0 + 1, starting with g 0 = 1 and continuing until a non-significant result was obtained it was concluded that g = 3 components were adequate for this data set.

120 Clustering Tissue Samples on the Basis of Gene Groups using EMMIX-GENE Tissue samples can be subdivided into two groups corresponding to 78 sporadic tumours and 20 hereditary tumours. When the two cluster assignment of EMMIX-GENE is compared to this genuine grouping, only 1 of the 20 hereditary tumour patients is misallocated, although 37 of the sporadic tumour patients are incorrectly assigned to the hereditary tumour cluster.

121 Using a mixture of factor analyzers model with q = 8 factors, we would misallocate: 7 out of the 44 members of 1 ; 24 out of the 34 members of 2 ; and 1 of the 18 BRCA1 samples. The misallocation rate of 24/34 for the second class, 2, is not surprising given both the gene expressions as summarized in the groups of genes and that we are classifying the tissues in an unsupervised manner without using the knowledge of their true classification.

122 Supervised Classification When knowledge of the groups true classification is used (van t Veer et al.), the reported error rate was approximately 50% for members of 2 when allowance was made for the selection bias in forming a classifier on the basis of an optimal subset of the genes. Further analysis of this data set in a supervised context confirms the difficulty in trying to discriminate between the disease-free class 1 and the metastases class 2. (Tibshirani and Efron, 2002, Pre-Validation and Inference in Microarrays, Statistical Applications In Genetics And Molecular Biology 1)

123

124 Investigating Underlying Signatures With Other Clinical Indicators The three clusters constructed by EMMIX- GENE were investigated in order to determine whether they followed a pattern contingent upon the clinical predictors of histological grade, angioinvasion, oestrogen receptor, lymphocytic infiltrate.

125

126 Microarrays have become promising diagnostic tools for clinical applications. However, large-scale screening approaches in general and microarray technology in particular, inescapably lead to the challenging problem of learning from high-dimensional data.

127 Hope to see you in Cairns in 2004!