Evaluation of Gene Selection Using Support Vector Machine Recursive Feature Elimination

Transcription

1 Evaluation of Gene Selection Using Support Vector Machine Recursive Feature Elimination Committee: Advisor: Dr. Rosemary Renaut Dr. Adrienne C. Scheck Dr. Kenneth Hoober Dr. Bradford Kirkman-Liff John Huynh

2 Contents Introduction Data Support Vector Machine Feature Selection Hypothesis & Experimental design Result Conclusion Future work Experience Reference

3 Terminology Sample is data set including Gene = feature = attribute = column Example = data point = slide = array = row r_1 r_2 r_i r_m x1 x_11 x_21 x_i1 x_m1 x2 x_12 x_22 x_i2 x_m xj x_1j x_2j x_ij x_mj xn x_1n x_2n x_in x_mn Class c_1 c_2 c_i c_m

4 Meningioma Dr. Adrienne C. Scheck s Lab, BNI (Barrow Neurological Institute) Meningioma: 20% of primary intracranial tumor Mortality/Morbidity: In one series by Coke et al, the overall survival rate for all patients at 5 and 10 years were 87% and 58%, respectively. Medial Sphenoid Wing Meningioma

5 Meningioma Correlating clinical process, microarray, NMR, and FISH with WHO classification grade I, II, and III. Tubercullum sellae meningioma

6 Anatomy Meningioma is tumor of arachnoid.

7 Histology Neuron & Purkinje cell (cerebellum) Neuroglial cells Astrocytes: nurture, support Protoplasmic astrocytes (gray matter) Fibrous astrocytes (white matter) Oligodendrocytes: myelin, support Microglia: immune system in brain Ependymal cells: epithelium Blood vessels

8 Meningioma - Histopathology Meningioma: whorl-like structure + psammoma bodies WHO grade I: benign WHO grade II: (atypical) A meningioma with increased mitotic activity or three or more of the following features: increased cellularity, small cells with high nucleus: cytoplasm ratio, prominent nucleoli, uninterrupted patternless or sheet-like growth, and foci of spontaneous or geographic necrosis. WHO grade III: (anaplastic) A meningioma exhibiting histological features of frank malignancy far in excess of the abnormalities present in atypical meningioma.

9 BNI Meningioma Data Affymetrix HG-U133 Plus 2.0 with 54,675 genes. Small data set with many genes Grade Primary Recurrence Total I II III Total

10 BNI Meningioma Data Plan A: consider data as large data set Plan B: consider data as small data set Grade Train Test I 11 4 II 5 2 Total 16 6 Total

11 BNI Meningioma Data High quality

12 Microarray Gene expression- Microarray Pattern of gene expressions for each tissue Oligo-microarray vs cdna High density Fixed probe length (25) In-situ synthesis

13 Microarray Microarray explores gene expression in global scale. PM & MM

14 Lymphoma Data Amersham cdna microarray with 7129 genes Tissue = bone marrow, blood ALL: acute lymphocytic leukemia AML: acute myelogenous leukemia Incidence: peak 2-3 yrs old: 80/1,000,000; 2400 new/yr/usa, 31% of all cancers ALL AML Total Train Test Total

15 Lymphoma Data Good quality Large sample size, smaller feature dimension

16 Inducer Problem The purpose of learning machine is to find the most accurate classifier by learning in the training set and testing in the testing set. It is the minimizing problem of the error function E in mathematics. Let call f is learning algorithm, data points X = {x1, x2,, xi,, xm} in Rn, target {y1, y2,, yi,, ym} in Y = {-1, +1} f: X Y xi f(xi) E = (yi - f(xi))2. Testing set requirement: the testing set must be never seen in the training process; otherwise the correctness of the testing phase is unexpected high.

17 Support Vector Machine Map data into the feature space Learn in the feature space Return the result to the output space Learning function f (xi) = xi w + b f(xi) > 0 for yi = +1, f(xi) < 0 for yi = -1 f(xi) = 0 for decision boundary Output space Input space Feature space

18 SVM Characteristics Maximum margin Low computer cost: Kernel function costs O(n). Training cost: the worst case costs O(nsv3 + nsv2m + nsvmn); the best case costs O(nm2). Testing cost: O(n).

19 Linear SVM - Separable Case No kernel = scalar dot product Margin = 2/ w minimizing w2 Constraints (xi w+b)yi >0

20 Linear SVM - Non-Separable Case Introduce slacks ξi to adjust the choosing of support vector when needed. This means adding a constraint C on the Lagrangean multipliers C = 100 in our experiment.

21 Non-Linear SVM There is no linear decision boundary in the input space

22 Non-Linear Support Vector Machine Introduce kernel function to map data into Euclidean high dimensional space: dot product.

23 Non-Linear Support Vector Machine Now the data and weight are in the hyperspace. Training and testing processes are in the high dimensional space

24 Problem of Microarray Data Instance space F1x F2 x x Fi x x Fn The training set must be a large enough subset of instance space. Over-fitting problem of small data set: the inducer performs well in training set, but acts poorly in test set. The computational cost of high dimensional data is so high (n = 54675). Multiple testing correction: FDR, SAM, Classical analysis methods are not suitable.

25 Feature Selection Benefits of feature selection are reducing Computer cost Over-fitting Feature selection actually is a search algorithm in the feature space to find the optimal feature subset. Given an inducer I, and a data set D with features X1,, Xi,, Xn from a distribution D over the labeled instance space, an optimal feature subset, Xopt, is a subset of the features such that the accuracy of the induced classifier C = I(D) is maximal (Kohavi97).

26 Feature Selection: How? Filter method vs wrapper method Feature ranking criteria Correlation coefficient Weight

27 Recursive Feature Elimination RFE is a top-down (backward) wrapper using weight as feature ranking criterion. Eliminate One feature in every loop: slow A subset in every loop: fast Are they the same optimal subsets? Is the feature ranking criteria are the same?

28 Feature Selection Meaning Create nested subsets Let define Rate of elimination Surviving subset Note that the feature selection module includes an inducer so the training set must be never seen in both Feature selection module Evaluation module (Kohavi97)

29 Full Two Factorial Experiment Design The evaluation cost is the accuracy. The evaluation methods: independent test and cross-validation. The inducer is SVM for both feature selection and evaluation (Guyon02). The factor A (row) is the rate of elimination. The factor B (column) is the surviving subset

30 Software Design Preprocessing data: linear normalization + log2 transformation (prep.java) SVM, feature selection and evaluation: Matlab 6.5R13

31 Result: Lymphoma The optimal subset is 32 genes.

32 Result: Lymphoma Box Plots Box Plots

33 Result: Lymphoma ANOVA Tsuc Vsuc

34 Result: Meningioma The optimal subset is 4.

35 Result: Meningioma Box Plots Small Plan: 4 Large Plan: 2 Large Small

36 Result: Meningioma ANOVA Correct choice is 4. Index Probe _at _s_at _a_at 21552_x_at

37 Conclusion No interaction between the rate of elimination and the feature optimal subset Small data set: rely on cross-validation

38 Future Works More published data set: large + small, difficult + easy How small is small? Evaluation method for small data set: master gene lists + LOOCV Over-fitting and cross-validation

39 Experience Not all the data mining task will be success. Business focus: communication, learning, negotiation, team work, leadership, Understand and live with data: a high dimensional small data set Never alternate the data in preprocessing process (time cost) Experimental design: good planning Observation + Think + Reaction = Strategy Loop, deal with the fact, not with who. Repeatable: Document experiment results and analysis Welcome new idea: good + bad; read, read, and read Never seen rule of test data, evaluation algorithm, over-fitting Feature selection SVM Software

40 References (Blum) Avrim L. Blum and Pat Langley, Selection of Relevant Features and Examples in Machine Learning, (Burges98) Christopher J.C. Burges, A Turtorial on Support Vector Machines for Pattern Recognition, (1998), Web-print: (Golub99) Golub et al, Molecular Classication of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring, Science 286 (1999), 531-7, et al 1999.pdf. (Guyon02) Isabelle Guyon et al., Gene Selection for Cancer Classication using Support Vector Machines, Machine Learning 46 (2002), no. 1-3, , Web-print: (Gunn98) Steve R. Gunn, Support Vector Machines for Classification and Regression, (1998), (Kohavi97) Ron Kohavi and George H. John, Wrappers for Feature Subset Selection, Artifcial Intelligence 97 (97), (Soroin03) Soroin Dr aghici, Data Analysis Tools for DNA Microarrays, Chapman and Hall/CRC, WHO Classification

41 Question?