A Support Vector Machine Classifier based on Recursive Feature Elimination for Microarray Data in Breast Cancer Characterization. Abstract.

Transcription

1 A Support Vector Machne Classfer based on Recursve Feature Elmnaton for Mcroarray Data n Breast Cancer Characterzaton. R.Campann, D. Dongovann, N. Lanconell, G. Palermo, A. Rccard, M. Roffll Dpartmento d Fsca,Unverstà degl Stud d Bologna campann@bo.nfn.t 1 Introducton Abstract. An effectve approach to cancer classfcaton based upon gene expresson montorng usng DNA mcroarray as ntroduced by [1]. Here they used DNA mcroarray analyss on prmary breast tumours of 78 young patents thout tumour cells n local lymph nodes at dagnoss, 34 from patents ho developed dstant metastass thn 5 years (poor prognoss group), 44 from patents ho contnued to be dsease-free after a perod of at least 5 years (good prognoss group) and appled a three step supervsed classfcaton (based on correlaton methods) to dentfy a gene expresson sgnature strongly predctve of a short nterval to dstant metastass ( poor prognoss sgnature). We use a Support Vector Machne (SVM) to face the same problem because such a method has already done ell n cancer classfcaton problems, and e thnk that e could obtan slghtly better results. In addton e also address the problem of selecton of a small subset of genes from the ntal number of genes (~5000). We use a method of gene selecton utlsng SVM methods based on Recursve Feature Elmnaton (RFE) nstead of the feature rankng th correlaton method used n [1], because the last method doesn t take nto account mutual nformaton beteen features n the feature selecton process, and ths could mpact classfcaton performance []. Breast cancer s one of the most common malgnant tumours affectng omen of the U.S. and European populatons. Heredtary breast cancer, hch accounts for less than 10 % of all cases, s due manly to germlne mutatons n the tumour suppressor genes BRCA1 and BRCA. Most cases of breast cancer occur n sporadc forms, n hch the nature of ther genetc determnants reman elusve[3]. Breast cancer patents th the same stage of dsease can have markedly dfferent treatment responses and overall outcome. The strongest predctors for metastass (for example, lymph nodes status and hstologcal grade) fal to classfy accurately breast tumours accordng to ther clncal behavour. Chemotherapy or hormonal therapy reduces the rsk of dstant metastass by approxmately one-thrd; hoever, % of patents recevng ths treatment ould have survved thout t [1]. Snce these therapes use pharmaceutcal agents, such as oestrogen modulators or cytotoxc drugs, that reach cancer cells through the bloodstream, t s not surprsng that these treatments frequently have toxc sde effects. Fnally dagnoss of cancer must be accurate n order for the patent to receve the correct treatment and so have the best chance of survval. It ould be very mportant to fnd a strategy to select patents ho ould beneft (or not) from such therapes as chemotherapy, and gene expresson profles are revealng a really poerful eapon to face ths knd of problems (compare to the tradtonal ay of cancer dentfcaton based on the locaton of tumour and ts morphologcal appearance ), especally for ther ablty to subtype dsease. All e need no s a classfcaton method that can afford to manage a such huge amount of data, because expresson datasets contan

2 measurements for thousands of genes hch proves problematc for many tradtonal methods. Also for ths reason e decded to use Support Vector Machne, a supervsed machne learnng technque, that have been shon to perform ell n evaluatng mcroarray gene expresson [4, 5, 6]. SVMs, though, are ell suted to orkng th hgh dmensonal data such as ths. In ths ork a systematc and prncpled method s ntroduced that analyses mcroarray expresson data from thousands of genes. The prmary goal s the proper classfcaton of ne samples. We do ths by tranng the SVM on samples classfed by experts, and then testng the SVM on samples t has non seen before. We also look at the method ntroduced n [1] to focus the analyss on a smaller subset of genes that appear to be the best dagnostc ndcators. Ths amounts to a knd of dmensonalty reducton on the dataset. If one can dentfy partcular genes that are dagnostc for the classfcaton one s tryng to make, then there s also hope that some of these genes may be found to be of value n further nvestgatons of the dsease and n future therapes. For ths purpose e use a method of gene selecton utlsng SVM methods based on RFE, because e are confdent that t could ork better than correlaton methods[] Materals and methods In recent years several methods have been developed for performng gene expresson experments. Measurements from these experments can gve expresson levels for genes n tssue or cell samples. Datasets used for our experments [11] consst of a relatvely small number of tssue samples (less than 100) each th expresson measurements for thousands of genes. Prevous methods used n the analyss of smlar datasets start th a procedure to extract the most relevant features. Most learnng technques do not perform ell on datasets here the number of feature s large compared to the number of examples. SVMs are beleved to be an excepton. We are able to begn th tests usng the full dataset, and systematcally reduce the number of features selectng those e beleve to be the most relevant. To understand our method a famlarty th SVM s requred, and a bref ntroducton follos..1 Support Vector Machnes Support Vector Machnes are learnng machnes used n pattern recognton and regresson estmaton problems [7]. They gro up from Statstcal Learnng Theory (SLT) [7]., hch gves some useful bounds on the generalzaton capacty of machnes for learnng tasks. The SVM algorthm constructs a separatng hypersurface n the nput space. Its ay to do that s: a) Mappng the nput space nto a hgh dmensonal features space through some non lnear mappng chosen a pror (kernel); b) Constructng n ths features space the Maxmal Margn Hyperplane Hyperplane are defned by x + b = 0 : hen tranng data ( x, y ), = 1,..., l are separated by ths hyperplane t happens that ( x 1, here y = ± 1 are the labels. It can be shon that the margn s to: y, so fndng the hyperplane hch separates data th maxmal margn s equal mnmze th y ( x 1 (1)

3 In order to allo for msclassfcaton errors, constrants are relaxed to y ( x 1, 0. (1) becomes then: mnmze + C th y ( x 1 () The dual formulaton of () reduces to : 1 maxmze α th α y = 0, 0 α α ( x, x α C ) y y (3) Ths formulaton, here example vectors are present only n dot products, makes qute smple the executon of pont a) because of a theorem by Mercer[7]. It gves an easy ay to compute dot products n feature spaces, here vectors n nput space are non-lnearly mapped by a functon φ (x). By usng a sutable functon K such that φ( x ) φ( x ) = K( x, x ) e do not need to calculate each sngular mappng φ (x). If nstead of controllng the overall tranng error one ants to control the trade-off beteen false postve and false negatve, t s possble to modfy the prmal n the follong ay: + + mnmze + C + C + th y ( x 1,( x 1+ (4) + here C and C gve dfferent costs to false-postve and false-negatve errors. We use an SVM softare mplemented by ourself. In order to verfy the relablty of our softare e tested t (before of usng t n ths specfc problem of breast cancer classfcaton) usng gene expresson data from [4]. In ths case the goal of usng SVM as to functonally classfy genes based on ther expressons (e used expresson data from 467 genes of the buddng yeast Saccharomyces Cerevsae measured n 79 dfferent DNA mcroarray hybrdsaton experments). Our results are comparable th the results obtaned n [4].. Feature Selecton Classcal gene selecton methods select the genes that ndvdually classfy best the tranng data. These methods nclude correlaton methods. Evaluatng ho ell an ndvdual feature contrbutes to the separaton (e.g. poor patents vs. good patent ) can produce a smple feature (gene) rankng. Varous correlaton coeffcents are used as rankng crtera. For nstance the coeffcent + µ used n [8] s defned µ = + here µ e σ are the mean and standard devaton σ + σ of the gene expresson values of gene for all patents of class(+) or class(-). Large postve values ndcate strong correlaton th class(+) hereas large negatve correlaton th class(-). values ndcate strong

4 What characterze feature rankng th correlaton methods s the mplct orthogonalty assumptons that are made. Each coeffcent s computed th nformaton about a sngle feature and does not take nto account mutual nformaton beteen features. Several authors have suggested usng the change n obectve functon hen one feature s removed as a rankng crteron [9]. For classfcaton problems, the deal obectve functon s the expected value of the error, that s the error rate computed on an nfnte number of examples. For the purpose of tranng ths deal obectve s replaced by a cost functon J computed on tranng examples only. Hence the dea s to compute the change n the cost functon DJ() caused by removng a gven feature or, equvalently, brngng ts eght to zero. The OBD algorthm [10] approxmates DJ() by expandng J n Taylor seres to second order. At the optmum of J, the frst order term can be neglected yeldng: 1 J DJ ( ) = ( ) ( D ). (5) The change n eght D = corresponds to removng feature. Snce SVM mnmze J =1 under certan constrants, then (5) reduces to and ths ustfes the use of as feature rankng crteron...1 Recursve Feature Selecton A good feature-rankng crteron s not necessarly a good feature subset rankng crteron. The crtera DJ() or estmate the effect of removng one feature at a tme on the obectve functon. They become very suboptmal hen t comes to removng several features at a tme, hch s necessary to obtan a small feature subset. Ths problem can be overcome by usng the follong teratve procedure that s called RFE: 1) Tran the classfer (optmse the eghts respect to J); ) Compute the feature th smallest rankng crteron; 3 Remove the feature th smallest rankng crteron. For computatonal reasons t may be more effcent to remove several features at a tme at the expense of possble classfcaton performance degradaton. It s relevant to be noted that RFE has no effect on correlaton methods snce the rankng crteron s computed th nformaton about a sngle feature; so as gene selecton e utlze an SVM method based on RFE, that should take nto account mutual nformaton beteen genes n the gene selecton process, and ths could mpact classfcaton performance. Snce a gene selecton utlsng SVM method based on RFE as already used n [] and they obtaned good results compared to classcal correlaton methods, e also decde to use such a method. 3.1 Data Set The data set e used n ths ork can be found at [11]. We used the gene expresson data from 98 prmary breast cancers: 34 from patents ho developed dstant metastass thn 5 years, 44 from patents ho contnued to be dsease-free after a perod of at least 5 years, 18 from patents th BRCA1 germlne mutatons and from BRCA. All sporadc patents ere lymph nodes negatve, and under 55 years of age at dagnoss. From each patent 5 µg total RNA as solated from snap frozen tumours materal and used to derve complementary RNA (crna). A reference crna pool as made by poolng equal amounts of crna from each of sporadc carcnomas. To hybrdsaton ere carred out for each tumour usng a fluorescent dye reversal technque on

5 mcroarray contanng approxmately 5000 genes. The 78 sporadc lymph node negatve patents ere selected specfcally to search for a prognostc sgnature n ther expresson profles [1]. 3 Results The data conssted of 78 tranng sample and 19 test samples. Each sample as a vector correspondng to ~ 5000 genes, but approxmately 5000 genes (sgnfcantly regulated n more than 3 tumours out of 78, that s, at least a tofold dfference and a p-value of less than 0.01 n more than 3 tumours) ere selected from the 5000 genes on the mcroarray. Frst e perform a smple pre-processng step: e normalze the data such that for each gene expresson value e subtract ts mean and dvde the result by ts standard devaton. Then e use the Recursve Feature Elmnaton method, as explaned n Secton..1. We elmnate chunks of genes at a tme (for nstance a ay to proceed could be: reach the number of genes hch s the closest poer of, and at subsequent teratons elmnate half of the remanng genes). We thus obtan nested subsets of genes of ncreasng nformatve densty. The qualty of these subsets of genes s then assessed by tranng an SVM (look at the next subsecton). 3.1 SVM Tranng and testng procedure Gven a number of feature, e begn by choosng a kernel and tune the tuneable parameters of SVM softare n order to acheve the best performance on a leave-one-out cross valdaton tests usng the tranng dataset (the leave-one-out procedure conssts of removng one example from the tranng set, constructng the decson functon on the bass only of the remanng tranng data and then testng on the removed example). In ths fashon one tests all examples of the tranng data and measures the fracton of errors over the total number of tranng examples). Then For the best parameters of the leave-one-out procedure, e test the SVM on the 19 test samples. Unfortunately at the moment e are not able to publsh our results because the ork s stll n progress References [1] Laura J. van t Veer, Hongyue Dal, and Marc J.van de Vver, Gene Expresson Proflng Predcts Clncal Outcome of Breast Cancer, Nature, VOL 415, 31 January 00, [] Isabelle Guyon, Jason Weston, Stephen Barnhll and Vladmr Vapnk, Gene Selecton for Cancer Classfcaton usng Support Vector Machne. [3] Vanessa Yen, Study of Genomc Alteratons n Human Breast Tumours by RDA, Boornual, VOL 3, 000. [4] Mchael Bron, Wllam Grundy, Davd Ln, Nello Crstann, Charles Sugnet, Terrence Furey, Manuel Ares and Davd Haussler, Knoledge-based analyss of mcroarray gene expresson data by usng support vector machnes, PNAS, VOL 97, no.1, January 4, 000, [5] T.furey, N.Crstann, N. Duffy, D. Bednarsk, M.Schummer and D Haussler, Support Vector Machne Classfcaton and Valdaton of Cancer Tssue Samples Usng Mcroarray Expresson Data, Bonformatcs, 000. [6] S. Mukheree, P. Tamao, D. Slonm, A. Verr, T. Golub, J.P. Mesrov and T. Poggo, Support Vector Machne Classfcaton of Mcroarray Data, A.I. Memo No.1677, C.B.C.L. Paper No.18, 1998.

6 [7] V. Vapnk, The nature of statstcal learnng theory, Sprnger Verlag, [8] T. R. Golub, D. K. Slonm, P. Tamao, C. Huard, M. Gaasenbeek, J.P. Mesrov, H. Coller, M.L. Loh, J. R. Donng, M. A. Calgur, C. D. Bloomfeld and E. S. Lander, Molecular Classfcaton of Cancer: Class Dscovery and Class Predcton by Gene Expresson montorng, Scence, VOL 86, 15 October 1999, [9] Ron Kohav and George John, Wrappers for feature subset selecton, Artfcal ntellgence ournal, VOL 97, Nos 1-, 1997, [10] Y. Le Cun, J. S. Denker and S. A. Solla, Optmum Bran Damage, n D. Touretzky Ed. Advances n Neural Informaton Processng Systems, 1990, [11] Mcroarray data are avalable at