JOINT SUB-CLASSIFIERS ONE CLASS CLASSIFICATION MODEL FOR AVIAN INFLUENZA OUTBREAK DETECTION

Transcription

1 JOINT SUB-CLASSIFIERS ONE CLASS CLASSIFICATION MODEL FOR AVIAN INFLUENZA OUTBREAK DETECTION Je Zhang, Je Lu, Guangquan Zhang Centre for Quantum Computaton & Intellgent Systems Faculty of Engneerng and Informaton Technology, Unversty of Technology, Sydney PO Box 123, Broadway, NSW 2007, Australa {je.zhang, je.lu, H5N1 avan nfluenza outbreak detecton s a sgnfcant ssue for early warnng of epdemcs. Ths paper proposes doman knowledge-based jont one class classfcaton model for avan nfluenza outbreak. Instead of focusng on manpulatons of the one class classfcaton models, we delve nto the one class avan nfluenza data-set, dvde t nto sub-classes by doman knowledge, tran the subclass classfers and unfy the result of each classfer. The proposed jont method solves the one class classfcaton and feature selecton problems together. The experment results demonstrate that the proposed jont model defntely outperforms the normal one class classfcaton model on the anmal avan nfluenza data-set. Keywords: One class classfcaton; avan nfluenza; outbreak detecton; jont model 1. Introducton H5N1 avan nfluenza outbreak detecton s a sgnfcant task wth a bg challenge, because there are a number of uncertan factors assocated wth the outbreaks 1. The Asan lneage, hghly pathogenc avan nfluenza (HPAI) vrus sub-type H5N1 was frst dentfed n Hong Kong n , and worldwde outbreaks ncreased dramatcally snce Now t has spread to more than 60 countres from Asan, to Europe and Afrca 3. Ths epdemc has nfected poultry and caused the cullng of mllons of brds resultng n a loss of bllons n the poultry trade, and sgnfcant loss of human lfe. Accordng to WHO 4, 502 humans have been nfected and among them 302 of them have ded. Ths vrus can mutate accordng to ts host and adapt to dfferent envronments 5, 6. Normally, the vrus transmts from brds to mammals, but so far, t seems t cannot be transmtted between mammals 5. Water brds are beleved to be the vral reservor of nfluenza A vruses 7 but the transmttng mechansm s stll unclear. Wld brds and human actvtes have broadened the channels for the vrus to spread; for example, poultry farm, brd trade and wld brds mgraton 5, 8, 9 are all possble channels to spread the vrus. In south Asa, free-grazng duck are also beleved to be a major reason for vrus transmsson 10. Scentsts are stll strugglng to dscover the reasons for transmsson. However, f we take outbreak events as our target observaton and the normal status wthout outbreaks as the outlers, then detecton turns nto a typcal one class classfcaton () ssue. The tem one class classfer was frst proposed by Moya 11 and one class classfcaton () has been studed n the area of the novelty detecton 12, outler detecton 13 n sgnal processng and pattern recognton applcatons. At the outset, the focus of s to dentfy noveltes and get rd of them before processng 1

2 2 Je Zhang, Je Lu and Guangquan Zhang normal sgnals. Ths focus subtly changes to analyzng the target class n the research area of data mnng 14, and method has been developed and greatly advanced by Tax and Dun 15 by ther dea of one class data descrpton. n data mnng depcts the only labeled target class by a sutable model and detects the new case f t s n the boundary of the target class or f t s out of the boundary as an outler. The stuaton of s very common n real world. These one label tasks can often be encountered n the real world, for example, nuclear plant falure, a medcal dsease case, and dentfyng a type of web pages 16, 17. has been wdely used n many areas, such as cyber-ntruson detecton, medcal dagnoss, mage processng, fraud detecton n fnancal ndustry 17, and defect detecton n the fabrc ndustry 18. Other applcatons nclude land cover classfcaton 19, envronmental montorng 20, document retreval and classfcaton 21, 22, vague stream data analyss 23 and the most promsng applcaton doman, whch we wll dscuss next, s the medcal and bologcal area. methodology s especally sutable for medcal and bologcal doman applcatons. Dun has mentons that has been used n dsease detecton 24. In many medcal dagnoses, the doctor only keeps the dsease case data whch can be used as the labeled target class, and all other dseases and healthy cases are taken as the outlers. In the area of epdemc dsease for nstance, avan nfluenza, anmal cases are only reported f there are epdemc outbreaks n a poultry farm or a number of dead wld brds are dentfed. Ths s smlar n applcatons appled n gene scence where only target gene samples are avalable. The stuaton n these areas s approxmately the same and researchers only focus on lmted target samples, whle outlers have a large populaton, such as healthy populatons versus target dsease subjects, or RNA of all other anmals versus the target RNA gene pattern. Though methods have many applcatons, there are lmtatons due to the nature of only one labeled class. Many researchers choose two-class or mult-class data-sets n ther experments 25, 26, 29, 30 to analyze the results. Other researchers artfcally generate outlers for evaluaton 27, 31. In real world examples, we can only obtan the target class 28 such as avan nfluenza outbreaks. The outbreaks are a typcal ssue wth only one label, whlst other ssues need to be addressed n depctng avan nfluenza anmal outbreaks: 1) All the features leadng to the outbreak are unclear because the H5N1 vrus has mutated to adapt to the envronment and the vrus dstrbuton channels are complcated 5. 2) The outbreak happens ncdentally. Ths makes t hard to tell exactly the dfferences between the outbreak and the non-outbreak cases. The above two reasons make outbreak detecton very dffcult. Frst, the feature space s uncertan and hard to evaluate. The factors causng an outbreak are not clearly dentfed and refnng of factors s dffcult because we have only the target labels. Secondly, the possbltes of outbreaks decrease the necessty and the effectveness of generatng artfcal outlers. Under ths crcumstance, we proposed an ensemble classfers approach, jont subclassfer one class classfcaton (JSC-) method, to overcome these dffcultes. The

3 Instructons for Typng Manuscrpts (Paper s Ttle) 3 ensemble classfers have long been studed 32 and have been verfed to mprove the performance of the classfcaton 33. Contemporary research for ensemble classfer s represented by baggng 34 and boostng 35 methods, and the extenson of the two methods. These methods make full use of the orgnal dataset by a dfferent method of samplng to make the classfcaton results more stable and accurate. However, ths research has some lmtatons: Frstly, these methods have lmtatons of mprovng the ndvdual base classfers by the learnng doman knowledge 36. In a real world dataset, the doman knowledge wll sgnfcantly affect the classfyng result, whch s why we apply doman knowledge nto the classfcaton; secondly, many applcatons only focus on mult-class classfcaton. There are many research applcatons on combnaton of classfers 37, 38, but these applcatons mostly address mult-class classfcatons. In the real world, there are many one class classfcaton problems whch focus on the target class dentfcaton and are dfferent to mult-class classfcaton problems, whch treat classes equally. Instead of seekng assstance from the second class n ths research, we delve nto the target cases by classfyng the outbreak cases nto sub-classes, tranng the classfers separately and ntegratng the separated models. We apply a supervsed feature selecton method to the sub-class and acheve better results wth only half of the features selected. The paper s organzed as follows. Secton 2 descrbes the methods, rated research work and revews the lmtatons of the prevous methods. Secton 3 presents the motvaton and the contents of the JSC- method. Secton 4 llustrates the method on the avan nfluenza anmal outbreak data-set and the results show that the JSC- method out-performs the normally appled method. Secton 5 concludes the paper. 2. METHODS AND RELATED WORK There are many methods whch have been appled n many real world applcatons. We descrbe the concept of the method and the related research results methods The basc dea of s descrbed by two essental elements. The frst s the dstance, or the possblty of a new case for the target class. The second element s the threshold on ths dstance or possblty. Whether or not the new case belongs to the target class can be defned by the dstance, less a threshold: f y ) = I( d( y) < θ ) (1) ( d or the possblty s larger than the possblty threshold: f y ) = I( p( y) > θ ) (2) ( p where I(.) s the ndctor functon, y s the new case, d (y) and p(y) s the dstance and the possblty on the target class. θ d and θ s the dstance and possblty threshold. p

4 4 Je Zhang, Je Lu and Guangquan Zhang Dfferent method have dfferent defntons of d (y) or p(y) and the evaluaton of the method can compare the hypercube or the hypersphere volumes depcted by d (y) or p (y). The threshold value s a trade off between whether a new comng case s a target class or an outler. methods can be dvded nto three man categores: the densty method, the boundary method and the reconstructon method 39. The densty method apples statstcal dstrbuton to depct the target class probablty densty, and uses a threshold to dstngush targets from outlers. The densty method examples are the Gaussan data descrpton (GaussanDD) model, mxture Gaussan data descrpton (MOGDD) and Parzen data descrpton (ParzenDD) model. The boundary method draws a boundary ncludng the targets wth the mnmum volume, such as support vector data descrpton (SVDD) model and k-nearest neghbor data descrpton (KNNDD) model. The reconstructon method reconstructs prevous space by the prototype model or data compress model and dentfy targets and outler after reconstructon, e.g. k-means data descrpton (KmeansDD) model, prncpal component analyss data descrpton (PCADD) model, or the self-organzng map data descrpton (SOMDD) model. In a real world applcaton, dfferent models wll have dfferent performances accordng to the mplementaton 39. The mnmum spannng tree data descrpton (MSTDD) method s a new promsng method whch out-performs many other prevous one class data descrpton methods by defnng the target boundary of the mnmum volume around the spannng tree 24. GaussanDD s a basc densty method, whch apply Gaussan dstrbuton to descrbe the one class case accordng to the Central Lmt Theorem. The possblty of d- dmensonal targets x s gven by: 1 p( x, u, Σ) = d / 2 (2π ) Σ 1/ 2 1 exp{ ( x u) 2 T 1 Σ ( x u)} (3) where u s the mean value and Σ s the co-varance matrx. The target class should be a strct unmodal and convex densty dstrbuton. The man calculaton cost s the nverson of the matrx Σ. The other densty models are the extenson of the GaussanDD model. SVDD model s a typcal boundary method whch s dfferent from one class support vector machne (SVM). One class SVM turns an method nto a two class classfcaton method by defnng the orgn as a second class 17. But the SVDD model draws a mnmum volume hypersphere to contan most of, or all of, the targets. SVDD s an optmzaton problem wth the object as: N 2 mn. f ( R, a, ξ ) = R + C ξ, (4) = 1

5 Instructons for Typng Manuscrpts (Paper s Ttle) 5 s. t. ( x T 2 a) ( x a) R + ξ = 1,..., N, ξ > 0 (5) where a s the center and R s the radus of the hypersphere, ξ s the slack varable and C s the varable to descrbe the trade off between the sphere volume and the number of the target objects rejected. After applyng Lagrange multplers to the problem we wll have the dual problem: L = N N α x. x ) = 1, j= 1 ( α α ( x. x ) j j (6) N N, 1 = where 0 α, = 1,..., N, α = 1 α x = a C = 1 We can predct f a new case s accepted or not by: y a = ( y 2 = ( y. y) 2 T = ( y a) ( y a) α x ) ( y α x ) (7) T α ( y. x ) +, j α α ( x. x ) R j j 2 If we substtute all the nner products x. ) wth a kernel functon: ( x j k( x, x ) = φ j ( x ). φ( x j ) (8) Then the problem can be mapped nto an nner product space. K_meansDD method s a very smple reconstructon method. It apples prototype vectors μ to mnmze the error: k ε = (mn x μ ) (9) Where μ s the vector of k-means center, k x s the target case vector. Dfferent reconstructon methods have dfferent error measurng methods. More detals of SVDD and other methods are descrbed by Tax 39. k k 2 Many new methods and mprovements have been dscovered. These methods can be manly categorzed as localzaton and combnatons. Localzaton tunes the parameters accordng to the local propertes whch dffer from the global ones. The localzaton method has been appled to KNNDD 17 and Gupta 29 ; t combnes local and global searches to mprove the one-class nformaton ball (IC-IB) method. The combnaton method combnes dfferent models to obtan better results and has been

6 6 Je Zhang, Je Lu and Guangquan Zhang explored n modelng. Combnng densty estmator wth two class probablty estmator has been proposed 30. The normal ensemble methods are appled n gene scence 26 and the combnaton of the model wth dfferent data-sets has also been studed 27. Statstcal learnng and case-based reasonng combnaton methods have also been proposed to ntegrate the smlarty measure and Bayesan statstcal nformaton to dentfy noveltes 40. Only a few research methods deal wth the genune one class ssue 30, where t s mpossble or mprobable to obtan an outler. Ths stuaton wll make many proposed methods unsutable, ncludng nformaton from the outlers Feature selecton ssue n Feature selecton chooses hgh varance features and removes low varance ones, but target class label provdes no nformaton at all 31. Ths means only unsupervsed feature selecton methods can be appled under ths crcumstance. Vllalba 41 evaluated four feature selecton algorthms and concluded that the Q-α algorthm and localty preservng projectons (LPP) have better performance. Generally, unsupervsed methods cannot compare wth supervsed ones. Most unsupervsed dmenson reducton methods just compress the orgnal feature space nto a smaller dmensons, whch hardly explans the meanng of each dmenson, for example LPP and PCA method 42.The most promsng supervsed feature selecton method s mutual nformaton feature selecton, for nstance the mnmum-redundancy maxmum-relevancy (mrmr) method 43, whch makes full use of mutual nformaton between the features and class labels to select the feature group wth most varaton. The above revew ndcates that we can nvestgate lmtatons of current methods n dealng wth real world examples. Avan nfluenza outbreak event detecton s ths type of ssue. Frstly, a genune problem can only obtan the target label, whch means that we cannot obtan all the other class data wthout outbreak events. If we can provde some defnte outlers for the models, the classfer can tune 31 and the result wll be sgnfcantly mproved. Ths means that many technques whch tune on the second class samples are not sutable. Secondly, the supervsed features selecton should not be appled because the outbreak factors are unclear and we need to select features. Therefore, we can only apply the unsupervsed feature selecton method. We next present our own approach for dealng wth ths real world problem. 3. Jont Sub-Classfer Method We descrbe the motvaton and the detaled processes of our JSC- method step-bystep The motvaton of proposed method The method s desgned for resolvng the classfcaton problem when the tranng data-set only has one label. If there s only one class labeled as target, e.g. the avan nfluenza outbreak events, we can mprove the method wth the followng method.

7 Instructons for Typng Manuscrpts (Paper s Ttle) 7 We dvde the outbreak events nto sub-categores to dscover whether the new sub-class of the outbreak events can help mprove the models to detect f a new comng case s lkely to be an outbreak or not. On one hand, f the sub-classes can be grouped closely together and at same tme apparently appear separate to each other, then we can apply the method for each sub-class, then combne every sub-classfer to obtan a better result. On the other hand, f the sub-groups cannot be separated from each other, then we should select the most sutable features to help classfy the sub-classes. The basc dea s shown very clearly n Fg. 1 by a Two-D SVDD method. We provde a well separated three sub-class example n Fg. 1 (a). The three black lne crcles are the SVDD boundares for the sub-classes. The magenta dashed lne s the SVDD boundary for the labeled one class. In ths context, the jont three crcle boundares are better than the one bg crcle because they have less volume. Therefore, f we can fnd a well separated sub-class and the features whch can help separate the sub-class, then we can mprove the detecton effectvely. In Fg. 1(b), we cannot observe that the combnaton of sub-class crcles mproved the precson of the method, or the combnaton cannot reduce the volume of the boundary. Ths means that the three sub-groups on ths feature space cannot mprove the classfcaton effectveness. wth three subsets wth three subsets Feature Feature 1 Feature Feature 1 a) three well separated sub-classes and features b) three poorly separated sub-classes and features Fg 1. One class SVDD and three separated sub-classes SVDD Therefore, we must address two ssues: frst, we delve nto one class and dvde t nto well separated sub-classes; second, we select good features to mprove the separated effect. In the medcal doman, epdemc doman and bologcal doman, the frst task s naturally completed by the doman expert. In the medcal doman, the doctor wll separate samples of an llness nto slght, and severely ll groups, male and female groups, or young and old categores. In the epdemc dsease doman such as avan nfluenza, the sub-class can be wld brd affected events and poultry farm events. Ths means the frst task s resolved by doman knowledge. The second task s also easy to accomplsh because we can select the features accordng to the sub-class labels. The

8 8 Je Zhang, Je Lu and Guangquan Zhang prevous unsupervsed feature selecton task becomes a supervsed feature selecton and we can easly resolve the two problems at the same tme Processes of the JSC- method The combnng method concludes the followng 5 steps: Step 1. Dvde all the one class cases nto sub-classes by doman knowledge; Even though we have only one class label, we can dvde the one class nto sub-classes by applyng doman knowledge. The effectve sub-class needs to be carefully consdered. In avan flu epdemc doman, there are many ways to group the outbreak events such as the outbreak seasons, the locatons and so on. Nevertheless, f we consder the transmsson of the vrus and the moblty of the populaton, we can dvde the affected populaton upon dfferent sub-groups as wld speces, backyard free range poultry and farm poultry. The wld brds have the maxmum moblty, the backyard brds have lmted moblty and the farm poultry has the less moblty. We don t need to dvde the farm poultry as broler chckens, layer chckens, or turkey and so on, because we consder that the transmsson ways to the farm maybe smlar, e.g. manly by human actvtes. The dvdng task s fully depended on the doman knowledge and the research objectve. If we have enough detals of the doman knowledge, then the sub-classes wll contan the more varaton nformaton. If the objectve s dfferent, the classfcaton standard wll be dfferent. For example, f we need to undertake research nto the vaccne effect of poultry, we must dvde farm or free-range poultry nto vaccnated and unvaccnated groups. If we only consder the transmsson methods of the vrus, we only need to dvde the brds nto the prevous mentoned three sub-groups. Step 2: Select the most varaton features accordng to the sub-classes; Here, we select features whch help enforce the classfcaton effect. In medcal and epdemcal areas, t s normal that there wll be many factors assocated wth the dsease case and s mportant to group the factors and select the sutable ones. In the normal method, we apply only the unsupervsed feature selecton method, but here we can select features by ether unsupervsed or supervsed methods because we have sub-class labels whch allow selecton of features based on the sub-class labels. Ths provdes a lot more choce than prevous models, for example, we can apply mrmr method to select the features. Step 3: Tran the classfer on each sub-class; We apply each sub-class cases as tranng data-set to tran sub- models. We can ether use the same classfer on dfferent sub-class data, or we can use dfferent classfers on dfferent sub-class data. We then obtan several sub one class classfers.

9 Instructons for Typng Manuscrpts (Paper s Ttle) 9 Step 4: Combne three sub-classfers to unon nto a jont model. We combne the sub-classfers results to obtan a jont result. We choose to jon the results of the sub-classfers by logcal or operator. Suppose we have n classfers, for each new comng case x, the fnal result can be obtan by: n = 1 I( x) = C ( x) (10) C s the th classfer, C (x) s true f x s detected as a target class and I s the output of jonng the output results. If one of the classfers classfes the nput case as target, the fnal output s the target class. The whole process s shown n Fg. 2

10 10 Je Zhang, Je Lu and Guangquan Zhang Dataset Subclass knowledge Preprocessng Dvdng nto subclasses Feature selecton on subclasses Sub-dataset 1 Sub-dataset 2... Sub-dataset n Classfer 1 Classfer 2... Classfer n Classfer combnng Jont model Testng data Fnal result Fg 2 The JSC- method processes 4. Experment and Result Analyss The experment s conducted on the avan nfluenza data-set collected from the nternet. Some features are transformed by GIS software before nput to the model. We apply Matlab platform and DD_Tools 44 to perform the experments Data source One secton of avan nfluenza anmal outbreak data was obtaned from reports on the webste: the other

11 Instructons for Typng Manuscrpts (Paper s Ttle) 11 data was obtaned from Nature News reporter Dr Declan Butler ( Each record contans the outbreak tme, outbreak locaton, nfected populaton, locaton type, and so on. The data-set has records dated from 2003 to If the outbreak locaton s a farm, the dead brd numbers or nfected numbers on the farm do not affect other farms. For processng purposes, we count ths farm event as only one event. Dead wld brd events are counted as one event even f only one dead brd s dentfed. The other feature data s collected from the nternet, such as poultry densty, poultry and wld brd trade, populaton densty, from ; other geographc data s collected from etc. These data wll frst be pre-processed by GIS software, and then processed to obtan the feature nformaton of an outbreak event by Matlab. The basc locaton data from OIE reports has errors, for example the wrong nformaton of longtude and lattude, whch ndcates a locaton n the sea and we cannot obtan correct nformaton from them, so record ths as mssng data. After removng the mssng data, we have 5,600 cases. Each case has 24 features plus the affected brd type nformaton. All these features are shown n Table 1. Table 1 All the selected features No Features Descrptons 1 'lattude' Lattude 2 'logtude' Longtude 3 'clm_tmp' Temperature 4 'clm_wet' Wet days 5 'clm_tmx' Daly Maxmum Temperature 6 'clm_tmn' Daly Mnmum Temperature 7 'clm_frs' Ground frost frequency 8 'clm_vap' Water vapour 9 'clm_dtr' Durnal temperature range 10 'clm_cld' Cloud cover 11 'pop_den' Populaton densty 12 'pty_den' Poultry densty 13 'ral_den' Rals lne densty 14 'mral_den' Man rals lne densty 15 'road_den' Road lne densty 16 'water_lne' Rver lne densty 17 'hydro_den' Man hydro pont densty 18 'pmeat_trd' Poultry meat mport volume 19 'brd_trd' Lve brd mport volume 20 'elevaton' Elevaton 21 'bat_cover' BAT Land cover type 22 'month' Month

12 12 Je Zhang, Je Lu and Guangquan Zhang 23 'year' Year Near how many mgraton 24 'mgr_rsk' routes 25 'brd_type' Affected brd populaton type These 25 features are possble features assocated wth the avan nfluenza outbreak. We select only 25, classfed as clmate factors, geographc factors, poultry densty factor, transportaton factors, water factor, brd trade and wld brd mgratory nformaton. For example, the brd mgratory rsk s defned by the followng processes: there are eght man brd mgratory routes around the world the mgr_rsk features are calculated by how near the outbreak locaton s to how many the mgraton routes. If the mnmum dstance to a brd mgratory route s less than 300 km, we count the mgr_rsk attrbute once. The transportaton level s measured by the lne densty per km 2 of ralroad length and road length. The ndcator of poultry meat trade s calculated from the quanttes of tons of mported poultry meat dvded by the country total area, and the same s done wth the lve brd trade Experment result and analyss Here, we follow the steps of the proposed JSC_ method and conduct the experment on an avan nfluenza data-set. Step 1. Dvde all the one class cases nto sub-classes by doman knowledge; In avan flu outbreak events, the affected brds can be dvded nto three sub-classes as wld brds, backyard brds and farm poultry. We dvde nto these three sub-class takng nto consderaton the moblty of the populatons. Wld brds have the greatest moblty, farm poultry has the lowest moblty, and backyard brds have lmted moblty. The movements of brds should be connected wth the transmsson of the vruses. The brd_type s chosen as the sub-class label and the remanng 24 wll be appled by the method. The whole data-set can be dvded nto 1,086 wld brd affected cases, 1,169 backyard poultry affected cases, and 3,345 farm poultry brd affected cases. Step 2: Select the most varaton features accordng to the sub-classes; Ths step apples the mrmr feature selecton method and we only choose 12 features, or half, of the total features to llustrate our approach. The 12 selected features are 'elevaton', 'ral_den', 'brd_trd', 'pmeat_trd', 'clm_wet', 'road_den', 'mral_den', 'year', 'clm_tmp', 'mgr_rsk', 'logtude', 'month'. Step 3: Tran the classfer on each sub-class; The tranng data-set randomly selects 80 percent of each sub-class case. We also merge the three tranng data-sets n to a fourth data-set as a comparson. We then have four traned models. The models appled are GaussanDD model, MOGDD model,

13 Instructons for Typng Manuscrpts (Paper s Ttle) 13 ParzenDD model, SVDD model, 1NNDD model, KNNDD model, KmeansDD model, PCADD model and SOMDD model. We choose the rejected threshold as 0.1 and apply the default parameters of DD_tools. Step 4: Combne three sub-classfers to unon nto a jont model. We combne the traned three sub-classfers by the results as (1). Fnally, the test data s the same as the 20 percent of remanng data. The results of the experments are shown n Table 2 and Table 3. Table 2 s the experment wth all the features and Table 3 llustrates the result wth only 12 selected features. Correct classfed rate of model on same test data Table 2 Experment wth all features Wld brd (Tdata1) Sub-1 correct rate tranng data Backyard poultry (Tdata2) Sub-2 correct rate Farm poultry (Tdata3 ) Sub- 3 correct rate Tdata4 =(Tdata 1+Tdat a2 +Tdata 3) collect rate JSC- collect rate Gaussan Parzen Kmeans (k=5) knn som nn mog(5) Svdd (σ=100) pca mst

14 14 Je Zhang, Je Lu and Guangquan Zhang Correct classfed rate of model on same test data Table 3 Experments results wth only 12 features Wld brd (Tdata1) Sub-1 correct rate Tranng data Backyard poultry (Tdata2) Sub-2 correct rate Farm poultry (Tdata3 ) Sub- 3 correct rate Tdata4 =(Tdata 1+Tdat a2 +Tdata 3) collect rate JSC- collect rate Gaussan Parzen Kmeans (k=5) knn som nn mog(5) Svdd (σ=100) pca mst The evaluaton standard s obvous for comparng the correct classfcaton rate to the target class, the true postve (TP) rate. In Table 2 and Table 3 the correct classfcaton rates are rates on the whole testng data. We observe that the JSC- method can outperform the normally appled models and we conclude from the two tables that, whether we apply feature selecton frst or not, the JSC- model wll outperform the normal model on the outbreak data-set. Though the sub- models on the testng data have a low TP rate, the JSC- model performances mprove sgnfcantly, as we expected. We also compare Table 3 to Table 2 wth the TP rate of applyng all the features, and only half of the features. From the data we observe that fve out of ten combned models wth the selected 12 features n the experments outperform the combned models wth all 24 features. These fve combned JSC- models are the GaussanDD, ParzenDD, K- meansdd, MogDD and PCADD models. But the results of JSC- wth only the 12 features data-set have more accuracy than the results of the normal method on the 24 data-set. Fve models gan better performances wth selected features and ths means the feature selectons on the sub-class wll not decrease the FP rate of JSC- models. We haven t tuned the parameters of the SVDD model, so t has the lower accuracy. The 1NNDD sub-classfers have very hgh accurate rate whch means they are not senstve

15 Instructons for Typng Manuscrpts (Paper s Ttle) 15 wth the groups dvded. But the JSC- result based on 1NNDD stll has better performance. We also notce that MSTDD performed poorly, whch s sad to outperform many other models. We explan ths by pontng out that MSTDD s a more strct method wth the mnmum volume boundary, whch means t can only perform best when the outlers are dentfed clearly. In the real world one class problem, you really cannot provde a clear descrpton of the outler. So the strct MSTDD wll perform poorly compared wth other models. 5. Dscusson The JSC- method wll mprove the accuracy of the classfcaton provded we have the doman knowledge to dvde the whole populaton nto sub-groups. Sometmes t s very dffcult, or even mpossble, to obtan the knowledge to dvde the sub-classes. Under these crcumstances, we can stll classfy the data-set by clusterng. Whether or not the unsupervsed sub-classes mprove the accuracy of models s an nterestng problem for study. Clusterng method s the most common unsupervsed classfcaton method, and we classfy the outbreak cases nto three sub-classes by k-means and the spectral clusterng method separately, and conduct the JSC- method on these two sub-classes. We do the experments both on the 'all features' and selected' 12 features' data-sets and the results are lsted n Table 4 and Table 5. Correc t classf ed rate Table 4 K-means: Three sub-groups wth all features: 24 feature s JSC- mprove d or not 12 feature s JSC- Impoved or not Gauss an TRUE TRUE Parzen TRUE FALSE Kmean s (k=5) TRUE TRUE knn TRUE TRUE som TRUE FALSE nn TRUE TRUE mog(5) FALSE FALSE Svdd (σ=100 ) FALSE TRUE pca TRUE TRUE mst TRUE TRUE

16 16 Je Zhang, Je Lu and Guangquan Zhang Correc t classf ed rate Table 5 Spectral clusterng: Three sub-groups: 24 feature s JSC- mprove d or not 12 feature s JSC- Impove d or not Gauss an TRUE TRUE Parzen FALSE TRUE Kmean s (k=5) TRUE TRUE knn FALSE TRUE som FALSE TRUE nn TRUE TRUE mog(5) FALSE FALSE Svdd (σ=100 ) TRUE FALSE pca TRUE TRUE mst TRUE TRUE Table 4 lsts the comparsons between results of JSC- and on the K-means three sub-groups and Table 5 lsts the comparsons between results of JSC- and on the three spectral clusters. The experments have been conducted wth ten models on both 'all features' and '12 features' data-sets. From Table 4, we observe that eght models employng the JSC- method perform well on the 24 features data-set and seven models employng the JSC- method perform better on the 12 features data-set. From Table 5 we also observe smlar results wth sx out of ten JSC- models performng better on 24 features data-set and eght out of 10 JSC- models performng better on 12 features data-set. We cannot reach a concluson that unsupervsed sub-classes mprove the accuracy of the JSC- method because not all JSC- methods provde better results than the normal method. However, the above experments are conducted on the three sub-groups data-set. In fact, we can have dfferent number of sub-groups, so we conduct the experments on two to twenty sub-groups on the 24 features data-set to compare the performances between JSC- method and normal method. Ths tme we only apply sx models as GaussanDD, MoGDD, ParzenDD, SOMDD, K-meansDD and KNNDD models. We stll appled K-means and spectral clusterng methods to cluster the data-sets. The results are shown n Fgure 3 and Fgure 4.

17 Instructons for Typng Manuscrpts (Paper s Ttle) K-means cluster JSC- ture postve rate 1.1 K-means cluster JSC- TP mproved rato TP rate Gaussan Mog(5) Parzen SOM K-means(k=5) KNN Cluster number TP mproved rato Gaussan Mog(5) Parzen SOM K-means(k=5) KNN Cluster number Fgure 3 (a) JSC- FP rate on dfferent K-means sub-groups. (b) JSC- FP rate mproved rato on dfferent K-means sub-groups TP rate Spectral Cluster JSC- ture postve rate Gaussan Mog(5) Parzen SOM K-means(k=5) KNN TP mproved rato Spectral Cluster JSC- TP mproved rato Gaussan Mog(5) Parzen SOM K-means(k=5) KNN Cluster number Cluster number Fgure 4 (a) JSC- FP rate on dfferent spectral clusterng sub-groups. (b) JSC- FP rate mproved rato on dfferent spectral clusterng sub-groups The TP rates and TP rate mproved rato of JSC- on dfferent number of K-means sub-groups are shown n Fgure 3 and smlar results on spectral cluster sub-groups are shown n Fgure 4. The TP rates of JSC- are llustrated n Fgure 3 (a) and Fgure 4 (a). The TP rate mproved rato s calculated by the TP rate of JSC- method dvded by the TP rate of method and s lsted n Fgure 3 (b) and Fgure 4 (b). We fnd that TP rates of JSC- method fluctuates wth the varatons of cluster number. There are no obvous trends wth almost all the JSC- models except TP rates of SOMDD and MoGDD models show a slghtly decreased trend. The results are smlar for both TP rate on K-means sub-groups and for TP rate on Spectral cluster sub-groups. We conclude that the unsupervsed cluster number has no obvous effect on the TP rate of the JSC- method and some models performances degenerate. We also found that most of the TP rate mproved ratos are above one and ths stuaton s clearer n Fgure 4 (b) wth the TP rate mproved ratos on spectral cluster sub-groups. Ths phenomenon ndcates that TP results on most of the sub-groups of the data-set can be mproved and t s most obvous wth spectral clusterng sub-groups.

18 18 Je Zhang, Je Lu and Guangquan Zhang The last observaton from Fgure 3 and Fgure 4 s that the TP mproved rato s lmted n comparson to the supervsed sub-groups. The average TP mproved rato of supervsed, or knowledge based, JSC- method s and the mnmum TP mproved raton s The two ratos are shown n Fgure 3 (b) and Fgure 4 (b) as black hard lne and magenta dashed lne. It s clear that most FP mproved ratos on unsupervsed sub-groups are lower than the magenta dotted lne- the mnmum TP mproved rato on knowledgebased sub-groups. 6. Conclusons Ths paper proposes a JSC- model for real world genune one class problems. Instead of focusng on the models, we delved nto the data-set and developed dvded sub-classes and combned sub-classfers model: the JSC- method. In genune one class problems, especally n the medcal and bologcal domans, the subclasses are a natural exploraton method. Therefore, the proposed method s very practcal n these applcaton areas. The experments show the results that we expected and ndcate mproved performances to normally applyng the method. The proposed JSC- method also helps features reducton. Wthout outler knowledge, genune one class problems can only select features by an unsupervsed method. Wth sub-classes dentfed, the supervsed feature selecton method can be appled to the sub-class. Most of the unsupervsed feature selecton or dmenson reducton technques just compress the orgnal feature space nto a target feature space and the selected features cannot be explaned. Ths dffculty can also be overcome by applyng the JSC- method. Though we change the feature selecton object, the experments show that ths feature selected unon model wll not decrease performance. We conclude that the approprate sub-class of data-set and features makes the JSC- model perform extremely well. The proposed JSC- method also has better performance on unsupervsed clusterng groups f we choose a sutable model, cluster method and cluster number. However, large cluster numbers degenerate the JSC- performances. We also found that the spectral cluster method s better than the K-means cluster method when applyng JSC-. The results of the JSC- method on unsupervsed sub-clusters cannot compete wth our proposed method on knowledge based sub-clusters. Further research nto ths method wll select approprate sub-classes. There should be enough cases n the data-set, otherwse there wll not be enough cases n the dvded the sub-data-set. Sometmes the mbalances n the sub-classes wll also affect the CS- method. If there are no sub-classes n the data-set, we wll apply the clusterng model to cluster the data-set nto sub-classes. Though ths s not as natural as the more meanngful sub-class, ths may be a soluton for ths knd of problem. The man concern s to fnd an approprate cluster number. These ssues wll be nvestgated n future research.

19 Instructons for Typng Manuscrpts (Paper s Ttle) 19 Acknowledgments The work presented n ths paper was supported by the Australan Research Councl (ARC) under Dscovery Project DP Specal thanks to Dr Declan Butler for provdng an mportant data-set whch was used n ths study. References 1. M. Bramer, S. Crone, J. Guajardoand R. Weber, A study on the ablty of Support Vector Regresson and Neural Networks to Forecast Basc Tme Seres Patterns. In Artfcal Intellgence n Theory and Practce, (Sprnger Boston, 2006), pp S. Vong, B. Coghlan, S. Mardy, D. Holl, H. Seng, S. Ly, M. J. Mller, P. Buchy,Y. Froehlch, J. B. Dufourcq, T. M. Uyek, W. Lmand T. Sok, Low frequency of poultry-tohuman H5NI vrus transmsson, southern Camboda, Emerg Infect Ds 12 (2006) OIE UPDATE ON HYGHLY PATHOGENIC AVIAN INFLUENZA IN ANIMALS (TYPE H5 and H7). (20/11), 4. WHO, Cumulatve Number of Confrmed Human Cases of Avan Influenza A/(H5N1) Reported to WHO M. Gauther-Clerc, C. Lebarbenchonand F. Thomas, Recent expanson of hghly pathogenc avan nfluenza H5N1: a crtcal revew. Ibs 149 (2007) J. D. Brown, D. E. Stallknechtand D. E. Swayne, Expermental nfecton of swans and geese wth hghly pathogenc avan nfluenza vrus (H5N1) of Asan lneage. Emerg Infect Ds 14 (2008) H. Chen, G. J. D. Smth, S. Y. Zhang, K. Qn,J. Wang, K. S. L, R. G. Webster, J. S. M. Persand Y. Guan, Avan flu H5N1 vrus outbreak n mgratory waterfowl. Nature 436 (2005) A. M. Klpatrck, A. A. Chmura, D. W. Gbbons, R. C. Flescher, P. P. Marraand P. Daszak, Predctng the global spread of H5N1 avan nfluenza. Proceedngs of the Natonal Academy of Scences (PANS) 103 (2006) T. Weberand N. Stlanaks, Ecologc mmunology of avan nfluenza (H5N1) n mgratory brds. Emerg Infect Ds 13 (2007) M. Glbert, X. Xao, D. U. Pfeffer, M. Epprecht, S. Boles, C. Czarneck, P. Chataweesub, W. Kalpravdh, P. Q. Mnh, M. J. Otte, V. Martnand J. Slngenbergh, Mappng H5N1 hghly pathogenc avan nfluenza rsk n Southeast Asa. Proceedngs of the Natonal Academy of Scences (PANS) 105 (2008) M. M. Moya, M. W. Kochand L. D. Hostetler One-class classfer networks for target recognton applcatons, n World Congress on Neural Networks, Portland, 1993, pp M. Markouand S. Sngh, Novelty detecton: a revew--part 1: statstcal approaches. Sgnal Processng 83 (2003) V. J. Hodgeand J. Austn, A Survey of Outler Detecton Methodologes. Artfcal Intellgence Revew 22 (2004) B. Scholkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smolaand R. C. Wllamson, Estmatng the Support of a Hgh-Dmensonal Dstrbuton. Neural Computaton 13 (2001)

20 20 Je Zhang, Je Lu and Guangquan Zhang 15. D. M. J. Taxand R. P. W. Dun, Support Vector Data Descrpton. Mach. Learn. 54 (2004) H. Yu, J. Hanand K. C. Chang, PEBL: Web Page Classfcaton wthout Negatve Examples. IEEE Transactons on Knowledge and Data Engneerng 16 (2004) C. Varun, B. Arndamand K. Vpn, Anomaly detecton: A survey. ACM Comput. Surv. 41 (2009) H. Bu,J. Wangand X. Huang, Fabrc defect detecton based on multple fractal features and support vector data descrpton. Engneerng Applcatons of Artfcal Intellgence 22 (2009) C. Sanchez-Hernandez, D. S. Boydand G. M. Foody, One-Class Classfcaton for Mappng a Specfc Land-Cover Class: SVDD Classfcaton of Fenland. IEEE Transactons on Geoscence and Remote Sensng 45 (2007) H. Garcesand D. Sbarbaro, Outlers detecton n envronmental montorng databases. Engneerng Applcatons of Artfcal Intellgence 24 (2011) T. Onoda, H. Murataand S. Yamada, Non-Relevance Feedback Document Retreval based on One Class SVM and SVDD. In Internatonal Jont Conference on Neural Networks, 2006 (IJCNN '06) Vol. (IEEE, Vancouver, Canada, 2006), pp L. Manevtz, M. and M. Yousef, One-class svms for document classfcaton. J. Mach. Learn. Res. 2 (2002) X. Zhu,X. Wuand C. Zhang, Vague One-Class Learnng for Data Streams. In Nnth IEEE Internatonal Conference on Data Mnng, 2009 (ICDM '09), Vol. (IEEE, Mam, FL,USA, 2009), pp P. Juszczak, D. M. J. Tax, E. P. Kalskaand R. P. W. Dun, Mnmum spannng tree based one-class classfer. Neurocomputng 72 (2009) Y. Xuand R. G. Brereton, Dagnostc Pattern Recognton on Gene-Expresson Profle Data by Usng One-Class Classfcaton. Journal of Chemcal Informaton and Modelng 45 (2005) J. C. Setubal, S. Verjovsk-Almeda, E. J. Spnosaand A. C. P. L. F. de Carvalho, Combnng One-Class Classfers for Robust Novelty Detecton n Gene Expresson Data. In Advances n Bonformatcs and Computatonal Bology, (Sprnger Berln / Hedelberg, 2005),pp A. Baroch, S. Cohen-Boulaka, C. Frodevaux, J. Reyesand D. Glbert, Combnng One- Class Classfcaton Models Based on Dverse Bologcal Data for Predcton of Proten- Proten Interactons. In Data Integraton n the Lfe Scences, (Sprnger Berln / Hedelberg, 2008),pp M. Yousef, S. Jung, L. C. Showeand M. K. Showe, Learnng from postve examples when the negatve class s undetermned--mcrorna gene dentfcaton. Algorthms Mol Bol 3 (2008) G. Guptaand J. Ghosh, Robust one-class clusterng usng hybrd global and local search. In Proceedngs of the 22nd nternatonal conference on Machne learnng, Vol. (ACM, Bonn, Germany, 2005). 30. W. Daelemans, B. Goethals, K. Mork, K. Hempstalk, E. Frankand I. Wtten, One-Class Classfcaton by Combnng Densty and Class Probablty Estmaton. In Machne Learnng and Knowledge Dscovery n Databases, (Sprnger Berln / Hedelberg, 2008), pp

21 Instructons for Typng Manuscrpts (Paper s Ttle) O. Kaynak, E. Alpaydn, E. Oja, L. Xu, D. Taxand K.-R. Müller, Feature Extracton for One-Class Classfcaton. In Artfcal Neural Networks and Neural Informaton Processng ICANN/ICONIP 2003, (Sprnger Berln / Hedelberg, 2003), pp R. Polkar, Ensemble based systems n decson makng. Crcuts and Systems Magazne, IEEE 6 (2006) T. Wndeatt, Accuracy/Dversty and Ensemble MLP Classfer Desgn. Neural Networks, IEEE Transactons on 17 (2006) L. Breman, Baggng predctors. Machne Learnng 24 (1996) R. E. Schapre, The strength of weak learnablty. Machne Learnng 5 (1990) B. Vermaand A. Rahman, Cluster Orented Ensemble Classfer: Impact of Mult-cluster Charactersaton on Ensemble Classfer Learnng. Knowledge and Data Engneerng, IEEE Transactons on PP F. N. Jula, K. M. Iftekharuddnand A. U. Islam, Dalog Act Classfcaton Usng Acoustc And Dscourse Informaton Of Maptask Data. Internatonal journal of computatonal ntellgence and applcaton 9 (2010) L. Rokach, O. Mamonand O. Arad, Improvng Supervsed Learnng by Sample Decomposton. Internatonal Journal of Computatonal Intellgence and Applcaton 5 (2005) D. M. J. Tax. One-class Classfcaton. Doctoral thess, (Delft Unversty of Technology, 2001). 40. P. Perner, Concepts for novelty detecton and handlng based on a case-based reasonng process scheme. Engneerng Applcatons of Artfcal Intellgence 22 (2009) S. Vllalbaand, P. Cunnngham, An evaluaton of dmenson reducton technques for one-class classfcaton. Artfcal Intellgence Revew 27 (2007) H. Farvareshand M. M. Sepehr, A data mnng framework for detectng subscrpton fraud n telecommuncaton. Engneerng Applcatons of Artfcal Intellgence 24 (2011) H. Peng, F. Longand C. Dng, Feature Selecton Based on Mutual Informaton: Crtera of Max-Dependency, Max-Relevance, and Mn-Redundancy. IEEE Transactons on Pattern Analyss and Machne Intellgence 27 (2005) D. M. J. Tax DDtools, the Data Descrpton Toolbox for Matlab.