Autoencoder networks for HIV classification

Transcription

1 Atoencoder networks for HIV classification Brain Leke Betechoh*, Tshilidzi Marwala and Thando Tettey In this paper, we introdce a new method to analyse HIV sing a combination of atoencoder networks and genetic algorithms. The proposed method is tested on a set of demographic properties of individals obtained from the Soth African antenatal srvey. When compared to conventional feedforward neral networks, the atoencoder network classifier model proposed yields an accracy of 92%, compared to an accracy of 84% obtained from the conventional feedforward neral network models. The area nder the ROC crve for the proposed atoencoder network model is 0.86 compared to an area nder the crve of 0.8 for the conventional feedforward neral network model. The atoencoder network model for HIV classification, proposed in this paper, ths otperforms the conventional feedforward neral network models and is a mch better classifier. Keywords: Atoencoder networks, genetic algorithms, HIV classification. ACQUIRED immnodeficiency syndrome (AIDS) was first defined 1 in 1982 to describe the first cases of nsal immne system failre that were identified in the previos year. The hman immnodeficiency virs (HIV) was later identified as the case of AIDS. Risk factor epidemiology examines the individal demographic and social characteristics and attempts to determine factors that place an individal at risk of acqiring a life-threatening disease 2. In this stdy, the demographic and social characteristics of the individals and their behavior are sed to determine the risk of HIV infection; referred to as biomedical individalism 2,3. By identifying the individal risk factors that lead to the HIV infection, it is possible to modify social conditions, which give rise to the disease, and ths design effective HIV prevention policies 2. A model will be created and sed to classify the HIV stats of individals based on demographic properties. In this stdy, the model is created sing atoencoder neral networks and genetic algorithms, which have been applied to classification. An artificial neral network (ANN) is an inter-connected strctre of processing elements. The ANN strctre 4 sed in this stdy consists of three main components (Figre 1) 5. Neral networks have been sccessflly sed for medical informatics, for decision making, clinical diagnosis, prognosis, and prediction of otcomes 6 10 and for classification. Marwala 11 sed a probabilistic committee of neral networks to classify falts in a poplation of nominally identical cylindrical shells and obtained an accracy of 95%, in classifying eight classes of falt cases. Ohno-Machado 12 depicted the limitation on the accracy of the neral network model de to lack of data balance The athors are in the School of Electrical and Information Engineering, University of the Witwatersrand, Private Bag 3, Wits, 2050, Soth Africa. *For correspondence. ( b.leke@ee.wits.ac.za) and increased the accracy by sing seqential neral networks. Lisboa 13 assessed the evidence of healthcarebenefits sing neral networks. Fernandez and Caballero 14 sed ANN to model the activity of cyclic rea HIV-1 protease inhibitors. They showed that ANN were capable of representing the nonlinearity in the HIV model. Lee and Park 15 applied neral networks to classify and predict the symptomatic stats of HIV/AIDS patients based on pblicly available HIV/AIDS data. A stdy was also performed to predict the fnctional health stats of HIV/ AIDS patients defined as in good health or not in good health, sing neral networks 16. Lamann and Yom 17 sed the racial and ethnic grop differences to model the prevalence of the disease and scceeded in relating the demographic properties to the transmission of the disease. Pondstone et al. 2 related demographic properties to the spread of HIV. Their work jstifies the se of sch demographic properties in creating a model to predict the HIV stats of individals, as done in this stdy. The above models conclded that ANN performed well in HIV classification problems. The methodology presented here aims at sing demographic and social factors, to predict the HIV stats of an individal, sing atoencoder neral networks. The most common neral network architectre is the mltilayer perceptron (MLP). An alternative network is the radial basis fnction (RBF) 5. The se of MLP over RBF can be attribted to the fact that the RBF sally reqires the implementation of the psedo-inverse of a matrix for training, which is often singlar while MLP ses conventional feedforward optimization methods, which are stable 5. In or stdy, preliminary design showed that the MLP otperformed the RBF. This can be attribted to the fact that MLP networks, also known as niversal approximators, are capable of modelling any complex relationship with one or two hidden layers 5 and are ths most sited for this stdy. More details on neral networks and CURRENT SCIENCE, VOL. 91, NO. 11, 10 DECEMBER

2 Demographic inpt parameters from antenatal dataset x 1 x 2 x 3 x 9 Inpt h 1 h 2 h 77 Hidden Otpt Node y (HIV stats of individal) Figre 1. Feed-forward MLP network architectre. MLP can be fond in refs In this stdy neral networks are sed with genetic algorithms. A genetic algorithm (GA) is an optimization method deriving its behavior from processes of evoltion in natre, inspired by Darwin s theory of natral evoltion 23,24. This is done by the creation within a machine/compter of a poplation of individals. In this stdy, the poplation of individals represents the missing inpt entries. The individals then go throgh the process of evoltion. GA ses fitness-proportionate or tornament selection to select the missing entries (individals) probabilistically that yields the right HIV stats for the individals. Althogh not garanteed to provide the globally optimm soltion, GA has been shown to be highly efficient at reaching a very near optimm soltion in a comptationally efficient manner 23,24. More details on GA can be fond in refs 25 and 26. In the literatre review, there is no method proposed ths far that investigates the se of atoencoder networks for HIV modelling. The aim of this paper will ths be to propose a new method, which is based on atoassociative models 27 combined with GA to classify the HIV stats of an individal based on demographic properties. The proposed method is tested on the classification of the HIV stats of individals sing a data set obtained from the Soth African antenatal seroprevalence srvey. The method is then compared with conventional feedforward neral network models that have already been applied in the HIV modelling problem as presented in the literatre review. Backgrond Atoassociative networks Atoassociative networks are models where the network is trained to recall the inpts 27. This network ths predicts 1468 the inpts as otpts, whenever an inpt is presented. These networks have been sed in a nmber of applications An atoassociative neral network encoder (or simply known as atoencoder) consists of an inpt and otpt layer with the same nmber of inpts and otpts, hence the name atoassociative, combined with a narrow hidden layer 27. The networks will be trained sing HIV/ AIDS demographic data. The hidden layer attempts to reconstrct the inpts to match the otpts, by minimizing the error between the inpts and the otpts when new data is presented. The narrow hidden layer forces the network to redce any redndancies, bt still allows the network to detect non-redndant data. However, it mst be noted that for missing data estimation it is absoltely crcial that the network mst be as accrate as possible and that this accracy is not necessarily realized throgh few hidden nodes as is the case when these networks are sed for data compression. It is therefore crcial that some process of identifying the optimal architectre be sed. GA is sed in this stdy to find the optimal atoencoder architectre by finding the global optimm soltion 23. The ato-encoder neral network architectre sed in this stdy is shown in Figre 2. Classification as a statistical pattern The goal of or classification is to develop an algorithm, which will assign an individal, represented by a vector {x} describing the demographic, social and behavioral characteristics of that individal, to one of the HIV classes, C 1 or C 2 (where C 1, C 2 represents the stats of an individal, which may be positive or negative). The data on which the model is based pon contains demographic examples of individals, as well as the classes to which those individals belong. The otpt of the classification system is assigned to the variable y. The classification CURRENT SCIENCE, VOL. 91, NO. 11, 10 DECEMBER 2006

3 model is therefore reqired to map the inpts x 1,..., x d to the otpt y. A mathematical fnction describes this mapping, and since it cannot be explicitly determined, the data is sed to determine the parameters. This can be written as follows: { y} = f({ x}, { w}). (1) Here {w} is the mapping weights and {x} represents the demographic inpt parameters and {y} represents the HIV stats. In this stdy, atoencoder neral networks are sed to obtain the fnctional mapping, and spervised learning is sed to obtain the parameters. The prpose of the classification model is to design the decision srface to assign new inpts to one of the classes 5. Methodology The literatre review showed that models for HIV prediction and classification have been developed sing conventional feedforward neral networks architectres and have worked well. However, it was fond from the literatre review that atoencoder networks have not been applied to HIV modelling, for prediction and classification. Or work ths focses on proposing a methodology for HIV classification from demographic properties sing atoencoder neral networks and GA. Or work also focses on comparing the proposed atoencoder method to a conventional feedforward neral networks model, by creating a feedforward MLP neral network model and comparing the reslts with the atoencoder network model reslts. HIV classification sing atoencoder networks The NETLAB toolbox 32 was sed to create and train an atoencoder MLP architectre. This toolbox has a 2-layer MLP network, which according to literatre review 5 is capable of modelling any complex relationship, sch as the HIV model. The network implemented consisted of an inpt layer, representing different demographic inpts and the HIV stats, mapped to an otpt layer representing the same characteristics as the inpt layer via the hidden layer. The network was ths trained to recall itself (predict the demographic inpts). This network is shown in Figre 2. One of the inpt nodes in Figre 2, x 2, represented the HIV stats of individals, which was ltimately represented by one of the otpt nodes, y 2, as well. The neral network eqation can be written as in eq. (1). Since the network is trained to recall the demographic inpts, the otpt vector {y} (predicted demographic properties) obtained will be approximately eqal to the inpt vector {x} (actal demographic properties). An error, however, exists between the inpt vector {x} and the otpt vector {y}, which can be expressed as the difference between the inpt and otpt vector. This error is formlated as e= {} x {}. y (2) Sbstitting for {y} from eq. (1) into eq. (2) we get e= { x} f({ x},{ w}). (3) In or work, a minimm and non-negative error is reqired. This can be obtained by sqaring the error fnction in eq. (3) to obtain e ({ x} f({ x},{ w})) 2 =. (4) To predict the HIV stats of individals, the HIV stats inpt, in the inpt vector {x} was assmed as an nknown inpt, while the demographic inpt properties were considered as the known inpts. When the inpt vector {x} has nknown elements, the inpt vector set can be categorized into {x} known represented by {x k } and {x} nknown represented by {x }. Rewriting (4) in terms of {x k } and {x }, we obtain Demographic Inpt properties with HIV stats as one of the inpts x 1 x 2 x 3 x 9 Inpt h 1 h 2 Hidden Otpt Figre 2. Ato-encoder neral network architectre. y 1 = x 1 y 2 = x 2 y 3 = x 3 y 9 = x 9 Predicted Demographic properties and HIV stats from the atoencoder network x x =,{}. e f w x k xk Here {x } represents the HIV stats of the individal, which is nknown, {x k } represents the demographic inpt parameters of the individals in Table 1, {w} represents the weight vector that maps the atoencoder network inpt vector {x} to the same inpt vector {x}. An estimated vale for the HIV stats is then obtained by minimizing eq. (5) sing a GA which was chosen becase it finds the global optimm soltion 25. GA, however, always finds the maximm vale. To cater for this, the negative of eq. (5) was sed as the fitness fnction for the GA. The error fnction to be minimized is ths 2 (5) CURRENT SCIENCE, VOL. 91, NO. 11, 10 DECEMBER

4 Start Demographic Inpts with HIV Stats Unknown Atoencoder neral networks Predicted otpt with the predicted HIV stats Generate Error Fnction (Fitness Fnction) Use GA to estimate HIV stats by minimizing the error fnction Retrn the global optimm as the HIV stats Minimm obtained Convert the continos otpt from the GA to a binary otpt sing the threshold END Figre 3. Flow chart of the proposed model Table 1. Smmary of inpt and otpt variables Variable Type Range Inpt variables Age grop Integer Age gap Integer 1 7 Edcation Integer 0 13 Gravidity Integer 0 11 Parity Integer 0 40 Province Integer 1 9 Race Integer 1 5 Region Integer 1 36 RPR Integer 0 2 WTREV Continos Otpt variable HIV stats Binary [0, 1] x x,{}. e= f w x k xk This estimated vale from the atoencoder network and genetic algorithm was a continos vale representing the HIV stats. A threshold was ths reqired to convert the 2 (6) HIV otpt node vale to a binary vale, representative of the HIV class of the individal. Figre 3 shows the implementation of this proposed model in a flowchart. HIV classification sing neral networks In this model, the NETLAB toolbox 32 was sed to create and train an MLP neral network architectre. The network implemented consisted of an inpt layer, representing different demographic inpts of an individal, mapped to an otpt layer representing the HIV stats of an individal via the hidden layer. The network ths mapped the demographic inpts of individals to the HIV stats. This network is shown in Figre 1. The neral network eqation can be written as in eq. (1). In this model, however, the otpt vector {y} represents the HIV stats of the individal. The network is ths trained to find the relationship between the HIV stats of the individal and the individal s demographic inpt properties. An error, however, exists between the individal s predicted HIV stats (otpt vector) {y} and the individal s actal HIV stats (target vector) {t} dring training, which can be expressed CURRENT SCIENCE, VOL. 91, NO. 11, 10 DECEMBER 2006

5 as the difference between the target and otpt vector. For the neral network HIV classification, the mean sqare error fnction between the target otpt vector {t} and the otpt vector {y} is insfficient as a classification accracy measre, as it only indicates the total nmber of correct classifications. A confsion matrix was ths constrcted and the accracy was obtained from the confsion matrix. The accracy can be formlated as TN + TP Accracy =. (7) TN + FN + TP + FP Here TN = tre negatives (where network predicts an HIV negative person as negative), FP = false positives (where network predicts an HIV negative person as positive), FN = false negatives (where network predicts an HIV positive person as negative) and TP = tre positives (where network predicts an HIV positive person as positive). The accracy fnction was then sed as the fitness fnction in the GA to obtain the optimal neral network parameters. GA was sed as it finds the maximm vale of the fitness fnction, which was reqired in this case. GA was also sed to obtain the threshold vale to convert the continos network otpt to a binary vale representative of HIV. provided for the network. The GA sed for the atoencoder network model proposed in this stdy and the neral network model sed arithmetic cross-over, non-niform mtation and normalized geometric selection. The probability of cross-over was chosen to be 0.75 as proposed in Marwala et al. 34. The probability of mtation was chosen to be as recommended by Marwala et al. 34. GA had a poplation of 40 and was rn for 150 generations. The first experiment investigated the se of atoencoder networks for HIV classification. An atoencoder network with 9 inpts and 9 otpts was constrcted and several nmber of hidden nits were investigated, sing Matlab (ref. 35). A GA was sed to obtain the optimm nmber of hidden nits and yielded an optimm nmber of hidden nits of 2, hence the strctre Linear optimization sing the mean sqare error verss hidden nits was also investigated. As shown in Figre 4, the Reslts and discssion The demographic and medical data, sed in this stdy, came from the Soth African antenatal seroprevalence srvey 33 of This is a national srvey, and pregnant women attending selected pblic health care clinics participating for the first time in the srvey were eligible. The variables obtained are shown in Table 1. These inclde: age of mother, age of partner, edcational level of mother, gravidity (nmber of complete or incomplete pregnancies), parity (nmber of complete pregnancies), province of origin, race of mother, and region of origin. The qalitative variables sch as the province of origin, race of mother and region of origin were encoded to integers. For example, the encoding scheme for race is shown in Table 2. The HIV stats was encoded sing an integer scheme, whereby a 1 represents a positive HIV stats meanwhile a 0 represents a negative HIV stats. The parameter distribtions are also listed in Table 1. A total of 1986 training inpts were Figre 4. The prediction error verss the nmber of hidden nodes. Table 2. Example of an encoding scheme of a qalitative parameter (race) Qalitative parameter (race) Integer encoding White 1 Black 2 Colored 3 Indian 4 Other 5 Figre 5. ROC crve for the atoencoder network classifier. CURRENT SCIENCE, VOL. 91, NO. 11, 10 DECEMBER

6 linear optimization yielded 6 hidden nits as the optimal network that gives the best prediction since as the error does not change significantly from 6 nits onwards (the difference in error is abot 8.5% from 6 hidden nits to 20 hidden nits). It mst be noted, however, that it is generally assmed that the best atoencoder network is the one that has the lowest possible nmber of hidden nits 36. A hidden nit of 2 was ths sed as the optimal atoencoder network nmber of hidden nits. The performance analysis for the atoencoder network model is based on classification accracy and the area nder the ROC crve. The proposed atoencoder network model obtained an HIV classification accracy of 92%. The confsion matrix obtained for the above network is shown in Table 3. The ROC crve for this classification is shown in Figre 5 and the area nder the crve was compted as 0.86, ths giving a very good classifier according to ROC crves docmentation 37. The second experiment investigated the se of conventional feedforward neral network MLP architectre to classify the HIV stats of an individal sing the demo- Table 3. Classifier confsion matrix of the atoencoder network classifier Confsion matrix Predicted positive Predicted negative Actal positive Actal negative Table 4. Classifier confsion matrix of conventional feed forward neral network classifier Confsion matrix Predicted positive Predicted negative Actal positive Actal negative Figre 6. ROC crve for the conventional feedforward neral network classifier graphic inpt properties. The MLP was constrcted with 9 inpts and 1 otpt. A GA was then sed to obtain the optimal strctre and yielded an optimal nmber of hidden nits of 77, hence the strctre was The performance analysis for this network model is also based on classification accracy and the area nder the ROC crve. This network gave an accracy of 84%. The confsion matrix obtained for the above network is shown in Table 4. The ROC crve obtained for this classification is shown in Figre 6 and the area nder this ROC crve obtained was 0.8, which according to ROC crves docmentation 37 is a very good classifier. The reason why atoencoder networks performed better than the conventional feedforward neral network can be attribted to the fact that the atoencoder network focses on characterizing the positive classes independently of the negative classes, whereas the conventional feedforward neral networks may overlook nder-represented classes. We hypothesize that this may be de to lower effective dimension of the atoencoder network classifier. The difference in performance can also be attribted to the fact that in the atoencoder network, classification is done by choosing the best fitting model sing probability distribtions. The class of the network with the smallest reconstrction error is selected. Conventional feedforward neral networks on the other hand jst map an inpt vector to an otpt vector sing scenario and encodes the classes directly. This plays a role becase, for nonlinear models sch as the HIV model, it is sally difficlt to compte the derivatives for the scenarios since they reqire that we integrate all the possible representations that cold have been sed for each particlar observed inpt vector. The distance measre in classification is ths better minimized in the atoencoder network than in the conventional feedforward network model. Conclsion A method based on atoassociative neral networks and genetic algorithms is proposed to classify the HIV stats of an individal from demographic properties. This method is proposed in order to investigate whether sing atoencoder networks improves on the accracy of classification, of an individal s HIV stats, from demographic properties. The proposed method is tested on an HIV data set obtained from the Soth African antenatal seroprevalence srvey of The method is then compared to a conventional feedforward neral network model, implemented sing the MLP architectre. A classification accracy of 92% was obtained for the atoencoder network compared to 84% obtained for the conventional feedforward neral network model implementation. The area nder the ROC crve for the atoencoder network classifier was compted as 0.86 compared to 0.8 compted for the conventional feedforward neral network classifier. The CURRENT SCIENCE, VOL. 91, NO. 11, 10 DECEMBER 2006

7 reslt ths sggest that atoencoder network models are more accrate and better classifiers for the HIV model than conventional feedforward neral network models, since atoencoder networks focs on characterizing the positive classes independently of the negative classes, whereas the conventional feedforward neral networks may overlook nder-represented classes. 1. Root-Bernstein, R., The evolving definition of AIDS. Rethinking AIDS. last accessed: Pondstone, K., Strathdee, S. and Celectano, D., The social epidemiology of hman immnodeficiency virs/acqired immnodeficiency syndrome. Epidemiologic Rev., 2004, 26, Fee, E. and Krieger, N., Understanding AIDS: historical interpretations and limits of biomedical individalism. Am. J. Pblic Health, 1993, 83, Nelson, M. M. and Illingworth, W. T., A Practical Gide to Neral Nets, Addison-Wesley, New York, 1991, 3rd edn. 5. Bishop, C. M., Neral Networks for Pattern Recognition, Oxford University Press, Oxford, Tandon, R., Adak, S. and Kaye, J. A, Neral network for longitdinal stdies in Alzheimer s disease. Artif. Intell. Med., 2006, 36, Alkan, A., Koklkaya, E. and Sbasi, A., Atomatic seizre detection in EGG sing logistic regression and artificial neral network. J. Nerosci. Methods, 2005, 148, Sawa, T. and Ohno-Machado, L., A neral network-based similarity index for clstering DNA microarray data. Compt. Biol. Med., 2003, 33, Szprek, D., Moszynski, R., Smolen, A. and Sajdak, S., Artificial neral network compter prediction of ovarian malignancy in women with adnexal masses. Int. J. Gynaecol. Obstet., 2005, 89, Tan, A-H. and Pan, H., Predictive neral network for gene expression data analysis. Neral Networks, 2005, 18, Marwala, T., Probabilistic falt identification sing a committee of neral networks and vibration data. J. Aircraft, 2001, 38, Ohno-Machado, L., Seqential se of neral networks for srvival prediction in AIDS. Proceedings: AMMA Annal Fall Symposim, 1996, pp Lisboa, P. J. G., A review of evidence of health benefit from artificial neral networks in medical intervention. Neral Networks, 2002, 15, Fernandez, M. and Caballero, J., Modeling of activity of cyclic rea HIV-1 protease inhibitors sing reglarized-artificial neral networks. J. Bioorg. Med. Chem., 2006, 14, Lee, C. and Park, J., Assessment of HIV/AIDS-related health performance sing an artificial neral network. J. Inf. Manage., 2001, 38, Sardari, S. and Sardari, D., Applications of artificial neral network in AIDS research and therapy. Crr. Pharmacet. Design, 2002, 8, Lamann, E. O. and Yom, Y., Racial/ethnic grop differences in the prevalence of sexally transmitted diseases in the United States: a network explanation. Sex Transm. Dis., 1999, 26, Hdson, D. L. and Cohen, M. E., Neral Networks and Artificial Intelligence for Biomedical Engineering. IEEE Press, NJ, Deo, M. C. and Jagdale, S. S., Prediction of breaking waves with neral networks. J. Ocean Eng., 2003, 30, Narendra, K. and Lewis, F., Introdction to the special isse on neral network feedback control. Atomatica, 2001, 37, Rafiq, M. Y., Bgmann, G. and Easterbrook, D. J., Neral network design for engineering applications. J. Compt. Strct., 2001, 79, Svozil, D., Kvasnicka, V. and Pospichal, J., Introdction to mltilayer feed-forward neral networks. J. Chemometrics Intell. Lab. Syst., 1997, 39, Holland, J., Adaptation in Natral and Artificial Systems, University of Michigan Press, Ann Arbor, Goldberg, D. E., Genetic Algorithms in Search Optimization and Machine Learning, Addison-Wesley, Reading, Davis, L., Handbook of Genetic Algorithms, Van Nostrand, New York, Michalewicz, Z., Genetic Algorithms + Data Strctres = Evoltion Programs. Berlin, Springer, 1996, 3rd edn. 27. L, P. J. and Hs, T. C., Application of atoassociative neral network on gas-path sensor data validation. J. Propl. Power, 2002, 18, Atalla, M. J. and Inman, D. J., On model pdating sing neral networks. Mech. Syst. Signal Proc., 1998, 12, Frolov, A., Kartashov, A., Goltsev, A. and Folk, R., Qality and efficiency of retrieval for Willshaw-like atoassociative networks. II. Recognition. Network: Comptat. Neral Syst., 1995, 6, Smaoi, N. and Al-Yakoob, S., Analyzing the dynamics of celllar flames sing Karhnen Loeve decomposition and atoassociative neral networks. Soc. Ind. Appl. Math., 2003, 24, Hines, J. W., Robert, E. U. and Wrest, D. J., Use of atoassociative neral networks for signal validation. J. Intell. Rob. Syst., 1998, 21, Nabney, I. T., NETLAB: Algorithms for Pattern Recognition, Springer Verlag, London, 2003, pp HIV Syphilis Srvey data 2001, Department of Health, Repblic of Soth Africa, 7 March Marwala, T. and Chakraverty, S., Falt classification in strctres with incomplete measred data sing atoassociative neral networks and genetic algorithm. Crr. Sci., 2006, 90, MATLAB 7.1 Manal, Matlab and Simlink for Technical Compting, Release 13, Mathworks, Kramer, M. A., Nonlinear principal component analysis sing atoassociative neral Networks. AIChE J., 1991, 37, ROC Crves Docmentation, roccrves_doc.html, Abot ROC crves, Monash University: last accessed: ACKNOWLEDGEMENTS. We thank Mr Chris Lines, Mr David Starfield, Mr David Vancci and the National Research Fondation for spport. Received 6 Jne 2006; revised accepted 30 Agst 2006 CURRENT SCIENCE, VOL. 91, NO. 11, 10 DECEMBER