Design of Classifier for Detection of Diabetes using Genetic Programming

Transcription

1 Design of Classifier for Detection of Diabetes using Genetic Programming M. A. Pradhan, Abdul Rahman, PushkarAcharya,RavindraGawade,AshishPateria Abstract Early diagnosis of any disease with less cost is always preferable. Diabetes is one of such disease. It has become the 4th leading cause of death in developed countries and is also reaching epidemic proportions in many developing and newly industrialized nations. Due to its importance the need for detection of this disease is increasing. A classifier is one such detection function which classifies between two or more classes by assigning labels to individuals. In this paper we propose a different approach of genetic programming (GP) called GP with comparative partner selection (CPS) for designing of classifier. Our system will consist of two stages, first one being the generation of a single feature from available features using GP from the training data. The second stage consists of using the test data for checking of this classifier. Diagnosis of diabetes can be supplemented by this GP based classifier. Keywords Classifier, comparative partner selection, diabetes, genetic programming. D I. INTRODUCTION IABETES is a metabolic disorder where in human body does not produce or properly uses insulin, a hormone that is required to convert sugar, starches and other into energy. This leads further complications like adult blindness, endstage renal disease (ESRD), gangrene, and amputations. Overweight, lack of exercise, family history and stress increase the likelihood of diabetes. There are three main types of diabetes - type 1, type 2 and gestational diabetes. Type 1 results from the body's failure to produce insulin while type 2 results from insulin resistance, a condition in which cells fail to use insulin properly sometimes combined with absolute insulin deficiency[3]. Gestational diabetes is a form of diabetes which affects pregnant women. Though, diabetes mellitus is not completely curable but, it is controllable to a great extent. Advances in diabetes research have led to better ways of controlling diabetes and treating its complications. Mrs. M. A. Pradhan is faculty (Asst. Professor in charge HOD) at AISSMS College of engineering, Pune, India. madhavipradhan@rediffmail.com Abdul Rahman is undergraduate student from AISSMS College of Engineering Pune. abdul.rahman1104@yahoo.com AcharyaPushkar is under graduate student from AISSMS College of Engineering Pune. pushkar.acharya@yahoo.in AshishPateriaisunder graduate student from AISSMS College of Engineering Pune. pateriaashish@yahoo.in RavindraGawade is under graduate student from AISSMS College of Engineering Pune. gawaderavindra11@gmail.com However, even if diabetes is under control it still contributes to heart disease[15]. Diagnosis of diabetes depends on many factors and thus it makes the physician's job very difficult. Thus it is dependent not only the physician's knowledge of the disease but also his/her experience in dealing with similar patients. Normally physicians make their decisions by comparing the test results of current patient with some previous patients who also had similar conditions. Also the demand for physicians will also increase if all the persons at risk need to be tested. As physicians need to have a look at previous results while making their decision, they may need a tool for listing all the previous decisions made on the patients having similar conditions. So a classifier system is needed which can classify that list according to the decision made by experts. There is no doubt that the most important factors in diagnosis are data taken from patient and expert's opinion on that data[3]. Here the artificial intelligence techniques and classifiers can help a lot. Thus the use of classifier system in healthcare industry is increasing rapidly. II. RELATED WORK A number of classification techniques have been used for the classification of diabetes disease.polatet. al. proposed two different approaches for diabetes data classification - firstly, principal component analysis and neuro-fuzzy inference and secondly, Generalized Discriminant Analysis (GDA) and least square support vector machine (LS-SVM).They achieved an accuracy of 89.47% and 79.16% respectively[10].deng and Kasabov obtained 78.4% classification accuracy with 10-fold cross-validation (FC) using ESOM[8].An instance counting algorithm ARTMAP-IC was presented by Carpenter and Markuzonand obtained an accuracy of 81%[6]. Kayaer and Yildirim used general regression neural networks to achieve an accuracy of 80.21%[11]. A neural network approach was proposed byhasantemurtas et al. for classification of diabetes data and achieved 82.37% accuracy [12]. All of these figures are indicative of the accuracy only and they do not report the standard deviations. Thus the robustness of these approaches cannot be established.many more algorithms have been proposed for the classification of diabetes data with accuracy between 59.5% and 77.7%. A study of these methods can be seen in Polatet. al[10]. In this paper, we have proposed the use of genetic programming (GP) with comparative partner selection (CPS), 125

2 and will explore various methods of selection, initialization and replacement strategies to yield better results, to design the classifier for detection of diabetes. M. W. Aslam and A. K. Nandi proposed a system using GP and achieved an accuracy of 78.5±2.2%[3]. They have given the use of GP and GP with CPS and have given a comparison between the two techniques. Day and Nandi had pioneered the idea of CPS in order to emphasize the importance of phenotype in GP[5]. III. THE PROPOSED SYSTEM A classifier is a function which takes a feature in p dimension as input and assigns a class label to it. Mathematically[4], a classifier can be defined as - C: S n L vc (1) Where, C is the classifier that maps search space to label vectors, S n is the 'n' dimensional search space and the L vc is the set of label vectors i.e. {Diabetic, Non-diabetic}. L vc can be further defined as - 2 L vc = { y i ϵ {0,1} for all i, ii=1 yy ii =1 } (2) In other words, consider a vector x ϵ S n then C(x) is a vector in 2 dimension with only 1 component as 1 and other a 0 i.e. a tuple in search space can only be diabetic or non-diabetic not both at the same time. Our objective here is to design the classifier using GP with CPS. Given a set of training samples X={x 1, x 2, x 3,..,x n } S n and its associated labels Y {y 1, y 2 } ={diabetic, non-diabetic} then the objective can be described as finding the mapping y=c(x) or C: X Y using Genetic Programming. For this two class problem, a classifier or an individual is represented as a single tree T in genetic programming as follows - ift(x) 0, x ϵ class 1 i.e. x is diabetic < 0, x ϵ class 2 i.e. x is non-diabetic. Fig.1 Block Diagram of Proposed System. A. Standard Genetic Programming Genetic programming (GP) is an evolutionary computation technique that automatically solves problems without requiring the user to know or specify the form or structure of solution in advance. GP has been used to solve a wide range of practical problems producing a number of human-competitive results and even patentable new inventions. At the most abstract level GP is a systematic, domain independent method for getting computers to solve problems automatically starting from a high level statement of what needs to be done[17]. GP evolves a set of individuals, usually represented in memory as tree structure generation by generation. It emulates evolution in nature to transform a population of solutions into new and hopefully better population of solutions. Just like nature, GP is also random and thus cannot always guarantee results, but the same randomness can help it to escape the traps in which the deterministic algorithms are captured by. The basic steps[1] in a GP system are 1. Randomly create an initial population of individuals from the available primitives. 2. Repeat 3. Evaluate the fitness value of each individual. 4. Select one or two individual(s) from the population with a probability based on fitness value to participate in genetic operations. 5. Create new individual(s) by applying genetic operations with specified probabilities. 6. Until an acceptable solution is found or some otherstopping condition is met(e.g. a maximum number of generations is reached). 7. Return the best-so-far individual. The idea of GP[4] can be described by the following equation- G t+1 = g 0 (f(g t )) (3) Where,g t+1 is new generation being produced, g t is the current generation being processed and function f chooses the fittest individuals g t. Function g 0 applies genetic operators like crossover, mutation and reproduction on g t to produce g t+1. B. Basic Terms of Standard GP: B.1 Function Pool It is a set of functions that will be used by the intermediate nodes in the structure of the tree. The function pool can contain different types of functions which are problem dependent. All these functions may have different number of inputs but always have a single output e.g. for logical problems logical functions like AND, ORetc. are used. The function pool[3] that will be used is {+, -, *, /, square,, sin, cos, asin, acos, log, abs, reciprocal}. B.2 Fitness Function The most significant concept of genetic programming is the fitness function. Genetic Programming can solve a number of problems; it is the fitness function which connects GP to the given problem. Each individual is assigned a fitness value by this function. It determines how well a solution is able to solve the problem. It varies greatly from one type of the problem to another. It is chosen in such a way that highly fitted solutions have high fitness value. Fitness function is the only index to select a chromosome to reproduce for the next generation. B.3 Genetic Operators 126

3 Genetic operators are the programs which operate on the current generation and then produce the next generation which we expect to be better than the current generation. These genetic operators operate similar to any other reproduction process for example sexual and asexual reproduction[3]. The genetic operators used in this approach are: Reproduction Reproduction is a simple genetic operator that copies the individual with the best fitness values directly in to the next generation without undergoing crossover and mutation. The objective of reproduction is preservation of the better genetic material as there may be some loss of genetic information during crossover and mutation. Crossover Crossover is the mostly used genetic operator in the reproduction process. In this operation, two solutions are combined to form two new solutions or offspring[3]. The parents are selected based on their fitness value or on their rank in the current population. Here we will be using CPS for selection of second the parent and when CPS fails to find proper partner then second parent is selected randomly. Its aim is to improve the diversity and the genetic fitness of the population. Crossover causes the exploitation of a region of search space to find better solutions in it. After the selection of the two parents, a random crossover point is selected in both of the parents. Then the sub-trees below the crossover points are interchanged to get offspring i.e. two new trees. Fig. 2 Example of Crossover Mutation Mutation is a different genetic operator which takes a single parent as an input and also returns a single child as output[3]. Its aim is to improve diversity as well as prevent the algorithm from being trapped in a local optimal solution. Mutation achieves this by causing the exploration of search space. The probability of mutation is important factor in achieving optimal solution. If the mutation probability is very high it can lead to random search while a very low mutation rate can increase the chances of getting a local optimal solution to the given. It is a genetic operator that randomly selects a sub-tree and replaces it with another randomly created one. Fig. 3 Example of Mutation C. GP with Comparative Partner Selection In standard GP, genetic operator crossover chooses the individual based solely on their fitness values. But between these individuals there are no criteria which individuals will be crossed with whom they are just picked randomly. The idea of choosing the parents only on a single fitness value maybe limiting in the sense that the individual is judged considering only one aspect. So, assigning an overall fitness ignores the task-wise performance of GP individual[3]. This has led to use of comparative partner selection (CPS). CPS is a pair-wise parent selection method that aims to minimize the population phenotype variance throughout evolutionary process. The intention is to maintain the search period of the evolutionary process, while individuals that do not satisfy all training cases equally, do not dominate the population[5]. An individual may be strong in one direction but weak in other direction so we need to explore the strengths and weaknesses of an individual. Strengths and weaknesses can be pointed out by checking the test cases for which the individual classifies correctly and for which it does not. Recognition of strengths and weaknesses can be important in determining which individuals to crossover so that there is chance of removal of those weaknesses. In other words crossover is discouraged between the individuals that exhibit weakness in the same region. For binary problems we can create a string representation such that 1 represents the correctly classified and 0 represents the incorrectly classified. Such a binary string is called Binary String Fitness Characterization (BSFC). This process is shown in Fig. 4 where gray and black colors represent the strength and weakness of individual respectively[5]. The probability of crossover[5] can be calculated by following equation - PP cc (bb 1, bb 2 ) = XXXXXX(bb 1, bb 2 ) NNNNNNNN(bb 1, bb 2 ) (4) Here,P c is the probability of crossover, b 1 and b 2 are binary strings of two individuals, and summation represents the summation of each bit in the binary string. 127

4 Fig. 4 An example of CPS process IV. THE PROPOSED MODEL This section describes the model that we will follow and also the dataset that will be used. It includes brief description about the initialization methods, fitness function, selection methods and replacement methods that may be used. A. Diabetes Disease Dataset The dataset that will be used here is obtained from UCI Repository of Machine Learning Databases[14]. This database is owned by National Institute of Diabetes and Digestive and Kidney Disease. All patients were Pima Indian females who were above 21 years of age. There are eight attributes and one output variable which has either a value '1' or '0', where '1' means positive test of diabetes and '0' means negative test for diabetes. There are 268(34.9%) cases for class '1' and 500(65.1%) cases for class '0'. The 8 attributes are - 1. Number of times pregnant 2. Plasma glucose concentration a 2hrs in an oral glucose tolerance test. 3. Diastolic Blood Pressure(mm Hg) 4. Triceps skin fold thickness(mm) 5. Hour serum insulin (mm U/ml) 6. Body mass index(weight in kg/[height in m] 2 ) 7. Diabetes Pedigree Function 8. Age in years. B. Initialization of Population Like in other evolutionary algorithms, in GP the individuals in the initial population are typically randomly generated. There are number of different approaches forgenerating this initial random population. The three of the major generation techniques used in GP are - full, grow and ramped half-andhalf method[1][16]. As the name implies the full method tends to produce full/bushy trees. It begins by randomly choosing a function to be the root node. Then it recursively assigns functions to the child nodes of the root node until the maximum depth hasbeen reached, at which point we choose only the terminals. Hence, the full method generates trees where all the leaves are at the same depth. Brief analysis of Diabetes[14] dataset TABLE I ANALYSIS OF DIABETES DATASET Attribute No Mean Standard Deviation Min/Max / / / / / / / /81 The grow method is very similar to the full method in that it is a recursive, random generation of a tree with maximum initial depth. The difference between grow and full is that the grow method does not require the functions to be chosen at any point. If the maximum depth has not yet been reached, either a function or a terminal may be selected. It causes morediverse tree structures with some branches longer than others. These trees tend to be less computationally complex[16]. Neither of the two tree generation methods described above is particularly better at solving all GP problems. Ramped halfand-half method, proposed by Koza, is the most common way of creation of the initial GP population. In this method, half of the individuals are created using the full method and other half are generated using the grow method. The maximum depth of the trees generated is ramped, such that the individuals are created in a range of sizes. Using this method a diverse initial population, both in structure and computational complexity is created[1]. C. Fitness Value and Fitness Function These eight attributes of diabetes data will be supplied as input to GP algorithm. GP should try to derive a classifier that will clearly classify between the two classes viz. diabetic and non-diabetic. Each individual in the population is assigned a fitness value using the fitness function. GP will then choose the individuals with better fitness values to go ahead to next stage of GP. Fitness value will be the driving function to arrive at an individual through generations and generations. The final output will be a single feature that is able to separate the two classes properly. Our aim is to increase the distance between these classes (i.e. increase the intra-class variance) and decrease the distance between the points within each class(i.e. decrease the inter-class variance). Fitness function[3] that will be used here is - FFFFFFFFFFFFFF = mm 1 1 mm 2 σσ σσ (5) 2 where, m 1, m 2 are the means of the two classes σ 1, σ 2 are the standard deviations of the two classes This fitness function tries to increase the distance between the means of the two classes while minimizing the variance of the same classes. 128

5 D. Selection of Individuals Genetic operators in GP are applied to individuals that are probabilistically selected on fitness. Better individuals are more likely to have more offspring than inferior individuals. The selection pressure is defined as the degree to which the better individuals are favored[2]. The objective of the selection pressure should be to improve the population fitness over the successive generations. Because, we have limited processing power and we would like GP to solve problems as effectively and efficiently as possible. The two most commonly used methods for selection are - Roulette Wheel Selection and Tournament Selection. Roulette wheel selection[16] is a classic fitness-proportional selection. In this selection method all individuals have a chance of being selected. Also the probability that an individual is selected is dependent on its normalized fitness value. Because more fit individuals have higher normalized fitness, they have a higher chance of being selected and their genetic material has higher chance of being preserved. In tournament selection a number of individuals are chosen at random from the population. These are compared with each other and best of them is chosen to be the parent[3]. Tournament selection is a more flexible selective procedure than fitness-proportional selection. It works by creating a tight selection pressure on a small local group of individuals. It only looks at which solution is better than another and not how much better. Since it is easy to implement and provides automatic fitness rescaling, it is most commonly used in GP. E. Replacement Strategies Once offspring are produced, a method must determine which of the current members of the population should be replaced by new solutions. Basically there are two kinds of methods for maintaining the population - Generational updates and steady state updates[2]. The basic generational update scheme consists of producing 'N' children from a population of size 'N' to form the population at the next generation and this new population of children completely replaces the parents. Clearly, this kind of update implies that an individual can only reproduce with individuals from the same generation. In steady state update, new individuals are inserted in the population as soon as they are created. The insertion of an individual usually necessitates the replacement of another population member. The individual to be deleted can be chosen as the worst member of the population or as the oldest member of the population. Generally, these updates use an ordinal-based method for both selection and replacement. V. CONCLUSION Genetic Programming (GP) is an evolutionary approach and is an alternative to traditional optimization or classification problems. The results of this approach can change drastically by experimenting with the crossover probability, mutation probability or by selection of different methods of population replacement and selection. It is very helpful when the developer does not have precise domain expertise as the algorithm possesses the ability to explore and learn from the domain. Such qualities of GP make it an excellent approach for designing the solutions for problems which are otherwise difficult to solve by deterministic ways. This paper discusses one such problem that is quite difficult to solve by deterministic algorithms such as greedy method, etc. We proposean approach using GP with comparative partner selection (CPS) to classify the diabetes dataset. The CPS logic eventually leads to enhanced performance of the proposed system. The effect of aging, diploidy, etc. on the performance and functionality of GP is yet to be researched thoroughly and can prove to be an interesting point for future work. REFERENCES [1] Riccardo Poli, William Langdon, Nicholas McPhee, John Koza, A Field Guide to Genetic Programming, Lulu Enterprises, UK Ltd (March 26, 2008) pp [2] S.N. Sivanandam, S.N. Deepa, "Introduction to Genetic Algorithms", Springer Berlin Heidelberg, New York (2008). pp [3] M.W. Aslam, A.K. Nandi, Detection Of Diabetes Using Genetic Programming, 18th European Signal Processing Conference(EUSIPCO-2010),Aalborg, Denmark, August [4] D.P. Muni, N.R. Pal, J. Das, A Novel Approach to Design Classifiers Using Genetic Programming, IEEE Trans. on Evolutionary Computation.,vol.8, no.2, April [5] P. Day, A.K. Nandi, "Binary String Fitness Characterization and Comparative Partner Selection in Genetic Programming", IEEE trans. on Evolutionary Computation., vol.12, no.6, December [6] Gail A. Carpenter, Natalya Markuzon, ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases, Neural Networks, vol.11, Issue 2, March 1998 pp [7] Hong Guo, A.K. Nandi, "Breast Cancer Diagnosis using Genetic Programming generated feature", Pattern Recognition, vol.39, 2006, pp [8] D.Deng, N. Kasabov, On-line pattern analysis by evolving self organizing maps, 5th biannual conference on artificial neural networks and expert systems(annes),2011, pp [9] K. Polat, S. Gunes, "An expert system approach based on principal component analysis and adaptive neuro-fuzzy inference system to diagnosis of diabetes disease", Digital Signal Processing, vol.17, July 2007, pp [10] K. Polat, S. Gunes, A. Aslan, "A cascade learning system for classification of diabetes disease: Generalized discriminant analysis and least square support vector machine", Expert systems with applications, vol.34(1), 2008, pp [11] K. Kayaer, T. Yildirim, "Medical diagnosis on Pima Indian Diabetes using General Regression Neural Networks", International conference on artificial neural networks and neural information processing (ICANN/ICONIP), pp [12] T. Hasan, Y. Nijat, T. Feyzullah, "A comparative study on diabetes disease using neural networks", Expert system with applications, vol.36, May 2009, pp [13] J. K. Kishore et al., "Application of genetic programming for multicategory pattern classification", IEEE trans. on Evolutionary Computation [14] UCI repository of machine learning Databases, Pima Indian Diabetes Dataset. [online] Available at: [15] Diabetes Information Hub.[online]. Available: [16] Genetic Programming Source: Evolutionary Computing at GeneticProgramming.us.[online]. Available: ml 129

6 [17] Wikipedia- The Free Encyclopedia [online]