Multilayer Perceptron Neural Network Classification of Malignant Breast. Mass

Transcription

1 Multilayer Perceptron Neural Network Classification of Malignant Breast Mass Joshua Henry 12/15/2017 Introduction Breast cancer is a very widespread problem; as such, it is likely that one may know someone who is affected by it during their lifetime. An unsettling statistic, as reported by breastcancer.org is that About 1 in 8 U.S. women (about 12%) will develop invasive breast cancer over the course of her lifetime. The projected casualties from breast cancer for 2017 is also quite worrisome: About 40,610 women in the U.S. are expected to die in 2017 from breast cancer, though death rates have been decreasing since (breastcancer.org) Since this problem affects so many people, I think it is important to use artificial intelligence in order to hasten the detection of malignant breast mass. But which method will produce the most accurate results with such a high-stakes problem? In this project I use a multilayer perceptron neural network and test different structures, as well as different training algorithms, in order to garner the best results. Data The data set used, Breast Cancer Wisconsin (Diagnostic) Data Set, from kaggle.com holds 569 samples; each sample is a description of a fine-needle aspirate, each taken from a

2 different breast mass. Descriptions include the mean, standard error and largest observation of the radius, perimeter, area, smoothness, compactness, concavity, concave points, symmetry, and fractal dimension of the fine-needle aspirate. This gives us 30 features per data sample to use in the neural network. There are two classes in this data set: malignant and benign. The original data file taken from kaggle denotes the classes with an M or B respectively, but for the purpose of training a neural network, I replaced each M with 1 and each B with 0. There was also a column in the data called id which holds the id of each sample, and I have removed that from the data set. Neural Network Since there are 30 features, each neural network uses 30 input nodes. These inputs are then forward propagated through each layer of the network until the two output nodes are reached (output node 1 malignant, output node 2 benign). The hidden structure of the network is to be tested, as well as four different training algorithms. These algorithms include: 1) Scaled conjugate gradient descent with backpropagation, 2) BFGS Quasi-Newton backpropagation, 3) Gradient descent with adaptive learning rate and backpropagation, and 4) Gradient descent with momentum and adaptive learning rate with backpropagation. Neural Network Implementation MATLAB I wrote a MATLAB program which uses patternnet in order to test different structures and training algorithms. The program prompts the user for information regarding the structure of the neural network, how the data will be partitioned into testing, validation, and training, as well as how many times the network will be retrained; retraining is crucial for obtaining the average performance (using cross-entropy) and the average classification accuracy. The best and

3 worst performance and classification accuracy are also computed. The training process has a maximum of 1000 epochs for each training period. The hidden neuron activation function used is tansig, and the output function used is softmax. Each test uses a 15% testing ratio, 15% validation ratio, and 70% training ratio. The program s input prompt to establish a neural network The program s output Testing Trial #1: Network Structure

4 For the first testing trial, I have used one hidden layer with ten neurons. Each training function was tested, and the neural network was retrained 100 times. Results are as follow: Best Accuracy Worst Accuracy Average Accuracy Scaled Conj. BFGS Quasi- Newton w/ Adaptive L- Rate w/ Momentum & Adaptive L-Rate % % % % % % % % % % % % Best Worst Average Overall, a network structure seems to garner accurate results for this data. Scaled conjugate gradient descent with backpropagation performed the best out of the four training algorithms, whereas gradient descent with an adaptive learning rate performed the worst. Using scaled conjugate as the training function seems to be the safest to use, since after being trained 100 times, its lowest accuracy percentage was 95.6%, which is still very useful. The gradient descent with adaptive learning rate algorithm had a low of 62.91%, which could cause major classification problems. Testing Trial #2: Network Structure For the second testing trial, I have used two hidden layer with ten neurons per layer. Each training function was tested, and the neural network was retrained 100 times. Results are as follow:

5 Best Accuracy Worst Accuracy Average Accuracy Scaled Conj. BFGS Quasi- Newton w/ Adaptive L- Rate w/ Momentum & Adaptive L-Rate % % % % % % % % % % % % Best Worst Average Again, the training function that performed the best (in terms of performance and accuracy percentages) was scaled conjugate gradient descent with backpropagation, but only slightly above BFGS Quasi-Newton backpropagation: BFGS had better best accuracy percentage and best performance measurement, but is only slightly below scaled conjugate in average accuracy and average performance. For this trial, the worst performing training function was gradient descent with momentum and an adaptive learning rate; the worst accuracy percentage got as low as 39.2%, and its average accuracy percentage was only 94.93%. Testing Trial #3: Network Structure For the third testing trial, I have used three hidden layers with ten neurons per layer. Each training function was tested, and the neural network was retrained 100 times. Results are as follow:

6 Best Accuracy Worst Accuracy Average Accuracy Scaled Conj. BFGS Quasi- Newton w/ Adaptive L- Rate w/ Momentum & Adaptive L-Rate % % % % % % % % % % % % Best Worst Average Scaled conjugate gradient descent with backpropagation performed the best again, but not by much: BFGS has a very similar average accuracy percentage and average performance. The best accuracy percentages are near identical across the board, but gradient descent with momentum and an adaptive learning rate scored the worst; with an average accuracy of % and an average performance of , it would be best not to use this training algorithm with a structure. However, adding a third hidden layer helped raise its worst accuracy percentage of 39.19% to 61.16% which is a strong improvement. Testing Trial #4: Network Structure For the fourth and final testing trial, I have used four hidden layers, this time with five neurons per layer. Each training function was tested, and the neural network was retrained 100 times. Results are as follow:

7 Best Accuracy Worst Accuracy Average Accuracy Scaled Conj. BFGS Quasi- Newton w/ Adaptive L- Rate w/ Momentum & Adaptive L-Rate % % % % % % % % % % % % Best Worst Average For a network structure, using BFGS Quasi-Newton as the training function will result in the most accurate results, with the second best being scaled conjugate gradient descent. The worst accuracy percentage for this neural network comes from using gradient descent with momentum and an adaptive learning rate, clocking in at 88.13%. Although scaled conjugate gradient descent backpropagation has a worst accuracy of 62.74%, its average accuracy is still efficient as it scored 97.03%. Conclusion: By using the MATLAB program I have written, one may tinker with different structures and training functions of a multilayer perceptron artificial neural network in order to find the best structure suited for the cancer data set by comparing the best, worst, and average accuracy percentages as well as best, worst and average performance of a network. Overall, scaled conjugate gradient descent has performed the best the majority of the time through the trials.

8 The best average accuracy percentage one can obtain with the above structures is around ~97.6%, and the best average performance is ~0.04. It is important to consider which training function one should use for a given neural network structure, as some perform better than others. An efficient network to use in order to classify malignant breast masses would be the structure with scaled conjugate gradient descent backpropagation, as the average accuracy percentage and the average performance was the best of all trials (97.62% and respectively); it is also capable of reaching a 99.12% accuracy score which would produce a very satisfactory classification model. WORKS CITED U.S. Breast Cancer Statistics. Breastcancer.org. Breastcancer.org. URL: March 10, Breast Cancer Wisconsin (Diagnostic) Data Set. Kaggle.com. URL: Patternnet: Pattern recognition network. Mathworks.com. URL: Professor Yu Hen Hu lecture slides