Eindhoven University of Technology MASTER. Extracting features to discriminate OSA and non-osa. Pei, W. Award date: Link to publication

Transcription

1 Eindhoven University of Technology MASTER Extracting features to discriminate OSA and non-osa Pei, W. Award date: 2013 Link to publication Disclaimer This document contains a student thesis (bachelor's or master's), as authored by a student at Eindhoven University of Technology. Student theses are made available in the TU/e repository upon obtaining the required degree. The grade received is not published on the document as presented in the repository. The required complexity or quality of research of student theses may vary by program, and the required minimum study period may vary in duration. General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. Users may download and print one copy of any publication from the public portal for the purpose of private study or research. You may not further distribute the material or use it for any profit-making activity or commercial gain

2 Department of Mathematics and Computer Science Databases and Hypermedia Research Group Extracting Features to Discriminate OSA and non-osa Master Thesis Wenjie Pei Supervisors: Dr. Toon Calders Senior Scientist Stefan Winter Drs. Thanh Lam Hoang Eindhoven, July 2013

3

4 Abstract This project is about analyzing anatomical data obtained by pharyngometry with respect to obstructive sleep apnea (OSA) and the goal is to extract valued feature to discriminate OSA and non-osa subjects. The key contributions of this thesis work are two-fold. First, we extract a rich set of 16 features from raw time series data and evaluate all them by ten-fold cross validation with different classifiers. We also did an extensive evaluation of the correlation of each feature with class labels based on t-test, Matthews Correlation Coefficient (MCC), Receiver Operating Characteristic (ROC), and Area Under the Curve (AUC). The experiment results show that several features such as the volume, volume variance, t-test selection feature and stretched length are very predictive. The accuracy of OSA and Non-OSA classification task can reach up to 77% with these features alone. The t-test conducted on these features show the significant correlation between the given features and the class labels. We also found an interesting result that on average OSA patients have smaller volume (stretch length) than non-osa people. From medical point of view, this finding is consistent because OSA likely occurs when the muscles relax during sleep, causing soft tissue in the back of the throat to collapse and block the upper airway [11]. The second major contribution of this work is a method to search for local part of the time series from which the extracted features are most predictive. The main idea is based on an intuitive observation that large part of the time series which usually corresponds to normal part of the throat is not predictive. Therefore, using features extracted from the entire time series for classification is not effective because normal part of the throat contributes a big factor to the value of the features. In order to avoid this type of noise information we propose method to search for small part of time series which likely corresponds to abnormal part of the throat from which the extracted features could be more predictive. We did an exhaustive search for such predictive local parts. We found local parts from which the volume and stretch length were extracted and improved the classification accuracy from 65% to 75%. An interesting finding is that the most predictive local parts found by our algorithm are very close to the oropharyngeal junction point (OPJ), which has been already confirmed by some prior work in the literature [2] as an important point for discriminating OSA and non-osa. They almost belong to the oropharynx region, which is consistent with the domain knowledge of OSA that the oropharynx region is the most possible obstructed site in the upper airway. Extracting Features to Discriminate OSA and non-osa iii

5

6 Acknowledgements First of all, my heartfelt gratitude goes out to my supervisor at Philips, Senior Scientist, Stefan Winter, for his support and encouragement during this project and successful completion of my thesis. I am so honored to take part in the internship in Philips under the supervision of Stefan Winter. Secondly, I would like to thank my supervisor in TU/e, Professor Toon Calders, for his patient guidance and advices. Thirdly, I also want to thank my second supervisor in TU/e, Drs. Thanh Lam Hoang, for providing me a lot of useful ideas and helps. Without his guidance, I could not complete this project. Dr. Alexander Serebrenik is gratefully acknowledged for accepting to be part of my defense committee. Moreover, I would like to thank many colleagues in Philips for their useful advices and concrete comments. They always shared their research experiences and gave me many useful suggestions. They made me enjoy doing this project. Last but certainly not least, friends were also supportive during my stay in the Netherlands. This particularly concerns Jie Yang, Xuefei Chen and JiaChun Cui. I also wish to thank the immeasurable help, though mostly operated at distance, of some close family members. This concerns my father, my mother, my brother and most particularly my grandfather to whom I dedicate this thesis. Extracting Features to Discriminate OSA and non-osa v

7

8 Contents Contents List of Figures List of Tables List of Algorithms vii ix xi xiii 1 Introduction Research Question Motivation Related Work Domain Knowledge Sleep Apnea Obstructive Sleep Apnea Upper Airway Structure Anatomical Determinants of Upper Airway Caliber in OSA Acoustic Pharyngometry Rationale Pros and Cons Workflow 7 4 Datasets Preprocessing of Datasets Remove Noise Normalization Performance Criteria Some Basic Performance Criteria k-fold Cross-Validation Classification Terminology ROC and AUC MCC T-test Classifier based on Information Gain and Optimal Split Point Classifier based on OSP Classification with Raw Data Nearest Neighbor Algorithm with Euclidean Distance Algorithm Experimental Evaluation Extracting Features to Discriminate OSA and non-osa vii

9 CONTENTS Analysis of Failure Conclusion Nearest Neighbor Algorithm with DTW(Dynamic Time Warping) Distance Algorithm Experimental Evaluation Analysis of Failure Conclusion Classifications with Popular Classification Algorithms Experimental Evaluation Analysis of Failure Conclusion Features Local Features OPJ Value OPJ Position OPJ-aligned Feature Maximum Value OPJ Value/Maximum Value Global Features Shapelets based Algorithms Using SAX to transform the time series into low-dimensional symbolic datasets Volume Volume Variance OPJ-aligned Volume Stretched Length Reference-based feature Auto-Correlation DFT-based and DWT-based features DTW Distance between Different Dimensions T-test Selection feature Feature Combination Correlation-based Feature Selection (CFS) Information Gain Evaluation Ranking Feature Combination Performance Conclusion Conclusions 60 Bibliography 62 viii Extracting Features to Discriminate OSA and non-osa

10 List of Figures 2.1 A: Midsagittal MR image in a normal subject demonstrating the upper airway regions [4]: (a) nasopharynx; (b)retropalatal; (c) retroglossal; (d) hypopharynx. B: Important sagittal upper airway structures demonstrated on MR imaging midsagittal magnetic resonance image (MRI) in a normal subject (left) and in a patient with severe OSA (right) [1]. Note that the upper airway is smaller, in both the retropalatal and retroglossal region The workflow of thole project Cross-sectional area over distance from teeth A normal time series This time series does not follow the normal pattern Confusion matrix Roc space (quoted from Wikipedia) Time alignment of two time series OPJ point in the time series ROC curve of results with four dimensions Best interval for Volume with four dimensions, which are indicated by blue lines The selected interval of Volume Variance, which is indicated by blue lines ROC curve of results with four dimensions Best interval for Stretched Length with four dimensions, which are indicated by blue lines Extracting Features to Discriminate OSA and non-osa ix

11

12 List of Tables 4.1 Datasets Classification results of 1-NN algorithm with Euclidean Distance with 10-fold cross validation Accuracy with 1-NN algorithm with DTW distance with cross validation Classification results based on several different classifiers with 10-fold cross validation. The number with blue color indicates the biggest accuracy in the results of all classifiers for one dimension, the tables in the later section are indicated in the similar way h-value of T-test for all points in time series with α = 0.05, totally 45 points in each time series Classification results of feature OPJ Value based on several different classifiers with 10-folds cross validation Statistical results of feature OPJ Value Classification results of feature OPJ Position based on several different classifiers with 10-folds cross validation Statistical results of feature OPJ Position Classification results of OPJ-aligned Feature based on several different classifiers with 10-fold cross validation, k is set h-value of T-test of OPJ-aligned feature Classification results of feature Maximum Value based on several different classifiers with 10-folds cross validation Statistical results of feature Maximum Value Classification results of feature OPJ Value/Maximum Value based on several different classifiers with 10-fold cross validation Statistical results of feature OPJ/Maximum Accuracy with feature Shapelets with 3-folds cross validation Accuracy with different popular classification algorithms on the symbolic sequence transformed by SAX. In this example, the 45 points are transformed into 9 dimensions and the alphabet size is Accuracy with different popular classification algorithms, taking the histogram vector of the symbolic sequence as the feature Accuracy for feature Volume. The total length of interval is Accuracy for feature Volume without interval selection, i.e., volume of whole time series data Classification results of feature Volume based on Decision Tree model Statistical results of feature Volume Evaluation result of feature Volume Variance Accuracy with feature OPJ-aligned Volume Accuracy with feature Stretched Length. The total length of interval is Extracting Features to Discriminate OSA and non-osa xi

13 LIST OF TABLES 7.21 Accuracy for feature Stretched Length without interval selection, i.e., stretched length of whole time series data classification results of feature Stretched Length based on Decision Tree model Statistical results of feature Stretched Length Classification results with RefModel1 based on several different classifiers with 10-folds cross validation. Note that in the test set, there are 20 non-osa subjects and 63 OSA subjects since 20 non-osa subjects are used to calculate the reference model Statistical results of reference-based feature based on RefModel Classification results with RefModel2 based on several different classifiers with 10-folds cross validation. Note that in the test set, there are 30 non-osa subjects and 63 OSA subjects since 10 non-osa subjects are used to calculate the reference model Statistical results of reference-based feature based on RefModel classification results of feature autocorrelation with time step = 1 based on several different classifiers with 10-fold cross validation classification results of feature autocorrelation with time step ranging from 1 to 30 based on several different classifiers with 10-folds cross validation h-value of T-test of feature autocorrelation with time step ranging from 1 to Accuracy of top-8 dimensions DFT-based features with several different classifiers Accuracy of top-8 dimensions DWT-based features with several different classifiers p-value of T-test of top-8 dimensions DFT-based features p-value of T-test of top-8 dimensions DWT-based features Classification results of DTW distance between different dimensions based on several different classifiers with 10-folds cross validation. Note that Su B Si B is the abbreviation of Supine Breath vs Sitting Breath while Su H Si H means Supine Hold vs Sitting Hold T-test results of DTW distance between different dimensions h-value of T-test of feature T-test Selection with α = 0.01, totally 45 points in each time series Classification results of feature T-test Selection based on several different classifiers with 10-folds cross validation Selected features by CFS among all features with 10-fold cross validation. The probability shown after the feature name indicates the probability of being selected in 10-fold cross validation Top 4 features ranked by Information Gain Evaluation Accuracy with Decision Tree and Random Forest classifiers for combined features. 59 xii Extracting Features to Discriminate OSA and non-osa

14 List of Algorithms 1 Feature-Evaluation-Algorithm(F, D train, D test ) NN Euclidean Distance (T rain set, T rain label, unknown object) DTW Distance calculation algorithm (s : array[1..n], t : array[1..m], w : int) Volume-Extraction-Algorithm(D, K) Stretched-Length-Extraction-Algorithm(D, K) CalculateStretchedLength(D, start, end) DFT DWT based Feature Extraction(D, l, m) Extracting Features to Discriminate OSA and non-osa xiii

15

16 Chapter 1 Introduction This document is the master thesis concerning the internship project of extracting features to discriminate OSA and non-osa subjects, which is finished at Philips Research in Eindhoven, supervised by Scientist Stefan Winter in Philips and Prof. Toon Calders in TU/e. The rest of the document is organized as follows. The research question, motivation and the related work are presented in this section. In section 2, the domain knowledge of obstructive sleep apnea (OSA) are given. Section 3 presents the workflow of the whole project. Then the datasets description including the preprocessing is presented in section 4. The performance criteria used in this thesis is given in section 5. We conduct the classifications directly on the raw datasets without any feature extraction in section 6, then a rich set of 16 features, including local features and global features, are proposed in section 7. Finally, section 8 indicates the conclusion of this thesis. 1.1 Research Question The topic is about analyzing anatomical data obtained by pharyngometry with respect to obstructive sleep apnea (OSA). We have recorded data from over 100 subjects (OSA and non-osa) that can be analyzed. The central question is whether we can define any features that discriminate OSA and non-osa subjects. We would look into the underlying anatomy and work on extracting discriminative features. The deliverable would be algorithms that extract these features from the given data and a description of their discriminative power along with an idea of their physiological meaning. The focus is on proposing (and investigating) discriminative (OSA/non-OSA) features. While this of course requires classification based on the proposed features to assess their discriminative power, the challenge is more in defining good features rather than implementing an optimized classifier. 1.2 Motivation Obstructive sleep apnea (OSA) is the most common type of sleep apnea and is caused by obstruction of the upper airway. It is characterized by repetitive pauses in breathing during sleep, despite the effort to breathe, and is usually associated with a reduction in blood oxygen saturation. These pauses in breathing, called apneas (literally, without breath ), typically last 20 to 40 seconds. The individual with OSA is rarely aware of having difficulty breathing, even upon awakening. It is recognized as a problem by others witnessing the individual during episodes or is suspected because of its effects on the body (sequelae). Diagnosis of OSA is often based on a combination of patient history and tests (lab- or homebased). Given the recorded test data of patients, how to define good features to discriminate the OSA and non-osa subjects is a challenging and meaningful task, which could greatly help the Extracting Features to Discriminate OSA and non-osa 1

17 CHAPTER 1. INTRODUCTION process of the diagnosis of OSA. This project is expected to propose good algorithms that extract discriminative (OSA/non-OSA) features from the given data. 1.3 Related Work Several research work has been done with respect to validity of Acoustic Pharyngometry, the structure characteristics of upper airway of OSA patients and features extraction about discriminating OSA and non-osa subjects. [5] develops a standard operating protocol for Acoustic Pharyngometry that ensures a comprehensive repeatability of upper airway measurements. This paper gives the detailed specifications to the following aspects: equipment, subject selection criteria and operation procedure of Acoustic Pharyngometry. In addition, as another contribution of this paper, a detailed survey about the development of acoustic pharyngometry is made in this paper and some important issues about Acoustic Pharyngometry are discussed. [7]assesses the repeatability of pharyngeal cross-sectional area measurements obtained from normal and snoring individuals. The evaluation results show that repeatability of Acoustic Pharyngometry results can be achieved following the standard operating protocol. This conclusion adds to the reliability of Acoustic Pharyngometry in assessing the pharyngeal airway in patients with snoring and OSA. The genetic basis of upper airway size as determined is assessed in [12] using Acoustic Pharyngometry. It concludes that the minimum cross-sectional area exhibits a heritability of 0.34 in white subjects and 0.39 in African-Americans, which suggests that 30 40% of the total variance in this measure is explained by shared familial factors. This conclusion indicates that the minimum cross-sectional area of the oropharynx is a highly heritable trait, which suggests the presence of an underlying genetic basis and demonstrates the potential utility of Acoustic Pharyngometry in dissecting the genetic basis of OSA. [2] investigates the predictability and usefulness of Acoustic Pharyngometry in diagnosis of OSA. It concludes that the oropharyngeal junction point (OPJ) of the supine position is the most predictive parameter to discriminate OSA and non-osa. However, as shown in Section 7.1.1, the OPJ point is not a valued feature for our datasets, the accuracy is only about 60%. [6] uses the snorers with and without OSA to conduct the experiments to assess the acoustic pharyngometry patterns. It found 2 patterns for non-osa snorers and 3 patterns for OSA snorers. [8] concludes that MCA(Minimal Cross-sectional Airway Area) would differentiate between mild and moderate/severe OSA (apnoea-hypopnoea index [AHI], < 15 and 15 events/hour). MCA was shown to predict the presence of moderate to severe OSA, independent of age, sex, and neck circumference. 2 Extracting Features to Discriminate OSA and non-osa

18 Chapter 2 Domain Knowledge 2.1 Sleep Apnea Sleep apnea [14] is a sleep disorder characterized by abnormal pauses in breathing or instances of abnormally low breathing during sleep. Each pause in breathing, called an apnea, can last from at least ten seconds to minutes, and may occur 5 to 30 times or more an hour. Similarly, each abnormally low breathing event is called a hypopnea. There are three forms of sleep apnea: central (CSA), obstructive (OSA), and complex or mixed sleep apnea (i.e. a combination of central and obstructive) constituting 0.4%, 84%and15% of cases respectively [13]. In CSA, breathing is interrupted by a lack of respiratory effort; in OSA, breathing is interrupted by a physical block to airflow despite respiratory effort, and snoring is common Obstructive Sleep Apnea Obstructive sleep apnea (OSA) [11] is a sleep-related breathing disorder that involves a decrease or complete halt in airflow despite an ongoing effort to breathe. It occurs when the muscles relax during sleep, causing soft tissue in the back of the throat to collapse and block the upper airway. This leads to partial reductions (hypopneas) and complete pauses (apneas) in breathing that last at least 10 seconds during sleep. Most pauses last between 10 and 30 seconds, but some may persist for one minute or longer. This can lead to abrupt reductions in blood oxygen saturation, with oxygen levels falling as much as 40 percent or more in severe cases. The brain responds to the lack of oxygen by alerting the body, causing a brief arousal from sleep that restores normal breathing. This pattern can occur hundreds of times in one night. The result is a fragmented quality of sleep that often produces an excessive level of daytime sleepiness. Most people with OSA snore loudly and frequently, with periods of silence when airflow is reduced or blocked. They then make choking, snorting or gasping sounds when their airway reopens. A common measurement of sleep apnea is the apnea-hypopnea index (AHI). This is an average that represents the combined number of apneas and hypopneas that occur per hour of sleep. Types Mild OSA: AHI of 5-15 Involuntary sleepiness during activities that require little attention, such as watching TV or reading. Moderate OSA: AHI of Involuntary sleepiness during activities that require some attention, such as meetings or presentations. Extracting Features to Discriminate OSA and non-osa 3

19 CHAPTER 2. DOMAIN KNOWLEDGE Severe OSA: AHI of more than 30 Involuntary sleepiness during activities that require more active attention, such as talking or driving. However, in our case, we consider the subjects with AHI < 15 to be normal and the subjects with AHI 15 to be OSA patients. 2.2 Upper Airway Structure Using sagittal imaging [4],the upper airway has been subdivided into three regions, as shown in Figure 2.1: 1. naso-pharynx (region between the nasal turbinates and hard palate); 2. oropharynx, which can be subdivided into the retropalatal (defined from the level of the hard palate to the caudal margin of the soft palate; also called the velopharynx) and retroglossal (defined from the caudal margin of the soft palate to the base of the epiglottis) regions. In the majority of patients with sleep apnea, airway closure during sleep occurs in the retropalatal and retroglossal regions. 3. hypopharynx (region from the base of the tongue to the larynx). Figure 2.1: A: Midsagittal MR image in a normal subject demonstrating the upper airway regions [4]: (a) nasopharynx; (b)retropalatal; (c) retroglossal; (d) hypopharynx. B: Important sagittal upper airway structures demonstrated on MR imaging Anatomical Determinants of Upper Airway Caliber in OSA [1] Studies using nasal pharyngoscopy, computer tomography and magnetic resonance imaging, or pharyngeal pressure monitoring have shown that one or more sites within the oral pharayngeal region are usually where closure occurs in most subjects with OSA, and this region is also smaller in 4 Extracting Features to Discriminate OSA and non-osa

20 CHAPTER 2. DOMAIN KNOWLEDGE OSA patients versus controls even during wakefulness (see Figure2.2). Although the retropalatal region of the oropharynx is the most common site of collapse, airway narrowing is a dynamic process, varying markedly among and within subjects and often includes the retroglossal and hypopharyngeal areas. Figure 2.2: midsagittal magnetic resonance image (MRI) in a normal subject (left) and in a patient with severe OSA (right) [1]. Note that the upper airway is smaller, in both the retropalatal and retroglossal region. The recent use of quantitative imaging techniques has allowed advances that reveal important differences in both craniofacial and upper airway soft tissue structures in the OSA patient. The reduced size of cranial bony structures in the OSA patient include a reduced mandibular body length, inferior positioned hyoid bone, and retro position of the maxilla, all of which compromise the pharyngeal airspace. Airway length, from the top of the hard palate to the base of the epiglottis, is also increased in OSA patients, perhaps reflecting the increased proportion of collapsible airway exposed to collapsing pressures. As expected, these craniofacial dimensions are primarily inherited, as the relatives of OSA patients demonstrated retroposed and short mandibles and inferiorly placed hyoid bones, longer soft palates, wider uvulas, and higher narrower hard palates than matched controls. Enlargement of soft tissue structures both within and surrounding the airway contributes significantly to pharyngeal airway narrowing in most cases of OSA. An enlarged soft palate and tongue would encroach on airway diameter in the anterior-posterior plane, while the thickened pharyngeal walls would encroach in the lateral plane. Volumetric time overlapped magnetic resonance imaging (MRI) or computer tomography (CT) images strongly implicate the thickness of the lateral pharyngeal walls as a major site of airway compromise, as the airway is narrowed primarily in the lateral dimension in the majority of OSA patients. Furthermore, treatment with CPAP, weight loss, or mandibular advancement all show increases in the lateral pharyngeal dimensions. 2.3 Acoustic Pharyngometry Several researches have shown that the pharyngeal size and the dynamic behavior of the upper airway are important factors in the production of OSA. Hence assessment of the upper airway for possible sites of obstruction is crucial. The measurements of the upper airway are difficult because it is a geometrically complex structure subject to considerable variability. Acoustic pharyngometry is a useful tool for localizing the possible sites of upper airway obstruction in cases of OSA Rationale Reflections of acoustic pulse disturbances introduced at the mouth can be used to infer the crosssectional airway area of the oral cavity and pharyngeal spaces down to the level of the larynx. In Extracting Features to Discriminate OSA and non-osa 5

21 CHAPTER 2. DOMAIN KNOWLEDGE this technique, phase and amplitude information of the reflected sound wave can be transformed into an area-distance relationship Pros and Cons Compared to other methods for objectively evaluating the upper airway, Acoustic pharyngometry has many advantages. Advantage: Portability: easy to operate, free tidal breathing during measurement. Real time display of airway area. Lack of radiation involvement. Noninvasive. Ability to assess the entire airway simultaneously. Disadvantage: Cannot provide information about specific tissue structures that impinge upon the airway. Reduce accuracy compared to CT. 6 Extracting Features to Discriminate OSA and non-osa

22 Chapter 3 Workflow Figure 3.1 shows the workflow of the whole project. Given the datasets, we first conduct the preprocessing operations on it, including the removing noise and normalization in Chapter 4. Then we try several classical time series classification algorithms directly on the raw datasets without any feature extraction to check the performance, seen in Chapter 6, such as 1 Nearest Neighbor Algorithm with Euclidean distance and Dynamic Time Warping distance, Support Vector Machine algorithm, Logistic Regression algorithm and so on. As we will see in the later section, direct classifications on the raw datasets fail to discriminate OSA and non-osa effectively, the failed reason is that large part of the time series which usually corresponds to normal part of the throat is not predictive. Therefore, using the entire time series as the features for classification is not effective because normal part of the throat contributes a big factor to the value of the features. Hence, we have to extract valued features from the raw datasets, which is also the primary goal of this project. In this project, we propose about 16 features, which can be classified into two parts according to the feature characteristics: (1) local features in Section 7.1, which focus on some local points and local properties of the time series, such as OPJ point,maximum point, OPJ Value / Maximum Value and so on, and (2) global features in Section 7.2, which investigate the whole time series and try to extract some valued features from the global view, such as volume, stretched length, DFT -based features and so on. For each feature, we first present the algorithm specification, then conduct the all-sided experimental evaluation on the performance of the feature, finally we give the detailed explanation from the view of domain knowledge about OSA. In addition, we combine all the features (only single-value feature) we proposed together and conduct a comprehensive analysis on them in Section 7.3, including the correlation-based feature selection, information gain ranking, performance evaluation on combined features. Extracting Features to Discriminate OSA and non-osa 7

23 CHAPTER 3. WORKFLOW Figure 3.1: The workflow of thole project. 8 Extracting Features to Discriminate OSA and non-osa

24 Chapter 4 Datasets The data to be analyzed in this project is the recorded data from over 100 subjects(osa and non-osa). The data consists of graphs (cross-sectional area over distance from teeth) as shown in Figure 4.1. It can be considered as 1D imaging data of the upper airway. The values of the graphs are available in simple ASCII format. The features need to be extracted from these graphs. Figure 4.1: Cross-sectional area over distance from teeth. In the datasets, since the data for each subject is measured at uniform distance interval to the teeth, hence it can be considered as the time series, thus, the problem is actually a time series feature extraction problem. In this way, we can apply many popular time series feature extraction techniques to our problem. As shown in Table 4.1, there are totally 103 subjects in the datasets which consists of 63 OSA subjects and 40 non-osa subjects. For each subject, there are four time series measured in four different postures (called four dimensions in this report): supine breath, supine hold, sitting breath, sitting hold. Each time series contains 45 steps of points. total number OSA non-osa measurements time series steps dimensions(supine breath, supine hold, sitting breath, sitting hold) 45 Table 4.1: Datasets. Extracting Features to Discriminate OSA and non-osa 9

25 CHAPTER 4. DATASETS 4.1 Preprocessing of Datasets Two preprocess operations are conducted over the datasets before further analyzing them. We first remove the noise contained in the datasets, then apply the normalization to make all the time series in a uniform scale Remove Noise We develop a visualization tool for the datasets and check whether each of the time series follows the normal pattern as shown in Figure 4.2. A normal time series always goes through a peak which is corresponding to the oral cavity, then a local minimal point named OPJ (oropharyngeal junction) point which is corresponding to the first minimal point in the oropharynx reagion. If a time series is totally different from this normal pattern, we will remove it manually. E.g., the time series shown in Figure 4.3 is removed since it does not follow the normal pattern. Figure 4.2: A normal time series. Figure 4.3: This time series does not follow the normal pattern Normalization As we known, though the basic structure of the upper airway is almost same between different persons, there are still some subtle differences, for example, some more obese and taller persons are prone to possess bigger size of upper airway than the thinner and lower persons. In order to avoid such impact, we apply the normalization technique to all the time series in the datasets. In our project, y-scale normalization is applied to the datasets, which is indicated in Equation 4.1. For each point in the time series, y value is divided by the maximum value max value in the whole time series. 10 Extracting Features to Discriminate OSA and non-osa

26 CHAPTER 4. DATASETS In this way, the y-axis value is scaled into the range [0, 1] while keeping the shape of the time series. The normalization operation makes two series indistinguishable, provided they are proportional to one another, i.e., a i = λb i for all i. Application of Normalization y = y max value We apply the normalization to some of our features, but not all. Since not all features need the normalization. For instance, the feature OPJ Value adopts the original value but not the normalized value. The normalized OPJ Value is equivalent to the feature OPJ value / maximum value. With respect to feature Volume, we tried both the normalized volume and non-normalized volume and find that non-normalized volume performs much better than normalized volume, which indicates that the actual value could better represent the difference between OSA and non-osa than the normalized value w.r.t. volume. One possible explanation is that if the OSA affect the whole upper airway, then the crosssectional area will be decreased in whole part of the upper airway, then the y-scale normalization would make this decrease invisible. In some cases, the normalization is necessary, for instance, the features concerned with DTW distance calculation need the normalization to make the two time series align more accurately. (4.1) Extracting Features to Discriminate OSA and non-osa 11

27 Chapter 5 Performance Criteria In this section, we introduce briefly several performance criteria that we use in this project. 5.1 Some Basic Performance Criteria k-fold Cross-Validation Cross-Validation is a statistical method of evaluating and comparing learning algorithms by dividing data into two segments: one used to learn or train a model and the other used to validate the model. In typical cross-validation, the training and validation sets must cross-over in successive rounds such that each data point has a chance of being validated against. The basic form of cross-validation is k-fold cross-validation. In k-fold cross-validation the data is first partitioned into k equally (or nearly equally) sized segments or folds. Subsequently k iterations of training and validation are performed such that within each iteration a different fold of the data is held-out for validation while the remaining k 1 folds are used for learning. The validation results are averaged over the k rounds. The biggest advantage of cross validation is to avoid the overfitting of predictive model to the datasets. In data mining and machine learning 10-fold cross-validation (k= 10) is the most common. In our experiments, we also apply 10-fold cross validation to evaluate the discriminative performance of feature Classification Terminology Suppose we have an experiment with the datasets which contains P positive instances and N negative instances. The four outcomes can be formulated in a 2 confusion matrix, as shown in Figure 5.1. sensitivity = true positive rate(tpr) = recall = hit rate T P R = T P T P + F N (5.1) specificity = true negative rate specif icity = T N T N + F P (5.2) precision = positive predictive value precision = T P T P + F P (5.3) 12 Extracting Features to Discriminate OSA and non-osa

28 CHAPTER 5. PERFORMANCE CRITERIA accuracy accuracy = T P + T N T P + F N + F P + T N (5.4) Figure 5.1: Confusion matrix ROC and AUC A receiver operating characteristic (ROC), or simply ROC curve, is a graphical plot which illustrates the performance of a binary classifier system as its discrimination threshold is varied. It is created by plotting the fraction of true positives out of the positives (TPR = true positive rate) vs. the fraction of false positives out of the negatives (FPR = false positive rate), at various threshold settings. TPR is also known as sensitivity (also called recall in some fields), and FPR is one minus the specificity or true negative rate. The best possible prediction method would yield a point in the upper left corner or coordinate (0,1) of the ROC space, representing 100% sensitivity (no false negatives) and 100% specificity (no false positives). The (0,1) point is also called a perfect classification. A completely random guess would give a point along a diagonal line (the so-called line of no-discrimination) from the left bottom to the top right corners. As shown in Figure 5.2, the algorithm represented by point A is better than B and C while the algorithm represented by B is equal to random guessing. AUC means the area under the curve, it ranges in [0, 1]. Higher AUC indicates a better performance of the classifier MCC The Matthews Correlation Coefficient (MCC) is used to evaluate the performance of binary classifications. It can be calculated from confusion matrix in Figure 5.1, as shown in Equation 5.5. MCC ranges from -1 to +1, +1 represents a perfect prediction, 0 is equal to random prediction and -1 means total disagreement between prediction and observation. MCC = T P T N F P F N (T P + F P )(T P + F N)(T N + F P )(T N + F N) (5.5) T-test Student s t-test is used to determine whether two sets of data are significantly different from each other. the premise of the t-test is that these two sets of data must follow normal distribution. Extracting Features to Discriminate OSA and non-osa 13

29 CHAPTER 5. PERFORMANCE CRITERIA Figure 5.2: Roc space (quoted from Wikipedia). In our experiments, since the size of OSA (63) is not equal to non-osa (40), and the variances are always not equal, hence we should apply the t-test as shown in Equation 5.6, where x and ȳ is the mean value of the sets, s x and s y is the standard deviations, n and m are the size of sets. h-value returned by T-test will be 1 if there is significant difference between two sets, which means a rejection of the null hypothesis that there are no significant difference between two sets. t = x ȳ s 2 x n + s2 y m (5.6) 5.2 Classifier based on Information Gain and Optimal Split Point Our algorithms need some metric to evaluate how well a feature can divide the entire combined datasets into two original classes, i.e., the discriminative power. In this project, for some features, we use Information Gain to find the Optimal Split Point(OSP). Then we can apply Decision Tree model to evaluate the performance for this feature based on the Optimal Split Point. The related definitions are defined as follows. Definition 1 (Entropy). A time series dataset D consists of two classes, A and B. Given that the proportion of objects in class A is p(a) and the proportion of objects in class B is p(b), the entropy of D is: I(D) = p(a) log(p(a)) p(b) log(p(b)). Given the value of a feature for all instances, each splitting point divides the whole dataset D into two subsets, D 1 and D 2. Therefore, the information remaining in the entire dataset after splitting is defined by the weighted average entropy of each subset. Suppose the fraction of objects in D 1 and D 2 are f(d 1 ) and f(d 2 ) respectively. The total entropy of D after splitting is Î(D) = f(d 1 )I(D 1 ) + f(d 2 )I(D 2 ) (5.7) Then the information gain for any splitting strategy is defined as: Definition 2 (Information Gain). Suppose a certain split point sp divides D into two subsets D 1 and D 2, the entropy before and after splitting is I(D) and Î(D), then the information gain for 14 Extracting Features to Discriminate OSA and non-osa

30 CHAPTER 5. PERFORMANCE CRITERIA Algorithm 1 Feature-Evaluation-Algorithm(F, D train, D test ) 1: Input: Feature value F for each subject in dataset D, the training set D train and the test set D test 2: Output: Classification accuracy A with this feature 3: Set 1 4: Set 2 5: OSP CalculateOptimalSplitP oint(d train ) // calculate the OSP in the training dataset 6: for each subject s in D test do 7: if F (s) < OSP then 8: Set 1 Set 1 s 9: else 10: Set 2 Set 2 s 11: end if 12: end for 13: A CalculateAccuracy(Set 1, Set 2, D test ) // check the similarity with the real value of label in the test set D test 14: Return A this splitting point is Gain(sp) = I(D) Î(D) (5.8) Given the value of each subject in the dataset for a given feature, to evaluate the discriminative power of the feature, we sort the subjects according to the its feature value and find an optimal split point between two neighboring feature value based on Information Gain. Definition 3 (Optimal Split Point). Our time series dataset D consists of two classes: OSA and non-osa. For a given feature F, we choose some threshold V th and split D into D 1 and D 2, such that for every time series subject T 1,i in D 1, F eaturev alue(t 1,i ) < V th and for every time series subject T 2,i in D 2, F eaturev alue(t 2,i ) > V th. An Optimal Split Point is a threshold that for any other threshold V th Classifier based on OSP Gain(F, V OSP (D,F ) ) Gain(F, V th) (5.9) For some single-value feature we propose, we use OSP and information gain to evaluate its discriminative power. Specifically, given a feature F, we first calculate the corresponding OSP Optimal Split Point, then calculate the accuracy based on the Decision Tree model with OSP, as shown in Algorithm 1.In practice, we use k-folds cross-validation technique to calculate the accuracy to avoid the overfitting for the datasets. Extracting Features to Discriminate OSA and non-osa 15

31 Chapter 6 Classification with Raw Data After preprocessing operations on the datasets, we first try to apply some classical time series classification algorithms directly on the raw data, i.e., we use all the 45 points in the time series as the input features of the classifier, to check the prediction performance. As one of the most popular time series classification algorithm, 1-Nearest Neighbor (1-NN) Algorithm with Euclidean distance are effective in most time series problem, hence we first apply this algorithm to our datasets. However, it performs poor on our data for the reasons explained later. Then in order to remove the shifting between different time series and align them together, we replace the Euclidean distance with Dynamic Time Warping distance in 1-NN algorithm. Finally, several popular classification models are applied to our datasets, taking the entire time series as input features. However, they all performs poor for the reason we explained in detail in the following subsection. 16 Extracting Features to Discriminate OSA and non-osa

32 CHAPTER 6. CLASSIFICATION WITH RAW DATA Nearest Neighbor Algorithm with Euclidean Distance 1-Nearest Neighbor (1-NN) algorithm is a one of most classical time series classification algorithm, recent empirical evidence has strongly suggested that the simple nearest neighbor algorithm is very difficult to beat for most time series problems. The biggest advantage of this algorithm is its simplicity of implementing. Hence, I first apply the 1-NN algorithm with Euclidean distance to our data to check the performance Algorithm Algorithm 2 shows the 1-NN algorithm process with Euclidean Distance. It works very simply: predict the label of the unknown object by the label of the object with the nearest Euclidean distance in the training set. Algorithm 2 1-NN Euclidean Distance (T rain set, T rain label, unknown object) 1: Input: T rain set, T rain label, unknown object //T rain set is the training set and T rain label indicates the label for each object in T rain set, the goal is to predict the label of the unknown object. 2: Output: the label predicted class of the unknown object 3: //initialization 4: best so far inf; 5: for i = 1 size(t rain set) do 6: distance sqrt(sum(t rain set(i) unknown object) 2 ) // calculate the Euclidean distance 7: if distance < best so far then 8: predicted class T rain label(i) 9: best so far distance 10: end if 11: end for 12: Return predicted class Experimental Evaluation In order to avoid the overfitting, we apply the 1-NN algorithm with Euclidean Distance to our datasets with 10-fold cross validation. Table 6.1 shows the experimental result. We can find that the accuracy is between 55% to 63%, which is not good enough. The AUC is around 0.5 and MCC is around 0.000, which indicates it is almost equivalent to random guess. Supine Breath Supine Hold Sitting Breath Sitting Hold Accuracy 56.3% 55.3% 57.3% 62.1% AUC MCC Precision Recall TP FP FN TN Table 6.1: Classification results of 1-NN algorithm with Euclidean Distance with 10-fold cross validation. Extracting Features to Discriminate OSA and non-osa 17

33 CHAPTER 6. CLASSIFICATION WITH RAW DATA Analysis of Failure There are two possible factors that lead to failure with 1-NN algorithm with Euclidean distance: It is observed that there is some shifting between one time series and another, i.e., the time series are not aligned well between each other. Calculating distance with entire time series is not effective since large part of the time series which is corresponding to normal part of the throat are not predictive but this part contribute a big factor to the calculation Conclusion From the evaluation results based on our datasets, we can conclude that 1-NN algorithm with Euclidean Distance is not feasible to our datasets. All kinds of quantitive indicators show that it is no better than random guess. 18 Extracting Features to Discriminate OSA and non-osa

34 CHAPTER 6. CLASSIFICATION WITH RAW DATA Nearest Neighbor Algorithm with DTW(Dynamic Time Warping) Distance Dynamic time warping (DTW) is a well-known technique to find an optimal alignment between two given (time-dependent) sequences under certain restrictions (Figure 6.1). Intuitively, the sequences are warped in a nonlinear fashion to match each other. In fields such as data mining and information retrieval, DTW has been successfully applied to automatically cope with time deformations and different speeds associated with time-dependent data and measure similarity between two sequences which may vary in time or speed. Figure 6.1: Time alignment of two time series. Since in our datasets, there may be some shifting along x-axis between two time series, hence DTW algorithm can be used to remove the offset and align them along x-axis Algorithm We use DTW to calculate the distance between two time series instead of Euclidean distance in line 6 in Algorithm 2. Algorithm 3 shows the DTW distance calculation algorithm using dynamic programming method. In order to make the alignment more precisely, y-scale normalization is applied to the data first. Algorithm 3 DTW Distance calculation algorithm (s : array[1..n], t : array[1..m], w : int) 1: Input: s : array[1..n], t : array[1..m], w : int //s and t are two time series with length of m and n, w is the window parameter indicating the constraint that two aligned points can only be in a same window. 2: Output: the DTW distance between s and t 3: //initialization 4: distance array[0..n, 0..m] 5: w max(w, abs(n m)) 6: for i = 0 n do 7: for j = 0 m do 8: distance[i, j] inf inity 9: end for 10: end for 11: distance[0, 0] 0 12: for i = 1 n do 13: for j = max(1, i w) min(m, i + w) do 14: cost d(s[i], t[j]) 15: distance cost + minimum(distance[i 1, j], distance[i, j 1], distance[i 1, j 1]) 16: end for 17: end for 18: Return distance[n, m] Extracting Features to Discriminate OSA and non-osa 19