Vital-Sign Monitoring Using Pulse Oximetry for Automated Triage and the Prediction of Patient Deterioration

Transcription

1 Vital-Sign Monitoring Using Pulse Oximetry for Automated Triage and the Prediction of Patient Deterioration John Prince Wolfson College University of Oxford Transfer of Status for Doctor of Philosophy Trinity 2016 Supervised by: Prof. Maarten de Vos Prof. David Clifton

2 Contents 1 Introduction and Clinical Need Overview Background Current Approaches to Paediatric Triage Current Approaches to EWS Respiration Rate Derivation from Pulse Oximetry The Paediatric Triage Study Introduction Paediatric Triage Study Patient Selection Logistic Regression Results of the PTS Analysis Matching Triage Decision Matching Admission Decision Discussion The CALMS-2 Database Introduction CALMS-2 Patient Selection Kernel Density Estimation Results of the CALMS-2 Analysis D Observational Novelty Model D Continuous Novelty Model D Continuous Novelty Model Model Comparison Discussion Conclusions and Future DPhil Work Conclusions Future Work ii

3 1 Introduction and Clinical Need 1.1 Overview The pulse oximeter has been a popular diagnostic tool in primary and secondary care for over three decades [1]. Its primary use is in measuring the percentage of oxygenated haemoglobin in the blood, a measurement highly correlated with respiratory health. Its use is widely adopted in assessing acute and chronic illnesses and, as a non-invasive tool, it is popular in paediatric care [2]. In secondary care, blood oxygen saturation is used extensively in intensive care units to guide the use of oxygen therapy and as a means of continuously monitoring high risk patients when at the bedside or ambulatory [3, 4]. In primary care, pulse oximetry can be used as a spot check for chronic obstructive pulmonary disease, asthma and community-acquired pneumonia, although the use for these purposes is highly variable amongst general practitioners [5]. The purpose of this research is to provide evidence that the diagnostic ability of pulse oximetry can not only be improved in its current clinical settings, but also adapted to be used in new and resource-poor settings. The current clinical uses of pulse oximetry under investigation in this research can be split into two areas; the emergency department and the post-surgical surveillance ward. Firstly, when a patient presents to an emergency department, a triage nurse will determine the urgency at which the patient needs to be seen by a clinician based on a series of clinical signs and vital sign measurements. Triage is the practice of prioritizing patients for treatment based on the severity of their state of health. Of the vital signs used in most triage systems, heart rate, respiration rate and blood oxygen saturation contribute significantly to the triage decision [6]. One aspect of this research is to automate the triage process via creating a predictive algorithm capable of matching the decision of the triage nurse using vital signs derived from pulse oximetry. Secondly, after undergoing major surgery, patients are at a high risk of experiencing an adverse event such as a cardiac arrest, a mortality event or requiring unplanned admission to the intensive care unit. At present, nurses periodically assign Track-and-Trigger (T&T) scores to patients 1

4 1. Introduction and Clinical Need 2 based on their vital signs. This research aims to improve the T&T system via investigating the benefit of using continuous pulse oximetry monitoring to provide continuous T&T scores which may allow earlier detection of patient deterioration. However, both of these current clinical uses are in developed countries where pulse oximetry and other diagnostic tools are widespread. Conversely, in developing countries many health facilities are resource poor and under-staffed. The third and final aspect of this research aims to investigate the role of pulse oximetry as a rapid and affordable triage tool in resource-poor settings. The United Nations Millennium Development Goal 4 (MDG-4) aimed to reduce the global under-five mortality rate by two-thirds between 1990 and 2015 [7]. Although the under-five mortality rate has decreased significantly, the goal was not fully achieved and the World Health Organisation (WHO) are now calling for new sustainable development goals to continue reducing child mortality with focus being on pneumonia, malaria and diarrhoea [8, 9]. In low-income settings, particularly in rural areas and paediatric care, there are often no formal triage systems being implemented leading to patients being seen in the order that they arrive at a hospital outpatient ward and thus often receive inappropriate treatment [10, 11]. Although initial patient triage is critical to ensuring timely treatment, and is a key part of training efforts to improve care, little work is being undertaken to improve triage and anecdotal observations suggest triage systems are often weak with staff unable to devote sufficient time to evaluation of each walk-in patient[12, 13]. However, some initiatives have been taken to improve upon these weaknesses in low-income healthcare environments. Emergency Triage Assessment and Treatment (ETAT [14]) was developed by the WHO and has since been implemented in many countries (including Niger, Malawi, Brazil, and Kenya) and provides a set of guidelines for rapid and effective triage [15, 16]. The ETAT triage system was developed purely based on 12 clinical signs including patient age, temperature, visible respiratory distress and malnutrition. The effectiveness of the ETAT system has been described in several studies showing a decrease in inpatient mortality [15, 16]. Although the ETAT system often benefits from its simplicity in only making use of clinical signs, this can also contribute to increasing the operational strain put upon the person performing the triage leading to many signs not being assessed [17]. The problem of assessing 12 clinical signs by a single staff member is amplified when combined with the severe staff shortages with half the countries in sub-saharan Africa having less than 1 health worker per 5000 of the population [18].

5 1. Introduction and Clinical Need 3 It can therefore be proposed that a technology capable of performing triage whilst simultaneously measuring and reporting vital signs would be a beneficial component to a health-facility s triage procedure. This research investigates whether a technological approach utilising signal processing and machine learning can predict the triage and admission decisions made by a clinician. Currently, pulse oximetry is either not present or not routinely used in many hospitals across Sub-Saharan Africa [19]. However, pulse oximetry allows the measurement of multiple vital signs (heart rate, blood oxygen saturation, and respiration rate) from a single, non-invasive sensor through signal processing of the signal it produces, the photoplethysmogram (PPG). Heart rate increases and decreases periodically during a respiration cycle thus allowing this variation to be automatically detected and extracted with signal processing revealing the respiration rate [20]. Although it is has been proven and it is widely accepted that respiration rate can be derived from the use of pulse oximetry [21 25], there is still the need to validate the robustness of current algorithms on more challenging data; such as that acquired from paediatric patients in low-income settings [24, 26]. The immediate benefit of deriving respiration rate from the PPG is that it eliminates the need for additional, and more expensive, methods for measuring respiration rate such as ECG impedance pneumography, capnography or manual counting which require additional technology and, in the latter case, can be subject to human error. The vital signs capable of being extracted from the PPG have proven to be sensitive predictors of outcomes for diseases such as pneumonia [27]. Studies investigating the effect of pulse oximeters in low-income settings suggest that their use improve mortality rates and shortens length of hospital stay whilst their use in triage in developed countries proved to detect significantly more cases of hypoxia than if just clinical signs were to have been used [28]. Consequently, due to the sparse use of pulse oximetry in low-income settings and the recent acknowledgement of the capabilities of deriving respiration rate from the PPG, there have been very few studies looking at the use of such a system in performing triage [29]. It is well described that pneumonia, malaria, dehydration, and meningitis are the most common diagnoses amongst paediatric admissions in Kenya and studies have shown that these diseases are often misclassified for severity [30]. The Clinical Information Network (CIN) study is present in 14 health facilities across Kenya and aims to improve the collection of routine mortality and morbidity data in an attempt to determine whether better information standards improve quality of care [31].

6 1. Introduction and Clinical Need 4 Further recent analyses (unpublished) using CIN data have indicated that many vital signs measurements are either missed or likely to be crude estimates rather than accurate measurements, including respiration rate [32]. One explanation of these crude estimates is that many hospitals in Kenya have a low nurse-to-patient ratio meaning that nurses are unable to spend an appropriate length of time assessing each patient individually [33]. This research introduces the use of pulse oximetry for automatically providing an estimate of patients health condition. In Chapter 2, a background to the current triage systems used in developed and developing countries is provided. The similarity between these triage systems and the T&T systems used to assess patients who have been admitted to hospital is highlighted as a means of expanding the literature to include more machine learning applications. Chapter 2 concludes by providing a background to the current state-of-the-art signal processing techniques for deriving respiratory rate from the PPG. Chapter 3 presents the Paediatric Triage Study dataset along with the methods and results when using pulse oximetry to automatically predict triage decisions. Although the ultimate aim of this research is to automate paediatric triage in resource-poor settings, due to the lengthy ethics application process, available datasets are limited. For this reason, analysis was expanded to investigate the benefit of using pulse oximetry for risk stratification in a developed clinical setting. Chapter 4 describes the CALMS-2 dataset along with the methods and results when using continuous pulse oximetry to predict patient deterioration. Finally, Chapter 5 provides the plans for future work, including a detailed description of the proposed internally validated prognosis study to be undertaken at a paediatric hospital in Nairobi, Kenya.

7 2 Background 2.1 Current Approaches to Paediatric Triage In 2013, roughly 6.3 million children died before the age of five years old with 95% of these occurring in developing countries [34]. The MDG-4, as set out by the WHO, has contributed to significantly reducing the under five mortality rate by an estimated 59% between 2000 and 2015 [35]. This value however is far from consistent across the targeted countries with some regions (Eastern Asia, Latin America and Northern Africa) now reporting a under five mortality rate of <30 per 1000 live births whereas other regions, such as Sub-Saharan Africa, continue to have a very high rate of 92 per 1000 live births - contributing nearly half of the 6.3 million under 5 deaths [36]. Furthermore, pneumonia is the leading cause of mortality in children under five years old and if correctly diagnosed can be treated with a course of antibiotics. Expanding the availability of antibiotics for diseases such as pneumonia has been highlighted as a key factor in contributing to reducing the under five mortality rate throughout MDG-4 [37]. Attention must then be turned to not just being able to treat a disease such as pneumonia, but how to initially diagnose such a disease. In developing and developed countries alike, overcrowding in emergency departments poses a serious risk to patient safety [38]. The primary purpose of triage is to rapidly assess and stratify patients according to their current condition in order to optimise level of care whilst simultaneously optimising the available facilities. Triage is most commonly used in emergency care, such as in accident and emergency (A&E) departments, ambulances, or war zones. However, over the last decade triage has become more popular in primary health settings such as GP clinics, Community Healthy Workers (CHW), and telephone triage systems such as NHS Direct and, most recently, NHS 111. Although some practices and standards have been suggested in the UK, there is no universalgold standard practice for triage, leading to a variety of triage techniques existing. The common skeleton that many of these techniques share is using a pre-defined chart of healthy vital signs 5

8 2. Background 6 to compare against the vitals signs measured from a patient when they present to an emergency department. It is difficult to say exactly how many triage systems are being implemented in the U.K. alone due to inconsistency of triage systems between each care trust. As a brief example, Brighton, Bristol, and Cardiff & Vale all have their own variation of a paediatric early warning score (PEWS) system varying in which vital signs they believe to be predictive of paediatric deterioration [6, 39, 40]. An alternative yet similar triage system, the paediatric advanced warning score (PAWS) [41], has recently shown that an elevated PAWS score corresponds poorly with predicting deterioration when tested on 140 patients in Leeds Children s Hospital, UK, yielding a sensitivity and specificity of 56% and 76% respectively leading to uncertainty of the effectiveness of the current triage systems [42]. These triage systems are intended for use when a child first presents to an emergency department. However, as will be discussed in Section 2.2, the same methodology for assessing patients who have already been admitted to hospital is applied. This usually entails periodically assigning an Early Warning Score (EWS) based on patients vital signs when compared to a table of pre-defined healthy vital signs. Several triage strategies have reached the international scale in developed countries including the Austrlian triage scale (ATS), the Canadian emergency department triage scale (CTAS), and the Manchester triage scale (MTS), which is widely used throughout Europe. However, all of these triage systems require multiple vital signs to be measured including heart rate, respiration rate, temperature, blood oxygen saturation and blood pressure. In developed countries the technologies for automatically measuring these vital signs (ECG, ECG pneumography, oral or rectal thermometer, pulse oximeter and sphygmomanometer) quickly and accurately are widely available whereas in resource-poor settings these measurements are measured manually or not measured at all. For this reason, the WHO established a set of triage guidelines called the Emergency Triage Assessment and Treatment (ETAT) system which was specifically designed for paediatric triage in resource-poor environments and the ETAT system has been linked with the current acceleration in reducing the under five mortality rate in highly vulnerable countries [14, 17, 43]. The ETAT system provides a paper-based framework for triage and immediate treatment for severely ill children in resource poor environments and generally shows a good performance when matching the decisions of a paediatrician.

9 2. Background 7 The ETAT system classifies children into three categories based on the prevalence of either an emergency sign, a priority sign or neither requiring immediate care, prompt assessment or waiting for care respectively. The emergency signs focus on the detection of diseases causing severe respiratory distress such as acute lower respiratory infections, including pneumonia. This is achieved via an assessment of the airway and breathing followed by an assessment of circulation, coma, convulsing, and severe dehydration. The first performance assessment of the ETAT system was carried out on a cohort of 3837 children aged below five years who presented to the A&E department at a large hospital in Brazil [44]. The ETAT system achieved a sensitivity of 85.7% with respect to identifying all patients who were subsequently admitted with a specificity of 85.2% when compared to the corresponding assessment made by a paediatrician. Although the nurses implementing the ETAT system in this study had only received 20 hours of ETAT training, they also had several years of nurse training likely making them more attuned to detecting severely ill children thus likely biasing the effectiveness of the ETAT system. Furthermore, the study notes a lack of children presenting with respiratory distress and malaria and not identifying respiratory distress contributed to half of the false negatives. As the ETAT system is intended to be easily implemented and effective across a wide range of diseases, by nurses with little experience and within varying facilities, its validation has been sought through many subsequent studies. Of 2281 children under five who presented to a hospital in Malawi, the ETAT system detected 85% of children who were subsequently admitted by a physician [17]. As there is no gold standard in the practice of triage, in either developed or developing countries, it is difficult to quantify the performance of the ETAT system in a quantifiable or reproducible manner - this is further complicated by the often subjective assessment made during triage such as Lethargy and Irritable which are included in the ETAT system. The South African Triage Scoring (SATS) system was developed in 2006 and was the first nation wide triage system implemented by South Africa [45]. The SATS system differs from the previous triage systems as it also incorporates the presence of trauma parameters into the final triage score. The performance of SATS was compared to a modified version of the ETAT system (which also included weight, temperature and blood oxygen saturation measurements) and combined to form a single triage system which was found to reduce the undertriage rate of the original SATS system from 24% to 12% [46].

10 2. Background 8 Very few studies have investigated the potential role of machine learning in the field of automated triage in resource-poor environments. The performance of using a support vector machine (SVM) classifier to predict patients who presented to an emergency department who would subsequently have a cardiac arrest or a mortality event within the next 72 hours was compared to the performance of the modified early warning score (MEWS) [47]. 925 patients were enrolled in this study and it was found that via using heart rate variability in combination with heart rate, respiration rate, temperature and systolic blood pressure the SVM classifier outperformed the MEWS approach with an Area Under Curve (AUC) of against respectively. The study has the significant limitations that it was performed only on adults (mean age of 61) and only to predict a very severe condition. A recent internally validated study implemented a multivariate logistic regression model to predict hospital admission for 3374 children under five years old in Bangladesh [48]. Although this study made use of a pulse oximeter for blood oxygen saturation measurement, this measurement was hidden from the physicians as pulse oximetry is not currently in use in the facility and thus would have biased their admission decisions. Furthermore, the pulse oximeter in this case was only used to measure HR and S p 0 2 and not respiration rate. This study did not attempt to predict any triage decisions made by the physicians with the primary outcome being to predict hospital admission. Using oxygen saturation, respiratory rate and temperature alongside the readily available clinical signs and symptoms the logistic model was found to have a sensitivity of 76.9% and a specificity of 87.3%. This study requires an external validation stage prior to being considered to be used in clinical practise. 2.2 Current Approaches to EWS Although there is a lack of studies in applying machine learning techniques to triage in emergency departments, much work has been done in stratifying patients conditions once already admitted to hospital. As discussed briefly in the previous section, once a patient has been admitted to hospital they are periodically assigned an EWS based on their vital signs when compared to a pre-defined table of healthy vital signs. An example of such a table can be seen in Table 2.1. These EWS tables are very similar to the tables used to perform triage in emergency departments. We can therefore consider the machine learning techniques having previously been used to automatically assign EWSs to potentially be useful to automatically perform triage.

11 2. Background 9 Table 2.1: An example of an EWS table - The Oxford EWS Score Variable HR [bpm] RR[breaths/min] Sp02 [%] > Temperature [0C] SBP [mmhg] AVPU Scale A V - P, U EWSs are primarily used to detect patients who are showing signs of deterioration once already admitted to the hospital. Via detection of deterioration clinical staff may be provided with sufficient time to intervene so as to prevent an adverse event such as a cardiac arrest, unplanned admittance to the ICU or a mortality event. As a member of the clinical staff performs routine observations on a patient, their measurements feed into the EWS system. An alternative name for EWS systems is a Track-and-Trigger (T&T) system. If the EWS exceeds a certain threshold then in theory the patient is flagged as showing abnormal vital signs and their care is reassessed by the clinical staff. However, in reality EWS systems place a greater importance on sensitivity than specificity. This results in many false alarms often creating so called alarm fatigue, thus diluting the importance of each alarm which runs the risk of the clinical staff assuming a true alarm to be the boy who cried wolf. As with triage systems, there is a vast choice of EWS systems within the literature and within clinical use. Smith et al. performed an evaluation on 33 T&T systems showing that on 9987 patients admitted the range of AUC was to [49]. All 33 of these EWS systems measured heart rate, respiration rate, systolic blood pressure, and Alert-Verbal-Painful- Unresponsive (AVPU) whereas only 26 measured temperature and only 6 measured blood oxygen saturation suggesting the previous literature had suggested blood oxygen saturation a relatively unimportant parameter when predicting deterioration. However, of the 33 EWS systems evaluated the best system was that proposed by Bakir et al., which utilises both temperature and blood oxygen saturation [50]. The major limitations of the T&T systems is that they only provide sparse periodic estimates of a patients health condition - roughly once every few hours when the nurse performs routine measurements. However, as many of these patients are also connected to bedside monitors recent EWS approaches have utilised machine learning techniques to the continuous bedside monitors

12 2. Background 10 to improve the temporal resolution of estimates of a patient s state of health. Furthermore, most bedside monitors already include some form of alert system but tend to be based on a single-channel framework which uses a fixed threshold set for each vital sign. Once more, this results in a high false alarm rate causing an inconvenience to clinical staff and often not being an accurate representation of a patient s true state [51]. A variety of machine learning methodologies have been proposed to improve the clinical usefulness of both observational and continuous patient monitoring including logistic regression, support vector machines (SVM), neural networks (NN), Kalman filtering, kernel density estimation (KDE), and Gaussian processes [52 60]. Saria et al. used logistic regression to predict the probability of preterm infants risk of a severe morbidity event [54]. Three hours of non-invasive measurements (heart rate, respiration rate and blood oxygen saturation) were collected during the first three hours of life for 138 infants. This study elegantly falls in-between triage and EWS as the birth of a preterm child is analogous to a child presenting to an emergency department. When only the collected vital signs are included in the model an AUC of 0.85 is found - which was improved to 0.92 when the gestational age and weight were subsequently included in the model. Also included in the model were short and long term variability features of the heart and respiration rates. It was found that short term variability contributed significantly to the final result whereas long term variability did not. This observation could be considered as further strengthening the argument that continuous EWSs are better than intermittent EWSs as they are capable of measuring short term variability. An alternative use of logistic regression was used by Hackmann et al. [52, 53]. when predicting the likelihood of ICU admittance of 19,116 hospital patients. Manual observations were combined with heart rate and blood oxygen saturation measured by a pulse oximeter to act as independent variables in the logistic regression model. Of the 10 variables with the highest coefficients in the model, six related to heart rate, respiration rate and blood oxygen saturation levels (including their maximum, minimum and mean values). Here specificity was fixed at 95% leading to a sensitivity of 49% whereas if specificity is not fixed, a sensitivity of roughly 80% and a specificity of 85% is achieved. Both the uses of logistic regression have highlighted heart rate, respiration rate and blood oxygen saturation as significant predictors of patient deterioration. The use of NN, SVMs, Kalman filtering and KDE in this context fall under the umbrella of novelty detection. There are many approaches to novelty detection including probabilistic, distance-based, reconstruction-based and domain-based and, in its simplest form, novelty

13 2. Background 11 detection can be defined as the task of recognising that test data differ in some respect from the data that are available during training. [61]. Novelty detection is commonly employed when there is a severe inbalance of data from each class. In the application of NN and Kalman filtering the model is trained to produce a label of patients vitals signs as either normal or abnormal whereas in KDE a simpler approach is taken in which only a normal class is defined. The limitation of using NN and Kalman filtering in this case is that the ratio of normal to abnormal data is severely inbalanced therefore making it difficult to confidently train a model on the somewhat limited amount of abnormal data - a limitation overcome by the KDE approach. The primary limitation of novelty detection methods is its constraint to being a one-class classifier. In the context of this research, this may be beneficial for predicting admission decision but in order to predict triage categories, and depending on the novelty detection method utilised, the results will require post-processing. 2.3 Respiration Rate Derivation from Pulse Oximetry The purpose of this research is to utilise signal processing and machine learning techniques on the PPG to perform automatic triage. Accordingly, a comprehensive literature review of the current state-of-the-art signal processing techniques utilised on the PPG was performed and a brief description of the key methods and the underlying physiology is described in this section [62]. All commercial pulse oximeters currently provide an estimate of heart rate and blood oxygen saturation level and the techniques used to extract these measurements are well established and will therefore not be covered in this literature review. Respiration Rate Estimation Methods Many signal processing techniques have been developed for estimating RR from the PPG waveform including the use of digital filtering [23, 63], Fourier transforms [24, 64], wavelet decomposition [21, 65 67], autoregressive modelling [22, 68, 69], and Gaussian processes [70]. The application of digital bandpass filters are beneficial as they allow the removal of the cardiac component of the PPG waveform [23, 63]. However, many studies applying filtering techniques are constrained by the fact they only utilise high-quality PPG data obtained from healthy volunteers, and discard recordings from patients who provided poor-quality data. Accordingly, the effectiveness and robustness of these digital filtering techniques is questionable.

14 2. Background 12 The use of Fourier transforms as a standalone method to derive RR from raw PPG waveforms has shown to be effective. When inspecting the frequency content of a PPG signal, there should be two distinct peaks; one being at a low frequency corresponding to respiration rate, and one at a higher frequency corresponding to the heart rate [71]. The FFT has also proven successful when used in conjunction with digital filtering [72]. The use of time-frequency methods to estimate RR from the PPG has been extensively demonstrated [65 67]. The CWT can successfully determine RR from clean PPG waveforms but, in many cases, requires the results to be manually selected via comparison with the reference signal, thus lacking clinical usability. The use of autoregressive (AR) models has accurately shown to be capable of estimating RR [22, 68, 69]. AR modelling is an alternative for spectral analysis and assumes that each value in a time series may be defined by a weighted sum of previous values: M x[n] = a i x[n i] + ɛ[n] (2.1) i=1 where x[n] is the current value being predicted, a i is the i th coefficient, M is the model order, and ɛ[n] is the prediction error or residual [73]. The spectral content may be visually inspected by forming the model transfer function, and converting into the z-domain allowing for each pole to be plotted in the complex plane, from which the magnitude and frequency can be derived. Fleming and Tarassenko found that first applying a low-pass filter to the PPG waveform, followed by applying an AR model, yielded an average error of 3.4 breaths/minute. This method often leads to rapid changes in RR, which was addressed by Lee et al. (2011) who proposed an AR model that detects if a sudden change in RR occurs, and which will automatically vary the model parameters accordingly. Respiratory-Induced Variations in PPG Waveforms It has been well-documented that three respiratory-induced variations in PPG waveforms based on the variation of the PPG amplitude, intensity, and HR can be derived [24]. Respiratory sinus arrthymia (RSA) is a phenomenon in which heart rate variability is dependant on respiration, and which has been found to be present in many vertebrates [74]. This respiratory-induced frequency variation (RIFV) leads to slightly increasing HR during inhalation and slightly decreasing HR during exhalation[20]. Thus, via detecting the time of

15 2. Background 13 (a) (b) Figure 2.1: (a) An annotated section of raw PPG signal showing the three respiratory induced variations and (b) the signals created when each of the respiratory induced variations are isolated and undergo FFT. The red dotted lined represent the upper and lower limits of respiration and the frequency with the largest magnitude is set as the respiratory rate. each pulse in a PPG waveform, the time between each peak may be calculated and plotted as a tachogram allowing derivation of a respiratory component. This can be seen as the black lines in Figure 2.1a and a RIFV tachogram can be seen in Figure 2.1b. Respiration-induced intensity variation (RIIV) can be observed as the variation of the amplitude of the systolic peaks. This variation occurs as a result of variation of pressure within the pleural cavity (intrapleural pressure) causing changes in the baseline of perfusion [75]. This can be seen as the green lines in Figure 2.1a and a RIIV tachogram can be seen in Figure 2.1b. Respiration-induced amplitude variation (RIAV) refers to the difference in amplitude between a systolic peak and the preceding trough representing the onset of the pulse. The pulse strength is believed to vary due to a reduction of cardiac output during ventricular filling [75]. This can be seen as the red lines in Figure 2.1a and a RIAV tachogram can be seen in Figure 2.1b. RR Estimation from RIV Extracting the three respiration induced variations in the PPG (amplitude variation, frequency variation and intensity variation) will provide three separate waveforms from which respiration may be estimated. The most common method for processing these waveforms to estimate RR is the FFT. Examples of each of the three respiration induced variation waveforms can be seen in Figure 2.1b. Once the frequency spectrum of each waveform has been calculated, the frequency with the largest magnitude, within a certain range of frequencies, is set as the respiration rate. The resulting three respiration rates may then be fused, providing a robust estimate [24, 64].

16 2. Background 14 Use of AR models using the RIAV and RIFV waveforms have been shown successful, yet susceptible to rapid variations in RR estimates requiring post-processing to be performed [76]. A recent method - entitled ARspec (AutoRegressive specral median) applies multiple AR models of varying orders upon all three variations, and fuses the results. The performance of the ARspec algoirthm was compared against nine alternative algorithms and yielded the most reliable RR estimation result with a mean absolute error of 1.67 breaths/minute [75] whilst maintaining a high data retention rate.

17 3 The Paediatric Triage Study 3.1 Introduction The Paediatric Triage Study (PTS) was conducted between January 2011 and August 2011 during which a total of 272 children aged between 0-12 years old were enrolled when they presented to the Paediatric Emergency Department at the John Radcliffe Hospital, Oxford UK. Paediatric nurses used pulse oximeters (Nonin Medical, Inc.) to record a period of PPG (ranging from 30 seconds to 8 minutes) for each patient whilst also measuring temperature and a manual estimate of respiratory rate via counting the number of breaths in a 15 or 30 second period. In addition to the vital signs estimated by the nurse, respiratory distress and child compliance were also measured and in some cases capillary refill time. All measurements were added to the child s medical records for use by the clinicians who would go on to provide care. Although the triage decision outcomes were specified by the triage nurse, the final outcomes including admission decision and diagnosis decision were decided by the clinician. The initial aim of the study that collected this database was to validate the respiratory rate derived from the PPG signal against the manual observation of respiratory rate as taken by the triage nurse. The primary aim of this research is to determine whether the use of the vital signs as measured using pulse oximetry may accurately predict the triage decision made by the nurse. The secondary objective is to then determine whether the admission decision of the clinician could also be predicted using the same pulse oximetry recording Paediatric Triage Study Patient Selection Within the PTS database there were 281 PPG recordings and 272 patients with manual recordings. The discrepancy between these numbers comes from the fact that PPG recordings were often taken prior to gaining consent from the parent/guardian of the patient. In nine cases, PPG recordings were taken and consent was not granted. These recordings were therefore not used in this research and no manual measurements for these patients were available. 15

18 3. The Paediatric Triage Study 16 Figure 3.1: The steps taken in identifying all patients in the PTS database who meet the criteria of having complete observational recordings and a PPG recordings of 60 seconds or longer. In order to confidently derive a respiratory rate from the PPG signal it was decided that a minimum of 60 seconds of continuous PPG should be recorded. This was true for 232 of the 281 PPG recordings. Additionally, any patients with any missing observational data were excluded from this research leaving 252 patients with no missing observational measurements. Combining the patients with PPG recordings over 60 seconds and patients with complete manual measurements resulted in 231 patients having met both these criteria, as shown in Figure 3.1. Triage decision was categorised according to the Manchester triage score (MTS) putting each patient into one of six groups: red, orange, yellow, green, blue and none with red being the most severe, blue the least and none with no triage decision. No knowledge exists as to the reason no triage decision is provided for the patients in the none category. The distribution of triage decision for each of the eligible patients as well as the number who went on to be admitted from each triage group can be seen in Figure 3.1. Admission decision was categorised into five groups: High Dependency Unit (HDU), ward, day ward, home or Out of Hours GP. For this research any patient with an admission decision of either the HDU, ward or day ward were considered positive admissions whereas any patients in the remaining two groups were considered negative admissions as they are assumed low urgency, regardless of their initial triage decision. From Figure 3.1 it is apparent that there is a significant inbalance in the number of patients admitted to the extrema of the triage categories. Furthermore, there is a very broad range of discharge diagnoses including entries as specific as New onset childhood epilepsy and as broad as Unwell child and Paeds. other. Due to the number of different discharge diagnoses

19 3. The Paediatric Triage Study 17 alongside their varying triage categories and discharge decision it was impossible to further separate the patients into diagnoses specific groups for classification. In order to perform a preliminary regression of patients into their triage categories and their discharge decision, a Logistic Regression (LR) approach was employed Logistic Regression Logisitic regression is well suited to the analysis of this dataset due to its ability to provide the probability of a patient being admitted to each triage group. The key difference between logisitic regression and its counterpart, linear regression, is the choice of the dependant variable prior. Logisitic regression bounds the dependant variable via the use of the logit function. ( ) p logit(p) = log e (3.1) 1 p We can therefore state that the dependant variable is a linear weighted sum of the independant variables [77]. ( ) p log e = β 0 + β 1 x β n x n 1 p (3.2) = β 0 + βx where β 0 is the interecept and β 1,..,n are the coefficients where n is the number of independant variables. We can therefore solve for p giving: p = exp(β 0 + βx) 1 + exp(β 0 + βx) (3.3) The coefficients, β, are here estimated via the use of Maximum Likelihood Estimation (MLE) which is a method that seeks to find the probability distribution that makes the observed data most likely via finding a parameter vector that maximises the likelihood function where L(w y) = f(y w) (3.4) and w is a chosen parameter given the observed data y and f(y w) is the probability density function that specifies the probability of observing data y given the parameter w. MLE therefore attempts to find a vector of w to maximise the likelihood function [78]. The benefit of using LR over its counterpart of linear regression is that via the introduction of the exponential term (coming from the use of the logit function) results in the dependant

20 3. The Paediatric Triage Study 18 Table 3.1: Summary of a subset of vital signs as measured by the triage nurse and as derived from the PPG waveform. Vital Sign Mean (min - max) Std. Dev Observational Age [months] 49.7 (1-162) 43.6 Heart Rate [bpm] 128 (53-197) 29.8 Respiration Rate [breaths/min] 33.4 (14-80) 12.4 Sp02 [%] 97.2 (85-100) 2.5 Temperature [ o C] 37.1 (34-40) 0.9 PPG Derived Heart Rate [bpm] (61-204) 27.9 Respiration Rate [breaths/min] 25.2 (11-44) 6.9 variable being bound between 0 and 1. Thus, LR conveniently expresses the probability of a data belonging to a certain class. Once LR has been performed, a threshold between 0 and 1 may be set and used to classify the probability outputted by the LR model depending on whether it exceeds this threshold or not. Due to the aforementioned inbalance of patients in each triage category, the preliminary classification approach entailed first determining the capability of a LR model to differentiate patients admitted into the three most severe categories (132 patients) and those admitted into the three least severe categories (99 patients). It has here been assumed that patients in the none category are of the lowest severity. Initially two multivariate LR models were constructed: one using observational measurements and one using the equivalent measurements but as derived from the PPG signal. A subset of the features used in each model are summarised in Table 3.1. In addition to these features, gender and respiratory distress were used. Here the respiratory rate from the PPG has been derived via the use of the ARspec method discussed in Section 2.3 using a window size of 60 seconds and an overlap of 50 seconds. This algorithm was chosen as it has been proven to outperform nine other competing algorithms in both respiration rate accuary and data retention rate [75]. The heart rate from the PPG has been derived via identifying the cardiac component in the PPG frequency content. Prior to running the LR models using these measurements, the PPG derived estimates for each patient were compared against their respective manual observations, results of which will be presented in the following section. 400-fold cross validation was performed for both the LR models with a training to test ratio of 75:25 (174:57 patients) whilst varying the threshold from 0 to 1 in increments of

21 3. The Paediatric Triage Study 19 (a) (b) (c) (d) Figure 3.2: Comparison of vital signs as manually recorded by the triage nurse and as derived from the PPG waveform for (a) heart rate (b) respiration rate (c) boxplot of errors and (d) relationship between observed HR and observed RR Further LR models were developed that sought the ability to differentiate the admission decision within each of the triage categories as well as patients admitted in the most severe triage categories against the patients who were not admitted from the least severe triage categories. 3.2 Results of the PTS Analysis Matching Triage Decision The methods described in Section 3.1 were used to perform multiple multivariate Logisitic Regression (LR) analyses across the population of the PTS database. Prior to implementing any LR models, a brief assessment of the algorithms used to derive heart rate and respiration rate from the PPG were performed, as shown in Figure 3.2. The frequency spectral analysis algorithm used to derive the heart rate yielded a median error of 3.77 bpm with 25th and 75th percentiles of 1.66 bpm and 7.19 bpm respectively. The

22 3. The Paediatric Triage Study 20 (a) (b) Figure 3.3: (a) The performance of the LR model in discriminating between patients in the top three and bottom three triage categories for both observational measurements and PPG derived estimates. (b) The ability of the LR to discriminate between patients in the top three triage categories who were admitted against patients in the bottom three triage categories who were not admitted for both observational measurements and PPG derives estimates. ARspec algorithm for deriving respiration rate yielded a median error of 1.93 breaths/min with 25th and 75th percentiles of 0.55 breaths/min and 5.64 breaths/min respectively. As the primary aim of this research is to match the triage decisions of the paediatric nurses an initial and broad model compared the performance of a LR model when using vital signs derived from PPG against a LR model using the manual observations to differentiate between patients admitted to the three most severe triage categories and patients admitted to the three least severe triage categories. Figure 3.3a shows the performance of both these models. Further models were developed to determine more specifically the difference between each pair of triage categories. Table 3.2 shows the performance of each of these models. The optimum performance of each model, P opt, was determined via: P opt = min (100 sensitivity) 2 + (FPR) 2 (3.5) i.e. the result corresponding to the smallest Euclidean distance from both sensitivity and specificity being 100% Matching Admission Decision The secondary objective sought to match the hospital admission decision of the clinician. Another LR model was implemented to differentiate patients who were subsequently admitted to hospital

23 3. The Paediatric Triage Study 21 Table 3.2: Summary of results of performing LR for differentiating triage decision for patients in specific triage groups. Negative Positive Sensitivity Specificity AUC Yellow Orange Green Orange None Orange Green Yellow None Yellow None Green from the three most severe triage categories and the patients who were not admitted to hospital from the three least severe triage categories. The results shown in Figure 3.3b compare the triage performance when using the PPG derived features and manual observations. When manual observations are used the AUC is 0.53 whereas when features derived from the PPG are used the AUC is Further models were developed to differentiate the patients who were admitted against those who were not within each triage category. This was not performed on the red, blue and none triage categories due to insufficient patients in each of these categories. Table 3.3 shows the results of this analysis for patients in the orange, yellow and green triage categories. Table 3.3: Results of using LR to predict which patients within each triage group are admitted to hospital. Triage Category Sensitivity Specificity AUC Orange Yellow Green Discussion The results of Section 3.2 provide limited information as to the performance of using the vital signs derived from PPG to predict triage decisions. However, as shown in Figure 3.3a, the performance of using the pulse oximeter derived features is very similar to using the manual observations. It can therefore be hypothesised that the nurses performing triage based some of their decisions on additional factors which may not have been recorded on the database, such as cyanosis or patient compliance/restlessness. Furthermore, due to the wide range of discharge

24 3. The Paediatric Triage Study 22 diagnoses and the inhomogeneity of vital signs within each diagnosis, it is impossible to further assess the performance of automating triage decisions for specific diagnoses using this database. There are some discrepancies between the manual vital sign measurements and the estimates derived from the PPG, shown in Figure 3.2. The primary aim of this research is to assess the ability to perform triage using the features derived from the PPG, even in the presence of noise. Nonetheless, the ARspec algorithm yielded similar results to those reported in literature and the positive correlation seen between the estimates (R = 0.93 for HR and R = 0.66 for RR) can allow these estimates to be used with confidence. Figure 3.2b shows several very high manually observed respiration rates (>60 breaths/min). As with any manual observations their recording are liable to errors such as pressing of the wrong key or caused by illegible hand writing. An additional source of error is introduced as the triage nurse will generally count the number of respirations in a 15 or 30 second period and scale up this value for breaths/minute, thus introducing a small rounding error. However, a rounding error is unlikely to cause such extreme measurements. Indeed, the diagnosis decisions of patients with questionably high respiration rates (>60 breaths/min, n=8, mean age = 11 months) were referenced and all were diagnosed with either a chest infection, upper or lower respiratory tract infection or a viral induced wheeze - all of which include breathlessness as a classic symptom. It can therefore be concluded that the manual measurements are likely to be correct and the ARspec algorithm performs less accurately at higher respiratory rates. The source of this drop in performance is due to the fact the ARspec algorithm bases its estimate on the frequency content of the PPG. The pre-processing of the PPG waveform prior to undergoing the ARspec algorithm entails applying a finite-impulse response low-pass filter, with a transition band from 0.5 to 1.2 Hz (30-72 breaths/min) to remove the cardiac component of the signal. Evidently, in the cases of very high respiratory rate, there is a significant overlap between the respiratory frequency and the expected range of cardiac frequencies resulting in the respiratory frequency being filtered out of the signal. Furthermore, a patient with an elevated respiration rate commonly also has an elevated heart rate, as shown in Figure 3.2d. It may therefore be proposed that the ARspec algorithm may be improved if the transition band of the filter was dependant upon the PPG estimated heart rate. The correlation between triage category with likelihood of admission is poor and contrary to what would be expected from the literature [17, 79], although the distribution of patients admitted to each triage category (with the exception of the blue category) is consistent with literature,

25 3. The Paediatric Triage Study 23 the subsequent admissions from each is highly disproportionate. Wuerz et al. report the highest triage category to admit between 90% to 100% of patients whereas the second and third highest admit 65% and 35% respectively. Figure 3.3b shows the performance of predicting admission amongst patients who were prioritised by nurses during triage providing very poor results.

26 4 The CALMS-2 Database 4.1 Introduction The Computer Alerting Monitoring System 2 (CALMS-2) study was conducted between May 2009 and December 2012 at the Oxford Cancer Hospital in the Oxford University Hospitals NHS Trust (Oxford, UK). The study was a before-and-after design prospective trial which aimed to assess whether introducing continuous patient monitoring could reduce patient length-of-stay after upper gastrointestinal surgery when compared to the currently employed Track-and- Trigger (T&T) system. Additionally, the continuous monitoring in the CALMS-2 trial made use of a computer-modelled alerting system of patient deterioration which differs from the paper based T&T system currently in place. The study was conducted in two phases; the first phase consisted of implementing the current T&T system post-surgery until the patient has been deemed fit to be discharged from high dependency unit (HDU). Upon leaving the HDU the patients continued on the T&T system but were switched over to telemetry monitoring systems (as opposed to a bedside monitoring system in the HDU) which were portable and allowed continuous monitoring of a subset of vital signs. Patients remained on the telemetry monitoring system until deemed fit to be discharged. The results of the computer-modelling system were hidden from the hospital staff throughout Phase I. During Phase II however, the results of the computer-modelling system were made available for the hospital staff to be used in their clinical decision making. A subset of the patients, either during or once having been deemed fit to leave the HDU experienced an adverse event which includes an unplanned admission to the emergency intensive care unit (EICU), a mortality event or a cardiac arrest. The conventional T&T level of care system remained in use throughout both phases of the study. The T&T system comprises of an assigned early warning score (EWS) based on six physiological parameters heart rate, respiration rate, blood oxygen saturation, systolic blood pressure, temperature and level of consciousness. These vital signs were measured manually at 24

27 4. The CALMS-2 Database 25 a frequency of roughly once every four hours. The dependency of the patient on any additional supplemental oxygen also contributes to their EWS. A manual observation for each vital sign is compared to a table of predefined healthy and unhealthy vital signs and each given a score from zero (healthy) up to three (unhealthy). Thus the extrema are zero for healthy patients and 20 for severely ill patients who are also receiving supplemental oxygen. Of particular interest to this research is the information acquired when the patients were using the telemetry monitoring system. The telemetry system comprised of a Bluetooth pulse oximeter (Nonin Medical, Inc.) which recorded the PPG waveforms at 75Hz as well as the PPG derived heart rate and S p 0 2 estimates. Throughout Phase II of the CALMS-2 trial only the heart rate and S p 0 2 values were made available for the clinical staff to monitor and use for the T&T system. This research investigates the potential benefit in detecting patient deterioration (leading to an adverse event) via deriving respiration rate from the PPG and producing a clinically useful continuous novelty score representing the patients condition CALMS-2 Patient Selection For this research, the 207 patients enrolled in Phase II of the trial were considered. Of these 207 patients, 27 experienced an adverse event and were labelled Abnormal. The remaining 180 patients were labelled Normal. It has been reported that the vital signs of patients vary in the days immediately after surgery [80]. Thus, for Normal patients it can be assumed that their most normal vital signs are in the period immediately preceding their discharge from the hospital. Furthermore, an Abnormal patient will show irregular vital signs in the time before their EICU admittance [81]. Normal patients with a length of stay shorter than 3 days (10th percentile) and longer than 20 days (90th percentile) were excluded from this research. Each patient in the Normal subset was checked for a PPG waveform 24 hours before their discharge and each Abnormal patient was checked for a PPG waveform any time before their EICU admittance. Subsequently, 74 Normal patients and 8 Abnormal patients met the above the criteria. The patients who met the criteria were split into three groups; one for training and two for test. The 74 Normal patients were split into a large training set and a smaller test set. The Normal Training set contained 64 Normal patients leaving 10 patients to form the Normal Test set. All of the 8 Abnormal patients formed the Abnormal Test set. Much of the previous research involving the CALMS-2 dataset has involved some use of novelty detection. Novelty detection can be deemed appropriate to be used in this research

28 4. The CALMS-2 Database 26 for two reasons. Firstly, novelty detection is most commonly used in datasets with severely unbalanced amounts of data from each class being investigated. In this case, this is an appropriate approach due to only having eight members of the Abnormal class. Furthermore, the use novelty detection will allow for direct comparison with previous research allowing for clearer conclusions to be drawn as results will be presented in a similar form Kernel Density Estimation Novelty detection can be conceptually thought of as determining a probability density function for normal data, and then when presented with a new piece of data, determining the probability of the new data as having come from the normal data. In this case, as there is no prior knowledge of the distribution of the normal data it is appropriate to use a non-parametric approach, thus learning the distribution from the normal data. The non-parametric method chosen is kernel density estimation. Kernel Density Estimation (KDE) aims to provide a proability that a piece of new test data x comes from the known training data. Lets start by defining a small region in the d dimeonsional feature space of volume V that contains x. As this is a non-parametric model we are assuming that our data is coming from some unknown probability density p(x). If we draw a large number of samples, N, from this distribution, and assume that the region width, h, is small enough such that the probability density is constant over its width we can say: p(x) = K NV (4.1) where K is the total number of points within the small region. The question then arises as how to specify K and V. In this method the K-nearest neighbour method is employed in which the value of K is fixed and the value V is determined from the surrounding data. As means of a preliminary introduction, if we assume the small region to be a d dimensional hypercube centred on the point x we wish to determine the probability density of the region thus we need to count how many points of the training data lie within the hypercube. We can define the hypercube kernel function as: k(u) = { 1, if ui 1/2 0, otherwise (4.2)

29 4. The CALMS-2 Database 27 where i=(1,...,d). Thus the total number of datapoints lying within the region can be calculated as: N ( ) x xn K = k n=1 h Substituting this expression into Equation 4.1 provides an updated estimate of the density at x. where h d p(x) = 1 N is the volume of the hypercube. N n=1 ( ) 1 x h k xn d h (4.3) (4.4) However, using a hypercube kernel introduces artificial discountinuous boundaries. It is therefore favourable to choose an alternative and smoother kernel function which will in turn provide a smooth density model, with the popular kernel of choice being a Gaussian. Replacing the hypercube kernel function with the Gaussian kernel function yields: p(x) = 1 N(2π) d/2 σ d N { } x xn exp n=1 2σ 2 where h n has been replaced by its equivalent of the standard deviation of the Gaussian distribution, σ. (4.5) The parameter σ specifies the kernel width and its value plays a pivotal role in ensuring a suitably tuned model as it determines the range of influence of each point in the d dimensional space. With an overly large value of σ the kernel widths will be large such that overlapping and smoothing of the data will occur resulting in a poor resolution. Conversely, if σ is too small, the kernels will be very narrow and will account poorly for noise in the data and will result in overfitting of the model. The use of k-means clusters may be employed to determine an estimate for σ analytically. Bishop s method assumes that the mean distance of the m points surrounding each cluster suitably estimates the kernel width [82]. For a single variable y, the variance may be calculated via: σs 2 = 1 m (y i y) 2 (4.6) m i=1 where y is the sample mean and m the number of samples of y. If we repeat Equation 4.6 at each cluster centre, Q i we can collect and estimate the local variance at each pattern. The global variance can then be set as the mean of the local variances:

30 4. The CALMS-2 Database 28 (a) (b) Figure 4.1: (a) Visualisation of all cluster centers transformed from 6D to 2D and (b) the most normal 400 cluster centres with a large random sample of training data. σ 2 s = 1 N ( N m i=1 j Q i y i y j 2 Each vital sign, prior to any use first underwent a selection process governed by their respective physiologically possible ranges. Any vital signs lying outside of this range were replaced by their respective mean values that lie within the physiologically plausible range. Additionally, in this model, it is assumed that each of the vital signs are equally weighted with respect to their importance and influence on the novelty score. It is therefore appropriate to perform zero mean and unit variance normalisation to each of the inputs to the model. ) (4.7) x n = x i µ x σ x (4.8) where x n is the normalised value of x i and µ x and σ x are the mean and standard deviation of the feature x as calculated from the training dataset, respectively. Using K-mean clustering 500 cluster centres were initially chosen as described in previous use of Bishop s method to find the general data distribution of the normalised training data in d dimensions. In an example to follow (Section 4.1.2) the feature space is six-dimensional (d = 6). Via the use of Sammon s Mapping the location of the cluster centres may be non-linearlly transformed

31 4. The CALMS-2 Database 29 from 6D into 2D space [83]. Upon visualisation it was apparent that outliers of the original training data had resulted in cluster centres being present at what can be considered less normal data as shown in Figure 4.1a. In order to account for this, the overall mean coordinates of the clusters was computed and the 100 clusters with the largest Euclidean distance from this value were discarded to ensure the clusters used in the model of normality only represent the most normal data. This can be confirmed when a large random sample of training data is mapped over the 400 most normal cluster centres as shown in Figure 4.1b. Application of Equations 4.6 and 4.8 can then be applied to each cluster centre and the surrounding m data points. Thus the final probability density at location x can be given by p(x) = 1 400(2π) d/2 σ d 400 n=1 The final novelty score may then be quantified via { } x xn exp 2σ 2 (4.9) novelty = log e p(x) (4.10) It should be noted that the novelty of each data point is calculated with respect to the Euclidean distance to the kernels placed over the cluster centres. Therefore a residual novelty will exist caused by the offset of the mean cluster center from the origin. The residual novelty is calculated via finding the novelty of the mean of the cluster centres. This residual novelty is to be subtracted from the training and subsequently also subtracted from any test novelty scores produced when these 400 cluster centres are used. 2D KDE Example As a means of providing a visual example of the KDE method, a simple 2D model using simulated data will be provided. We start with a sample of 10 patients, from whom HR and RR have been measured from five times at regular intervals. We can normalise and plot these data and then apply K-means clustering assuming two cluster centres will be present. We apply Bishop s method by drawing a circle around each cluster centre which encapsulates m data points, as seen in Figure 4.2a. Using these encapsulated data points we are able to determine the global variance σ, allowing us to fit 2D Gaussian distributions over each cluster centre. Furthermore, when measurements are taken from a test patient and presented to the model, we can visually examine their novelty over the feature and novelty space, as seen in Figure 4.2b.

32 4. The CALMS-2 Database 30 (a) (b) Figure 4.2: 2D Visualisation of the KDE technique on simulated normalised heart rate and respiratory rate measurements. (a) A scatter plot showing the normalised data measurements, the two cluster centres and the area surrounding the cluster centres which in turn determines the global variance and (b) the resulting feature space showing the Gaussian kernels over each cluster centre. The magenta triangles represent measurements taken from a test patient - the points furthest from the cluster centres represent a period of high novelty. Observational Data As a means of model validation, kernel density estimation was first performed on the manual vital sign observations recorded by the clinical staff and compared to the corresponding T&T scores. In this model heart rate, respiration rate, blood oxygen saturation, temperature, systolic and diastolic blood pressures were used, thus forming a 6D feature space. The manual observations taken in the last 24 hours for the Normal Training patients were extracted and any recording with a missing measurement replaced by the mean of the Normal Training dataset. Furthermore, if any of the vital signs were out of their physiologically possible ranges stated in Table 4.1 the measurement was replaced by the mean value. A 6D KDE model was formed as described in the previous section yielding a novelty score at each point a valid manual observation was taken. This model will now be referred to as the 6D Observational KDE Model. Continuous PPG Data Two subsequent KDE models were developed using features derived from the continuous PPG recordings. The 75 Hz PPG waveform was analysed in one minute windows with 30 seconds

33 4. The CALMS-2 Database 31 Table 4.1: Physiological limits applied to obervational limits. Lower Limit Upper Limit Heart Rate [bpm] Respiration Rate [breaths/min] 3 45 Sp02 [%] Temperature [ o C] Systolic Blood Pressure [mmhg] Table 4.2: Summary of novelties produced by the 6D Observational KDE model. Mean Median Std. Dev 95% Normal Training Novelties Normal Test Novelties Abnormal Test Novelties overlap. An estimate for HR, RR and S p 0 2 was therefore produced for each of these windows. Once again, the use of the ARspec algorithm was used to derive RR from the PPG. The first model used HR, RR and S p 0 2 as features and thus is to be referred as the 3D Continuous KDE Model. The second model made use of the same features as the 3D Continuous KDE Model but also introduced variability features based on the difference between the current measurement and the measurement from the previous window. referred as the 6D Continuous KDE Model. This model is now to be 4.2 Results of the CALMS-2 Analysis D Observational Novelty Model The first kernel density estimation (KDE) was performed using the six vital signs collected manually by nurses on the ward. This allows direct comparison between the 6D Observational KDE novelty score and the Track and Trigger (T&T) score provided by the nurse at the same time points. The model of normality was built as described in Section The results for two Normal Test and two Abnormal Test patients can be seen in Figure 4.3. The normalised histograms of the observational novelties for the Normal Training and both test sets of patients can be seen in Figure 4.4 and are summarised in Table 4.2 This model had a residual novelty of 9.50 and a global variance of 2.82.

34 4. The CALMS-2 Database 32 (a) (b) (c) (d) Figure 4.3: Time series plots of the 6D Observational KDE novelties for four test patients. The upper plot in each shows the manual observations for the 5 vital signs. Systolic and diastolic blood pressures have here been combined into Mean Arterial Blood Pressure (MAP). (a) Normal Test patient 294 (b) Normal Test patient 371 (c) Abnormal Test patient 349 and (d) Abnormal Test patient 464. (a) (b) (c) Figure 4.4: Novelty distributions for the 6D Observational KDE Model for (a) Normal Training (b) Normal Test and (c) Abnormal Test.

35 4. The CALMS-2 Database 33 (a) (b) (c) (d) Figure 4.5: Time series plots of the 3D Continuous KDE novelties for four test patients. The upper plot in each shows the 3 vital signs derived from the PPG. (a) Normal Test patient 294 (b) Normal Test patient 371 (c) Abnormal Test patient 349 and (d) Abnormal Test patient 464. Table 4.3: Summary of novelties produced by the 3D Continuous KDE model. Mean Median Std. Dev 95% Normal Training Novelties Normal Test Novelties Abnormal Test Novelties D Continuous Novelty Model The second KDE model was performed using features extracted from the continuous PPG waveform. The features in this model were heart rate, respiratory rate and S p 0 2. The results for two Normal Test and two Abnormal Test patients can be seen in Figure 4.5. The normalised histograms of the 3D continuous novelties for the Normal Training and both test sets of patients can be seen in Figure 4.6 and are summarised in Table 4.3

36 4. The CALMS-2 Database 34 (a) (b) (c) Figure 4.6: Novelty distributions for the 3D Continuous KDE Model for (a) Normal Training (b) Normal Test and (c) Abnormal Test. Table 4.4: Summary of novelties produced by the 6D Continuous KDE model. Mean Median Std. Dev 95% Normal Training Novelties Normal Test Novelties Abnormal Test Novelties This model had a residual novelty of 3.4 and a global variance of D Continuous Novelty Model The final KDE model was implemented using the same three features as in the 3D continuous model plus an additional three variability features as described in Section The results for two Normal Test and two Abnormal Test patients can be seen in Figure 4.7. The normalised histograms of the 6D continuous novelties for the Normal Training and both test sets of patients in Figure 4.8 and are summarised in Table 4.4. This model had a residual novelty of 8.3 and a global variance of Model Comparison The primary objective of this part of the research is to determine whether the use of continuous patient monitoring via pulse oximetry can outperform the current observational T&T monitoring system. The three characteristics used to classify each model are sensitivity, false positive rate (FPR)/false alarm rate and the time prior to an adverse event each model detects a patient as being abnormal. In order to visually highlight the benefit of using continuous monitoring as opposed to periodic observational monitoring, a section of 48 hours from two Abnormal Test patients can be seen in Figure 4.9.

37 4. The CALMS-2 Database 35 (a) (b) (c) (d) Figure 4.7: Time series plots of the 6D Continuous KDE novelties for four test patients. The upper plot in each shows the 3 vital signs derived from the PPG. (a) Normal Test patient 294 (b) Normal Test patient 371 (c) Abnormal Test patient 349 and (d) Abnormal Test patient 464. (a) (b) (c) Figure 4.8: Novelty distributions for the 6D Continuous KDE Model for (a) Normal Training (b) Normal Test and (c) Abnormal Test.

38 4. The CALMS-2 Database 36 (a) (b) (c) (d) Figure 4.9: A closer inspection of 48 hours of novelty scores prior to ICU admittance for (a) Abnormal Test patient 389 using the 3D Continuous KDE (b) Abnormal Test patient 424 using the 3D Continuous KDE (c) Abnormal Test patient 389 using the 6D Continuous KDE (d) Abnormal Test patient 424 using the 6D Continuous KDE. Figure 4.10: Relationship between t ew and FPR for the three KDE models and the T&T system in the 12 hours prior to an adverse event. In order to be clinically useful, a threshold must be set on the novelty such that if a novelty score exceeds this threshold an alarm will sound to alert the clinical staff. We therefore aim to determine the optimum threshold to reduce false alarm rate whilst simultaneously ensuring detection of a true alarm with sufficient intervention time. Figure 4.10 shows the performance of each model with respect to their FPR (which we would like to minimise) and their time of early warning, t ew, (which we would like to maximise) across a novelty threshold range of 0 to 45 with increments of 0.1. T ew was calculated for each patient via finding the first time a novelty exceeds the threshold

39 4. The CALMS-2 Database 37 Figure 4.11: A schematic representing the windowing technique used to determine a threshold for various time frames prior to ICU admittance, within the 12 hours preceding an event in the Abnormal Test patients. FPR was calculated via finding any time a patient in the Normal Test group yielded a novelty above this same threshold. Thus with a low threshold we can expect to have a high FPR and a high t ew and vice versa. The sensitivity and specificity of the 6D continuous model was then considered. Building on the knowledge that patients vital signs are abnormal prior to an event [81], we can assume that a patient will show their highest novelty just before an event. However, here we are attempting to maximise the length of time prior to an event we can detect a potentially abnormal patient. Via finding the sensitivity and specificity of a model whilst varying the time frame, τ, prior to a known event we can produce a variety of candidate thresholds. Figure 4.11 shows this process schematically. The novelties prior to each event were windowed into 1 hour segments with no overlap. Each window then produced a sensitivity and specificity value based on the following definitions. A true positive (TP) was reported if any novelty value within a window exceeded the novelty threshold for an Abnormal Test patient prior to their adverse event. A false negative (FN) was reported if none of the novelties within an hour window exceeded the novelty threshold for an Abnormal Test patient prior to their adverse event. A true negative (TN) was reported if none of the novelties within an hour window exceeded the novelty threshold for a Normal Test patient. Finally, a false positive (FP) was reported if any novelty value within a window exceeded the novelty threshold for a Normal Test patient. Figure 4.12a shows the results of running this analysis on a range of τ values. Furthermore, the optimum threshold in each case can be calculated via:

40 4. The CALMS-2 Database 38 (a) (b) Figure 4.12: (a) The sensitivites and FPRs seen when looking at increasing periods of time prior to ICU admittance. (b) Via taking the optimum sensitivity and FPR value in each time frame, the corresponding threshold can be plotted against the window size from which it was derived. For example, if we were to select a threshold of 3.9, we can achieve 2 hours of warning prior to ICU admittance with a sensitivity of 81% and specificity of 88%. Once more, it is a trade off between sensitivity, specificity and TEW. T opt = arg min thresh (100 sensitivity) 2 + (FPR) 2 (4.11) A plot of the relationship between each of the optimum thresholds and their corresponding τ can be seen in Figure 4.12b. 4.3 Discussion Section 4.2 presented the results of applying novelty detection to both observational and continuous vital sign measurements. The first model used the six vital signs measured routinely by nurses. Figure 4.4 and Table 4.2 show that the Novelty Test patients show a much wider range of novelty scores with a maximum novelty of 44. Tables 4.2, 4.3 and 4.4 show the statistics of all three KDE models. In all three KDE models the Abnormal Test patients have higher mean, median, standard deviations, and 95th percentiles than the Normal Test and Normal Training patients. The 6D Observational KDE model however shows the largest difference between the Abnormal and Normal patients. Although the 6D Observational KDE model was limited in temporal resolution via only using data obtained manually, the additional three features appear to increase its ability to discriminate between Normal and Abnormal patients. An example of this can be seen at the start of 04/23 in Figure 4.3a (patient 294) where a decrease in MAP corresponds to an increase in novelty whilst

41 4. The CALMS-2 Database 39 the HR, RR and S p 0 2 values remain relatively constant. This change in physiology is also detected in the T&T system cuasing an elevated T&T score. Furthermore, Figure 4.9 provides a closer inspection of a 48 period of interest for two Abnormal Test patients prior to ICU admittance for the 3D Continuous KDE and the 6D Continuous KDE. Figure 4.9a (patient 389) shows the result when using the 3D Continuous KDE model. There is a clear spike in novelty missed by the T&T system in the early afternoon of 07/07. In Figure 4.9c (also patient 389), when the 6D Continuous KDE model is used, this peak is more distinct and an additional spike in novelty is detected roughly 24 hours beforehand, again missed by the T&T system. Figure 4.9b (patient 424) shows the result when using the 3D Continuous KDE model. Again, this shows a similar phenomenon where there is an increase in novelty in-between manual observations in the evening of 08/05 and just prior ot ICU admittance. When using the 6D Continuous KDE model, Figure 4.9d these spikes in novelty are again more distinct. The comparison of t ew between the three KDE models and the T&T system is shown in Figure When a very low threshold is used, t ew is at its maximum allowable value whilst also yielding a very high false alarm rate. The noisy nature of each plot when the false alarm rate is low is due to a limited number of patients having novelties at the extreme values. If a greater population were to be available this data would be smoother. Nonetheless, it can be concluded that there is an improvement on t ew when continuous monitoring is used in comparison to when using routine manual observations. If we were to fix the false alarm rate to 5% (specificity of 95%) the T&T system, 6D Observational Novelty system, 3D Continuous Novelty system and the 6D Continuous Novelty system would allow 6.09, 8.18, and hours of early warning respectively. One limitation of all the KDE models developed is the assumption that Normal vital signs are constant over the entire length of stay. In reality, it is known that after having undergone major surgery, such as in this research, a patient s vital signs immediately after surgery will likely be irregular. This has been demonstrated for Normal patients showing high novelty scores immediately after surgery followed by a period of steady decline in novelty over the following days until discharge [80]. This time dependency of novelty distributions could be accounted for via training a series of KDE models at regular time intervals, for example every 24 hours, after surgery with the training data coming from patients who experienced no adverse events. This would allow the models to account for expected vital sign irregularities in post-surgery patients. A further time dependency of Normal vital signs may be introduced via accounting for circadian rhythms. Circadian rhythms are the daily variation in vital signs which tend to

42 4. The CALMS-2 Database 40 Figure 4.13: Circadian rhythms exhibited by Abnormal Test patient 424 resulting in a cyclic nature of novelty scores. As this patient has been discharged from the ICU and as circadian rhythms are commonly associated with healthy autonomic systems it would be expected that the novelty score would be a relatively low and constant value. be higher whilst a patient is awake and lower when asleep and are normally associated with healthly and mature autonomic nervous systems [84]. It was found that several test patients exhibited circadian rhythms, which in turn led to a cyclic nature in novelty score, as seen in Figure The relationship between cyclic novelty scores and circadian rhythms could be explained by limitations in the methodology in forming the training dataset. No account was made for the sleep-wake state of the training patients which is likely to end in a bias in Normal data being taken from one of the sleep-wake states. A further limitation was introduced in the estimation of σ in each KDE model. Here it was assumed that all vital signs were of equal importance and therefore contributed equally to the novelty score. This limitation may be overcome via estimating a specific value of σ for each vital sign proportional to its contribution to the final novelty score. Additionally, pulse oximetry is infamous for its susceptibility to signal artefacts. This research has not made any attempts to differentiate high quality PPG from low quality PPG, though it should be noted that much research exists in the literature to do this so as to ensure robust vital sign derivation [85]. Although this research performed short term noise reduction via applying a short term 5-minute median filter to the derived vital signs this methodology would be ineffective for long term applications.