arxiv: v5 [cs.lg] 7 Jan 2016

Transcription

1 LEARNING TO DIAGNOSE WITH LSTM RECURRENT NEURAL NETWORKS REVISED Zachary C. Lipton 1 Department of Computer Science and Engineering University of Caifornia, San Diego La Joa, CA 92093, USA zipton@cs.ucsd.edu David C. Kae 1 Department of Computer Science PUSH OVER TO LEFT University of Southern Caifornia Los Angees, CA dkae@usc.edu arxiv: v5 [cs.lg] 7 Jan 2016 Chares Ekan Department of Computer Science and Engineering University of Caifornia, San Diego La Joa, CA 92093, USA ekan@cs.ucsd.edu ABSTRACT Randa Wetze Laura P. and Leand K. Whittier Virtua PICU Chidren s Hospita Los Angees Los Angees, CA rwetze@cha.usc.edu Cinica medica data, especiay in the intensive care unit (ICU), consist of mutivariate time series of observations. For each patient visit (or episode), sensor data and ab test resuts are recorded in the patient s Eectronic Heath Record (EHR). Whie potentiay containing a weath of insights, the data is difficut to mine effectivey, owing to varying ength, irreguar samping and missing data. Recurrent Neura Networks (RNNs), particuary those using Long Short-Term Memory (LSTM) hidden units, are powerfu and increasingy popuar modes for earning from sequence data. They effectivey mode varying ength sequences and capture ong range dependencies. We present the first study to empiricay evauate the abiity of LSTMs to recognize patterns in mutivariate time series of cinica measurements. Specificay, we consider mutiabe cassification of diagnoses, training a mode to cassify 128 diagnoses given 13 frequenty but irreguary samped cinica measurements. First, we estabish the effectiveness of a simpe LSTM network for modeing cinica data. Then we demonstrate a straightforward and effective training strategy in which we repicate targets at each sequence step. Trained ony on raw time series, our modes outperform severa strong baseines, incuding a mutiayer perceptron trained on hand-engineered features. 1 INTRODUCTION Time series data comprised of cinica measurements, as recorded by caregivers in the pediatric intensive care unit (PICU), constitute an abundant and argey untapped source of medica insights. Potentia uses of such data incude cassifying diagnoses accuratey, predicting ength of stay, predicting future iness, and predicting mortaity. However, besides the difficuty of acquiring data, severa obstaces stymie machine earning research with cinica time series. Episodes vary in ength, with stays ranging from just a few hours to mutipe months. Observations, which incude sensor data, vita signs, ab test resuts, and subjective assessments, are samped irreguary and pagued by missing vaues (Marin et a., 2012). Additionay, ong-term time dependencies compicate earning with many agorithms. Lab resuts that, taken together, might impy a particuar diagnosis may be separated by days or weeks. Long deays often separate onset of disease from the appearance of symptoms. For exampe, symptoms of acute respiratory distress syndrome may not appear unti hours after ung injury (Mason et a., 2010), whie symptoms of an asthma attack may present shorty after admission but change or disappear foowing treatment. Author website: Author website: 1

2 Recurrent Neura Networks (RNNs), in particuar those based on Long Short-Term Memory (LSTM) (Hochreiter & Schmidhuber, 1997), mode varying-ength sequentia data, achieving state-of-the-art resuts for probems spanning natura anguage processing, image captioning, handwriting recognition, and genomic anaysis (Aui et a., 2013; Sutskever et a., 2014; Vinyas et a., 2015; Karpathy & Fei-Fei, 2015; Liwicki et a., 2007; Graves et a., 2009; Poastri et a., 2002; Vohradský, 2001; Xu et a., 2007). LSTMs can capture ong range dependencies and noninear dynamics. Some sequence modes, such as Markov modes, conditiona random fieds, and Kaman fiters, dea with sequentia data but are i-equipped to earn ong-range dependencies. Other modes require domain knowedge or feature engineering, offering ess chance for serendipitous discovery. In contrast, neura networks earn representations and can discover unforeseen structure. This paper presents the first empirica study using LSTMs to cassify diagnoses given mutivariate PICU time series. Specificay, we formuate the probem as mutiabe cassification, since diagnoses are not mutuay excusive. Our exampes are cinica episodes, each consisting of 13 frequenty but irreguary samped time series of cinica measurements, incuding body temperature, heart rate, diastoic and systoic bood pressure, and bood gucose, among others. Associated with each patient are a subset of 429 diagnosis codes. As some are rare, we focus on the 128 most common codes, cassifying each episode with one or more diagnoses. Because LSTMs have never been used in this setting, we first verify their utiity and compare their performance to a set of strong baseines, incuding both a inear cassifier and a MutiLayer Perceptron (MLP). We train the baseines on both a fixed window and hand-engineered features. We then test a straightforward target repication strategy for recurrent neura networks, inspired by the deep supervision technique of Lee et a. (2015) for training convoutiona neura networks. We compose our optimization objective as a convex combination of the oss at the fina sequence step and the mean of the osses over a sequence steps. Additionay, we evauate the efficacy of using additiona information in the patient s chart as auxiiary outputs, a technique previousy used with feedforward nets (Caruana et a., 1996), showing that it reduces overfitting. Finay, we appy dropout to nonrecurrent connections, which improves the performance further. LSTMs with target repication and dropout surpass the performance of the best baseine, namey an MLP trained on hand-engineered features, even though the LSTM has access ony to raw time series. 2 RELATED WORK Our research sits at the intersection of LSTMs, medica informatics, and mutiabe cassification, three mature fieds, each with a ong history and rich body of research. Whie we cannot do justice to a three, we highight the most reevant works beow. 2.1 LSTM RNNS LSTMs were originay introduced in Hochreiter & Schmidhuber (1997), foowing a ong ine of research into RNNs for sequence earning. Notabe earier work incudes Rumehart et a. (1985), which introduced backpropagation through time, and Eman (1990), which successfuy trained RNNs to perform supervised machine earning tasks with sequentia inputs and outputs. The design of modern LSTM memory ces has remained cose to the origina, with the commony used addition of forget gates (Gers et a., 2000) (which we use), and peep-hoe connections (Gers & Schmidhuber, 2000) (which we do not use). The connectivity pattern among mutipe LSTM ayers in our modes foows the architecture described by Graves (2013). Pascanu et a. (2014) expores other mechanisms by which an RNN coud be made deep. Surveys of the iterature incude Graves (2012), a thorough dissertation on sequence abeing with RNNs, De Muder et a. (2015), which surveys natura anguage appications, and Lipton et a. (2015), which provides a broad overview of RNNs for sequence earning, focusing on modern appications. 2.2 NEURAL NETWORKS FOR MEDICAL DATA Neura networks have been appied to medica probems and data for at east 20 years (Caruana et a., 1996; Baxt, 1995), athough we know of no work on appying LSTMs to mutivariate cinica time series of the type we anayze here. Severa papers have appied RNNs to physioogic signas, incuding eectrocardiograms (Siipo & Marchesi, 1998; Amari & Cichocki, 1998; Übeyi, 2009) and 2

3 gucose measurements (Tresp & Briege, 1998). RNNs have aso been used for prediction probems in genomics (Poastri et a., 2002; Xu et a., 2007; Vohradský, 2001). Mutipe recent papers appy modern deep earning techniques (but not RNNs) to modeing psychoogica conditions (Dabek & Caban, 2015), head injuries (Rughani et a., 2010), and Parkinson s disease (Hammera et a., 2015). Recenty, feedforward networks have been appied to medica time series in siding window fashion to cassify cases of gout, eukemia (Lasko et a., 2013), and critica iness (Che et a., 2015). 2.3 NEURAL NETWORKS FOR MULTILABEL CLASSIFICATION Ony a few pubished papers appy LSTMs to mutiabe cassification tasks, a of which, to our knowedge, are outside of the medica context. Liu et a. (2014) formuates music composition as a mutiabe cassification task, using sigmoida output units. Most recenty, Yeung et a. (2015) uses LSTM networks with mutiabe outputs to recognize actions in videos. Whie we coud not ocate any pubished papers using LSTMs for mutiabe cassification in the medica domain, severa papers use feedforward nets for this task. One of the eariest papers to investigate muti-task neura networks modeed risk in pneumonia patients (Caruana et a., 1996). More recenty, Che et a. (2015) formuated diagnosis as mutiabe cassification using a siding window mutiayer perceptron. 2.4 MACHINE LEARNING FOR CLINICAL TIME SERIES Neura network methodoogy aside, a growing body of research appies machine earning to tempora cinica data for tasks incuding artifact remova (Aeks et a., 2009; Quinn et a., 2009), eary detection and prediction (Stancuescu et a., 2014a; Henry et a., 2015), and custering and subtyping (Marin et a., 2012; Schuam et a., 2015). Many recent papers use modes with atent factors to capture noninear dynamics in cinica time series and to discover meaningfu representations of heath and iness. Gaussian processes are popuar because they can directy hande irreguar samping and encode prior knowedge via choice of covariance functions between time steps and across variabes (Marin et a., 2012; Ghassemi et a., 2015). Saria et a. (2010) combined a hierarchica dirichet process with autoregressive modes to infer atent disease topics in the heart rate signas of premature babies. Quinn et a. (2009) used inear dynamica systems with atent switching variabes to mode physioogic events ike bradycardias. Seeking deeper modes, Stancuescu et a. (2014b) proposed a second ayer of atent factors to capture correations between atent states. 2.5 TARGET REPLICATION In this work, we make the task of cassifying entire sequences easier by repicating targets at every time step, inspired by Lee et a. (2015), who pace an optimization objective after each ayer in convoutiona neura network. Whie they have a separate set of weights to earn each intermediate objective, our mode is simper owing to the weight tying in recurrent nets, having ony one set of output weights. Additionay, unike Lee et a. (2015), we pace targets at each time step, but not foowing each ayer between input and output in the LSTM. After finishing this manuscript, we earned that target repication strategies simiar to ours have aso been deveoped by Ng et a. (2015) and Dai & Le (2015) for the tasks of video cassification and character-eve document cassification respectivey. Ng et a. (2015) ineary scae the importance of each intermediate target, emphasizing performance at ater sequence steps over those in the beginning of the cip. Dai & Le (2015) aso use a target repication strategy with ineary increasing weight for character-eve document cassification, showing significant improvements in accuracy. They ca this technique inear gain. 2.6 REGULARIZING RECURRENT NEURAL NETWORKS Given the compexity of our modes and modest scae of our data, reguarization, incuding judicious use of dropout, is crucia to our performance. Severa prior works use dropout to reguarize RNNs. Pham et a. (2014), Zaremba et a. (2014), and Dai & Le (2015) a describe an appication of dropout to ony the non-recurrent weights of a network. The former two papers estabish the method and appy it to tasks with sequentia outputs, incuding handwriting recognition, image captioning, and machine transation. The setting studied by Dai & Le (2015) most cosey resembes ours as the authors appy it to the task of appying static abes to varying ength sequences. 3

4 2.7 KEY DIFFERENCES Our experiments show that LSTMs can accuratey cassify mutivariate time series of cinica measurements, a topic not addressed in any prior work. Additionay, whie some papers use LSTMs for mutiabe cassification, ours is the first to address this probem in the medica context. Moreover, for mutiabe cassification of sequentia cinica data with fixed ength output vectors, this paper is the first, to our knowedge, to demonstrate the efficacy of a target repication strategy, achieving both faster training and better generaization. 3 DATA DESCRIPTION Our experiments use a coection of anonymized cinica time series extracted from the EHR system at Chidren s Hospita LA (Marin et a., 2012; Che et a., 2015) as part of an IRB-approved study. The data consists of 10, 401 PICU episodes, each a mutivariate time series of 13 variabes: diastoic and systoic bood pressure, periphera capiary refi rate, end-tida CO 2, fraction of inspired O 2, Gascow coma scae, bood gucose, heart rate, ph, respiratory rate, bood oxygen saturation, body temperature, and urine output. Episodes vary in ength from 12 hours to severa months. Each exampe consists of irreguary samped mutivariate time series with both missing vaues and, occasionay, missing variabes. We resampe a time series to an houry rate, taking the mean measurement within each one hour window. We use forward- and back-fiing to fi gaps created by the window-based resamping. When a singe variabe s time series is missing entirey, we impute a cinicay norma vaue as defined by domain experts. These procedures make reasonabe assumptions about cinica practice: many variabes are recorded at rates proportiona to how quicky they change, and when a variabe is absent, it is often because cinicians beieved it to be norma and chose not to measure it. Nonetheess, these procedures are not appropriate in a settings. Backfiing, for exampe, passes information from the future backwards. This is acceptabe for cassifying entire episodes (as we do) but not for forecasting. Finay, we rescae a variabes to [0, 1], using ranges defined by cinica experts. In addition, we use pubished tabes of norma vaues from arge popuation studies to correct for differences in heart rate, respiratory rate, (Feming et a., 2011) and bood pressure (NHBPEP Working Group 2004) due to age and gender. Each episode is associated with zero or more diagnostic codes from an in-house taxonomy used for research and biing, simiar to the Ninth Revision of the Internationa Cassification of Diseases (ICD-9) codes (Word Heath Organization, 2004). The dataset contains 429 distinct abes indicating a variety of conditions, such as acute respiratory distress, congestive heart faiure, seizures, rena faiure, and sepsis. Because many of the diagnoses are rare, we focus on the most common 128, each of which occurs more than 50 times in the data. These diagnostic codes are recorded by attending physicians during or shorty after each patient episode and subject to imited review afterwards. Because the diagnostic codes were assigned by cinicians, our experiments represent a comparison of an LSTM-based diagnostic system to human experts. We note that an attending physician has access to much more data about each patient than our LSTM does, incuding additiona tests, medications, and treatments. Additionay, the physician can access a fu medica history incuding free-text notes, can make visua and physica inspections of the patient, and can ask questions. A more fair comparison might require asking additiona cinica experts to assign diagnoses given access ony to the 13 time series avaiabe to our modes. However, this woud be prohibitivey expensive, even for just the 1000 exampes, and difficut to justify to our medica coaborators, as this annotation woud provide no immediate benefit to patients. Such a study wi prove more feasibe in the future when this ine of research has matured. 4 METHODS In this work, we are interested in recognizing diagnoses and, more broady, the observabe physioogic characteristics of patients, a task generay termed phenotyping (Oerich et a., 2015). We cast the probem of phenotyping cinica time series as mutiabe cassification. Given a series of observations x (1),..., x (T ), we earn a cassifier to generate hypotheses ŷ of the true abes y. Here, t indexes sequence steps, and for any exampe, T stands for the ength of the sequence. Our proposed LSTM RNN uses memory ces with forget gates (Gers et a., 2000) but without peephoe 4

5 connections (Gers et a., 2003). As output, we use a fuy connected ayer atop the highest LSTM ayer foowed by an eement-wise sigmoid activation function, because our probem is mutiabe. We use og oss as the oss function at each output. The foowing equations give the update for a ayer of memory ces h (t) where h (t) 1 stands for the previous ayer at the same sequence step (a previous LSTM ayer or the input x (t) ) and h (t 1) stands for the same ayer at the previous sequence step: g (t) i (t) f (t) o (t) = φ(w gx h (t) 1 + W gh = σ(w ix h (t) 1 + W ih = σ(w fx h (t) 1 + W fh = σ(w ox h (t) 1 + W oh s (t) = g (t) h (t) i (i) + s (t 1) h (t 1) + b g ) h (t 1) + b i ) h (t 1) + b f ) h (t 1) = φ(s (t) ) o (t). + b o ) f (t) In these equations, σ stands for an eement-wise appication of the sigmoid (ogistic) function, φ stands for an eement-wise appication of the tanh function, and is the Hadamard (eement-wise) product. The input, output, and forget gates are denoted by i, o, and f respectivey, whie g is the input node and has a tanh activation. 4.1 LSTM ARCHITECTURES FOR MULTILABEL CLASSIFICATION We expore severa recurrent neura network architectures for mutiabe cassification of time series. The first and simpest (Figure 1) passes over a inputs in chronoogica order, generating outputs ony at the fina sequence step. In this approach, we ony have output ŷ at the fina sequence step, at which our oss function is the average of the osses at each output node. Thus the oss cacuated at a singe sequence step is the average of og oss cacuated separatey on each abe. oss(ŷ, y) = 1 = L (y og(ŷ ) + (1 y ) og(1 ŷ )). L =1 Figure 1: A simpe RNN mode for mutiabe cassification. Green rectanges represent inputs. The recurrent hidden ayers separating input and output are represented with a singe bue rectange. The red rectange represents targets. 4.2 SEQUENTIAL TARGET REPLICATION One probem with the simpe approach is that the network must earn to pass information across many sequence steps in order to affect the output. We attack this probem by repicating our static targets at each sequence step (Figure 2), providing a oca error signa at each step. This approach is inspired by the deep supervision technique that Lee et a. (2015) appy to convoutiona nets. This technique is especiay sensibe in our case because we expect the mode to predict accuratey even if the sequence were truncated by a sma amount. The approach differs from Lee et a. (2015) because 5

6 we use the same output weights to cacuate ŷ (t) for a t. Further, we use this target repication to generate output at each sequence step, but not at each hidden ayer. For the mode with target repication, we generate an output ŷ (t) at every sequence step. Our oss is then a convex combination of the fina oss and the average of the osses over a steps: α 1 T T oss(ŷ (t), y (t) ) + (1 α) oss(ŷ (T ), y (T ) ) t=1 where T is the tota number of sequence steps and α [0, 1] is a hyper-parameter which determines the reative importance of hitting these intermediary targets. At prediction time, we take ony the output at the fina step. In our experiments, networks using target repication outperform those with a oss appied ony at the fina sequence step. Figure 2: An RNN cassification mode with target repication. The primary target (depicted in red) at the fina step is used at prediction time, but during training, the mode back-propagates errors from the intermediate targets (purpe) at every sequence step. 4.3 AUXILIARY OUTPUT TRAINING Reca that our initia data contained 429 diagnostic abes but that our task is to predict ony 128. Given the we-documented successes of mutitask earning with shared representations and feedforward networks, we wish to train a stronger mode by using the remaining 301 abes or other information in the patient s chart, such as diagnostic categories, as auxiiary targets (Caruana et a., 1996). These additiona targets serve reduce overfitting as the mode aims to minimize the oss on the abes of interest whie aso minimizing oss on the auxiiary targets (Figure 3). Figure 3: Our dataset contains many abes. For our task, a subset of 128 are of interest (depicted in red). Our Auiiary Output neura network makes use of extra abes as additiona training targets (depicted in purpe). At inference time we generate predictions for ony the abes of interest. 6

7 4.4 REGULARIZATION Because we have ony 10, 401 exampes, overfitting is a considerabe obstace. Our experiments show that both target repication and auxiiary outputs improve performance and reduce overfitting. In addition to these ess common techniques we depoy 2 2 weight decay and dropout. Foowing the exampe of Zaremba et a. (2014) and Pham et a. (2014), we appy dropout to the non-recurrent connections ony. We first compute each hidden ayer s sequence of activations in the eft-to-right direction and then appy dropout before computing the next ayer s activations. In our experiments, we find that dropout decreases overfitting, enabing us to doube the size of each hidden ayer. 5 EXPERIMENTS A modes are trained on 80% of the data and tested on 10%. The remaining 10% is used as a vaidation set. We train each LSTM for 100 epochs using stochastic gradient descent (SGD) with momentum. To combat expoding gradients, we scae the norm of the gradient and use 2 2 weight decay of 10 6, both hyperparameters chosen using vaidation data. Our fina networks use 2 hidden ayers and either 64 memory ces per ayer with no dropout or 128 ces per ayer with dropout of 0.5. These architectures are aso chosen based on vaidation performance. Throughout training, we save the mode and compute three performance metrics (micro AUC, micro F1, and precision at 10) on the vaidation set for each epoch. We then test the mode that scores best on at east two of the three vaidation metrics. To break ties, we choose the earier epoch. We evauate a number of baseines as we as LSTMs with various combinations of target repication (TR), dropout (DO), and auxiiary outputs (AO), using either the additiona 301 diagnostic abes or 12 diagnostic categories. To expore the reguarization effects of each strategy, we record and pot both training and vaidation performance after each epoch. Additionay, we report performance of a target repication mode (Linear Gain) that scaes the weight of each intermediate target ineary as opposed our proposed approach. Finay, to show that our LSTM earns a mode compementary to the baseines, we evauate an ensembe of the best LSTM with the best baseine. 5.1 MULTILABEL EVALUATION METHODOLOGY We report micro- and macro-averaged versions of Area Under the ROC Curve (AUC). By micro AUC, we mean a singe AUC computed on fattened Ŷ and Y matrices, whereas we cacuate macro AUC by averaging each per-abe AUC. The bind cassifier achieves 0.5 macro AUC but can exceed 0.5 on micro AUC by predicting abes in descending order by base rate. Additionay, we report micro- and macro-averaged F1 score, computed in simiar fashion to the respective micro and macro AUCs. F1 metrics require a threshoding strategy, and here we seect threshods based upon vaidation set performance. We refer to Lipton et a. (2014) for an anaysis of the strengths and weaknesses of each type of mutiabe F-score and a characterization of optima threshods. Finay, we report precision at 10, which captures the fraction of true diagnoses among the mode s top 10 predictions, with a best possibe score of on the test spit of this data set because there are on average diagnoses per patient. Whie F1 and AUC are both usefu for determining the reative quaity of a cassifier s predictions, neither is taiored to a rea-word appication. Thus, we consider a medicay pausibe use case to motivate this more interpretabe metric: generating a short ist of the 10 most probabe diagnoses. If we coud create a high reca, moderate precision ist of 10 ikey diagnoses, it coud be a vauabe hint-generation too for differentia diagnosis. Testing for ony the 10 most probabe conditions is much more reaistic than testing for a conditions. 5.2 BASELINE CLASSIFIERS We provide resuts for a base rate mode that predicts diagnoses in descending order by incidence to provide a minimum performance baseine for micro-averaged metrics. We aso report the performance of ogistic regression, which is widey used in cinica research. We train a separate cassifier for each diagnosis but choose an overa 2 2 penaty for a individua cassifiers based on vaidation performance. For a much stronger baseine, we train a mutiabe MLP with 3 hidden ayers of 300 hidden units each, rectified inear activations, and dropout of 0.5. A MLPs were trained for 1000 epochs, with hyperparameters chosen based on vaidation set performance. Each baseine is tested 7

8 with two sets of inputs: raw time series and hand-engineered features. For raw time series, we use the first and ast six hours. This provides cassifiers with tempora information about changes in patient state from admission to discharge within a fixed-size input, as required by a baseines. We find this works better than providing the first or ast 12 hours aone. Our hand-engineered features are inspired by those used in state-of-the-art severity of iness scores (Poack et a., 1996): for each variabe, we compute the first and ast measurements and their difference scaed by episode ength, mean and standard deviation, median and quarties, minimum and maximum, and sope of a ine fit with east squares. These 143 features capture many of the indicators that cinicians ook for in critica iness, incuding admission and discharge state, extremes, centra tendencies, variabiity, and trends. They previousy have been shown to be effective for these data (Marin et a., 2012; Che et a., 2015). Our strongest baseine is an MLP using these features. 5.3 RESULTS Our best performing LSTM (LSTM-DO-TR) used two ayers of 128 memory ces, dropout of probabiity 0.5 between ayers, and target repication, and outperformed the MLP with hand-engineered features. Moreover simpe ensembes of the best LSTM and MLP outperformed both on a metrics. Tabe 1 shows summary resuts for a modes. Tabe 2 shows the LSTM s predictive performance for six diagnoses with the highest F1 scores. Fu per-diagnosis resuts can be found in Appendix C. Target repication improves performance on a metrics, acceerating earning and reducing overfitting (Figure 4). We aso find that the LSTM with target repication earns to output correct diagnoses earier in the time series, a virtue that we expore quaitativey in Appendix A. As a comparison, we trained a LSTM-DO-TR variant using the inear gain strategy of Ng et a. (2015); Dai & Le (2015). In genera, this mode did not perform as we as our simper target repication strategy, but it did achieve the highest macro F1 score among the LSTM modes. Cassification performance for 128 ICU phenotypes Mode Micro AUC Macro AUC Micro F1 Macro F1 Prec. at 10 Base Rate Logistic Regression, First 6 + Last Logistic Regression, Expert features MLP, First 6 + Last MLP, Expert features LSTM Modes with two 64-ce hidden ayers LSTM LSTM, AuxOut (Diagnoses) LSTM-AO (Categories) LSTM-TR LSTM-TR-AO (Diagnoses) LSTM-TR-AO (Categories) LSTM Modes with Dropout (probabiity 0.5) and two 128-ce hidden ayers LSTM-DO LSTM-DO-AO (Diagnoses) LSTM-DO-AO (Categories) LSTM-DO-TR LSTM-DO-TR-AO (Diagnoses) LSTM-DO-TR-AO (Categories) LSTM-DO-TR (Linear Gain) Ensembes of Best MLP and Best LSTM Mean of LSTM-DO-TR & MLP Max of LSTM-DO-TR & MLP Tabe 1: Resuts on performance metrics cacuated across a abes. DO, TR, and AO indicate dropout, target repication, and auxiiary outputs, respectivey. AO (Diagnoses) uses the extra diagnosis codes and AO (Categories) uses diagnostic categories as additiona targets during training. Auxiiary outputs improved performance for most metrics and reduced overfitting. Whie the performance improvement is not as dramatic as that conferred by target repication, the reguarizing effect is greater. These gains came at the cost of sower training: the auxiiary output modes required more epochs (Figure 4 and Appendix B), especiay when using the 301 remaining diagnoses. This 8

9 may be due in part to severe cass imbaance in the extra abes. For many of these abes it may take an entire epoch just to earn that they are occasionay nonzero. Top 6 diagnoses measured by F1 score Labe F1 AUC Precision Reca Diabetes meitus with ketoacidosis Scoiosis, idiopathic Asthma, unspecified with status asthmaticus Neopasm, brain, unspecified Deayed miestones Acute Respiratory Distress Syndrome (ARDS) Tabe 2: LSTM-DO-TR performance on the 6 diagnoses with highest F1 scores. The LSTMs appear to earn modes compementary to the MLP trained on hand-engineered features. Supporting this caim, simpe ensembes of the LSTM-DO-TR and MLP (taking the mean or maximum of their predictions) outperform the constituent modes significanty on a metrics (Tabe 1). Further, there are many diseases for which one mode substantiay outperforms the other, e.g., intracrania hypertension for the LSTM, septic shock for the MLP (Appendix C). 6 DISCUSSION Our resuts indicate that LSTM RNNs, especiay with target repication, can successfuy cassify diagnoses of critica care patients given cinica time series data. The best LSTM beat a strong MLP baseine using hand-engineered features as input, and an ensembe combining the MLP and LSTM improves upon both. The success of target repication accords with resuts by both Ng et a. (2015) and Dai & Le (2015), who observed simiar benefits on their respective tasks. However, whie they saw improvement using a ineary increasing weight on each target from start to end, this strategy performed worse in our diagnostic cassification task than our uniform weighting of intermediate targets. We beieve this may owe to the pecuiar nature of our data. Linear gain emphasizes evidence from ater in the sequence, an assumption which often does not match the progression of symptoms in critica inesses. Asthma patients, for exampe, are often admitted to the ICU severey symptomatic, but once treatment begins, patient physioogy stabiizes and observabe signs of disease may abate or change. Further supporting this idea, we observed that when training fixed-window baseines, using the first 6 and ast 6 hours outperformed using the ast 12 hours ony. Whie our data is of arge scae by cinica standards, it is sma reative to datasets found in deep earning tasks ike vision and speech recognition. At this scae, reguarization is critica. Our experiments demonstrate that target repication, auxiiary outputs, and dropout a work to reduce the generaization gap. as shown in Figure 4 and Appendix B. However, some of these techniques are compementary whie others seem to cance each other out. For exampe, our best mode combined target repication with dropout. This combination significanty improved upon the performance using target repication aone, and enabed the effective use of arger capacity modes. In contrast, the benefits of dropout and auxiiary output training appear to wash each other out. This Figure 4: Training curves showing the impact of the DO, AO, and TR strategies on overfitting. may be because target repication confers more than reguarization, mitigating the difficuty of earning ong range dependencies by providing oca objectives. 9

10 7 CONCLUSION Whie our LSTMs produce promising resuts, this is ony a first step in this ine of research. Recognizing diagnoses given fu time series of sensor data demonstrates that LSTMs can capture meaningfu signa, but utimatey we woud ike to predict deveoping conditions and events, outcomes such as mortaity, and treatment responses. In this paper we used diagnostic abes without timestamps, but we are obtaining timestamped diagnoses, which wi enabe us to train modes to perform eary diagnosis by predicting future conditions. In addition, we are extending this work to a arger PICU data set with 50% more patients and hundreds of variabes, incuding treatments and medications. On the methodoogica side, we woud ike to both better expoit and improve the capabiities of LSTMs. Resuts from speech recognition have shown that LSTMs shine in comparison to other modes using raw features, minimizing need for preprocessing and feature engineering. In contrast, our current data preparation pipeine removes vauabe structure and information from cinica time series that coud be expoited by an LSTM. For exampe, our forward- and back-fiing imputation strategies discard usefu information about when each observation is recorded. Imputing norma vaues for missing time series ignores the meaningfu distinction between truy norma and missing measurements. Aso, our window-based resamping procedure reduces the variabiity of more frequenty measured vita signs (e.g., heart rate). In future work, we pan to introduce indicator variabes to aow the LSTM to distinguish actua from missing or imputed measurements. Additionay, the fexibiity of the LSTM architecture shoud enabe us to eiminate age-based corrections and to incorporate non-sequentia inputs, such as age, weight, and height (or even hand-engineered features), into predictions. Other next steps in this direction incude deveoping LSTM architectures to directy hande missing vaues and irreguar samping. We aso are encouraged by the success of target repication and pan to expore other variants of this technique and to appy it to other domains and tasks. Additionay, we acknowedge that there remains a debate about the interpretabiity of neura networks when appied to compex medica probems. We are deveoping methods to interpret the representations earned by LSTMs in order to better expose patterns of heath and iness to cinica users. We aso hope to make practica use of the distributed representations of patients for tasks such as patient simiarity search. 8 ACKNOWLEDGEMENTS Zachary C. Lipton was supported by the Division of Biomedica Informatics at the University of Caifornia, San Diego, via training grant (T15LM011271) from the NIH/NLM. David Kae was supported by the Afred E. Mann Innovation in Engineering Doctora Feowship. The VPICU was supported by grants from the Laura P. and Leand K. Whittier Foundation. We acknowedge NVIDIA Corporation for Tesa K40 GPU hardware donation and Professors Juian McAuey and Greg Ver Steeg for their support and advice. REFERENCES Aeks, Norm, Russe, Stuart J, Madden, Michae G, Morabito, Diane, Staudenmayer, Kristan, Cohen, Mitche, and Maney, Geoffrey T. Probabiistic detection of short events, with appication to critica care monitoring. In Advances in Neura Information Processing Systems (NIPS) 21, pp , Amari, Shun-ichi and Cichocki, Andrzej. Adaptive bind signa processing-neura network approaches. Proceedings of the IEEE, 86(10): , Aui, Michae, Gaey, Miche, Quirk, Chris, and Zweig, Geoffrey. Joint anguage and transation modeing with recurrent neura networks. In Empirica Methods in Natura Language Proessing (EMNPL), voume 3, Baxt, W.G. Appication of artificia neura networks to cinica medicine. The Lancet, 346(8983): ,

11 Caruana, Rich, Bauja, Shumeet, Mitche, Tom, et a. Using the future to sort out the present: Rankprop and mutitask earning for medica risk evauation. In Advances in Neura Information Processing Systems (NIPS) 8, pp , Che, Zhengping, Kae, David C., Li, Wenzhe, Bahadori, Mohammad Taha, and Liu, Yan. Deep computationa phenotyping. In Proceedings of the 21th ACM SIGKDD Internationa Conference on Knowedge Discovery and Data Mining (KDD), pp ACM, Dabek, Fiip and Caban, Jesus J. A neura network based mode for predicting psychoogica conditions. In Brain Informatics and Heath, pp Springer, Dai, Andrew M and Le, Quoc V. Semi-supervised sequence earning. In Advances in Neura Information Processing Systems (NIPS) 28, pp , De Muder, Wim, Bethard, Steven, and Moens, Marie-Francine. A survey on the appication of recurrent neura networks to statistica anguage modeing. Computer Speech & Language, 30(1): 61 98, Eman, Jeffrey L. Finding structure in time. Cognitive Science, 14(2): , Feming, Susannah, Thompson, Matthew, Stevens, Richard, Heneghan, Car, Pddemann, Annette, Maconochie, Ian, Tarassenko, Lione, and Mant, David. Norma ranges of heart rate and respiratory rate in chidren from birth to 18 years: A systematic review of observationa studies. The Lancet, pp , Gers, Feix and Schmidhuber, Jürgen. Recurrent nets that time and count. In Proceedings of the IEEE-INNS-ENNS Internationa Joint Conference on Neura Networks (IJCNN), voume 3, pp IEEE, Gers, Feix A., Schmidhuber, Jürgen, and Cummins, Fred. Learning to forget: Continua prediction with LSTM. Neura Computation, 12(10): , Gers, Feix A., Schraudoph, Nico N., and Schmidhuber, Jürgen. Learning precise timing with LSTM recurrent networks. The Journa of Machine Learning Research, 3: , Ghassemi, Marzyeh, Pimente, Marco AF, Naumann, Tristan, Brennan, Thomas, Cifton, David A, Szoovits, Peter, and Feng, Menging. A mutivariate timeseries modeing approach to severity of iness assessment and forecasting in ICU with sparse, heterogeneous cinica data. In Proceedings of the Twenty-Ninth AAAI Conference on Artificia Inteigence, Graves, Aex. Supervised Sequence Labeing with Recurrent Neura Networks, voume 385 of Studies in Computationa Inteigence. Springer-Verag Berin Heideberg, Graves, Aex. Generating sequences with recurrent neura networks. arxiv preprint arxiv: , Graves, Aex, Liwicki, Marcus, Fernández, Santiago, Bertoami, Roman, Bunke, Horst, and Schmidhuber, Jürgen. A nove connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 31(5): , Hammera, Nis Y, Fisher, James M, Andras, Peter, Rochester, Lynn, Waker, Richard, and Pötz, Thomas. PD disease state assessment in naturaistic environments using deep earning. In Proceedings of the Twenty-Ninth AAAI Conference on Artificia Inteigence, Henry, Katharine E, Hager, David N, Pronovost, Peter J, and Saria, Suchi. A targeted rea-time eary warning score (trewscore) for septic shock. Science Transationa Medicine, 7( ra122): 1 9, Hochreiter, Sepp and Schmidhuber, Jürgen. Long short-term memory. Neura Computation, 9(8): , Karpathy, Andrej and Fei-Fei, Li. Deep visua-semantic aignments for generating image descriptions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp , June

12 Lasko, Thomas A., Denny, Joshua C., and Levy, Mia A. Computationa phenotype discovery using unsupervised feature earning over noisy, sparse, and irreguar cinica data. PLoS ONE, 8(6): e66341, Lee, Chen-Yu, Xie, Saining, Gaagher, Patrick, Zhang, Zhengyou, and Tu, Zhuowen. Deepysupervised nets. In Proceedings of the Eighteenth Internationa Conference on Artificia Inteigence and Statistics (AISTATS), Lipton, Zachary C, Ekan, Chares, and Naryanaswamy, Baakrishnan. Optima threshoding of cassifiers to maximize F1 measure. In Machine Learning and Knowedge Discovery in Databases, pp Springer, Lipton, Zachary C., Berkowitz, John, and Ekan, Chares. A critica review of recurrent neura networks for sequence earning. arxiv preprint arxiv: , Liu, I, Ramakrishnan, Bhiksha, et a. Bach in 2014: Music composition with recurrent neura network. arxiv preprint arxiv: , Liwicki, Marcus, Graves, Aex, Bunke, Horst, and Schmidhuber, Jürgen. A nove approach to on-ine handwriting recognition based on bidirectiona ong short-term memory networks. In Proceedings of the Ninth Internationa Conference on Document Anaysis and Recognition, voume 1, pp , Marin, Ben M., Kae, David C., Khemani, Robinder G., and Wetze, Randa C. Unsupervised pattern discovery in eectronic heath care data using probabiistic custering modes. In Proceedings of the 2nd ACM SIGHIT Internationa Heath Informatics Symposium (IHI), Mason, Robert J., Broaddus, V. Courtney, Martin, Thomas, King Jr., Tamadge E., Schraufnage, Dean, Murray, John F., and Nade, Jay A. Murray and Nade s textbook of respiratory medicine: 2-voume set. Esevier Heath Sciences, Nationa High Bood Pressure Education Program Working Group on Chidren and Adoescents. The fourth report on the diagnosis, evauation, and treatment of high bood pressure in chidren and adoescents. Pediatrics, 114: , Ng, Joe Yue-Hei, Hausknecht, Matthew, Vijayanarasimhan, Sudheendra, Vinyas, Orio, Monga, Rajat, and Toderici, George. Beyond short snippets: Deep networks for video cassification. arxiv preprint arxiv: , Oerich, Anika, Coier, Nige, Groza, Tudor, Rebhoz-Schuhmann, Dietrich, Shah, Nigam, Bodenreider, Oivier, Boand, Mary Regina, Georgiev, Ivo, Liu, Hongfang, Livingston, Kevin, Luna, Augustin, Maon, Ann-Marie, Manda, Prashanti, Robinson, Peter N., Rustici, Gabriea, Simon, Michee, Wang, Liqin, Winnenburg, Rainer, and Dumontier, Miche. The digita revoution in phenotyping. Briefings in Bioinformatics, Pascanu, Razvan, Gucehre, Cagar, Cho, Kyunghyun, and Bengio, Yoshua. How to construct deep recurrent neura networks. In Internationa Conference on Learning Representations (ICLR), Pham, Vu, Buche, Théodore, Kermorvant, Christopher, and Louradour, Jérôme. Dropout improves recurrent neura networks for handwriting recognition. In Frontiers in Handwriting Recognition (ICFHR), th Internationa Conference on, pp IEEE, Poack, M. M., Pate, K. M., and Ruttimann, U. E. PRISM III: an updated Pediatric Risk of Mortaity score. Critica Care Medicine, 24(5): , Poastri, Gianuca, Przybyski, Darisz, Rost, Burkhard, and Badi, Pierre. Improving the prediction of protein secondary structure in three and eight casses using recurrent neura networks and profies. Proteins: Structure, Function, and Bioinformatics, 47(2): , Quinn, John, Wiiams, Christopher KI, McIntosh, Nei, et a. Factoria switching inear dynamica systems appied to physioogica condition monitoring. IEEE Transactions on Pattern Anaysis and Machine Inteigence, 31(9): ,

13 Rughani, Anand I., Dumont, Travis M., Lu, Zhenyu, Bongard, Josh, Horgan, Michae A., Penar, Pau L., and Tranmer, Bruce I. Use of an artificia neura network to predict head injury outcome: cinica artice. Journa of Neurosurgery, 113(3): , Rumehart, David E, Hinton, Geoffrey E, and Wiiams, Ronad J. Learning interna representations by error propagation. Technica report, DTIC Document, Saria, Suchi, Koer, Daphne, and Penn, Anna. Learning individua and popuation eve traits from cinica tempora data. In Proc. Neura Information Processing Systems (NIPS), Predictive Modes in Personaized Medicine Workshop. Citeseer, Schuam, Peter, Wigey, Fredrick, and Saria, Suchi. Custering ongitudina cinica marker trajectories from eectronic heath data: Appications to phenotyping and endotype discovery. In Proceedings of the Twenty-Ninth AAAI Conference on Artificia Inteigence, Siipo, Rosaria and Marchesi, Caro. Artificia neura networks for automatic ecg anaysis. IEEE Transactions on Signa Processing, 46(5): , Stancuescu, Ioan, Wiiams, Christopher K, Freer, Yvonne, et a. Autoregressive hidden markov modes for the eary detection of neonata sepsis. Biomedica and Heath Informatics, IEEE Journa of, 18(5): , 2014a. Stancuescu, Ioan, Wiiams, Christopher KI, and Freer, Yvonne. A hierarchica switching inear dynamica system appied to the detection of sepsis in neonata condition monitoring. In Proceedings of the 30th Conference on Uncertainty in Artificia Inteigence (UAI), 2014b. Sutskever, Iya, Vinyas, Orio, and Le, Quoc VV. Sequence to sequence earning with neura networks. In Advances in Neura Information Processing Systems (NIPS) 27, pp , Tresp, Voker and Briege, Thomas. A soution for missing data in recurrent neura networks with an appication to bood gucose prediction. In Advances in Neura Information Processing Systems (NIPS) 10, pp Übeyi, Eif Derya. Combining recurrent neura networks with eigenvector methods for cassification of ecg beats. Digita Signa Processing, 19(2): , Vinyas, Orio, Toshev, Aexander, Bengio, Samy, and Erhan, Dumitru. Show and te: A neura image caption generator. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp , June Vohradský, Jiří. Neura network mode of gene expression. The FASEB Journa, 15(3): , Word Heath Organization. Internationa statistica cassification of diseases and reated heath probems, voume 1. Word Heath Organization, Xu, Rui, Wunsch II, Donad, and Frank, Ronad. Inference of genetic reguatory networks with recurrent neura network modes using partice swarm optimization. IEEE/ACM Transactions on Computationa Bioogy and Bioinformatics (TCBB), 4(4): , Yeung, Serena, Russakovsky, Oga, Jin, Ning, Andriuka, Mykhayo, Mori, Greg, and Fei-Fei, Li. Every moment counts: Dense detaied abeing of actions in compex videos. arxiv preprint arxiv: , Zaremba, Wojciech, Sutskever, Iya, and Vinyas, Orio. Recurrent neura network reguarization. arxiv preprint arxiv: ,

14 Appendices A HOURLY DIAGNOSTIC PREDICTIONS Our LSTM networks predict 128 diagnoses given sequences of cinica measurements. Because each network is connected eft-to-right, i.e., in chronoogica order, we can output predictions at each sequence step. Utimatey, we imagine that this capabiity coud be used to make continuousy updated rea-time aerts and diagnoses. Beow, we expore this capabiity quaitativey. We choose exampes of patients with a correcty cassified diagnosis and visuaize the probabiities assigned by each LSTM mode at each sequence step. In addition to improving the quaity of the fina output, the LSTMs with target repication (LSTM-TR) arrive at correct diagnoses quicky compared to the simpe mutiabe LSTM mode (LSTM-Simpe). When auxiiary outputs are aso used (LSTM- TR,AO), the diagnoses appear to be generay more confident. (a) Asthma with Status Asthmaticus (b) Acute Respiratory Distress Syndrome (c) Diabetic Ketoacidosis (d) Brain Neopasm, Unspecified Nature (e) Septic Shock (f) Scoiosis Figure 5: Each chart depicts the probabiities assigned by each of four modes at each (houry resamped) time step. LSTM-Simpe uses ony targets at the fina time step. LSTM-TR uses target repication. LSTM-AO uses auxiiary outputs (diagnoses), and LSTM-TR,AO uses both techniques. LSTMs with target repication earn to make accurate diagnoses earier. 14

15 Our LSTM-TR,AO effectivey predicts status asthmaticus and acute respiratory distress syndrome, ikey owing to the severa measures of pumonary function among our inputs. Diabetic ketoacidosis aso proved easy to diagnose, ikey because gucose and ph are incuded among our cinica measurements. We were surprised to see that the network cassified scoiosis reiaby, but a deeper ook into the medica iterature suggests that scoiosis often resuts in respiratory symptoms. This anaysis of step-by-step predictions is preiminary and informa, and we note that for a sma number of exampes our data preprocessing introduces a target eak by back-fiing missing vaues. In future work, when we expore this capabiity in greater depth, we wi reprocess the data. B LEARNING CURVES We present visuaizations of the performance of LSTM, LSTM-DO (with dropout probabiity 0.5), LSTM-AO (using the 301 additiona diagnoses), and LSTM-TR (with α = 0.5), during training. These charts are usefu for examining the effects of dropout, auxiiary outputs, and target repication on both the speed of earning and the reguarization they confer. Specificay, for each of the four modes, we pot the training and vaidation micro AUC and F1 score every five epochs in Figure 6. Additionay, we pot a scatter of the performance on the training set vs. the performance on the vaidation set. The LSTM with target repication earns more quicky than a simpe LSTM and aso suffers ess overfitting. With both dropout and auxiiary outputs, the LSTM trains more sowy than a simpe LSTM but suffers consideraby ess overfitting. (a) AUC earning curves (b) F1 earning curves (c) AUC training vs. vaidation (d) F1 training vs. vaidation Figure 6: Training and vaidation performance potted for the simpe mutiabe network (LSTM- Simpe), LSTM with target repication (LSTM-TR), and LSTM with auxiiary outputs (LSTM-AO). Target repication appears to increase the speed of earning and confers a sma reguarizing effect. Auxiiary outputs sow down the speed of earning but impart a strong reguarizing effect. 15