Multi-modal Patient Cohort Identification from EEG Report and Signal Data

Transcription

1 Multi-modal Patient Cohort Identification from EEG Report and Signal Data Travis R. Goodwin and Sanda M. Harabagiu The University of Texas at Dallas Human Language Technology Research Institute

2 Conflicts Nothing to disclose.

3 Presentation Outline 1. Introduction 2. Data 3. Methods 4. Evaluation 5. Conclusions

4 Introduction: The Problem Background: An electroencephalogram (EEG) measures the electrical activity of the brain. EEGs are an important investigational tool in the diagnosis and management of epilepsies and other types of brain disorders. Problem: EEG interpretation is known to have only moderate agreement (Beniczky et al., 2013).

5 Introduction: The Solution Solution: Leverage Big Data of EEG reports Improve interpretation agreement by allowing neurologists to search for patients that exhibit similar EEG characteristics Automatically identifying patient cohorts can inform the clinical decision of neurologists enable comparative clinical effectiveness research

6 Introduction: Examples Scenario 1: Neurologist suspects their patient has epilepsy potential and wants to review similar cases Query: Patients with a history of seizures and EEG with TIRDA without sharps, spikes, or electrographic seizures. Temporal Intermittent Rhythmic Delta Activity Scenario 2: Neurologist wants to investigate effective interventions for epilepsy accompanied by mental health disorders. Query: Patients with a history of Alzheimer and abnormal EEG

7 Multi-modal Encephalogram Patient Cohort Discovery (MERCuRY) Goal: Identify patients satisfying cohort criteria expression in a natural language query. MERCuRY System: Considers both the EEG report as well as the signal data Natural language processing to identify inclusion/exclusion criteria in the query Deep learning to represent EEG signals and produce a multi-modal index of EEG information Ranks patients using relevance models

8 Presentation Outline 1. Introduction 2. Data 3. Methods 4. Evaluation 5. Conclusions

9 Data: TUH EEG Corpus Temple University Hospital (TUH) EEG Corpus Largest publically available dataset of EEG data 25,000 EEG sessions 15,000 patients Collected over 12 years Contains both EEG Reports and EEG signal data

10 Data: EEG Reports American Clinical Neurophysiology Society (ACNS) Guidelines for writing EEG reports Clinical History: patients age, gender, relevant medical conditions and medications Introduction: EEG technique/configuration digital video EEG, standard system with 1 channel EKG Description: describes any notable waveform activity, patterns, or EEG events sharp wave, burst suppression pattern, very quick jerks of the head Impression: interpretation of whether the EEG indicates normal or abnormal brain activity, as well as a list of contributing epileptiform phenomena abnormal EEG due to background slowing Clinical correlation: relates the EEG findings to the over-all clinical picture of the patient very worrisome prognostic features

11 Data: EEG Signals Each report is associated with its EEG signal recording EEG signals contain 24 to 36 channels with an additional annotation channel identifying events of interest to the physicians and/or technicians Sampled at a rate of 250 Hz or 256 Hz using 16-bits per sample Each EEG recording contains roughly 20 MB of raw data! Uses the European Data Format (EDF+) schema

12 Presentation Outline 1. Introduction 2. Data 3. Methods 4. Evaluation 5. Conclusions

13 Methods: Overview Queries are processed by natural language analysis: term filtering, query formulation, query expansion EEG reports are identified by a relevance model Case 1: EEG report ONLY Case 2: EEG report + EEG signal (Multi-modal) Relevance model relies on multi-modal Index Term/phrase dictionary, tiered inverted lists, EEG signal fingerprints EEG Report Processing: section identification, medical language processing EEG Signal Processing: deep neural learning: fingerprint detection Query Relevance Model Index

14 Methods: MERCuRY System Overview EEG Cohort Description Analysis 1. Term Filtering 2. Query Formulation 3. Query Expansion Relevance Model Case 1 Case 2 Patient Cohort CASE 1 EEG Report EEG Report Processing 1. Section identification 2. Medical language processing Term / Phrase Dictionary Tiered Inverted Lists Patient Cohort CASE 2 EEG Signal EEG Signal Processing Deep Neural Network EEG Signal fingerprints Multi-Modal EEG INDEX

15 Methods: Indexing EEG Reports Concept dictionary Identified 5 types of medical concepts: PROBLEMS (e.g. seizure ) TREATMENTS (e.g. Topamax ) TESTS (e.g. video EEG ) EEG PATTERNS/ACTIVITIES (e.g. focal slowing, polyspike ) EEG EVENTS (e.g. blink, jerk of the head ) Links each medical concept to the terms expressing it: e.g., spike and slow wave spike, and, slow, wave

16 Methods: Indexing EEG Reports Term Dictionary Each term is associated with two inverted lists positive polarity: mentions in which the term is positive negative polarity: mentions in which the term is negated Each cell in the inverted list contains: the EEG report containing the mention the name of the section containing the mention the position of the mention within the section (e.g. term offset) position(s) of the term in any associated medical concepts the EEG signal fingerprint associated with the EEG report

17 TERM DICTIONARY Overview of the Tiered Multi-Modal Index term ID term ID term ID. term ID term ID term ID term ID term ID term ID. term ID alpha beta hypertension. lovenox seizure sharp slow spike stroke. wave POSITIVE POLARITY NEGATIVE POLARITY Tiered Inverted Lists EEG Report ID EEG Signal Fingerprint ID Report Section Report Section Position Medical Concept ID Concept Position EEG Report ID EEG Signal Fingerprint ID Report Section Report Section Position Medical Concept ID Concept Position Next EEG Signal fingerprints Next Medical Concept ID. Medical Concept ID Concept Type. Concept Type alpha. Sharp and slow wave term ID. term ID Medical Concept DICTIONARY

18 Methods: Fingerprinting EEG Signals EEG signal encoded as a dense floating-point matrix, D R N,L N is the number of electrode channels L is the number of samples in the recording (e.g. L / 250 = duration of the recording in seconds) One pass over the EEG signals requires considering over 1.8 terabytes of information! We need a more compact representation: EEG fingerprints

19 Methods: Learning EEG Fingerprints Deep neural learning Process EEG signals in a matter of hours rather than weeks Reduce each EEG signal from 20 MB to a few hundred bytes Recurrent Neural Network Consider the EEG signal as a sequence of samples For each sample x t, learn to predict the next sample x t+1 Long Short-Term Memory Can learn long-range interactions in the EEG signal Maintains & updates an internal memory m t Final internal memory m L becomes the EEG fingerprint

20 Methods: Relevance Model Purpose: measure the relevance between a query and an EEG report Case 1: consider EEG reports only BM25F ranking function Gives a different weight to query matches in each field, and for each polarity Case 2: consider EEG report + EEG fingerprint In our experiments, Retrieve initial set of EEG reports as in Case 1 λ = 5; δ = 3 Identify the λ top-ranked EEG reports Lookup the δ most-similar EEG fingerprints for top-ranked reports

21 Presentation Outline 1. Introduction 2. Data 3. Methods 4. Evaluation 5. Conclusions

22 Evaluation: Queries Asked neurologists to provide patient cohort descriptions (queries) Patient Cohort Description (Queries) 1. History of seizures and EEG with TIRDA without sharps, spikes, or electrographic seizures 2. History of Alzheimer dementia and normal EEG 3. Patients with altered mental status and EEG showing nonconvulsive status epilepticus (NCSE) 4. Patients under 18 years old with absence seizures 5. Patients over age 18 with history of developmental delay and EEG with electrographic seizures

23 Evaluation: Patient Cohort Quality For each query Identified the 10 most relevant EEG reports Random sample of 10 EEG reports retrieved between ranks 11 and 100. Asked neurologists to judge whether each EEG report was relevant : 1: the patient described in the report definitely belongs to the cohort 0: the patient described in the report does not belong to the cohort Measured using standard information retrieval metrics: Mean Average Precision (MAP) Normalized Discounted Cumulative Gain (NDCG) Precision at Rank 10 (P@10)

24 Evaluation: Patient Cohort Quality Relevance Model MAP NDCG 10 Baseline 1: BM % 66.41% 80.00% Baseline 2: LMD 50.37% 65.90% 80.00% Baseline 3: DFR 46.22% 59.35% 70.00% MERCuRY: Case 1 (a) 58.59% 72.14% 90.00% MERCuRY: Case 1 (b) 57.95% 70.34% 90.00% MERCuRY: Case 2 (a) 70.43% 84.62% % MERCuRY: Case 2 (b) 69.87% 83.21% % (a) With concept dictionary (b) Without concept dictionary

25 Presentation Outline 1. Introduction 2. Data 3. Methods 4. Evaluation 5. Conclusions

26 Conclusions Including EEG signals with EEG reports lead to improved patient cohort retrieval No noticeable improvement when indexing medical concepts: can recognize concepts in the query only maintaining positional information is sufficient to identify medical concepts in EEG reports Considering polarity & section information substantially improved performance EEG fingerprints are able to fill in the gaps in EEG reports Future work: Tune parameters δ and λ Jointly encode EEG reports + EEG signals

27 Acknowledgements Research reported in this publication was supported by the National Human Genome Research Institute of the National Institutes of Health under award number 1U01HG The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

28 Questions?

29 Background: Multi-modal Retrieval Text REtrieval Conference (TREC) Medical Records Track 2011 & 2012: systems were given natural language queries and text-only EHRs Goal was to retrieve sets of electronic medical records relevant to the query Biomedical Multi-modal Retrieval (Demner-Fushman et al, 2012) Considers both images and text of scientific articles Allows users to discover similar images to those used in a scientific article AALIM (Syeda-Mahmood et al, 2007) Allows users to discover similar ECG, echo, or audio recordings given the ECG, echo, or audio recording of a patient