Using Electronic Medical Records to Identify Complex Health Outcomes

Transcription

1 Using Electronic Medical Records to Identify Complex Health Outcomes Mary Anne Armstrong MA; Maqdooda Merchant, MA, MSc; Amy Alabaster, MS, MPH; Tina Raine-Bennett, MD, MPH; Debbie Postlethwaite RNP, MPH Division of Research, Oakland, CA HCSRN April 12, 2018 Kaiser Permanente, Division of Research

2 Background Electronic Medical Records (EMRs) have been used successfully in health research to identify outcomes and risk factors in a cohort of interest. Uterine perforation and IUD expulsion are complex health outcomes that are not easily captured using administrative and claims data. Can algorithms be developed to identify these 2 outcomes using clinical data from the EMR? No publications could be identified that validated these outcomes in EMR-associated databases. 2

3 Purpose To validate algorithms using a combination of ICD-9-CM codes, CPT codes and natural language processing (NLP) of clinical notes to identify uterine perforations and IUD expulsions following IUD insertions. 3

4 Methods 1 Study design: Cross-sectional study IUD insertions: Inclusion criteria Data: Evidence in EMR of IUD insertion (any brand) 1/1/2009 9/30/2015 Women age 50 at time of IUD insertion No required minimum enrollment as KPNC members after IUD insertion => 168,744 IUD insertions All IUD insertions in the study period were identified using the KPNC Virtual Data Warehouse (VDW) Separate algorithms were developed to identify potential uterine perforations and expulsions using data from the VDW, the surgical procedure (Optime) database and the Clarity and radiology notes tables For both outcomes, 100 potential cases were randomly chosen for manual medical record review and validation 4

5 Methods 2 Algorithm development Uterine perforation algorithm Used a combination of structured (e.g., ICD-9-CM and CPT codes) and unstructured (e.g., NLP) data; Only structured codes for suspected or confirmed perforations No structured codes for delineating complete vs partial IUD expulsion algorithm Used unstructured data only There were no structured codes for expulsions Statistical analysis For both outcomes, positive predicted Value (PPV) with 95% confidence intervals comparing the algorithm results to the manual medical record review results (gold standard) were calculated and used to assess the accuracy of the algorithms. 5

6 6 Methods 3 Uterine perforation algorithm (1) Requiring/confirmed at operation Structured data from OPTIME CASE LOG table Values (dx): Uterus perforation, IUD malfunction, IUD malposition, IUD removal, etc Values (px): laparoscopy diag, pelviscopy Subset to operations that mention IUD in dx or px text Unstructured data form hospital notes tables NLP phrases (examples) IUD abdomen IUD omentum IUD protruding Extrauterine IUD (2) No operation, dealt with/discussed in clinic Structured data from VDW VDW dx table: codes for IUD perforation, IUD complications, IUD malfunction, etc VDW px table: codes for hysteroscopy Unstructured data from Clinic/Ambulatory notes tables Subset to progress, procedure notes, etc NLP phrases (examples) IUD embedded IUD buried IUD perforated Embedded IUD

7 7 Methods 4 IUD expulsion algorithm (1) Structured data VDW dx table: codes for IUD surveillance, presence of IUD, contraceptive surveillance, IUD complication, etc Also searched for evidence of new IUD insertion without coded removal (2) Unstructured data From hospital notes tables Subset to short and long operative notes NLP phrases (examples) IUD fell out IUD expelled IUD not in uterus IUD extruding cervix From radiology tables Use structured codes to get radiology reports from abdominal ultrasounds, x-rays, etc NLP phrases (strategy was to look for evidence of no IUD) No IUD IUD was not General NLP strategy Search for key phrases Search for and exclude cases with negation phrases (e.g., not embedded ) Flag but do not exclude cases with phrases related to history of

8 8 Results

9 Table 1. Positive Predictive Value for Uterine Perforation Category N PPV, % (95% CI) Identified By EMR algorithm 444 Sampled 100 Case status Yes Overall * (68 85) Partial 28 Complete 48 No 16 Undetermined 7 * Subcategorization of the uterine perforation cases into Partial or Complete could not be ascertained for all cases reviewed, so the number of cases classified with a case status of Yes for Partial or Complete will not add to the Overall Yes total. 9

10 Table 2. Positive Predictive Value for IUD Expulsion Category N PPV, % (95% CI) Identified By EMR algorithm 4,185 Sampled 100 Case status Yes (66 85) No 21 Undetermined 2 10

11 Strengths and Limitations Limitations: Algorithms developed on KPNC data might not be generalizable to other EMRs. The algorithms were developed using data through September 2015, when only ICD-9-CM coding was in use; expanding beyond that date for ICD-10 coding involves mapping Strengths: Large sample size Use of both structured and unstructured data resulted in robust algorithms 11

12 Conclusions The PPV was 77% for both uterine perforation and expulsions, indicating excellent performance of both algorithms for the capture of these complex health outcomes. Refine algorithms to further improve PPV 12