A comparative study of different methods for automatic identification of clopidogrel-induced bleeding in electronic health records

Transcription

1 A comparative study of different methods for automatic identification of clopidogrel-induced bleeding in electronic health records Hee-Jin Lee School of Biomedical Informatics The University of Texas Health Science Center at Houston

2 Introduction Pharmacogenomics The study of how genes affect a person s response to drugs Identifying common and rare genetic variants that contribute to drug efficacy and toxicity (e.g., adverse drug events, ADR) Identifying patients with specific ADR from EHR Costly and time consuming Automatic tools for ADR identification have been investigated. 2

3 Clopidogrel-induced bleeding Clopidogrel for coronary artery disease Often in association with aspirin (dual anti-platelet therapy, DAPT) Recommended by current guidelines Potentially life-threatening adverse bleeding by clopidogrel Genetically determined response Only 12% of the inter-individual variations are explained so far. On going study at Vanderbilt University Medical Center (VUMC) 3

4 Overview Automatic identification of clopidogrel-induced bleeding from EHR Use both structured and narrative data Comparative study of different features and classification methods EHR at VUMC Ini3al Screening Candidate Pa3ents Feature Extraction Features Automatic Classification Case Non-case 4

5 Data 2,268 candidate patients Patients who potentially had clopidogrel-induced bleeding Screening criterion: at least one bleeding ICD-9 code within the time period of clopidogrel mentions in a patient s medical record Manual classification into case or non-case 1,921 cases of clopidogrel-induced bleeding 347 non-cases Splitting into training/test set 1,521 patients for training 756 patients for test 5

6 Features Bleeding information Bleeding ICD-9 codes E.g , blood in stool (109 codes curated by experts) Bleeding CUIs as identified by KMCI (KnowledgeMap Concept Identifier) E.g. C , bleeding from larynx (465 CUIs curated by experts) Bleeding keywords E.g. epistaxis, melena (51 keywords curated by experts) Clopidogrel information Clopidogrel mention and signature (i.e., dosage, route, frequency, etc.) as identified by MedEx 6

7 Features Temporality between bleeding and clopidogrel Timestamps of bleeding, clopidogrel exposure, or other possible causes of bleeding (e.g., chest tube, stab wound ) Shortest time durations between clopidogrel exposure and bleeding, and between other possible causes of bleeding and bleeding Explicit evidence from physicians assertion E.g., Bleeding was substantial owing to the fact that the patient was on DAPT with aspirin and clopidogrel. Identified by simple rule: both clopidogrel and bleeding mention in a sentence without any negation 7

8 Classification methods Rule-based methods Temporality rule: case if there was bleeding after a mention of clopidogrel without any other possible bleeding causes in-between. Explicit evidence rule: case if there is an explicit evidence of clopidogrel-caused bleeding. Machine learning-based methods SVM classifier Bleeding, clopidogrel, temporality, and explicit evidence features and their combinations 8

9 Classification methods Scoring-based methods Use both temporality features and explicit evidence features. s = W duration (1 f duration /10, 000) +W explicit f explicit F duration S f duration f explicit W duration,w explicit score shortest 3me dura3on from clopidogrel exposure to bleeding existence of explicit evidence of clopidogrel-caused bleeding weights case if s cutoff, non-case otherwise. 9

10 Results Category System Description P R F + temporality rule Rule + explicit evidence rule union of temporality and explicit evidence rules bleeding and clopidogrel features ML + temporality feature explicit evidence feature temporality and explicit evidence features cutoff = Scoring cutoff = cutoff =

11 Results Explicit evidence feature shows the best precision. Combination of temporality feature and explicit evidence feature shows the best F-score. Overall, ML-based and scoring-based systems outperforms rule-based systems (F-score). When targeting 95% precision level, scoring system with cutoff 6 may be the best choice. 11

12 Error case Date: September 20th 2011 Recent episode of gross hematuria at OSH with resultant demand ischemia and NSTEMI. "Per Urology, no obvious source of hematuria and has not reoccured in house, will proceed with andcoaguadon and LHC. Please take plavix for three more days total. Bleeding in the past No bleeding as of Sep. 20th Taking clopidogrel as of Sep. 20th non-case, but identified as case by the systems Detailed analysis on temporality and drug status is required. 12

13 Summary Informatics approaches for identifying patients with clopidogrel-induced adverse bleeding. Useful features Temporality between clopidogrel and bleeding Textual evidence from physicians assertion Comparison of different classification methods ML and scoring methods are effective in terms of F-score. Scoring method can achieve a high precision with a reasonable recall. 13

14 Discussion The gold standard data set (candidate patients) was generated using ad-hoc rules. Different criteria for candidate case selection would result in different precision and recall. The generalizability of the findings from this study may need further validation. Application of the proposed methods to other ADR types would require additional manual work. 14

15 Acknowledgements University of Texas Health Science Center at Houston Min Jiang PhD Yonghui Wu PhD Hua Xu PhD Vanderbilt University School of Medicine Christian M. Shaffer MS John H. Cleator MD Eitan A. Friedman MD Dan M. Roden MD Josh Denny MD,MS University of Maryland School of Medicine Joshua P. Lewis MD 15

16 Thank you!!! Questions? 16