Identifying Patients With Ischemic Heart Disease in an Electronic Medical Record

Transcription

1 Original Research Identifying Patients With Ischemic Heart Disease in an Electronic Medical Record Journal of Primary Care & Community Health 2(1) The Author(s) 2011 Reprints and permission: sagepub.com/journalspermissions.nav DOI: / Noah Ivers, MD, CCFP 1-3, Bogdan Pylypenko, BScN 2, and Karen Tu, MD, MSc, CCFP, FCFP 2-4 Abstract Purpose: Increasing utilization of electronic medical records (EMRs) presents an opportunity to efficiently measure quality indicators in primary care. Achieving this goal requires the development of accurate patient-disease registries. This study aimed to develop and validate an algorithm for identifying patients with ischemic heart disease (IHD) within the EMR. Methods: An algorithm was developed to search the unstructured text within the medical history fields in the EMR for IHD-related terminology. This algorithm was applied to a 5% random sample of adult patient charts (n = 969) drawn from a convenience sample of 17 Ontario family physicians. The accuracy of the algorithm for identifying patients with IHD was compared to the results of 3 trained chart abstractors. Results: The manual chart abstraction identified 87 patients with IHD in the random sample (prevalence = 8.98%). The accuracy of the algorithm for identifying patients with IHD was as follows: sensitivity = 72.4% (95% confidence interval [CI]: ); specificity = 99.3% (95% CI: ); positive predictive value = 91.3% (95% CI: ); negative predictive value = 97.3 (95% CI: ); and kappa = 0.79 (95% CI: ). Conclusions: Patients with IHD can be accurately identified by applying a search algorithm for the medical history fields in the EMR of primary care providers who were not using standardized approaches to code diagnoses. The accuracy compares favorably to other methods for identifying patients with IHD. The results of this study may aid policy makers, researchers, and clinicians to develop registries and to examine quality indicators for IHD in primary care. Keywords electronic medical record, administrative data, ischemic heart disease, chart audit Ischemic heart disease (IHD) is a leading cause of morbidity and mortality that is often managed in primary care; 1 however, recent evidence indicates a wide variability in the management of such patients. 2,3 The increasing use of electronic medical records (EMRs) could facilitate improved monitoring of patients with IHD. However, unlike the case of diabetes, where structured fields in the EMR (such as laboratory results and treatments) can be used to accurately identify patients, 5 using the EMR to develop a registry of IHD patients is difficult. 6 Therefore, in jurisdictions where providers are not incentivised to systematically code diagnoses, approaches to help identify patients with IHD are needed to facilitate quality improvement efforts. With this goal in mind, we developed and tested a novel EMR search algorithm focusing on the unstructured text in the medical history fields. We also examined whether searching for nitrate prescriptions in the EMR would increase the accuracy of identification of patients with IHD as has been proposed in similar studies. 8 Methods Setting As part of a study for the Canadian Cardiovascular Outcomes Research Team (CCORT), selected Ontario family physicians using Practice Solutions EMR share their data with researchers at the Institute for Clinical Evaluative Sciences (ICES) to create an Electronic Medical Record 1 Women s College Hospital, Toronto, Canada 2 Institute for Clinical Evaluative Sciences, Toronto, Canada 3 Department of Family and Community Medicine, University of Toronto, Canada 4 Toronto Western Hospital Family Health Team, University Health Network, Toronto, Canada Corresponding Author: Noah Ivers, Institute for Clinical Evaluative Sciences, G1 06, 2075 Bayview Avenue, Toronto, Ontario M4N 3M5 noah.ivers@ices.on.ca

2 50 Journal of Primary Care & Community Health 2(1) Table 1. Initial Search Algorithm Applied to Medical History Fields in Electronic Medical Records IHD, ischemic heart disease ASHD, arteriosclerotic heart disease CAD, coronary artery disease Angina ACS, acute coronary syndrome MI, myocardial infarction, STEMI, NSTEMI, non-stemi, AMI PCI, percutaneous coronary intervention, coronary stent, coronary angioplasty CABG, coronary bypass graft surgery Administrative Data Linked Database (EMRALD). ICES is a prescribed entity under the province of Ontario s Personal Health Information Protection Act, which allows for the collection of individual level health information for use in planning and managing the health care system. Practice Solutions is Ontario s most used EMR system, with 46.2% of the market as of November Algorithm Development The search algorithm was developed between August 2009 and January 2010 and was conducted on the EMR charts from all active, rostered, adult patients in the EMRALD database at that time. Active was defined as having at least 2 visits within 3 years. These patients belonged to 39 community-based family physicians from across Ontario who had used the EMR for at least 2 years. The algorithm was developed using the SQL database management system, starting by applying the clinically derived list of terms in Table 1 to the free-text entries in the medical history fields ( problem list and past medical history ) of the EMR. This initial IHD-related dictionary was developed based on the clinical experience of the investigators. It was refined iteratively by applying the terms in a search of the medical history fields and then adding any new IHD-related terms discovered. Physicians rarely used structured entries (eg, ICD-9 code 413 for angina), and usually entered data into the medical history fields using free text. Idiosyncratic descriptors were common in the medical history fields, such as stent-lad (meaning angioplasty for the left anterior descending coronary artery). Therefore, the search algorithm was designed to find exact matches to phrases/ words/acronyms and also allow for some variation in syntax or spelling (eg, LAD-stent vs stent-lad, AMI vs A.M.I.). Many patients had multiple IHD-relevant descriptors. To illustrate, some patients had a history of myocardial infarction (MI) and a prior stent. Because it seemed likely that stent was another IHD-related term, it was added to the algorithm. In the next iteration, new patients were identified with stent as their only IHD descriptor. Search-term-exclusions were also added to the algorithm to remove terms when they were not related to IHD. For instance, stent would be excluded if adjacent to aneurysm because the stent was not necessary related to IHD. Similarly, bypass (as in coronary artery bypass) was included as a search term, but was excluded from the algorithm if near the term gastric. We tried to increase specificity by excluding words preceded by? or query or possible. We also excluded IHD-related words occurring next to mother/father and occurring next to the phrase no known. All such exclusions were developed inductively. The algorithm was refined iteratively until no new search terms or exclusions were uncovered. The refinement of the search algorithm was conducted by 2 of the investigators (Noah Ivers and Bogdan Pylypenko) and any disagreements were settled through discussion with all investigators. The final search algorithm has 317 search terms and automatically removes 52 exclusion terms. Our methods were very similar to Natural Language Processing approaches recently reported in the medical informatics literature. 9,10 In these papers, it is argued that using text-dictionaries with modifier detection is a simple but efficient approach, and that clinical expertise is needed to adjust the search terms used to find exact matches, similar matches, and exclusion terms for specific clinical domains and contexts. Although there are open-source software options to assist medical informatics professionals to develop algorithms for searching free-text (such as NegEx 11 for finding exclusions), we chose to avoid such software to ensure that our methods were accessible to those with a clinical or policy focus. We also developed an algorithm to search for nitrate prescriptions in the treatments field of the EMR. We counted patients as nitrate positive if they had 1 long-acting nitrate prescription and/or 2 or more prescriptions for short-acting nitrates. The requirement for 2 short-acting nitrates was implemented to try to eliminate patients who were prescribed a nitro-spray only once prior to a normal stress test. Validation Set The validation set was based on a 5% random sample of patients from the first 17 physicians to contribute data to EMRALD. The 5% random sample resulted in 969 patients. As described in a related paper, 12 3 trained abstractors reviewed the full text from the entire patient chart. Patients that had documented evidence of a MI, percutaneous coronary intervention (PCI), or coronary artery bypass graft (CABG), or had IHD documented in a specialist letter, in the medical history fields, or in the family physician s

3 Ivers et al. 51 Table 2. Accuracy of Four Different Search Algorithms for Identifying Patients With IHD Compared to Manual Abstracted Occurrences (N = 87) From a Random Sample of Primary Care Electronic Medical Records a Search Technique Sensitivity (%) Specificity (%) PPV (%) NPV (%) Kappa Initial search, medical history fields ONLY 26.4, ( ) 99.6, ( ) 85.2, ( ) 93.2, ( ) 0.37, ( ) Initial search PLUS Rx nitrate 40.2, ( ) 99.3, ( ) 85.4, ( ) 94.4, ( ) 0.52, Refined search, medical history fields ONLY 72.4, ( ) 99.6, ( ) 94.0, ( ) 97.3, ( ) 0.80, Refined search PLUS Rx nitrate 74.7, ( ) 99.3, ( ) 91.2, ( ) 97.6, ( ) 0.81, Abbreviations: IHD, ischemic heart disease; PPV, positive predictive value; NPV, negative predictive value. a Values are given, followed by exact 95% confidence intervals. progress notes were IHD positive. The intraobserver reliability was high, with kappa values exceeding Analysis We tested the initial, clinical terms only (Table 1) and the final, refined search algorithm, with and without a nitrate prescription, against the results from the manual abstraction. Sensitivity was calculated as the proportion of patients identified by the manual abstraction as having IHD (used as the reference standard), who also had IHD according to the algorithm. Specificity was calculated in the same manner except that it was based on individuals who were not identified as having IHD. Positive predictive value (PPV) was defined as the proportion of IHD patients identified by the algorithm that were also identified as IHD by the manual abstraction. Negative predictive value (NPV) was defined similarly for patients that did not have IHD. Kappa statistics for agreement between the results from the manual abstraction and from the algorithm data were also calculated. Exact confidence interval (CIs) were calculated for all proportions and tests of accuracy. All analyses were conducted using SAS version 9.2 (SAS Institute, Cary, NC). In a secondary analysis, we evaluated the algorithms in a practice of 6 physicians that were new to the EMRALD database. One researcher (Dr Ivers) manually audited the complete electronic charts of all patients identified by each algorithm as having IHD. If the manual abstraction suggested that the patient did not have IHD, that patient was considered a false positive. This project received ethics approval through Sunnybrook Health Sciences Centre Research Ethics Board. Results The manual chart abstraction of 969 randomly selected charts identified 87 patients meeting criteria for IHD (prevalence = 8.98%). Table 2 shows that the refined search algorithm outperforms the initial search terms, greatly improving sensitivity with limited change to specificity. The addition of nitrate prescriptions substantially increased sensitivity compared to the initial search terms, but had little impact on the diagnostic accuracy of the refined search algorithm. When the algorithms were applied to a new EMRALD practice, the prevalence of IHD in this practice was estimated to be 9.68% to 12.66%, depending on the algorithm used. Table 3 shows that the false positive rate ranged from 4.14% to 12.54%. Compared to the initial search terms, the refined search resulted in a lower false positive rate, while identifying more IHD patients. Of the extra patients identified by adding a search for nitrate prescriptions, 51.92% were false positives. Of these false positives, 62.96% represented patients who had a negative work-up (eg, investigations revealed normal coronary arteries), while 25.93% seemed to have received nitrates for either congestive heart failure or for refractory hypertension, rather than angina. Using the refined algorithm, 85.71% of the false positives had a negative workup and the others had no further evidence in the chart of IHD. Discussion The refined algorithm vastly outperformed a set of clinically derived search terms for identifying patients with IHD. Using the same set of manually abstracted charts, algorithms using administrative databases identify patients with IHD with sensitivity of 77.0% but a PPV of only 78.8%. 12 The approach described here has slightly lower sensitivity but much better PPV. The PPV is also superior to methods described in the UK. 7,8 When the refined algorithm was applied to a new set of over 3000 patient charts, the false positive rate was <5%. This low false positive rate suggests a clinically useful tool for practice-based quality improvement. Furthermore, this approach relies only on primary care clinical data and therefore may capture patients in the earlier stages of disease. 13 Another major advantage of this search technique is that it does not require interventions to standardize data. 14 A Norwegian study has previously found that the use of the International Classification for Primary Care (ICPC) system for diagnostic codes could identify 75% of patients in primary care with IHD, 15 but the ICPC is not in common

4 52 Journal of Primary Care & Community Health 2(1) Table 3. Audit of IHD Patients Identified by Each Algorithm in New Primary Care Practice Search Technique # IHD Patients (estimated prevalence) # False Positives (% false positive) Initial search, medical history fields ONLY 299 (9.68%) 17 (5.69%) Initial search PLUS Rx nitrate 351 (11.37%) 44 (12.54%) Refined search, medical history fields ONLY 338 (10.95%) 14 (4.14%) Refined search PLUS Rx nitrate 390 (12.66%) 41 (10.51%) Abbreviation: IHD, ischemic heart disease. use in North America. Internet-based tools have been shown in New Zealand 16 and Australia 17 to help EMR-using clinicians manage cardiovascular risk, but these required extra data entry into structured fields. Another option for creating a registry is to search for all patients prescribed a cardiovascular drug followed by a manual abstraction to confirm those with IHD. 18 The uptake of such methods by typical primary care providers with limited resources is uncertain. There are a number of limitations with this study. The accuracy of the algorithm depends on updated medical history fields, but there is some evidence that keeping up-todate patient profiles in an EMR can lead to improved processes of care for patients at risk of cardiovascular disease. 19 In this study, physicians appear to be proactively updating the medical history fields as patients present with new problems. In fact, this created an unexpected limitation most false positives occurred when physicians quickly added IHD to the patient s problem list and neglected to remove it when subsequent investigations were negative. In contrast, removal of IHD-related words from the algorithm when preceded by possible may have improved specificity at the expense of sensitivity. The overlap between the development and validation data sets was necessary to use data from as many physicians as possible in creating the search algorithm (thereby including as many idiosyncratic IHD descriptors as possible). Though this might falsely elevate specificity in our analysis, the secondary analysis applying the algorithm to an entirely new population was concordant with the initial results. Finally, although the algorithm was developed using the charts of physicians with a variety of characteristics from across the province, these were a convenience sample of participants all using the same software. These physicians are experienced EMR users and may be systematically different from other physicians with respect to how they populate the medical history fields. Therefore, application to other physicians using other EMR software may not have the same accuracy. The inclusion of billing/diagnostic codes, prescriptions other than nitrates, or cardiac investigations in an EMRbased search algorithm for IHD patients requires further investigation. In general, primary care providers in North America lag behind other developed countries with respect to utilization of EMRs, 1 due in part to a lack of standardized documentation procedures. Innovative approaches are needed to develop IHD patient registries and maximize the potential of EMRs. The approach described above compares favorably to other methods and could facilitate primary care physicians and researchers in using EMRs to facilitate quality improvement activities for patients with IHD. Acknowledgement The authors thank the physician participants of EMRALD. Declaration of Conflicting Interests The authors declared no potential conflicts of interests with respect to the authorship and/or publication of this article. Financial Disclosure/Funding This work was funded by a Canadian Institutes of Health Research Team grant in Cardiovascular Outcomes Research to the Canadian Cardiovascular Outcomes Research Team (CCORT). Dr Ivers is supported by a Fellowship Award in Primary Care from the Canadian Institutes of Health Research (CIHR). References 1. Schoen C, Osborn R, Doty MM, Squires D, Peugh J, Applebaum S. A survey of primary care physicians in eleven countries, 2009: Perspectives on care, costs, and experiences. Health Aff (Millwood). 2009;28:w1171-w DeWilde S, Carey IM, Richards N, Whincup PH, Cook DG. Trends in secondary prevention of ischaemic heart disease in the UK : use of individual and combination treatment. Heart. 2008;94: Austin PC, Tu JV, Ko DT, Alter DA. Factors associated with the use of evidence based therapies after discharge among elderly patients with myocardial infarction. CMAJ. 2008;179: Mitiku TF, Tu K. Using data from electronic medical records: theory versus practice. Healthc Q. 2008;11: Tu K, Manuel D, Lam K, Kavanagh D, Mitiku T, Guo H. Diabetics can be identified in an electronic medical record using lab tests and prescriptions. J Clin Epidemiol Fokkens AS, Wiegersma PA, Reijneveld SA. A structured registration program can be validly used for quality assessment in general practice. BMC Health Serv Res. 2009;9:241.

5 Ivers et al Gray J, Majeed A, Kerry S, Rowlands G. Identifying patients with ischaemic heart disease in general practice: cross sectional study of paper and computerized medical records. BMJ. 2000;321: Donnan PT, Dougall HT, Sullivan FM. Optimal strategies for identifying patients with myocardial infarction in general practice. Fam Pract. 2003;20: Farkas R, Szarvas G, Hegedus I, et al. Semi-automated construction of decision rules to predict morbidities from clinical texts. J Am Med Inform Assoc. 2009;16: Yang H, Spasic I, Keane JA, Nenadic G. A text mining approach to the prediction of disease status from clinical discharge summaries. J Am Med Inform Assoc. 2009;16: Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG. A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform. 2001;34: Tu K, Mitiku T, Lee DS, Guo H, Tu JV. Validation of physician billing and hospitalization data to identify patients with ischemic heart disease using EMRALD. Can J Cardiol. In press. 13. Elo SL, Karlberg IH. Validity and utilization of epidemiological data: a study of ischaemic heart disease and coronary risk factors in a local population. Public Health. 2009;123: de Lusignan S. An educational intervention, involving feedback of routinely collected computer data, to improve cardiovascular disease management in UK primary care. Methods Inf Med. 2007;46: Treweek S. The potential of electronic medical record systems to support quality improvement work and research in Norwegian general practice. BMC Health Serv Res. 2003;3: Wells S, Furness S, Rafter N, Horn E, Whittaker R, Stewart A, et al. Integrated electronic decision support increases cardiovascular disease risk assessment four fold in routine primary care practice. Eur J Cardiovasc Prev Rehabil. 2008;15: Peiris DP, Joshi R, Webster RJ, Groenestein P, Usherwood TP, Heeley E, et al. An electronic clinical decision support tool to assist primary care providers in cardiovascular disease risk management: development and mixed methods evaluation. J Med Internet Res. 2009;11:e Gray J, Ekins M, Scammell A, Carroll K, Majeed A. Workload implications of identifying patients with ischaemic heart disease in primary care: population-based study. J Public Health Med. 2003;25: Hartung DM, Hunt J, Siemienczuk J, Miller H, Touchette DR. Clinical implications of an accurate problem list on heart failure treatment. J Gen Intern Med. 2005;20: Bios Noah Ivers, MD, CCFP, is a primary care research fellow and a family physician at Women s College Hospital. Bogdan Pylypenko, BScN, is a health informatician at the Institute for Clinical Evaluative Sciences. Karen Tu, MD, MSc, CCFP, FCFP, is a family physician and a scientist at the Institute for Clinical Evaluative Sciences.