Numerous studies have indicated shortcomings in. Effect of Concurrent Computerized Documentation of Comorbid Conditions on the Risk of Mortality Index

Transcription

1 Effect of Concurrent Computerized Documentation of Comorbid Conditions on the Risk of Mortality Index Jerry Stonemetz, MD, Julius Cuong Pham, MD, Robert J. Marino, MD, John A. Ulatowski, MD, PhD, and Peter J. Pronovost, MD, PhD Abstract Objective: To evaluate the effect of adding concurrent computerized documentation of comorbid conditions to standard medical record review on the severity of illness index and risk of mortality index. Design: Cross-sectional study. Setting and participants: 377 surgical patients at 2 academic centers undergoing inpatient surgery from September 2004 to January Standard medical record review was performed by the hospital s professional medical records coders. Concurrent computerized coding was performed by an anesthesiologist doing preoperative assessment using a software tool. Measurements: Severity of illness index, risk of mortality index, and number of comorbid conditions. Results: Concurrent coding averaged 5.3 additional comordid conditions that were not identified by medical record review. 9 of 13 comorbid conditions increased with concurrent coding. Mean severity of illness index and risk of mortality index scores were 1.92 and 1.44 with medical record review and 2.23 and 1.68 for concurrent coding. Concurrent coding increased severity of illness by 16% and risk of mortality by 17%; severity of illness and risk of mortality increased by 1 category in 27% and 23% of patients, respectively. Conclusion: Concurrent coding significantly increased the severity of illness index and risk of mortality index, which can have important implications for health outcomes research, perceived quality of care, and financial reimbursement. Numerous studies have indicated shortcomings in the quality of the U.S. health care system [1]. Concern about these deficiencies have led to numerous quality improvement initiatives, including widespread use of quality measures and the public reporting of quality data. A primary goal of quality measurement is to increase the number of patients exposed to high-quality providers [2,3], either by converting low-quality providers into highquality providers or by moving patients away from providers of low quality to providers of high quality. Yet, measuring quality of care is an imprecise and evolving science. While most hospitals want to measure health outcomes such as mortality, the challenges of measuring these outcomes are significant [4]. Although studies have raised concern about the validity of mortality as an outcome measure, its use continues [5,6]. As with all outcome measures, hospital mortality rates may be affected by random or systematic error. One systematic error in measuring hospital mortality results from inadequately estimating a patient s risk for death. A common method to account for patient differences in risk is to use risk adjustment [7]. The risk of mortality index is a relatively new method of risk adjustment that is part of the All Patient Refined Diagnosis-Related Group (APR-DRG) system (3M, St. Paul, MN) [8]. The APR-DRG expands the basic CMS-DRG structure by adding subclasses to each DRG that describe patient differences relating to risk of mortality and severity of illness. The methodology yields 2 subcomponent scores: the risk of mortality index and the severity of illness index. Hospital payment rates for inpatient care are based on DRGs. Relative weights are assigned to each DRG to determine the reimbursement rate. These weights are affected by the patient severity of illness index score. Because the severity of illness index score influences hospital reimbursement, hospitals have an incentive to code more thoroughly and accurately. A primary strategy has been the implementation of tools to help better identify comorbid conditions. However, unless such tools are uniformly applied by all hospitals, From the Department of Anesthesiology and Critical Care Medicine (Drs. Stonemetz, Ulatowski, and Pronovost) and the Department of Emergency Medicine (Dr. Pham), Johns Hopkins University School of Medicine, Baltimore, MD; and Oschner Health System, New Orleans, LA (Dr. Marino). Vol. 14, No. 9 September 2007 JCOM 499

2 concurrent coding risk-adjusted mortality rates could misrepresent a hospital s level of quality of care and result in over- or underpayment for care delivered. The aim of this study was to evaluate the impact of adding concurrent computerized coding of comorbid disease to standardized retrospective medical record review on the severity of illness index and risk of mortality index. We hypothesized that the use of concurrent computerized coding of comorbid diseases would alter severity of illness index and risk of mortality index scores. Methods Study Design and Population A cross-sectional study identifying comorbid diseases in a surgical population was performed between September 2004 and January A convenience sample of patients from the daily operating room schedule at 2 academic medical centers (Baltimore, MD, and New Orleans, LA) were studied. Eligibility criteria for the study included age 18 years and older; scheduled for inpatient admission following surgery, discharged at the time of chart review; and Class 2, 3, or 4 on the American Society of Anesthesiologists (ASA) Physical Status Classification System. Institutional review board approval was obtained at both institutions. Informed consent was waived because data identifying a patient (eg, name, date of birth) were not collected. The medical record for each study subject was coded using both retrospective medical record review and concurrent computerized coding. Trained personnel from the medical records department at each participating hospital reviewed every medical record to identify comorbid diseases, converted them to ICD-9-CM diagnosis codes, and then assigned DRGs. The patient s age, gender, race (ie, white, black, and other), and type of surgery (ie, classified by major organ involved) were abstracted from the discharge record. An anesthesiologist (JS or RJM) concurrently conducted an independent assessment of comorbid conditions using a computerized preoperative assessment tool (DocuCode by Docusys, Mobile, AL). This tool prompts the user to identify clinical conditions and abnormal laboratory or diagnostic tests from the medical record and assigns ICD-9-CM diagnosis codes. Clinical conditions were determined through the standard paper preoperative assessment form and a thorough chart review. Outcomes Primary outcome variables were the severity of illness index and risk of mortality index. Secondary outcomes were the number of comorbid conditions coded using each coding method. ICD-9-CM codes were entered into the APR-DRG software tool, called a Grouper, to calculate the severity of illness index and the risk of mortality index. The coding logic is proprietary; however, both the severity of illness index and risk of mortality index were calculated from the aggregation of comorbid conditions, principle diagnosis, and procedures. The severity of illness index and risk of mortality index classify subjects into 1 of 4 mutually exclusive risk categories (1 signifying the lowest risk and 4 signifying the highest risk). For the concurrent coding, the additional ICD- 9-CM diagnosis codes assigned by DocuCode were added to the codes obtained from the medical records, and the severity of illness index and risk of mortality index were recalculated. Because of complex rules for ICD-9-CM coding, all cases in which the severity of illness index or risk of mortality index were altered after adding the concurrent coding were validated by the respective medical records department. Statistical Analysis Proportions, means, and standard deviation were used to describe patient demographic characteristics, comorbid conditions, severity of illness index, and risk of mortality index. Impact of the enhanced comorbid disease coding system was evaluated by percentage of patients whose severity of illness and risk of mortality increased by at least 1 category. The percentage of patients that changed 1, 2, or 3 categories was also evaluated. Mean severity of illness index and risk of mortality index scores with and without the concurrent coding were compared with a matched t test. A Wilcoxon signed-rank test was used to test for differences in the distribution of severity of illness index and risk of mortality index categories. A chi-square test was used to evaluate differences in proportions; p value was significant at All analyses were performed using the Intercooled Stata 8.2 (College Station, TX) statistical package. Results Study Subject Characteristics A total of 400 patients were enrolled; 23 were excluded because the discharge data were not available at the completion of the study (lost by medical records or not discharged). 72% were white, and average age was 59 years (Table 1). 44% of subjects were ASA Physical Status Class 2 and 47% were Class 3. The most common surgeries were gastrointestinal (30%), genitourinary (29%), and musculoskeletal (20%). Comparison of Methods On average, medical record review identified 8.3 ± 4.7 (95% confidence interval [CI], ) comorbid conditions and concurrent computerized coding identified 13.6 ± 2.6 (95% CI, ). This resulted in an average increase of 5.3 ± 2.6 (95% CI, ) comorbid diseases identified with computerized 500 JCOM September 2007 Vol. 14, No. 9

3 coding (Table 2). Of the 13 comorbid conditions, 9 increased in diagnoses with computerized coding. Of these 9 comorbidities, the likelihood of diagnoses increased by at least 5% for hypertension, obesity, anemia, diabetes, and chronic obstructive pulmonary disease. Mean severity of illness index and risk of mortality index scores were 1.92 and 1.44 on medical chart review and 2.23 and 1.68, respectively, with concurrent coding (Table 3). With concurrent computerized coding, severity of illness index and risk of mortality index increased by 16% and 17%. In addition, severity of illness index increased by 1 category in 27% of patients and risk of mortality index increased by 1 category in 23% of patients. Table 1. Study Subject Demographic Characteristics (n = 377) Demographic Characteristics Percentage of Sample* Gender Male 45 Race White 72 Black 20 Other 08 ASA class Primary procedure Cardiovascular 10 Gastroenterology 30 Endocrine 00 ENT 03 Genitourinary 29 Hematolymphatic 01 Integumentary 04 Musculoskeletal 20 Respiratory 03 Miscellaneous 00 Average age, yr (SD) 59 (15) ASA = American Society of Anesthesiology Physical Status Score; ENT = ear, nose, and throat; SD = standard deviation. *Unless otherwise indicated Discussion Mortality risk for surgical patients as measured using the APR-DRG risk of mortality index was strongly influenced by the number of comorbid conditions coded. Concurrent computerized coding identified more comorbid conditions than retrospective medical record review, which substantially increased a patient s estimated mortality risk. A hospital s mortality rating is affected by the extent to which comorbid diseases, severity of illness, and complications are documented in the medical record. Assignment of ICD-9-CM codes by medical records department personnel are based on medical record documentation. Thorough documentation is critical to this process, as coders cannot infer more complex or higher acuity diagnosis codes without supporting documentation. For example, our study demonstrated that obesity was frequently not documented and consequently could not be coded as a comorbid condition, even if height and weight were noted on the chart. Despite concerns about random and systematic error, the use of outcome measures to evaluate hospital performance continues to grow. Yet, if quality measures are not scientifically sound, they can misinform. Error from inadequate risk adjustment is well documented [9 12]. Our study raised additional concerns about the errors introduced by methods of identifying clinical variables. Hospitals that have the resources to implement information technology to enhance coding may appear to have lower risk-adjusted mortality rates compared with hospitals that lack such resources. Researchers and policy makers, while acknowledging limitations in risk-adjusted mortality, generally assume that variation in coding among hospitals are random and thus does not introduce systematic error [13]. Our results suggest that a hospital s ability to invest in information technology to enhance coding may introduce bias in risk-adjusted mortality. Researchers and policy makers would be well served by adhering to basic clinical research principles in collecting the variables that influence risk for mortality. That is, the variables need explicit definitions, the methods of surveillance for those variables should be standardized, and data quality control for data abstractors should be implemented. This type of approach will incur costs. Yet in the absence of such basic quality control principles, hospital mortality data should be used cautiously [14,15]. Implementing standardized concurrent coding during the preoperative assessment may provide a more accurate assessment of a patient s mortality risk. Another method for obtaining accurate risk-adjusted mortality is to use trained nurses to collect perioperative data in a standardized fashion. This methodology has been effectively employed by the Department of Veterans Affairs (VA) in the National Surgical Quality Improvement Program, which has been credited with improving the quality of care across all VA hospitals [16]. This program has been praised as the best in the nation by the Institute of Medicine for measuring surgical quality and outcomes [1]. Unfortunately, the methodology is resourceintensive and data are only available from limited institutions for several diseases. There are potential limitations to this study. First, we used clinical definitions to identify comorbid conditions. These Vol. 14, No. 9 September 2007 JCOM 501

4 concurrent coding Table 2. Comorbid Conditions of Study Subjects by ICD-9-CM (n = 377) Percentage of Sample* Comorbid Conditions Medical Records, % Concurrent, % Odds Ratio (95% CI) p Value Prior myocardial infarction ( ) < 0.01 Dementia ( ) COPD ( ) < 0.01 Mild liver disease Severe liver disease ( ) DM ( ) < 0.01 DM with complications ( ) < 0.01 Renal disease Malignancy ( ) < 0.01 Metastatic solid tumor ( ) Obesity ( ) < 0.01 Hypertension ( ) < 0.01 Anemia ( ) < 0.01 Number of comorbidities, mean (SD) 8.3 (4.7) 13.6 (2.6) < 0.01 CI = confidence interval; COPD = chronic obstructive pulmonary disease; DM = diabetes mellitus; SD = standard deviation. *Unless otherwise indicated. Table 3. Risk of Mortality and Severity of Illness Indices of Study Subjects by ICD-9-CM (n = 377) Percentage of Sample* Medical Records Concurrent Relative Change, % P Value Severity of illness index, mean (SD) 1.92 (0.80) 2.23 (0.76) +16 < 0.01 Patients by severity of illness class, % I II III < 0.01 IV Risk of mortality index, mean (SD) 1.44 (0.70) 1.68 (0.80) +17 < 0.01 Patients by risk of mortality class, % I II III < 0.01 IV SD = standard deviation. *Unless otherwise indicated. conditions were defined by patient self-report, physician understanding of medical disease, and diagnostic test interpretation as is standard clinical practice. However, our approach was based on the current methods employed by hospitals. Second, the reliability in identifying comorbid conditions in this study is unknown. Only 1 anesthesiologist and 1 medical records coder reviewed each case. Nevertheless, this is also the routine practice employed by hospitals. Third, our results may have been due to study bias (Hawthorne effect). The physicians who did the coding knew they were being 502 JCOM September 2007 Vol. 14, No. 9

5 studied and may have been more astute in documenting References comorbid conditions. Fourth, we studied surgical patients at 2 academic medical centers. The external validity of these results 1. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): Na- for other settings, patient populations, and operations is tional Academy Press; Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005;293: unknown. Fifth, we did not evaluate how the change in risk of mortality index impacted the hospital s mortality ranking. 3. Marshall MN, Shekelle PG, Leatherman S, Brook RH. The Because many of these hospital ranking systems use proprietary risk-adjustment formulas, we were not able to evaluate gain? A review of the evidence. JAMA 2000;283: public release of performance data: what do we expect to how rankings were influenced. Nevertheless, the increase in 4. Gaynes RP, Platt R. Monitoring patient safety in health care: risk of mortality index was significant. Conclusion The use of concurrent coding of comorbid diseases in surgical patients resulted in 24% of patients having an increased building the case for surrogate measures. Jt Comm J Qual Patient Saf 2006;32: Krumholz HM, Rathore SS, Chen J, et al. Evaluation of a consumer-oriented internet health care report card: the risk of quality ratings based on mortality data. JAMA 2002; risk of mortality index score and 27% having an increased 287: Mack MJ, Herbert M, Prince S, et al. Does reporting of coronary artery bypass grafting from administrative databases severity of illness index score. These results raise further questions about the validity of risk-adjusted mortality accurately reflect actual clinical outcomes? J Thorac Cardiovasc Surg 2005;129: as a measure of quality. Until we develop standardized methods to identify comorbid diseases, patients, providers, 7. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care employers, insurers, regulators, and consumers should be cautious about using risk-adjusted mortality to inform their 1998;36:8 27. perceptions of quality and their health care purchasing decisions. 8. Averill R, Goldfield N, Hughes J, et al. What are APR-DRGs? An introduction to severity of illness and risk of mortality adjustment methodology. St. Paul (MN): 3M; Acknowledgments: We thank Paul Allen, RHIA, CCS, and the Casemix Information Management Department at the Johns Hopkins Health System for generation of the APR-DRG and associated indices 9. Werner RM, Bradlow ET. Relationship between Medicare s hospital compare performance measures and mortality rates [published erratum appears in JAMA 2007;297:700]. JAMA 2006;296: on all patients; Chris Erwin and Teecie Cozad, BSN, DocuSys, Inc., for technical assistance with the use of the computerized module; 10. Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005;293: and Christine Holzmueller, BLA, for assistance in editing the manuscript. 11. Pronovost PJ, Miller MR, Wachter RM. Tracking progress in patient safety: an elusive target. JAMA 2006;296: Corresponding author: Peter J. Pronovost, MD, PhD, 1909 Thames St., 2nd Fl., Baltimore, MD 21231, ppronovo@jhmi.edu. 12. Wachter RM, Pronovost PJ. The 100,000 lives campaign: a scientific and policy review. Jt Comm J Qual Patient Saf 2006;32: Support: DocuSys, Inc. provided the software used for the concurrent computerized coding. 13. Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care 2004;42: Financial disclosures: Drs. Stonemetz and Pronovost are consultants for Docusys and own company stock. 14. Macario A, Vitez TS, Dunn B, et al. Hospital costs and severity of illness in three types of elective surgery. Anesthesiology 1997;86: Author contributions: conception and design, JS, JAU, PJP; analysis and interpretation of data, JCP, JAU, PJP; drafting of the article, JS, JCP, JAU; critical revision of the article, JCP, JAU, PJP; provision of 15. Pronovost P, Dorman T, Sadovnikoff N, et al. The association between preoperative patient characteristics and both clinical and economic outcomes after abdominal aortic surgery. J Cardiothorac Vasc Anesth 1999;13: study materials or patients, JS; statistical experience, JCP; administrative or technical support, PJP; collection and assembly of data, JS, RJM. 16. Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs. Arch Surg 2002;137:20 7. Copyright 2007 by Turner White Communications Inc., Wayne, PA. All rights reserved. Vol. 14, No. 9 September 2007 JCOM 503