Preventing Genetic Testing Order Errors With a Laboratory Utilization Management Program

Transcription

1 AJCP /ORIGINAL ARTICLE Preventing Genetic Testing Order Errors With a Laboratory Utilization Management Program Patrick C. Mathias, MD, PhD, 1 Jessie H. Conta, MS, 2 Eric Q. Konnick, MD, 1 Darci L. Sternen, MS, 2 Shannon M. Stasi, MS, 2 Bonnie L. Cole, MD, 2 Michael L. Astion, MD, PhD, 1,2 and Jane A. Dickerson, PhD 1,2 From the 1 Department of Laboratory Medicine, University of Washington, Seattle; and 2 Department of Laboratories, Seattle Children s Hospital, Seattle, WA. Key Words: Order errors; Genetic testing; Utilization management Am J Clin Pathol August 216;146: DOI: 1.193/AJCP/AQW15 ABSTRACT Objectives: To characterize error rates for genetic test orders between medical specialties and in different settings by examining detailed order information. Methods: We performed a retrospective analysis of a detailed utilization management case database, comprising 2.5 years of data and almost 1,4 genetic test orders. After review by multiple reviewers, we categorized order modifications and cancellations, quantified rates of positive results and order errors, and compared genetics with nongenetics providers and inpatient with outpatient orders. Results: High cost or problems with preauthorization were the most common reasons for modification and cancellation, respectively. The cancellation rate for nongenetics providers was three times the rate for geneticists, but abnormal result rates were similar between the two groups. The approval rate for inpatient orders was not significantly lower than outpatient orders, and abnormal result rates were similar for these two groups as well. Order error rates were approximately 8% among tests recommended by genetics providers in the inpatient setting, and tests ordered or recommended by nongeneticists had error rates near 5% in both inpatient and outpatient settings. Conclusions: Clinicians without specialty training in genetics make genetic test order errors at a significantly higher rate than geneticists. A laboratory utilization management program prevents these order errors from becoming diagnostic errors and reaching the patient. Diagnostic errors, or failures in establishing an accurate and timely explanation of a patient s health problem and communicating that explanation, are estimated to affect 5% of outpatient care visits, or 12 million visits, yearly in the United States. 1,2 These errors are inherently more complex than other forms of medical errors because of the high cognitive requirements of the diagnostic process. Previous studies characterizing the nature of diagnostic errors have demonstrated that a significant proportion of errors is attributable to the diagnostic testing process, including test selection and ordering, test result retrieval, and test interpretation. 3 The diagnostic error literature has focused on primary care settings since these visits are entry points for patients into the health care system. However, diagnostic processes in inpatient and specialty settings may be more complex because of an enrichment of rarer disease entities. To date, the impact of diagnostic error in specialty and inpatient settings is unclear because of limited study in these settings. Of the various diagnostic testing modalities, genetic testing has experienced the largest recent growth (2% yearly) 4 and will likely continue to grow with increased federal funding for precision medicine. The growth of genetic testing menus has added complexity to the test order process, which is arguably one of the simplest steps in the diagnostic process. In addition, genetic testing has expanded into various specialties outside of medical genetics with less experience in diagnosing genetic disorders. At many institutions, a substantial proportion of laboratory send-out budgets comprises genetic testing requests, and increased growth has adversely affected laboratory budgets and American Society for Clinical Pathology, 216. All rights reserved. For permissions, please journals.permissions@oup.com 221 Am J Clin Pathol 216;146: DOI: 1.193/ajcp/aqw15

2 Mathias et al /PREVENTING GENETIC TESTING ORDER ERRORS operations. The combination of these factors has prompted many institutions to establish utilization management (UM) programs to decrease and optimize laboratory test utilization. 5-9 While numerous past studies have described a variety of UM approaches and associated cost savings, there has been limited characterization of the impact of these programs on genetic test order errors and their impact on the diagnostic process. Characterizing these errors is inherently difficult for laboratories because test order requisitions and computerized order entry offer limited ancillary information with which to determine test appropriateness and the specific clinical setting in which a test was ordered. Furthermore, access to electronic health records (EHRs) may not provide the detailed information required to assess a specific test order if the documentation lacks sufficient detail. Despite such challenges, some UM programs have had success in characterizing the reasons orders were cancelled or modified. Starting with information from test requisitions, Miller et al 1 have described overall modification/ cancellation rates of 26% from the perspective of a reference laboratory, with more than one-third of these orders requiring the cancellation of an incorrect order. Because a small number of commonly misordered tests drove many of their cancellations, it is unclear how well their statistics characterize a single medical center. Riley et al 9 provide multiple examples of order errors in describing their twotiered approach of UM by electronic clinical decision support and order review, but the frequency of such errors was not characterized. Greenblatt et al 8 described the rejection of 12% of send-out tests (including but not limited to genetic tests) through their UM program and determined that more than half had questionable indications given the patient s situation. In this study, we describe the impact of a UM program that was established in 212 to improve the value of genetic testing services for patients and providers. 5 To characterize genetic test order errors, we leveraged a UM case database, in which every test under UM was logged by a consultant, as a detailed data source with relevant diagnostic information ancillary to computerized test orders and the EHR. Because each genetic test order entry includes details of the clinical situation as recorded by a laboratory genetic counselor or pathologist, we were able to categorize specific reasons for test modification or cancellation and determine the proportion of orders in which a test order error was detected and corrected. In this study, we describe the rate of genetic test order errors observed across almost 1,4 genetic test orders over a 3-month period and characterize differences between genetics and nongenetics providers as well as between outpatient and inpatient test orders. Materials and Methods UM Program The laboratory UM program was established in 212 at a pediatric, tertiary care medical center and has been described previously. 5 Laboratory orders fulfilling the following criteria were reviewed by doctoral-level faculty or genetic counselors before sending samples: orders with costs (to the laboratory) above a cost threshold, multiple genetic tests on one order, orders requested to be sent to a nonpreferred laboratory, orders requested to be sent out for tests performed in-house, orders requested to be sent to an international laboratory, and orders for tests predefined as requiring review because they are known to be associated with errors or other problems during the order process (eg, familial mutation testing). The cost threshold was lowered from $1, to $7 during the course of this study (starting January 1, 214). Case data for all tests fulfilling review criteria were entered in a custom Microsoft Access (Microsoft, Redmond, WA) database as part of the routine workflow, which included patient demographics, inpatient or outpatient status at the time of test order, ordering physician name and specialty, and specific order details such as test name and cost. Ordering providers who were listed as members of the institution s genetics division (including biochemical genetics) in the institutional provider database were classified as genetics providers. All other providers were classified as nongenetics providers. Pertinent information for any orders fulfilling the specified criteria were organized by send-out staff and ed to send-out consultants for review. Consultants gathered information about whether the desired test was ordered, the medical necessity of the testing, whether insurance preauthorization was obtained, and whether each of these items was obtained through electronic medical chart review or by direct communication by or telephone. Consultants and laboratory staff also played a significant role in managing the orders and results for tests ordered in a sequential process by helping to develop the testing plan and facilitating reflexive testing steps. Case Categorization On the basis of the clinical indication and test that was ordered, the consultant recommended the order be approved, modified, or cancelled. If the ordering provider agreed with the modification or cancellation, the test was categorized as such. Tests that were modified were subclassified based on whether there was an error in the order that was corrected, a lower cost laboratory was selected, or the laboratory test order was improved by selecting an alternate 222 Am J Clin Pathol 216;146: American Society for Clinical Pathology 222 DOI: 1.193/ajcp/aqw15

3 AJCP /ORIGINAL ARTICLE test. Cancelled tests were similarly categorized into groups reflecting cancellations based on cost alone, duplicate test orders, lack of insurance preauthorization, deferral of testing (eg, to wait for the results of other testing), new information during the course of review that made the test no longer necessary, or inappropriateness given the medical situation. Test results were classified as negative, positive, variant of uncertain significance, or inconclusive. If multiple variants were detected, the test result was classified as positive only if at least one of the variants could explain the patient s clinical presentation. The test result was classified as containing a variant of uncertain significance or inconclusive if at least one of the variants was reported as a variant of clinical significance in the clinical test result report or if it was unclear whether the variant detected was causative for the patient s condition. Otherwise, the test was classified as having a negative result. Retrospective Analysis For this study, we performed a retrospective analysis of orders reviewed by consultants between September 212 and February 215. We excluded orders from patients not seen at this institution as well as nongenetic test orders. To establish a rate of test order error, we included test orders that were modified to a correct test due to an error in addition to tests cancelled because they were deemed inappropriate given the medical situation (and the ordering provider agreed to cancel the order). Since establishing the reason a test was modified or cancelled requires some subjectivity, all modified and cancelled orders were reviewed retrospectively by two authors not involved in the original order adjudication for comparison with the original reason for modification/cancellation. Discrepancies in the reason for modification or cancellation were resolved by consensus between three genetic counselors and two pathologists. In addition, cases were flagged by the group if there was consensus that an order error had diagnostic implications for the patient. Summary statistics for data extracted from the database were calculated in the R statistical computing environment (R Foundation for Statistical Computing, Vienna, Austria). Comparisons of outcomes to test requests, rates of positive results and results of uncertain significance, and error rates between relevant subgroups (genetics vs nongenetics providers and outpatient vs inpatient orders) were performed using the two-sample test of equality of proportions (v 2 test with Yates continuity correction), with adjustment of P values for multiple comparisons by the Holm-Bonferroni method when applicable. This work was approved by the local institutional review board. Results We reviewed data from 1,393 genetic test orders over 2.5 years to quantify the impact of the UM program on decreasing unnecessary testing and correcting test order errors. Overall, more than 18% of orders were modified or cancelled and 15% of orders were performed as part of a sequential diagnostic pathway with UM consultant input Table 1. The most common reason for modification of an order was due to cost (5.2% of all orders), and the most common reason for cancellation was a lack of preauthorization (3.3% of all orders), which is required for genetic testing at this institution Table 2. The overall error rate, defined as the sum of orders that were corrected during review (modified corrected) and orders cancelled because they were deemed inappropriate given the clinical situation (cancelled wrong) divided by the total number of orders, was 3.%. Because orders from genetics specialists constituted the majority of orders and each individual specialty outside of genetics had a smaller number of orders, we stratified the data set to determine UM decision rates and positive result rates for genetics (n ¼ 861) and nongenetics (n ¼ 532) ordering physicians Figure 1. While the approval rate between genetics and nongenetics providers was not dramatically different (69% vs 65%, P ¼.25), the cancellation rate was significantly higher for nongenetics providers (15% vs 4.5% Table 1 Overall s for All Orders No. (%) of Total Orders Approved 936 (67) Sequential 25 (15) Modified 131 (9.4) Cancelled 121 (8.6) Table 2 Detailed s for Modified and Cancelled Orders No. of Cases % of Modified/ Cancelled Orders Modified cost Cancelled no preauthorization Modified improved Cancelled deferred Cancelled wrong Modified corrected Cancelled duplicate Cancelled cost Cancelled new information % of Total Orders American Society for Clinical Pathology Am J Clin Pathol 216;146: DOI: 1.193/ajcp/aqw15

4 Mathias et al /PREVENTING GENETIC TESTING ORDER ERRORS A 6 B P <.1 P < P =.39 Approved Cancelled Sequential for genetics providers, P <.1), and the rate of sequential testing was higher among genetics providers (17.9% vs 9.6% for nongenetics providers, P <.1). The positive result rate was not significantly different between genetics and nongenetics providers (27.8% and 28.6%, respectively), although tests ordered by geneticists more frequently contained a variant of uncertain significance in the results (13.% for genetics and 7.9% for nongenetics, P ¼.39). By stratifying genetic test orders by setting in which the test was ordered (inpatient, n ¼ 23, or outpatient, n ¼ 1,19), we determined the approval rate, positive result rate, and error rate differences between the two settings Figure 2. The difference between approval rates was not significantly different between the two settings (64% for inpatient and 68% for outpatient, P ¼.53), but the inpatient order modification rate was more than twice the outpatient rate (17% for inpatient vs 8.1% for outpatient, P <.1). Negative Positive Result Variant/Inconclusive Figure 1 Comparison of solutions (A) and rates (B) of different types of results for genetics and nongenetics providers. A P <.1 Approved Cancelled Sequential B Negative Positive Result Variant/Inconclusive Figure 2 Comparison of solutions for tests ordered in outpatient and inpatient settings (A) and rates of different types of results for outpatient and inpatient test orders (B). There was no significant difference in positive result rate (27% for inpatient and 28% for outpatient, P ¼.99). Orders from nongenetics providers had a significantly higher rate of error than genetics orders (5.2% for nongenetics vs 1.7% for genetics, P <.1). orders were more likely to have an order error than outpatient orders (6.9% of inpatient orders vs 2.4% of outpatient orders, P ¼.1). Figure 3 illustrates a significant contrast in the error rate of inpatient orders vs outpatient orders by genetics providers (8.3% vs 1.%, P <.1), while orders from nongenetics providers had a similar rate of error in both settings (5.9% for inpatient and 4.8% for outpatient, P ¼.83). Upon detailed review of order errors, we determined that 11 of the 42 ordering errors may have had diagnostic implications for the patient either a diagnosis could have been missed or the testing would have failed to rule out a condition on the differential (Supplemental Table 1; all supplemental materials can be found at American Journal of Clinical Pathology online). 224 Am J Clin Pathol 216;146: American Society for Clinical Pathology 224 DOI: 1.193/ajcp/aqw15

5 AJCP /ORIGINAL ARTICLE Percent Error Discussion Specialty and Setting Figure 3 Percentage of order errors (calculated from total orders) by combination of specialty and order setting. UM of laboratory testing has demonstrated significant financial savings across multiple institutions and promises to improve the diagnostic testing process for patients, but the clinical benefits from UM programs can be difficult to measure. In this work, we take advantage of a detailed view of a large number of laboratory orders in a UM database to better understand the program s impact on the test selection and ordering steps of the genetic diagnostic testing process at a pediatric tertiary care institution. Overall, the management process affects one-third of all genetic test orders fulfilling the criteria for review: greater than 18% of orders reviewed were modified or cancelled, and another 15% of orders were performed sequentially, with substantial laboratory staff involvement. When comparing medical genetics providers with those in other specialties, we found that overall approval rates were not different, but geneticists were more likely to pursue sequential testing, and nongeneticists had more orders cancelled following UM review. Interestingly, the positive result rate was not different between geneticists and nongeneticists. However, this may reflect the influence of the UM program on cancelling tests that are unlikely to be informative rather than differences in how effectively the providers are using the tests. Comparison of inpatient and outpatient testing also showed no significant differences in positive result rate, but the modification rate for inpatient orders was almost twice that for outpatient orders. Because we had a combination of manually curated clinical information and details about the adjudication for a particular order, we were able to specifically classify the reasons why each order was modified or cancelled. More than half of modifications or cancellations decreased costs by selection of an alternate reference laboratory or were prompted by the institution s insurance preauthorization policies (which were instituted to prevent patients from receiving unexpected large bills due to rejection of claims by insurance carriers). Fifteen percent of modified or cancelled orders, or 3% of the genetic test orders overall, were cancelled/modified due to an order error, which we defined as a cancellation because the test was not appropriate given the clinical situation or a modification because testing was indicated but an incorrect test was ordered. The order error rate for nongenetics providers (5% of total orders from nongenetics providers) was approximately three times the order error rate for genetics providers, which may reflect the combination of a lack of experience and the increasing complexity of genetic testing. Similarly, the inpatient order error rate (7% of total inpatient orders) was three times that observed in outpatients. When evaluating the error rates based on the combination of provider specialty and patient setting, inpatient orders from genetics providers had the highest rate of error, 8%. This high order error rate likely reflects the fact that inpatient orders are entered by residents who have less experience ordering genetic tests and are acting on behalf of consultant recommendations, which may not be communicated in sufficient detail. Of the orders with errors, we found that approximately one-fourth (or 1% of all orders) could have had diagnostic implications for the patient if the order was not corrected, demonstrating that the UM program not only decreases waste but also helps prevent diagnostic errors. This study is the first to our knowledge that quantifies the genetic test order error rate for multiple provider subgroups at a pediatric tertiary care center. One important strength of this work was the additional test information stored in the UM database, which combines test order information with classifications of the results of the UM process, case histories, and testing rationales added by genetic counselors and pathologists. Determining test appropriateness and the risk of diagnostic error would be much more difficult based on retrospective clinical note review alone because details about why tests are ordered are often limited in notes, and the interaction with laboratory consultants will not necessarily be captured in the clinical record. Another strength of the study was the ability to review test ordering patterns in almost 1,4 orders, which was possible because of the upfront work completed by UM consultants. Although the review was retrospective, important information about genetic test orders was assembled at the time of order, which greatly reduced the work required to review such a large number of cases. While the study leverages a unique data source to quantify the impact of one laboratory UM program on eliminating waste, decreasing order errors, and preventing diagnostic errors, there are some limitations in applying these findings to broader populations. This study was American Society for Clinical Pathology Am J Clin Pathol 216;146: DOI: 1.193/ajcp/aqw15

6 Mathias et al /PREVENTING GENETIC TESTING ORDER ERRORS performed in a single pediatric tertiary care center, which receives referrals for complex patients across a broad geographic area. Genetic test orders may be more complex than those seen in other institutions, but many specialists have expertise in rare diseases, so it is unclear whether the overall error rate of 3% would be observed at other institutions. Quantification of error rates using a uniform method across multiple institutions could provide a better gauge of the problem in other settings. Another limitation of our study is that the UM program evaluated only approximately one-third of the genetic tests ordered, since only send-out orders above a certain cost or tests flagged based on prior knowledge of ordering difficulties are reviewed. The data set is therefore enriched for send-out testing and does not capture ordering errors for in-house genetic tests that providers may be more familiar with. Finally, the detailed review of order errors for diagnostic impact was retrospective and relied heavily on information entered into the UM database as well as available clinical notes. The complete details required to best classify an order error may not have been available, so we relied on consensus from multiple reviewers for accurate classification. The findings of this study reinforce past work suggesting that send-out testing presents a high risk for order entry error. 11 We find that in addition to order entry error, additional ordering errors occur even when a provider correctly inputs an order for the test he or she intends to enter. While a significant proportion of these errors represents waste, we discovered that one-fourth of order errors in this population could have diagnostic implications for the patients. These findings emphasize that developing system solutions to handle the total genetic testing process should be a priority for laboratories and health systems. For institutions without the necessary resources to put in place a structured UM program, privileging of genetic test orders to genetics specialists could prevent errors associated with inpatient orders or orders from nongenetics providers, since error rates for these groups were greater than 5%. However, even orders from genetics providers had an error rate of nearly 2%, which emphasizes the utility of a structured UM program that relies on expert review by laboratory professionals, including genetic counselors, geneticists, and pathologists. The diagnostic process for patients who have a potentially rare disease is inherently complex and has become more so with growth of genetic testing menus. Establishing systems in which the laboratory is more actively involved in the diagnostic workup is important to provide the highest value care to patients. Corresponding author: Jane A. Dickerson, PhD; jane.dickerson@seattlechildrens.org. References 1. National Academics of Science Engineering, Medicine. Improving Diagnosis in Healthcare. Washington, DC: National Academies Press; Singh H, Meyer AND, Thomas EJ. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations [published online April 17, 214]. BMJ Qual Saf. 3. Singh H, Giardina T, Meyer A, et al. Types and origins of diagnostic errors in primary care settings. JAMA Intern Med. 213;173: United Health Center for Health Reform & Modernization. Personalized Medicine: Trends and Prospects for the New Science of Genetic Testing and Molecular Diagnostics UNH-Working-Paper-7.ashx. 5. Dickerson JA, Cole B, Conta JH, et al. Improving the value of costly genetic reference laboratory testing with active utilization management. Arch Pathol Lab Med. 214;138: Kim JY, Dzik WH, Dighe AS, et al. Utilization management in a large urban academic medical center: a 1-year experience. Am J Clin Pathol. 211;135: Warren JS. Laboratory test utilization program: structure and impact in a large academic medical center. Am J Clin Pathol. 213;139: Greenblatt MB, Nowak JA, Quade CC, et al. Impact of a prospective review program for reference laboratory testing requests. Am J Clin Pathol. 215;143: Riley JD, Procop GW, Kottke-Marchant K, et al. Improving molecular genetic test utilization through order restriction, test review, and guidance. J Mol Diagn. 215;17: Miller CE, Krautscheid P, Baldwin EE, et al. Genetic counselor review of genetic test orders in a reference laboratory reduces unnecessary testing. Am J Med Genet Part A. 214;164: Valenstein PN, Walsh MK, Stankovic AK. Accuracy of sendout test ordering: a College of American Pathologists Q-Probes study of ordering accuracy in 97 clinical laboratories. Arch Pathol Lab Med. 28;132:26-21.[26:AOSTOA]2..CO; Am J Clin Pathol 216;146: American Society for Clinical Pathology 226 DOI: 1.193/ajcp/aqw15