Total testing process applied to therapeutic drug monitoring: impact on patients outcomes and economics

Clinical Chemistry 44:2 370 374 (1998) TDM Conference Total testing process applied to therapeutic drug monitoring: impact on patients outcomes and economics Gerald E. Schumacher* and Judith T. Barr The Total Testing Process (TTP) refers to the sequence of 11 steps of laboratory testing, beginning with a clinical question prompted by the patient clinician encounter and concluding with the impact of the test result on patient care. TTP when applied to therapeutic drug monitoring (TDM) emphasizes that TDM must be considered a process involving a series of steps and interrelated activities and not viewed simply as a numerical value for a serum drug concentration. TTP is also an ideal format for organizing and identifying the system-related and patientcentered variables used in outcomes assessment of TDM, as well as providing a template for collecting the cost data needed for economic analyses. Examples are provided for improving application of TDM by practitioners, clinical laboratories, and educators. Center for Outcomes Assessment in Healthcare, Bouvé College of Pharmacy and Health Sciences, Northeastern University, 360 Huntington Ave., 105 DK, Boston, MA 02115. *Author for correspondence. Fax 617-373-2968, e-mail G.Schumacher@ lynx.neu.edu. Received July 28, 1997; revision accepted October 27, 1997. The Total Testing Process (TTP) is one of the systems used in applying quality-management approaches to the clinical laboratory [1 4]. The relationship between TTP, therapeutic drug monitoring (TDM), and outcomes assessment is clear, as shown by the following definitions: TTP refers to all aspects of the 11 steps of laboratory testing that begin with a clinical question prompted by a patient clinician encounter and conclude with the impact of the test result on patient care [3, 4]. TDM is a practice applied to a small group of drugs in which there is a direct relationship between serum drug concentration and pharmacological response, as well as a narrow range of concentrations that are effective and safe, and for which the concentration measurements are used in conjunction with other measures of clinical observation to assess a patient s status [5]. Outcomes assessment refers to the quantitative evaluation of certain variables that characterize the result or impact of an intervention; in this case, the use of TDM or clinical pharmacokinetic services represents the intervention [6 8]. Therefore, TTP describes the full sequence of laboratory testing activities, which, when applied to the analysis and interpretation of serum drug concentrations, leads to decisions that influence patient outcome resulting from drug therapy. the total testing process The 11-step sequence for TTP, outlined in Fig. 1 [3, 4], begins and ends with patient care and consists of four major components: preanalytical, analytical, postanalytical, and the regulatory environment in which the test is performed. The 4 steps of the preanalytical component (steps 1 4) begin with a clinician patient encounter that raises a clinical question capable of being answered by a test. The clinician considers what tests might be appropriate to answer the clinical question and then selects one or more. The selection is translated into a test order. The preanalytical component concludes with specimen collection and the transport of the specimen to the clinical laboratory. The 3 steps of the analytical component (steps 5 7) involve the intralaboratory processing and testing of specimens and the verification of test results. The 4 steps of the postanalytical component (steps 8 11) consist of the reporting of the test result, the interpretation of the test result in light of the original clinical question, the actions taken in response to the result, and the result of these actions on the patient s care and the ultimate outcome for the patient. TTP, therefore, has direct application to the field of TDM [4]. TTP requires that we consider TDM as a full system, not simply a numerical value for a serum drug concentration. At a minimum, we should improve the interpretation of TDM testing by identifying all steps of the TDM testing procedure, determining where variations and errors can occur, and interpreting TDM results in the light of these factors. But we should also move beyond improved interpretation to build 370

Clinical Chemistry 44, No. 2, 1998 371 Fig. 1. The total testing process [3, 4]. Reproduced with permission from Appleton & Lange. TDM testing systems that prevent errors from occurring in all steps of the TTP and that maximally contribute to an improved patient outcome. outcomes assessment Outcomes assessment of TDM is complicated by definitional issues [6]. For some, TDM is a laboratory test, but for others, the test is but part of a larger TDM system in healthcare delivery. As a laboratory test, TDM is used to determine the serum drug concentration for medications that have a narrow concentration range associated with therapeutic effectiveness. According to the principles of TTP, the performance of a TDM test should incorporate an assessment of the clinical indication for testing, the timing and collection of the specimen, the performance of the analysis, the proper interpretation of the result, the initiation of indicated action, and the impact of the action on patient care. TDM, however, can also connote an organized system of care, usually called a TDM service or a clinical pharmacokinetic service. As a system of care, a TDM service guides or intervenes in the TTP to ensure that the serum drug concentration will have maximal positive impact on patient care. To be optimally effective, assessment of a TDM service should include its impact not only on patient care, but also on the economic efficiency of the delivery system. TTP provides a coherent and effective template for performing economic analyses of TDM. Direct and indirect costs, effects, or benefits of TDM as an intervention may be assigned to the appropriate steps in TTP to calculate cost-effectiveness or cost benefit analyses of TDM as a monitoring activity or as a complete service. How do we assess the outcomes of TDM? In both definitions, be it as a test result or a full system of care, the critical outcome measure for TDM is the effect on patient care. Donabedian, in speaking generally about the quality of care, considered the interrelationship of three components influencing care: structure, process, and outcome [9]. As applied to healthcare, the structural component refers to the patient and provider variables involved in the system of care: (a) patients variables such as age, gender, and comorbidity and providers variables such as the site of care (e.g., hospital, clinic, physician office, home), (b) the characteristics and training of the practitioners providing the care (e.g., generalist or specialty practitioner, technologist, technician), and (c) the method of payment. With reference to TTP depicted in Fig. 1, the structural component is most closely aligned with steps 1 4. the process component includes the procedures for delivering the intervention, the methods of communication and counseling, and (for the laboratory) the types of tests and the quality of test performance. In Fig. 1, the process component, while clearly reflecting all 11 steps of TTP as a system, is most identified with steps 3 8. the outcome component describes the noneconomic results (e.g., cure, failure, change in physiological variables or functional status or health-related quality of life, patient s satisfaction) and economic consequences (e.g., costs of care) resulting from the structure and process components of the intervention. Steps 9 11 of Fig. 1 are most closely associated with the outcome component. The outcome component of care is what is then used to provide feedback for modifying the structure and process components of care, where needed. Applied to TTP, an evaluation of the outcomes resulting from steps 9 11 over time may be used to modify the structure and process variables characterized in steps 2 8, to improve care. Unfortunately, analyses of TDM to date have focused more not only on process rather than outcome components but also on what we refer to as system-related rather than patient-centered outcomes [6]. While all of these considerations process, outcome, system-related, patient-centered are important indices in measuring performance, in the future more emphasis needs to be placed on patient-centered outcomes. System-related measures refer to and characterize how well the procedure works. Patient-centered measures characterize the effect of the intervention on the status of the patient. Examples of these distinctions, in comparing process to outcome measures related to TDM, are as follows: Process measure the fraction of patients who have serum drug concentrations measured, and at the correct times, for expected drugs. In Fig. 1, this is included in steps 3 and 4. Outcome measure the fraction of patients whose serum drug concentrations are in the appropriate therapeutic ranges. This is characterized by step 8, and sometimes steps 9 and 10, of Fig. 1. Examples of contrasting system-related and patientcentered outcome measures in TDM include: System-related outcome measure the fraction of

372 Schumacher and Barr: TDM impact on outcomes and economics patients who have serum drug concentrations in the appropriate therapeutic range (step 8, and perhaps steps 9 and 10, of Fig. 1). Patient-centered outcome measure the fraction of patients in TDM who are treated effectively and without drug-induced adverse effects. This is included in steps 10 and 11 of Fig. 1. Examining the difference between a system-related and patient-centered outcome measure shows an important distinction between determining how effectively a system of TDM achieves drug concentrations within the appropriate therapeutic or target concentration range (a measure of the effectiveness of adherence to the system s procedures) and how effectively TDM achieves drug effectiveness without drug-induced toxicity a true measure of patients response. A drug value within a commonly accepted therapeutic range is not synonymous with an effective and safe response to treatment with the drug. Patient-centered outcome measures are most directly related to clinical effectiveness of the test or service; historically, however, studies evaluating TDM have concentrated on examining the processes of TDM testing and providing TDM services, rather than on assessing their impact on patient care [6]. Furthermore, these studies have considerably more often used system-related rather than patient-centered outcomes in assessing the effectiveness of clinical pharmacokinetic services. improving use of tdm by practitioners, laboratorians, and educators We believe that practitioners, laboratory services, and educators can do more to promote effective TDM for individualizing drug therapy than is commonplace [4, 10 15]. Practitioners and laboratories can devise better methods for monitoring comprehension of and compliance with the various components within the 11 steps of TTP, practitioners can use information contained in the serum drug concentration more expansively and effectively than is typical, laboratories can improve the application and reporting of serum drug concentrations and interpretation, and educators can teach students and practitioners the theory and application of Bayesian probability revision, test performance characteristics, and economic analyses necessary to make more effective use of TDM. It is unclear which of the above strategies is most likely to be effective in improving TDM. Improved communication and education, especially if provided at regular intervals, should reap some benefit. Improved surveillance of the unit processes involved in TDM is important. However, because it is so rarely discussed, and because we have worked on this strategy for some time, we shall focus for the remainder of this presentation on improving practitioner use of test performance characteristics. We do not suggest that this is the most important of the above strategies, but we do believe that practitioners can use drug concentrations more effectively than is commonly done. improving practitioner use of tdm and test performance characteristics If practitioners are to extract more information from the patient s serum drug concentration than is currently realized, they will need to become more familiar with the application of probability to fully interpret the significance of serum drug concentrations. We consider that measuring a patient s serum drug concentration represents a diagnostic test for TDM, akin to the use of clinical laboratory tests in medicine. The drug measurement contains probabilistic information that allows practitioners to assess the probability of the patient s status. Therefore, to guide decisionmaking, practitioners need to become more facile with using test performance characteristics, Bayesian probability revision, and probability thresholds. Test performance characteristics associated with use of drug concentrations as diagnostic tests for assessing the probability of drug effectiveness or drug-induced toxicity, or both, in patients requires knowledge of provider-focused indices such as predictive values, likelihood ratio, and ratio of net consequences. Table 1 records the use of these test performance characteristics for two prototypical drugs used in TDM, theophylline and phenytoin, examples selected to demonstrate the different influences of these indices on practitioner interpretation of drug concentration data [11]. Given that 20 and 22 mg/l for theophylline and phenytoin, respectively, represent commonly accepted drug concentration cutoff values values used to represent the top of the therapeutic range for separating nontoxic patients from patients with drug-induced toxicity the positive predictive values for these concentrations say that just 43% of theophylline patients and only 20% of phenytoin patients with drug values greater than the cutoff are truly drug toxic; on the other hand, the negative predictive value of drug Table 1. Test performance characteristics (TPC) for theophylline and phenytoin serum drug concentrations. TPC Serum concn cutoff, mg/l a Theophylline, 20 Phenytoin, 22 Positive predictive value b 0.43 0.2 Negative predictive value c 0.96 0.83 Likelihood ratio positive d 5.6 1.4 Likelihood ratio negative e 0.4 0.9 Ratio of net consequences f 0.27 0.5 a Value used as a separator for classifying patients as toxic or nontoxic. Sources: [11] for theophylline, [12] for phenytoin. b The fraction of patients with a positive test result ( test cutoff) who are truly drug-induced toxic. c The fraction of patients with a negative test result ( test cutoff) who are truly not drug-induced toxic (nontoxic). d Ratio of true-positive rate to false-positive rate; also, the ratio of posttest odds to pretest odds after a positive test result. e Ratio of false-negative rate to true-negative rate; also, the ratio of posttest odds to pretest odds after a negative test result. f Approximated by the ratio of the consequences (utility) of a false-positive result to the consequences (utility) of a false-negative result. More formally, the range for likelihood ratio positive times the ratio of the prevalence of toxicity to the prevalence of nontoxicity.

Clinical Chemistry 44, No. 2, 1998 373 concentrations below the cutoff values indicates that 96% and 83% of the theophylline and phenytoin patients, respectively, are truly nontoxic. This is much more useful information than merely using the cutoff value as a dichotomous measure for classifying patients as toxic or nontoxic. One must appreciate, however, that the predictive value is population-sensitive; that is, the value changes with the prevalence of the condition in the population being sampled. Accordingly, when practitioners use predictive value information, it is necessary to be sure that the population from which the predictive value was obtained is similar to the population in which it is being applied [11 13, 16]. Using serum drug concentrations in conjunction with likelihood ratio information enhances the application of the drug value in decisionmaking. This requires recognizing that the odds ratio form of Bayes probability revision [where odds % probability/(100 - % probability)] states that [10, 16]: Pretest odds of condition likelihood ratio posttest odds of condition The data in Table 1 show that for theophylline a positive test result ( 20 mg/l) increases the posttest odds of toxicity to 5.6 times more than the pretest odds, whereas a negative test result ( 20 mg/l) cuts the posttest odds to 40% of the pretest odds. Phenytoin serum drug concentration data are much less informative. A positive test result increases pretest odds by 40%, whereas a negative test result decreases the pretest odds by just 10%. Using test performance characteristics in this manner allows practitioners to engage a personal pretest assessment of a patient s status and then apply the probabilistic information contained in the serum drug concentration to estimate a posttest assessment of status. The remaining index in Table 1, the ratio of net consequences, expresses the relative consequences of a false-positive to a false-negative classification [10, 11]. Knowledge of this index provides practitioners with the relative risk associated with classification errors that is associated with a given cutoff value. For theophylline, the selected cutoff shows that a false-positive error is deemed 25% as risky as a false-negative error; for phenytoin, the ratio is 50%. Adjusting the cutoff upward increases the relative concern for a false-positive vs that for a falsenegative, whereas adjusting the cutoff downward increases the relative concern for a false-negative error. Knowledge of test performance characteristics also allows the practitioner to know under what conditions the information about a patient s serum drug concentration provides data that can alter clinical decisions. In other words, when does ordering a serum drug concentration actually have the potential to contribute useful information? A basic principle of decisionmaking dictates that a drug measurement (or any diagnostic test, for that matter) should be ordered only if the test result has the potential for revising the pretest probability sufficiently that the posttest probability may cause practitioners to change their decisions about the status of patients or about the most appropriate course of action [10, 15]. In practice, a clinician has a perception about what posttest probability of a patient s condition will lead to a change in the course of action. In the case of continuing or discontinuing a drug on the basis of the probability of drug-induced toxicity, used as an example in Table 2, T dc represents the practitioner s threshold probability for discontinuing the drug, and T c,t and T t,dc bound the probability testing window, below which ( T c,t ) the likelihood ratio positive (for a positive result) is insufficient for a positive test result to raise the posttest probability of drug-induced toxicity above the decision threshold, T dc, and above which ( T t,dc ) the likelihood ratio negative (for a negative result) is insufficient for a negative test result to decrease the posttest probability of toxicity below the decision threshold. For illustration, when the practitioner sets the decision threshold at a probability of 0.33 for drug-induced toxicity, as shown in Table 2, a pretest probability 0.08 for theophylline suggests that the practitioner should continue the drug without first ordering a test because the likelihood ratio positive resulting from a test result is insufficient to alter the decision. At the opposite end of the scale, when the pretest probability is 0.57, discontinuing the drug without ordering its measurement is appropriate. When the pretest probability is between 0.08 and 0.57, however, ordering a drug measurement may lead to altering the decision: A negative test result urges continuing treatment; a positive test warrants discontinuing treatment. As noted earlier in Table 1, the phenytoin serum concentration is less useful in providing informa- Table 2. Threshold probability values [15] for the decision alternatives of (a) acting without testing first and (b) testing before acting. Drug Test cutoff concn, mg/l Likelihood ratio a b T dc c T c,t d T t,dc Theophylline 20 5.6 0.33 0.08 0.57 Phenytoin 22 1.4 0.33 0.26 0.36 a For a positive test result (serum drug concentration test cutoff). b Threshold probability (pretest or posttest probability) for the decision alternatives of continuing the drug regimen (c) and discontinuing the drug regimen (d). c Threshold probability (pretest) for the decision alternatives of continuing the regimen (c) without drug concentration testing, and obtaining a serum drug concentration (t) before taking action; then discontinuing drug if the test is positive and continuing if the test is negative. d Threshold probability (pretest) for the decision alternatives of discontinuing the regimen (dc) without drug concentration testing, and obtaining a serum drug concentration (t) before taking action; discontinuing drug if the test is then positive and continuing if the test is negative.

374 Schumacher and Barr: TDM impact on outcomes and economics tion. In Table 2 this finding is reinforced, because only over the narrow probability testing window of 0.26 0.36 does ordering a serum drug concentration yield useful information with respect to drug-induced toxicity. In general, as the T dc decreases (i.e., as the probability threshold for the decision to discontinue a drug is lowered), the window for testing before deciding shrinks; T c,t increases and the T t,dc decreases [10, 15]. Although the examples in Tables 1 and 2 focus on decisions of drug-induced toxicity and nontoxicity, the same principles apply for decisions of drug effectiveness vs noneffectiveness; however, the available data are more sparse. Therefore, knowledge of test performance characteristics and the use of these data in Bayesian probability revision is important for practitioners. It improves the value of TDM and should improve patients outcomes. But practitioners can use data only when they are made available. At present, much of the data are limited by small sample studies or are simply not yet published. Herein lies the potential for improved laboratory participation in TDM through use of steps 8 and 9 of TTP. Laboratories should design better methods of communicating test results providing test performance characteristics, probability data, reference citations, and recommendations accompanying the serum concentration where the information is available. For drugs for which the test performance values are not known, or when patient populations or healthcare sites have the potential to differ from known population values, the laboratory should work with clinicians to collect, prospectively or retrospectively, the site-specific or population-specific data. Laboratories should also collaborate with other clinicians to collect economic data necessary to perform cost-effectiveness or cost benefit analyses of TDM. what has been done to evaluate tdm Some 90% of studies assessing TDM have focused on system-related rather than patient-centered evaluations, as shown in a thorough review of studies through 1993 [6]. The majority of these studies examined, with positive results for the use of TDM, the effects on (a) the fraction of patients with serum concentrations in the commonly accepted therapeutic range, (b) the fraction of patients whose concentrations were sampled at the appropriate time and for the appropriate indication, and (c) the percentage of practitioner responses to TDM results and recommendations that seemed correct. Too few studies have demonstrated with use of sound research methodology that TDM improves patient-centered outcomes by reducing the rate of drug-induced toxicity, improving the success rate of treatment, or decreasing the time for resolution or improvement of morbidity [6]. Ried et al., presenting a metaanalysis of studies through the 1980s, concluded that TDM does reduce drug-induced toxicity for a few drugs [17]. A few studies have used appropriate methodology to effectively evaluate the cost-benefit or cost-effectiveness of TDM [6]. Destache reviewed some of these efforts [18]. In conclusion, TTP represents a coherent framework for evaluating system-related and patient-centered processes and outcomes of care, as well as economic analyses. Using TDM for individualizing treatment will be enhanced when (a) practitioners and laboratories improve techniques for monitoring adherence to the steps of TTP, (b) clinical laboratories improve the breadth and depth of reporting TDM results, and (c) practitioners and laboratories increase the application of test performance characteristics and the use of Bayesian probability techniques. References 1. Inhorn SL, Addison BV, eds. Proceedings of the 1986 institute on critical issues in health practices: managing the quality of laboratory test results in a changing health center environment. Wilmington, DE: DuPont, 1987. 2. Martin ML, Addison BV, Eagner WM, Essien JDK, eds. Proceedings of the 1989 institute on critical issues in health laboratory practice: improving the quality of health management through clinician and laboratorian teamwork. Wilmington, DE: DuPont, 1991. 3. Barr JT, Silver S. The total testing process and its implications for laboratory administration and education. Clin Lab Manage Rev 1994;8:526 42. 4. Barr JT, Schumacher GE. The total testing process applied to therapeutic drug monitoring. In: Schumacher GE, ed. Therapeutic drug monitoring. East Norwalk, CT: Appleton & Lange, 1995:47 82. 5. Schumacher GE. Introduction to therapeutic drug monitoring. In: Schumacher GE, ed. Therapeutic drug monitoring. East Norwalk, CT: Appleton & Lange, 1995:2. 6. Barr JT, Schumacher GE. Outcomes assessment of therapeutic drug monitoring. In: Schumacher GE, ed. Therapeutic drug monitoring. East Norwalk, CT: Appleton & Lange, 1995:191 236. 7. Epstein AM. The outcomes movement will it get us where we want to go? N Engl J Med 1990;323:266 70. 8. Ellwood PM. Outcome management: a technology of patient experience. N Engl J Med 1988;318:1549 56. 9. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44:166 206. 10. Schumacher GE, Barr JT. Decision analysis applied to therapeutic drug monitoring. In: Schumacher GE, ed. Therapeutic drug monitoring. East Norwalk, CT: Appleton & Lange, 1995:143 90. 11. Schumacher GE, Barr JT. Using population-based serum drug concentration cut-off values to predict toxicity: test performance and limitations compared with Bayesian interpretation. Clin Pharm 1990;9:88 96. 12. Schumacher GE, Barr JT, Browne TR, Collins JF. Test performance characteristics of the serum phenytoin concentration (SPC): the relationship between SPC and patient response. Ther Drug Monit 1991;13:318 24. 13. Schumacher GE, Barr JT. Applying decision analysis in therapeutic drug monitoring: using decision trees to interpret serum theophylline concentrations. Clin Pharm 1986;5:325 33. 14. Schumacher GE, Barr JT. Making serum drug levels more meaningful. Ther Drug Monit 1989;11:580 4. 15. Schumacher GE, Barr JT. Bayesian and threshold probabilities in therapeutic drug monitoring: when can serum drug concentrations alter clinical decisions? Am J Hosp Pharm 1994;51:321 7. 16. Weinstein MC, Fineberg HV, eds. Clinical decision analysis. Philadelphia: WB Saunders, 1980:105 8. 17. Ried LD, Horn JR, McKenna DA. Therapeutic drug monitoring reduces toxic drug reactions: a meta-analysis. Ther Drug Monit 1990;12:72 8. 18. Destache CJ. Use of therapeutic drug monitoring in pharmacoeconomics. Ther Drug Monit 1993;15:608 10.