A Primer on the Complicated Subject of Cost-Effectiveness Analyses for Pulmonary Embolism

Transcription

1 HEALTH POLICY AND CLINICAL PRACTICE/EDITORIAL A Primer on the Complicated Subject of Cost-Effectiveness Analyses for Pulmonary Embolism Jeffrey A. Kline, MD From the Department of Emergency Medicine, Carolinas Medical Center, Charlotte, NC /$-see front matter Copyright 2010 by the American College of Emergency Physicians. doi: /j.annemergmed SEE RELATED ARTICLE, P.321. [Ann Emerg Med. 2010;56: ] In this issue of Annals, Duriseti and Brandeau 1 perform a model-based cost-effectiveness analysis of pulmonary embolism. This editorial is intended to help explain how cost-effectiveness analyses work, and their strengths and limitations. Although cost-effectiveness analyses have been published in health care literature for more than 40 years, their importance and frequency have increased sharply in the past decade. Given that the new Patient Protection and Affordable Care Act has a specific section calling for increased research in emergency care, with attention to comparative effectiveness, it can be reasonably conjectured that cost-effectiveness analyses will become more numerous in emergency care literature. According to the results of an informal survey of colleagues, I conclude that few emergency physicians have a basic understanding of costeffectiveness analyses. The goal of the cost-effectiveness analysis is to quantify and compare the economic cost of health strategies. Cost is measured in dollars, and the strategies are often a sequence of diagnostic tests and treatments in a clinical protocol. Two broad types of cost-effectiveness analyses exist: measured and modeled. Measured cost-effectiveness analyses use data obtained from a prospective study to compare costs between 2 or more groups that had one or two experimental variables controlled by the researchers. A measured cost-effectiveness analysis uses real charges and costs obtained from administrative databases, as well as real outcomes, such as missed diagnoses, survival, and adverse events, to compute the costs of each strategy. The present cost-effectiveness analysis is modeled such that the researchers had to estimate the costs and diagnostic performances of tests, as well as the probability and magnitude of effect of many patient-oriented events, such as false-negative testing. THE TRAFFIC ANALOGY TO AID UNDERSTANDING The next time you get in your car to drive home from work, imagine having a device that could automatically switch all stoplights to green on your route. Fewer barriers should mean increased flow capacity, which logically should decrease your travel time. However, traffic time-pattern analyses, which are similar to cost-effectiveness analyses, simulate and summarize the entire traffic network over many iterations and often find the opposite effect: that over the course of your driving life, you will save time by hitting red lights. Using a unidimensional velocity analysis will reveal that hitting a red light adds time because your velocity is zero for a while. Suppose you have 4 stoplights between work and home. For any one trip home, if 4 stoplights are red, this adds, say, 5 minutes to your drive compared with hitting all green lights. However, a comprehensive traffic time-pattern analysis conducted during 20 simulated years will introduce a few wildcard unlikely but high-impact events in terms of your time. For example, having all green lights slightly increases the probability of a speeding ticket, which adds time as you argue with the police officer, only to lose the argument, and then take the time to pay the ticket or go to court, and then expend more time at work to earn the money required to pay for your increased automobile insurance. Hitting more green lights also slightly increases the probability of a high-speed automobile accident and the worst-case, very low-probability event that you might then be injured, be hospitalized, and have to spend the time required to recover from and pay for that event. The model would also value the very, very low probability of your death as the remainder of your life expectancy. When the model incorporates these wildcard events, although their probabilities are low, their effect on your time is high, and during a simulated 20-year span, a comprehensive model may favor your hitting more red lights than you would like on your way home. Last, assuming that a predictable sequence of green lights equates to more choices for all the drivers besides you on the road, this increase in choice can lead to increased selfish driving behavior, increasing risk and net transit time for all motorists. In network theory, this unexpected inefficiency owing to selfish human behavior describes a derivative of gaming theory known as the Braess paradox, which states that increasing capacity in a network controlled by humans can actually decrease net flow. 2 More about gaming later. Although an imperfect analogy, this traffic example helps explain how cost-effectiveness analyses work and forwards the message that quantitative computerized models of complex 334 Annals of Emergency Medicine Volume 56, NO. 4 : October 2010

2 Kline networks often elucidate surprising conclusions compared with human thought. Instead of using gestalt and common sense, the cost-effectiveness analysis uses values of variables and the probability of their occurrence in a complex multivariate equation to predict how much a particular management strategy will cost. As opposed to cost being expressed as a single unit such as the amount of time of your life span or the dollars per evaluation, cost-effectiveness analyses typically express cost by normalizing it according to how many units of health are gained (or lost) per dollar spent. The unit of measure for health is something called a quality-adjusted life-year, which represents the theoretical value of 1 year of perfect health. Although costeffectiveness analyses provide mechanical logic and reproducibility, as with the Braess paradox in driving networks, cost-effectiveness analyses cannot model human factors that might perturb the model, such as selfish behavior or impatience. Thus, cost-effectiveness analyses include 3 main categories of variables. The first is patient-related variables, which include baseline age and health, diagnostic tests, diagnoses, treatments, and outcomes. The wildcards, analogous to the traffic ticket or accident in my traffic analogy, include the joint probability of a false-positive pulmonary embolism diagnosis and death from hemorrhage from warfarin treatment for this false-positive diagnosis of pulmonary embolism. These patient variables are manipulated by the investigator and require some highly speculative assumptions about their probability of occurrence and their magnitude of effect. The second category is economic costs of the tests, treatments, and outcomes. The economic costs are put in by the researcher and generated by the computer program. The third category is quality-adjusted life-years gained or lost by each test, diagnosis, and treatment. In some cases, the second and third variables can be controlled by the investigator (eg, cost of a test or quality-adjusted life-year loss caused by missing a diagnosis), but these variables are also outputs of the model of primary interest, and to that extent, these are analogous to dependent variables. A robust cost-effectiveness analysis recognizes the increase in quality-adjusted life-years afforded by a correct diagnosis and, conversely, the loss in quality-adjusted life-years exacted by a false-positive diagnosis. To critically analyze a model cost-effectiveness analysis, readers should first look at the assumptions made for the patient-related variables. These assumptions determine what the model will generate. For example, in the traffic analogy, the effect of a yearly incremental increase in probability of an accident of 0.001% with all green lights might produce a model that favors all green lights, but a 0.1% increase might favor all red lights. Of course, these probabilities would actually depend on other factors, such as time of day and traffic density. The assumptions used by Duriseti and Brandeau 1 are stated in Table 1 of their article. The authors undertook a comprehensive analysis of the literature (including results of meta-analyses) as the basis for these values, as well as their sequencing. It can be reasonably conjectured that not all readers will agree with all of these values, nor will all readers agree with Cost-Effectiveness Analysis for Pulmonary Embolism the imaging strategies proposed in Figure 2 of their article. For example, not all patients evaluated for pulmonary embolism are aged 55 years, and not all computed tomography pulmonary angiogram scans have 93% sensitivity and 98% specificity. For the latter point, consider that a multicenter study recently found that 10% of 64-channel computed tomography pulmonary angiogram scans were of inadequate quality to read. 3 To help address this problem of variable variability, Duriseti et al performed a sensitivity analysis, which essentially repeatedly redid the model: the patient variables provided to the computer program are changed across the ranges shown in Table 1 of their article, and the quality-adjusted life-years gained or lost per dollar spent (quality-adjusted life-years/$) are recalculated. Moreover, the analysis can be repeatedly conducted across a matrix to examine different management strategies and diagnostic test sequencing, in which in one run the D-dimer is the first test, and in the next, the computed tomography pulmonary angiogram is the first test, applied to patients with different pretest probabilities of pulmonary embolism. Then each of these tests is reconducted to assume different patient ages. In consideration of a recently published cost-effectiveness analysis, a salient omission in Figure 1 of their article was the absence of a decision node that uses clinical criteria to exclude pulmonary embolism without any diagnostic test. 4 Obviously, this type of analysis leads to a complex maze of nested results, which can be bewildering and difficult to summarize succinctly for the reader. Figures 2 and 3 in the appendix of their article provide a dashboard plot of the ratio of quality-adjusted life-years gained (y axis) versus the cost of each strategy on the x axis. The diagnosis-oriented reader will observe the remarkable similarity of these plots to the receiver operating characteristic curve for diagnostic accuracy, except that the best test in the plot of quality-adjusted life-years is found at the upper left corner, as opposed to the upper right corner with the receiver operating characteristic curve plot. A thoughtful reader may ask this: Instead of using this complex model that requires potentially sketchy assumptions that may or may not apply to my next patient, why not do a measurement cost-effectiveness analysis? I could use a very large sample of emergency department (ED) patients and possibly an existing database such as the National ED Sample (available at extract and examine data for all patients evaluated for pulmonary embolism, add up their tests and their health care costs, assess their outcomes at discharge, and then figure out the cheapest strategy. The first problem with an observational economic study is that clinical truth can obfuscate cost and lost quality-adjusted life-years. For example, if a patient has a computed tomography pulmonary angiogram that is interpreted as positive for a subsegmental pulmonary embolism and clinical truth led to anticoagulation therapy for 6 months, but the hidden truth was that the patient had no pulmonary embolism, then the real-time study would consider 6 months of warfarin treatment as Volume 56, NO. 4 : October 2010 Annals of Emergency Medicine 335

3 Cost-Effectiveness Analysis for Pulmonary Embolism beneficial when it should have been considered unnecessary risk. In contrast, Table 1 of the article allows that 2% of patients without pulmonary embolism will be labeled as having pulmonary embolism (ie, computed tomography pulmonary angiogram specificity 98%), and for these 2% of incorrectly treated patients, the model imparts a coefficient that modestly lowers the quality-adjusted life-years. Additionally, data aggregated from even the largest of feasible sample sizes would allow little inference into the cheapest strategy for an individual patient because we would again be looking at the average or median cost for a certain evaluation strategy for a large number of patients, each of whom had different underlying problems and risks. On the other hand, with their sensitivity analysis, Duriseti and Brandeau 1 are able to retest the cost per qualityadjusted life-year gained or lost for patients aged 30 years and again for patients aged 75 years, and again and again, using the ranges specified for each of the variables in Table 1 of the article. To use a longitudinally collected database in an analogous way would require direct access to the database and a custom program that could extract only the data of patients who look alike (ie, share a similar clinical profile in terms of age, sex, and comorbidity) from the database and report only the costs and outcomes for particular diagnostic strategies. This lookup process would require a nearest neighbor, or attributematching methodology. 5 Even if such a precise search engine existed in the public or commercial domain, the evaluations for the matched patients would be at the mercy of the capricious nature of standard care. Thus, it could be anticipated that few patients with similar clinical profiles would also have matched evaluations in terms of sequence and testing. Hence, despite the vulnerable assumptions about patient-related variables, a costeffectiveness analysis conducted with transparent assumptions and a sensible sensitivity analysis is the best way to measure and compare the cost of health gained or lost from a large set of hypothetical management protocols. ANALYSIS OF THE ARTICLE AT HAND First, a word about reproducibility. Duriseti and Brandeau wrote their own computer program, which was used to inform a commercially available software package (Netica; Norsys Software, Vancouver, British Columbia, Canada) that can be used to construct and manipulate Bayesian networks. The only practical way to reproduce the authors work would be to borrow their program, which is not unusual for well-done, complicated cost-effectiveness analyses. Second, and most important, are the assumptions in Table 1 of their article. The central theorem of the cost-effectiveness analysis is that the quality of the input variables determines the quality of the output. I believe that the authors performed a commendable literature review to obtain their assumptions, and as a topic expert in this area, I agree with most of their assumptions, but I respectfully disagree with a few. The first is that the assumed pretest probability of pulmonary embolism equals 1.3% for patients with a low Wells score. In the appendix, the authors Kline present a complex justification for their assumed overall low prevalence of pulmonary embolism inasmuch as their target population was any patient with any complaint that could be caused by pulmonary embolism, as opposed to patients selected for evaluation for pulmonary embolism. Several large sample studies in ED populations have found this number closer to 3% to 4% among patients who actually were selected for testing for pulmonary embolism by a board-certified emergency physician. 6,7 However, I do not believe that this 2% difference in pretest probability for low-risk patients would change the conclusions. The second is the presumed prevalence of acute coronary syndrome of 15% in patients being evaluated for pulmonary embolism. (A well-described, large database of 7,940 patients evaluated for pulmonary embolism contained 48 [0.6%] patients with a discharge diagnosis of myocardial infarction and another 62 (0.8%) with a discharge diagnosis of angina, together accounting for only 1.4% of the pulmonary embolism population. 8 ) Again, I do not believe that difference would have affected the conclusions. However, 3 estimates about deep venous thrombosis and its diagnosis may have misinformed the model in a way that overestimated the value of the compression ultrasonography. In my opinion, the authors low-end estimate of a 25% prevalence of concomitant deep venous thrombosis when the patient has pulmonary embolism, when deep venous thrombosis was not clinically suspected, is too high. I would also submit that most clinicians perform a bilateral compression ultrasonography when searching for deep venous thrombosis as a surrogate test for pulmonary embolism, and it is unclear whether the direct cost estimate of $100 is enough for compression ultrasonography of both legs. Also, the highest considered indirect cost rate for compression ultrasonography was $200, equal to that of the D-dimer. In many hospitals, compression ultrasonography is not available at night, so the indirect cost would need to consider the opportunity costs of requiring a patient to occupy bed space while waiting for the ultrasonographic technologist or the cost and risk of an empiric dose of enoxaparin. The role of emergency physician performed compression ultrasonography in this scenario remains uncertain. 9,10 A critical appraisal of these assumptions is vital because for all recommendations in Tables 3 and 4 of the article, compression ultrasonography was chosen as the second test to follow a positive D-dimer result for patients without suspected deep venous thrombosis. Importantly, emergency physicians do not suspect deep venous thrombosis for 85-90% of patients chosen for whom they initiate a pulmonary embolism workup. 11 This recommendation of compression ultrasonography as the next step may be met with some skepticism. First, a large randomized trial found no improvement in the rate of detection of venous thromboembolism when compression ultrasonography was added to computerized tomographic angiography in patients evaluated for pulmonary embolism. 12 Second, I respectfully submit that many emergency practitioners in crowded EDs will be reluctant to adopt a protocol that mandates compression ultrasonography after a 336 Annals of Emergency Medicine Volume 56, NO. 4 : October 2010

4 Kline positive D-dimer result (even when using an 800 ng/ml cutoff, which I consider the right thing to do), knowing that the majority of these compression ultrasonography study results will be negative, which will then oblige the clinician to order a computed tomography pulmonary angiogram test while the waiting room fills. It could be argued that emergency physicians eschew sequential testing in favor of parallel testing whenever possible. I would not be surprised if physicians anticipated the protocol by simultaneously ordering a D-dimer test, a compression ultrasonography, and a computed tomography pulmonary angiogram scan, planning to cancel the latter 2 tests if the D-dimer result were low enough but then forgetting to check the D-dimer result before the compression ultrasonography and computed tomography pulmonary angiogram tests were conducted. Third, in conventional clinical practice, even when compression ultrasonography reveals a deep venous thrombosis in a patient with suspected pulmonary embolism, it would be rare for no one to order a follow-up computed tomography pulmonary angiogram. Fourth, a more speculative concern would be the more insidious effect of imperfect physician behavior, similar to the Braess paradox with drivers. The time and hassle of obtaining a bilateral compression ultrasonography might lead to a syndrome of protocol fatigue, whereby the clinical team loses interest in excluding pulmonary embolism in deference to making a disposition, especially near the change of shift. This might lead to undertesting. So, for these reasons, I am skeptical about the clinical sensibility of recommendation for the compression ultrasonography to follow the D-dimer test for all cases. I recognize that patients with a positive compression ultrasonography result may not need a follow-up computed tomography pulmonary angiogram, and in an age of increasing computed tomography pulmonary angiogram recidivism, emergency clinicians have an obligation to at least try to decrease patient exposure to medical radiation from computed tomography scans whenever possible. 13 And it remains important to recognize that cost-effectiveness analyses show that the right way to do things often contradicts our beliefs. Thus, in consideration of all risks and benefits, I believe that the authors recommendation of compression ultrasonography to follow the positive D-dimer result is near equipoise. One logical next step to the present article would be to perform a measurement cost-effectiveness analysis, using the database from the large randomized controlled trial by Righini et al. 12 The authors provided a best evidence method of how the D-dimer can be more optimally used, dividing the D-dimer concentrations into quintiles, with evidence-based interval sensitivity and specificity values. Consistent with previous reports conducted as secondary analyses of existing data sets, the present cost-effectiveness analysis recommended using a higher D-dimer cutoff (800 ng/ml) as the initial test for patients with low pretest probability. 7 Curiously, this current cost-effectiveness analysis has almost opposite conclusions from one published in 2006 by the same authors, which concluded that D-dimer as the first test was almost never the most cost-effective plan. 14 This difference Cost-Effectiveness Analysis for Pulmonary Embolism highlights the importance of the assumptions in Table 1 of the article and shows how small changes in key patient-related variables can completely change the conclusions of the costeffectiveness analysis. The authors point out that the present article considers patients with any symptom attributable to pulmonary embolism, leading to the very low assumed prevalence of pulmonary embolism at less than 2%. However, this is difficult to reconcile with the fact that both articles assumed use of the Well s score and both articles assumed the same prevalence of pulmonary embolism within each strata of the Wells score (40.6%, 16.2%, and 1.3% for high, moderate, and low, respectively). Thus, I remain uncertain about the assumption(s) that caused the D-dimer, when used as the first test, to maximize quality-adjusted life-years per dollar in the present article, whereas in the 2006 report, the computed tomography pulmonary angiogram as the first test tended to maximize the quality-adjusted life-years per dollar under most scenarios. 14 WHAT CAN WE TAKE TO WORK NOW? For the patient with suspected pulmonary embolism and no suspected deep venous thrombosis, I believe the take-home points of this cost-effectiveness analysis should be the following: First, categorize the patient as having a low, moderate, or high pretest probability for pulmonary embolism. I believe you can use the Well s criteria or your own gestalt for this process. Then, order a quantitative D-dimer test as the first test for all patients, regardless of your estimated pretest probability. A D-dimer result below the standard cutoff (500 ng/ml) rules out pulmonary embolism for patients with high pretest probability, a value less than 650 ng/ml rules out pulmonary embolism for patients with moderate pretest probability, and a value less than 800 ng/ml rules out pulmonary embolism for patients with low pretest probability. For patients with a positive D-dimer result, I hold a more dubious outlook on the longevity of a protocol that prescribes compression ultrasonography as the next step. However, this article justifies at least a trial run of a protocol that mandates ordering the compression ultrasonography as the next step, followed by an assiduous quality assurance audit to determine the frequency of computed tomography pulmonary angiogram scanning after compression ultrasonography. If most patients with a positive compression ultrasonography result undergo computed tomography pulmonary angiogram anyway, or even a small percentage of patients with a negative compression ultrasonography result do not receive a follow-up pulmonary vascular imaging study, then the protocol should be changed to follow the positive D-dimer result directly with the computed tomography pulmonary angiogram or ventilationperfusion lung scan. Supervising editor: Steven M. Green, MD Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this Volume 56, NO. 4 : October 2010 Annals of Emergency Medicine 337

5 Cost-Effectiveness Analysis for Pulmonary Embolism Kline article that might create any potential conflict of interest. See the Manuscript Submission Agreement in this issue for examples of specific conflicts covered by this statement. Dr. Kline owns stock in CP Diagnostics LLC. BreathQuant manufactures a breath device designed to diagnose and monitor for pulmonary embolism. Address for reprints: Jeffrey A. Kline, MD, Department of Emergency Medicine, Carolinas Medical Center, Charlotte, NC; Jkline@carolinas.org. REFERENCES 1. Duriseti R, Brandeau ML. Cost-effectiveness of strategies for diagnosing pulmonary embolism among emergency department patients presenting with undifferentiated symptoms. Ann Emerg Med. 2010;56: Braess D. On the paradox of traffic planning. Transportation Sci. 2005;39: Courtney DM, Miller CD, Smithline HA, et al. Prospective multicenter assessment of interobserver agreement for radiologist interpretation of multidetector CT angiography for pulmonary embolism. J Thromb Haemost. 2010;8: Lessler AL, Isserman JA, Agarwal R, et al. Testing low-risk patients for suspected pulmonary embolism: a decision analysis. Ann Emerg Med. 2010;55: Kline JA, Courtney DM, Than MP, et al. Accuracy of very low pretest probability estimates for pulmonary embolism using the method of attribute matching compared with the Wells score. Acad Emerg Med. 2010;17: Runyon MS, Webb WB, Jones AE, et al. Comparison of the unstructured clinician estimate of low clinical probability for pulmonary embolism to the Canadian score or the Charlotte rule. Acad Emerg Med. 2005;12: Kabrhel C, Courtney DM, Camargo CA Jr, et al. Potential impact of adjusting the threshold of the quantitative D-dimer based upon pretest probability of acute pulmonary embolism. Acad Emerg Med. 2009;16: Courtney DM, Kline JA, Kabrhel C, et al. Clinical features from the history and physical examination that predict the presence or absence of pulmonary embolism in symptomatic emergency department patients: results of a prospective, multicenter study. Ann Emerg Med. 2010;55: Burnside PR, Brown MD, Kline JA. Systematic review of emergency clinician-performed ultrasound for deep venous thrombosis. Acad Emerg Med. 2008;15: Kline JA, O Malley PM, Tayal VS, et al. Emergency clinicianperformed compression ultrasonography for deep venous thrombosis of the lower extremity. Ann Emerg Med. 2008;52: Kline JA, Courtney DM, Kabrhel C, et al. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. J Thromb Haemost. 2008;6: Righini M, et al. Diagnosis of pulmonary embolism by multidetector CT alone or combined with venous ultrasonography of the leg: a randomised non-inferiority trial. Lancet. 2008;371: Kline JA, Courtney DM, Beam DM, et al. Incidence and predictors of repeated computed tomographic pulmonary angiography in emergency department patients. Ann Emerg Med. 2009;54: Duriseti RS, Shachter RD, Brandeau ML. Value of quantitative D-dimer assays in identifying pulmonary embolism: implications from a sequential decision model. Acad Emerg Med. 2006;13: IMAGES IN EMERGENCY MEDICINE (continued from p. 333) DIAGNOSIS: Patellar tendon rupture. Patella fractures, patellar tendon ruptures, and quadriceps tendon ruptures are the 3 most common injuries to the extensor mechanism of the knee. 1 The patellar tendon connects the patella to the tibial tuberosity, and acute traumatic patellar tendon rupture occurs most frequently in patients younger than 40 years. The mechanism of injury is forceful quadriceps contraction when the foot is planted and the knee is partially flexed. Patellar tendon rupture can also occur after harvest of the middle third of the patellar tendon for autograft replacement of the anterior cruciate ligament. 2 Physical examination findings include effusion, high-riding patella (patella alta), palpable soft tissue defect, tenderness along the retinacula, and inability to extend at the knee joint. As many as 38% of patellar and quadriceps tendon ruptures are missed on initial presentation. Prompt operative repair of the damaged tendon provides definitive management of this injury and better outcome than delayed repair. 3 REFERENCES 1. Ramseier LE, Werner CML, Heinzelmann M. Quadriceps and patellar tendon rupture. Injury. 2006;37: Bonamo JJ, Krinick RM, Sporn AA. Rupture of the patellar ligament after use of its central third for anterior cruciate reconstruction. A report of two cases. J Bone Joint Surg Am. 1984;66: Siwek CW, Rao JP. Rupture of the extensor mechanism of the knee joint. J Bone Joint Surg Am. 1981;63: Annals of Emergency Medicine Volume 56, NO. 4 : October 2010