Diagnostic strategy for excluding pulmonary embolism in primary care Lucassen, W.A.M.

Transcription

1 UvA-DARE (Digital Academic Repository) Diagnostic strategy for excluding pulmonary embolism in primary care Lucassen, W.A.M. Link to publication Citation for published version (APA): Lucassen, W. A. M. (2013). Diagnostic strategy for excluding pulmonary embolism in primary care General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. UvA-DARE is a service provided by the library of the University of Amsterdam ( Download date: 28 Sep 2018

2 CHAPTER 3 CLINICAL DECISION RULES FOR EXCLUDING PULMONARY EMBOLISM: A META-ANALYSIS WAM Lucassen, GJ Geersing, PMG Erkens, JB Reitsma, KGM Moons, HR Büller, HCPM van Weert Annals of Internal Medicine 2011 Oct 4; 155(7):

3 CHAPTER 3 ABSTRACT Background Clinical probability assessment is combined with D-dimer testing to exclude pulmonary embolism (PE). Purpose To compare the test characteristics of gestalt (a physician s unstructured estimate) and clinical decision rules for evaluating adults with suspected PE and assess the failure rate of gestalt and rules when used in combination with D- dimer testing. Data Sources Articles in MEDLINE and EMBASE in English, French, German, Italian, Spanish, or Dutch that were published between 1966 and June Study Selection Three reviewers, working in pairs, selected prospective studies in consecutive patients suspected of having PE. Studies had to estimate the probability of PE by using gestalt or a decision rule and verify the diagnosis by using an appropriate reference standard. Data Extraction Data on study characteristics, test performance, and prevalence were extracted. Reviewers constructed 2x2 tables and assessed the methodological quality of the studies. Data Synthesis 52 studies, comprising patients, were selected. Meta-analysis was performed on studies that used gestalt (15 studies; sensitivity, 0.85; specificity, 0.51), the Wells rule with a cutoff value less than 2 (19 studies; sensitivity, 0.84; specificity, 0.58) or 4 or less (11 studies; sensitivity, 0.60; specificity, 0.80), the Geneva rule (5 studies; sensitivity, 0.84; specificity, 0.50), and the revised Geneva rule (4 studies; sensitivity, 0.91; specificity, 0.37). An increased prevalence of PE was associated with higher sensitivity and lower specificity. Combining a decision rule or gestalt with D-dimer testing seemed safe for all strategies, except when the less-sensitive Wells rule (cutoff value 4) was combined with less-sensitive qualitative D-dimer testing. 40

4 Clinical decision rules for excluding PE Limitations Studies had substantial heterogeneity due to prevalence of PE and differences in threshold. Many studies (63%) had potential bias due to differential disease verification. Conclusion Clinical decision rules and gestalt can safely exclude PE when combined with sensitive D-dimer testing. The authors recommend standardized rules because gestalt has lower specificity, but the choice of a particular rule and D-dimer test depend on both prevalence and setting. Primary Funding Source Dutch Heart Foundation Introduction Pulmonary embolism (PE) has an estimated annual incidence of 2 to 3 cases per 1000 persons and a high fatality rate if left untreated. 1 A major problem in diagnosing PE is that the signs and symptoms are often nonspecific, and most patients with suspected PE do not actually have it. 2 Physicians in various fields frequently face the dilemma of not missing PE while avoiding unnecessar y diagnostic procedures that are expensive and possibly harmful, such as multidetector computed tomography. Diagnostic strategies in patients suspected of having PE initially focus on identifying patients in whom PE can be ruled out. 3 In these strategies, the first step is to assess clinical probability by using either empirical clinical assessment (gestalt) or a standardized clinical decision rule. The most widely used clinical decision rule is the Wells rule, which includes the physician s judgment of whether PE is more likely than an alternative diagnosis; however, this criterion is subjective. New rules, such as the Geneva, Pisa, Charlotte, and Pulmonar y Embolism Rule-out Criteria (PERC) rules, contain only objective items (AppendixTable ). 4 Additional D-dimer testing is performed only if clinical probability is low; otherwise, the patient is referred for additional pulmonar y vascular imaging without D-dimer testing. 5 Both quantitative and qualitative D-dimer testing is used. Quantitative tests are more sensitive but less specific 41

5 CHAPTER 3 than qualitative tests 6, but the latter can be used at the point of care. Because the relative merits of gestalt and different decision rules remain in dispute, we performed a systematic review and meta-analysis that compared the sensitivity and specificity of gestalt and different clinical decision rules, using a novel bivariate analysis approach. 7 We also assessed the failure rate and efficiency of gestalt and rules when used in combination with D-dimer testing, with a focus on their role as a triage tool for excluding PE without referral for imaging tests. Methods Data Sources and Searches We performed a systematic search to identify studies that used gestalt or a decision rule to assess the clinical probability of PE. We searched MEDLINE and EMBASE for articles published in English, French, German, Italian, Spanish, or Dutch between 1966 and June 2011, using the keywords pulmonar y embolism, epidemiologic research design, epidemiologic studies, predictive value of tests, probability, sensitivity, and specificity. Appendix Table 3.8 contains the full search strategy. Finally, we examined reference lists of selected articles. Study Selection To be eligible for inclusion, studies had to enroll consecutive patients aged 16 years or older who had symptoms or signs that suggested acute PE; be based on original, prospectively collected data; and use gestalt or a clinical decision rule to estimate the clinical probability of PE. Gestalt was defined as a physician s unstructured clinical probability estimate after collecting routine data from patient histor y, physical examination with or without basic laborator y tests, electrocardiography, or chest radiography. A clinical decision rule had to be based on a multivariate logistic regression model and provide a structured estimate of the probability of PE. Clinical probability had to be assessed while the researchers were blinded to the results of D-dimer testing or pulmonar y vascular imaging. Sufficient data had to be reported to allow construction of a 2x2 table from which we could extract the number of true-positive, false-positive, truenegative, and false-negative results. If a study derived a clinical decision rule, more than 50 patients with 42

6 Clinical decision rules for excluding PE confirmed PE had to be included to ensure a minimum level of accuracy for the derived rule. 8 A diagnosis of PE had to be confirmed by an appropriate (composite) reference standard, such as ventilationperfusion lung scanning, computed tomography, pulmonar y angiography, or autopsy; we also accepted a diagnosis of deep venous thrombosis as a surrogate for a diagnosis of PE. 9 If the diagnosis was refuted without pulmonar y imaging (for example, in patients with a negative D-dimer result), we required a clinical follow-up period of at least 45 days. Finally, D-dimer testing had to be performed in all patients with an assessment of low probability for a study to be included in our meta-analysis. Three reviewers, working in pairs, participated equally in identifying potentially eligible articles. Any disagreements were resolved by discussion among the 3 reviewers. Data Extraction and Quality Assessment Three reviewers, working in pairs, independently extracted the mean or median age of patients, proportion of outpatients, prevalence of PE (including follow-up period), and type of verification from each included study (Appendix Table 3.9). Authors were contacted to provide any data on these predefined characteristics that were missing in the original publication. To construct 2x2 tables, the probability assessment of PE had to be dichotomized. For studies that did not report dichotomized data, we took the lowest probability category for each rule (Wells score <2, Geneva score <5, revised Geneva score <4, simplified revised Geneva score 1, or Pisa score 10% [Appendix Table ]) versus the other probability categories. Among studies that used gestalt, the cutoff value for the lowest probability category ranged from less than 10% to less than 40% (Appendix Table 3.9). Finally, on the basis of the constructed 2x2 tables, the number of true-positive, false-positive, true-negative, and false-negative results were extracted from each study. The methodological quality of the selected studies was assessed independently by 3 reviewers, working in pairs, by using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies) instrument. 10 This instrument includes 14 items, all of which can be scored yes (no bias), no (bias present), or unclear. All discrepancies were resolved by discussion among the 3 reviewers. 43

7 CHAPTER 3 Data Synthesis and Analysis Data From Gestalt or Rules Alone On the basis of the number of true-positive, false-positive, true-negative, and false-negative results, we calculated sensitivity and specificity with 95% CIs (the Wilson method) and constructed forest plots. We used a bivariate model for diagnostic meta-analysis to obtain summary estimates of sensitivity and specificity. 7;11 The bivariate approach simultaneously models pairs of logit-transformed sensitivity and specificity from studies, thereby incorporating any correlation that might exist between sensitivity and specificity. It also uses a random-effects approach for both sensitivity and specificity, which allows for heterogeneity beyond chance due to clinical and methodological differences between studies. Data from the different rules were analyzed in a single model to obtain summary estimates of sensitivity and specificity for each rule, and we subsequently tested whether these summar y values significantly differed from each other. Our model was fitted to allow the between-study variation in sensitivity and specificity to differ across rules. A minimum of 4 studies was required for a rule to be included in our metaanalysis, to account for interstudy variability and to ensure a reliable assessment of summar y estimates of sensitivity and specificity by using the bivariate approach. We extended our basic bivariate model with study-level covariates to assess the effect of bias caused by specific design and patient population characteristics on sensitivity, specificity, or both. Three study characteristics were examined: type of study (derivation, in which a rule was developed, or management, in which the rule was actually used in clinical practice), type of verification method, and prevalence. Data From Rules and Gestalt Combined With D-Dimer Testing Combination testing is used as a diagnostic strategy to exclude PE. For patients with a low probability of PE (according to the rule) and a negative D-dimer result, the studies examined whether PE could be safely excluded and the patients discharged without anticoagulation. Because these studies were evaluating a diagnostic strategy rather than the characteristics of a specific test, they used failure rate as the outcome measure. 5 Failure rate was calculated as the proportion of patients with symptomatic and confirmed venous thromboem- 44

8 Clinical decision rules for excluding PE bolism during follow-up divided by the total number of patients with negative results on both the rule and D-dimer testing (missed cases). We combined these logit-transformed proportions by using a randomeffects model to estimate the overall failure rate for each combination of gestalt or decision rule with D-dimer testing. Covariates were added to the model to examine whether summary failure rates differed on the basis of the type of rule or D-dimer test used. We considered a strategy that used clinical probability assessment in combination with D-dimer testing to exclude PE safely if the failure rate was less than 2% and the maximum upper confidence limit was 2.7% (being the upper confidence limit of the 3-month rate of thromboembolism in patients in whom PE was suspected but who had normal findings on pulmonary angiography). 12 Another relevant outcome measure is the efficiency of the diagnostic strategy, expressed as the number of patients who receive a negative result on both the rule and D-dimer testing among all included patients (successful avoidance of imaging). This proportion was analyzed in the same manner as the failure rate. Analyses were performed by using the nonlinear mixed-models procedure (PROC NLMIXED) in SAS, version 9.1 (SAS Institute, Car y, North Carolina). Role of the Funding Source Our study was funded by the Dutch Heart Foundation. The funding source had no role in question formulation, searches and data collection, data interpretation, manuscript preparation, or approval of the manuscript for publication. Results Our search yielded citations. We retrieved 313 studies for detailed reading, of which 52 met our inclusion criteria (Figure 3.1). The 52 included studies were all published in English between 1990 and 2011 and comprised patients (median, 753 [range, 77 to 8138]). Pulmonar y embolism was diagnosed in 8987 patients (overall prevalence, 16% [range, 4% to 44%]). The mean age of included patients ranged from 45 to 72 years (Appendix Table 3.9). We identified 5 sets of clinical decision rules: the Wells rules (with a cutoff value <2 [Wells2] or 4 [Wells4] 13 and the simplified Wells 45

9 CHAPTER 3 rule 14 ), the Geneva rules (original 15, revised 16, and simplified revised 17 ), the Pisa rules (original 18 and revised 19 ), the Charlotte rule 20, and the PERC rule. 21 (Appendix Table ) The original Geneva rule and the Pisa rules use additional diagnostic testing (electrocardiography and chest radiography); the other rules use only clinical data. All studies used the appropriate cross-sectional study design and collected data prospectively. Differential disease verification, a potential source of bias, was present in 63% of the studies. For example, a patient with a positive result from a clinical decision rule received pulmonar y vascular imaging as a reference test, whereas a patient with a negative result from the rule and from D-dimer testing did not undergo the reference test but received only follow-up. Figure 3.1. Literature search and selection 46

10 Clinical decision rules for excluding PE Representative patient sample Selection criteria clearly described Adequate reference standard Cross-sectional design Complete verification of diagnosis No differential verification No incorporation bias Adequate index test description Adequate reference test description Blinding for reference test results Blinding for index test results Clinical dataavailable as in practice Uninterpretable test results reported Withdrawals explained Yes No Unclear Figure 3.2. Quality of included studies Scores (%) on Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool criteria that were fulfilled In addition, 37% of studies did not report uninterpretable test results. Figure 3.2 provides an overall impression of the methodological quality of studies. Results of Individual Studies Figures 3.3 and show the forest plots of sensitivity and specificity, respectively, in individual studies. Results varied considerably both among and within rules. The CIs for specificity were substantially smaller than those for sensitivity because more patients did not have PE. The low sensitivity in studies that used the Charlotte rule and the high sensitivity in those that used the Pisa rule were of interest. Outliers were partly explained by differences in cutoff values. For example, Perrier and colleagues 54, Righini and colleagues 55, Parent and colleagues 56, and Elias and colleagues 57 all used high cutoff values, which yielded fewer false-positive and more false-negative results (and thus higher specificities and lower sensitivities). The forest plot for specificity (Figure 3.4) shows that studies that used gestalt (with a low cutoff value) or the PERC rule had lower specificities than those that used other decision rules. 47

11 CHAPTER 3 0 Figure 3.3. Forest plot of sensitivity of gestalt and different decision rules 48

12 Clinical decision rules for excluding PE 0 Studies are ordered from low to high prevalence of pulmonary embolism. PERC=Pulmonary Embolism Rule-out Criteria; PIOPED=Prospective Investigation of Pulmonary Embolism Diagnosis. * Proportion of patients with pulmonary embolism (N) who had a positive test result (n). Validation study. Derivation study. Study excluded from meta-analysis because of high cutoff value. Study with revised Geneva dichotomized cutoff. Figure 3.3. Continued 49

13 CHAPTER 3 Study, Year (Reference) Patients, n/n* Wells rule, cutoff <2 Kline et al, /2194 Runyon et al, /2333 Kabrhel et al, /7395 Yap et al, /571 Wells et al, /844 Anderson et al, /776 Kabrhel et al, /546 Wolf et al, /118 Kline et al, /154 Wells et al, /211 Legnani et al, /298 Wells et al, /807 Penaloza et al, /274 Stein et al, /620 Siragusa et al, /257 Chagnon et al, /206 Sanson et al, /292 Calisir et al, /100 Miniati et al, /122 Wells rule, cutoff 4 Kabrhel et al, /546 Wolf et al, /118 Goekoop et al, /768 Steeghs et al, /284 Wells et al, /211 Kearon et al, /954 Wells et al, /798 Penaloza et al, /274 van Belle et al, /2632 Douma et al, /715 Douma et al, /615 Simplified Wells rule Douma et al, /715 Douma et al, /615 Gestalt Miniati et al, /436 Runyon et al, /1148 Kline et al, /2194 Runyon et al, /2333 Kabrhel et al, /7387 Kline et al, /7577 Kline and Hogg, /154 Cross et al, /64 Musset et al, /681 Ten Wolde et al, /504 Perrier et al, /749 PIOPED Investigators, /635 Sanson et al, /287 Miron et al, /73 Nilsson et al, /117 Kabrhel et al, /524 Parent et al, /216 Elias et al, /163 Barghouth et al, / Specificity, % Figure 3.4. Forest plot of specificity of gestalt and different decision rules 50

14 Clinical decision rules for excluding PE Study, Year (Reference) Patients, n/n* Geneva rule Perrier et al, /743 Chagnon et al, /206 Wicki et al, /719 Aujesky et al, /182 Miniati et al, /122 Perrier et al, /562 Revised Geneva rule Klok et al, /251 Le Gal et al, /735 Le Gal et al, /557 Calisir et al, /100 Righini et al, /1336 Douma et al, /706 Simplified revised Geneva rule Klok et al, /808 Douma et al, /615 Pisa rule Miniati et al, /660 Miniati et al, /230 Miniati et al, /122 Revised Pisa rule Miniati et al, /660 Miniati et al, /235 PERC rule Kline et al, /7577 Wolf et al, /188 Hugli et al, /1318 Charlotte rule Kline et al, /2194 Runyon et al, /2333 Kline et al, / Specificity, % Studies are ordered from low to high prevalence of pulmonary embolism. PERC=Pulmonary Embolism Rule-out Criteria; PIOPED=Prospective Investigation of Pulmonary Embolism Diagnosis. * Proportion of patients without pulmonary embolism (N) who had a negative test result (n). Validation study. Derivation study. Study excluded from meta-analysis because of high cutoff value. Study with revised Geneva dichotomized cutoff. Figure 3.4. Continued 51

15 CHAPTER 3 Sensitivity and Specificity of Gestalt and Clinical Decision Rules A pooled analysis was performed for studies that used gestalt (19 studies) or the Wells2 (19 studies), Wells4 (11 studies), Geneva (6 studies) or revised Geneva (6 studies) rules by using the bivariate model. The other clinical decision rules were used in fewer than 4 studies and could not be included in the meta-analysis. For the meta-analyses of studies that used gestalt and the Geneva and revised Geneva rules, we included 15, 5, and 4 studies, respectively. One study 58 used the revised Geneva rule with a dichotomized cutoff value, and the cutoff values were considered too high (probability >40%) in the other 6 studies 54-57;59;60 to be useful for excluding PE. Table 3.1 shows the pooled sensitivity and specificity of gestalt and the Wells2, Wells4, Geneva, and revised Geneva rules. For both sensitivity and specificity, differences between Wells4 and the other rules or gestalt were statistically significant (P<0.010). For specificity, the difference between Wells2 and the revised Geneva rule was also statistically significant (P=0.026). Other differences between the rules or gestalt were not significant. The summar y estimates of sensitivity indicate that a negative result from any clinical decision rule alone is insufficient for decision making. Even when the revised Geneva rule was used, 9% of patients with PE (1-sensitivity) would have false-negative results and not be treated. All rules and gestalt showed substantial between-study variability for sensitivity and specificity. Covariate Analysis The prevalence of PE in the included studies had a significant effect on sensitivity and specificity for gestalt and the Wells and Geneva rules; in all cases, an increased prevalence was associated with higher sensitivity and lower specificity. For example, a change in prevalence from 5% to 30% increased the sensitivity of gestalt from 0.63 to 0.90, of Wells2 from 0.67 to 0.91, of Wells4 from 0.34 to 0.72, of the Geneva rule from 0.53 to 0.85, and of the revised Geneva rule from 0.63 to 0.90 (Figures 3.5 and 3.6). 52

16 Clinical decision rules for excluding PE Table 3.1. Pooled Sensitivity and Specificity of Gestalt and Clinical Decision Rules Gestalt Studies Preva- Pooled Pooled Estimated Estimated or Rule n lence Sensitivity Specificity Sensitivity Specificity % (95% CI) (95% CI) at a Prevalence of at a Prevalence of 15% (95% CI)* 15% (95% CI)* Gestalt ( ) 0.51 ( ) 0.83 ( ) 0.52 ( ) Wells Cutoff value < ( ) 0.58 ( ) 0.85 ( ) 0.58 ( ) Cutoff value ( ) 0.80 ( ) 0.58 ( ) 0.81 ( ) Geneva ( ) 0.50 ( ) 0.76 ( ) 0.61 ( ) Revised Geneva ( ) 0.37 ( ) 0.82 ( ) 0.45 ( ) * Using a theoretical population with a 15% prevalence of pulmonary embolism. 53

17 CHAPTER 3 Because the overall prevalence of PE differed among studies of gestalt and the Wells and Geneva rules, we performed an additional analysis that adjusted for these differences in prevalence by adding it as a covariate to the model. Table 3.1 demonstrates the accuracy of the different rules in a virtual population with a 15% prevalence of PE (actual overall prevalence in included studies, 16%). Because the actual prevalence in studies of gestalt and the Wells rules is near 15%, their accuracy showed little change, whereas the revised Geneva rule showed considerably lower sensitivity and higher specificity. To study the influence of type of verification, we classified studies by whether all patients underwent pulmonar y vascular imaging or whether PE was excluded on the basis of a negative result on both a decision rule (or gestalt) and D-dimer testing (including clinical follow-up) or a negative D-dimer result (including clinical followup) alone (Appendix Table 9). Wells2=Wells with cutoff<2, Wells4=Wells with cutoff 4, Geneva rev.=revised Geneva. Figure 3.5. Relationship between prevalence of PE and sensitivity as derived from the bivariate model with type of rule and log transformed prevalence added as covariates 54

18 Clinical decision rules for excluding PE 100 Specificity (%) Wells 4 Geneva Wells 2 Gestalt Geneva rev Prevalence (%) Wells2= Wells with cutoff<2, Wells4=Wells with cutoff 4, Geneva rev.=revised Geneva. Figure 3.6. Relationship between prevalence of PE and specificity as derived from the bivariate model with rule and log transformed prevalence added as covariates The sensitivity and specificity of gestalt and the Wells and Geneva rules did not vary significantly by type of verification (data not shown). To examine the influence of type of study, we excluded studies that derived a new rule from the meta-analysis and reanalyzed those that remained. The sensitivity and specificity of gestalt and the Wells and Geneva rules did not change (data not shown). Adding D-Dimer Testing to the Result of Clinical Assessment Twenty studies reported data on combining the result of clinical assessment with D-dimer testing. Some studies reported on more than 1 rule or gestalt. One study 61 that did not perform D-dimer testing on all patients with a low probability assessment and 4 studies that used a high cutoff value were excluded from analysis. Figures 3.7 and 3.8 show the failure rate and efficiency of individual studies. Twelve studies combined a rule or gestalt with quantitative D-dimer 55

19 CHAPTER 3 Study, Year (Reference) Patients, n/n* Wells rule, cutoff <2 Wells et al, /73 Wells et al, /276 Wells et al, /437 Anderson et al, /369 Kline et al, /1241 Wells rule, cutoff 4 Steeghs et al, /170 van Belle et al, /1057 Goekoop et al, /450 Douma et al, /184 Wells et al, /118 Wells et al, /448 Kearon et al, /373 Simplified Wells rule Douma et al, /178 Gestalt Kline et al, /1239 Runyon et al, /599 Ten Wolde et al, /95 Elias et al, /60 Parent et al, /80 Geneva rule Aujesky et al, /46 Perrier et al, /238 Perrier et al, /232 Revised Geneva rule Klok et al, /98 Righini et al, /561 Douma et al, /185 Simplified revised Geneva rule Klok et al, /223 Douma et al, /190 Charlotte rule Kline et al, /1511 Quantitative D-dimer test Qualitative D-dimer test Failure Rate, % * Patients with symptomatic and confirmed venous thromboembolism during follow-up (n) divided by the total number of patients with negative results on both rule or gestalt and D- dimer testing (N). Validation study. Derivation study. Study excluded from meta-analysis because of high cutoff value. Study with revised Geneva dichotomized cutoff. Figure 3.7. Forest plot of the failure rate of gestalt and the different decision rules 56

20 Clinical decision rules for excluding PE Study, Year (Reference) Patients, n/n* Wells rule, cutoff <2 Wells et al, /247 Wells et al, /972 Wells et al, /930 Anderson et al, /858 Kline et al, /2302 Wells rule, cutoff 4 Steeghs et al, /331 van Belle et al, /3306 Goekoop et al, /876 Douma et al, /796 Wells et al, /247 Wells et al, /964 Kearon et al, /1126 Simplified Wells rule Douma et al, /803 Gestalt Kline et al, /2302 Runyon et al, /1193 Ten Wolde et al, /631 Elias et al, /274 Parent et al, /352 Geneva rule Aujesky et al, /259 Perrier et al, /965 Perrier et al, /756 Revised Geneva rule Klok et al, /300 Righini et al, /1693 Douma et al, /796 Simplified revised Geneva rule Klok et al, /1049 Douma et al, /795 Charlotte rule Kline et al, /2302 Quantitative D-dimer test Qualitative D-dimer test Efficiency, % * Patients with a negative result on both the rule or gestalt and D-dimer testing (n) divided by all included patients (N). Validation study. Derivation study. Study excluded from meta-analysis because of high cutoff value. Study with revised Geneva dichotomized cutoff. Figure 3.8. Forest plot of the efficiency of gestalt and different decision rules 57

21 CHAPTER 3 Table 3.2. Failure Rate and Efficiency of Gestalt and the Clinical Decision Rules When Combined With Either Quantitative or Qualitative D-Dimer Testing* Gestalt or Rule Studies, n Patients, n Prevalence of Failure Rate Efficiency Pulmonary (95% CI), % (95% CI), % Embolism, % All ( ) 35 (30-41) Quantitative D-dimer testing All ( ) 27 (22-34) Wells, cutoff value ( ) 39 (31-47) Geneva ( ) 21 (14-31) Simplified revised Geneva ( ) 23 (15-33) Qualitative D-dimer testing All ( ) 45 (39-52) Gestalt ( ) 52 (40-64) Wells Cutoff value ( ) 42 (32-52) Cutoff value < ( ) 40 (33-48) * Separate results shown only when 2 studies were available. 58

22 Clinical decision rules for excluding PE testing, and 11 studies combined a rule or gestalt with qualitative D- dimer testing (Table 3.2). The failure rate for all studies was 0.7% (CI, 0.5% to 1.0%), with an overall efficiency of 35%. Studies that used qualitative D-dimer testing had a higher failure rate than those that used quantitative D-dimer testing (1.0% [CI 0.8% to 1.3%] vs. 0.4% [CI 0.2% to 0.7%]; P<0.010) but also a higher efficiency (45% vs. 27%; P<0.010). Using qualitative D-dimer testing doubled the failure rate of Wells4 compared with Wells2 (1.7% vs. 0.9%). Combining Wells4 with quantitative D-dimer testing decreased the failure rate to 0.5%. The 2 gestalt studies that used qualitative D-dimer testing 62;63 had unexpectedly high efficiency, which was probably due to the low prevalence (4% and 5%) in both studies. Despite having a low cutoff value (<15%), the specificities of both studies were significantly higher than the summar y estimate. Discussion We performed a pooled bivariate analysis of the ability of gestalt and the Wells (with cutoff values <2 and 4), Geneva, and revised Geneva rules to exclude PE. Gestalt, Wells2, and the Geneva and revised Geneva rules had similarly high sensitivity for detecting PE. However, none was sensitive enough to exclude PE on its own. The specificities of gestalt and the revised Geneva rule were considerably lower, which yielded more false-positive results and unnecessar y computed tomography. Increased prevalence of PE was associated with higher sensitivity and lower specificity, yielding fewer false-negative and more false-positive results. A strategy of using both clinical probability assessment and quantitative D-dimer testing to exclude PE seemed safe for gestalt and the Wells and Geneva rules. When less-sensitive qualitative D-dimer testing was used, the 95% CI exceeded our prespecified threshold of a 2.7% failure rate for Wells4. The more sensitive gestalt and Wells2 were safe in combination with qualitative D-dimer testing. Our findings are consistent with those of a previous meta-analysis 64 showing that different clinical decision rules had similar accuracy in assessing clinical probability, as well as those of a previous review and meta-analysis 3;65 showing that PE can be safely excluded by a 59

23 CHAPTER 3 low clinical probability assessment and a negative D-dimer result. However, these studies did not report on gestalt, the clinical implications of the specificity of the rules used, or the influence of prevalence of PE. Our meta-analysis has limitations. First, we found substantial heterogeneity in sensitivity and specificity among studies of gestalt and the Wells and Geneva rules, which correlated with differences in prevalence of PE. Increased prevalence was associated with higher sensitivity and lower specificity. Differences in prevalence can indirectly reflect differences in the case mix of included patients. In studies with lower disease prevalence, more patients may be in an early stage of the condition, which would hamper detection and lead to more false-negative results (lower sensitivity). The prevalence of PE ranged from 5% to 44% in included studies. Studies performed in the United States and Canada had a considerably lower prevalence of PE than did those performed in Europe (8% and 13% vs.26%). The low threshold for testing for PE in North America due to medicolegal concerns results in lower prevalence 69, whereas preselection by primar y care physicians in Europe results in higher prevalence. The heterogeneity in sensitivity and specificity could also be due to differences in threshold. Threshold differences are clearly present in the studies of gestalt, because the definition of low probability ranged from less than 10% to less than 40%. These differences are highly subjective. In studies of the Wells rules, such threshold differences are also important because one of the items in the Wells rule, PE is as likely as or more likely than an alternative diagnosis, is a subjective measure. A positive score on this item counts as 3 points, which substantially influences test positivity (and thus both sensitivity and specificity). Second, the included studies used different reference standards to diagnose or exclude PE. Adding a covariate to our bivariate model for different types of reference methods did not influence sensitivity or specificity of gestalt or the Wells or Geneva rules. Including studies that used methods of excluding PE without imaging could lead to small overestimations of both the sensitivity and specificity of gestalt or a rule, because a small subsegmental embolus could be missed in a patient with a negative result and only uneventful follow-up

24 Clinical decision rules for excluding PE Third, we included studies that reported data on the use of both gestalt and rules in the same study population. In these studies, physicians might have been influenced by use of both gestalt and a rule. Because of the limited number of studies available, we could not perform a covariate analysis. Finally, our analysis is based on only published studies. Although publication bias may be a concern, we did not check for it. In a selected population with a high prevalence of PE, such as one referred by general practitioners, applying a decision rule with high sensitivity and low specificity (such as Wells2 or the Geneva rule) is less desirable. The high prevalence decreases the already moderate or low specificity, which leads to an absolute increase in false-positive results from the rule and the referral of too many patients for pulmonar y vascular imaging. A less sensitive and more specific rule, such as Wells4, is recommended in this setting. Because of the lower sensitivity of Wells4, combination with high-sensitivity quantitative D-dimer testing is advised. In an unselected population with low prevalence of PE, such as from an emergency department or primar y care, a highly sensitive rule (such as Wells2 or the Geneva rule) is recommended because the lower prevalence could further decrease sensitivity, yielding too many false-negative results. In this setting, PE can be safely excluded by using the more-sensitive Wells2 and a less-sensitive qualitative (point of care) D-dimer test. In conclusion, neither the clinical decision rules nor gestalt are sensitive enough to safely exclude PE on their own, but all may do so when combined with sensitive D-dimer testing. However, the sensitivity of a rule is not the only concern; a rule with lower specificity (and thus more patients with false-positive results, in whom no D-dimer test will be performed) would result in the referral of more patients for pulmonar y vascular imaging. Gestalt and the revised Geneva rule had lower specificity than the other rules. The physician who uses gestalt tends to assign a higher probability to PE to avoid missing it, thus causing more false-positive results and exposing more patients to unnecessar y pulmonar y imaging. We therefore recommend that physicians use a standardized decision rule instead of gestalt. Physicians should be aware that the sensitivity of a decision rule increases and its specificity decreases as the 61

25 CHAPTER 3 prevalence of PE increases. Because prevalence of PE is an indirect reflection of the case mix of their population, physicians should use the diagnostic strategy that fits their situation best. In high-prevalence situations (a referred population), a rule with higher specificity (such as Wells4) is desirable, whereas rules with higher sensitivity (such as Wells2 or the Geneva rule) are more desirable in a lower-prevalence situation. Because all of the studies we reviewed were performed in a hospital setting (emergency department, referred patients, or inpatients), a diagnostic strategy to exclude PE in primary care without imaging must be evaluated before implementation. Reference List 1 Anderson FA, Jr., Wheeler HB, Goldberg RJ, Hosmer DW, Patwardhan NA, Jovanovic B et al. A population-based perspective of the hospital incidence and case-fatality rates of deep vein thrombosis and pulmonary embolism. The Worcester DVT Study. Arch Intern Med 1991; 151(5): Moser KM. Venous thromboembolism. Am Rev Respir Dis 1990; 141(1): Kruip MJ, Leclercq MG, van der Heul C, Prins MH, Buller HR. Diagnostic strategies for excluding pulmonary embolism in clinical outcome studies. A systematic review. Ann Intern Med 2003; 138(12): Kline JA, Courtney DM, Kabrhel C, Moore CL, Smithline HA, Plewa MC et al. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. J Thromb Haemost 2008; 6(5): van Belle A, Buller HR, Huisman MV, Huisman PM, Kaasjager K, Kamphuisen PW et al. Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, D-dimer testing, and computed tomography. JAMA 2006; 295(2): Geersing GJ, Janssen KJ, Oudega R, Bax L, Hoes AW, Reitsma JB et al. Excluding venous thromboembolism using point of care D-dimer tests in outpatients: a diagnostic meta-analysis. BMJ 2009; 339:b Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005; 58(10): Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules. Applications and methodological standards. N Engl J Med 1985; 313(13): Anderson DR, Barnes D. The use of leg venous ultrasonography for the diagnosis of pulmonary embolism. Semin Nucl Med 2008; 38(6):

26 Clinical decision rules for excluding PE 10 Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003; 3: Hamza TH, van Houwelingen HC, Stijnen T. The binomial distribution of meta-analysis was preferred to model within-study variability. J Clin Epidemiol 2008; 61(1): van Beek EJ, Brouwerst EM, Song B, Stein PD, Oudkerk M. Clinical validity of a normal pulmonary angiogram in patients with suspected pulmonary embolism--a critical review. Clin Radiol 2001; 56(10): Wells PS, Anderson DR, Rodger M, Ginsberg JS, Kearon C, Gent M et al. Derivation of a simple clinical model to categorize patients probability of pulmonary embolism: increasing the models utility with the SimpliRED D- dimer. Thromb Haemost 2000; 83(3): Gibson NS, Sohne M, Kruip MJ, Tick LW, Gerdes VE, Bossuyt PM et al. Further validation and simplification of the Wells clinical decision rule in pulmonary embolism. Thromb Haemost 2008; 99(1): Wicki J, Perneger TV, Junod AF, Bounameaux H, Perrier A. Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score. Arch Intern Med 2001; 161(1): Le Gal GG, Righini M, Roy PM, Sanchez O, Aujesky D, Bounameaux H et al. Prediction of pulmonary embolism in the emergency department: the revised Geneva score. Ann Intern Med 2006; 144(3): Klok FA, Mos IC, Nijkeuter M, Righini M, Perrier A, Le GG et al. Simplification of the revised Geneva score for assessing clinical probability of pulmonary embolism. Arch Intern Med 2008; 168(19): Miniati M, Monti S, Bottai M. A structured clinical model for predicting the probability of pulmonary embolism. Am J Med 2003; 114(3): Miniati M, Bottai M, Monti S, Salvadori M, Serasini L, Passera M. Simple and accurate prediction of the clinical probability of pulmonary embolism. Am J Respir Crit Care Med 2008; 178(3): Kline JA, Nelson RD, Jackson RE, Courtney DM. Criteria for the safe use of D- dimer testing in emergency department patients with suspected pulmonary embolism: a multicenter US study. Ann Emerg Med 2002; 39(2): Kline JA, Mitchell AM, Kabrhel C, Richman PB, Courtney DM. Clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism. J Thromb Haemost 2004; 2(8): Miniati M, Pistolesi M, Marini C, Di RG, Formichi B, Prediletto R et al. Value of perfusion lung scan in the diagnosis of pulmonary embolism: results of the Prospective Investigative Study of Acute Pulmonary Embolism Diagnosis (PISA-PED). Am J Respir Crit Care Med 1996; 154(5):

27 CHAPTER 3 23 Kline JA, Hogg M. Measurement of expired carbon dioxide, oxygen and volume in conjunction with pretest probability estimation as a method to diagnose and exclude pulmonary venous thromboembolism. Clin Physiol Funct Imaging 2006; 26(4): Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the prospective investigation of pulmonary embolism diagnosis (PIOPED). The PIOPED Investigators. JAMA 1990; 263(20): Miron MJ, Perrier A, Bounameaux H, de MP, Slosman DO, Didier D et al. Contribution of noninvasive evaluation to the diagnosis of pulmonary embolism in hospitalized patients. Eur Respir J 1999; 13(6): Perrier A, Miron MJ, Desmarais S, de MP, Slosman D, Didier D et al. Using clinical evaluation and lung scan to rule out suspected pulmonary embolism: Is it a valid option in patients with normal results of lower-limb venous compression ultrasonography? Arch Intern Med 2000; 160(4): Sanson BJ, Lijmer JG, Mac Gillavry MR, Turkstra F, Prins MH, Buller HR. Comparison of a clinical probability estimate and two clinical models in patients with suspected pulmonary embolism. ANTELOPE-Study Group. Thromb Haemost 2000; 83(2): Ten Wolde WM, Hagen PJ, Macgillavry MR, Pollen IJ, Mairuhu AT, Koopman MM et al. Non-invasive diagnostic work-up of patients with clinically suspected pulmonary embolism; results of a management study. J Thromb Haemost 2004; 2(7): Nilsson T, Mare K, Carlsson A. Value of structured clinical and scintigraphic protocols in acute pulmonary embolism. J Intern Med 2001; 250(3): Cross JJ, Kemp PM, Walsh CG, Flower CD, Dixon AK. A randomized trial of spiral CT and ventilation perfusion scintigraphy for the diagnosis of pulmonary embolism. Clin Radiol 1998; 53(3): Musset D, Parent F, Meyer G, Maitre S, Girard P, Leroyer C et al. Diagnostic strategy for patients with suspected pulmonary embolism: a prospective multicentre outcome study. Lancet 2002; 360(9349): Wolf SJ, McCubbin TR, Feldhaus KM, Faragher JP, Adcock DM. Prospective validation of Wells Criteria in the evaluation of patients with suspected pulmonary embolism. Ann Emerg Med 2004; 44(5): Kabrhel C, McAfee AT, Goldhaber SZ. The contribution of the subjective component of the Canadian Pulmonary Embolism Score to the overall score in emergency department patients. Acad Emerg Med 2005; 12(10): Steeghs N, Goekoop RJ, Niessen RW, Jonkers GJ, Dik H, Huisman MV. C- reactive protein and D-dimer with clinical probability score in the exclusion of pulmonary embolism. Br J Haematol 2005; 130(4):

28 Clinical decision rules for excluding PE 35 van Belle BA, Buller HR, Huisman MV, Huisman PM, Kaasjager K, Kamphuisen PW et al. Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, D-dimer testing, and computed tomography. JAMA 2006; 295(2): Goekoop RJ, Steeghs N, Niessen RW, Jonkers GJ, Dik H, Castel A et al. Simple and safe exclusion of pulmonary embolism in outpatients using quantitative D-dimer and Wells' simplified decision rule. Thromb Haemost 2007; 97(1): Kearon C, Ginsberg JS, Douketis J, Turpie AG, Bates SM, Lee AY et al. An evaluation of D-dimer in the diagnosis of pulmonary embolism: a randomized trial. Ann Intern Med 2006; 144(11): Douma RA, Gibson NS, Gerdes VE, Buller HR, Wells PS, Perrier A et al. Validity and clinical utility of the simplified Wells rule for assessing clinical probability for the exclusion of pulmonary embolism. Thromb Haemost 2009; 101(1): Penaloza A, Melot C, Motte S. Comparison of the Wells score with the simplified revised Geneva score for assessing pretest probability of pulmonary embolism. Thromb Res 2011; 127(2): Wells PS, Anderson DR, Rodger M, Stiell I, Dreyer JF, Barnes D et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer. Ann Intern Med 2001; 135(2): Chagnon I, Bounameaux H, Aujesky D, Roy PM, Gourdier AL, Cornuz J et al. Comparison of two clinical prediction rules and implicit assessment among patients with suspected pulmonary embolism. Am J Med 2002; 113(4): Anderson DR, Kovacs MJ, Dennie C, Kovacs G, Stiell I, Dreyer J et al. Use of spiral computed tomography contrast angiography and ultrasonography to exclude the diagnosis of pulmonary embolism in the emergency department. J Emerg Med 2005; 29(4): Miniati M, Bottai M, Monti S. Comparison of 3 clinical models for predicting the probability of pulmonary embolism. Medicine (Baltimore) 2005; 84(2): Stein PD, Fowler SE, Goodman LR, Gottschalk A, Hales CA, Hull RD et al. Multidetector computed tomography for acute pulmonary embolism. N Engl J Med 2006; 354(22): Siragusa S, Malato A, Falaschi F, Porro F, Anastasio R, Giarratano A et al. Deferral of assessment of pulmonary embolism. Haematologica 2007; 92(3): Yap KS, Kalff V, Turlakow A, Kelly MJ. A prospective reassessment of the utility of the Wells score in identifying pulmonary embolism. Med J Aust 2007; 187(6):

29 CHAPTER 3 47 Calisir C, Yavas US, Ozkan IR, Alatas F, Cevik A, Ergun N et al. Performance of the Wells and Revised Geneva scores for predicting pulmonary embolism. Eur J Emerg Med 2009; 16(1): Legnani C, Cini M, Scarvelis D, Toulon P, Wu JR, Palareti G. Multicenter evaluation of a new quantitative highly sensitive D-dimer assay, the Hemosil D- dimer HS 500, in patients with clinically suspected venous thromboembolism. Thromb Res 2010; 125(5): Aujesky D, Hayoz D, Yersin B, Perrier A, Barghouth G, Schnyder P et al. Exclusion of pulmonary embolism using C-reactive protein and D-dimer. A prospective comparison. Thromb Haemost 2003; 90(6): Perrier A, Roy PM, Aujesky D, Chagnon I, Howarth N, Gourdier AL et al. Diagnosing pulmonary embolism in outpatients with clinical assessment, D- dimer measurement, venous ultrasound, and helical computed tomography: a multicenter management study. Am J Med 2004; 116(5): Klok FA, Kruisman E, Spaan J, Nijkeuter M, Righini M, Aujesky D et al. Comparison of the revised Geneva score with the Wells rule for assessing clinical probability of pulmonary embolism. J Thromb Haemost 2008; 6(1): Miniati M, Monti S, Bauleo C, Scoscia E, Tonelli L, Dainelli A et al. A diagnostic strategy for pulmonary embolism based on standardised pretest probability and perfusion lung scanning: a management study. Eur J Nucl Med Mol Imaging 2003; 30(11): Hugli O, Righini M, Le GG, Roy PM, Sanchez O, Verschuren F et al. The pulmonary embolism rule-out criteria (PERC) rule does not safely exclude pulmonary embolism. J Thromb Haemost 2011; 9(2): Perrier A, Roy PM, Sanchez O, Le GG, Meyer G, Gourdier AL et al. Multidetector-row computed tomography in suspected pulmonary embolism. N Engl J Med 2005; 352(17): Righini M, Le GG, Aujesky D, Roy PM, Sanchez O, Verschuren F et al. Diagnosis of pulmonary embolism by multidetector CT alone or combined with venous ultrasonography of the leg: a randomised non-inferiority trial. Lancet 2008; 371(9621): Parent F, Maitre S, Meyer G, Raherison C, Mal H, Lancar R et al. Diagnostic value of D-dimer in patients with suspected pulmonary embolism: results from a multicentre outcome study. Thromb Res 2007; 120(2): Elias A, Cazanave A, Elias M, Chabbert V, Juchet H, Paradis H et al. Diagnostic management of pulmonary embolism using clinical assessment, plasma D-dimer assay, complete lower limb venous ultrasound and helical computed tomography of pulmonary arteries. A multicentre clinical outcome study. Thromb Haemost 2005; 93(5): Douma RA, Mos IC, Erkens PM, Nizet TA, Durian MF, Hovens MM et al. Performance of 4 Clinical Decision Rules in the Diagnostic Management of Acute Pulmonary Embolism: A Prospective Cohort Study. Ann Intern Med 2011; 154(11):