The diagnosis and prognosis of venous thromboembolism : variations on a theme Gibson, N.S.

Transcription

1 UvA-DARE (Digital Academic Repository) The diagnosis and prognosis of venous thromboembolism : variations on a theme Gibson, N.S. Link to publication Citation for published version (APA): Gibson, N. S. (2008). The diagnosis and prognosis of venous thromboembolism : variations on a theme General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. UvA-DARE is a service provided by the library of the University of Amsterdam ( Download date: 21 Oct 2018

2 Furthervalidationandsimplification ofthewellsclinicaldecisionrule inpulmonaryembolism NADINES.GIBSON,MAAIKESÖHNE,MARIEKEJ.H.A.KRUIP,LIDWINEW.TICK, VICTORE.A.GERDES,PATRICKM.M.BOSSUYT,PHILIPS.WELLS,HARRYR.BÜLLER THROMBOSISANDHAEMOSTASIS2008.JAN;99(1):22934

3 ABSTRACT 38 Background TheWellsruleisawidelyappliedclinicaldecisionrule,inthediagnosticworkup ofpatientswithsuspectedpulmonaryembolism(pe).theobjectiveofthisstudy wastoreplicate,validateandpossiblysimplifythisrule. Methods Weuseddatacollectedin3306consecutivepatientswithclinicallysuspectedPEto recalculatetheoddsratiosforthevariablesintherule,tocalculatetheproportion ofpatientswithpeintheprobabilitycategories,theareaundertheroccurveand theincidenceofvenousthromboembolismduringfollowup.wecomparedthese measureswiththoseforamodifiedandasimplifiedversionofthedecisionrule. Results Inthereplication,theoddsratiosinthelogisticregressionmodelwerefoundtobe lower for each of the seven individual variables (p=0.02) but the proportion of patientswithpeintheprobabilitycategoriesinourstudygroupwerecomparable tothoseintheoriginalderivationandvalidationgroups.theareaundertheroc of the original, modified and simplified decision rule was similar: 0.74 (p=0.99; p=0.07). The venous thromboembolism incidence at 3 months in the group of patientswithawellsscore4andanormalddimerwas0.5%,versus0.3%witha modified rule and 0.5% with a simplified rule. The proportion of patients safely excludedforpewas32%,versus31%and30%,respectively. Conclusions ThisstudyfurthervalidatesthediagnosticutilityoftheWellsruleandindicates thatthescoringsystemcanbesimplifiedtoonepointforeachvariable.

4 ValidationandsimplificationoftheWellsrule INTRODUCTION Thediagnosticworkupofpatientswithclinicallysuspectedpulmonaryembolismis challengingbecauseoftherelativelylowprevalenceofthediseaseinthispopulation. In the past, several attempts have been made to include clinical information in the diagnosticprocessinordertoruleoutpulmonaryembolismandwithholdexpensive and timeconsuming imaging techniques without compromising patient s safety. However, the majority of these attempts have not been clinically successful 15. The mainconcernwiththeseassessmentsofclinicalprobabilityinvolvedtheuseofmany variables including subjective elements as well as the often complicated scoring methods. Furthermore clinical judgment by the doctor, also called gestalt, is the simplest method of selecting low risk patients. Yet when this method is used, it appears that only a low percentage of patients can be withheld from additional imagingtesting 68. The quantitative clinical decision rule, published by Wells and colleagues in 2000, incorporated seven items from the medical history and physical examination easily obtainedintheinitialdiagnosticworkup 9.Becauseofitsrelativecomprehensiveness andeaseofuseinaclinicalsettingthisruleisnowwidelyacceptedintheexclusionof pulmonaryembolism.ithasbeenincorporatedinseveralguidelines,despitecertain limitations The decision rule was obtained by selecting variables that were significantly associatedwiththepresenceorabsenceofpulmonaryembolismfromanextended40 item list. These variables were initially tested in a univariate logistic regression analysis.thosevariablesthatwerealsosignificantafterastepwiseregressionanalysis were selected for the final rule. According to the value of the odds ratios in the regressionanalysis1,1.5or3pointsareassignedforeachfeature(table1).therule can be used to assign patients to one of three probability categories (low, moderate andhigh)ortoclassifythemas pulmonaryembolismunlikely or likely. Thereisevidencethatoddsratios,calculatedthiswayforthedecisionrule,showa biasupwardandthatreplicationstudiesproducelowervaluesforthesamevariables. Thismechanismhasbeensuggestedasoneoftheexplanationsforthephenomenon that many decision rules tend to loose their discriminative power in subsequent evaluations 16,17. CHAPTER 3 39

5 Table 1. Scoring of the various variables in the original, the modified and simplified Wells rule. Original Modified Simplified 1. Clinical signs & symptoms DVT Tachycardia (>100/min) Immobilization or surgery in the previous four weeks Previous PE or DVT Hemoptysis Malignancy An alternative diagnosis is less likely than PE Cut-off for PE unlikely DVT: Deep Venous Thrombosis PE: Pulmonary Embolism 40 Ifthetrueoddsratiosintheclinicaldecisionrulearelowerthantheonesreportedby Wellsandcolleagues,theremaybelessneedtousethreedifferentsetsofpoints:1,1.5 or 3 points. It is possible that unit weights produce very similar, if not identical, results,astheoriginalrule.ifso,amuchsimplifiedrulecouldbeusedinpractice,one that is easier to memorize and leads to fewer summing mistakes in the acute care setting. Thethreepurposesofthisstudywereareplicationoftheweightsinthedecisionrule developed by Wells and colleagues, a validation of the rule, and, if possible, simplification.fortheseaimsweusedthedataofalargemanagementstudy 18. METHODS Datawereobtainedinalargeprospectivediagnosticmanagementstudythatincluded patientswithclinicallysuspectedpulmonaryembolismbetweennovember2002and August 2004 in 12 hospitals in the Netherlands. That study, described in detail elsewhere, demonstrated that a diagnostic management strategy with a clinical decision rule, a Ddimer test and spiral CT, is safe in the workup of patients with clinicallysuspectedpulmonaryembolism 18. Patients and management Consecutiveinandoutpatientswithclinicallysuspectedacutepulmonaryembolism

6 ValidationandsimplificationoftheWellsrule were eligible for this study. Patients were excluded if they had received (low molecularweight)heparinformorethan24hours,wereyoungerthan18yearsofage, were pregnant, had a known hypersensitivity for iodinated contrast fluid or renal failure, had a life expectancy of less than three months, if there was geographic inability for followup or if no informed consent was obtained. The institutional reviewboardsofallparticipatinghospitalsapproved thestudyprotocol. Eligiblepatientswereaskedforwrittenororalinformedconsent.Atpresentationthe clinicaldecisionruleofwellsandcolleagueswasusedbythetreatingphysician 9.The physician assigned three points for clinical signs and symptoms of deep venous thrombosis (DVT), three more points when pulmonary embolism was more likely thananalternativediagnosis,oneandahalfpointseachforaheartrategreaterthan 100,immobilizationorsurgeryinthepreviousfourweeks,andapreviousepisodeof DVT or pulmonary embolism, and one point each for hemoptysis and malignancy. The total score was obtained by summing these points. It takes values in the range from0to12.5. Withascore of4orlower,pulmonaryembolismwas consideredunlikelyanda D dimer test was performed (Tinaquant, Roche Diagnostica, Mannheim, Germany or VidasDdimer,Biomerieux,MarcyL Etoile,France) 18.TheDdimertestwasdefined asnormaliftheconcentrationwas0.5mg/l.thecombinationofascoreover4anda normal Ddimer result was considered to rule out pulmonary embolism and anticoagulanttreatmentwaswithheld. In all other patients a spiral CT scan was performed. The CT scan was considered positiveforpulmonaryembolismifcontrastmaterialoutlinedanintraluminalfilling defectorifavesselwastotallyoccludedbylowattenuationmaterialonatleasttwo adjacentslices.thedecisiononthepresenceorabsenceofpulmonaryembolismwas madebyatrainedattendingradiologist. Followupwasperformedinallpatientswithoutpulmonaryembolismatbaselineby the study physician, through a hospital visit, or a telephone interview at three months,andtheinstructiontocontactthestudycentreorthegeneralpractitionerin case of complaints suggestive of DVT or pulmonary embolism. In case of clinically suspected DVT or pulmonary embolism during the followup period, compression CHAPTER 3 41

7 ultrasound for suspected DVT and ventilationperfusion scintigraphy or CT for suspectedpulmonaryembolismwererequiredtoconfirmorrefutethediagnosis.in case of death, information was obtained from the general practitioner, from the hospitalrecordsorfromautopsy. Replication of the Wells clinical decision rule We fitted a multivariable logistic regression model to our data that contained the samevariablesasusedbywellsandcolleaguestodeveloptheclinicaldecisionrule. Theestimatedcoefficientsandthe95%confidenceintervalsweretransformedtoodds ratiosandcomparedtotheoddsratiosasreportedbywellsetal. 9.Weusedthesign testtotestthedirectionsofthedifferencesforsignificance. Validation of the Wells clinical decision rule The original study by Wells and colleagues assigned patients to one of three probability categories, based on their score: low (0 to 1 points), moderate (2 to 6 points) and high (more than 6 points). We calculated the prevalence of pulmonary embolism in these three probability categories in our study group and compared those with the prevalence in the corresponding categories in the derivation and validationgroupsofwellsetal Simplification of the Wells clinical decision rule To evaluate whether the original Wells rule could be simplified, we developed a modifiedandasimplifiedrule.insteadofassigningone,oneandahalf,orthreepoints foreachofthefeatures,themodifiedruleassignstwopointsforthepresenceofthe twovariableswiththehighestoddsratiosandonepointtotheremainingvariablesin themodel. Wealsodevelopedasimplifiedrule,onethatassignsunitweightsforthepresenceof all individual variables in the model. With this simplified rule, a patient receives a scorebetweenzeroandseven,dependingonthepresenceorabsenceofeachofthe sevenvariablesinthewellsdecisionrule. We calculated the scores for all patients in our study with the modified and the simplifiedrule.toevaluatetheperformanceofthemodifiedandthesimplifiedrules theareaunderthereceiveroperatingcharacteristic(roc)curvewascalculatedand comparedtotheareaunderthecurvefortheoriginalwellsrule.roccurvesshow

8 ValidationandsimplificationoftheWellsrule the discriminatory performance of a test; the area under the curve of a perfectly discriminating rule would be 1.00 and that of a useless rule We used the bivariate binormal method to estimate the respective ROC curves and test the differences of these correlated rules for significance, using the ROC kit computer program1.1b2 19. We also calculated and compared the proportion of patients in the likely and unlikely categoriesfortheoriginalwellsrule(usingacutoff4for unlikely ),the modifiedrule(usingacutoffof2)andthesimplifiedrule(cutoff1) 9. Safety and clinical utility of the clinical decision rules combined with D-dimer testing Since the clinical decision rule is never used as the only test to rule out pulmonary embolism, the diagnostic safety and utility of the combination of the decision rules andtheddimertestwerealsoevaluated.thesafetyofthisstrategywasdefinedin termsoftheobservedincidenceofsymptomaticvenousthromboembolismduringthe threemonthsoffollowupinpatientsinwhompulmonaryembolismwasconsidered unlikely,basedontheclinicaldecisionrulecutoffandanormalddimertestresult. The clinical utility was assessed by calculating the proportion of patients in whom furtherdiagnostictestingcouldbesafelywithheld.thesafetyaswellastheclinical utilityofthetwosimplifiedruleswascomparedtothoseoftheoriginaldecisionrule, usingthecutoffvaluesmentionedearlier 18. The 95 % confidence interval for the three months venous thromboembolism incidence rate for each possible score in combination with a normal Ddimer result werecalculated.astrategywasdefinedasacceptableiftheconfidenceintervalwas locatedbelow3.0%. CHAPTER 3 43 RESULTS Patient characteristics Ofthe3503screenedpatientswithclinicallysuspectedpulmonaryembolism,184had to be excluded because of predefined exclusion criteria. A total of 3306 consecutive patientswithclinicallysuspectedacutepulmonaryembolismcouldbeincludedinthe study.themeanageinthestudygroupwas53years(range18to110years)andthere were2701outpatients(82%).

9 Table 2. Odds ratios of the Wells rule items observed in the original and present study. Original study of Wells et al. 9 Present study (95% CI) 1. Clinical signs & symptoms DVT ( ) 2. Tachycardia (>100/min) ( ) 3. Immobilization or surgery in the previous four weeks ( ) 4. Previous PE or DVT ( ) 5. Hemoptysis ( ) 6. Malignancy ( ) 7. An alternative diagnosis is less likely than PE ( ) DVT: Deep Venous Thrombosis PE: Pulmonary Embolism 44 Thescoreontheclinicaldecisionrulewasavailablefor3298patients(99.8%).Ofthe variables in the decision rule the one most frequently observed was pulmonary embolism is more likely than an alternative diagnosis (n=2032, 62%). The two variables with the lowest frequency were clinical signs and symptoms of DVT (n=190, 6%) and hemoptysis (n=176, 5%). The prevalence of the other variables variedfrom11%(malignancy)to26%(tachycardia). Ddimerresultswereavailablein98%ofthe2199patientswithascoreof4orlower onthedecisionrule.ofthese2199patients1105hadaddimertestresult>0.5mg/l. Theprevalenceofprovenpulmonaryembolismatbaselineintheentirestudygroup was20%(n=674). Replication of the Wells clinical decision rule ThesecondcolumnofTable2showstheoddsratiosofthevariablesinthedecision rule, as obtained in the multivariable regression analysis of our data as well as the oddsratiosinthecomparableanalysisofthedataobtainedbywellsandcolleagues 9. Theoddsratiosforallsevenvariablesarelowerintheanalysisinourstudygroup comparedtothoseobtainedinthewellsstudy(p=0.02). Validation of the Wells clinical decision rule A comparison of the prevalence of pulmonary embolism in the three probability categoriesisshowninourstudygroup,andinthederivationandvalidationgroups ofwellsetal.(figure1).theprevalenceintherespectivecategoriesinthederivation group,thevalidationgroupandinourstudygroupisverysimilar.

10 ValidationandsimplificationoftheWellsrule Patients with pulmonary embolism (%) Low Moderate High Low Moderate High Low Moderate High 3.6% 20.5% 66.7% 2.0% 18.8% 50.0% 7.1% 25.5% 57.6% Derivation set (n=972) Validation set (n=247) Present study (n=3298) Figure 1. Prevalence of pulmonary embolism with 95% CI in the three probability groups with the original Wells rule in the study of Wells et al.9 (derivation and validation set), and in the present study. DifferencesbetweentheWellsvalidationgroupandourgroupwerenotstatistically significant(p=0.06,0.09and0.63forthelow,intermediateandhighprobabilitygroup, respectively). CHAPTER 3 45 Simplification of the Wells clinical decision rule Table 1 details the two new rules. In the modified rule two points are assigned to clinicalsignsandsymptomsofdvt andto pulmonaryembolismismorelikelythan analternativediagnosis.inthesimplifiedruleallvariablesareassignedonepointif present. In Figure 2 the ROC curves of the original scoring method of Wells et al. of the modifiedandthesimplifiedrulearedepicted.theareaunderthecurvewas0.744for theoriginalwellsrule(95%ci:0.724to0.764),0.744forthemodifiedrule(95%ci: to 0.764), and (95% CI: to 0.756) for the simplified rule. The differencesbetweentheoriginaldecisionruleandthemodifiedrule,andtheoriginal andthesimplifiedrule,werenotsignificant(p=0.99andp=0.07,respectively).

11 Figure 2. The area under the ROC of the original, modified and simplified decision rule was similar: 0.74 (p=0.99; p=0.07). 46 Safety and clinical utility of the different scoring options of the clinical decision rules combined with D-dimer testing Figure3showstheproportionofpatientswithpulmonaryembolisminthe unlikely and likely categoriesfortheoriginalruleandthemodified(cutoffscore2)andthe simplifiedrule(cutoffscore1).theproportionswereverysimilar.therewereno significantdifferencesbetweenthethreerules,neitherforthe likely orthe unlikely categories. Table 3. Safety and clinical utility; venous thromboembolic events during 3-months follow-up in untreated patients for the three scoring models of the clinical decision rule. Proportion of patients in n Total VTE n (%;95% CI) whom spiral CT can be withheld Original score 4 and normal (0.5%; 0.2 to 1.1%) 32% (95% CI 30-33%) D-dimer Modified score 2 and normal D-dimer Simplified score 1 and normal D-dimer (0.3%; 0.1 to 0.9%) 31% (95% CI 29-32%) (0.5%; 0.2 to 1.0%) 30% (95% CI 28-31%)

12 ValidationandsimplificationoftheWellsrule Patients with pulmonary embolism (%) Unlikely Likely Unlikely Likely Unlikely Likely 12.6% 38.5% 11.5% 37.3% 11.0% 35.8% Original (cut-off 4) Modified (cut-off 2) Simplified (cut-off 1) Figure 3. Prevalence of pulmonary embolism with 95% CI depicted for the original, the modified and the simplified Wells rule, using pulmonary embolism likely and unlikely categories. Wealsocalculatedthe3monthincidenceofVTEincaseofan unlikely resultonthe decision rule in combination with a normal Ddimer test result (Table 3). For comparisontheresultswiththeoriginaldichotomizedwellsscorecombinedwithd dimer, as used in our study, are also shown. Followup was incomplete in three patientsofthe2632patientswithoutpulmonaryembolismatbaseline.themodified rule and the simplified rule, in combination with a normal Ddimer, had similar incidenceratesofvteduringfollowup:0.3%(95%ci:0.1to0.9%)and0.5%(95%ci 0.2 to 1.0%), respectively. None of the differences in incidence rate was significant (p>0.70). Theclinicalutility,intermsoftheabilitytoreliablyexcludepatientsforpulmonary embolismwithoutfurtherimaging,wasaround30%forallthreedecisionruleswhen combinedwithanormalddimertest. Ifthecutoffscoreofthemodifiedrulewasincreasedto3,the3monthincidenceof VTEwouldincreaseto1.0%(95%CI0.5%to1.8%).Theadditionalpatientgroupwith a score of 3 points consisted of 109 patients of whom 5 patients had VTE (4.6%). Similarly,as6of135patientswith2pointsusingthesimplifiedrulehadpulmonary CHAPTER 3 47

13 embolism, the incidence of VTE in those patients would be 4.4% (95% CI: 1.7% to 9.4%). Acutoffscoreof2onthesimplifiedruleindicatesthatpulmonaryembolismcanbe considered likely if one or less of the seven features in the clinical decision rule is present.nofurthermultiplicationorsummationisrequired. DISCUSSION 48 Althoughtheliteratureindicatesthatshrinkageandregressiontothemeanareoften seen in the life span of prediction rules 16,17, the present analysis shows that the discriminativepowerofthewellsdecisionrulecomparesfavourablywiththeoriginal derivationandvalidationset. Ourfindingsindicatethatingeneraltheoddsratiosofthesevenvariableswerelower thanobservedbywellsetal.,howeverthisoccurredwithoutaffectingthevalidityof the rule. Moreover, the two most informative variables remained the same: alternative diagnosis less likely than pulmonary embolism and clinical signs & symptomsofdvt. In the original rule three different weights were assigned to the various variables basedontheiroddsratiosinordertoproduceauserfriendlydecisionrule(tables1 and2).whenwesimplifiedthisrulebygivingtwodifferentweightstothevariables two points for the two variables with the highest odds ratios and one point to the othervariableslittlediagnosticinformationwaslost.mostinterestingly,whenusing unit weights one point for each variable in what we call the simplified rule the diagnosticaccuracyremainedunchanged(figure2). ThesimplifiedWellsruleindicatesdiagnostictestingisrequirediftwoormoreofthe sevenvariablesarepresentoriftheddimertestispositive.ifonlyone,ornoneofthe variables,ispresentandtheddimertestisnormal,alessthanonepercentincidence rate of VTE during followup can be expected. With this combination of tests approximately 30% of patients with suspected pulmonary embolism can be safely withheldfromfurtherdiagnosticimaging,aproportionsimilartothatobservedwith theoriginalwellsrule.

14 ValidationandsimplificationoftheWellsrule Unitweightdecisionrulesarenotnew.Ithasbeenshownbeforethatalinearmodel containing equally weighted variables may be just as good at prediction in a validating sample as a model containing variables whose weights have been more preciselyestimatedinadifferentstudygroup 20.Havingunitweightsmakesdecision rulesalsomoreparsimonious.inthiscase,asthesevenvariablesareeitherpresentor absent in a patient,the physician should only check to see whether two or more of thesefeaturesareavailable. Our study results show that assigning different weights to the variables does not improvethediagnosticefficiencyoftheclinicaldecisionrule.thiscouldbecausedby theupwardbiasintheoddsratiosandaslightregressiontothemean.theoretically, the explanations for regression to the mean are diverse. It could be caused by the influenceofinterobservervariability,differencesininterraterreliabilityandvariation in referral pattern 13. Yet chance is also a decisive factor. A variable with an overestimated regression coefficient is more likely to be selected for the specific decisionrulethananunderestimatedone.consequentlytheselectedvariableswere likelytohavetoolargecoefficients 17. It is tempting to speculate why the Wells rule has been widely accepted while previousattemptswithsometimessimilarvariableshavefailed 15,21.Thiscouldpartly be due to the fact that the variables in these rules were too numerous and were complicated by the need for additional tests such as blood gas analysis, electrocardiographyorchestxray. The strategy of neural networks which are computerized clinical decision rules, popular in the early nineties, was also quickly forgotten, most likely because the networkswereperceivedtobetoocomplicated 22,23.Morerecentlytheapplicationof multivariable logistic regression techniques gave more insight into the predictive strengthandindependencyofthesignsandsymptomsforpulmonaryembolism,and thereforereliablediagnosticmodelscouldbecreatedwhichbecamemoreappealing toclinicians 3. ThewideacceptanceoftheWellsruleispossiblyduetotheinclusionofonlyseven relevant variables which are simple to obtain at the bedside 24. The addition of the clinical opinion of the clinician in the subjective variables clinical signs and symptomsof DVT and alternativediagnosislesslikelythanpulmonaryembolism probablyfurthercontributedtoitspopularity. CHAPTER 3 49

15 50 Several aspects of our study require comment. Our analysis was based on data collectedpreviouslyandthesimplifiedrulewouldbenefitfromvalidationinanother prospective study. Since the data were carefully collected in a large cohort of consecutive patients and the simplified rule builds on the work of Wells and colleagues, we are confident that this simplified rule will survive such further validation. The safety of excluding pulmonary embolism was determined by the subsequent incidenceofvteduringthethreemonthfollowup.usingfollowuphasincreasingly beenacceptedasanappropriatereferencestandardforclinicaloutcome,ifsomebasic methodological principles are adhered to, such as withholding anticoagulant treatment, complete followup and appropriate diagnostic workup in case of suspectedrecurrenceofvte 25. WeusedthecombinationofanunlikelyclinicalprobabilityandanormalDdimertest to rule out pulmonary embolism. It should be noted, that the addition of Ddimer testingismainlyresponsibleforthelowsubsequentincidenceofvteduringfollow up. Insummary,wevalidatedtheWellsruleandalthoughtheoddsratiosdiddiminish slightly, the performance of the rule was more than adequate. Simplification of the rule by assigning only one point to each of the seven variables had a similar diagnostic accuracy and clinical utility. This simplified rule requires prospective validation.

16 ValidationandsimplificationoftheWellsrule REFERENDCELIST 1. Celi A, Palla A, Petruzzelli S et al. Prospective study of a standardized questionnaire to improve clinical estimate of pulmonary embolism. Chest. 1989;95: Hoellerich VL, Wigton RS. Diagnosing pulmonary embolism using clinical findings. Arch Intern Med. 1986;146: Michel BC, Kuijer PM, McDonnell J, van Beek EJ, Rutten FF, Buller HR. The role of a decision rule in symptomatic pulmonary embolism patients with a non-high probability ventilation-perfusion scan. Thromb Haemost. 1997;78: Petruzzelli S, Palla A, Citi M, Giuntini C. Improvement of screening for pulmonary embolism with a standardized questionnaire. Respiration. 1990;57: Miniati M, Prediletto R, Formichi B et al. Accuracy of clinical assessment in the diagnosis of pulmonary embolism. Am J Respir Crit Care Med. 1999;159: ten Wolde M, Hagen PJ, Macgillavry MR et al. Non-invasive diagnostic work-up of patients with clinically suspected pulmonary embolism; results of a management study. J Thromb Haemost. 2004;2: Sanson BJ, Lijmer JG, Mac Gillavry MR, Turkstra F, Prins MH, Buller HR. Comparison of a clinical probability estimate and two clinical models in patients with suspected pulmonary embolism. ANTELOPE-Study Group. Thromb Haemost. 2000;83: Kabrhel C, Camargo CA, Jr., Goldhaber SZ. Clinical gestalt and the diagnosis of pulmonary embolism: does experience matter? Chest. 2005;127: Wells PS, Anderson DR, Rodger M et al. Derivation of a simple clinical model to categorize patients probability of pulmonary embolism: increasing the models utility with the SimpliRED D-dimer. Thromb Haemost. 2000;83: Chunilal SD, Eikelboom JW, Attia J et al. Does this patient have pulmonary embolism? JAMA. 2003;290: Righini M, Aujesky D, Roy PM et al. Clinical usefulness of D-dimer depending on clinical probability and cutoff value in outpatients with suspected pulmonary embolism. Arch Intern Med. 2004;164: Wells PS, Anderson DR, Rodger M et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer. Ann Intern Med. 2001;135: Wolf SJ, McCubbin TR, Feldhaus KM, Faragher JP, Adcock DM. Prospective validation of Wells Criteria in the evaluation of patients with suspected pulmonary embolism. Ann Emerg Med. 2004;44: Clinical policy: critical issues in the evaluation and management of adult patients presenting with suspected pulmonary embolism. Ann Emerg Med. 2003;41: British Thoracic Society guidelines for the management of suspected acute pulmonary embolism. Thorax. 2003;58: Charlson ME, Ales KL, Simon R, MacKenzie CR. Why predictive indexes perform less well in validation studies. Is it magic or methods? Arch Intern Med. 1987;147: Copas JB. Using regression models for prediction: shrinkage and regression to the mean. Stat Methods Med Res. 1997;6: Writing Group for the Christopher Study Investigators. Effectiveness of Managing Suspected Pulmonary Embolism Using an Algorithm Combining Clinical Probability, D-Dimer Testing, and Computed Tomography. JAMA. 2006;295: Metz CE, Herman BA, Roe CA. Statistical comparison of two ROC-curve estimates obtained CHAPTER 3 51

17 from partially-paired datasets. Med Decis Making. 1998;18: Dawes RM. The robust beauty of improper linear models in decision making. American Psychologist. 1979;34: Wicki J, Perneger TV, Junod AF, Bounameaux H, Perrier A. Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score. Arch Intern Med. 2001;161: Patil S, Henry JW, Rubenfire M, Stein PD. Neural network in the clinical diagnosis of acute pulmonary embolism. Chest. 1993;104: Wyatt J. Nervous about artificial neural networks? Lancet. 1995;346: Wyatt JC, Altman DG. Commentary: Prognostic models: clinically useful or quickly forgotten? BMJ. 1995;311: Kruip MJ, Slob MJ, Schijen JH, van der HC, Buller HR. Use of a clinical decision rule in combination with D-dimer concentration in diagnostic workup of patients with suspected pulmonary embolism: a prospective management study. Arch Intern Med. 2002;162:

18

19