Clinical decision theory: the threshold concept

Transcription

1 ELSEVIER Netherlands Journal of Medicine 47 (1995) The Netherlands JOURNAL OF MEDICINE Seminar 1 Clinical decision theory: the threshold concept J.D.F. Habbema Department of Public Health, Erasmus University Rotterdam, POB 173& 3000 DR Rotterdam, Netherlands Keywords: Decision analysis; Clinical decision theory 1. Introduction The rapidly increasing diagnostic and therapeutic possibilities, the high-quality demands by patients and public and the pressure for cost containment all contribute to the importance of trying to develop more explicit approaches to clinical reasoning and decision making. The rise of clinical decision theory, which is concerned with the quantitative study of decision problems in individual patient care, can partly be seen as a response to this challenge. A science of clinical decision making is possible because reasoning processes in diagnostics (what is wrong?), prognosis (what will happen?) and therapy choice (what can be done about it?) obey laws which are independent of the particular clinical domain of application. Clinical decision theory is truly a multi-disciplinary science. It uses methods and results from many fields including mathematical decision theory, probability theory, epidemiology, statistics, informatics, and cog- I Seminars in Clinical Epidemiology and Decision Analysis. Series Editor: D.E. Grobbee, MD, PhD, Department of Epidemiology, Erasmus University, and Consultation Centre for Clinical Research, Erasmus University and University Hospital, Rotterdam, Netherlands. nitive psychology. The core method is decision analysis, i.e. the comparative assessment of clinical strategies. Some basic concepts of decision analysis will be discussed and illustrated in this paper. An annotated bibliography is included for further study. 2. Diagnosis Diagnosis is not an aim in itself in clinical medicine; it is important because of its implications for prognosis and treatment. Contrary to the idea of many lay people, absolute diagnostic certainty can only seldom be reached in clinical practice. Moreover, certainty can sometimes only be obtained by performing risky (invasive) diagnostic tests. The notions of diagnostic probability and its revision because of further diagnostic evidence are central to diagnostic reasoning. Although diagnostics is an important subject in clinical decision sciences, it will not be discussed in the present paper. In our discussion of therapy choice we will simply assume that we are already at the end of the diagnostic process for a particular patient, and that the treatment decision remains (see, however, Section 5). We will illustrate the concepts with the case of a patient with possible deep venous thrombosis (DVT). We assume that we are at the end of the /95/$ Elsevier Science B.V. All rights reserved SSDI (95)

2 J.D.F Habbema / Netherlands Journal of Medicine 47 (1995) Table 1 Cross-tabulation of treatment choice and underlying disease, indicating the two types of right and wrong decisions that can be made. P(DVT) and P(noDVT) are the probabilities of DVT resp. no DVT at the moment that the treatment decision has to be made Choice for Choice for treatment No treatment Patient has Rightly treated Wrongly not treated DVT P(DVT) P(DVT) Patient has Wrongly treated Rightly not treated no DVT P(noDVT) P(noDVT) diagnostic process and that uncertainty remains as to whether a patient has DVT or not. We indicate this by the following symbolism: P(DVT): Probability that the patient has DVT, P(noDVT): Probability that the patient has no DVT. Whether the patient has the disease or not, these probabilities have to add up to 100%. The clinician has to choose for anti-coagulation treatment or not, but because of the diagnostic uncertainty, he will not know if he makes the right choice (see the schedule in Table 1). When the clinician chooses treatment, he will be choosing rightly when the disease is present, i.e. with probability P(DVT), but with a complementary probability P(noDVT) he will treat the patient wrongly. When the option "no treatment" is chosen, the two probabilities are reversed: with P(noDVT) the patient is rightly not treated, and with P(DVT) the patient is wrongly not treated. The general strategy that a clinician will follow is to choose for treatment when the probability of DVT, P(DVT), is high, and to choose not to treat when the probability of no DVT, P(noDVT), is high. One of the questions which can be addressed by decision analysis is: What is the treatment threshold ualue for the diagnostic probability P(DVT), above which the patient should be treated and below which he should not be treated. The value of this threshold will depend on the relative seriousness of the two types of wrong decisions. In other words: What adverse outcomes may result with what chances for the two types of wrong decisions? This leads us to a discussion of outcomes, before we go to calculate the treatment threshold. 3. Outcome Outcome plays a central role in clinical medicine because we aim to choose the treatment which makes the expected outcome for the patient as favourable as possible. Depending on the clinical situation at hand, outcome may include many aspects, such as inconvenience and risks of diagnostic tests, risks, side-effects and duration of the treatment process, and longevity and health status during the patient's further life. It is not known beforehand what long-term outcome the patient will experience and what risks and side-effects will be realized. Thus, at the time of treatment choice the outcome is uncertain and bears a probabilistic character. Most aspects of outcome can be classified under the two broad categories of length of life and quality of life. For our patient with possible DVT, four outcomes have to be assessed: those of non-dvt patients and DVT patients, under the therapeutic options yes or no anti-coagulation therapy. We will limit ourselves to what in this situation is the most important aspect of outcome, namely shortterm mortality. Based on evidence from the literature we have made the following short-term mortality estimates: DVT patient without therapy: 10% chance of a fatal embolism; DVT patient with therapy: 2% chance of a fatal bleeding or a fatal embolism; non-dvt patient with therapy: 1% chance of fatal bleeding; non-dvt patient without therapy: 0% mortality chance. These outcomes can be specified in the same cross-tabulation as in Table 1.

3 304 J.D.F. Habbema / Netherlands Journal of Medicine 47 (1995) Because we have limited ourselves to the short-term mortality aspect of outcome, there is no problem of incomparable types of outcome. When mortality is spread out over a long time period, the theory of survival curves should be used in comparing the mortality experience between treatment options. When health status is also affected by (non)treatment, the impact on quality of life should be assessed. There are several techniques for doing this, although they are not yet applied in routine practice. Survival in the short and the long run has to be combined with quality-of-life considerations. How these different aspects can be weighted in order to make an overall comparison between therapeutic options is beyond the scope of this paper. The most widely used concept in this respect is that of "quality-adjusted life year" (QALY) [see Refs. 1, 3]. 4. The treatment threshold Should we treat our patient with possible DVT or not? (Remember that we assumed that we are at the end of the diagnostic process.) Apparently, treatment is of benefit to the patient when he has DVT, and is of risk to the patient when he has no DVT. The magnitude of these benefits and risks can be calculated from Table 2: The benefit of treatment to the DVT patient amounts to a reduction in mortality risk of 8%, because the mortality risk with treatment is 2% instead of the 10% risk without treatment. The risk of anti-coagulation treatment to the nodvt patient is 1%. This is the fatal bleeding risk incurred by the treatment. Table 2 Consequences in terms of short-term mortality for the four combinations of treatment choice and underlying disease. See also Table 1 Choice for treatment Choice foi" No treatment Patient has 2% mortality 10% mortality DVT P(DVT) P(DVT) Patient has 1% mortality 0% mortality no DVT P(noDVT) P(noDVT) Mortality Mortality 8% risk benefit 0ojo,4,0 o'.'o,'ojo 7'o o 8', 9;,,00o, o ~.' ' p(dvt) Probability of Deep Venous Thrombosis Fig. 1. Graphical representation of the calculation of the treatment threshold in the case of a patient with possible DVT. The benefit of treating a DVT patient amounts to a 8% mortality reduction, and the risk of treating a non-dvt patient is 1%. Benefit and risk are exactly in balance for a DVT probability of 11.1% (this is the value where the benefit/risk ratio of 8:1 in favour of treatment is balanced by a ratio in diagnostic probabilities of 1:8 against DVT). For diagnostic values over 11.1%, anti-coagulation treatment is preferred, for values under 11.1% it is not. The balance of benefits and risks in an actual patient depends on the diagnostic probability P(DVT) that the patient has DVT. When we are absolutely certain that the patient indeed has DVT (i.e., a diagnostic probability of 100%), treatment is only of benefit and carries no risk. In the other extreme situation, a patient with 0% chance of DVT, we run only the full 1% risk and have no benefit. In all intermediate cases, benefits and risks have to be weighted with the probability of DVT or no DVT (see Fig. 1). The conclusion from Fig. 1 is that anti-coagulation treatment should be chosen when the probability of DVT exceeds 11%. But remember that this threshold value critically depends on the 8:1 benefit/risk ratio assumption. Sensitivity analysis is the name of the method that is used to assess this dependency. It can be used to explore how sensitive the conclusions are to the assumptions about benefit and risk. For example, when you don't know the exact benefit/risk ratio, but you are pretty sure that it will lie somewhere

4 J.D.F. Habberna /Netherlands Journal of Medicine 47 (1995) between 15:1 and 5:1, you are pretty sure by implication that the treatment threshold will lie somewhere between 6% and 16% (try to check these values). This gives you a more realistic first clinical guideline: In the case of low (< 6%) or "high" (> 16%) values for the probability of deep venous thrombosis, the results of the analysis point rather compellingly to no treatment or treatment; but for the range of diagnostic probabilities between 6% and 16%, the preferred course of action is not so clear-cut, and other considerations than short-term mortality which might influence the treatment choice will become more important. Most treatment decisions are more complex than the one we analyzed here (more diagnostic and therapeutic alternatives, more aspects of outcome, a long-term time-perspective, etc.), but the process of balancing the possible benefits and risks remains essentially the same. 5. Treatment, or further diagnostics? The testtreatment thresholds How much uncertainty at the end of the diagnostic process is realistic in patients with possible DVT? Ultrasonography can indeed lead to inconclusive results, which leave a wide range of possible diagnostic probabilities. But venography, the golden standard diagnostic method, should give (near)certainty about the presence or absence of DVT. Should we therefore always perform venography when diagnostic uncertainty remains after ultrasound? The answer would be affirmative-- apart from cost considerations--in case venography would have been risk-free. But this is not the case: it carries a non-negligible risk of fatal complications. Decision analysis can again be used to address the question under what conditions the benefit of venography--namely, a safer treatment choice afterwards--exceeds the risk of venography. If not, it would be better to make a direct treatment choice. Let us assume a fatal risk of venography of 0.2%. Graphically, we have to add a third line to the two lines in Fig. 1, representing the choice of venography. This has been done in Fig. 2. The Mortality risk Mortality l benefit 1% ~i&~-, 8% 0.2% 0.2% b I I I r i I I i 0% I 10% 20*/* 30% 40% 50% % I 90% 100% 2'~ / p(dvt) ~O /o Probability of Deep Venous Thrombosis Fig. 2. Graphical representation of the calculation of the two test-treatment thresholds in the case of a patient with possible DVT. Venography carries a 0.2% mortality risk, and is assumed to be a gold standard diagnosis. The no-treatment-test threshold lies at a 2.5% probability of DVT, and the testtreatment threshold at 80% probability. diagnostic probabilities on the horizontal axis should now be interpreted as representing the diagnostic judgement after ultrasound but before venography. After venography, we will always end up in the "risk-free" situation of 100% diagnostic certainty, so no other risk except the venography risk is incurred. The venography-risk line will therefore be a horizontal line of a constant 0.2% risk, irrespective of the diagnostic probabilities before venography. We will now end up with two thresholds: one at 2.5% probability of DVT and the other at 80% probability. The interpretation is as follows. When DVT is at least 80% certain, or the absence of DVT is at least 97.5% certain, a direct (no-)treatment choice should be made. If the diagnostic probability of DVT lies between 2.5% and 80%, the risk of venography is acceptable in view of the diagnostic information it produces. Sensitivity analysis can be used to assess the dependency of the threshold values on the three risk assumptions. Graphical illustration of decision problems by figures like those discussed so far is only possible

5 306 J.D.F. Habbema / Netherlands Journal of Medicine 47 (1995) PATIENT WITH POSSIBLE DVT TREAT (Anticoagulantia) P~I DVT) mortality: o OVT 4- l~q. p(no DVT) FATAL COMPLICATION } 0.2% positive ~ITREAT [ J p(ovt) ]VENOG RAPHY ( ~. ~ DO NOT TREAT 9&~ ] negative [] DO NOT TREAT i~ ~ p(no DVT) DVT P~i DVT) 0 DVT p(no DVT) Fig. 3. Decision tree for a patient with possible DVT, with the three options of treatment, no treatment and further diagnostics. Squares indicate choices, circles indicate uncertain outcomes or test results, and rectangles indicate outcomes. The quantification of the tree will be completed by specifying values for the probability of DVT, P(DVT), and the probability of no DVT, P(noDVT), It can subsequently be calculated what option gives the lowest mortality risk. (See Fig. 2 for an alternative representation of this choice situation.) For the relevant calculations for this tree, including the derivation of the treatment threshold and the two test-treatment thresholds, see Appendix 1. in rather simple situations. In the more general case, a decision tree can be used to illustrate the clinical problem, and to guide the subsequent calculations (see Fig. 3). 6. Discussion This completes our introduction to some basic concepts of clinical decision theory. We have chosen to embark directly on the study of a clinical problem, in order to demonstrate how decision theory can help with comparing clinical management options. Clinical decision theory is not an attempt to develop a theory of dehumanized automated decision making. On the contrary, the value of decision theory will depend on how well it can provide decision support in the context of the traditional patient-physician encounter. In a decision analysis it is impossible to take all (subtle) considerations into account which might influ- ence the subsequent choice of action. Clinician and patient, in their often difficult decision making process, have to judge the relevance and exhaustiveness of the results of the decision analysis for their particular problem. They have to adapt the results, sometimes radically, when important medical or other considerations have not been taken into account in the decision analysis. The experienced clinician-decision analyst will only selectively use decision theory as in our analysis of the DVT example. But he will continuously apply the principles of decision analysis informally, and will develop an increasingly sharp intuitive judgement on the conclusion he would have drawn from a more complete analysis. See the references for a further discussion of use and limitations of decision theory, and for further development of theory and applications. Appendix I: Calculation for the decision tree of Fig. 3, or equivalent, for Fig. 2 Probability of mortality (M) for the three options of Treatment (T), Venography (V) and no Treatment (not) P(MI T) = 2% X P(DVT) + 1% P(noDVT) P(MIV) = 0.2% % X 2% XP(DVT) P(MlnoT) = 10% XP(DVT) The threshold values for the probability of DVT are derived by equating the relevant mortality chances, and solving for P(DVT). Treatment - no treatment threshold P( M I T) = P( M InoT) This gives: 8% P(DVT) = 1% P(noDVT) Thus, P(DVT) = 1/9 = 11.1% No treatment - venography threshold P( M I not) = P( M [ V)

6 J.D.F. Habbema / Netherlands Journal of Medicine 47 (1995) This gives: 8% P( DVT) = 0.2% Thus, P( DVT) = 2.5% Venography - treatment threshold: P(MIV) This gives: =P(MIT) 0.2% = 1% P(noDVT) Thus, P( nodvt) = 20%, and P(DVT) = 80% References [1] Sox HC Jr, Blatt MA, Higgins, MC, Marton KI. Medical decision making. Stoneham: Butterworths, [Presently the best clinician oriented textbook on clinical decision making] [2] Kassirer JP, Moskowitz AJ, Lau J, Pauker SG. Decision analysis: a progress report. Ann Intern Med 1987;106: [A review of progress made until 1987, with a list of applications in different clinical specialities] [3] Weinstein MC, Fineberg HV, Elstein AS, et al. Clinical decision analysis. Philadelphia: WB Saunders, [A standard textbook. More mathematically oriented than the book of Sox et al.] [4] Kassirer JP, Kopelman RI. Learning clinical reasoning. Baltimore, MD: Williams & Wilkins, [A brief introduction to basic concepts, followed by a discussion of a large number of clinical cases] [5] Kahnemann K, Slovic P, Tversky A, eds. Judgement under uncertainty: heuristics and biases. Cambridge: Cambridge University Press, [A selection of interesting studies on problems that human beings have with making probability and value judgements] [6] Eddy DM. A manual for assessing health practices and designing practice policies: the explicit approach. Philadelphia: The American College of Physicians, [Discusses concepts, methods and processes concerned with the development of clinical guidelines] [7] Medical Decision Making, official journal of the Society for Medical Decision Making (quarterly), published by Hanley & Belfus, Inc. Philadelphia. [Publishes articles of theory and applications of medical decision sciences]