Evidence-based clinical biochemistry

Transcription

1 Personal View Ann Clin Biochem 1997; 34: 3-7 Evidence-based clinical biochemistry R A Moore From the Pain Research and Nuffield Department of Anaesthetics, University of Oxford. The Churchill, Oxford OX] 7LJ, UK 'Evidence-based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients'. 1 This definition of evidencebased medicine (EBM) by Professor David Sackett and his colleagues specifically includes pathology and diagnostic tests, because these are important parts of the decision-making process. The practice of EBM requires the integration of individual clinical expertise with the best available external clinical evidence from systematic research. The key here is the word systematic, a synthesis of all the information available, avoiding those which have known sources of bias. In clinical trials of effectiveness, bias is known to occur in trials which are either not randomized or which are inadequately randomized.? Lack of blinding is another source of bias. In both these cases biased trials overestimate the effects of treatment-and are more likely to give rise to positive effects of treatments. The problem is the sheer amount of published literature. Approximately new biomedical books are published annually. There are perhaps biomedical journals in the world, and they have grown steadily by 7% a year since the seventeenth century. Although 15% of medical interventions are supported by solid scientific evidence and only 1% of the articles in medical journals can be considered scientifically sound;' recent work in Oxford, UK found that about 80% of in-patient general medicine was evidence- based. 4 Clinical review articles should be as scientific as the articles they review, which is why systematic reviews of randomized controlled trials represent the gold standard in EBM. The best application of quality standards in this sort This article was prepared at the invitation of the Clinical Laboratory Investigations Standing Committee of the Association of Clinical Biochemists, but does not necessarily reflect its views. of research comes from the Cochrane Collaboration. THE COCHRANE COLLABORATION The Cochrane Collaboration was launched in It is an international network of individuals committed to preparing, maintaining and disseminating systematic reviews of research on the effects of health care.' The collaboration is guided by six principles: 1 Collaboration 2 Building on people's existing enthusiasm and interests 3 Minimizing duplication of effort 4 Avoidance of bias 5 Keeping up-to-date 6 Ensuring access There are now 11 Cochrane Centres around the world, and a large number of collaborative review groups working or planned. There is a Cochrane Methods Working Group on Screening and Diagnostic Tests (Convenor, Les Irwig, University of Sydney) which has now posted their recommended methods for systematic reviews of screening and diagnostic tests on the Internet ( The objectives of The Cochrane Collaboration include the maintenance of high quality standards and the avoidance of duplicated effort. It enables health care systems worldwide to share the benefits of the work of this organization published through the Cochrane Database of Systematic Reviews whilst sharing the effort and cost of the work. STANDARDS FOR DESCRmING DIAGNOSTIC TESTS Seven methodological standards have been set out for diagnostic tests" (Table 1). Read, Lachs and Feinstein examined reports of diagnostic tests published in the Lancest, British Medical Journal, New England Journal of Medicine and 3

2 4 Moore Seven standardsfor reports ofdiagnostic tests. Shortened version ofstandards ofreporting and frequency ofstandards met in reports from Ref6) TABLE I. Standard Conditions to be met % of reports meeting the standard Spectrum composition The sensitivity and specificity of a test depend on the characteristics of the population studied Pertinent subgroups For successful use of the test, separate indices of accuracy are needed for pertinent individual sub groups within the spectrum of tested patients Avoidance of workup bias This form of bias can occur when patients with positive or negative diagnostic test results are preferentially referred to receive verification of diagnosis by the gold standard procedure Avoidance of review bias This form of bias can be introduced if the diagnostic test or the gold standard is appraised without precautions to achieve objectivity in their sequential interpretation-like blinding in clinical trials of a treatment. It can be avoided if the test and gold standard are interpreted separately by persons unaware of the results of the other Precision of results for test accuracy The stability of sensitivity and specificity depends on how many patients have been evaluated. Like many other measures, the point estimate should have confidence intervals around it, which are easily calculated Presentation of indeterminate test results The frequency of indeterminate results and how they are used in calculations of test performance represent critically important information about the test's clinical effectiveness Test reproducibility Test may not always give the same result-for a whole variety of reasons of test variability or observer interpretation. The reasons for this, and its extent, should be investigated The report had to contain information 27 on three of these four criteria: age distribution, sex distribution, summary of presenting clinical symptoms and/or disease stage, and eligibility criteria for study subjects When results for indices of accuracy were 8 reported for any pertinent demographic or clinical sub group (for example symptomatic versus asymptomatic patients) In cohort studies, all subjects had to be 46 assigned to receive both the diagnostic test and the gold standard verification either by direct procedure or by clinical follow up. In case-control studies credit depended on whether the diagnostic test preceded or followed the gold standard procedure In either prospective cohort studies or case- 38 control studies, a statement was required regarding the independent evaluation of the two tests Confidence intervals, or standard errors must be quoted, regardless of magnitude A study had to report all of the appropriate 23 positive, negative or indeterminate results generated during the evaluation and whether indeterminate results had been included or excluded when indices of accuracy were calculated In tests requiring observer interpretation, at 23 least some of the tests should have been evaluated for a summary measure of observer variability. For tests wihtout observer interpretation, credit was given for a summary measure of instrument variability II Journal of the American Medical Association from 1978 through 1993 to see how many reports of diagnostic tests meet these standards. They found 112 articles, predominantly in radiological tests and immunoassays. Few met these standards, 46% avoided workup bias, 8% reported accuracy in subgroups and only half of the reports mentioned the sex or age of the patients being investigated. The more recent period studied ( ) showed some improvement with 24% meeting up to four standards, and 6% up to six. DIFFERENT ASPECTS OF EBCB When asked Professor Sackett requires three things from evidence-based clinical biochemistry Ann Clin Biochem 1997: 34

3 Evidence-based clinical biochemistry 5 (EBCB), a colleague with whom he could discuss the investigations of a patient's illness removal of the unhelpful concept of normality or abnormality evidence on the validity, importance, and clinical usefulness of biochemical tests offered. Likelihood ratio One promising approach to the assessment of diagnostic probability is the likelihood ratio (explored extensively by Panzer and colleagues"), The likelihood ratio (LR) can be calculated from the sensitivity and specificity of a test. It expresses the odds that a given finding would occur in a patient with, as opposed to without, the target disorder or condition. The likelihood ratio of a positive result is: LRpos= sensitivity/(i- specificity) With the LR above I, the probability of the disease or condition being present goes up; when it is below 1 the probability of it being present goes down, and when it is exactly I the probability is unchanged. LR can also be calculated for negative test results. To find the odds that a given finding would occur in a patient with, as opposed to without, the target disorder or condition, LRneg is: LRneg= (1- sensitivityj/specificity Likelihood ratios are used with a nomogram (Fig. I); the post-test probability of a patient having a disease or disorder is derived by joining the pre-test probability and the likelihood ratio. Their use is well described.s? The pre-test probability might simply be the prevalence of a disease or disorder. An example of prevalence being used as the pre-test probability for LR analysis'? is given in the tests of smoking status published in this journal in A good illustration of the use of likelihood ratios is in testing for hypothyroidism in general practice where the prevalence is low but simple clinical scoring can increase pre-test probabilities. When a general practitioner (GP) sends a sample to the laboratory for a thyroid-stimulating hormone (TSH) test from a patient with no ~ 5 70 ~ :c 60 :c.c 50 '" 10.c e '" 40 e c c. iii iii.'j! 'J! ~ 40 iii a FIGURE 1. A likelihood ratio nomogram showing a hypothetical example of testing for hypothyroidism in general practice. Pre-test probability comes from prevalence for screening (A). or from clinical scoring for a patient with several signs and symptoms of hypothyroidism (B). overt clinical indication of hypothyroidism the pre-test probability is less than 1% (prevalence of hypothyroidism in an ambulatory population). If a serum TSH of > 15mU/L had a sensitivity and specificity each of 98% 7 then the LRpos would be 56 and the post-test probability of the patient being hypothyroid is less than 30% (line A, Fig. 1). If a clinical scoring system showed that the patient had several signs and symptoms of hypothyroidism with a 30% pre-test probability the same TSH result would have a post-test probability of over 90% (line B, Fig. I). These post-test probabilities simplify decisions on whether to treat, refer, or wait. The prevalence, sensitivity and specificity data needed for these calculations can be obtained from a clinical audit of GP requests for hypothyroid tests. GPs using clinical scoring would demonstrate the prevalence of hypothyroidism at increasing levels of clinical suspicion, and intelligent use of information technology could easily generate useful ways of AIlIl Clin Biochem 1997: 34

4 6 Moore integrating the information. It might even simplify the way in which thyroid function tests are now used.f The advantages of using likelihood ratios are: I They are comprehensible and easier to handle than sensitivity and specificity 2 They can be used sequentially, so that the post-test probability from one diagnostic test becomes the pre-test probability for the next 3 They combine clinical judgement with laboratory science, and any other type of test 4 They can be calculated at different levels of test result 5 They demonstrate unequivocally the almost impossibly high standards needed for screening tests, but suggest ways in which screening could be made more effective 6 They highlight the way in which clinical audit can be used to generate local data relevant for local use Randomized trials of effectiveness of diagnostic strategies Making an accurate diagnosis is one important objective to which clinical biochemists can contribute. However, randomized trials of diagnostic strategies are rare so how can we help? There are no easy answers. If EBCB is to progress, we need to develop methods for determining the contribution diagnostic tests make to enhanced patient outcome. THE WAY FORWARD The profession of clinical biochemistry needs to grasp the nettle of EBCB and apply itself in several areas. (I) Systematic reviews of the evidence are needed to be undertaken in important areas of clinical biochemistry. This will not be easy because the field is large numerically (about 1000 publications a year on PSA) and in its breadth (from genetics and molecular biology through to clinical effectiveness). Nevertheless, important information on the usefulness of tests can easily be overlooked unless the work is systematic. (2) Deriving and understanding the rules and methods to be used. Cooperation with the Cochrane Collaboration groups working in this area. (3) Ensuring that modules on EBM, systematic review, critical appraisal of the literature included in educational programmes for those preparing for professional qualifications. (4) Organizing individuals and groups to pioneer examples of the application of EB methods to specific investigaitons. Concrete examples will serve to influence others. (5) Examining ways in which the clinical and cost-effectiveness information on laboratory tests can best be disseminated and implemented. Cost-effectiveness calculations should also benefit from the discipline of an EB approach if it is to be really worthwhile. (6) Explore the use of audit in development of local EB guidelines. Sensitivity, specificity and likelihood ratios can be applied to populations for which tests are being used in practice, not always as they were intended in the original research. So combining a definition of a clear clinical question with locally collected information and results should be an important way of influencing the use of diagnostic tests. SOME FINAL THOUGHTS It can be argued that there are three distinct phases in clinical biochemistry. Most clinical biochemists spend much of their time with the analysis stage-which is driven by issues of management, productivity, quality and cost. There is a pre-analytical stage, that of research into new or better tests which help in particular investigations. Relatively few biochemists have the opportunity to contribute to this area which is increasingly the domain of academic or industrial research groups. There is no reason, however, why all biochemists could not contribute to a post-analytical stage where the use of laboratory tests in investigations can be harnessed to bring increased clinical and cost effectiveness. Several studies have now demonstrated clear links between the intensity of diagnostic testing and subsequent therapy, both geographically'? and temporally'<e-test more, treat more. The accompanying editorial" makes the point that these studies 'are often better at raising important questions than answering them'-but we simply do not know whether by testing more and treating more we improve the quality and efficiency of the care we provide. In a cost-conscious world where operational analysis is often performed by young accountants still using text books, both the cost Ann Clin Biochem 1997: 34

5 Evidence-based clinical biochemistry 7 implications of intensive diagnostic testing and the high costs of employing doctors and clinical scientists in laboratories will attract increasing, and perhaps unwelcome, attention. A Dutch radiologist recently commented: 'Our technology needs not only to be effective but to be efficient and economically sound. We as radiologists had better get used to proving this, otherwise we will all be out of business'. It is a sentiment clinical biochemistry needs to consider. Acknowledgements Almost none of the thoughts in this article are original. I gratefully acknowledge the way in which my thinking has been moulded by Henry McQuay, Muir Gray, David Sackett, lain Chalmers, Alex Jadad and Martin Tramer, and the enthusiastic support of, and discussions with, Chris Price, Danielle Freedman, Adam Fleck, Jonathan Kay, Catherine Hawke and Jon Deeks. REFERENCES Sackett DL, Rosenberg WMC, Gray lam, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn't. BMJ 1996; 312: Schulz KF, Chalmers I, Hayes Rl, Altman DG. Empirical evidence of bias. JAMA 1995; 273: Smith R. Quotation from Professor D Eddy. BMJ 1991; 303: Ellis 1, Milligan I, Rowe 1, Sackett DL. In-patient general medicine is evidence-based. Lancet 1995; 346: Bero L, Rennie D. The Cochrane Collaboration. Preparing, maintaining, and disseminating systematic reviews ofthe effects ofhealth care. JAMA 1995; 274: Read MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research: getting better but still not good. lama 1995; 274: Panzer Rl, Black ER, Griner PF. Diagnostic Strategies for Common Medical Problems. American College ofphysicians, Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology, 2nd edn. Little, Brown & Co, Sackett DL. Evaluation ofclinical methods. In: D Weatherall, D Warrell, 1 Ledingham, eds. Oxford Textbook of Medicine, 3rd edn. 1996: How good is that test-using the result. Bandolier 28 lune 1996 II Arnold GPR, cupples M, McKnight A, Linton T. Measurement of markers of tobacco smoking in patients with coronary heart disease. Ann Clin Biochem 1995; 32: Barth lh, Seth 1, Howlett TA, Freedman DB. A survey of endocrine function testing by clinical biochemeistry laboratories in the UK. Ann Clin Biochem 1995; 32: Wennberg DE, Kellett MA, Dickens ld, Malenka Dl, Keilson LM, Keller RB. The association between local diagnostic testing intensity andinvasive cardiac procedures. JAMA 1996; 275: Verrilli D, Welch HG. The impact of diagnostic testing on therapeutic interventions. lama 1996; 275: Epstein AM. Use of diagnostic tests and therapeutic procedures in a changing health care environment. JAMA 1996; 275: Acceptedfor publication 2 July 1996 Ann C/in Biochem 1997: 34