Evidence:WhatIsIt,WhereDoWeFindIt,

Transcription

1 European Urology Supplements European Urology Supplements 2 (2003) 3 9 Evidence:WhatIsIt,WhereDoWeFindIt, and How Do We Use It? $ Jay Y. Gillenwater *, Mikel Gray Department of Urology, University of Virginia, School of Medicine, P.O. Box , Charlottesville, VA 22908, USA Abstract The purpose of this paper is to review the history and rationale for evidence-based medicine (EBM). The development of EBM is briefly described, together with the pros and cons of evidence-based research, review techniques, and resources. The current status of EBM with regard to the treatment of overactive bladder (OAB) is also discussed. In short, EBM can be defined as the conscientious, explicit and judicious use of current best evidence to make decisions about the care of individual patients. The four main steps are: (1) formulate a clear question from a patient s problem, (2) search the literature for relevant clinical articles, (3) evaluate and critically appraise existing evidence for its validity and usefulness, and (4) implement useful findings in clinical practice. The power of the evidence-based approach can be enhanced by the development of techniques such as systematic review and metaanalysis. However, although EBM allows us to use current best evidence to make decisions about patient care, the evidence gained from systematic review and meta-analysis only applies to an average patient and is not readily adaptable to issues such as etiology, diagnosis and prognosis. # 2003 Published by Elsevier B.V. Keywords: Evidence-based medicine (EBM); Cochrane reviews; Clinical evidence; Overactive bladder (OAB) 1. Introduction The concept, use and definition of evidence has evolved significantly during recent years. For many of us, terms such as clinical evidence, evidence-based medicine (EBM), evidence-based diagnosis and evidence-based practice were not part of our education. Today, the concept of EBM has redefined the standard by which major medical journals evaluate manuscripts, and has changed the way all of us practice medicine. Sackett [1] defines EBM as the conscientious, explicit and judicious use of current, best evidence in making decisions about the care of individual patients... integrating individual clinical expertise with the best available external clinical evidence from systematic research. The four steps of EBM are: (1) formulate a clear question from a patient s problem, (2) $ Sources of funding: Based on a sponsored symposium presentation at the XVIIIth European Association of Urology (EAU) Meeting, March 2003, Madrid, Spain. * Corresponding author. Tel. þ ; Fax: þ address: jyg@virginia.edu (J.Y. Gillenwater). search the literature for relevant clinical articles, (3) evaluate and critically appraise existing evidence for its validity and usefulness, and (4) implement useful findings in clinical practice [2]. 2. The history of EBM EBM, in its earliest form, dates back to the great French Encyclopedia of the 18th century and the subsequent work of Pierre Louis in Paris in the early 19th century [3]. The power of the evidence-based approach has since been enhanced by the development of techniques of systematic review and meta-analysis. There are at least two important events in the evolution of EBM. The first is the work of a Welsh pulmonary epidemiology professor, Archibald Leman Cochrane ( ), who first started to promote the idea in the 1970s that there would always be limits to health care resources [4]. He suggested that the effectiveness of healthcare interventions should be judged on the basis of evidence from randomized clinical trials. In 1979, believing that systematic /$ see front matter # 2003 Published by Elsevier B.V. doi: /s (03)

2 4 J.Y. Gillenwater, M. Gray / European Urology Supplements 2 (2003) 3 9 up-to-date and easy-to-access reviews hold the key to well-informed health care choices, Cochrane wrote It is surely a great criticism of (the medical) profession that we have not arranged a critical summary, by specialty or subspecialty, adapted periodically of all randomized controlled trials [4]. Cochrane must have been an interesting and probably controversial person in his time. He was born 12 January 1909 in Galashiels, Scotland. As an undergraduate ( ), he studied at King s College, Cambridge, UK, continuing his research there to work on tissue culture studies. Between 1931 and 1934 he studied psychoanalysis with Theodor Reik, in Berlin, Vienna, and The Hague. In he was a medical student at University College Hospital, London, before serving in the International Brigade in the Spanish Civil War (1936), and in the Second World War ( ) as a captain in the Royal Army Medical Corps. In June 1941 he was taken prisoner by the German army in Crete and served as a physician in the prisoner of war camps, at times acting as the only physician for as many as 20,000 captives. He wrote about treating many epidemics including typhoid and diphtheria, and infections such as tuberculosis and jaundice. He taught the prisoners how to be malingerers, staging small epidemics of mumps, acute nephritis, headaches, strains and sprains. After the war ( ), Cochrane studied the epidemiology of tuberculosis at the Henry Phipps Institute in Philadelphia, USA. Between 1948 and 1960, he served as a member of the Medical Research Council of the Pneumococcus Research Unit in Penarth, Wales, and then as the David Davies Professor of Tuberculosis and Chest Diseases at the Welsh National School of Medicine in Cardiff from 1960 to It wasn t until 1967 that he first started to write about controlled trials, in 1971 reporting the results of a controlled clinical study comparing home versus hospital treatment. In 1972 he published a book Effectiveness and Efficiency Random Reflections on Health Services. He died on 18 June 1988 [5]. Cochrane s landmark criticism of the obstetrics and gynecology field for not applying evidence generated by research was published in 1972 as a monograph. This criticism led Dr. Iain Chalmers and coworkers to develop a system for summarizing randomized clinical trials relating to pregnancy and childbirth. This huge effort took 10 years to complete. Following Cochrane s death in 1988, the British National Health System adopted his principles of systematic review and meta-analysis. This led to the formation of the Cochrane Collaboration, an international initiative concerned with the preparation and dissemination of systematic reviews of health care research [5]. The second important influence on EBM was the change made in the medical school curriculum at McMaster University, Canada. During the 1960s, McMaster University developed teaching methods built around problem-based, self-directed learning. Clinical practice was integrated with research and the use of research principles to inform about diagnosis, treatment and prognosis. This novel educational approach was termed clinical epidemiology [1]. The term evidence-based medicine was coined at the School of Medicine at McMaster University in the 1980s to describe this clinical learning strategy, which the faculty had developed over more than a decade. They defined EBM as the process of systematically finding, appraising and using contemporaneous research findings as the basis for clinical decisions [2]. Dr. Murphy Enkin, Professor Emeritus of Obstetrics and Gynecology, spent a sabbatical in 1979 launching the Oxford database of perinatal clinical trials. Drs. Iain Chalmers and Brian Haynes (Director of the Canadian Cochrane Center) then published two books, Effective Care in Pregnancy and Childbirth (1989) and Effective Care of the Newborn (1992). Haynes developed a pilot system of reviewers and editorial teams that ultimately evolved into the Cochrane Collaboration. The British National Health System Research and Development Program was awarded funds and expanded their reviews into other practice areas. The Cochrane Collaboration was officially founded at a meeting of approximately fifty professionals from several countries in the autumn of By 1995, Cochrane Centers had been established in Australia, Canada, Denmark, Holland, Italy and the USA [5]. 3. Sources of medical information Many sources of evidence are available to the practicing physician. A growing number of online resources contain summaries of individual randomized clinical trials, and systematic reviews of meta-analyses (Table 1). The urology journals carry updated reviews covering most relevant topics, and all of the major journals are peer reviewed by knowledgeable clinicians and editors. However, practically speaking, urologists also gather medical information from less reliable resources including promotional materials from pharmaceutical manufacturers, newspapers, television, seminars and conferences, textbooks, professors and colleagues. Ideally, the hierarchy of resources that should be consulted when evaluating treatment effects is, firstly, systematic reviews or meta-analyses, then, individual randomized clinical trials with robust designs and

3 J.Y. Gillenwater, M. Gray / European Urology Supplements 2 (2003) Table 1 Online sources for EBM Resources Cochrane database of systematic reviews EBM American College of Physicians (ACP) journal club Database of Reviews of Effectiveness (DARE) Inforetriever Patient-Oriented Evidence that Matters (POEM) Description Includes reviews of a variety of areas in medicine Brief reviews of clinical trials in all areas of medicine; co-administered by the ACP and the British Medical Journal Group, producing summaries of randomized trials Critical assessment of reviews from various journals, including diagnosis and rehabilitation Collection of evidence-based resources and reference tools s regularly alerting physicians about randomized clinical trials and EBM resources definitive results, followed by randomized clinical trials with non-definitive findings, cohort or case control studies, and cross sectional studies or (least preferable) case studies (Fig. 1) [6]. It is particularly important for practicing clinicians to remain up to date with the published literature and research findings, as studies have shown a significant and inverse correlation between a physician s knowledge of contemporary care and the number of years since graduation from medical school [1]. As the number of years in practice lengthens, physicians tend to rely more on anecdotal evidence, expert opinion, pharmaceutical company promotional materials and clinical experience than on clinical research developments and scientific evaluation of new methods of diagnosis and treatment. 4. Reviewing, extracting and synthesizing clinical evidence Systematic review of existing research is the principal means of generating evidence-based resources. A systematic review seeks to answer a clinical question Quality of evidence Level 5 Level 4 Level 3 More than 1 RCT supports safety Level 1 & efficacy of intervention Majority of RCTs support safety & efficacy; others are equivocal / fail to support efficacy Quasi-experimental studies (non-randomized trials) support safety & efficacy Level 2 Case series or case studies suggest potential for safety & efficacy Consensus or expert opinion (best practice) Fig. 1. Quality of evidence hierarchy. that is typically related to the efficacy and safety of a particular treatment or intervention [7]. The review begins with formulation of a research question. Its focus should be narrow, and it should clearly define the patient group of interest, intervention under study, comparative interventions, and outcomes used to determine treatment effect. To avoid bias, the method for data extraction must be explicitly defined, including criteria for inclusion or exclusion of specific types of studies. The aim of the review is to identify all studies that meet the specified inclusion criteria, including both published and (whenever feasible) unpublished research. This task has been greatly aided by the advent of online databases, such as MEDLINE or CINAHL, but journals related to the topic of interest should be hand-searched, as should the reference lists from articles identified by database search. A systematic literature review should also interrogate the gray literature, including research abstracts presented at pertinent meetings and unpublished theses or dissertations. Personal communication with researchers identified though the initial review is recommended, since this technique may reveal ongoing research pertinent to the clinical question [8]. Searching more than the electronic databases is particularly important, since no single electronic database indexes all healthcare journals published throughout the world, and none is able to index all citations or abstracts on the date of their publication [9]. It is estimated that sole reliance on electronic databases identifies only about 50% of the pertinent research related to a specific question [8], but it is unknown what proportion of randomized clinical trials are extracted by this method. The literature review should also include foreign language research reports whenever possible. Electronic databases can be particularly helpful in this task, since many provide English language abstracts for major medical journals. Even with the aid of electronic databases and hand searches of recent publications or gray literature, it is

4 6 J.Y. Gillenwater, M. Gray / European Urology Supplements 2 (2003) 3 9 not possible to ensure that every relevant study has been identified. Probable unidentified sources include unpublished studies, ongoing or recently completed studies that have not been published, foreign language research that is not abstracted in English, and unpublished abstracts from smaller conferences. Unpublished clinical trials are an especially relevant source of possible bias, since it is known that research yielding positive results is more likely to be published than studies reporting negative findings [10]. Greenhalgh and Taylor [11] recommend statistical estimation of the probability of study extraction by calculating the number of unpublished reports that contain negative evidence necessary to refute evidence supporting the efficacy of a particular treatment. Alternatively, a scattergram might be used to quantify the potential for publication bias [12]. Experience suggests, however, that explicit review of methods, including the criteria for excluding any studies, is the most common method of accounting for the possibility of publication bias. Once all available studies have been identified, they should be reviewed for inclusion in the review. This review is based on four criteria: (1) subject characteristics, (2) intervention, (3) study outcomes, and (4) research design. Systematic reviews are typically limited to studies that employ all elements of a true experimental design including randomization of subjects, use of an active intervention, and comparison of the intervention to a control or comparison group. Following its selection, each study is evaluated for the integrity of its research design. For example, a randomized clinical trial is assessed for potential biases in terms of patient selection, performance, attrition, and detection [13]. Selection bias focuses on the methods used for randomization. A double blind study is considered ideal whenever feasible. Performance bias refers to differences in the treatments subjects receive, other than the intervention under study. The rate of attrition between groups is scrutinized, since a higher proportion of dropouts in a particular group may influence results and provide insights about the tolerability of an intervention. The evaluation of detection bias focuses on the outcomes measured during the research. Studies included in a systematic review must be identical in terms of the subjects in the treatment and control or comparison groups, and outcomes must be the same across studies if data are to be extracted for analysis and synthesis. Once pertinent studies have been identified, data are extracted and analyzed using qualitative or quantitative methods. The most common method for quantitatively analyzing data is meta-analysis [14]. Meta-analysis combines mean standard differences to compare study outcomes reporting interval or ratio data, or compares Study A Study B Study C Meta-analysis (a) Study A Study B Study C Meta-analysis (b) 95% Confidence Intervals Favors control Favors control Favors treatment 95% Confidence Intervals Favors treatment Fig. 2. Fictional meta-analysis example comparing control and active treatment data from studies A, B and C, in which (a) the control group is favored, and (b) no definitive conclusions can be drawn. odds ratios when nominal (dichotomous) data are available. A meta-analysis is often presented as a graph that compares results from individual studies. The graph is bisected by a vertical line representing no effect ;the column to the left of the line represents results favoring the control or comparison group while the column to the right of this line reflects results favoring the treatment group (Fig. 2a). A point estimate of differences between groups is represented by a circle or box, and 95% confidence intervals representing the power of the result are presented as a solid line extending to the left and right of this central point. When the line representing the confidence interval derived from the point estimate of differences between control and treatment groups crosses the no effect line, the sample size may have been too small to achieve significant differences, or the magnitude of differences between the groups may be too modest to be entirely confident that treatment is more effective than the results observed in the control and comparison groups (Fig. 2b). Although meta-analysis is a potentially powerful tool for analyzing pooled results from multiple studies, it must be remembered that this technique is limited by heterogeneity between trials [14,15]. From a statistical perspective, meta-analysis assumes that all clinical trials employ subjects with similar characteristics, similar interventions in both the treatment and control or comparison group, and the same outcome variables.

5 J.Y. Gillenwater, M. Gray / European Urology Supplements 2 (2003) However, even when these factors appear to be consistent across trials, analysis may reveal heterogeneity among results that must be explained. A statistician can employ a variant of the w 2 test to quantify the magnitude of heterogeneity, but this mathematical maneuver does not remove differences or unveil the correct value. Instead, it is the responsibility of the clinicians completing a systematic review to determine the source of heterogeneity, and to interpret these results within the context of clinical practice. Additional statistical tests can be completed after meta-analysis to further define the meaning of a systematic review [7,15]. A sensitivity analysis can be carried out to determine whether the results of the meta-analysis would change if data were analyzed using different exclusion criteria (e.g. including only doubleblinded studies, or studies with treatment periods of at least 12 weeks). Data used to generate a meta-analysis can also be used to determine the number needed to treat. This is an estimate of the number of patients who must undergo a specific intervention before a single positive result is expected. It should be balanced against the number needed to harm, a statistical estimate of the number of patients predicted to undergo treatment before experiencing a harmful side effect or outcome. Evaluation of these values enables physicians to better balance benefit and risk, particularly when applied to a disorder such as OAB that primarily affects quality of life and psychosocial well-being. Descriptive statistics are additional quantitative measures that may be used to summarize pooled sample size, subject characteristics, and inclusion/exclusion criteria. Alternatively, qualitative methods may be used to summarize results from a systematic review, particularly when pertinent randomized clinical trials demonstrate inadequate consistency in subject selection, randomization procedures, interventions or outcome measures to justify meta-analysis [6]. Qualitative analysis also may be useful when available research does not yield an adequate number of randomized clinical trials for meta-analysis, for example in unusual conditions or diseases whose characteristics render investigation by randomized clinical trial infeasible. In such cases, inclusion criteria, group selection, interventions and outcome measures are summarized in a narrative format, and evidence may be assessed in terms of its strength. Assessment of the strength or level of evidence is typically described using an ordinal scale with 3 5 levels. Expert opinion is typically described as the weakest or lowest level of evidence. The term best practice is often used to describe practices based primarily on tradition or expert opinion; the utility of these resources can be maximized by content validation. Case studies typically comprise the next level of evidence. Even though no numbers of anecdotal cases provide the evidence that can be gleaned from a single, well-designed randomized clinical trial, they frequently serve as the best available evidence for rare disorders or atypical responses to treatments or diseases, and they are a rich source of hypotheses. Clinical practices supported by quasi-experimental designs, such as cohort, case control or cross-sectional studies, usually comprise the next level. Although these studies suggest effectiveness, they lack randomization or fail to use the control or comparison group needed for a robust evaluation of treatment efficacy and safety. Clinical decisions supported by randomized clinical trials provide the best evidence. Any intervention supported by positive results from at least one randomized clinical trial tends to be ranked at the highest level in three-tier taxonomies. Taxonomies with five or more levels differentiate between interventions supported by the results of multiple trials versus those supported by one or two trials and judged as ineffective by others. 5. Problems with EBM A scholarly report on the problems in the evidence of EBM was written by Feinstein and Horowitz [16]. The authors draw attention to the fact that large series used to evaluate problems do not include many types of treatments, and are not readily adaptable to relevant issues such as etiology, diagnosis or prognosis. Data gleaned from systematic reviews only apply to an average patient and may not be suitable for the individual receiving care. In many cases an intent-totreat analysis does not reflect important post-randomization events leading to altered treatment. For example, a patient assigned to receive medical therapy in an intent-to-treat trial comparing medicine versus surgery may ultimately undergo surgery, but the outcomes of this patient are recorded in the medical group. Clinical judgment is needed to determine whether research findings apply to an individual patient. The clinician must judge if his or her patient is similar to those patients included in clinical trials of the systematic review, and if those findings are applicable to this patient s clinical situation. 6. Current evidence in OAB Application of the tenets of EBM to the specific therapeutic area of OAB is, to an extent, still in its infancy. There are large numbers of published studies

6 8 J.Y. Gillenwater, M. Gray / European Urology Supplements 2 (2003) 3 9 documenting the safety and efficacy of anticholinergic treatments in OAB (for an overview of these studies see [17 19]), but there are surprisingly few good systematic reviews. Comparisons can be made across the data arising from individual studies, but such evaluations can be unreliable and confounded by inconsistencies in study design (e.g. the use of nonstandardized outcome measures, fixed and/or inadequate drug dosage regimens, small sample sizes, insufficient durations of treatment, etc). Up until mid 2002, only two metaanalyses of an anticholinergic drug (tolterodine) in the management of OAB had been published [20,21], and neither of these could be considered an adequate, objective review of the available data [22]. Both publications failed to report the objectives of the systematic review, the search strategy, or the inclusion/ exclusion criteria for the trials, all of which would be considered prerequisites for a good meta-analysis. To address the need for a comprehensive and unbiased review of the efficacy and safety of anticholinergic agents in the treatment of OAB, the Cochrane Collaboration initiated four systematic analyses of the available data. The first, which compared anticholinergic drugs with placebo or no treatment, has been published under the auspices of the Cochrane Collaboration [22] and by the British Medical Journal [23]. Three Cochrane reviews are in progress. These will consider: (1) whether different anticholinergic drugs exert different effects, (2) whether anticholinergic drugs are better than other active (non-pharmacologic) therapies, and (3) whether some anticholinergic medications are better than others. The first Cochrane review [22] included a total of 6713 adult subjects across 51 trials (n ¼ 32 parallel designs, n ¼ 19 cross-over studies). The seven anticholinergic agents evaluated were darifenacin, emepronium, oxybutynin, propiverine, propantheline, tolterodine and trospium. The methodologies of the trials were considered by the authors of the Cochrane review to be of moderate to high quality, although there were some notable omissions. For example, not all studies mentioned baseline comparability of treatment groups, and the descriptions of treatment withdrawals were not always adequate. There was also considerable variation in the way outcomes were measured and reported. In particular, data from validated quality of life measures were rarely included. In terms of clinical efficacy, systematic review of the OAB study data showed that placebo had a significant effect (45%), but that pharmacotherapy provided significant additional benefit (15%). This benefit translated into one less void per 48 hours, one less episode of urine loss per 48 hours, an average increase in bladder capacity of 50 ml, and an increase in residual urine of 4 ml. The authors concluded that administration of anticholinergic drugs for the treatment of OAB resulted in statistically significant differences compared to placebo [22]. As all trials included in the Cochrane analysis were of relatively short-term duration, the initiation of randomized controlled studies to assess the long-term utility of anticholinergic agents was identified as a priority for future research. 7. Summary and conclusions EBM the use of current best evidence to make decisions about patient care is an essential component of clinical practice today. Being based upon the preponderance of good published evidence rather than the opinion of one or more individuals, it removes the potential bias of expert opinion, and allows for a more objective decision making process in the management of patients. It should not completely dictate practice, however, nor replace clinical reasoning or judgment. Evidence-based recommendations are, by necessity, based on data from groups of individuals, and care should be taken to apply them to the particular patient under consideration. References [1] Sackett DL, Straus S. Evidence-based medicine: how to practice and teach EBM. Edinburgh: Livingston; [2] Rosenberg W, Donald A. Evidence-based medicine: an approach to clinical problem solving. Br Med J 1995;310: [3] Sackett DL, Rosenberg W. Evidence-based medicine: what it is and what it isn t. Br Med J 1996;312:71 2. [4] Cochrane AL. A critical review, with particular reference to the medical profession. Medicine for the year London: Office of Health Economics; p [5] The Cochrane Collaboration. [6] Gray M, Bliss DZ, Bookout K, Colwell J, Dutcher JA, Engberg S, et al. Evidence-based nursing practice: a primer for the WOC nurse. J Wound Ostomy Continence Nurs 2002;29: [7] Magarey JM. Elements of a systematic review. Int J Nurs Pract 2001;7: [8] McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias RS, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. Br Med J 1998;317: [9] Rosenfield RM. How to systematically review the medical literature. Otolaryngol Head Neck Surg 1996;115: [10] Easterbrook PJ, Berlin JA, Goplan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337: [11] Greenhalgh T, Taylor R. How to read a paper: papers that go beyond numbers (qualitative research). Br Med J 1997;315:740 3.

7 J.Y. Gillenwater, M. Gray / European Urology Supplements 2 (2003) [12] Lau J, Ionnaidis JPA, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997;127: [13] Evans D. Conducting a systematic review. Adelaide (Australia): The Joanna Biggs Institute for Evidence Based Nursing and Midwifery; [14] Greenhalgh T. Papers that summarize other papers (systematic reviews and meta-analyses). Br Med J 1997;315: [15] McQuay HJ, Moore RA. Using numerical results from systematic reviews in clinical practice. Ann Intern Med 1997;126: [16] Feinstein AR, Horwitz RI. Problems in the evidence of evidencebased medicine. Am J Med 1997;103: [17] Chapple CR. Muscarinic receptor antagonists in the treatment of overactive bladder. Urology 2000;55(Suppl 5A): [18] Thüroff JW, Chartier-Kastler E, Corcus J, Humke J, Jonas U, Palmtag H, et al. Medical treatment and medical side effects in urinary incontinence in the elderly. World J Urol 1998;16(Suppl 1):S [19] Yoshimura N, Chancellor MB. Current and future pharmacological treatment for overactive bladder. J Urol 2002;168: [20] Appell RA. Clinical efficacy and safety of tolterodine in the treatment of overactive bladder: a pooled analysis. Urology 1997;50(Suppl 6A): [21] Larsson G, Hallen B, Nilvebrant L. Tolterodine in the treatment of overactive bladder: analysis of the pooled phase II efficacy and safety data. Urology 1999;53(5): [22] Hay-Smith J. Herbison P. Ellis G. Moore K. Anticholinergic drugs versus placebo for overactive bladder syndrome in adults (Cochrane Review). In: The Cochrane Library. Oxford: Update Software; Issue 3. [23] Herbison P, Hay-Smith J, Ellis G, Moore K. Effectiveness of anticholinergic drugs compared with placebo in the treatment of overactive bladder: systematic review. Br Med J 2003;326:8 11.