THE EFFICACY AND TOXICITY OF METHOTREXATE MONOTHERAPY VERSUS METHOTREXATE COMBINATION THERAPY WITH NON-BIOLOGIC DISEASE-MODIFYING ANTI-

Transcription

1 THE EFFICACY AND TOXICITY OF METHOTREXATE MONOTHERAPY VERSUS METHOTREXATE COMBINATION THERAPY WITH NON-BIOLOGIC DISEASE-MODIFYING ANTI- RHEUMATIC DRUGS IN RHEUMATOID ARTHRITIS A SYSTEMATIC REVIEW AND META-ANALYSIS by Wanruchada Katchamart A thesis submitted in conformity with the requirements for the degree of Masters of Science in Clinical Epidemiology Graduate Department of Institute of Medical Science University of Toronto Copyright by Wanruchada Katchamart 2009

2 The Efficacy and Toxicity of Methotrexate Monotherapy versus Methotrexate Combination Therapy with Non-biologic Disease-modifying Anti-rheumatic Drugs in Rheumatoid Arthritis A Systematic Review and Meta-analysis Wanruchada Katchamart Master of Science in Clinical epidemiology Institute of Medical Science, University of Toronto 2009 ABSTRACT Objective to systematically review randomized trials that compared methotrexate (MTX) monotherapy to MTX in combination with other non-biologic disease-modifying antirheumatic Drugs (DMARD) and to compare the performances of PubMed versus MEDLINE (Ovid ) and EMBASE. Methods We performed a systematic review of randomized trials comparing MTX alone and MTX in combination with other non-biologic DMARDs. Heterogeneity was investigated and explored. The performances of Pubmed and MEDLINE were evaluated. The EMBASE unique trials were identified and investigated. Results A total of 19 trials were included and grouped by the type of patients randomized. Trials in DMARD naive patients showed no significant advantage of the MTX combination versus monotherapy. The recall was 85% vs. 90% for Ovid and PubMed, respectively, while the precision and number-needed-to read of Ovid and Pubmed were comparable. Only 23% of trials were EMBASE unique trials Conclusions In DMARD naive patients, the balance of efficacy/toxicity favours MTX monotherapy. ii

3 ACKNOWLEDGMENTS I would like to express my deepest and sincerest gratitude to my supervisor, Dr. Claire Bombardier for successfully convincing and supporting me to study Clinical Epidemiology. Luckily, I believed her! I really appreciate her important support throughout this work. Her broad knowledge, logical way of thinking, and sense of humour have been of great value to me. Her understanding, encouragement, and personal guidance helped me to complete my degree. I would like to express my most sincere appreciation to my thesis committee members, Dr. George Tomlinson and Dr. Brian Feldman for their detailed and constructive suggestions and untiring help. I would also like to thank the thesis defense examiners, Dr. Paula Rochon and Dr. Robert Inman for their thoughtful advice on study results and interpretation. I would like to express my deep gratitude to Abbott International and Abbott Canada for allowing me to participate in the 3-e Initiative project. This project ignited and introduced me to the field of Evidence-based guideline. It has also broadened the horizons of my future career. I would like to warmly and deeply thank Dr. Vivian Bykerk for her advice, help, and support throughout my training in Canada and Julie Amato for reviewing this thesis and for her friendship. My training in Canada would not have been possible without the support of my colleagues in Rheumatology Division, Department of Medicine, Siriraj hospital at Mahidol University. I would also like to especially thank my friends and colleagues, Dr. Praveena Chewchanwisawakit for her help, support, and friendship and Dr. Vararak Srinonprasert for her knowledgeable advices and support. Finally, I would like to deeply thank my family for their understanding and unconditional love, constant and untiring support, and ongoing encouragement. iii

4 TABLE OF CONTENTS Chapter 1 Introduction and Background...1 Rheumatoid arthritis and Methotrexate...1 Systematic review and meta-analysis.2 Role and importance of systematic review in clinical practice..2 Problems and pitfalls of systematic review 4 3-e Initiative (Evidence, Expertise, Exchange) in Rheumatology..4 Chapter 2 The Efficacy and Toxicity of Methotrexate Monotherapy versus Methotrexate Combination Therapy with Non-biologic Disease-modifying Antirheumatic Drugs in Rheumatoid Arthritis: A Systematic Review and Meta-analysis 9 Introduction.9 Materials and methods..15 Results...18 Discussion and conclusions..24 Chapter 3 Investigating Heterogeneity in Systematic Review and Meta-analysis.29 Introduction...29 Methods 29 Results...33 Discussion and conclusions.36 Chapter 4 Search Strategy for Systematic Reviews: Comparisons between Ovid (MEDLINE) versus Pubmed and EMBASE versus Pubmed...41 Introduction...41 Methods 46 Results...52 Discussion and conclusions.53 Chapter 5 Discussion...58 Tables...62 Figures.100 References 126 Appendices iv

5 List of Tables Table 1 Excluded studies and reason for exclusion...62 Table 2A Characteristics of included studies...63 Table 2B Detailed characteristics of the included studies.64 Table 3 Quality assessment of included studies...75 Table 4 Summary the results of efficacy (dichotomous data) comparing between MTX Combination versus Monotherapy...76 Table 5 Summary the results of efficacy (continuous data) comparing between MTX Combination versus Monotherapy...77 Table 6 Summary the results of toxicity (dichotomous data) comparing between MTX Combination versus Monotherapy...78 Table 7 Real and artifactual causes of between-study variation in effect...79 Table 8 Potential sources of heterogeneity...80 Table 9 Tests of heterogeneity using forest plots and eyeball test, Cochran s Q test, and I 2 test for withdrawal due to lack of efficacy and toxicity, withdrawal due to lack of efficacy, and withdrawal due to toxicity...81 Table 10 Outcome measures for efficacy used in included studies...82 Table 11 Toxicity reported in included studies...83 Table 12 Subgroup analysis: - Study design and Population (Pervious DMARD use)...84 Table 13 Subgroup analysis: - Trial duration...85 Table 14 Subgroup analysis: - Study quality (high versus low)...86 Table 15 Subgroup analysis and Meta-regression analysis using Withdrawal due to lack of efficacy and toxicity...87 Table 16 Subgroup analysis and Meta-regression analysis using Withdrawal due to lack of efficacy...88 Table 17 Subgroup analysis and Meta-regression analysis using Withdrawal due to toxicity...89 Table 18 Summary the results of relative risk and 95% confidence interval comparing between fixed and random effects models...90 Table 19 Search strategies comparing between Ovid-MEDLINE and PubMed...92 v

6 Table 20 Performances of Ovid-MEDLINE and Pubmed search for randomized controlled trials comparing the efficacy and safety of MTX combination therapy versus MTX monotherapy in RA patients...94 Table 21 Results of citations retrieved from PubMed compared to EMBASE...95 Table 22 Reasons why studies were missed by searching in PubMed...96 vi

7 List of Figures Figure 1 Flow diagram: - Results of the literature search and disposition of the potentially relevant studies Figure 2 Combined withdrawal due to lack of efficacy and toxicity: - Comparisons between MTX combinations versus MTX monotherapy Figure 3 Withdrawal due to lack of efficacy in DMARD naïve, MTX inadequate responder, and non-mtx inadequate responder: - Comparisons between MTX combinations versus MTX monotherapy Figure 4 ACR responses in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Figure 5 EULAR responses in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Figure 6 Tender joint counts in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Figure 7 Pain in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Figure 8 Patient global assessment of disease activity in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Figure 9 ESR in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Figure 10 CRP in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Figure 11 HAQ in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Figure 12 Radiographic outcomes in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Figure 13 ACR responses in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 14 Tender joint counts in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy vii

8 Figure 15 Swollen joint counts in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 16 Pain in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 17 Patient global assessment of disease activity in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 18 Physician global assessment of disease activity in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 19 CRP in MTX inadequate responders: - A comparison between MTX combinations versus MTX monotherapy Figure 20 HAQ in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 21 ESR in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 22 Radiographic outcomes in MTX inadequate responders: - A comparison between MTX combinations versus MTX monotherapy Figure 23 ACR responses in non-mtx inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 24 EULAR responses in non-mtx inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 25 Tender joint counts in non-mtx inadequate responders: - A comparison between MTX combinations versus MTX monotherapy Figure 26 Swollen joint counts in non-mtx inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Figure 27 Patient global assessment of disease activity in non-mtx inadequate responders: - A Comparison between MTX combinations versus MTX monotherapy Figure 28 Physician global assessment of disease activity in non-mtx inadequate responders: - A Comparison between MTX combinations versus MTX monotherapy Figure 29 Pain in non-mtx inadequate responders: -Comparisons between MTX combinations versus MTX monotherapy viii

9 Figure 30 ESR in non-mtx inadequate responders: -Comparisons between MTX combinations versus MTX monotherapy Figure 31 CRP in non-mtx inadequate responders: -A comparison between MTX combinations versus MTX monotherapy Figure 32 HAQ in non-mtx inadequate responders: -A comparison between MTX combinations versus MTX monotherapy Figure 33 Total adverse reactions: -Comparisons between MTX combinations versus MTX monotherapy Figure 34 Gastrointestinal side effects: -Comparisons between MTX combinations versus MTX monotherapy Figure 35 Abnormal liver functions: -Comparisons between MTX combinations versus MTX monotherapy Figure 36 Mucositis: -Comparisons between MTX combinations versus MTX monotherapy Figure 37 Hematological side effects: -Comparisons between MTX combinations versus MTX monotherapy Figure 38 Infection: -Comparisons between MTX combinations versus MTX monotherapy Figure 39 Withdrawal due to adverse reaction: -Comparisons between MTX combinations versus MTX monotherapy Figure 40 PubMed usage data from June, 2007 to March, Figure 41 Diagram illustrating the use of Boolean logic operators: OR, AND, and NOT for searching in the electronic bibliographic databases ix

10 List of Appendices Appendix 1 Search strategy Appendix 2 Data abstraction form Appendix 3 Study quality assessment checklist (Cochrane Back review group) x

11 Chapter 1 Introduction and Background Rheumatoid arthritis and Methotrexate Rheumatoid arthritis (RA) is the most common form of inflammatory arthritis affecting approximately 0.5 to 1 % of the global population 1. It is characterized by the inflammation of the synovial tissue, which if untreated, leads to permanent structural damage and eventual long term disability and impaired quality of life 2. Early diagnosis and aggressive treatment, therefore, is the fundamental strategy to suppress inflammation before patients develop irreversible damage 3-6. Disease modifying anti-rheumatic drugs (DMARD) are the mainstay of treatment in RA. Methotrexate (MTX) is the most commonly used DMARD in RA with less toxicity and better tolerability than others 7. It is pharmacologically classified as an anti-metabolite due to its antagonistic effect on folic acid metabolism. Although the mechanism by which low dose MTX modulates inflammation in RA is still unclear, the evidence for its efficacy has been well documented in several randomized controlled trials (RCT) and meta-analyses Unfortunately, MTX alone may not fully control disease activity. MTX rarely induces remission 12, 15, 16 and does not halt erosive disease Increasingly, in the last two decades, MTX has been used in combination with other non-biologic DMARDs based on accumulating evidence that in early RA, initial combination therapy is associated with earlier clinical improvement, less progression of joint damage 24-26, and improved productivity 27. MTX and non-biological DMARDS in combinations have diverse efficacy and toxicity indices. Although many MTX and traditional DMARD combination regimens have been studied, it is still not clear whether MTX alone or MTX combinations should be used in RA patients when balancing the efficacy and toxicity. Due to its relevance to clinical practice, this clinical question was voted by international rheumatologists from 17 countries in Europe, North and South America as one for which a recommendation should be developed 28. To provide the most up-to-date evidence to experts for formulating recommendations about MTX therapy, we performed a systematic review and meta-analysis comparing the efficacy and toxicity of MTX combination therapy versus monotherapy in RA patients. 1

12 2 Systematic review and meta-analysis The concept of reviews in medicine is not new. Preparation of reviews has traditionally depended on implicit, idiosyncratic methods of data collection, and interpretation. Narrative reviews, therefore, have a number of disadvantages that may distort their results and conclusions. They are usually broad in scope and usually provide a general perspective on a topic. Reviewers often use informal, subjective methods to identify, include, and interpret studies. They tend to select the studies that reinforce their preconceived ideas or promote their own views on a topic 29. They rarely assess and incorporate the quality of studies into their results. This method is potentially unsound and subject to bias. The overall conclusions may be inaccurate or distorted. Systematic review is a form of research that provides a summary of medical reports on a specific clinical question, using explicit methods to search, critically appraise, and synthesize the world literature systematically 30. Systematic reviews are recognized as the highest level of research evidence 31, and their methodology is well documented 32. Meta-analysis is a specific methodological and statistical technique for combining quantitative data 33, which generally aims to produce a single estimate of a treatment effect 34. It is one of the tools used in preparing systematic reviews. Meta-analysis, if appropriate, will enhance the precision of estimates of treatment effects, leading to a reduced probability of false negative results. It is always desirable and appropriate to systematically review a body of data, but it may be inappropriate or even misleading, to statistically pool results from separate studies in certain situations 35. Role and importance of systematic review in clinical practice Although primary research is potentially useful to clinicians, researchers, and guideline developers, many situations hinder its use. For many common disease conditions or commonly used interventions, the volume of relevant articles is high. Healthcare providers, researchers, and policy makers are inundated with unmanageable amounts of information. Systematic review is a tool that can be used to refine overwhelming amounts of information. Through critical exploration, evaluation, and synthesis, the systematic review can separate insignificant, unsound, or redundant studies from salient and critical studies

13 3 that are worthy of reflection 36. Furthermore, evidence may be inconsistent or contradictory. Systematic reviews can determine consistency among studies of the same intervention or even among studies of different interventions (e.g., varying doses or intensities of classes of therapeutic agents) 37, and also explain inconsistencies and conflict in the data 38. Frequently, the generalizability of evidence can be established in systematic reviews. The diversity of multiple included studies provides an interpretive context, not available in any one study 39. This is because studies addressing similar questions often use different eligibility criteria for participants, different definitions of disease, different methods of measuring or defining exposure, different variations of a treatment, and different study designs 40. Generalizability addressed systematically is important for developing clinical practice guidelines (CPG) that aim to provide guidance for broad spectrums of clinical conditions in daily practice. Sometimes many important studies assessing outcomes with a low event rate or small effects of intervention are underpowered to demonstrate a clinically significant benefit. In this situation, the systematic review and especially meta-analysis is a useful tool to increase power and precision yielding a definitive combined effect size 38. In other situations, data may be sparse or not available. In this case, performing a comprehensive search, as part of a systematic review, will identify and suggest future research relevant to clinical practice. Although performing a systematic review is arduous and time-consuming, it is an efficient scientific technique yielding robust evidence and is usually faster and less expensive than embarking on a new study 38. This is also relevant to CPG development, as a CPG should be based on the most recent evidence for current daily practice. If guideline developers have to conduct a new study, guideline development may be delayed past the point of being useful. Systematic reviews either previously published or created de novo by guideline developers are, therefore, the most important evidence resource to incorporate into guideline development. However, systematic reviews can lay the foundation for evidencebased guideline only when they are well-developed using rigorous and explicit methodology to reduce bias. Otherwise, they can distort the integrity of guidelines.

14 4 Problems and pitfalls of systematic review Although systematic review is often recognized as the highest level of research evidence 31, it is a form of retrospective observational study, which is potentially subject to many biases like other retrospective studies 32. Methodological flaws can create bias that affects the quality of reviews. Jadad, et al 41 evaluated 36 Cochrane reviews and 39 systematic reviews published in peer-review journals. They found a number of deficiencies in the process of conducting the reviews including lack or inadequate description of inclusion and exclusion criteria, lack of a clearly identified primary outcome, lack of assessment of trial quality, language restrictions, inappropriate pooling of data, and lack of updating reviews. Even with well-conducted methods, a review based on poor quality or poorly reported trials is unlikely to provide useful data for informing clinical decisions 42. Making recommendations usually needs information pertaining to certain outcomes such as quality of life, patient preferences, or rare adverse events. These outcomes are sometimes inadequately represented in reviews 43 because most primary studies focus on clinically objective outcomes, e.g., mortality rate, infection rate, or duration of hospitalization, and are underpowered for rare or usual adverse events. Systematic reviews usually narrowly focus on specific clinical questions; these sometimes limit their generalizability for daily practice. Finally, the number of systematic reviews and metaanalyses published has increased in recent years, leading to a rise in the number of reviews addressing the same or very similar clinical questions, with a concomitant increase in conflicts among reviews. Such discordance can confuse rather than clarify and causes difficulties for decision-makers including clinicians, patients, and policy-makers who rely on these reviews e Initiative (Evidence, Expertise, Exchange) in Rheumatology The 3-e Initiative (Evidence, Expertise, Exchange) in rheumatology is a multinational effort, aimed at promoting evidence-based medicine by formulating evidencebased recommendations addressing clinical problems in daily practice. It involves a broad international panel of practising rheumatologists from Europe, North and South America. Starting in 2006, two phases of the 3-e Initiative have been successfully accomplished. The

15 5 first 3-e Initiative, , addressed issues relevant to ankylosing spondylitis in the areas of diagnosis, monitoring, and treatment 45. The second 3-e Initiative, , developed practical recommendations for the use of MTX in rheumatic disorders 28. These recommendations were developed with rigorous and explicit methodology by integrating systematically generated evidence and expert opinion from a broad panel of international rheumatologists. In the 3-e Initiative of , a total of 751 rheumatologists from 17 countries participated in three rounds of discussion and voted on the recommendations. Each country was represented by a scientific committee, consisting of one principal investigator and 5-16 members. The bibliographic team consisted of 6 international fellows from 4 countries (Canada, France, the Netherlands, and Spain), 3 mentors experienced in systematic review, and a scientific organizer. During the first round of international meetings, 87 participants formulated and selected the 10 clinical questions using a systematic approach. The first step was a small group discussion within their countries to propose a maximum of 10 clinical questions per country pertaining to the use of MTX in rheumatic disorders. Then all the questions were pooled, rephrased, and grouped into similar topics by a group composed of the bibliographic team and a principal investigator from each country. Subsequently, all participants came together to discuss and vote for the final, most important, 10 clinical questions via the Delphi method. The 10 questions comprised: 1) What is the best dosing strategy and optimal route of administration for MTX in patients with RA to optimize rapid early clinical and radiographic response and minimize toxicity? 46 2) What are indications for pausing/stopping/restarting MTX in case of elevated liver tests and when is liver biopsy indicated? 47 3) What is the long term safety of MTX with regards to cardiovascular, malignancies, liver and infections? 48 4) What is the difference between MTX combination therapy versus monotherapy in terms of efficacy and toxicity in RA? 49 5) Is folic/folinic acid supplementation to MTX useful in reducing toxicity for adult patients with RA?

16 6 6) What is the optimal (clinical, laboratory, imaging) safety monitoring of patients with MTX? Which interval of time? 7) What pre-administration work-up is necessary (co-morbidities / social behavior, physical, laboratory and radiographic data) to identify MTX exclusions and/or get a baseline? 8) What is the optimum management of usual dose MTX in RA patients in the perioperative period to minimize perioperative morbidity and while maintaining RA control? 50 9) How should MTX use be managed when planning pregnancy (male and female patients), during pregnancy and after pregnancy? 51 10) Is MTX effective as a glucocorticoid-sparing (adjuvant) treatment compared with placebo or other DMARDS in chronic inflammatory rheumatic disorders? The bibliographic team conducted a systematic literature review with the same methodology following the updated guidelines of the Cochrane Collaboration under supervision of their mentors. In the first step, the clinical questions were rephrased into epidemiological questions and constructed using the PICO model (population-interventioncontrol-outcome) for developing the comprehensive search strategy 30. The comprehensive search was formulated and conducted by collaborating with experienced information specialists. The relevant articles were included according to the pre-specified inclusion and exclusion criteria. The data were abstracted using standardized forms. The quality of evidence was assessed using the Oxford Level of Evidence 52. The evidence was summarized and meta-analysis was performed if appropriate. In the second round, the results of 10 systematic reviews were presented to national scientific committees consisting of rheumatologists, who work in academic centers or communities in each country. Subsequently, a national meeting was held in each country (total of 751 participants in 17 countries) to discuss the generated evidence and propose a set of recommendations. In a third joint meeting, the scientific committees (n = 94 participants) merged all propositions into 10 final recommendations via discussion and Delphi vote. The grade of recommendation according to the Oxford Levels of Evidence was assessed, and the level of agreement was measured on a 10-point visual analog scale (1 = no agreement, 10 =

17 7 full agreement). Finally, the potential impact among the participants was assessed using 3 statements: this recommendation will change my practice ; this recommendation will not change my practice as it is already my practice ; and this recommendation will not change my practice as I don t want to change my practice for this aspect. For their dissemination, the 10 recommendations were presented in the plenary session of the annual scientific meeting of American College of Rheumatology (ACR) 2008 and in the poster session of the annual scientific meeting of European League Against Rheumatism (EULAR) There was encouragement that the recommendations and individual systematic reviews be published in an international peer reviewed journal. This thesis is part of evidence developed for the 3-e Initiative, I worked in this initiative as a bibliographic fellow under Professor Dr. C Bombardier s supervision. We conducted a systematic review to address the clinical question number 4: What is the difference between MTX combination therapy versus monotherapy in terms of efficacy and toxicity in RA patients? The clinical questions and concerns were uncovered during the conduct of this systematic review. When performing this review, much variation among the included studies was revealed. This raised questions concerning: 1) the potential sources of heterogeneity in this review; 2) the extent and direction of the heterogeneity; and lastly, 3) how to manage or incorporate the heterogeneity in the review to reduce bias that may mislead audiences. Although the process of review was standardized across the 3-e Initiative to ensure that similar methodology was employed in all the 3-e reviews, we found a discrepancy in the database providers used for the literature search among 6 fellows from 4 different countries. The bibliographic team agreed to search in 3 main electronic databases: MEDLINE, EMBASE, and the Cochrane Controlled Clinical Trial Registry. However, the database provider differed between users of MEDLINE. All except the Canadian fellows accessed MEDLINE via PubMed, while the 2 Canadian fellows had access to Ovid, a fee-based web database provider. This issue was not discussed before performing the search, and this led to the following two questions: Is there a difference between the 2 MEDLINE database providers, Ovid-MEDLINE and PubMed when performing a comprehensive search for

18 8 systematic reviews? and What is the impact of using the different database providers on systematic reviews? To answer these questions, we used the search strategy in the review of The efficacy and safety of MTX combination therapy versus monotherapy in rheumatoid arthritis patients, which was first searched through Ovid-MEDLINE. We ran the search again using both PubMed and Ovid-MEDLINE. Another methodological question that we planned to answer by the 10 3-e reviews was what additional benefit searching EMBASE provided over MEDLINE when searching for evidence in Rheumatology. All fellows were asked to record the unique citations retrieved by EMBASE but not MEDLINE in each review. However, we had data from only 8 questions; the search for 2 clinical questions (question 6 and 7) found no evidence to answer these 2 questions. This thesis is organized as follows: chapter 1 lays out the background relating to systematic reviews, meta-analysis, and the 3-e Initiative project; chapter 2 presents the detailed systematic review and meta-analysis of The efficacy and safety of MTX combination therapy versus monotherapy in rheumatoid arthritis patients ; chapter 3 demonstrates the investigation of and approach to heterogeneity in this review; chapter 4 explores the consequences of using PubMed versus Ovid-MEDLINE and EMBASE in systematic reviews in Rheumatology; and the discussion and conclusions are in chapter 5.

19 Chapter 2 The Efficacy and Toxicity of Methotrexate Monotherapy versus Methotrexate Combination Therapy with Non-biologic Disease-modifying Anti-rheumatic Drugs in Rheumatoid Arthritis: A Systematic Review and Meta-analysis INTRODUCTION RA is a chronic systemic inflammatory disease of unknown etiology. It is the most common form of inflammatory arthritis and affects approximately 0.5 to 1 % of the global population 1.The disease affects individuals at any age; however, it presents most commonly in women aged years. It is characterized by the inflammation of the synovial tissue, which if untreated, leads to permanent structural damage and eventual long term disability and impaired quality of life. Systemic inflammation usually occurs and leads to increased morbidity and mortality associated with cardiovascular disease and malignancy The typical manifestation is symmetrical polyarthritis involving the small joints of the hands and feet, the large joints of the extremities, and the upper cervical spine 64. A number of extra-articular manifestations can occur and vary with the duration and severity of the disease 64. Approximately, 36% to 75% of patients with early RA have joint erosions and develop the initial erosions within the first 2 years of symptom onset 2, 65. Joint damage progresses at a consistent rate over the course of the disease and accounts for approximately 25 % of the disability. The association between radiographic damage and disability is strongest with disease duration of more than 8 years; however, prevention of structural damage early in the disease is likely to preserve patient function 66. Early diagnosis and aggressive treatment, therefore, is the fundamental strategy to suppress inflammation before patients develop irreversible damage 3-6, 67. Disease modifying anti-rheumatic drugs (DMARDs) are the mainstay of treatment in RA. This heterogeneous group of drugs has delayed onset of action, ranging from 3 weeks to 3 months and has no immediate analgesic effect 68. It suppresses the inflammatory process and may retard joint destruction. 9

20 10 Methotrexate (MTX) (4-amino-N-10-methylpteroyl glutamic acid) is among the most effective of the DMARDs in the treatment of RA. It is an analogue of folic acid and of aminopterin (4-amino-pteroyl glutamic acid), a folic acid antagonist 69 and was first used in 1948 to treat acute leukemia. However, over the last 20 years, it has been used in the treatment of RA as well as many rheumatic diseases 7. Many pharmacological mechanisms of MTX action have been suggested including: 1) Inhibition of de novo purine synthesis 70 2) Promotion of adenosine release leading to inhibition of production of proinflammatory cytokines [TNF-α, interleukin-6 (IL-6), and IL-8] and leukocyte accumulation 71 3) Induction of activated T cell apoptosis and clonal deletion 72 4) Inhibition of IL-1 production 73, 74 5) Reduction of IL-6, IL-8, soluble TNF receptor, and soluble IL-2 receptor level 75. However, the mechanism by which low dose MTX modulates inflammation in RA is still unclear. It has been shown that in RA patients low dose MTX inhibits neutrophil chemotaxis 76, 77 by inhibiting leukotriene B4 78 and reduces in vitro prostaglandin E 2 release 79 and superoxide production 80. MTX has become the most commonly used DMARD in RA with less toxicity and better tolerability than other DMARDs 7. In the mid 1980s, four randomized, doubled blinded, placebo-controlled trials in RA showed a superior efficacy of MTX over placebo in short term treatment In a randomized, 24-week, double-blind, crossover trial comparing oral MTX (2.5 to 5 mg every 12 hours for three doses weekly) with placebo in 28 refractory RA patients 9, the MTX group had statistically significant reductions in the number of tender or painful joints (26 ± 4 vs. 4 ± 4, p < 0.01), the number of swollen joints (14 ± 2 vs. 5 ± 2, p < 0.05), disease activity according to physician and patient assessments (1.6 ± 0.2 vs. 0.1 ± 0.2, p < 0.01 and 1.5 ± 0.2 vs. 0 ± 0.2, p < 0.01, respectively), and 15-meter walking time (5 ± 1 vs. 3 ± 1, P < 0.03) at the 12-week crossover visit. These variables, as well as erythrocyte sedimentation rate, showed significant improvement at 24 weeks in the population crossed over to MTX (no detailed data were provided, P < 0.01). Anderson, et al 11 studied 12 refractory RA patients treated with weekly pulse MTX in a double-blind,

21 11 placebo-controlled, crossover trial. After 13 weeks of therapy, marked clinical improvement occurred in the patients after MTX therapy compared to after placebo therapy, judged by the number of swollen joints (6.9 ± 1.5 vs.19.4 ± 3.5 vs., p < 0.002) and the number of tender joints (12.6 ± 4.1 vs ± 4.9, p < 0.001), the duration of morning stiffness (minute) (78 ± 34 vs. 242 ± 38, p = 0.002), and the subjective assessments of clinical condition on visualanalogue scale (VAS) (5.8 ± 0.8 vs. 1.6 ± 0.6, p < 0.001). This improvement was associated with a decrease in sedimentation rate (33 ± 7.5 vs ± 7.4, p < 0.001) and decreases in the serum levels of immunoglobulin (Ig) G (1069 ± 83 vs.1311 ± 110, p < 0.001), IgM (170 ± 30 vs. 253 ± 40, p < 0.002), and IgA (282 ± 55 vs. 361 ± 71, p < 0.005). Thompson, et al 10 compared 2 doses of injected MTX (10 and 25 mg/wk versus placebo) in 48 refractory RA patients. At 6 weeks, the effect of the two MTX doses did not differ significantly, but patients on MTX had fared significantly better than those given placebo [swollen joint score: 18 ± 15 vs. 35 ± 18 (p < 0.001), tender joint score : 25 ± 32 vs. 55 ± 29 (p < 0.002), pain score on mm VAS : 33 ± 24 vs. 65 ± 20 (p < 0.001), grip strength (mm/kg) : 126 ± 8 vs. 97 ± 7 (p < 0.005), and erythrocyte sedimentation rate (ESR) (mm/hr) : 29 ± 14 vs. 43 ± 13 (p < 0.001)]. At 12 weeks, all indices showed statistically significant improvement in two MTX groups (no data were shown in the published article). Adverse reactions necessitated change from 25 mg to 10 mg in some patients, but no major adverse events of MTX were noted. William, et al 8 conducted a prospective, controlled, double-blind, multicenter trial comparing placebo and 7.5 mg/wk of MTX in 189 RA patients. After 18 weeks, 110 patients able to tolerate MTX therapy were significantly improved, compared with patients receiving placebo therapy for all clinical variables measured, including the number of joint pain/tenderness [10 (11%) vs. 30 (32%), p = 0.001], the number of the swollen joint count [4 (4%) vs. 20 (21%), p = 0.002], and patient and physician assessment of disease activity on 10 cm. VAS. [7 (7%) vs. 10 (11%), p = and 3 (3%) vs. 9 (10%), p = 0.163]. However, nearly one-third of the patients receiving MTX were withdrawn at the end of the trial due to adverse drug reactions, of which elevated levels of liver enzymes was the most common. They concluded that MTX appears to be effective in the treatment of active rheumatoid arthritis but requires close monitoring for toxicity especially for liver toxicity. A metaanalysis of these trials by Tugwell, et al 12 showed that MTX-treated patients received a 37% greater improvement in swollen joint and tender joint scores, a 39% greater improvement in

22 12 joint pain, and a 46% greater improvement in morning stiffness. MTX was generally welltolerated with 0 to 32 % withdrawal due to minor toxicities, e.g., stomatitis, nausea. In 1990, Felson, et al 13 performed a meta-analysis of placebo-controlled and comparative clinical trials to examine the relative efficacy and toxicity of MTX, injectable gold, D penicillamine (DP), sulfasalazine (SSZ), auranofin (AUR), and antimalarial drugs (Hydroxychloroquine- HCQ), the second-line drugs most commonly used to treat RA. For efficacy based on the number of the tender joint count, grip strength, and ESR, pooled data of 66 trials showed that MTX was superior to placebo [mean improvement ± standard error (SE): 12.6 ± 1.9 vs. 2.9 ± 0.8 for the number of the tender joint count, 42.3 ± 8.2 vs. 9.7 ± 3.4 for grip strength (mm/kg), and 12.5 ± 4.1 vs. 0.7 ± 1.6 for ESR (mm/hr)], oral gold [mean improvement ± SE : 12.6 ± 1.9 vs. 8 ± 0.7 for the number of the tender joint count, 42.3 ± 8.2 vs ± 3.6 for grip strength (mm/kg), and 12.5 ± 4.1 vs. 9.2 ± 1.5 for ESR (mm/hr)], HCQ [mean improvement ± SE : 12.6 ± 1.9 vs. 6.9 ± 1.7 for the number of the tender joint count, 42.3 ± 8.2 vs. 39 ± 6.9 for grip strength (mm/kg), and 12.5 ± 4.1 vs ± 2.6 for ESR (mm/hr)], and comparable to SSZ [mean improvement ± SE : 12.6 ± 1.9 vs ± 1.5 for the number of the tender joint count, 42.3 ± 8.2 vs ± 7.8 for grip strength (mm/kg), and 12.5 ± 4.1 vs ± 2.9 for ESR (mm/hr)], DP [mean improvement ± SE : 12.6 ± 1.9 vs. 9.4 ± 1 for the number of the tender joint count, 42.3 ± 8.2 vs ± 4.5 for grip strength (mm/kg), and 12.5 ± 4.1 vs ± 2 for ESR (mm/hr)] and intra-muscular gold [mean improvement ± SE : 12.6 ± 1.9 vs. 9.1 ± 0.9 for the number of the tender joint count, 42.3 ± 8.2 vs ± 4.6 for grip strength (mm/kg), and 12.5 ± 4.1 vs ± 1.8 for ESR (mm/hr)]. For toxicity, the pooled data of 71 trials showed that injectable gold had higher toxicity rates (no data on the toxicity rate, P < 0.05) and higher total dropout rates than any other drugs (29.9% vs %, P < 0.01), while HCQ and AUR had relatively low rates of toxicity. Unfortunately, the toxicity rate for MTX in 195 patients was imprecise because of discrepancies between trials. Two years later, Felson, et al 14 updated their previous review and compared the benefittoxicity tradeoffs of various second-line drugs in 9 efficacy/toxicity tradeoffs plots. MTX was ranked ahead of SSZ, azathioprine (AZA), DP, AUR, antimalarial, and injectable gold. (The results were displayed in graph of effect size between composite efficacy and toxicity index. No actual numbers of effect size and p values were provided).

23 13 Despite all the favorable efficacy and acceptable toxicity results reported, MTX alone may not fully control disease activity. MTX rarely induces remission 12, 15, 16 and does not halt erosive disease The evidence of slowing radiographic progression is still controversial Increasingly, MTX is used in combination with other non-biologic DMARDs. In 1992, William, et al 81 compared the safety and efficacy of auranofin (AUR), MTX, and the combination of both in the treatment of 335 active RA in a 48-week, prospective, controlled, double-blind, multi-center trial. There were no statistically significant differences among the treatment groups in the clinical or laboratory variables measured (50% improvement in the swollen joint count was 34% in AUR, 43% in MTX, and 36% in AUR+MTX). Withdrawals because of adverse drug reactions were slightly more common for those taking combination therapy, but the differences were not statistically significant (14% in AUR, 15% in MTX, and 21% in AUR + MTX). Withdrawals because of lack of response were more common for single-drug therapy, with the difference between AUR and the combination reaching statistical significance (13% in AUR, 7% in MTX, and 2% in AUR + MTX). Willkens, et al 82 compared the safety and efficacy of azathioprine (AZA), MTX, and the combination of both in a 24-week prospective, controlled, doubleblind, multi-center trial of 212 RA patients. Intention-to-treat analysis showed that combination therapy was not statistically superior to MTX therapy alone, but both combination therapy and MTX alone were superior to AZA alone. A greater number of AZA patients (38%; 28/73) than combination (26%; 18/69) or MTX patients (7%; 5 of 67) terminated therapy prior to week 24 (P < 0.01). Trnavsky, et al 83 compared the efficacy of HCQ alone and in combination with MTX in a randomized, placebo-controlled study lasting 6 months. The results from 40 RA patients found that combination of MTX + HCQ significantly more efficacious than HCQ alone in pain on mm VAS (14 vs. 30, p < 0.05), patient global assessment on Dixon s index (77 vs. 100, p < 0.05), and ESR (mm/hr) (24 vs.38, p < 0.05). However, statistically significant difference between the 2 groups was not found in other clinical (Ritchie s articular index, swollen joint count, and morning stiffness,minute) or laboratory outcomes (Haemoglobin, IgG, IgM, and circulating immune complex). Toxicity was comparable in both groups. Ferraz, et al 84 found that in 68 RA patients who completed the trial, patients receiving 7.5 mg/wk of MTX plus 250 mg/d of chloroquine (CQ) ended the study with a significantly lower joint count (4.5 vs. 7.5; P <

24 ), greater grip strength (mm/kg) (113.3 vs. 89.1; P < 0.05), and better functional ability measured by Health Assessment Questionnaire (HAQ 0-3) (0.636 vs ; P < 0.05) than the patients in the MTX alone group. Mild adverse events were more frequently observed in the MTX + CQ combination, 17 events in 15 patients, compared to 9 events in 8 patients in the MTX group, but it was not statistically significant. Haagsma et al 85 compared the efficacy and safety of MTX + SSZ or MTX alone in 24 week, open-label, randomized controlled trial (RCT) of 40 RA patients with active arthritis despite adequate SSZ therapy. The mean decrease in the Disease Activity Score (DAS) in the group of patients receiving the combination was significantly greater than in the MTX group (-2.6 ± 0.7 vs ± 0.7; p < 0.001) with no difference in the occurrence of toxicity between the two groups. The same combination was studied in a large, double-blind, RCT with a longer follow-up period (52 week) 86. But the population was early RA (< 1 year of disease duration) and was never treated with DMARDs before randomization. MTX + SSZ was more efficacious than SSZ or MTX alone based on the DAS [-1.26 for MTX + SSZ, for SSZ, and for MTX; p = 0.019]. However, there were no statistically significant differences in terms of other outcomes, either the European league against rheumatism (EULAR) good responders (SSZ:34%; MTX:38%; SSZ + MTX:38%) or the American college of rheumatology (ACR) criteria responders (SSZ:59%; MTX:59%; SSZ + MTX:65 %). Adverse events occurred more frequently in the SSZ + MTX group (91%) than in SSZ (75%) and MTX group (75%) (p = 0.025). Although many MTX and traditional DMARD combination regimens have been studied, several important questions remain unclear. What is the relative benefit and toxicity of MTX monotherapy versus MTX combination with other DMARDs? When should the combination DMARD therapy be used: initially or only after a trial of MTX monotherapy? Finally, which is the preferred combination DMARD strategy? These questions are particularly salient as formularies in many countries require the use of MTX monotherapy and MTX combination therapy prior to reimbursing for the more expensive biologic drugs. The objective of this paper is to systematically review randomized trials that compared MTX monotherapy to MTX in combination with other non-biologic DMARDs.

25 15 MATERIALS AND METHODS Literature Search We performed a comprehensive search of electronic bibliographic databases including MEDLINE (1950 to June Week ), EMBASE (1980 to 2007 Week 25), and the Cochrane Central Register of Controlled Trials (CENTRAL) (2nd Quarter 2007) using a search strategy that combined Medical subject headings (MeSH) and keywords for rheumatoid arthritis, Methotrexate, and randomized controlled trials (appendix 1). We also searched the abstracts of the Annual scientific meetings of American College of Rheumatology (ACR) and European League Against Rheumatism (EULAR) from 2005 to 2007, as well as the references lists of all relevant studies, letters, and review articles. All languages were included. Study Selection Two reviewers independently screened the titles and abstracts of the citations and retrieved relevant articles. The following selection criteria were used: 1) Randomized controlled trials (RCTs) of MTX monotherapy versus MTX combined with other DMARDs of at least 12 weeks of trial duration (open label extensions were excluded as well as studies comparing DMARDs not currently used, e.g., oral gold) 2) Participants met the American College of Rheumatology (ACR; formerly, the American Rheumatism Association) 1987 revised criteria for RA with age equal to or older than 18 years 3) Data available on one or more of the following pre-specified outcomes: Outcome measures included in the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) 1993 core set 87 including: number of tender joints, number of swollen joints, pain (Visual Analogue Scale or VAS), patient global assessment (VAS), physician global assessment (VAS), functional status (Health Assessment Questionnaire or HAQ, Arthritis Impact Measurement Scales or AIMS, and Problem Elicitation Technique or PET), and acute phase reactant erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP)

26 16 American College of Rheumatology (ACR) core set 88 : ACR 20, 50, or 70 responses 89 ; ACR remission Disease activity score (DAS) and Disease activity score 28 (DAS28) European League Against Rheumatism (EULAR) response 93 Withdrawal due to lack of efficacy (LOE) Withdrawal due to adverse events (AE) Number of patients who experienced total adverse events or individual adverse events such as gastrointestinal (GI) adverse events (any adverse events except liver toxicity), hepatotoxicity (transaminitis), mucositis, haematological adverse events (anemia, leucopenia, and/or thrombocytopenia), or infection. These are the most common adverse events related to the use of MTX and MTX-DMARD combination. GI adverse events included all symptoms related to GI tract except transaminitis and mucositis. Transaminitis was separately analyzed from other GI adverse events because it is one of the most common adverse events related to MTX, azathioprine, and leflunomide. Additionally, the risk of liver toxicity may increase with the use of MTX combination therapy such as the combination of MTX + leflunomide, MTX + azathioprine. Transaminitis was defined as an increase in the level of Aspartate aminotransferase (AST) or Alanine transaminase (ALT) above the upper limit of normal. Mocositis also commonly occurs in MTX, intramuscular gold, and the combination of both DMARDS. It, therefore, is analyzed separately. Data Abstraction and Quality Assessment Two reviewers independently extracted the data (Appendix 2) and assessed the quality of relevant studies. If the reviewers found any discrepancy between their information, then a consensus was reached by looking at the original article and discussing with the senior reviewer. Study quality was assessed using the van Tulder s scale (Appendix 3), which was developed by the Cochrane back review group 94. This scale comprises questions on the following eleven criteria: addressing randomization, blinding details about the procedure (patients, provider, and outcome assessor), concealed treatment allocation, similarity of the important baseline characteristics, co-intervention, timing of the outcome

27 17 assessment, compliance and withdrawals, and intention-to-treat analysis. Each item is rated as yes = 1 and no or not clear = 0. The total score is the sum of all items so ranges from 0 to 11. The quality of trials was assessed without masking of trial identifiers. Data Synthesis We used RevMan for analysis. The efficacy analysis was stratified into 3 groups based on previous DMARD use. The DMARD naïve, parallel strategy refers to trials where patients who never received DMARDs (including MTX) were randomized to start MTX alone or MTX plus another DMARD. The MTX inadequate response, step-up strategy refers to trials where patients with inadequate response to MTX were randomized to continue the use of MTX alone or to add a second DMARD. The Non-MTX DMARDs inadequate response, step-up strategy refers to trials where patients with inadequate response to DMARDs (other than MTX) were randomly switched to MTX alone or MTX plus another DMARD. The toxicity analysis was stratified by DMARD combination and pooled across trials for each combination. For continuous measures of efficacy, we used either end-of-trial data or change from baseline and pooled them as weighted mean differences (WMD) or standardized mean difference (SMD) as appropriate using a random effects model 95. For the categorical measures of efficacy and toxicity, the end-of-trial results were pooled and estimated using the relative risk (RR) with a random effects model. For efficacy, a RR greater than 1 favors MTX combination therapy: MTX combination therapy increases efficacy, while for toxicity and withdrawals, a RR greater than 1 favors MTX monotherapy: MTX combination therapy increases toxicity or withdrawal. Our pre-specified primary analysis was based on total withdrawal rates for efficacy or toxicity. The heterogeneity of the trials for each pooled analysis was assessed using a visualgraphical method (forest plot), the Cochran s Q (or chi-square) test, and the I 2 test. The method and results of this analysis are discussed in detail in Chapter 3. Publication bias was assessed using funnel plots the plot of effect estimate against standard error on a reverse

28 18 scale. Funnel plots of relative risks and their standard errors were performed only for the primary outcome, the withdrawal due to lack of efficacy and toxicity. RESULTS Our search retrieved 6,938 citations. After review of titles and abstracts and removal of duplicates across databases, 6,846 citations were excluded. The reasons for exclusion are presented in figure 1. Thirty-nine (39) full-text articles were retrieved for further evaluation, and 20 articles (from 19 studies) were retained for our analysis (Figure 1). Sixteen (16) of these 20 articles were found in all 3 databases, and 4 were identified from EMBASE only. Two abstracts from the EULAR annual meeting were also published in full article and included in our review. No additional citations were retrieved by the hand search of the reference lists of relevant articles. The excluded studies and reason for exclusion after full text review are summarized in table 1. Characteristics of the included studies (Table 2A and 2B) The total number of patients in the trials was 2,025. Most of the trials were 6 or 12 months in duration. The doses of MTX ranged between 5 to 20 mg/wk, but most were between 7-15 mg/wk. MTX was administered orally in all trials. Three trial strategies were identified according to the DMARDs used before randomization: a DMARD naïve group, a MTX inadequate response group, and a non-mtx DMARD inadequate response group. Six trials used a parallel strategy where DMARD naïve patients were started either on MTX alone or on a combination of MTX + sulfasalazine (SSZ) 86, 96, 97, MTX + cyclosporine (CSA) 98, 99, and MTX + doxycycline 100. All included early rheumatoid arthritis patients with less than 1 year of disease duration. Five trials used a step-up strategy in MTX inadequate responders where the patients were either continued on MTX plus placebo or a second DMARD was added : MTX + leflunomide (LEF) 101, MTX + CSA 102, MTX+intramuscular gold 103, MTX + levofloxacin 104, and MTX + zolendronic acid 105. The criteria for MTX failure or inadequate response were different across the studies. The inadequate response dose of MTX in these studies ranged 7.5 to 20 mg /wk. The duration on MTX before randomization was 1 105, 3 102, 103, or 6 101, 104 months. Eight trials used a step-up strategy in non-mtx inadequate responders (who had never received MTX before

29 19 randomization), they were randomized to MTX alone or to MTX + SSZ 85, 106, MTX + azathioprine (AZA) 17, 82, MTX + chloroquine (CQ) 84, MTX + SSZ + hydroxychloroquine (HCQ) 107, MTX + bucillamine (BUC) 108, and MTX + previous DMARDs including: gold, D-penicillamine, BUC, and SSZ 109. The criteria for DMARD failure or inadequate response were different across the studies. Previous DMARD use was not clear in one study 110. Methodological quality of studies The detailed quality of studies and total quality score are presented in table 3. Ten studies 17, 82, 84, 96, , 103, 106, 107 demonstrated appropriate randomization, adequate blinding of intervention in both patients and care providers as well as clearly reported number and reason for withdrawal and drop out. Seven of these 17, 82, 84, 96, 99, 101, 107 were high quality (comparison groups had similar baseline characteristics and co-interventions and acceptable withdrawals and drop outs), but one study 100 had a high drop out rate (~59%). In two studies 103, 106, there were unequal co-interventions including steroid or NSAIDS between the treatment groups. In five studies 86, 102, 104, 105, 108, the method of randomization was not explicitly described. Additionally, co-intervention (NSAIDS or steroid) was either unclear 86, 102 or higher 105, 108 in the MTX treatment group Due to their open label nature, five studies 85, 97, 98, 109, 110 were rated lower, and the method of randomization was also unclear in three studies 97, 109, 110. Combined withdrawal due to lack of efficacy and toxicity (Figure 2) Our primary analysis was based on withdrawals for both efficacy and safety; data were available for 13 of the 19 trials. The results showed no advantage of combination therapy over MTX monotherapy in either pooled data across all trial or subgroups [RR 0.89 (95% confidence interval (CI) 0.66 to 1.21) for all trials, RR 1.16 (95%CI 0.70 to 1.93) for DMARD naïve, RR 0.86 (95% CI 0.49 to 1.51) for MTX inadequate response, and RR 0.75 (95% CI 0.41 to 1.35) for non-mtx DMARD inadequate response]. However, there was statistically significant heterogeneity in non-mtx DMARD inadequate response group (I 2 =57.4%, χ 2 = 9.39, df = 4, p= 0.05) with one important outlier: the combination of MTX +

30 20 SSZ + HCQ showed better efficacy/ toxicity ratio over MTX alone with RR of 0.3 (95%CI 0.14 to 0.65). Efficacy Table 4 and 5 summarize an overall view of the efficacy of the DMARD treatments according to various efficacy measures found in individual trial. DMARD naïve, Parallel design The number of patients who withdrew due to lack of efficacy was available in five of the six trials (405 patients) with combination of MTX + SSZ 86, 96, 97, MTX + CSA 98, and MTX + doxycycline 100, MTX combination therapy yielded less patient withdrawal than monotherapy, but it was not statistically significant [RR 0.63 (95%CI 0.34 to 1.17)] (Table 4, Figure 3). The ACR responses were available in three of the six trials that compared MTX monotherapy to MTX combination therapy in MTX naïve patients (Figure 4). These trials included a total of 287 patients. Combination arms were MTX + cyclosporine (CSA) and MTX + doxycycline 100. The only statistically significant result was for the ACR 70 response in one CSA trial with RR of 2.41 (95% CI 1.07 to 5.44) favouring the MTX combination arm. The EULAR response was available in an additional two trials (368 patients) with combinations of MTX+ Sulfasalazine (SSZ) 86, 96 or MTX + CSA 99. There was no statistically significant difference between the two groups for a good or moderate EULAR response or remission (Table 4, Figure 5). Individual continuous efficacy measures were available in two of the six studies comparing MTX alone to MTX + SSZ (Table 5). There were no statistically significant differences in responses for the number of the tender joint count 96 (Figure 6), pain 97 (Figure 7), patient global assessment 96 (Figure 8), ESR 96, 97 (Figure 9), and CRP 97 (Figure 10), [WMD -1.7 (95% CI to 2.71), WMD (95% CI-5.11 to 2.4), WMD 0.7 (95%CI to 11.64), WMD (95% CI to 3.74), and SMD 0.66 (95% CI -2.7 to 4.1), respectively]. However, the HAQ score response was slightly higher in the combination group of those 2 studies 96, 97 [WMD 0.1 (95% CI 0.09 to 0.11)] (Figure 11). 98, 99

31 21 The radiographic outcome from one study of MTX+CSA 98 showed a small but statistically significant reduced progression in the combination therapy [WMD of Modified Sharp s score (95%CI to -0.45)] (Figure 12). MTX inadequate response, Step-up design The number of patients who withdrew due to lack of efficacy was available in 3 of the 5 trials (476 patients) with combination of MTX + LEF 101, MTX + CSA 102, and MTX+ intramuscular (im) gold 103 showing significantly fewer patient withdrawals than in the MTX monotherapy group with RR of 0.42 (95%CI 0.21 to 0.84) (Table 4, Figure 3). The ACR responses were available in four of five trials (552 patients) that compared MTX monotherapy to MTX combination therapy in MTX inadequate response patients (Figure 13). Combination arms included MTX + leflunomide (LEF) 101, MTX + CSA 102, MTX + im gold 103, and MTX + levofloxacin 104. In this group of trials, combination therapy was significantly more effective than MTX monotherapy with RR of 2.51 (95%CI 1.92 to 3.28), RR of 4.54 (95%CI 2.51 to 8.2), and RR of 5.59 (95%CI 2.08 to 15.01) for ACR 20, 50, and 70 response, respectively. There was no data on ACR remission and EULAR response. Individual continuous efficacy measures were also available in four of these five trials (Table 5). There were statistically significant differences in responses for the number of the tender joint count (Figure 14), the number of the swollen joint count (Figure 15), pain , 105 (Figure 16), patient global assessment , 105 (Figure 17), physician global assessment , 105 (Figure 18), CRP 101 (Figure 19), and HAQ (Figure 20). The MTX combination responses were significantly greater than monotherapy [SMD (95% CI to -0.33), SMD (95% C I-0.63 to -0.27), WMD (95% CI to -1.79), WMD (95% CI to -1.79), WMD (95%CI , ), SMD (95% CI to -4.36), and WMD (95% CI to -0.21), respectively]. However, a statistically significant difference was not found for ESR [WMD (95% CI to 10.41)] (Figure 21). The radiographic outcome from one study 105 showed less progression in the combination of MTX+ Zolendronic acid, but it was not statistical significance. [WMD of Modified Sharp s score -1.4 (95%CI to 0.01)] (Figure 22).

32 22 Non-MTX DMARD inadequate response, Step-up design The number of patients who withdrew due to lack of efficacy was available in 5 of 8 trials (329 patients) (Table 4, figure 3) with combinations of MTX + Chloroquine(CQ) 84, MTX + SSZ+ hydroxychloroquine (HCQ) 111, MTX + SSZ 106, MTX + BUC 108, and MTX + previous DMARDs (BUC, D-penicillamine, and im gold) 109. MTX combination therapy yielded significantly fewer patient withdrawals than monotherapy with RR of 0.37 (95%CI 0.16 to 0.87). The ACR responses were available in two of eight trials (158 patients) that compared MTX monotherpay to MTX combination therapy in non- MTX inadequate responders (Figure 23). Only the pooled ACR 20 showed a statistically significant benefit for the combinations of MTX+SSZ 106 and MTX+bucillamine(BUC) 108 over monotherapy with RR of 1.85 (95%CI 1.21 to 2.83). There was no data on ACR remission. The EULAR response criteria was available for one of these two studies 106 and showed no statistically significant difference between two groups (Table 4, Figure 24). Individual continuous efficacy measures were available in five of the eight trials (Table 5). There were statistically significant differences in responses for the number of the tender joint count 107 (Figure25), the number of the swollen joint count, 84, 85, 107 (Figure26), patient global assessment 107 (Figure27), and physician global assessment 107 (Figure28). The MTX combination responses were significantly greater than monotherapy [WMD -4 (95% CI to -1.18), SMD (95% CI to -0.17), WMD -10 (95% CI to ), and WMD -10 (95% CI to -5.2), respectively]. However, statistically significant difference were not found for pain 84, 85 (Figure29), ESR 84, 85, 107, 109 (Figure30), CRP 109 (Figure31), and HAQ 84 (Figure32) [WMD (95% CI to13.02), WMD (95% CI to 2.13), WMD -1.2 (95% CI to 0.55), and WMD (95% CI to 0.14), respectively]. Toxicity The toxicity analysis was stratified and pooled by DMARD combinations. Table 6 summarizes an overall adverse event of the DMARD treatments according to various adverse events.

33 23 Total adverse reactions (Figure33) were reported in eight of the nineteen trials (797 patients: 400 in the combination vs. 397 in the monotherapy groups). Overall, the number of adverse events was not increased in the MTX + SSZ [RR 1.13 (95%CI )] 85, 86, 96, 97 and MTX + LEF 101 combinations [RR 1 (95%CI ) versus MTX monotherapy. There was a non-significant trend for increased adverse events in the MTX + CSA combination [RR 3.62 (95%CI )] 98. Both the MTX + AZA 17 and MTX+ im gold 103 combinations increased the risk of total adverse events with RR of 1.67 (95%CI 1.21 to 2.3) and RR of 2.61 (95%CI 1.22 to 5.55), respectively. Gastrointestinal adverse events (Figure34) Gastrointestinal (GI) related adverse events (excluding liver toxicity reported below) were available for seven trials (692 patients: 351 patients in combination vs. 341 in monotherapy groups). Both MTX + SSZ 85, 86, 96, 97 and MTX + LEF 101 combination increased the risk of GI adverse events significantly (RR 1.75, 95%CI 1.14 to 2.67 for MTX + SSZ and RR 1.67, 95%CI 1.17 to 2.4 for MTX + LEF). GI adverse events were not increased in MTX + CSA 98 [RR 4.13 (95%CI ) and MTX + intramuscular gold 103 combinations [RR 0.71 (95%CI )]. Abnormal liver function (Figure35) was analyzed in seven trials (673 patients: 336 in combination vs. 337 in monotherapy groups). MTX + LEF 101 significantly increased the risk of abnormal liver function with RR of 4.3 (95%CI 2.58 to 7.15). While MTX + SSZ 85, 86, 96, 97, MTX + CSA 98, and MTX + BUC 108 showed a non-statistically significant, increased risk of abnormal liver enzymes (RR 1.77, 95%CI 0.29 to for MTX + SSZ, RR 3.1, 95%CI 0.13 to for MTX + CSA, and RR 3, 95%CI 0.13 to for MTX + BUC). Mucositis (Figure36) was analyzed in four trials (229 patients: 123 patients in combination vs. 106 in monotherapy groups). MTX + intramuscular gold 103 increased the risk of mucositis (RR 9.33, 95%CI 0.55 to ), but it was not significant. There was no increased risk in MTX + SSZ [RR 0.62 (95%CI )] 85, 86, 96, 97.

34 24 Hematological adverse events (Figure37) were reported in six trials (415 patients: 215 in combination vs. 200 in monotherapy groups). No difference was demonstrated for the combinations of MTX+SSZ 85, 86, 96, 97, MTX+ im gold 103, and MTX + BUC 108 compared with MTX monotherapy with RR of 2.36 (95%CI ), RR of 1.42 (95%CI ), and RR of 0.32 (95%CI ), respectively. Infection (Figure38) was analyzed in four trials (454 patients: 231 in combination vs. 223 in monotherapy).the risk of infection slightly increased in MTX + SSZ [RR 1.35 (95%CI ) 96, 97 and MTX + im gold [RR 1.6 (95%CI )] 103, but it was not statistically significant. MTX + LEF did not increase the risk of infection [RR 0.79 (95%CI ) 101. Withdrawal due to adverse reaction Figure 39 summarizes the RR and its corresponding 95%CI of withdrawal due to adverse reactions comparing between MTX combination and MTX monotherapy. In 17 of the 19 trials (1,624 patients: 824 in combination group vs. 800 monotherapy group), combination therapy resulted in more withdrawal due to adverse reactions than monotherapy, but the differences were statistically significant only for the CSA 98, 99, 102 and AZA 82 combinations with RR of 1.88 (95%CI 1.02 to 3.5) and RR of 5.18 (95%CI 1.58 to 16.95), respectively. DISCUSSION Despite the introduction of new biologic therapies, methotrexate alone or in combination with other traditional DMARDs remains the recommended first line therapy for most patients with RA 112. Our systematic review addressed the respective risks and benefits of monotherapy versus combination. Nineteen studies met our inclusion/exclusion criteria. Trials of DMARD combinations used different designs: DMARD naïve, parallel strategy, MTX inadequate response, step-up strategy, and Non-MTX DMARDs inadequate response, step-up strategy. These 3 strategies were studied in the different populations (according to previous treatment prior to randomization) and therefore answered different clinical questions.

35 25 The DMARD naïve, parallel strategy is the only design that addresses the questions of whether a combination DMARD therapy should be used initially or only after a trial of MTX monotherapy. Only ACR70 responses showed a statistically significant improvement for the combination therapies but with increasing risk of withdrawals due to toxicity. Additionally, none of the trials that reported other composite or single outcome measures could demonstrate a benefit of an initial course of combination therapy over MTX monotherapy in DMARD naïve patients over 12 to 24 months of follow-up. Although the pooled RR of the primary outcome, withdrawal due to lack of efficacy and toxicity, showed a trend in favour of MTX monotherapy, the benefit of MTX combination therapy over monotherapy cannot be clearly addressed because the confidence interval was wide and crossed 1. The MTX inadequate response, step-up strategy included five trials where the overall combination therapy was significantly more effective than MTX monotherapy. However, when balancing between risk and benefit, the statistically significant benefit of combination therapy was not demonstrated because the confidence interval was wide and crossed 1. This design does not address the question of what is the preferred therapeutic approach when patients fail MTX monotherapy because these trials continue patients who are considered inadequate responders on the same low dose of MTX in both arms. The response of patients in MTX monotherapy arm would be expected to be less than the response of the patients in MTX combination arm. The results, therefore, showed the greater benefit of combination therapy in this group than in DMARD naïve. These trials also do not reflect current practice. The dose of MTX (7-15 mg/wk) is lower than current use, and patients randomized to the MTX monotherapy arm were kept on the same inadequate low dose of MTX. In actual practice, physicians would increase the dose of MTX or change to parenteral administration before adding another DMARD. For all of these reasons, the superiority of the combination therapies in this group of trials does not have clinical credibility. Therefore, the current evidence for patients with inadequate response to MTX is inconclusive pending results from new trials that compare maximum doses of MTX monotherapy with combination therapies. On the other hands, these studies may be useful for patients who cannot tolerate high-dose MTX because they address the question of which

36 26 approach is preferred, MTX combination or monotherapy, in patients who could not tolerate high-dose MTX? In the Non-MTX DMARDs inadequate response, step-up strategy 6 out of 7 studies 84, 85, were available for efficacy analysis. This study design answers the question of which approach is preferred, MTX combination or monotherapy, in patients who did not respond to non-mtx DMARDs. This study design would be useful if patients failed or had inadequate response to a DMARD and then were switched to another DMARD combined with MTX vs. MTX alone. When balancing the risk and benefit, no conclusions can be reached. Although, there was a trend in favour of MTX combination therapy, the confidence interval was wide and crossed 1. In Capell s study, patients who already failed sulfasalazine 2 g/d were randomized to receive MTX alone or MTX+ the same dose of sulfasalazine. In fact, this study compared the efficacy of MTX in both arms and did not address this question. This study actually addressed a different clinical question that is in patients who did not achieve sufficient benefit with SSZ, what is the preferred strategy, adding MTX to SSZ or switching to MTX monotherapy. The Ichikawa s and Hanyu s study are trials of bucillamine, which is not commonly used in North America or Europe. For toxicity analyses, GI and liver adverse events were higher in the sulfasalazine and leflunomide MTX combinations but did not lead to statistically significant differences in withdrawal rates. The total number of adverse events was higher with the gold and azathioprine MTX combinations. Withdrawal rates due to adverse reactions were higher in all the combination therapies, but the differences were statistically significant only for the combinations of MTX + cyclosporine and MTX + azathioprine. The simplest criterion of benefit/risk ratio for drug evaluation is whether a drug is stopped for inefficacy or adverse events. This data was available for 13 of the 19 trials and therefore represents the most powerful results from our meta-analysis. Overall there was no benefit of MTX combination therapy over monotherapy either within the three design strategies or across all trials. However, one study of the combination MTX, SSZ, and HCQ showed better efficacy/ toxicity ratio over MTX alone. To answer our question of which is the preferred combination DMARD strategy, our study suggests that one trial has a clear benefit/toxicity advantage: MTX + SSZ + HCQ, but this result needs to be confirmed in additional trials; also the combinations of MTX+ CSA

37 27 and MTX + AZA should be avoided due to their serious toxicities. The combination of MTX with SSZ or LEF should be used cautiously due to increased GI and liver adverse events. There are important limitations to our findings, mostly stemming from characteristics of the primary studies they are based on. There were diverse regimens of combination therapy. DMARDs have different efficacy and toxicity, and they also have drug interactions when used in combination. We intended to perform subgroup analysis stratified by regimen to demonstrate their efficacy comparing with MTX monotherapy. However, since there were too few trials comparing the same combination regimen, the benefit of specific combinations of therapy can not be addressed. Furthermore, outcome measures were inconsistently reported across the trials. Some studies reported the efficacy as a composite score. The European trials used DAS or DAS 28, while the others reported ACR response measures. Some studies especially those prior to 2000 reported individual variables of clinical and laboratory outcomes. Few studies reported radiographic outcomes and those that did used different methods of assessment. Lastly, the duration of follow up was also different, likely affecting the assessment and interpretation of efficacy as well as toxicity. All of this heterogeneity complicated the pooling of results across studies. In addition, most studies used lower doses of MTX than in current practice, and several studies were done with drugs that are not commonly used (bucillamine, doxycycline, levofloxacin, chloroquine, im gold, and cyclosporine). Lastly, most of the studies included in our review were short-term trials; drawing firm risk benefit conclusions regarding MTX combination therapy is difficult. Nonetheless, this meta-analysis presents useful information particularly when looking at total withdrawal rates where combination across a number of studies is possible. Three previous systematic reviews and 2 meta-analyses 116, 117 compared DMARD monotherapy with combination therapy. Felson and Verhoeven studied nonbiological DMARDs with or without MTX. Hoehberg, Choy, and Douahue included both biological and non-biological DMARDs. Felson, et al (1994) 116 and Verhoeven, et al (1998) 114 concluded that combination DMARD therapy does not substantially improve efficacy with an increase in toxicity. This is consistent with our overall results that include more

38 28 recent trials. Hochberg, et al (2001) 115 included only 4 studies of MTX combined with both biological and non-biological DMARDs (cyclosporine, leflunomide, etanercept, and infliximab) in MTX inadequate responder studies and found that ACR responses improved significantly when a second DMARD was added. Choy, et al (2005) 117 reached the same conclusion in subgroup of MTX and non-mtx inadequate responders based on analysis of withdrawals [OR 0.51 (95%CI 0.3 to 0.82)] but added that improved efficacy is associated with an increased risk of adverse events. Donahue, et al (2008) 113 reported only on a small subset of our included trials (mostly of SSZ combinations), and the remainder of his and Choy s data cannot be compared to our study because they included biologics as well as monotherapies with non-biologic DMARDs other than MTX. CONCLUSIONS In summary, when the balance of efficacy and toxicity is taken into account, the evidence from our systematic review showed no statistically significant advantage of the MTX combination versus monotherapy; only one study with the specific combination of MTX + SSZ + HCQ showed a better efficacy/ toxicity ratio over MTX alone. Adding leflunomide to MTX non-responders improved efficacy but increased the risk of gastrointestinal adverse events and liver toxicity. Withdrawals because of toxicity varied but were most significant with the cyclosporine and azathioprine combinations. Trials are needed that compare currently used MTX doses and combination therapies.

39 Chapter 3 Investigating Heterogeneity in Systematic Review and Metaanalysis INTRODUCTION Meta-analysis aims to compare and possibly combine estimates of effect across related studies 118. However, an uncritical use of the technique can be very misleading. In some situations, it can be inappropriate to combine effects; for example, if the primary studies are substantially heterogeneous or are at risk of bias 119. Heterogeneity in a systematic review refers to any kind of variability among the studies in a systematic review 119. It can occur by the differences in some study characteristics (systematic variation) 120. The different types of heterogeneity have been described. Thompson 121 described the distinction between statistical and clinical heterogeneity. Clinical heterogeneity refers to the differences in the characteristics of the studies; for example, differences in the study population, such as age or disease severity, differences in the intervention, such as the dose or duration or type of intervention, differences in outcome measures, such as timing of outcome assessment, or differences in study design. Statistical heterogeneity is the quantitative observation that there is variability in the review, and it may be caused by known clinical differences or unknown characteristics. The source of heterogeneity was stratified by Glasziou and Sanders 122 into real and artifactual variation. Real variation is the true variation in treatment effect caused by the differences in characteristics of the population, intervention, and/or outcome, while the differences in the study design or methodology are considered an artifact resulting in the artifactual variation. Table 7 lists and categorizes some potential sources of variation causing the different effect size between different studies. In this chapter, we have investigated the potential sources of heterogeneity in our systematic review and meta-analysis. METHODS 1. Identifying Heterogeneity 1.1 Statistical heterogeneity 29

40 30 To identify the heterogeneity among the included studies, we used visual-graphical methods (forest plot with the eyeball test) 120, 123 and statistical methods: Cochran s Q test 124 and I 2 test 125. Forest plot and eyeball test The forest plot is used to display the point estimate (which is a relative risk for dichotomous data and a weighted mean difference or standardized mean difference for continuous data in our review) and its corresponding confidence interval (CI) for individual studies. Using the eyeball method, the extent to which the CIs from different studies overlap each other represents the degree of homogeneity among the studies. Non-overlap or a small degree of overlap of the CIs indicates likely heterogeneity among the studies. Cochran s Q test (chi-square test) Statistical tests assess the observed variability between observed treatment effects is compatible with chance alone. The usual test statistic (Cochran s Q) is computed by summing the squared deviations of each study s estimate from the overall fixed-effects estimate, weighting each study s contribution in the same manner as in the meta-analysis. P- values are obtained by comparing the statistic with a χ 2 distribution with k 1 degrees of freedom (where k is the number of studies). A large p value suggests that the observed variation across studies is plausibly due to chance, and therefore, the hypothesis that studies are estimating the same effect size cannot be safely rejected. Conversely, a small p value indicates a small possibility that the observed variation between studies is due to chance, and therefore, a very small p value indicates statistically significant heterogeneity across studies. I 2 test We also quantified the inconsistency across the studies using the I 2 test 125. I 2 can be calculated using the formula below.

41 31 I 2 = Q-df x 100% Q Where Q = Cochran s heterogeneity s statistics df = degree of freedom = number of studies -1 The value of I 2 is expressed as a percentage of the variability in the trials that may be attributable to between study variations. No universal rule covers the definitions of mild, moderate, or severe heterogeneity. Higgin J &Thompson S 125 suggested that a value of less than 30 % may represent mild heterogeneity, and more than 50% may represent substantial heterogeneity. I 2 can be compared across meta-analyses for different types of outcomes. The major problem of statistical testing for heterogeneity is that it is underpowered. The power of statistical tests for heterogeneity is dependent on the following two factors: the number of studies included and the weight (inverse of the corresponding variance) allocated to each study. The power of test for heterogeneity will be reduced when a large proportion of the total information resides with one study 126. Therefore, the statistical tests for heterogeneity may fail to detect the existence of true differences in the results between studies, particularly when the number of studies included is fewer than or the variance of the outcome varies significantly across studies 126. A non-significant result does not reliably identify lack of heterogeneity in the treatment effects 128. To compensate for this problem, a P-value of less than 0.1 is usually used to indicate significant heterogeneity among the studies as a conservative method 40. Because the statistical power of heterogeneity tests is low, and a number of the potential sources of clinical heterogeneity identified in this review are relevant to clinical practices and may influence the pooled effect sizes, we investigated observed differences between studies regardless of the results of the statistical tests of heterogeneity.

42 Clinical heterogeneity From the clinical perspective in our review, there are some considerable variations between the included studies. Table 8 summarizes the potential sources of heterogeneity in our study. These were explored and discussed in the results section. 2. Investigating and approaching sources of heterogeneity 2.1. Subgroup analysis We used subgroup analysis to separate included studies into subgroups according to certain study-level variables that might explain the variations among the studies. Many characteristics were considered to be sources of heterogeneity in our review (as shown in table 8); however, we opted to explore only 3 important characteristics: treatment received prior to randomization, timing of outcome assessment, and study quality. These study characteristics are clinically relevant and important. The other potential sources of heterogeneity were not explored because of too few studies in each subgroup. Population was stratified into 3 groups: DMARD naïve, MTX inadequate responder, and non-mtx DMARD inadequate responder. Timing of outcome assessment was stratified into 6 months or more than 6 months. Study quality was stratified into high vs. low quality and studies that met vs. did not meet each criterion of the quality assessment list. For assessing the quality of an individual study, we used the quality assessment checklist developed by the Cochrane Back Review group 129. The details of this score were described in chapter 2 and in appendix 3. High vs. low quality stratification was pre-defined by cut-off points provided by the Cochrane Back review group. High-quality trials were defined in two ways: as those that fulfilled at least 50% of the criteria ( 6 of 11) and as those with scores higher than the mean of included studies (> 7.15). The overall estimate of treatment effect was pooled within subgroups. The difference between subgroups was assessed using meta-regression analysis. STATA 9.0 software was used to perform the meta-regression analysis on the log RR scale, with each trial weighting being equal to the inverse of the variance of the estimate for that study and between-study variance estimated with the restricted maximum likelihood (REML) method, a variant of

43 33 maximum likelihood estimation that provides unbiased and accurate estimates of variances by taking account of residual heterogeneity 130. In subgroup analysis, we used only primary outcomes including withdrawal due to lack of efficacy, withdrawal due to toxicity, and combined withdrawal due to lack of efficacy and toxicity Constructing a random effects estimate across all studies Both fixed and random effects models were used for all outcomes irrespective of statistical heterogeneity. If there is no component of variability between studies, the results based on fixed and random effects models are essentially identical 131, 132 because using the Dersimonian and Laird method, when there is no component of variability between studies, the random effects analysis becomes a fixed effects analysis. When there is heterogeneity between studies, the CI of a summary estimate of effect size will be wider with the random effects model. For this analysis, we used all outcomes Meta-regression analysis Due to sample size limitation, we did not perform multivariable regression analysis to adjust for other potential predictors of treatment effect. RESULTS 1.1. Statistical heterogeneity (Table 9) For withdrawal due to lack of efficacy and withdrawal due to toxicity, the heterogeneity was not significant based on the eyeball test, Cochran s Q test, and I 2 test. However, when withdrawal due to lack of efficacy and toxicity were combined, the eyeball test showed moderate overlap reflecting some degree of variability between studies. This variability was statistically significant by Cochran s Q test (χ 2 = 20.98, df = 12, p = 0.05) with a moderate degree of heterogeneity according to the I 2 criterion (I 2 = 42.8%) Clinical heterogeneity Population According to DMARD treatment prior to randomization, 6 trials studied DMARD naïve patients 86, , 5 studied MTX inadequate responders , and 7 studied non-mtx

44 34 inadequate responders 82, 84, 85, In one study, the treatment received before randomization was not clear 110. Intervention Ten combinations were included in our review, 6 studies of MTX+ Sulfazalazine, 3 of MTX + Cyclosporin, 1 of MTX + Leflunomide, MTX + Azathioprine, MTX + Doxycycline, MTX + Levofloxacin, MTX + intramuscular gold, MTX + Zolendronic acid, MTX + chloroquine, MTX + Sulfazalazine + hydroxychloroquine, MTX + Bucillamine, and MTX + previous DMARDs. These combinations have diverse efficacy and toxicity profiles. The dose of MTX used in these trials was also different across the trials. The mean dose of MTX ranged between 5 and 18 mg/wk, which is somewhat lower than current practice. These variations may influence the treatment effect. However, we did not perform subgroup analysis based on this variation due to there being too few studies in each subgroup. Outcome Previously, there was no consensus on what outcome is the most appropriate tool to measure the efficacy of interventions in the RA clinical trials, so the outcome measures used in the included studies varied. Some used composite scores, such as the ACR response criteria, the EULAR response criteria, the DAS, and the DAS28. Some used single variables, such as the swollen joint count, the tender joint count, a pain score, a patient or physician global assessment of disease activity, ESR, CRP, or withdrawal due to lack of efficacy. Table 10 and 11 summarize the efficacy and toxicity outcomes reported in the included studies. The different outcome measures may assess the different aspect of the response to the treatment. This discrepancy also complicates the pooling of results across studies. The timing of outcome assessments was also different across the trials. Nine 82, 84, 85, 101, 102, 104, 105, 107, 110, 7 86, 96-99, 103, 106, 2 100, 108, and studies had 6, 12, 24, and 60-month follow up, respectively. Most DMARDs are slow-acting 64. Therefore, it is likely that the studies that had longer follow-up would result in a higher response rate.

45 35 Study characteristics The mean of the quality score of included studies was 7.15 (range was 3 to 10 out of 11). Fourteen studies were rated as high quality studies using fulfillment of at least 50% ( 6 of 11) criterion, and 11 studies were above the mean of included studies. More than half had adequate randomization (10 studies), blinded patients (13 studies), blinded care providers (14 studies), adequate treatment allocation concealment (10 studies), acceptable drop-out rates (10 studies), and used intention-to-treat analysis (13 studies). Only 6 studies blinded outcome assessors. Six studies reported acceptable compliance, while others did not provide this information. Comparable co-intervention in both arms was reported only in 7 studies. Due to the variation in study quality and characteristics, we performed subgroup analysis to assess their influence on the pooled effect size. 2.1 Subgroup analysis Study design and Population (Table 12) For withdrawal due to lack of efficacy, all study designs favoured MTX combination therapy over MTX monotherapy, but it was statistically significant only MTX inadequate response group [RR 0.42, 95%CI 0.21 to 0.84] and non-mtx DMARD inadequate response group [(RR 0.47, 95%CI 0.16 to 0.87]. There was no statistically significant difference between the pooled RR from the 3 study designs in the meta-regression analysis. For withdrawal due to toxicity, MTX combination therapy significantly increased toxicity compared to MTX monotherapy in DMARD naïve [RR 1.72, 95%CI 1.04 to 2.83] and MTX inadequate response group [RR 1.89, 95% CI 1.05 to 3.41]. This finding was not statistically significant in non-mtx DMARD inadequate response group [RR1.53, 95%CI 0.74 to 3.18]. Subgroup analysis showed no statistically significant difference between the 3 groups. When balancing between efficacy and toxicity, there was a trend in favor of MTX monotherapy in DMARD naïve [RR 1.16, 95% CI 0.7 to 1.93], while in MTX inadequate response and non- MTX inadequate response group, there was a trend in favor of MTX combination therapy [RR 0.86, 95% CI 0.49 to 1.51 and RR 0.75, 95% CI 0.41 to 1.35, respectively]. However, the difference between groups was not statistically significant.

46 36 Timing of outcome assessment (Table 13) When stratifying the trials according to trial duration, the pooled RR in the trial duration of 6 months or > 6 months resulted in the same direction that MTX combination therapy was significantly more efficacious than monotherapy with significantly higher toxicity. However, the increased toxicity was statistically significant only in the trial with 6 month duration. Toxicity increased in the trial duration of more than 6 months, but with the lower end of the CI of 1. The difference between trial duration was not statistically significant in 3 outcomes. Study quality (Table 14) When stratifying the studies according to quality score, there was a trend that low quality studies (using both criteria) overestimated the pooled effect size, except for the analysis of the withdrawal due to lack of efficacy and toxicity using the mean as a cut-off value. However, the difference in RR according to quality was not statistically significant. When stratifying the studies according to each criterion of study quality assessment, there was no statistically significant difference in the primary outcomes between studies that met and did not meet a criterion (table 15-17) Fixed vs. random effects models (Table 18) When comparing the results of the fixed and random effects models, the pooled treatment effects and their 95%CI were slightly different in both analyses using all combined 19 studies and subgroup analysis stratified by study design. The confidence interval was wider in the random effects models as compared to the fixed effects models reflecting some degree of heterogeneity between the included studies. DISCUSSION The statistical heterogeneity was not significant In our review, the statistical heterogeneity was not significant in the three statistical tests used, although many types of clinical heterogeneity in our review were found. One major problem of statistical tests of heterogeneity is that they may have low power to detect a clinically important degree of heterogeneity 126, 131, 133. In addition, our sample size is quite

47 37 small, perhaps leading to a lack of power to demonstrate significant heterogeneity. From a clinical point of view, we believe that heterogeneity exists in our review. The differences in populations and interventions are clinically relevant and should be incorporated into our analysis. Careful investigation of heterogeneity can provide clinically important results by examining subgroups of studies looking at patients who might benefit more or less from a treatment. Clinical heterogeneity and subgroup analysis Why stratify by study design? One clinically important characteristic of the included studies is the treatment received before randomization. These three strategies answer different clinical questions. The DMARD naïve subgroup is the most relevant and useful study design for clinicians in the light of the accumulative evidence of the benefit of early diagnosis and aggressive treatment in RA patients 6, 67, Ideally, RA patients should be diagnosed and treated in the early stages of the disease before they develop irreversible deformities and functional disabilities. It, therefore, is important to use the most effective treatment possible for these patients who would usually never receive any treatment. This study design answers the question, Should MTX alone or MTX combination be started in patients who never received a DMARD? The MTX inadequate response group answers the clinical question, Does the addition of a second DMARD to MTX show a greater benefit than continued MTX alone in patients who did not fully respond to MTX? This study design may not be of practical use to clinicians because in actual practice physicians generally increase the dose of MTX or change to parenteral administration before adding another DMARD. In addition, the patients in this group were randomized to receive placebo or non-mtx DMARD in combination with the same dose of MTX that they had already failed in both arms; therefore, the response of patients in MTX monotherapy arm would be expected to be less than the response of the patients in MTX combination arm. The non-mtx DMARD inadequate response group answers the clinical question, Should MTX combination or monotherapy be used in patients who did not respond to non-mtx DMARDs? This study design would be useful if patients failed a non-mtx DMARD and then were switched to another DMARD in combination with MTX vs. MTX alone. Due to the difference in

48 38 clinical implication between these 3 groups, the treatment effect should not be pooled across the trials even though no statistically significant differences were found between the 3 groups in any outcomes. From this subgroup analysis, we found that the response to the treatment varied depending on the treatment status before randomization. MTX combination is more beneficial than MTX monotherapy only in patients who failed MTX or a non-mtx DMARD, while the benefit of MTX combination over MTX alone was not found in patients who had never received any DMARDs. The toxicity significantly increased in all populations. Based on this information clinicians can tailor the most appropriate regimen for an individual patient. The impact of trial duration Most DMARDs are slow-acting 64. It is likely that the studies that had longer followup would result in a higher response rate. However, our subgroup analysis showed similar results in the trials of less than or equal to 6 months and more than 6 months, that in both MTX combination therapy was more efficacious than MTX alone. For toxicity, the increased toxicity of MTX combination was statistically significant only in the trials of less than or equal to 6 months. This finding may be explained by the attrition bias rather than the trial duration itself. The adverse events of DMARDs usually occur in the first 6 months. The patients who continued in the trial longer than 6 months were the patients who could tolerate to the treatment. This group of patients might be less likely to withdraw due to toxicity afterward. The withdrawal due to toxicity in MTX combination, therefore, did not significantly increase in those trials of more than 6 months. The impact of study quality The quality of the studies included in the meta-analysis influences the conclusion. It has been shown that the results and conclusions of reviews are sensitive to methods for appraising trial quality and incorporating quality into data synthesis 140. Differences in quality in the reported trials may lead to increased variation in the results and also to bias 141. Empirical studies have shown that poor quality studies influence the pooled effect size; however, the direction is inconclusive; some overestimate and some underestimate 146, 147 the effect size.

49 39 Some studies have investigated the influence of different components of methodology on pooled effect size in randomized controlled trials. Moher, et al 142 found that trials that used inadequate allocation concealment, compared with those that used adequate methods, were associated with a 37% increased estimate of benefit. Linde, et al 144 investigated the influence of indicators of methodological quality on study outcomes in a set of 89 trials of homeopathy and found that studies that were explicitly randomized and were double-blind yielded significantly lower effect sizes. Similarly, Juni 143, Djulbegovic, et al 148, Pildal, et al 149, and Poolman, et al 150 showed that the lack of blinding overestimated the treatment effect. Wood, et al 151 also demonstrated that the overall estimated effects were exaggerated by 7% in non-blinded studies and by 17% in studies with unclear or inadequate allocation concealment, but this finding was restricted to trials assessing subjective outcomes, not mortality. Perhaps due to sample size limitations, similar impacts on the effect size of study quality and its components were not found in our review. Despite these results, methodologically sound procedures in clinical trials are still important in the field of Rheumatology. In Rheumatology, many outcomes used are subjective, e.g., pain score, patient or physician global assessment, treatment allocation concealment; thus, blinding procedures are important and necessary in maintaining the integrity of trials. Why use random effects models? Although the Cochran s Q test, the I 2 test, and subgroup analysis showed no statistically significant difference between the studies, the somewhat different results from fixed and random effects models indicated some component of variability. The confidence interval was wider in the random effects models compared to the fixed effects models even in the subgroup analysis. This suggested that there was residual heterogeneity, perhaps in both known and unknown (or unrecorded) trial characteristics, and this should be taken into account. Our conclusions, therefore, were based on subgroup analysis with random effects models.

50 40 Why combine the different combinations? One variation amongst included studies is the DMARD combination. DMARDs have differential efficacy and toxicity profiles. Combining the treatment effect of different DMARDs may obscure, under-, or over-estimate the results. However, we aimed to evaluate the treatment strategy (combination vs. monotherapy), rather than to focus on the specific regimen. The conclusions, therefore, were based on combining data of the different combinations. CONCLUSIONS A meta-analysis attempts to gain greater objectivity, generalizability, and precision by including all the available evidence from trials that pertain to the issue. For this purpose, the trials included in the reviews usually encompass a substantial variety of study characteristics. Investigating potential sources of heterogeneity and the influence of these differences on the overall results needs to be explored carefully. The exploration of heterogeneity provides opportunity to increase the relevant and valid conclusions drawn from studies pertaining to a topic.

51 Chapter 4 Search Strategy for Systematic Reviews: Comparisons between Ovid (MEDLINE) versus Pubmed and EMBASE versus Pubmed This chapter will focus on the performance of 3 different databases commonly used in systematic reviews: Ovid-MEDLINE, PubMed, and EMBASE. If there is a discrepancy between databases, the causes will be explored. The objectives of this study are to: 1) Compare the performance of Ovid-MEDLINE and PubMed for identifying RCTs evaluating the efficacy and safety of non-biologic DMARDs in RA. 2) Determine whether EMBASE identifies RCTs and non-experimental studies not indexed in PubMed for use in the systematic review of RA treatments. INTRODUCTION Systematic review in healthcare is designed to synthesize and summarize the available research evidence for a certain clinical question. Rigorous methods are used to minimize bias and maximize validity of the review. These methods are based on systematic and exhaustive search, extensive analysis, and critical appraisal methods. Unbiased and complete identification of relevant studies is a first important step in performing methodologically sound systematic reviews 152. Failure to include all relevant sources of evidence could diminish the validity and reliability of the review. To completely identify all potentially relevant articles, 2 strategies should be applied: 1) choose appropriate databases; 2) develop comprehensive and sensitive search strategies. 1. Choose appropriate databases The Cochrane collaboration suggested that a comprehensive search should include the following sources 153 :- 1) Bibliographic databases include: International healthcare databases, e.g., MEDLINE, EMBASE National and regional databases which concentrate on the literature produced in those regions, and which often include journals and other literature not indexed 41

52 42 elsewhere, e.g., Chinese Biomedical Literature Database (CBM), Index Medicus for the South-East Asia Region (IMSEAR) Subject-specific databases, e.g., International Pharmaceutical Abstracts, Allied and Complementary Medicine (AMED) Science Citation Index / Science Citation Index Expanded is a database that lists published articles from approximately 6,000 major scientific, technical, and medical journals and links them to the articles in which they have been cited (a feature known as cited reference searching) Dissertations and theses databases. To identify relevant studies published in dissertations or theses, it is advisable to search specific dissertation sources, e.g., Index to Theses in Great Britain and Ireland Grey literature databases. Literature that is not formally published in sources such as books or journal articles may be identified in the conference abstract or some databases, e.g., Healthcare Management Information Consortium (HMIC) 2) Non-bibliographic databases include hand search of relevant journals and reference lists of relevant articles, conference abstracts, and proceedings 3) Unpublished and ongoing studies Searches of health-related bibliographic databases are generally the easiest and least time-consuming way to identify an initial set of relevant reports of studies. A key advantage of the bibliographic databases is that they can be assessed online and searched electronically using search engine technology. The three bibliographic databases generally considered to be the most important sources to search for reports of trials are MEDLINE, EMBASE, and CENTRAL (Cochrane Central Register of Controlled Trials) What is MEDLINE? MEDLINE (Medical Literature Analysis and Retrieval System Online) is a literature database of life sciences and biomedical information including medicine, nursing, pharmacy, dentistry, veterinary medicine, and health care. It contains more than 18 million records (2008) from approximately 5,000 selected publications (Feb 2007) from 1950 to the present 154. Reviewers can access MEDLINE through the fee-based database providers such as

53 43 Ovid, COS (Community of Science), Datastar, and DIALOG or the free-web based version provided by PubMed. What is PubMed? PubMed is a free database developed and maintained by the National Center for Biotechnology Information (NCBI) at the U.S. National Library of Medicine (NLM) located at the National Institutes of Health (NIH). It provides access to bibliographic information that includes MEDLINE, as well as: In-process citations which provide a record of an article before it is indexed with Medical subject heading (MeSH) and added to MEDLINE or converted to out-ofscope status. Some OLDMEDLINE citations that have not yet been updated with current vocabulary and converted to MEDLINE status. Out-of-scope citations (e.g., articles on plate tectonics or astrophysics) from certain MEDLINE journals, primarily general science and chemistry journals, for which the life sciences articles are indexed for MEDLINE. Citations that precede the date that a journal was selected for MEDLINE indexing. Some additional life science journals that submit full texts to PubMedCentral (PMC) and receive a qualitative review by NLM Citations to author manuscripts of articles published by NIH-funded researchers. Many of the scientists who receive research funding from NIH publish the results of their research in journals that are not available in PMC. In order to improve access to these research articles, NIH's Public Access policy asks these authors to provide PMC both author manuscripts and final, peer-reviewed manuscripts of such articles once they have been accepted for publication. The author manuscripts in PMC are the author versions without revision by the independent peer reviewers and the journal's editors.

54 44 What is the difference between MEDLINE and PubMed? In addition to the difference in the databases contained in PubMed-MEDLINE and Ovid-MEDLINE, the methodology used for searching in these 2 database providers, e.g., field tag, Boolean operators, etc. is also different. Many reviewers cannot access the feebased versions of MEDLINE and have increasingly turned to the free web-based version, PubMed. Data from the U.S. National Center for Biotechnology Information (NCBI) has shown that PubMed searches have tremendously increased from 54,663,426 searches in June 2007 to 74,518,774 searches in March 2009 (Figure 40) 155. Searches in MEDLINE using these 2 database providers may yield different citations retrieved with differential trade-offs such as the time needed to perform searches, complexity of the search process, time to review the citations retrieved, etc. What is EMBASE? As mentioned above, EMBASE is one of the major bibliographic databases used for searches in systematic review produced by Elsevier. It covers over 4,800 active peerreviewed journals published in 70 countries / 30 languages, of which nearly 2,000 are unique in that they are not found in MEDLINE. Using PubMed versus EMBASE It has been shown that searching only the MEDLINE database for randomized controlled trials will inevitably miss pertinent information in many fields and topics. It has been shown in several studies that searching in EMBASE retrieves additional articles that may be related to reviews in many fields of Medicine However, the database providers used were different across studies. Two of these studies 161, 162 accessed to MEDLINE via Ovid. Sampson, et al and Suarez-Almazor, et al 161, 162 compared the performance of MEDLINE vs. EMBASE in the identification of articles regarding controlled clinical trials (CCTs) using Ovid-MEDLINE. Topfer, et al 159 and Brazier, et al 160 performed searches in MEDLINE using different commercial suppliers (DIALOG and Datastar, respectively). Database providers were not clear in the other three studies The PubMed database covers many extra-sources of published information in addition to Ovid-MEDLINE. Therefore, searching in EMBASE in addition to PubMed may not retrieve

55 45 as many additional articles as shown in the previous studies using EMBASE in addition to Ovid- MEDLINE. In the field of Rheumatology, Suarez-Almazor, et al compared the performance of MEDLINE and EMBASE for the identification of CCTs in rheumatoid arthritis, osteoarthritis, and low back pain using hand search as the gold standard. MEDLINE and EMBASE were searched for literature published between 1988 and They found that the EMBASE search identified 85% of the CCT compared to 73% identified by MEDLINE, and EMBASE retrieved 16% more CCT than MEDLINE. However, this study performed the MEDLINE search via Ovid, and the study was not specific for the topic/clinical question of RA. This study also did not test the ability to find non-experimental studies, e.g., cohort or case control studies. PubMed and EMBASE have been improved and have updated the indexing process, and new journals are added every year. The result from searching in the most updated databases may not be significantly different between these 2 databases. 2. Developing comprehensive and sensitive search strategies Developing the most comprehensive search strategy consists of a complete list of search terms related to the clinical question, appropriate use of database-specific search term syntax, and appropriate use of Boolean operators Complete list of search terms related to the clinical question A useful framework for developing the comprehensive search terms related to specific clinical questions is PICO 30. The PICO model consists of 4 domains: P = Population or Problem I = Intervention C = Comparison interventions (if relevant) O = Outcome of interest The search term should comprise all relevant terms in each component of PICO. In addition, some reviewers may want to focus their searches on study design. Many sensitive search strategies related to specific study design or type of study such as randomized

56 46 controlled trials 152, , prognostic studies 166, 167, causation studies 168, and diagnostic studies 169, 170 have been developed. These search terms can be added to a search strategy to increase the precision of a search Appropriate use of a database-specific search term syntax The field tags used in each search term will specify the section where the search term will be searched in the database. The field tag system is also different across databases. Inappropriate use of field tags may cause relevant articles to be missed. For example, in Ovid, the search term randomized controlled trial.ti./ab. will be searched under titles and abstracts. If some studies are randomized controlled trials, but this term appears only in publication type, a search using randomized controlled trial.ti./ab will miss these citations Appropriate use of Boolean operator Since developing a search strategy using the PICO method will result in a set of terms for each domain individually; an efficient method is to combine all search terms in each domain together by using Boolean logic operators. Three Boolean operators: OR, AND, and NOT are commonly used and illustrated in figure 41. OR is used to combine synonyms or similar concepts. It allows articles containing either term to be retrieved. AND is used to combine different concepts and will allow only articles containing both terms to be retrieved. NOT is used to eliminate a concept that is not wanted. It allows only articles containing one term but not the other terms to be retrieved. METHODS Objective 1 To compare the performance of Ovid-MEDLINE and PubMed for identifying RCTs evaluating the efficacy and safety of non-biologic DMARDs in RA. Step 1: Development of search strategy for MEDLINE provided by Ovid A comprehensive search strategy for Ovid-MEDLINE database was developed using the PICO method as following:

57 47 Population = Rheumatoid arthritis Intervention = MTX combination therapy with non-biologic DMARDs Comparison = MTX monotherapy Outcome = Report on efficacy and toxicity compared between both arms using any outcome measures for RA The study design was limited to randomized controlled trials (RCT). The database from 1950 to January 2009 was used with no language restriction. For Rheumatoid arthritis, the Medical subject heading (MeSH) and textword were used without explosion. MeSH is the controlled vocabulary term assigned by indexers at the US National Library of Medicine to describe the contents of the articles included in the MEDLINE database. Textword is used to search for a word that appears in the citation exactly as typed in. Ovid automatically maps a user's search term to a database's controlled vocabulary, providing the opportunity for a user to select subject headings that closely match the search term. If a user searches a database that has a Tree structure, they can opt to Explode or Focus the search term, and also choose subheadings that apply all as part of the Mapping process 171. For Rheumatoid arthritis, its mapping includes juvenile rheumatoid arthritis, Caplan s syndrome, Felty's syndrome, rheumatoid nodule, Sjogren s syndrome, and Still's disease, adult-onset. To include only terms related to our interests, we, therefore, opted not to explode the RA subheadings and included only the following subheadings separately: Caplan s syndrome, Felty's syndrome, rheumatoid nodule, and Still's disease, adult-onset. For MTX, we included the comprehensive list of generic and trade names of MTX. These terms were used as textword with field tag mp, e.g., Methotrexate.mp, mp stands for multi-purpose. An mp search looks in the title, original title, abstract, subject heading, name of substance, and Registry Word fields. Truncation ( $ or : ) was used in textword searching to retrieve variant words, e.g., amethopterin$ will search for both amethopterin and amethopterine. For RCT, we combined terms from the Cochrane Highly Sensitivity Search Strategy (CHSSS) 152 and Glanville JM, et al s search strategy 163.

58 48 To make these terms more comprehensive, we modified some field tags. For example, CHSSS suggests placebo.ti and placebo.ab which will retrieve articles that have the term placebo in the title and abstract, respectively. We changed this to placebo.mp. We also added the term meta-analysis and its variants to capture all related reviews. All terms within the same subset of RA, MTX, and RCT were combined using the Boolean operator OR. All citations under the subset of RA, MTX, and RCT were then combined using the Boolean operator AND. We decided not to include the list of terms related to DMARD combination used with MTX and outcome measures because using only the term MTX, (which would be included in all the desired studies) the trials related to MTX combination would be retrieved. Step 2: Translation of Ovid-MEDLINE search strategy to PubMed format Due to the differences in field tag functions between Ovid and PubMed, the following strategies were applied to make the search strategy for each term as similar as possible. 1. Explosion For the term Rheumatoid arthritis, as mentioned above we searched this term and its related syndromes separately as this strategy was more relevant and comprehensive than exploding Rheumatoid arthritis which included all subheadings. Some subheadings were not related to our topic, e.g., juvenile arthritis, and Sjogren s syndrome. However, this was modified in our PubMed search because PubMed automatically searches the MeSH headings as well as the more specific terms beneath that heading in the MeSH hierarchy 172. We, therefore, did not explode Rheumatoid arthritis by turning off automatic explosion of MeSH headings using the field tag [mesh:noexp] for Rheumatoid arthritis and its related terms. 2. Field tag 2.1 mp Because the field tag mp in Ovid is not available in PubMed, the closest field tag/method to mp in PubMed is untagged term. In PubMed, untagged terms or terms without a field tag that are entered in the search box are matched against a MeSH translation table, a Journals translation table, the Full Author translation table, Author index, the Full Investigator (Collaborator) translation table, and an Investigator (Collaborator) index. If a match is found in this

59 49 translation table, the term will be searched as MeSH (that includes the MeSH term and any specific terms indented under that term in the MeSH hierarchy), and in all fields. If there is no match, the individual terms will be searched in all fields 173. This is not exactly the same as, but close to the use of mp in Ovid. 2.2 adj In Ovid, we used the proximity operator adj (called adjacent ) in textword to search for words in a phrase close together rather than any occurrences spread throughout the article title and abstract by specifying the number of words within the required terms were to appear, for example clinical$ adj2 trial$. This feature is not available in PubMed, so we used AND instead of adj in PubMed, for example clinical* and trial*. This searched both terms in any distance in the specific field. 2.3 ab In Ovid, ab is used to search terms in an abstract field. This feature is not available in PubMed. The similar field tag in PubMed is [ti/ab], which will search specific terms in titles and abstract areas. 2.4 fs In Ovid, the subheadings under each MeSH term are provided to help describe more precisely a particular aspect of a subject, e.g., etiology, diagnosis, laboratory, treatment, drug therapy, etc. fs (or floating subheading) is used to search in a subset of MeSH terms in specific areas. In our search, we used dt.fs to search the subheading of drug therapy. In PubMed, the same feature is available with the field tag MeSH subheading. The dt.fs was then translated to drug therapy[mesh subheading]. 2.5 pt In Ovid, field tag pt is used to search terms in the publication type field. This can be directly translated to PubMed into [pt] with the same function. 3. Truncation The truncation $ or : used in Ovid to capture variant words was replaced with * in PubMed, e.g., methotrexat$ methotrexat* with the same function. Step 3: Comparing the citation results from 2 searches The citation results from Ovid and PubMed were downloaded into a bibliographic program (Endnote ) and reviewed. The inclusion and exclusion criteria for this review were applied to retrieve the eligible studies from these 2 searches.

60 50 Step 4: Test performance characteristics of search strategy In information retrieval, the science of searching for documents, the performance of information retrieval systems are evaluated using the terms recall and precision. Recall is analogous to sensitivity, a term used to assess the performance of diagnostic test. It is defined as the proportion of the total number of eligible studies identified by a specific database provider 152, 163, 174, 175. It evaluates the completeness of the search to identify the relevant articles; thus, recall is the most important search parameter from a scientific perspective 176. Comprehensive searching, with a high recall, is considered standard practice when conducting systematic reviews 177. Precision is defined as the proportion of publications retrieved by the specific database provider that are actually eligible in the review 152. It is equivalent to positive predictive value 174. Precision evaluates the burden on the reviewers to unnecessarily review irrelevant citations. Number-needed-to-read (NNR) was recently coined in analogy to the number-needed-to-treat (NNT) to describe the number of irrelevant references that one has to screen to find one of relevance 170. It is the inverse of precision. NNR helps researchers to easily interpret the performance of the search in the context of systematic reviews. The recall, precision, and NNR of the search strategy using 2 database providers were calculated using the total included studies identified by the 3 electronic databases, handsearching, and the abstracts of annual scientific meeting in our review as a gold standard. Recall = Number of eligible studies identified by a database provider x 100 Total number of eligible studies in the review Precision = Number of eligible studies identified by a database provider x 100 Total number of citations retrieved by a database provider Number-needed-to-read (NNR) = 1 Precision Objective 2 To determine whether EMBASE identifies RCT and non-experimental studies not indexed in PubMed in the systematic review of RA patients. We used the data of 8 clinical questions from the 3-e Initiative projects 28. The following is the list of 8 clinical questions.

61 51 a) What is the best dosing strategy and optimal route of administration for MTX in patients with RA to optimize rapid early clinical and radiographic response and minimize toxicity? (Dosing strategy) 46 b) What are indications for pausing/stopping/restarting MTX in case of elevated liver tests and when is liver biopsy indicated? (Liver toxicity management) 47 c) What is the long term safety of MTX with regards to cardiovascular, malignancies, liver, and infections? (Long term safety of MTX) 48 d) What is the different between MTX monotherapy versus MTX combination therapy in terms of efficacy and toxicity in rheumatoid arthritis? (MTX mono vs. combo therapy) 49 e) Is folic/folinic acid supplementation to MTX useful in reducing toxicity for adult patients with RA? (Folate supplementation) f) What is the optimum management of usual dose MTX in RA patients in the perioperative period to minimize perioperative morbidity and while maintaining RA control? (Management for perioperative period) 50 g) How should MTX use be managed when planning pregnancy (male and female patients), during pregnancy, and after pregnancy? (Management in pregnancy period) 51 h) Is MTX effective as a glucocorticoid-sparing (adjuvant) treatment compared with placebo or other DMARDs in chronic inflammatory rheumatic disorders? (Efficacy in other rheumatic diseases) The search for all reviews was performed using PubMed and EMBASE. All reviewers were asked to record the total citation hits stratified by database and the included studies that were identified by EMBASE only. The reason for missing articles when searching in Pubmed was investigated by reviewing the search strategy and the PubMed/MEDLINE citations related to each article.

62 52 RESULTS Objective 1 To compare the performance of Ovid-MEDLINE and PubMed for identifying RCT evaluating the efficacy and safety of non-biologic DMARDs in RA. The search strategies comparing Ovid-MEDLINE and PubMed are shown in table 19. The search strategy in PubMed retrieved a slightly higher number of citations than in Ovid in all combined search terms for RA (104,892 vs. 97,213), MTX (35,593 vs.34,747), and RCT (4,740,614 vs. 4,215,641). After combining the citations from 3 subsets with AND, 2,036 were retrieved from PubMed vs. 1,930 citations from Ovid. From a total of 20 citations included in our review, PubMed retrieved all 17 eligible citations retrieved by Ovid with an additional 1 citation, Ichikawa et, al s study 108. Ichikawa, et al was the Japanese study published in the Japanese journal, Modern Rheumatology. This journal was indexed in MEDLINE in This article was published in 2005; therefore, it was not indexed in MEDLINE but indexed in PubMed and EMBASE. The other 2 eligible citations not retrieved by Ovid and PubMed were retrieved by EMBASE. The performances of Ovid-MEDLINE and PubMed were summarized in table 20. The recall was 85% for Ovid vs. 90% PubMed, while the precision was comparable, % for Ovid vs % for PubMed. The NNR was 114 for Ovid vs. 113 for PubMed. Objective 2 To determine whether EMBASE identified RCT and non-experimental studies not indexed in PubMed in the systematic review of RA patients. Of the 8 clinical questions related to the MTX use in rheumatic diseases, 4,218 vs. 6,651 citations were indentified in PubMed vs. EMBASE, respectively. After applying the inclusion and exclusion criteria specific to each topic, 208 articles from 2 databases were included in the 8 reviews (Table 21). Twenty-two (22) included articles (11 %) were retrieved by EMBASE, but not PubMed. EMBASE did not retrieve the additional eligible articles to PubMed in 2 out of the 8 reviews. PubMed did not retrieve any additional eligible articles that were not indexed in EMBASE. Reasons for missing trials in the PubMed search Table 22 shows the reasons why 22 eligible studies were missed when searching in PubMed. Eight articles (36%) were not indexed with the search terms used in the search

63 53 strategies In another 8 articles (36%), insufficient or restricted search strategies including an inappropriate or too limited field tag or incomprehensive search terms were used One study was misclassified as the trial retrieved by only EMBASE search, but it was actually indexed and retrieved by the search in both PubMed and EMBASE 194. Finally, in a total of 22 articles, only 5 (23%) 97, 109, were EMBASE unique trials (EMUT) because these trials were published in the journals that were not indexed in PubMed. These journals include the Revue du Rhumatisme (English Edition), the Journal of Internal Medicine of India, the Journal of Postgraduate Medical Institute, the Japanese Journal of Rheumatology, and the Journal of Rheumatology and Medical Rehabilitation from France, India, Pakistan, Japan, and Turkey, respectively. All were published in English. DISCUSSION PubMed versus Ovid MEDLINE is one of the most commonly used databases for the comprehensive search in systematic review. It is provided by many database providers, but the most commonly used database providers are PubMed and Ovid. Both of them have different features that users may have to consider when incorporating them into their reviews. The major differences include the citations contained in databases, the search engines, and costs. As we mentioned in the introduction that the largest component of PubMed is MEDLINE; however, it also contains the additional citations that may be relevant to reviews in Medicine. Searching in PubMed will result in a higher number of citation hits rather than searching in MEDLINE (via Ovid) only. The benefit is that reviewers can be confident that they will not miss any relevant articles, but the drawback is that they may have to review more citations, which may or may not be relevant to their reviews. Our study demonstrated that searching in PubMed in a specific topic in Rheumatology yielded only a slightly higher number of hits and identified one additional relevant article compared to searching in Ovid. The additional study identified by PubMed is Ichikawa, et al. The impact of missing this study may not be statistically significant. Although this study was a high-quality study, it

64 54 provided only 4 out of 22 outcomes used in our analysis. Additionally, it studied MTX in combination with Bucillamine, a DMARD that is not commonly used outside Japan. Other limitations we experienced when using PubMed is that the PubMed website is not stable especially when we ran a high number of search terms and used many Boolean operators to refine our search. The website crashed many times when we combined all terms and limited the results to only adult. Unfortunately, we could not save or retrieve our results. We finally completed our search by re-running all the search strategies several times. This problem did not occur when we used Ovid. The save feature in PubMed is also not suitable for the search process in systematic reviews. When users save their search strategies, the search strategies can be saved only with the final combination of all terms. The lists of each search term and its citation hits are not saved separately. So if users would like to modify their search terms, they have to enter each term and re-run all search terms manually then again combine them together. While in Ovid each search term and the final combination are saved separately, so the users can modify their search terms, and then the final combination will be run automatically. Most MEDLINE search strategies for a specific topic such as randomized controlled trial , prognostic study 166, or diagnostic study 170, 198 were developed specifically for searching via Ovid , 198 or Datastar 170. Some groups also provided the optional format that is compatible with the Pubmed search strategies 165, 198. However, the performances of the suggested search strategies were based on the Ovid-MEDLINE search strategies. Using the search strategies that were translated from Ovid format to PubMed format may not yield the similar recall and precision as demonstrated in the literature. The translation should be performed or suggested by an experienced information specialist. Searching MEDLINE via Ovid for systematic reviews is comprehensive and technically more convenient for users than searching in PubMed. However, PubMed is free to access, while Ovid is costly; therefore, it is not available in many institutes. Our findings were based on the results of searching in only one systematic review of one specific topic. The generalizability of the results needs to be tested in the other reviews in other subject

65 55 areas. This finding can benefit from replication in other fields. Despite studying only one review, the problems and obstacles in the use of PubMed are universal and can be useful for other reviewers. PubMed versus EMBASE The benefit of searching in EMBASE in addition to MEDLINE has been shown in several studies, in different topics and populations including RA However, in most studies, the MEDLINE search was performed via fee-based database providers, e.g., Ovid, Datastar, and DIALOG. As we mentioned that PubMed has indexed citations complementary to MEDLINE from many sources, searching via PubMed may result in less missed citations compared to searching in MEDLINE. Our study showed that searching in PubMed still missed relevant articles in the review of randomized controlled trials and observational studies in RA. Our study confirmed that a comprehensive search requires that at least 2 databases be searched. Furthermore, we demonstrated that two other important strategies to ensure its comprehensiveness are developing the comprehensive list of the search terms and using the appropriate syntax and field tags related to individual database. Seeking the suggestion from information specialists experienced in electronic searching is always necessary to design and run search strategies for systematic reviews. The assistance of information specialists should help to complete a list of the search terms, avoid as many errors, and ensure that the database-specific search term syntax is appropriate, and advanced searching techniques (e.g., exploding) can be employed where available. The results of our study are similar to Sampson, et al s 162 in that EMUTs were more likely to be published outside North America. They also showed that EMUTs influenced the pooled effect size at only 6%. When comparing the performance of search strategies or databases, the relevance of the article identified in the database should be the main concern as this is the ultimate goal of the search process. Searching in EMBASE usually results in publications relating to RCT but may not be eligible in the reviews such as synopsis, editorial, letter, or correspondence. Suarez-Almazor, et al 161 reported that 53% of the randomized controlled trials in RA,

66 56 osteoporosis, and low back pain were EMUTs. However, their relevance to the specific reviews was not explored. In the review of pre-hospital care, Brazier, et al 160 found that 33% (280 out of 849) of their search results before the reviewers assessed the eligibility of the citations retrieved were EMUTs. However, when applying the inclusion and exclusion criteria, only 1 out of the 41 eligible studies (or 2.4%) was EMUT. This finding was similar to ours, although this study was done 12 years ago. In addition, several aspects including the area or topic of reviews, the type of study designs, the time of conducting studies, and the database providers used in the reviews have to be taken into account when comparing the results among the studies since these factors influence the performance of databases and search strategies. Lastly, MEDLINE has a time lag for indexing articles in the database. In 1999, Toper, et al 159 also found that 60% of EMUTs were finally indexed in MEDLINE in the follow-up search. Unfortunately, the duration of time lag was not mentioned in this study. However, PubMed has been developed to help searchers find the recent references by indexing some of the in-process references or the online-published versions of articles before they are officially published in journals. The time lag in PubMed is usually shorter than in MEDLINE. This may improve the precision (or sensitivity) of searching MEDLINE through PubMed. The additional benefit of an EMBASE search may not be as much as demonstrated in previous studies. CONCLUSIONS Searching in Ovid-MEDLINE and PubMed has differential benefits and drawbacks with diverse impact to individuals. Ovid is user-friendly especially for comprehensive searches in systematic reviews; however, its accessibility is a major limitation. PubMed is free web-based and contains the most recent references as well as references other than in the MEDLINE database, but it may not be suitable for the complex search of some reviews. We found a small additional benefit of EMBASE search on the specific topic of RA. These findings are subject to change over time because the bibliographic databases are constantly evolving products. The most common reasons for missing trials in the PubMed search are

67 57 the use of insufficient or restricted search strategies and the use of inappropriate search terms. Consulting an experienced information specialist is an important strategy to perform a comprehensive search for systematic reviews.

68 Chapter 5 Discussion In this thesis, a comparison of the net benefit of MTX combination and MTX monotherapy in RA patients is summarized. Clinical and statistical heterogeneity in this review are investigated. In addition, the impact of a literature search using different databases and database providers is explored and discussed. Strengths The efficacy of a treatment is perhaps the most important factor when making decisions on the choice of treatments. On the other hand, toxicity is one of the factors determining whether patients continue or discontinue treatments. The balance between benefit and harm, therefore, is the ultimate index to determine the success of a treatment and should be considered when summarizing the results of trials in an evidence synthesis. This outcome also assesses patient s preference to the medications. Some adverse events may not be serious or life-threatening but cause withdrawal due to patient s preference such as alopecia especially in female patients. To summarize and compare the net benefit between MTX combination and MTX monotherapy, combined withdrawal due to lack of efficacy and toxicity was, therefore, used as a primary outcome in this systematic review. This information is relevant and assists clinicians in clinical decision-makings. It is also easy to understand from clinicians and patients perspectives. Challenges Although the benefit/risk trade off is the simplest criterion to assess the net benefit of a treatment, the limitation in the use of this outcome is the lack of detailed information concerning the magnitude of specific benefits and harms. By using only this outcome, we have no data on how well the patients would be if they use MTX combination therapy compared to MTX monotherapy. We also have no data on the severity of toxicity because some adverse events are not serious but cause withdrawal such as alopecia from MTX or skin hyperpigmentation from HCQ. In addition, the criteria used for withdrawal due to lack of efficacy and toxicity usually are not clearly defined in each trial. These criteria may be 58

69 59 different across the included trials. This means that they may measure the different entities of the same outcome. Other limitations mostly stem from characteristics of the primary studies they are based on. The included studies are explanatory trials, which are designed to determine the effect of intervention in ideal circumstances. Participants in these trials were thought to have a high chance to respond to the interventions with the lowest risk of having unfavourable outcomes. While in actual practice, RA patients may have underlying conditions that are prone to experience adverse events or may not be able to tolerate the interventions studied in the trials. Consequently, the efficacy of the interventions in actual practice may not be similar as found in the clinical trials. In addition, the conclusions of this review were based on short-term trials, the firm conclusion for the long-term effectiveness and toxicity of the MTX and MTX-DMARD combinations cannot be addressed. This issue is important in the RA treatment because most RA patients have to continue DMARD treatment for at least a year. Some have to continue DMARD treatment life long to maintain remission. This information is more likely to be addressed in observational studies. Accordingly, the results of our systematic reviews need to be confirmed in pragmatic trials and long-term, observational studies. In clinical practice where clinical decision-making depends on individual patients and their circumstances, specific factors need to be considered and discussed, such as the availability of the treatment, care providers experience, the value the patient places on the treatment, and the context or personal circumstances of the patient, for example, medical conditions, financial status, reimbursement system, and so on. Systematic reviews do not usually provide such information as this information is less likely to be studied and summarized in systematic reviews. Future directions Withdrawal due to lack of efficacy and toxicity are the only in common outcomes provided by many studies in the review in this thesis. Previously, there was no consensus on the outcome measures that should be used to assess the efficacy of interventions in RA trials; therefore, the outcome measures used to assess disease activity, functional status, and radiographic damage in eligible studies are diverse. This meant that a significant number of

70 60 studies were excluded when each specific outcome measure was pooled across studies. This would have led to bias in our review if the (unreported) results for excluded studies were systematically different from the results of studies that reported an outcome. Recently, the OMERACT (Outcome Measures in Rheumatoid Arthritis Clinical Trials) initiative 87 and the American College of Rheumatology (ACR) and the European League Against Rheumatism (EULAR) 199. provided recommendations on how to report disease activity in clinical trials of RA. These recommendations will harmonize the presentations of results from clinical trials and facilitate the comparison of outcomes across different trials and pooling of trial results. Although tests of heterogeneity were not statistically significant in this review, many types of relevant clinical heterogeneity were found, such as population, intervention (dose), and trial duration. The variations may be obscured and difficult to statistically identify due to the low power of heterogeneity tests and inadequate sample size. Since clinical heterogeneity was identified and incorporate in the analysis, the conclusions drawn in this review are more specific and clinically relevant. This also enhances the generalizability of this review as this review provides the information related to different clinical settings seen in clinical practice including patients who never receive any DMARD treatments, patients who inadequately responded to MTX or non-mtx DMARDs. These patients may differentially respond to DMARD treatment. Patients who inadequately respond to MTX or non-mtx DMARDs are less likely to respond to the next DMARD treatment compared to DMARD-naïve patients. Our findings suggest that clinical heterogeneity should be explored and taken into account when analyzing data and interpreting the results of reviews, even when tests of heterogeneity are not statistically significant at traditional levels 121. Comparisons between DMARDs or DMARD combinations in RA trials may be influenced by various factors, including the dose of DMARDs, the type of DMARD combinations, and trial duration. However, the patient s DMARD status also influences the

71 61 response to treatments and should be taken into account when analyzing and interpreting the results of RA trials and systematic reviews. Developing a comprehensive search strategy is a very important step in conducting a systematic review. Choices of databases and database providers are also important as searching only one database is clearly insufficient. Pubmed is one of the most common database providers used in systematic reviews, as it provides free access to MEDLINE and other additional citations; however, its performance has not been assessed. This study suggests that searching Pubmed yields additional relevant articles in the specific topic of RA trials compared to searching Ovid (MEDLINE). Furthermore, this study demonstrated that searching EMBASE in addition to Pubmed also yielded additional articles relevant to specific topics in RA. This finding was demonstrated in systematic reviews of RCTs as well as of cohort and case control studies in the RA field. However, the performance of Pubmed should be confirmed in other fields of study as well as in systematic reviews of observational studies. A systematic review is a type of scientific research that provides the qualitative and/or quantitative summary of research evidence relating to a specific clinical question. Despite the laudable attempts to achieve objectivity in developing scientific evidences, considerable subjective judgment is necessary in carry out systematic reviews. These judgments relate to the relevance of studies, the quality of eligible studies, the approach to heterogeneity, the method to incorporate heterogeneity in the analysis, as well as drawing unbiased conclusions. Systematic reviews will be useful and valuable scientific evidence only when they are developed using rigorous and explicit methodology, and their results and conclusions are unbiased and relevant to current clinical practices.

72 62 TABLES Table 1 Excluded studies and the reasons for exclusion Study Calguneri 1999 Clegg 1997 Haagsma 1995 Kremer 2004 Maillefert 2003 Matucci-Cerinic 2003 Mottaghi 2005 Mroczkowski 1999 Nagashima 2006 Nisar 1994 O'Dell 1996 Rou 1998 Stein 1997 Trnavsky 1993 Willkins 1996 Krause D et al (German) 1998 Geokoop-Ruiterman YPM 2005 Geokoop-Ruiterman YPM 2007 Mottonen T et al 1999 Reason for exclusion no MTX monotherapy arm (data combined with sulfazalazine and Hydroxychloroquine monotherapy) no outcome of interest summary of Haagsma et al. British Journal of Rheumatology 1994;33: (included in this review) open-label extension of randomized controlled trial open-label extension of randomized controlled trial summary of Kremer et al. Annals of Internal Medicine 2002;137: (included in this review) non randomized controlled trial open-label extension of randomized controlled trial non randomized controlled trial non randomized controlled trial open-label extension of randomized controlled trial non randomized controlled trial open-label extension of randomized controlled trial no MTX monotherapy arm published in the journal supplements and the key data has been reported in Willkins 1995 (included in this review) non randomized controlled trial no MTX monotherapy arm no MTX monotherapy arm no MTX monotherapy arm Abbreviation: - MTX = methotrexate

73 63 Table 2A Characteristics of the included studies Study Sample size Study duration Strategy MTX (mg/wk) DMARD Quality Rating# (Month) Mono Combo (0-11) Haagsma DMARD SSZ 3 g/d N Dougados DMARD- Up to Up to 15 SSZ 3 g/d 7 86 N 15 Marchesoni DMARD CSA N 2.5 mg/kg/d Tascioglu DMARD SSZ 2 g/d 4 97 N Hetland DMARD CSA 8 N 2.5 mg/kg/d O Dell DMARD Doxycycline 7 N 20 or 100 mg twice/d Tugwell MTX-IR CSA mg/kg/d Kremer MTX-IR LEF 20 mg/d 10 Lehman MTX-IR im Gold mg/wk Jarette MTX-IR Zolendronic acid 7 5 mg iv twice Ogrendik MTX-IR Levofloxacin mg/d Willkins Non AZA 8 82, MTX-IR mg/d Ferraz Non CQ 250 mg/d 9 MTX-IR Haagsma Non SSZ 2 g/d 7 85 MTX-IR O Dell Non- Up to Up to SSZ 1 g/d 8 MTX-IR HCQ 400 mg/d Hanyu Non Previous 3 MTX-IR DMARD (Penicillamine, Bucillamine, im Gold) Ichikawa Non- 8 8 Bucillamine MTX-IR 200 mg/d Capell Non SSZ 2 g/d 9 MTX-IR Islam Not clear SSZ 2 g/d 3 Abbreviation: - DMARD-N = DMARD naïve patients, MTX-IR = MTX inadequate response patients, Non-MTX-IR = Non-MTX DMARDs inadequate response patients, # Van Tulder s scale, MTX = methotrexate, SSZ = sulfazalazine, HCQ =hydroxychloroquine, CQ = chloroquine, LEF= leflunomide, CSA =cyclosporin, AZA = azathioprine, im Gold = intramuscular gold

74 64 Table 2B Detailed characteristics of the included studies Study Study design Participants Intervention Outcomes measured Haagsma 1997 Randomized controlled trial, double blind Design:- Parallel Sample size :- 36 (MTX+SSZ), 35 (MTX+placebo), 34(SSZ) Trial duration:- 52 weeks Analysis:- Intention-totreat, Last observation carried forward for missing data Active RA (DAS >3) with disease duration less than 1 year and DMARDS naive Mean age (SD) :- 57(12.2) yrs MTX+SSZ, 59.4(13.2) yrs MTX Female :- 67% MTX+SSZ, 66% MTX Disease duration (SD) :- 2.6(1.4) months- MTX+SSZ, 3(2.3) months- MTX Rheumatoid factor + :-94% MTX+SSZ, 94% MTX Mean dose of MTX:- 7.5 mg/wk MTX+SSZ, 15 mg/wk MTX Concomitant treatment - no data SSZ 3g/d+MTX 7.5 mg/wk vs. MTX 15 mg/wk+placebo vs. SSZ 3 g/d +placebo Ritchie articular index Swollen joint Tender joint Pain (0-100 VAS) Patient global assessment (0-100 VAS) HAQ ESR Dougados 1999 Merchesoni 2003 Randomized controlled trial, double blind Design :- Paralell Sample size :- 68 (MTX+SSZ), 69 (MTX+placebo) Trial duration:- 52 weeks Analysis:- Intention-totreat, Last observation carried forward for missing data Randomized controlled trial, single blind(the clinical investigator) Design :- Paralell Sample size :- 30 (MTX+CSA), 30 (MTX) RA with less than 1 year of disease duration Active disease (DAS>3) DMARDS and steroid naive Mean age (SEM) :- 52(2) yrs MTX+SSZ, 50(2) yrs MTX Female :- 77% MTX+SSZ, 74% MTX Disease duration (SEM) :- 10.6(1) months- MTX+SSZ, 18.4(5.2) months- MTX Rheumatoid factor + :- 71% MTX+SSZ, 62% MTX Concomitant treatment - no data Active RA, DMARDS naïve Mean age (SD) :- 46.6(10.5) yrs MTX+CSA, 49.3(10.2) yrs MTX Female :- 93% MTX+CSA, 90% MTX Disease duration (SD) :- 0.9(0.7) SSZ 3 g/d+mtx 15 mg/wk vs. MTX, SSZ 3 g/d+placebo CSA up to 4 mg/kg/d+mtx up to 20mg/wk im vs.mtx up to 20 mg/wk ACR EULAR response Tender joint Swollen joint Pain (0-100 VAS) Patient global assessment (1-5 Likert scale) Physician global assessment (1-5 Likert scale) HAQ Sharp's score ACR HAQ Sharp/vDH

75 65 Trial duration:- 12 months Analysis:- Intention-totreat, Last observation carried forward for missing data yrs- MTX+CSA, 0.9(0.7) yrs- MTX Rheumatoid factor + 36% MTX+CSA, 74% MTX Mean dose of MTX (SD):- 9.5 (1.7) mg/wk in CSA+MTX, 11.2(3.4) mg/wk in MTX Mean dose of CSA (SD):- 2.5 (0.6) mg/kg/d Concomitant treatment : no more than 10 mg/d of prednisolone and NSAIDS was permitted Taschioglu 2003 Hetland 2006 Randomized controlled trial, open label Design:- Paralell Sample size :- 35 (MTX+SSZ), 35 (MTX) analysed only completers:- 27(MTX+SSZ) + 28 (MTX) Trial duration:- 12 months Analysis of missing data or withdrawals was not described Randomized controlled trial, double blind Design :- Paralell Sample size :- 80 (MTX+CSA), 80 (MTX +placebo) Trial duration:- 52 weeks Active (DAS>3) RA with less than 1 year of disease duration, DMARDS and steroid naive Mean age (SD) : (6.46) yrs MTX+SSZ, 45.16(5.93) yrs MTX Female :- 81% MTX+SSZ, 86% MTX Disease duration (SD) :- 6.9(6.35) months- MTX+SSZ, 7.48(4.03) months- MTX Rheumatoid factor + :- 85% MTX+SSZ, 88% MTX Mean dose of MTX:- 7.5 mg/wk both groups Concomitant treatment - NSAIDS were allowed during the study period Active RA with less than 6 months' duration and DMARDS naive Median age (IQR) :- 53.2( ) yrs MTX+CSA, 51( ) yrs MTX Female :- 64% MTX+CSA, 70% MTX 7.5 mg/kg + SSZ 2g/d vs. MTX 7.5 mg/wk MTX 20 mg/wk+csa 2.5 mg/kg/d vs. MTX 20 mg/wk Intra-articular injection of betamethasone was given in all swollen joints (maximum 4 joints or 4 ml per visit) at weeks 0,2, 4, 6, 8 and every 4 Swollen joint Ritchie articular index Pain (0-100 VAS) HAQ ESR,CRP ACR DAS28 HAQ Larsen's score

76 66 O Dell 2006 Analysis:- Intention-totreat, Last observation carried forward for missing data Randomized controlled trial, double blind Design :- Paralell Sample size :- 18 (Doxycycline 20 mg twice daily+mtx), 24 Doxycycline 100 mg twice daily+mtx and 24 MTX+placebo Trial duration:- 2 years Analysis:-Intention-totreat MTX Disease duration, median (IQR) :- 3.2( ) months MTX+CSA, 3.9( ) months MTX Rheumatoid factor + :- 70% MTX+CSA, 59% MTX Mean dosage of MTX : mg/wk MTX+CSA, 15 mg/wk MTX Concomitant treatment :- nonsteroidal anti-inflammatory drugs 60% MTX+CSA, 66% MTX. Active RA with 6 weeks to less than 1 year of disease duration and had positive rheumatoid factor DMARD naive Mean age (range) :- 49.5(35-47) yrs doxy 100 mg+mtx, 49.9(27-74) yrs doxy 20 mg+mtx, 55.7(41-47) yrs MTX+placebo Female :- 67% doxy 100 mg+mtx, 89% doxy 20 mg+mtx and 70% MTX Disease duration (SD) :- 5(3.1) months-doxy 100 mg+mtx, 5.4(2.9) doxy 20 mg+mtx, 4.8(2.7) months- MTX Rheumatoid factor + :- 100% all three groups Mean dose of MTX:- no data Concomittant treatment - NSAIDS and prednisolone less than 7.5 mg/d were permitted. A total 2 intra-articular steroid injection were allowed but not within 6 weeks prior to the weeks thereafter up to week 52 Doxycycline 20 mg twice daily + MTX mg/wk Doxycycline 100 mg twice daily + MTX mg/wk Placebo + MTX mg/wk Tender joint Swollen joint Patient global assessment (0-10 VAS) Physician global assessment (0-10 VAS) HAQ ESR

77 67 Tugwell 1995 Randomized controlled trial, double blind Design:- Step-up Sample size :- 75 (MTX+CSA), 73 (MTX+placebo) Trial duration:- 6 months Analysis:- Intention-totreat, Last observation carried forward for missing data evaluation period RA who had partial response to MTX 15 mg/wk at stable dose for at least 3 months Mean age (SD) :- 55.4(12.9) yrs MTX+CSA, 54.3(14.5) yrs MTX Female :- 72% MTX+CSA, 73% MTX Mean dose of MTX (SD):- equal or less than 15 mg/wk Mean dose of CSA (SD):- 2.97(1.02) mg/kg/d Disease duration (SD) :- 11.2(8.3) months- MTX+CSA, 9.4(7.8) months- MTX Concomitant treatment - no more than 10 mg/d of prednisolone was permitted CSA 2.5 mg/kg/d - 5mg/kg/d+ previous dose of MTX vs. previous dose of MTX Tender joint Swollen joint Pain (0-100 VAS) Patient global assessment (0-10 VAS) Physician global assessment (0-10 VAS) ACR HAQ ESR Kremer 2002 Randomized, doubleblind, placebo-control trial Design :- Step-up Sample size at entry : MTX+Placebo -133, MTX+Leflunimide -133 Trial duration : 24 wk Analysis:- Intention-totreat Active RA despite taking stable dose of MTX mg/wk for at least 6 months Mean age (SD) :- 55.6(11.7) LEF, 56.6(11.37) placebo Female: - 99(76.2%) LEF, 107(81%) placebo Disease duration :- 10.5(8.35) y LEF, 12.7(9.56) y placebo Positive rheumatoid factors: - 99(79%) LEF, 113(88%) placebo Mean dose of MTX(SD):- 16.8(2.7) MTX+LEF, 16.1(2.9) MTX Concomitant treatment:- systemic steroid: - 77(59.2%) LEF, 86(64.7%) placebo LEF 20 mg/d + previous dose of MTX vs. Placebo + previous dose of MTX Tender joint Swollen joint Pain (0-100 VAS) Patient global assessment (0-100 VAS) Physician global assessment (0-100 VAS) HAQ ESR CRP

78 68 Lehman 2005 Randomized controlled trial, double blind Design :- Step-up Sample size :- 38 (MTX+ intramuscular gold), 27 (MTX+placebo) Trial duration:- 48 weeks Analysis:- Intention-totreat RA with disease duration more than 4 months and less than 10 years suboptimal response at least 12 wk of MTX equal or more than 15 mg/wk Mean age (SD) :- 51(11) yrs MTX+gold, 54(13) yrs MTX+Placebo Female :- 84% MTX+gold, 78% MTX Disease duration (SD) :- 3.4(2.5) yrs MTX+gold, 2.8(2.7) yrs MTX Rheumatoid factor + :- 67% MTX+gold, 63% MTX Mean dosage of MTX (SD) :- 18(5.1) mg/wk MTX+ gold, 18(6.5) mg/wk MTX Concomitant treatment :- nonsteroidal anti-inflammatory drugs:- 92% MTX+gold, 89% MTX systemic steroid:- 18% MTX+gold, 52% MTX - MTX, folic acid and prednisolone (up to 10 mg/d) had to have been received at a stable dosage for 4 weeks - stable dose of previous of MTX+Gold mg/wk or as tolerated vs. Previous dose of MTX +placebo Tender joint Swollen joint Pain (0-100 VAS) Patient global assessment (0-100 VAS) Physician global assessment (0-100 VAS) HAQ ESR ACR Jarette 2006 Randomized controlled trial, double blind Design :- Step-up Sample size :- 18 (MTX+Zoledronic acid), 21 (MTX+placebo) Trial duration:- 26 weeks Analysis:- Intention-totreat RA symptom for less than 2 years active disease and clinical synovitis in at least the wrist or hand joints were on MTX 4 weeks before randomization Mean age (range) :- 50.2(30-76) yrs MTX+Zoledronic acid, 53.5(33-72) yrs MTX+placebo Female :- 55.6% MTX+Zolindronic acid, 57.1% MTX+placebo Disease duration (range) :- 5.6 ( ) months Zoledronic acid 5 mg infusion at baseline and 13 week +MTX mg/wk vs. MTX +placebo infusion Tender joint Swollen joint Pain (0-100 VAS) Patient global assessment (0-100 VAS) Physician global assessment (0-100 VAS) MRI of the second to fifth proximal interphalangeal and metacarpophalageal joints and of wrist

79 69 MTX+Zolindronic acid, 5.5 ( ) months MTX Rheumatoid factor + :- 83% MTX+Zolindronic acid, 71% MTX Mean dosage of MTX :- 14 mg/wk MTX+ Zolindronic acid, 11.9 mg/wk MTX Concomitant treatment:- nonsteroidal anti-inflammatory drugs:- 94% MTX+Zolindronica acid, 100% MTX Intra-articular or intra-muscular steroid was not permitted Ogrendik 2007 Randomized controlled trial, double blind Design:- Step-up Sample size :- 38 (MTX+levofloxacin), 38 (MTX+placebo) Trial duration:- 6 months Analysis :- Intention-totreat, Last observation carried forward for missing data Active RA Mean age (SD) :- 51(9) yrs MTX+levofloxacin, 49(10) yrs MTX+Placebo Female :- 71% MTX+ levofloxacin, 74% MTX Disease duration (SD) :- 13(9) yrs MTX+ levofloxacin, 12(8) yrs MTX Rheumatoid factor + :- 84% MTX+ levofloxacin, 74% MTX Mean dosage of MTX (SD) :- 15(8.3) mg/wk MTX+ levofloxacin, 17.5(9.1) mg/wk MTX Concomitant treatment:- systemic steroid:- 63% MTX+ levofloxacin, 68% MTX Before receiving the study drugs, all patients had been taking MTX mg/wk for at least 6 months and then Levofloxacin 500 mg/d + previous dose of MTX vs. Placebo + previous dose of MTX Tender joint Swollen joint Pain (0-10 VAS) Patient global assessment (0-10 VAS) Physician global assessment (0-10 VAS) HAQ, ESR,CRP Willkins 1992 Randomized controlled trial, double-blind Design:- Step-up Sample size :- 69( RA with inadequate disease control or toxic response to treatment with injectable gold (at least 750 mg total dose), Level I :- MTX 5 mg/week + AZA 50 mg/day vs MTX 5 mg/week Level II:- MTX 7.5 mg/week Tender joint Swollen joint Patient global assessment (1-4)

80 70 Willkins 1995 Ferraz 1994 MTX+AZA), 67(MTX+placebo) Trial duration:- 24 weeks Analysis:- Intention-totreat Continue to the Willikins 1992's study and cross over design at week 24 and were followed up in an open protocol for the ensuring 24 weeks. For those patients who continued with the initial regimen, were followed up in double-blind manner. Analysis :- Intention-totreat analysis and patients in whom therapy was terminated at any time because of toxicity or lack of efficacy and those who crossed over at week 24 were considered nonresponders at week 48 Randomized controlled trial, double blind Design :- Paralell auranofin(6 months of 6 mg daily dose), penicillamine (500 mg daily dose for 3 months) Mean age (range) :- 56(29-79) yrs MTX+AZA, 54(21-85) yrs MTX Female :- 82% MTX+AZA, 74% MTX Disease duration (range) :- 8(1-54) yrs- MTX+AZA, 10(1-40) yrs- MTX Median dose of MTX and AZA:- no data Concomitant treatment - no data Median dose of MTX:- 7.5 mg/wk in both groups at week 48 Median dose of AZA:- 75 mg/d Active disease RA and had failed to respond to NSAIDS and at least one DMARD + AZA 100 mg/day vs MTX 7.5 mg/week Level III:- MTX 15 mg/week + AZA 150 mg/day vs MTX 15 mg/week All subjects entered the study at dosage level I and dosage increase were instituted at week 6, 12 and or 18 in nonresponsive patients. Non-responders from Willkins 1992 's study were invited to cross to one of the other treatment regimens (same protocol) MTX 7.5 mg/week + CQ 250 mg/day vs MTX 7.5 mg/week + placebo Physician global assessment (1-4) HAQ The primary response was a clinical response of more than 30% improvement from baseline in at least 3 of these parameter at week 24. Same as Willikins 1992 and radiograph of hands and wrists Tender joint Swollen joint Pain (0-10 VAS)

81 71 Haagsma 1994 O Dell 1996 Sample size :- 41 (MTX+CQ), 41 (MTX+Placebo) Trial duration:- 6 months Analysis of missing data or withdrawals was not described Randomized controlled trial, open label Design :- Step-up Sample size :- 22 (MTX+SSZ), 18 (MTX) Trial duration:- 24 weeks Analysis:- Intention-totreat, Last observation carried forward for missing data Randomized controlled trial, double blind had not used any DMARDS in the past 2 months, were on stable dose (up to 7.5 mg/d) of prednisolone and NSAIDS for at least 4 weeks Mean age (SD) :- 49.7(13.9) yrs MTX+CQ, 43.6(11.9) yrs MTX Female :- 82% MTX+CQ, 85% MTX Disease duration (SD) :- 9.24(7.94) yrs- MTX+CQ, 6.19(4.74) yrs- MTX Rheumatoid factor + :-24% MTX+CQ, 26% MTX Mean dose of MTX:- 7.5 mg/wk both groups Concomitant treatment - systemic steroid 24% MTX+CQ, 23% MTX RA patients who had an insufficient response to SSZ according to their treating physician were considered for selection. Mean age (SD) :- 59.3(12.3) yrs MTX+SSZ, 51.8(13.9) yrs MTX Female :- 82% MTX+SSZ, 78% MTX Disease duration (SD) :- 4.7(4.2) yrs- MTX+SSZ, 5.3(4.2) yrs- MTX Rheumatoid factor + :-77% MTX+SSZ, 72% MTX Mean dose of MTX:- 7.9 mg/wk MTX+SSZ, 8.3 mg/wk MTX Concomitant treatment - no data Active RA with more than 6 months of disease duration and MTX 15 mg/week + SSZ 2 g/day vs. MTX 15 mg/week MTX 17.5mg/wk+SSZ 2 g/d+ HCQ 400mg/d vs. MTX HAQ ESR Ritchie articular index Swollen joint Pain (0-100 VAS) DAS ESR Tender joint Swollen joint

82 72 Hanyu 1999 Design:- Paralell Sample size :- 31(MTX+SSZ+HCQ), 36 (MTX+placebo) Trial duration:- 12 months Analysis:- Intention-totreat and patients who withdrew were considered to have had treatment failure Randomized controlled trial, open label Design :- Step-up Sample size :- 19 (MTX+previuos DMARDS), 18 (MTX) Trial duration:- 5 years Analysis of missing data or withdrawals was not described poor response to treatment with at least one of the following :gold, HCQ, Penicillamine, SSZ,MTX Mean age (range) :- 50(27-67) yrs MTX+SSZ+HCQ, 50(21-69) yrs MTX+placebo Female :- 66% MTX+SSZ+HCQ, 70% MTX Disease duration (SD) :- 10(10) months- MTX+SSZ+HCQ, 10(8) months- MTX Rheumatoid factor + :- 84% MTX+SSZ+HCQ, 89% MTX Mean dose of MTX:- no data Concomitant treatment - syetemic steroid of equal or less than 10 mg/d 52% in MTX+SSZ+HCQ and 53% in MTX and NSAIDS were permitted Active RA and insufficient response to treatment with gold, D-penicillamine, bucillamine or sulfasalazine. Mean age (SD) :- 55.6(2.2) yrs MTX+ previous DMARDS, 54(2.9) yrs MTX Female :- 79% MTX+previous DMARDS, 83% MTX Disease duration (SD) :- 13.1(2.1) yrs- MTX+previous DMARDS, 13.8(1.5) yrs- MTX Rheumatoid factor + :-89% MTX + previous DMARDS, 94% MTX Mean dose of MTX:- 5 mg/wk MTX combo, 7.5 mg/wk MTX Concomitant treatment mg/wk+placebo Patient global assessment (0-10 VAS) Physician global assessment (0-10 VAS) ESR MTX 5 mg/wk+ previous DMARDS(SSZ, Penicillamine, Bucillamine, Gold) vs. MTX 7.5 mg/wk Joint score Lansbury index (calculated from duration of morning stiffness, grip strength, ESR, joint score) grip strength morning stiffness ESR CRP

83 73 Ichikawa 2005 Capell 2007 Randomized controlled trial, double blind Design :- Paralell Sample size :- 24 (MTX+Bucillamine), 41 (MTX+Placebo) Trial duration:- 96 weeks Analysis:- Intention-totreat Randomised controlled trial, double-blind Design :- Step-up Sample size :- 687 in phase I and 56 (MTX+SSZ), 55(SSZ+placebo) and 54(MTX+placebo) in phase II Trial duration:- 18 months Analysis:- Intention-totreat, Last observation carried forward for missing data systemic steroid 74% MTX+previous DMARDS, 78% MTX Active RA, MTX or Bucillamine(BUC) naive, never received prednisolone more than 7.5 mg/day Mean age (SD) :- 49.2(13.9) yrs MTX+BUC, 52.7(9.3) yrs MTX Female :- 83.3% MTX+BUC, 69.6% MTX Disease duration (SD) :- 10.6(6.6) yrs- MTX+BUC 8.2(4.8) yrs- MTX Rheumatoid factor + :-96% MTX+BUC, 83% MTX Mean dose of MTX:- 8 mg/wk both groups Concomitant treatment - systemic steroid 17% MTX+BUC, 35% MTX RA with less than 10 years of disease duration Active RA ( defined by DAS>2.4) Mean age (range) :- 56(30-78) yrs MTX+SSZ, 53(34-79) yrs MTX Female :- 75% MTX+SSZ, 79% MTX Disease duration (range) :- 1(1-9) yrs MTX+SSZ, 1(1-9) yrs MTX Rheumatoid factor + :- 68% MTX+SSZ, 65% MTX Median dose of MTX: mg/wk MTX+SSZ, 15 mg/wk MTX Concomitant non-steroidal anti- MTX 8 mg/wk +Bucillamine 200 mg/d vs MTX 8 mg/wk Phase I:- All patients recieved SSZ 40 mg/kg/d or 4 g/d maximum for the initial 6 months and then patients who had DAS equal or more than 2.4 were enrolled into phase II Phase II:- Patients were randomly assigned to one of three groups including MTX 25 mg/wk+ssz 2g/d MTX mg/wk + placebo SSZ 40 mg/kg/d or 4 g/d maximum+placebo Tender joint Swollen joint Pain (0-100 VAS) Patient global assessment (0-100 VAS) Physician global assessment (0-100 VAS) HAQ ESR, CRP ACR EULAR response DAS Ritchie articular index Swollen joint Pain (0-100 VAS) Patient global assessment (0-100 VAS) Physician global assessment (0-100 VAS) ESR, CRP HAQ Sharp/vDH's score,

84 74 inflammatory drugs and other drugs were continued. Intraarticular or intra-muscular steroid was permitted but not within 1 months of assessment period Islam 2000 Randomized controlled trial, open label Design :- Parallel Sample size :- 27 (MTX+SSZ), 27 (MTX) Trial duration:- 6 months Analysis:- analysed only completers 19 (MTX+SSZ), 23 (MTX) RA with less than 3 months of disease duration Mean age (SD) : (11.08) yrs MTX+SSZ, 32.35(14.79) yrs MTX Female :- 79% MTX+SSZ, 83% MTX Mean dose of MTX:- no data Concomitant treatment - no data SSZ 2 g/d+mtx mg/wk vs. MTX mg/wk Tender joint Swollen joint Patient global assessment (0-10 VAS) Physician global assessment (0-10 VAS) Functional class (I-IV) ESR Abbreviation: RA= Rheumatoid arthritis, DAS = Disease Activity Score, HAQ = Health Assessment Questionnaire, ESR = erythrocyte sedimentation rate, ACR = American College of Rheumatology, EULAR = European League Against Rheumatism, SD = standard deviation, VAS =visual analogue scale, SEM = standard error of mean, IQR = interquatile range, MTX = methotrexate, SSZ = sulfazalazine, HCQ =hydroxychloroquine, CQ = chloroquine, LEF= leflunomide, CSA =cyclosporin, AZA = azathioprine, im Gold = intramuscular gold

85 75 Table 3 Quality assessment of the included studies Study Total score (0-11)* Adequate randomization Blinded patients Blinded care provider Blinded outcome assessor Treatment allocation concealed Acceptable drop-out rate Acceptable compliance Similarity of baseline characteristics between groups Cointervention similar or avoid Haagsma Yes Yes Yes Unclear Yes Yes Yes Yes Yes Yes Yes Dougadous Unclear Yes Yes Unclear Unclear Yes Yes Yes Unclear Yes Yes Marchesoni Yes No Yes No Yes Yes Unclear Yes Yes Yes Yes Tascioglu Unclear Unclear Unclear Yes No Yes Unclear Yes Unclear No Yes Hetland Yes Yes Yes No Yes Yes Unclear Yes No Yes Yes O' Dell Yes Yes Yes Unclear Yes No Unclear Yes Unclear Yes Yes Tugwell Unclear Yes Yes Yes Yes No Unclear Yes Unclear Yes Yes Kremer Yes Yes Yes Unclear Yes Yes Yes Yes Yes Yes Yes Lehman Yes Yes Yes Yes Yes Yes Unclear Yes No Yes Yes Jarette Unclear Yes Yes Unclear Unclear Yes Yes Yes No Yes Yes Ogrendik Unclear Yes Yes Unclear Unclear Yes Yes Yes Yes Yes Yes Willikins Yes Yes Yes Unclear Yes No Unclear Yes Yes Yes Yes Ferraz Yes Yes Yes Yes No Yes Yes Yes Yes No Yes Haagsma Yes No No No No Yes Yes Yes Yes Yes Yes O'Dell Yes Yes Yes Yes Yes No Unclear Yes No Yes Yes Hanyu Unclear No No No No No Unclear Yes Yes Unclear Yes Ichikawa Unclear Yes Yes No Unclear Yes Unclear Yes No Yes Yes Capell Yes Yes Yes Yes Yes No Yes Yes No Yes Yes Islam Unclear Unclear Unclear Unclear Unclear Yes Unclear Yes Unclear No Yes ITT ** analysis used Similarity of the timing of outcome assessment * Van Tulder s scale, ** ITT = Intention-to-treat analysis

86 76 Table 4 Summary of the results of efficacy (dichotomous data) comparing MTX combination to monotherapy DMARD naïve RR (95%CI) ACR ( ) ACR ( ) ACR * ( ) ACR remission 1.27 ( ) EULAR good 0.97 response (0.69,1.37) EULAR moderate 1.37 response (0.81,2.33) EULAR remission 1.26 (0.84,1.88) Withdrawal due to 0.63 lack of efficacy (0.34,1.17) MTX inadequate response RR (95%CI) 2.51* ( ) 4.54* ( ) 5.59* ( ) No data Non-MTX inadequate response RR (95%CI) 1.85* ( ) 1.69 ( ) 1.93 ( ) No data No data 3.38 (0.73,15.53) No data No data No data 3.86 (0.45,33.42) 0.42* 0.37* ( ) ( ) * Statistically significant Abbreviation: - RR = relative risk, 95%CI = 95% confidence interval, MTX = methotrexate, DMARD = Disease modifying anti-rheumatic drug, ACR = American College of Rheumatology, EULAR = European League Against Rheumatism

87 77 Table 5 Summary of the results of efficacy (continuous data) comparing MTX combination to monotherapy DMARD naïve WMD or SMD (95%CI) Swollen joint (-2.7,0.6) Tender joint (-0.64, 0.29) Pain (VAS 1-10 cm.) (-5.12, 2.4) Patient global 0.7 assessment (VAS 1-10 cm.) (-10.24, 11.64) MTX inadequate response WMD or SMD (95%CI) -0.45* (-0.63, -0.27) -0.51* (-0.69, -0.33) -9.72* (-14.7, -4.75) -8.15* (-14.52, -1.79) Physician global assessment (VAS 1-10 cm.) * (-18.98,-2.84) ESR (-6.98,3.74) (-11.47,10.41) CRP * (-0.38,0.56) (-0.62,-0.13) HAQ 0.1* -0.28* (0.09,0.11) (-0.36, -0.21) Modified Sharp s score -3.15* (-5.85, -0.45) -1.4 (-2.81, 0.01) Non-MTX inadequate response WMD or SMD (95%CI) (-1.62, -0.3) -0.51* (-0.69, -0.33) (-24.99, 13.02) -10* (-19.6, -0.4) -10* (-14.8,-5.2) (-10.72, 2.13) (-1.09, 0.21) (-0.48, 0.14) - * Statistically significant Abbreviation: - WMD = weighted mean difference, SMD = standardized mean difference, 95% CI = 95% confidence interval, MTX = methotrexate, DMARD = Disease modifying anti-rheumatic drug, VAS = visual analogue scale, ESR = erythrocyte sedimentation rate, CRP = C-reactive protein, HAQ = Health Assessment Questionnaire

88 78 Table 6 Summary of the results of toxicity (dichotomous data) comparing MTX combination to monotherapy Total adverse event MTX+SSZ MTX+LEF MTX+CSA MTX+AZA MTX+im Gold Gastrointestinal adverse events MTX+SSZ MTX+LEF MTX+CSA MTX +im Gold Abnormal liver function test MTX+SSZ MTX+LEF MTX+CSA MTX+BUC Mucositis MTX+SSZ MTX +im Gold Hematological adverse events MTX+SSZ MTX +im Gold MTX+BUC Infection MTX+SSZ MTX+LEF MTX +im Gold Withdrawal due to adverse events MTX+SSZ MTX+SSZ+HCQ MTX+CQ MTX+CSA MTX+AZA MTX+LEF MTX +im Gold MTX+Antibiotics MTX +BUC MTX+ miscellaneous RR (95%CI) (Random effects) 1.13 ( ) 1 ( ) 3.62 ( ) 1.67 ( ) * 2.61 ( )* 1.75 ( )* 1.67 ( )* 4.13 ( ) 0.71 ( ) 1.77 ( ) 4.3 ( )* 3.1 ( ) 3 ( ) 0.62 ( ) 9.33 ( ) 2.36 ( ) 1.42 ( ) 0.32 ( ) 1.35 ( ) 0.79 ( ) 1.6 ( ) 1.19 ( ) 0.5 ( ) 3 ( ) 1.88 ( )* 5.18 ( )* 1.82 ( ) 2.84 ( ) 1.9 ( ) 2.88 ( ) 1.42 ( ) * Statistically significant Abbreviation: - MTX = methotrexate, SSZ = sulfazalazine, HCQ =hydroxychloroquine, CQ = chloroquine, LEF= leflunomide, CSA =cyclosporin, AZA = azathioprine, im Gold = intramuscular gold, BUC = bucillamine, RR = relative risk, 95%CI = 95% confidence interval

89 79 Table 7 Real and artifactual causes of between-study variation in effect Patient Intervention Co-intervention Outcome Real Disease severity Age Co-morbidity Time Duration Dose Drug Therapy Timing of outcome Event type Artifactual Improper randomization Differential follow-up (noncomparable groups) Non-compliance Cross-over Undetected co-intervention Differential and non-differential measurement error

90 80 Table 8 Potential sources of heterogeneity Study Characteristics Patient Potential source of heterogeneity Treatment received prior to randomization Subgroup DMARD naïve MTX inadequate responder Non-MTX DMARD inadequate responder Intervention Different DMARD combination MTX+ SSZ (6 studies) MTX+ SSZ+HCQ (1 study) MTX +Leflunomide (1 study) MTX + CSA (3 studies) MTX + CQ (1 study) MTX + AZA (1 study) MTX + Doxycycline (1 study) MTX +Levofloxacin (1 study) MTX + Bucillamine (1 study) MTX + im Gold (1 study) MTX + Zolindronic acid (1 study) MTX+ Previous DMARDs (Penicillamine, Bucillamine and in Gold) (1 study) Different dose of MTX and other MTX dose mg/wk DMARD SSZ dose 1-3 g/d CSA dose mg/kg/d Outcome Timing of outcome assessment 6, 12, 24, and 60 months Outcome measure used Single variable outcome, e.g., Number of swollen joint count and tender joint count, pain score, patient or physician global assessment of disease activity, ESR, and CRP Composite outcome, e.g., ACR response, EULAR response, DAS, and DAS28 Study Quality of study (overall) High versus low characteristics Criterion for assessing the quality of study Met versus not met each criterion Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrextae, SSZ = sulfazalazine, HCQ =hydroxychloroquine, CQ = chloroquine, LEF= leflunomide, CSA =cyclosporin, AZA = azathioprine, im Gold = intramuscular gold, BUC = bucillamine, ESR = erythrocyte sedimentation rate, CRP = C-reactive protein, ACR = American College of Rheumatology EULAR = European League Against Rheumatism, DAS =Disease activity score

91 81 Table 9 Tests of heterogeneity using forest plots and eyeball test, Cochran s Q test, and I 2 test for withdrawal due to lack of efficacy and toxicity, withdrawal due to lack of efficacy, and withdrawal due to toxicity Outcome Withdrawal due to Withdrawal due to Withdrawal due to lack of efficacy and lack of efficacy toxicity toxicity Forest plot Eyeball Moderately Highly overlapped Highly overlapped overlapped Cochran s Q test χ 2 = 20.98, df = 12, p = 0.05 χ 2 = 9.52, df = 11 p = 0.57 χ 2 = 15.13, df = 16, p = 0.52 I % 0% 0% Abbreviation: - χ 2 = chi-square, df = degree of freedom, RR = relative risk, 95%CI = 95% confidence interval

92 82 Table 10 Outcome measures for efficacy used in included studies Study ACR EULAR DAS DAS 28 Haagsma 1994 Haagsma 1997 Dougados 1999 Islam 2000 Tascioglu 2003 Withdra wal due to LOF RAI SJC TJC Pain (VAS) PGA (VAS) MDGA (VAS) X X X X X ESR CRP HAQ X-ray X X X X X X X X X X X X X X X X X X* X X X X X X X X X X X X Capell 2007 X X X X X X X X X X X X X X O Dell 1996 X X X X X X X X X X X X X X X X Lehman 2005 Ferraz 1994 X X X X X X X X X X X X X X X X Kremer 2002 Tugwell 1995 Marchesoni 2003 Hetland 2006 X X X X X X X X X X X X X X X X X X Willkins 1992 Willkins 1995 O Dell 2006 X X X X X X X X X X X X X X X X X Ogrendik 2007 Hanyu 1999 X X X Ichikawa 2005 X X Jarette 2005 X X X X X X Abbreviation: - ACR =American College of Rheumatology Response criteria, EULAR = European League Against Rheumatism response criteria, DAS =disease activity score, LOF = lack of efficacy, RAI =Ritchie s articular index, SJC =sswollen joint count, TJC = tender joint count, VAS Visual analogue scale, PGA = patient global assessment, MDGA = physician global assessment, ESR = erythrocyte sedimentation rate, CRP = C-reactive protein, HAQ = Health Assessment Questionnaire

93 83 Table 11 Toxicity reported in the included studies Study Total adverse events Gastrointestinal adverse events Abnormal liver function Mucositis Hematological adverse events Infection Withdrawal Due to adverse events Haagsma 1994 X X X X X Haagsma 1997 X X X X X X X Dougados 1999 X X X X X Islam 2000 X Tascioglu 2003 X X X X X X X Capell 2007 X O Dell 1996 X Lehman 2005 X X X X X X Ferraz 1994 X Kremer 2002 X X X X X Tugwell 1995 X Marchesoni X X X X 2003 Hetland 2006 X Willkins 1992 X Willkins 1995 X O Dell 2006 X Ogrendik 2007 X Hanyu 1999 X Ichikawa 2005 X X X Jarette 2005

94 84 Table 12 Subgroup analysis: - Study design and Population (Pervious DMARD use) Outcome Study design RR (95%CI) Withdrawal due to lack of efficacy Withdrawal due to toxicity Withdrawal due to lack of efficacy and toxicity DMARD naive 0.63 (0.34 to 1.17) MTX-IR 0.42 (0.21 to 0.84) Non MTX-IR 0.47 (0.16 to 0.87) DMARD naive 1.72 (1.04 to 2.83) MTX-IR 1.89 (1.05 to 3.41) Non MTX-IR 1.53 (0.74 to 3.18) DMARD naive 1.16 (0.7 to 1.93) MTX-IR 0.86 (0.49 to 1.51) Non MTX-IR 0.75 (0.41 to 1.35) P-value for comparison to MTX-IR Non-MTX- IR Abbreviation:- DMARD = disease modifying anti-rheumatic drug, MTX = methotrexate, MTX-IR = MTX inadequate responder, Non-MTX-IR = Non-MTX DMARD inadequate responder, RR = relative risk, 95%CI = 95% confidence interval

95 85 Table 13 Subgroup analysis: - Trial duration Outcome Withdrawal due to lack of efficacy Withdrawal due to toxicity Withdrawal due to lack of efficacy and toxicity RR, random effects (95%CI) Duration > Duration 6 6 months months (0.29 to 0.77) (0.21 to 0.94) (1 to 2.05) (1.09 to 3.6) (0.65 to 1.23) (0.39 to 1.99) Meta-regression analysis (Univariate analysis) Coefficient RR ratio P value (95%CI) (-0.88 to 0.97) (0.41 to 2.64) (-0.97 to 0.43) (0.38 to 1.53) (-0.78 to 0.98) (0.46 to 2.67) Abbreviation:- RR = relative risk, 95%CI = 95% confidence interval

96 86 Table 14 Subgroup analysis: - Study quality (high versus low) Using 50% met criteria Dependent variable Withdrawal due to lack of efficacy Withdrawal due to toxicity Withdrawal due to lack of efficacy and toxicity Dependent variable Withdrawal due to lack of efficacy Withdrawal due to toxicity RR, random effects (95%CI) High quality (score 6) 0.45 (0.3 to 0.67) 1.61 (1.16 to 2.22) 0.87 (0.61 to 1.23) Low quality (score < 6) 0.74 (0.17 to 3.16) 1.75 (0.75 to 4.08) 1.18 (0.57 to 2.46) RR, random effects (95%CI) High quality (score > 7.15) 0.46 (0.23 to 0.93) 1.81 (1.23 to 2.67) Low quality (score 7.15) 0.49 (0.28 to 0.85) 1.91 (1.11 to 3.29) Meta-regression analysis (Univariate analysis) Coefficient RR ratio (95%CI) (-2.23 to 1.2) (-1.14 to 0.97) (-1.5 to 0.92) Using Mean Coefficient (95%CI) (-1.05 to 0.7) (-0.82 to 0.72) 0.59 (0.1 to (0.32 to 2.64) 0.75 (0.22 to 2.51) Meta-regression analysis (Univariate analysis) RR ratio 0.84 (0.35 to 2) 0.95 (0.44 to 2.05) P value P value Withdrawal due to lack of efficacy and toxicity 0.95 (0.52 to 1.73) 0.9 (0.66 to 1.23) 0.06 (-0.77 to 0.88) 1.06 (0.46 to 2.4) 0.88 Abbreviation: - RR = relative risk, 95%CI = 95% confidence interval

97 87 Table 15 Subgroup analysis and Meta-regression analysis using Withdrawal due to lack of efficacy Item Randomization, adequate Blinding patients, performed Blinding care providers, performed Blinding outcome assessors, performed Treatment allocation concealment, adequate Drop-out rate, acceptable Compliance, acceptable RR, random effects (95%CI) Meta-regression analysis (Univariate analysis) Met Not met Coefficient RR ratio P value (95%CI) (95%CI) (0.29 to 0.81) (0.2 to 0.91) (-0.9 to 1.12) (0.41 to 3.06) (0.3 to 0.67) (0.17 to (-2.23 to 1.2) (0.11 to 3.32) 3.16) 0.45 (0.3 to 0.67) 0.35 (0.18 to 0.66) 0.46 (0.3 to 0.7) 0.57 (0.32 to 1) 0.73 (0.39 to 1.35) 0.74 (0.17 to 3.16) 0.55 (0.34 to 0.88) 0.49 (0.22 to 1.09) 0.4 (0.24 to 0.67) 0.36 (0.22 to 0.58) (-2.23 to 1.2) (-1.4 to 0.43) (-1.08 to 0.99) 0.31 (-0.56 to 1.19) 0.66 (-0.24 to 1.56) 0.59 (0.11 to 3.32) 0.62 (0.25 to 1.54) 0.96 (0.34 to 2.69) 1.36 (0.57 to 3.29) 1.93 (0.79 to 4.76) Co-intervention, similar 0.74 (0.36 to 1.53) ITT, performed 0.44 (0.3 to 0.65) 0.39 (0.25 to 0.61) 0.94 (0.25 to 3.53) 0.6 (-0.36 to 1.56) (-2.32 to 0.91) 1.82 (0.7 to 4.76) 0.49 (0.1 to 2.48) Abbreviation: - RR = relative risk, 95%CI = 95% confidence interval, ITT = Intention-to-treat analysis

98 88 Table 16 Subgroup analysis and Meta-regression analysis using Withdrawal due to toxicity Item Randomization, adequate Blinding patients, performed Blinding care providers, performed Blinding outcome assessors, performed Treatment allocation concealment, adequate Drop-out rate, acceptable Compliance, acceptable Co-intervention, similar RR, random effects (95%CI) Meta-regression analysis (Univariate analysis) Met Not met Coefficient RR ratio P value (95%CI) (95%CI) (1.06 to (1.2 to 2.12) (-0.82 to 0.66) (0.44 to 2.62) 1.93) 1.54 (1.11 to 2.13) 1.61 (1.16 to 2.22) 1.06 (0.66 to 1.69) 1.63 (1.07 to 2.48) 1.93 (1.29 to 2.88) 1.32 (0.87 to 2) 2.35 (1.45 to 3.79) ITT, performed 1.6 (1.14 to 2.25) 2.09 (1 to 2.12) 1.75 (0.75 to 4.08) 2.03 (1.42 to 2.91) 1.78 (1.03 to 3.08) 1.44 (0.81 to 2.56) 1.89 (1.28 to 2.8) 1.28 (0.9 to 1.84) 1.88 (0.85 to 4.13) -0.3 (-1.23 to 0.62) (-1.14 to 0.97) (-1.29 to ) (-0.91 to 0.65) 0.38 (-0.29 to 1.04) (-1.02 to 0.31) 0.6 (-0.05 to 1.25) (-1.16 to 0.82) 0.74 (0.29 to 1.86) 0.91 (0.32 to 2.64) 0.52 (0.28 to 1) 0.88 (0.4 to 1.92) 1.46 (0.75 to 2.83) 0.72 (0.36 to 1.36) 1.82 (0.95 to 3.49) 0.84 (0.31 to 2.27) Abbreviation: - RR = relative risk, 95%CI = 95% confidence interval, ITT = Intention-to-treat analysis

99 89 Table 17 Subgroup analysis and Meta-regression analysis using Withdrawal due to lack of efficacy and toxicity Item Randomization, adequate Blinding patients, performed Blinding care providers, performed Blinding outcome assessors, performed Treatment allocation concealment, adequate Drop-out rate, acceptable Compliance, acceptable Co-intervention, similar RR, random effects (95%CI) Meta-regression analysis (Univariate analysis) Met Not met Coefficient RR ratio P value (95%CI) (95%CI) (0.54 to 1.44) (0.66 to 1.38) (-0.94 to (0.39 to 2) 0.71) 0.81 (0.58 to 1.12) 0.87 (0.61 to 1.23) 0.72 (0.42 to 1.24) 0.86 (0.56 to 1.33) 1.13 (0.75 to 1.71) 1.07 (0.77 to 1.48) 1.47 (0.91 to 2.39) ITT, performed 0.83 (0.59 to 1.17) 1.47 (0.76 to 2.83) 1.18 (0.57 to 2.46) 1.02 (0.74 to 1.43) 0.95 (0.63 to 1.43) 0.71 (0.47 to 1.08) 0.78 (0.5 to 1.2) 0.71 (0.52 to 0.96) 1.34 (0.67 to 2.68) (-1.64 to 0.37) (-1.5 to 0.92) (-1.16 to 0.33) (-1.03 to 0.66) 0.52 (-0.19 to 1.22) 0.41 (-0.34 to 1.17) 0.79 (0.13 to 1.45) (-1.61 to 0.55) 0.53 (0.19 to 1.45) 0.75 (0.22 to 2.5) 0.66 (0.31 to 1.39) 0.84 (0.36 to 1.93) 1.68 (0.83 to 3.39) 1.51 (0.71 to 3.22) 2.2 (1.14 to 4.26) 0.59 (0.2 to 1.73) Abbreviation: - RR = relative risk, 95%CI = 95% confidence interval, ITT = Intention-to-treat analysis

100 90 Table 18 Summary the results of relative risk and 95% confidence interval comparing between fixed and random effects models Outcome Fixed effects RR (95%CI) Random effects RR (95%CI) ACR 20 Total 1.77 (1.53,2.04) 1.82 (1.33,2.49) DMARD naive 1.24 (1.05,1.48) 1.22 (0.88,1.68) MTX inadequate response 2.53(1.94,3.31) 2.51 (1.92,3.28) Non-MTX inadequate response 1.87 (1.20,2.91) 1.85 (1.21,2.83) ACR 50 Total 2.53 (1.83,3.5) 2.41 (1.43,4.05) DMARD naive 1.66 (1,1.48) 1.76 (0.64,4.85) MTX inadequate response 4.61 (2.54,8.36) 4.54 (2.51,8.2) Non-MTX inadequate response 1.65 (0.91,2.97) 1.68 (0.94,2.99) ACR 70 Total 3.79 (2.05,7) 3.26 (1.78,5.99) DMARD naive 2.41 (1.07,5.44) 2.41 (1.07,5.44) MTX inadequate response 5.99 (2.23,16.11) 5.59 (2.08,15.01) Non-MTX inadequate response 1.93 (0.18,20.65) 1.93 (0.18,20.65) ACR remission Total DMARD naive 1.27 (0.8,2.03) 1.27 (0.8,2.03) MTX inadequate response - - Non-MTX inadequate response - - EULAR good response Total 1.09 (0.78,1.52) 1.06 (0.7,1.6) DMARD naive 0.97 (0.69,1.37) 0.97 (0.69,1.37) MTX inadequate response - - Non-MTX inadequate response 3.38 (0.73,15.53) 3.38 (0.73,15.53) EULAR moderate response Total - - DMARD naive 1.37 (0.81,2.33) 1.37 (0.81,2.33) MTX inadequate response - - Non-MTX inadequate response - - EULAR remission Total 1.35 (0.91,2.01) 1.33 (0.83,2.12) DMARD naive 1.26 (0.84,1.88) 1.26 (0.84,1.88)

101 91 MTX inadequate response - - Non-MTX inadequate response 3.86 (0.45,33.42) 3.86 (0.45,33.42) Withdrawal due to lack of efficacy Total 0.45 (0.31, 0.65) 0.47 (0.32, 0.69) DMARD naive 0.65 (0.35,1.22) 0.63 (0.34,1.17) MTX inadequate response 0.43 (0.23,0.79) 0.42 (0.21,0.84) Non-MTX inadequate response 0.34 (0.17,0.67) 0.37 (0.16,0.87) Withdrawal due to toxicity Total 1.6 (1.26,2.2) 1.56 (1.17, 2.09) DMARD naive 1.76 (1.07,2.87) 1.72 (1.04,2.83) MTX inadequate response 1.91 (1.06,3.44) 1.89 (1.05,3.41) Non-MTX inadequate response 1.47 (0.97,2.24) 1.53 (0.74,3.18) Withdrawal due to Lack of efficacy and toxicity Total 0.89 (0.72, 1.1) 0.89 (0.66, 1.21) DMARD naive 1.18 (0.79,1.74) 1.16 (0.7,1.93) MTX inadequate response 0.92 (0.63,1.34) 0.86 (0.49,1.51) Non-MTX inadequate response 0.69 (0.49,0.98) 0.75 (0.41,1.35) Abbreviation: - RR = relative risk, 95%CI = 95% confidence interval, DMARD = disease modifying antirheumatic drug, MTX = methotrexate, ACR = American College of Rheumatology, EULAR = European League Against Rheumatism

102 92 Table 19 Search strategies comparing between Ovid-MEDLINE and PubMed Ovid Search term (number of citation) PubMed Search term (number of citation) 1. arthritis, rheumatoid/ (67002) 1. Arthritis, Rheumatoid [Mesh:noexp] (67291) 2. caplan's syndrome/ (133) 2. caplan syndrome[mesh:noexp] (133) 3. felty's syndrome/ (595) 3. Felty's Syndrome[Mesh:noexp] (595) 4. rheumatoid nodule/ (762) 4. Rheumatoid Nodule[Mesh:noexp] (757) 5. still's disease, adult-onset/ (611) 5. Still's Disease, Adult Onset [Mesh:noexp] (599) 6. rheumatism.mp. (6945) 6. rheumatism[all Fields] (11910) 7. caplan's syndrome$.mp. (84) 7. caplan syndrome* (161) 8. felty's syndrome$.mp. (739) 8. felty's syndrome* (751) 9. rheumatoid.mp. (91164) 9. rheumatoid[all Fields] (93993) 10. Methotrexate$.mp. (33979) 10. methotrexate* (34426) 11. amethopterin$.mp. (356) 11. Amethopterin* (397) 12. mexate$.mp. (1) 12. mexate* (1) 13. Abitrexate$.mp. (0) 13. Abitrexate* (0) 14. Amethopterin$.mp. (356) 14. Amethopterin* (397) 15. A Methopterine$.mp. (1) 15. A Methopterine* (1) 16. Ametopterine$.mp. (2) 16. Ametopterine* (16) 17. Antifolan$.mp. (0) 17. Antifolan* (0) 18. Emtexate$.mp. (0) 18. Emtexate* (0) 19. Emthexate$.mp. (0) 19. Emthexate* (0) 20. Emtrexate$.mp. (1) 20. Emtrexate* (1) 21. Enthexate$.mp. (0) 21. Enthexate* (0) 22. Farmitrexate$.mp. (0) 22. Farmitrexate* (0) 23. Folex.mp. (3) 23. Folex* (3) 24. Ledertrexate.mp. (1) 24. Ledertrexate* (1) 25. Methoblastin$.mp. (0) 25. Methoblastin * (0) 26. Methohexate$.mp. (0) 26. Methohexate* (0) 27. Methotrate$.mp. (0) 27. Methotrate* (0) 28. Methotrexat$.mp. (34001) 28. Methotrexat* (34448) 29. Methylaminopterin$.mp. (9) 29. Methylaminopterin* (8) 30. Metotrexat$.mp. (134) 30. Metotrexat* (30) 31. Mtx.mp. (6274) 31. MTX (6405) 32. Novatrex$.mp. (0) 32. novatrex* (0) 33. Rheumatrex.mp. (3) 33. rheumatrex* (3) 34. randomized controlled trial.pt. 34. randomized controlled trial

103 93 (263105) [Publication Type] (258826) 35. placebo$.mp. (125474) 35. placebo* (128221) 36. random$.mp. (561059) 36. random* (582840) 37. clinical trial.pt. (446706) 37. clinical trial[publication Type] (548249) 38. exp clinical trial as topic/ (210045) 38. Clinical Trials as Topic[Mesh] (206536) 39. clinical trial$.mp. (613169) 39. clinical trial* (614699) 40. controlled clinical trial.pt. (78151) 40. controlled clinical trial [Publication Type] (77549) 41. clinical trial, phase i.pt. (9286) 41. clinical trial, phase I [Publication Type] (9071) 42. clinical trial, phase ii.pt. (14496) 42. clinical trial, phase ii [Publication Type]) (14182) 43. clinical trial, phase iii.pt. (4571) 43. clinical trial, phase iii [Publication Type]) (4433) 44. clinical trial, phase iv.pt. (430) 44. clinical trial, phase iv [Publication Type]) (412) 45. (clinical$ adj2 trial$).mp. (614943) 45. ((clinical*) AND (trial*)) (668136) 46. single-blind method/ ( Single-Blind Method [Mesh:noexp] (12200) 47. Double-Blind Method/ (98848) 47. Double-Blind Method [Mesh:noexp] (97431) 48. "double blind:".mp. (118209) 48. double blind* (118293) 49. ((singl$ or doubl$ or trebl$) adj2 (blind$ or mask$)).mp. (133392) 49. (((singl*) OR (doubl*) OR (trebl*)) AND ((blind*) OR (mask*))) (142681) 50. exp Research Design/ (244739) 50. Research Design[Mesh] (241338) 51. comparative study.pt. ( ) 51. comparative study [Publication Type] ( ) 52. exp Evaluation Studies as topic/ 52. Evaluation Studies as Topic [Mesh] (730924) (654983) 53. follow-up studies/ (373404) 53. Follow-Up Studies [Mesh:noexp] (367572) 54. prospective studies/ (250873) 54. prospective studies [Mesh:noexp] (245950) 55. (control$ or prospectiv$ or 55. control* ( ) OR prospectiv* volunteer$).mp. ( ) (361547) OR volunteer* (112176) 56. meta-analysis.pt. (19931) 56. meta analysis [Publication Type]) (19115) 57. meta-analy$.mp. (35003) 57. meta-analy* (35652) 58. meta analy$.mp. (35003) 58. meta analy* (35652)

104 metaanaly$.mp. (926) 59. metaanaly* (828) 60. metanaly$.mp. (112) 60. metanaly* (113) 61. or/1-9 (97213) 61. or/1-9 (104892) 62. or/10-33 (34747) 62. or/10-33 (35593) 63. or/34-60 ( ) 63. or/34-58 ( ) and 62 and 63 (2078) and 62 and 63 (2181) 65. limit 64 to "all child (0 to 18 years)" (288) 65. limit 64 to "all child (0 to 18 years)" (290) 66. limit 64 to "all adult (19 plus years)" (1182) 66. limit 64 to "all adult (19 plus years)" (1190) 67. or/64-66 (1330) 67. or/64-66 (1335) not 67 (748) not 67 (846) or 68 (1930) or 68 (2036) Table 20 Performances of Ovid-MEDLINE and Pubmed search for randomized controlled trials comparing the efficacy and safety of MTX combination therapy versus MTX monotherapy in RA patients Ovid-MEDLINE PubMed Total citations retrieved Eligible citations identified Total eligible citations in this review Recall 85 % 90 % Precision % % Number-Needed-to-read

105 95 Table 21 Results of the citations retrieved from PubMed compared to EMBASE Number of study identified Number of included studies Question Pubmed EMBASE Total (Pubmed and EMBASE) EMBASE retrieved article A. Dosing strategy (RCT) B. Liver toxicity management (Cohort, case control) C. Long term safety 1, (Cohort, case control) D. MTX Combo vs. Mono 1,662 4, therapy (RCT*) E. Folate supplementation (RCT) F. Management during Perioperative period (2RCTs, 2 cohorts) G. Management during pregnancy (case series) H. Efficacy in other rheumatic diseases (4 RCTs, 1 cohort) Total citations retrieved 4,218 6, Abbreviation: - RCT = Randomized controlled trial

106 96 Table 22 Reasons why studies were missed by searching in PubMed Citation Reason Cannon GW, Reading JC, Ward JR et al. Clinical and laboratory outcomes during the treatment of rheumatoid arthritis with methotrexate. Scandinavian Journal of Rheumatology 1990;19(4): Dosing strategy Indexed in PubMed Not indexed with the search term used by researcher (a set of term for Drug administration route/dosing ) Williams HJ, Ward JR, Reading JC et al. Comparison of auranofin, methotrexate, and the combination of both in the treatment of rheumatoid arthritis: A controlled clinical trial. Arthritis & Rheumatism 1992; 35(3): Westedt ML, Dijkmans BAC, Hermans J. Azathioprine compared with methotrexate for rheumatoid arthritis: An open randomized clinical trial. Revue du Rhumatisme (English Edition) 1994; 61(9)( ) Rau R, Herborn G, Menninger H et al. Progression in early erosive rheumatoid arthritis: 12 month results from a randomized controlled trial comparing methotrexate and gold sodium thiomalate. British Journal of Rheumatology 1998; 37(11): Strand V, Cohen S, Schiff M et al. Treatment of active rheumatoid arthritis with leflunomide compared with placebo and methotrexate. Archives of Internal Medicine 1999; 159(21): Sharp JT, Strand V, Leung H et al. Treatment with leflunomide slows radiographic progression of rheumatoid arthritis: Results from three randomized controlled trials of leflunomide in patients with active rheumatoid arthritis. Arthritis & Rheumatism 2000; 43(3): Indexed in PubMed Not indexed with the search term used by researcher (a set of term for Drug administration route/dosing ) Journal not indexed in MEDLINE/PubMed Indexed in PubMed Not indexed with the search term used by researcher (a set of term for Drug administration route/dosing ) Indexed in Pubme Not indexed with the search term used by researcher (a set of term for Drug administration route/dosing ) Indexed in PubMed Not indexed with the search term used by researcher (a set of term for Drug administration route/dosing )

107 97 Singh CP, Kaur BG, Singh G et al. The open, randomised trial of cyclosporine vs. methotrexate in refractory rheumatoid arthritis. Journal of Internal Medicine of India 2000; 3(1): Ferraccioli GF, Gremese E, Tomietto P et al. Analysis of improvements, full responses, remission and toxicity in rheumatoid patients treated with step-up combination therapy (methotrexate, cyclosporin A, sulphasalazine) or monotherapy for three years. Rheumatology 2002;41(8): Bao C, Chen S, Gu Y et al. Leflunomide, a new disease-modifying drug for treating active rheumatoid arthritis in methotrexate-controlled phase II clinical trial. Chinese Medical Journal 2003; 116(8): Journal not indexed in MEDLINE/PubMed Indexed in PubMed Not indexed with the search term used by researcher (a set of term for Drug administration route/dosing ) Indexed in PubMed Not indexed with the search term used by researcher (a set of term for Drug administration route/dosing ) Liver Toxicity Hoffmeister RT. Methotrexate therapy in rheumatoid arthritis: 15 years experience. American Journal of Medicine ;75(6 A):69-73 Indexed in PubMed Insufficient or restricted search strategy Term Liver [majr] was used in the search strategy (which would search this term when it is used as MeSH as major topic). This article used liver as MeSH but liver is not the main topic Groff GD, Shenberger KN, Wilke WS et al. Low dose oral methotrexate in rheumatoid arthritis: An uncontrolled trial and review of the literature. Seminars in Arthritis & Rheumatism 1983; 12(4): Weinstein A, Marlowe S, Korn J et al. Lowdose methotrexate treatment of rheumatoid arthritis. Long-term observations. American Journal of Medicine 1985; 79(3): Indexed in PubMed Insufficient or restricted search strategy Term Liver [majr] was used in the search strategy (which would search this term when MeSH is used as major topic only). This article used liver as MeSH but liver is not the main topic Indexed in PubMed Insufficient or restricted search strategy Term Liver [majr] was used in the search strategy (which would search

108 98 Boh LE, Schuna AA, Pitterle ME. Low-dose weekly oral methotrexate therapy for inflammatory arthritis. Clinical Pharmacy 1986; 5(6): Scully CJ, Anderson CJ, Cannon GW. Longterm methotrexate therapy for rheumatoid arthritis. Seminars in Arthritis & Rheumatism 1991; 20(5): Taqweem MA, Ali Z, Takreem A et al. Methotrexate induced hepatotoxicity in Rheumatoid Arthritis patients. Journal of Postgraduate Medical Institute 2005; 19(4): this term when MeSH is used as major topic only). This article used liver as MeSH but liver is not the main topic Indexed in PubMed Insufficient or restricted search strategy Term related to liver not used as MeSH in this article Term Liver enzyme appeared in the abstract but the search strategy Liver enzyme[ti] was used which would search this term in title only Indexed in PubMed Insufficient or restricted search strategy Term related to liver not used as MeSH in this article Term Liver enzyme appeared in the abstract but the search strategy Liver enzyme[ti] was used which would search this term in title only Journal not indexed in PubMed Long term safety of MTX Nagashima M, Matsuoka T, Saitoh K, et al Treatment continuation rate in relation to efficacy and toxicity in long-term therapy with low-dose methotrexate, sulfasalazine, and bucillamine in 1358 Japanese patients with rheumatoid arthritis. Clin. Exp. Rheumatol. 2006; 24(3): Found in PubMed Insufficient or restricted search strategy Publication type was clinical trial and journal article which was not included in the search term MTX combination versus MTX monotherapy Hanyu T, Arai K, Ishikawa H. Long-term Journal not indexed in PubMed methotrexate (MTX) combination therapy versus MTX alone for active rheumatoid arthritis. Jpn J Rheumatol. 1999; 9: Tascioglu F, Oner C, Armagan O. Comparison of low dose methotrexate and combination Journal not indexed in PubMed

109 99 therapy with methotrexate and sulphasalazine in the treatment of early rheumatoid arthritis. J Rheumatol Med Rehab. 2003; 14: MTX as steroid-sparing in other rheumatic diseases Carneiro JR, Sato EI. Double blind, randomized, placebo controlled clinical trial of methotrexate in systemic lupus erythematosus. J Rheumatol 1999;26(6): Ramanan AV, Campbell-Webster N, et al. The effectiveness of treating juvenile dermatomyositis with methotrexate and aggressively tapered corticosteroids. Arthritis & Rheum 2005; 52(11)(3570-8) Pregnancy outcome in MTX treated RA Chakravarty EF, Sanchez-Yamamoto D, Bush TM.The use of disease modifying antirheumatic drugs in women with rheumatoid arthritis of childbearing age: a survey of practice patterns and pregnancy outcomes. J Rheumatol. 2003; 30(2): Not indexed with the search term used by researcher (this article used prednisone as MeSH. It also appeared in abstract but researcher used steroid and glucocoticoid search as [MeSH] and in title abstract [tiab] Misclassified as not retrieved by Pubmed by the reviewer Indexed in PubMed Insufficient or restricted search strategy (Publication type was journal article which was not included in the search term) Donnenfeld AE, Pastuszak A, Noah JS, et al MTX exposure prior to and during pregnancy Teratology 1994;49(2):79-81 Indexed in PubMed Insufficient or restricted search strategy (Publication type was letter which was not included in the search term)

110 100 FIGURES Figure 1 Flow diagram: - Results of the literature search and disposition of the potentially relevant studies 6938 Titles and abstracts identified through searches MEDLINE n = 1662 EMBASE n = 4591 CENTRAL n =374 ACR and EULAR( )n = 311 Hand search n = 0 Inclusion criteria - RA, age 18 y - RCT - MTX vs. MTX combo 39 Full-text articles retrieved* (Not in English) MEDLINE n = 31(1) EMBASE n = 35(0) CENTRAL n = 24 (0) ACR and EULAR abstract n = Citations excluded Non-RA (n = 324 ) Non-article type(n = 51 ) Non-RCT (n =112) Non-MTX or MTX combination (n = 647) Non-relevant (n = 5,627) Relevant review (n = 39) No outcome of interest (n = 46) Exclusion criteria - RCT (Open label extension)/ non-rct - No outcome of interest - No MTX ± placebo arm - Uncommonly use DMARD 19 Full text articles excluded Non-RCT (n = 9) Not original article (n = 2) No outcome of interest (n = 3) No MTX ± placebo arm (n = 5) 20 Articles included (19 studies) Duplicate in all database n = 16 EMBASE only n = 4 * Number is not equal to the sum of number from each database due to duplication among databases Abbreviation: - CENTRAL= Cochrane Central Register of Controlled Trials, ACR = American College of Rheumatology Response criteria, EULAR = European League Against Rheumatism response criteria, RCT = randomized controlled trial, RA = rheumatoid arthritis, MTX = methotrexate, DMARD = disease modifying anti-rheumatic drug

111 101 Figure 2 Combined withdrawal due to lack of efficacy and toxicity: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, CSA = cyclosporin, DOXY = doxycycline, LEF = leflunomide, im Gold = intramuscular gold, CQ = chloroquine, HCQ = hydroxychloroquine, BUC = bucillamine, RR = relative risk, 95%CI = 95% confidence interval

112 102 Figure 3 Withdrawal due to lack of efficacy in DMARD naïve, MTX inadequate responder, and non-mtx inadequate responder: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = Methotrexate, SSZ = sulfazalazine, CSA = cyclosporin, DOXY = doxycycline, LEF = leflunomide, im Gold = intramuscular gold, CQ = chloroquine, HCQ = hydroxychloroquine, BUC = bucillamine, RR = relative risk, 95%CI = 95% confidence interval

113 103 Figure 4 ACR responses in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - ACR = American college of Rheumatology, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, CSA = cyclosporin, DOXY = doxycycline, RR = relative risk, 95%CI = 95% confidence interval

114 104 Figure 5 EULAR responses in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - EULAR = European League Against Rheumatism, DMARD = Disease modifying antirheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, CSA = cyclosporin, RR = relative risk, 95%CI = 95% confidence interval

115 105 Figure 6 Tender joint counts in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 7 Pain in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

116 106 Figure 8 Patient global assessment of disease activity in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 9 ESR in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - ESR = erythrocyte sedimentation rate, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

117 107 Figure 10 CRP in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Abbreviation: - CRP = C-reactive protein, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 11 HAQ in DMARD naïve: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - HAQ = Health Assessment Questionnaire, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

118 108 Figure 12 Radiographic outcomes in DMARD naïve: - A comparison between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, CSA = cyclosporin, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 13 ACR responses in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - ACR = American college of Rheumatology, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, CSA = Ccyclosporin, LEF = leflunomide, im Gold = intramuscular gold, LEV = levofloxacin, RR = relative risk, 95%CI = 95% confidence interval

119 109 Figure 14 Tender joint counts in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, LEF = leflunomide, im Gold = intramuscular gold, CSA =cyclosporin, SD = standard deviation, SMD = standardized mean difference, 95%CI = 95% confidence interval Figure 15 Swollen joint counts in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, LEF = leflunomide, im Gold = intramuscular gold, CSA = cyclosporin, SD = standard deviation, SMD = standardized mean difference, 95%CI = 95% confidence interval

120 110 Figure 16 Pain in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, LEF = leflunomide, im Gold = intramuscular gold, CSA = cyclosporin, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 17 Patient global assessment of disease activity in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, LEF = leflunomide, im Gold = intramuscular gold, CSA = cyclosporin, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

121 111 Figure 18 Physician global assessment of disease activity in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, LEF = leflunomide, im Gold = intramuscular gold, CSA = cyclosporin, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 19 CRP in MTX inadequate responders: - A comparison between MTX combinations versus MTX monotherapy Abbreviation: - CRP = C-reactive protein, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

122 112 Figure 20 HAQ in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - HAQ = Health Assessment Questionnaire, MTX = methotrexate, CSA = cyclosporin, LEF = leflunomide, im Gold = intramuscular gold, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 21 ESR in MTX inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - ESR = erythrocyte sedimentation rate, MTX = methotrexate, CSA = cyclosporin, LEF = leflunomide, im Gold = intramuscular gold, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

123 113 Figure 22 Radiographic outcomes in MTX inadequate responders: - A comparison between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 23 ACR responses in non-mtx inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - ACR = American College of Rheumatology, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, BUC = bucillamine, SSZ = sulfazalazine, RR = relative risk, 95%CI = 95% confidence interval

124 114 Figure 24 EULAR responses in non-mtx inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - EULAR = European League Against Rheumatism, DMARD = Disease modifying antirheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, RR = relative risk, 95%CI = 95% confidence interval Figure 25 Tender joint counts in non-mtx inadequate responders: - A comparison between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, HCQ = hydroxychloroquine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

125 115 Figure 26 Swollen joint counts in non-mtx inadequate responders: - Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, CQ = chloroquine, SSZ = sulfazalazine, HCQ = hydroxychloroquine, SD = standard deviation, SMD = standardized mean difference, 95%CI = 95% confidence interval Figure 27 Patient global assessment of disease activity in non-mtx inadequate responders: - A Comparison between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, HCQ = hydroxychloroquine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

126 116 Figure 28 Physician global assessment of disease activity in non-mtx inadequate responders: - A Comparison between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, SSZ = sulfazalazine, HCQ = hydroxychloroquine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 29 Pain in non-mtx inadequate responders: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, CQ = chloroquine, SSZ = sulfazalazine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

127 117 Figure 30 ESR in non-mtx inadequate responders: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - ESR = erythrocyte sedimentation rate, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, CQ = chloroquine, SSZ = sulfazalazine, HCQ = hydroxychloroquine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval Figure 31 CRP in non-mtx inadequate responders: -A comparison between MTX combinations versus MTX monotherapy Abbreviation: - CRP = C-reactive protein, MTX = methotrexate, DMARD = Disease modifying antirheumatic drug, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

128 118 Figure 32 HAQ in non-mtx inadequate responders: -A comparison between MTX combinations versus MTX monotherapy Abbreviation: - HAQ = Health Assessment Questionnaire, DMARD = Disease modifying anti-rheumatic drug, MTX = methotrexate, CQ = chloroquine, SD = standard deviation, WMD = weighted mean difference, 95%CI = 95% confidence interval

129 119 Figure 33 Total adverse reactions: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, SSZ = sulfazalazine, LEF = leflunomide, CSA = cyclosporin, AZA = azathioprine, im Gold = intramuscular gold, RR = relative risk, 95%CI = 95% confidence interval

130 120 Figure 34 Gastrointestinal adverse events: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - GI = gastrointestinal, MTX = Methotrexate, SSZ = sulfazalazine, LEF= leflunomide, CSA = cyclosporin, RR = relative risk, 95%CI = 95% confidence interval

131 121 Figure 35 Abnormal liver functions: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, SSZ = sulfazalazine, LEF= leflunomide, CSA = cyclosporin, RR = relative risk, 95%CI = 95% confidence interval Figure 36 Mucositis: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, SSZ = sulfazalazine, RR = relative risk, 95%CI = 95% confidence interval

132 122 Figure 37 Hematological adverse events: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, SSZ = sulfazalazine, RR = relative risk, 95%CI = 95% confidence interval Figure 38 Infection: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, SSZ = sulfazalazine, LEF= leflunomide, RR = relative risk, 95%CI = 95% confidence interval

133 123 Figure 39 Withdrawal due to adverse reaction: -Comparisons between MTX combinations versus MTX monotherapy Abbreviation: - MTX = methotrexate, SSZ = sulfazalazine, HCQ =hydroxychloroquine, CQ = chloroquine, CSA = cyclosporin, AZA = azathioprine, LEF = leflunomide, DMARD = Disease modifying anti-rheumatic drug, RR = relative risk, 95%CI = 95% confidence interval

134 Figure 40 PubMed usage data from June, 2007 to March,