Systematic review: the methodological quality of trials affects estimates of treatment efficacy in functional (non-ulcer) dyspepsia

Transcription

1 Aliment Pharmacol Ther 2004; 19: doi: /j x Systematic review: the methodological quality of trials affects estimates of treatment efficacy in functional (non-ulcer) dyspepsia N. S. ABRAHAM*, P. MOAYYEDI, B. DANIELSà & S. J. O. VELDHUYZEN VAN ZANTENà *Sections of Health Services Research and Gastroenterology, Houston Veterans Affairs Medical Center and Baylor College of Medicine, Houston, TX, USA; University of Birmingham, Birmingham, UK; àdivision of Gastroenterology, Dalhousie University, Halifax, NS, Canada Accepted for publication 19 December 2003 SUMMARY Aim: To evaluate treatment efficacy using objective quality criteria. Methods: A systematic review was performed of randomized controlled trials of endoscopically investigated dyspepsia ( ) using the Jadad score and Rome II guidelines. The Jadad score differentiated studies as high quality (score 4 5/5) vs. poor quality (score 1 3/ 5). Three key Rome II guidelines on study design (cut-off of 0/3 or > 0/3) were also compared with the Jadad score. Results: Poor quality trials suggested a benefit of prokinetic therapy [relative risk (RR) of remaining dyspeptic, 0.47; 95% confidence interval (CI), ), which was not confirmed in high quality trials (RR, 1.0; 95% CI, ). There was a marked benefit of H 2 -receptor antagonist therapy in poor quality trials (RR, 0.68; 95% CI, ), but a marginal benefit in good quality trials (RR, 0.87; 95% CI, ). Trial quality did not affect the small statistically significant benefit seen with Helicobacter pylori eradication. Two high quality trials suggested a limited benefit with the use of proton pump inhibitors, with no poor quality trials to provide a comparison. Separation of the studies by the Rome II criteria had a similar impact on the calculated treatment estimates. Conclusions: The magnitude of benefit of prokinetic and H 2 -receptor antagonist therapies reported in previous meta-analyses has been over-estimated. The quality of trials has an impact on the efficacy estimates of treatment. The Rome II criteria for study methodology may be appropriate for judging study quality. INTRODUCTION Functional (non-ulcer) dyspepsia is defined as continuous or frequently recurring epigastric pain or discomfort for which no organic cause can be determined. 1 Epigastric pain or discomfort may be associated with other symptoms, such as upper abdominal bloating, excessive burping or belching, early satiety and nausea. There is controversy with regard to whether symptoms of heartburn and acid regurgitation should be considered as part of the dyspepsia symptom complex. The Correspondence to: Dr N. S. Abraham, Houston VAMC-HCQUS, 2002 Holcombe Blvd (152), TX 77030, USA. nabraham@bcm.tmc.edu Rome II definition of dyspepsia specifically excludes these symptoms and considers them to be indicative of gastro-oesophageal reflux disease (GERD). 2 Others believe that heartburn and acid regurgitation should be considered as part of dyspepsia, especially in the primary care setting. 3, 4 There is consensus that dominant symptoms of heartburn and acid regurgitation are suggestive of GERD. For a diagnosis of functional dyspepsia to be made, patients need to have a normal upper gastrointestinal endoscopy. Over the last decade, several reviews have evaluated the efficacy of therapies for functional dyspepsia. These include treatment with H 2 -receptor antagonists, 5 prokinetic agents (mainly cisapride), 6 9 proton pump inhibitors, 10, 11 anti-helicobacter therapy Ó 2004 Blackwell Publishing Ltd 631

2 632 N. S. ABRAHAM et al. (in patients who are H. pylori positive) and 17, 18 antidepressants or psychological interventions. These studies have revealed problems in study design, including the use of a variety of outcome measures, an unclear definition of treatment response, a heterogeneous population mix, a cross-over study design and an insufficient sample size. An important problem in existing reviews is the lack of strict selection criteria for study quality. Consequently, these meta-analyses and systematic reviews are subject to the methodological weaknesses of the individual studies they include and pool Moher et al. identified the importance of methodological quality assessment as part of meta-analysis to ensure an accurate estimation of effect size in intervention trials. 25 They concluded that studies of low methodological quality significantly inflated the estimates of effect size when included in meta-analyses. In 1999, the Rome II Working Group of Functional Gastrointestinal Disorders published its guidelines on the conduct, design and execution of studies of functional gastrointestinal disorders. 2 This report provided recommendations regarding optimal study methodology, including a need for a clear definition of a treatment responder, the use of validated outcome measures and guidelines for explicit inclusion and exclusion criteria. We believe that the key Rome criteria for appropriate study methodology for functional dyspepsia studies are: (i) the definition of subjects by Rome II criteria with clear inclusion and exclusion criteria, particularly with regard to the way in which patients with possible GERD are handled; (ii) an a priori definition of a patient responder that clarifies what improvement constitutes a treatment success; and, finally, (iii) the use of validated outcome measures that are able to detect change. In this review, we have specifically evaluated functional dyspepsia studies to determine whether the application of the Jadad quality criteria 26 or the Rome criteria influences estimates of effect size. We hypothesize that a consideration of the methodological quality of the studies will decrease previously published estimates of treatment efficacy. METHODS Identification of relevant studies Three independent computer searches were performed using the PubMed database going back to 1979, followed by a recursive review of identified studies and published meta-analyses. The following search terms were used: dyspepsia, functional dyspepsia, non-ulcer dyspepsia, treatment, clinical trial, H 2 -receptor antagonists, prokinetic agent, cisapride, domperidone, proton pump inhibitor, H. pylori and psychotherapy. We also included studies identified by hand-search (recursive) review of bibliographies from previously published 19 24, 27, 28 systematic reviews in this area. Study selection criteria Only randomized, placebo-controlled trials of (endoscopically) investigated functional dyspepsia published in English as full manuscripts were eligible. The main objective of a study needed to be the evaluation of the efficacy of treatment for functional dyspepsia symptoms. All eligible studies were assessed to determine whether the primary outcome could be dichotomized as improved/not improved. A priori, it was determined that only studies that could be dichotomized in this fashion would be abstracted for subsequent aggregation of study results. The Jadad score was used as the gold standard to assess the methodological quality of studies. 26 This validated score lies in the range 0 5. Studies are scored according to the presence of three key methodological features of randomization, blinding and accountability of all patients, including withdrawals. For example, the score is two if appropriate methods of randomization are described, one if the study is merely described as randomized and zero when no details are provided to evaluate randomization. Two points can be given for blinding in the study: a score of two is allocated if patients and investigators are made blind by appropriate methods, one if the study is described merely as double blind and zero if details about blinding are not provided. The third item to be scored is the reporting of withdrawals. The study receives a score of one if all patients are accounted for in the analysis and reasons for withdrawals are provided. A score of zero is given when information regarding withdrawals is incomplete. A priori, it was decided that studies should be scored as high quality if they received a Jadad score of four or five (of a possible five points) and low quality if the score was equal to or less than three. In addition to the Jadad score, three key Rome Consensus Working Group recommendations on the design of clinical trials were chosen by a panel of

3 SYSTEMATIC REVIEW: QUALITY AND EFFICACY IN NUD TREATMENT STUDIES 633 international experts in dyspepsia and research methodologies, to be assessed as a marker of sound research methodology. These included: (i) evidence of a clear description of how heartburn was evaluated in patients and whether a diagnosis of GERD, either based on a dominant symptom of heartburn or on the presence of endoscopic oesophagitis, was a reason for exclusion; (ii) the use of a validated outcome measure; and (iii) an a priori definition of what amount of improvement was required for a patient to be considered as a treatment responder. Studies were scored for the presence or absence of these three key methodological recommendations, one point for each recommendation for a total score of 3/3. To facilitate the operation of this potential new methodological quality score, the expert panel also defined good quality studies as those that scored > 0/3, whereas poor quality studies were those that lacked all of these three key recommendations (i.e. scored 0/3). Study assessment and data abstraction Two investigators (NSA, SJOVvZ) independently performed methodological quality assessments of each selected study using a standardized abstraction form. This form assessed not only the Jadad score and three Rome criteria, but also a variety of items described in the Rome study design report and published previously. 29 This form was pre-tested, after which minor changes were made. Agreement on data abstraction exceeded 95%, and disagreements were resolved through discussion. Data synthesis The data from identified studies were dichotomized into dyspepsia improved or not improved, as described previously. 24 Formal meta-analytical mathematical techniques were used to combine dichotomized results where possible into a single estimate of treatment efficacy, according to the methodological quality of the trials ( good trials ¼ Jadad score of 4 or 5 or Rome criteria > 0/3 vs. poor trials ¼ Jadad score of 1 3 or Rome criteria of 0/3). A fixed effect model (Mantel Haenszel method) was used to aggregate results where possible. This was chosen, as it is generally accepted that the fixed effect approach is valid as a test of significance of the overall null hypothesis (i.e. no effect in all studies ). 30 Results were reported as the relative risk (RR) of persistent dyspepsia (or failure of therapy), such that an aggregated estimate score of > 1 favours placebo (Figures 1 6, see later). In some therapeutic categories (proton pump inhibition, psychotherapy), the generation of a pooled estimate of efficacy was not possible due to differences in the outcome measures used and/or the small number of studies. In that case, conclusions were synthesized in a qualitative fashion to summarize the treatment response. RESULTS Study characteristics Literature searches yielded 61 studies published as full manuscripts. In total, 39 were included for analysis. The remaining 22 were excluded for a variety of reasons, including no randomization, no placebo group, sample size of < 20, no appropriate follow-up after completion of therapeutic intervention and failure to utilize an appropriate definition of dyspepsia. There was 100% agreement between investigators with regard to study selection for data extraction (Table 1). H 2 -receptor antagonists Twelve studies met the inclusion criteria. The effect size of all studies combined favoured the use of H 2 -receptor antagonists [RR, 0.76; 95% confidence interval (CI), ]. However, when the impact of study quality according to the Jadad criteria was examined, the effect size became less pronounced in good quality studies (RR, 0.87; 95% CI, ) compared with poor quality studies (RR, 0.68; 95% CI, ) (Figure 1). When the Rome criteria were used to assess the effect size, this was more pronounced, with good quality studies showing no statistically significant effect (RR, 1.01; 95% CI, ) compared with a significant effect for poor quality studies (RR, 0.73; 95% CI, ) (Figure 2). Proton pump inhibitors Only two citations were eligible for data abstraction. These were the combined Bond and Opera studies by Talley et al. 10 and the study by Wong et al. 11 Both achieved a Jadad score of 4 and high Rome quality

4 634 N. S. ABRAHAM et al. H 2 RA versus placebo stratified by Jadad score Jadad Citation Treated Control Effect Lower Upper 1 to 3 Delattre / / to 3 Gotthard / / to 3 Hadi 33 0 / / to 3 Kelbaek / / to 3 Mackinnon 35 1 / 11 5 / to 3 Muller / / to 3 Nesland / / to 3 Olubuyide / / to 3 Singal / / Total 1 to 3 (9) 277 / / to 5 Blum / / to 5 Hansen 5 51 / / to 5 Saunders / / Total 4 to 5 (3) 216 / / Total Combined (12) 493 / / Favours H 2 RA Favours placebo Figure 1. H 2 -receptor antagonist (H 2 RA) therapy (Jadad). H 2 RA versus placebo stratified by Rome criteria Rome Citation Treated Control Effect Lower Upper 0 Blum / / Delattre / / Gotthard / / Hadi 33 0 / / Kelbaek / / Mackinnon 35 1 / 11 5 / Muller / / Nesland / / Olubuyide / / Saunders / / Total 0 (10) 432 / / Hansen 5 51 / / Singal / / Total 1 (2) 61 / / Total Combined (12) 493 / / Favours H 2 RA Favours placebo Figure 2. H 2 -receptor antagonist (H 2 RA) therapy (Rome). Prokinetic versus placebo stratified by Jadad score Jadad Citation Treated Control Effect Lower Upper Jadad 2-3 Al-Quorain 42 6 / / Jadad 2-3 Bekhti 8 7 / / Jadad 2-3 Champion / / Jadad 2-3 Chung 44 4 / / Jadad 2-3 DeGroot / / Jadad 2-3 DeNutte 46 3 / 17 8 / Jadad 2-3 Francois 47 3 / / Jadad 2-3 Hannon 48 3 / 11 9 / Jadad 2-3 Kellow 49 5 / / Jadad 2-3 Rosch / / Jadad 2-3 Wood 51 1 / 6 2 / Jadad 2-3 Yeoh / / Total Jadad 2-3 (12) 123 / / Jadad 4-5 Hansen 5 41 / / Jadad 4-5 Holtman 7 51 / / Total Jadad 4-5 (2) 92 / / Total Combined (14) 215 / / Favours prokinetic Favours placebo Figure 3. Prokinetic therapy (Jadad).

5 SYSTEMATIC REVIEW: QUALITY AND EFFICACY IN NUD TREATMENT STUDIES 635 Prokinetic versus placebo stratified by Rome criteria Rome Study Treated Control Effect Lower Upper 0 Al-Quorain 42 6 / / Bekhti 8 7 / / Champion / / Chung 44 4 / / DeGroot / / DeNutte 46 3 / 17 8 / Francois 47 3 / / Hannon 48 3 / 11 9 / Kellow 49 5 / / Rosch / / Yeoh / / Total 0 (11) 122 / / to 2 Hansen 5 41 / / to 2 Holtman 7 51 / / to 2 Wood 51 1 / 6 2 / Total 1 to 2 (3) 93 / / Total Combined (14) 215 / / Figure 4. Prokinetic therapy (Rome) Favours prokinetic Favours placebo H. pylori eradication versus placebo stratified by Jadad score Figure 5. Helicobacter pylori (Hp) eradication therapy (Jadad). Jadad Citation Treated Control Effect Lower Upper 2 to 3 des Varennes / / to 3 Gisbert / 34 8 / to 3 Hsu / / to 3 Koskenpato / / to 3 Miwa / / Total 2 to 3 (5) 209 / / to 5 Blum / / to 5 Froehlich / / to 5 Koelz / / to 5 McColl / / to 5 Talley / / to 5 Talley / / Total 4 to 5 (6) 520 / / Total Combined (11) 729 / / Favours Hp eradication Favours placebo H. pylori eradication versus placebo stratified by Rome criteria Rome Citation Treated Control Effect Lower Upper 1 to 2 Froehlich / / to 2 Koelz / / to 2 Koskenpato / / Total 1 to 2 (3) 153 / / Blum / / des Varennes / / Gisbert / 34 8 / Hsu / / McColl / / Miwa / / Talley / / Talley / / Total 3 (8) 576 / / Figure 6. Helicobacter pylori eradication therapy (Rome). Total Combined (11) 729 / / Favours H.pylori eradication Favours placebo

6 636 N. S. ABRAHAM et al. Table 1. Abstracted studies Author Delattre et al. 31 Gotthard et al. 32 Hadi 33 Kelbaek et al. 34 Mackinnon et al. 35 Muller et al. 36 Nesland and Berstad 37 Olubuyide and Atoba 38 Singal et al. 39 Blum et al. 40 Hansen et al. 5 Saunders et al. 41 Al-Quorain et al. 42 Bekhti and Rutgeerts 8 Champion et al. 43 Chung 44 DeGroot and De Both 45 De Nutte et al. 46 Francois and De Nutte 47 Hannon 48 Kellow et al. 49 Rosch 50 Wood et al. 51 Yeoh et al. 52 Holtman et al. 7 des Varennes et al. 53 Gisbert et al. 54 Hsu et al. 55 Koskenpato et al. 56 Miwa et al. 57 Blum et al. 13 Froehlich et al. 16 Koelz et al. 58 McColl et al. 59 Talley et al. 15 Calvert et al. 18 Hamilton et al. 17 Wong et al. 11 Talley et al. 14 Reference scores. Due to the limited number of studies, an aggregate score was not derived and results are described qualitatively. In the combined Bond and Opera studies, 1248 patients were randomized to a 4-week treatment with omeprazole 20 mg, omeprazole 10 mg or placebo. The main outcome measure was relief of epigastric pain and/or discomfort using a validated four-point Likert scale. Symptoms were assessed 3 days prior to the first and last visits during the trial, and complete absence of epigastric pain and discomfort on each of the 3 days was the primary end-point of interest. This outcome measure was validated and the definition of a treatment responder was explicit. Omeprazole was superior to placebo: omeprazole 20 mg, 38% (161/421); omeprazole 10 mg, 36% (146/405); placebo, 28% (119/422). The difference between omeprazole 20 mg or 10 mg and placebo was statistically significant (omeprazole 20 mg: 95% CI, %; omeprazole 10 mg: 95% CI, %). When randomization to omeprazole 20 mg was compared with placebo, the number needed to treat was 10. In the study by Wong et al., carried out in Hong Kong, 453 patients were randomized to a 4-week treatment with lansoprazole 30 mg/day, lansoprazole 15 mg/day or placebo. The severity of dyspepsia symptoms was assessed using a validated five-point Likert scale. There was no difference in the proportion of patients with complete relief of dyspepsia symptoms amongst the three treatment groups: lansoprazole 30 mg, 23%; lansoprazole 15 mg, 23%; placebo, 30%. It is worth mentioning that this study was carried out in Hong Kong, an area in which the prevalence of GERD is low. Specific criteria were used to exclude GERD patients and an a priori definition of treatment response was given. Prokinetic agents A total of 17 studies, using cisapride or domperidone, were retrieved through computer search, 14 of which met the inclusion criteria. When studies were grouped according to the Jadad score, the effect size was larger for poor quality studies (RR, 0.47; 95% CI, ) when compared with good quality studies (RR, 1.0; 95% CI, ) (Figure 3), which demonstrated no significant therapeutic effect at all. Similarly, when the Rome criteria were used (Figure 4), the statistically significant effect for poor quality trials (RR, 0.47; 95% CI, ), favouring the use of prokinetic agents, was not observed in good quality trials (RR, 0.99; 95% CI, ). Coating agents One study was identified. The study by Holtmann et al., which compared simethicone with cisapride and placebo, received a Jadad score of > 4. 7 In this study, the

7 SYSTEMATIC REVIEW: QUALITY AND EFFICACY IN NUD TREATMENT STUDIES 637 coating agent simethicone proved to be more efficacious than both cisapride and placebo in the relief of upper abdominal pain or discomfort over an 8-week treatment period. However, the statistical manipulation of outcome measures made the study difficult to interpret. Helicobacter treatment studies A total of 17 studies were identified; however, only in 11 papers was it possible to dichotomize the change in dyspepsia symptoms. In contrast with H 2 -receptor antagonists and prokinetic therapy, the difference in effect size between high and low Jadad scores was much less marked (Figure 5). Amongst the good quality studies, there was a small therapeutic benefit favouring eradication therapy (RR, 0.93; 95% CI, ), with poor quality studies also showing a statistically significant benefit (RR, 0.87; 95% CI, ). When the Rome criteria were used to define methodological quality (Figure 6), all the studies were classified as good quality (i.e. Rome quality score of > 0). When the studies were divided post hoc into trials fulfilling up to two Rome criteria compared with those fulfilling all three criteria, there was no difference in effect size between these categories (Figure 6), with an overall effect size consistent with a small but statistically significant effect in favour of H. pylori eradication (RR, 0.91; 95% CI, ). Psychotherapy Four studies were identified evaluating complementary psychological therapy as the primary intervention, but only two were eligible for abstraction. These two studies were qualitatively reviewed. Hamilton et al. randomized 96 patients to either standardized supportive therapy in the control arm or psychodynamic interpersonal therapy given over a 12-week period. 17 Patients randomized to psychodynamic interpersonal therapy underwent a 3-h first session of therapy followed by six sessions of 50 min over the 12-week period. A global assessment of severity of dyspepsia symptoms was used, and the primary outcome was the patient s self-rating of their dyspeptic symptoms measured on a five-point Likert scale. Overall psychological status was assessed using the Global Symptom Index using the SCL-90-R. 24 Intention-to-treat analysis showed no significant difference when the two treatment arms were compared at 12 weeks and 1 year. The study allowed the enrolment of patients with GERD symptoms, and a post hoc analysis excluding patients who had a dominant symptom of heartburn demonstrated that psychodynamic therapy was superior to the control group. No definition of a treatment responder was given. The main outcome measure was not validated, but the face validity of the measure was high. Calvert et al. examined the impact of hypnotherapy on dyspepsia symptoms. 18 In this study, 126 patients were randomized to one of three treatment arms: hypnotherapy (n ¼ 32), supportive therapy (n ¼ 48) and conventional treatment with ranitidine (n ¼ 46). Those receiving hypnotherapy or supportive therapy underwent twelve 30-min visits over a 16-week treatment phase. Those who received conventional treatment presented four times for the dispensing of medications. Patients were assessed with a symptom score using a visual analogue scale previously validated in an irritable bowel syndrome population. Response was measured by calculating the percentage change after treatment compared with baseline. An a priori definition of treatment response was not provided. Hypnotherapy was superior in both the short-term and long-term relief of dyspeptic symptoms. In the short term, symptom scores improved by 59% for those undergoing hypnotherapy vs. 41% for those given supportive therapy (P ¼ 0.01) and 33% for those given medical treatment (P ¼ 0.057). At 56 weeks, hypnotherapy improved symptoms by 73% vs. 34% in supportive therapy (P < 0.02) and only 20 percent; in the medical arm (P < 0.001). DISCUSSION Before 1995, most functional dyspepsia treatment studies suffered from important methodological weaknesses, making it difficult to make definitive statements about the efficacy of treatments. Another serious problem in trying to perform a formal meta-analysis in this area is that the differences in outcome measures utilized make statistical pooling difficult, if not impossible, due to study heterogeneity. One of the key findings of this meta-analysis is that the quality of the included studies has an impact on the estimate of the overall treatment effect size. We showed that, by including studies of lower quality assessed by the Jadad score, 26 the effect size was inflated. This was most evident

8 638 N. S. ABRAHAM et al. for cisapride studies, but an effect was also seen for H 2 -blockers. Moher et al., in their re-evaluation of 127 randomized controlled trials published between 1960 and 1995, abstracted in meta-analyses, detected a clinically significant 30 50% over-estimation of treatment efficacy when the results of lower quality trials were included. 25 Although quality assessment of study methodology adds another layer of complexity to the systematic review process, their results suggest that an incorporation of an estimate of study quality of randomized controlled trials is important. Schulz et al. came to a similar conclusion in their methodological review of studies included in meta-analyses in obstetrics and gynaecology. 60 The addition of Jadad scores makes weaknesses in study design and execution explicit. We also believe that the three additional criteria, which were highlighted in the Rome II Consensus Report on Study Design, are a useful additional assessment for high quality functional dyspepsia studies. These are: (i) a clear description of how patients with possible GERD are handled in the inclusion and exclusion criteria of the study; (ii) the use of validated outcome measures; and (iii) the need to define a priori what constitutes a treatment success, i.e. a definition of what amount of improvement is considered clinically meaningful. If functional dyspepsia treatment studies addressed these three key issues in their design, the overall quality of the studies would markedly improve. This is the first attempt to validate the Rome criteria for functional dyspepsia study quality. Both the Jadad score and the Rome quality criteria influenced the estimation of the effect size, with lower quality studies causing an over-estimation of the aggregate scores. We believe that our selection of the three key Rome II methodological criteria may provide a reproducible disease-specific methodological quality assessment tool to evaluate functional dyspepsia studies. A limitation of our study is that few of the published studies abstracted met the cut-off for a good quality study according to the Rome II methodological criteria. This is probably a reflection of publication prior to the Rome II methodological recommendations in To ensure the operation of the Rome II methodological recommendations as a quality assessment tool, further prospective validation is required. Furthermore, we do not believe that our choice of cutoff was completely arbitrary. Rather, it was developed following consensus discussion with international experts in dyspepsia to determine the Rome methodological criteria that best addressed the current perceived weaknesses of the published literature. These diseasespecific Rome criteria may provide useful additional information on trial quality. They may, for example, clarify why both good and poor quality trials evaluating H. pylori eradication gave a similar effect size. The Rome criteria suggested that all trials were of reasonable quality, despite variations in Jadad score, and this could explain the relatively homogeneous results. Of all the evaluated treatments, the best evidence is for treatment with a proton pump inhibitor, based on the combined Bond and Opera studies. 10 However, the recent well-designed study using lansoprazole 30 mg and 15 mg by Wong et al. was negative for all outcome measures. 11 This may be explained by the low prevalence of GERD in Hong Kong where the study was carried out. In a Chinese population, there may not be a problem with contamination by unrecognized GERD patients. Indeed, there is evidence that the prevalence of reflux oesophagitis is low in Chinese patients. Wong et al. reported a prevalence of 3.8% in over endoscopies. 61 It is possible that the presence of heartburn in addition to epigastric pain is a predictor of response. In the Bond and Opera studies, there was a suggestion that response was primarily seen in patients with associated heartburn. Currently, there are too few trials of proton pump inhibitor therapy in functional dyspeptics to permit a thorough quantitative assessment of treatment efficacy in this sub-group of patients. This is an important question for future research. There is equivocal evidence in favour of H 2 -blockers. Most studies suffered from small sample sizes and a lack of use of validated outcome measures. Another important weakness was that, generally, low doses of H 2 -blockers were studied. Larger studies are needed which should test higher doses of medication. As is the case with proton pump inhibitors, it certainly is possible that heartburn (but not the dominant symptom) is a driver of response to an H 2 -blocker. As has been highlighted in previous systematic reviews on cisapride, many of the studies suffered from important flaws in design and execution, making it difficult to make definitive statements about efficacy. Our results clearly show that the magnitude of the estimated effect size is inversely related to study quality. Due to rare but

9 SYSTEMATIC REVIEW: QUALITY AND EFFICACY IN NUD TREATMENT STUDIES 639 serious cardiac side-effects, cisapride is no longer available in most countries. Whether the cure of H. pylori infection leads to a sustained improvement in symptoms in Helicobacterpositive functional dyspepsia patients has been controversial for the past 10 years. The two recent meta-analyses by Moayyedi et al. and Laine et al. came to opposite conclusions. 27, 28 Our data suggest that the reason for the differences between these two systematic reviews does not relate to trial quality, as this did not dramatically influence the effect size, but may relate to the overall numbers of patients included. Moayyedi et al. included more trials and patients and therefore had a greater power to detect the small treatment effect. 27 The combined analysis of all studies and the analysis of high quality studies both showed a small benefit of H. pylori treatment. Of the high quality studies, only the study by McColl et al. 12 clearly gave a positive result in favour of anti-helicobacter treatment. One possible explanation for this positive finding is the underlying high background ulcer prevalence in Scotland. 59 Based on this systematic review and previous analyses, there is evidence that H. pylori eradication will benefit functional dyspepsia patients, but the effect size is likely to be small. However, given the fact that H. pylori is a true pathogen, and may also be a co-factor in the development of gastric cancer, it is reasonable to treat Helicobacter pylori-positive functional dyspepsia patients. Of the two studies using psychotherapy, the study by Hamilton et al. favoured treatment with psychotherapy, but this was only seen in a post hoc analysis that excluded patients with dominant symptoms of heartburn. 17 A disadvantage of psychotherapy is that it is time consuming and hence expensive. The study by Calvert et al. suggested that there is a benefit of hypnotherapy in functional dyspepsia. 18 Both psychological modalities require future studies. CONCLUSION The lack of methodological coherence in studies published prior to 1995 makes it difficult to make definitive statements about the efficacy of treatments. The publication of the Rome II recommendations for the design of treatment trials for functional gastrointestinal disorders should result in an improvement in methodological quality in future trials. 2 Further refinement of study design criteria should be encouraged in functional dyspepsia studies, which should incorporate the tenets of the Jadad score 26 and the use of validated outcome measurements, the exclusion of heartburn/gerd-dominant patients (particularly if acid suppression therapy is being evaluated) and a clear a priori clinical definition of expected treatment response. 2 The examination of both generic (Jadad score) and disease-specific (Rome II) methodological quality indicators is helpful in assessing the true magnitude of treatment effect in clinical trials of functional dyspepsia treatment. ACKNOWLEDGEMENTS This meta-analysis was commissioned by EBMed LLC with funding received from Novartis Pharmaceuticals. Dr Van Zanten has received financial support/honoraria and speaker s fees from Astra Zeneca, Abbott Laboratories, Glaxo Wellcome Smith Kline and Janssen Pharmaceuticals. Dr Moayeddi has received speaker s and consultancy fees from Astra Zeneca, Wyeth, Takeda and Abbott Pharmaceuticals. REFERENCES 1 Talley NJ, Stanghellinin V, Heading RC, Koch KL, Malageleda JR, Tytgat GNJ. Functional gastroduodenal disorders. Gut 1999; 45(Suppl. II): Veldhuyzen van Zanten SJO, Talley NJ, Bytzer P, Klein KB, Whorwell PJ, Zinsmeister AR. Design of treatment trials for functional gastrointestinal disorders. Gut 1999; 45(Suppl. II): Veldhuyzen van Zanten SJ, Flook N, Chiba N, et al. An evidence-based approach to the management of uninvestigated dyspepsia in the era of Helicobacter pylori. Canadian Dyspepsia Working Group. CMAJ 2000; 162(12 Suppl.): S Moayyedi P, Feltbower R, Brown J, et al. Effect of population screening and treatment for Helicobacter pylori on dyspepsia and quality of life in the community: a randomised controlled trial. Leeds HELP Study Group. Lancet 2000; 355(9216): Hansen JM, Bytzer P, Schaffalitzky de Muckadell OB. Placebocontrolled trial of cisapride and nizatidine in unselected patients with functional dyspepsia. Am J Gastroenterol 1998; 93(3): Chen JD, Ke MY, Lin XM, Wang Z, Zhang M. Cisapride provides symptomatic relief in functional dyspepsia associated with gastric myoelectrical abnormality. Aliment Pharmacol Ther 2000; 14(8): Holtmann G, Schossman J, Mayr P, Talley NJ. A randomized placebo-controlled trial of simethicone and cisapride for the treatment of patients with functional dyspepsia. Aliment Pharmacol Ther 2002; 16(9):

10 640 N. S. ABRAHAM et al. 8 Bekhti A, Rutgeerts L. Domperidone in the treatment of functional dyspepsia in patients with delayed gastric emptying. Postgrad Med J 1979; 55(Suppl. 1): Van Outryve M, De Nutte N, Van Eeghem P, Gooris JP. Efficacy of cisapride in functional dyspepsia resistant to domperidone or metoclopramide: a double-blind, placebo-controlled study. Scand J Gastroenterol Suppl 1993; 195: 47 52; discussion Talley NJ, Meinech-Schmidt V, Pare P, et al. Efficacy of omeprazole in functional dyspepsia: double-blind, randomized, placebo-controlled trials (the Bond and Opera studies). Aliment Pharmacol Ther 1998; 12(11): Wong WM, Wong BCY, Hung WK, et al. Double blind, randomized, placebo controlled study of four weeks of lansoprazole for the treatment of functional dyspepsia in Chinese patients. Gut 2002; 51: McColl K, Murray L, El-Omar E, et al. Symptomatic benefit from eradicating Helicobacter pylori infection in patients with nonulcer dyspepsia. N Engl J Med 1998; 339(26): Blum AL, Talley NJ, O Morain C, et al. Lack of effect of treating Helicobacter pylori infection in patients with nonulcer dyspepsia. Omeprazole plus Clarithromycin and Amoxicillin Effect One Year after Treatment (OCAY) Study Group. N Engl J Med 1998; 339(26): Talley NJ, Janssens J, Lauritsen K, Racz I, Bolling-Sternevald E. Eradication of Helicobacter pylori in functional dyspepsia: randomised double blind placebo controlled trial with 12 months follow up. The Optimal Regimen Cures Helicobacter Induced Dyspepsia (ORCHID) Study Group. Br Med J 1999; 318(7187): Talley NJ, Vakil N, Ballard ED 2nd, Fennerty MB. Absence of benefit of eradicating Helicobacter pylori in patients with non-ulcer dyspepsia. N Engl J Med 1999; 7(34): Froehlich F, Gonvers JJ, Wietlisbach V, et al. Helicobacter pylori eradication treatment does not benefit patients with nonulcer dyspepsia. Am J Gastroenterol 2001; 96(8): Hamilton J, Guthrie E, Creed F, et al. A randomized controlled trial of psychotherapy in patients with chronic functional dyspepsia. Gastroenterology 2000; 119(3): Calvert EL, Houghton LA, Cooper P, Morris J, Whorwell PJ. Long-term improvement in functional dyspepsia using hypnotherapy. Gastroenterology 2002; 123: Finney JS, Kinnersley N, Hughes M, O Bryan-Tear CG, Lothian J. Meta-analysis of antisecretory and gastrokinetic compounds in functional dyspepsia. J Clin Gastroenterol 1998; 26: Dobrilla G, Comberlato M, Steele A, Vallaperta P. Drug treatment of functional dyspepsia. A meta-analysis of randomized controlled clinical trials. J Clin Gastroenterol 1989; 11: Redstone HA, Barrowman N, Veldhuyzen van Zanten SJO. H 2 -receptor antagonists in the treatment of functional (nonulcer) dyspepsia: a meta-analysis of randomized controlled clinical trials. Aliment Pharmacol Ther 2001; 15: Bytzer P. H 2 receptor antagonists and prokinetics in dyspepsia: a critical review. Gut 2002; 50(Suppl. 4): iv: Veldhuyzen van Zanten SJ, Jones MJ, Verlinden M, Talley NJ. Efficacy of cisapride and domperidone in functional (nonulcer) dyspepsia: a meta-analysis. Am J Gastroenterol 2001; 96: Soo S, Moayyedi P, Deeks J, Delaney B, Innes M, Forman D. Pharmacological interventions for non-ulcer dyspepsia. Cochrane Database Syst Rev 2000; 2: CD Moher D, Jones A, Cook DJ, et al. Does quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352: Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17: Moayyedi P, Soo S, Deeks J, et al. Eradication of Helicobacter pylori for non-ulcer dyspepsia. Cochrane Database Syst Rev 2001; 1: CD Laine L, Schoenfeld P, Fennerty MB. Therapy for Helicobacter pylori in patients with nonulcer dyspepsia. A meta-analysis of randomized, controlled trials. Ann Intern Med 2001; 134(5): Veldhuzyzen van Zanten SJO, Cleary C, Talley NJ, et al. Drug treatment of functional dyspepsia: a systematic analysis of trial methodology with recommendations for design of future trials. Am J Gastroenterol 1996; 91(4): Section Fixed effect versus random effects analysis. Cochrane Reviewers Handbook. Available at v.4.1.6, January Delattre M, Malesky M, Prinzie A. Symptomatic treatment of non-ulcer dyspepsia with cimetidine. Curr Ther Res 1985; 37(5): Gotthard R, Bodemar G, Brodin U, Jonsson KA. Treatment with cimetidine, antacid, or placebo in patients with dyspepsia of unknown origin. Scand J Gastroenterol 1988; 23(1): Hadi S. Clinical investigation of ranitidine in patients with gastritis. Clin Ther 1989; 11(5): Kelbaek H, Linde J, Eriksen J, Mungaard S, Moesgaard F, Bonnevie O. Controlled clinical trial of treatment with cimetidine for non-ulcer dyspepsia. Acta Med Scand 1985; 217(3): Mackinnon M, Willing RL, Whitehead R. Cimetidine in the management of symptomatic patients with duodenitis: a double-blind controlled trial. Dig Dis Sci 1982; 27(3): Muller P, Hotz J, Franz E, Simon B. Ranitidine in the treatment of non-ulcer dyspepsia. A placebo-controlled study in the Federal Republic of Germany. Arzneim-Forsch/Drug Res 1994; 10: Nesland AA, Berstad A. Effect of cimetidine in patients with non-ulcer dyspepsia and erosive prepyloric changes. Scand J Gastroenterol 1985; 20(5): Olubuyide OA, Atoba MA. Non-ulcer dyspepsia in Nigerians: clinical and therapeutic results. Scand J Gastroenterol 1986; 21(Suppl. 124): 83 7.

11 SYSTEMATIC REVIEW: QUALITY AND EFFICACY IN NUD TREATMENT STUDIES Singal AK, Kumar A, Broor SL. Cimetidine in the treatment of non-ulcer dyspepsia: results of a randomized double-blind, placebo-controlled study. Curr Med Res Opin 1989; 11(6): Blum AL, Arnold R, Stolte M, Fischer M, Koelz HR. Short course acid suppressive treatment for patients with functional dyspepsia: results depend on Helicobacter pylori status. Gut 2000; 47: Saunders JH, Oliver RJ, Higson DL. Dyspepsia: incidence of a non-ulcer disease in a controlled trial of ranitidine in general practice. Br Med J Clin Res Ed 1994; 1986(6521): Al-Quorain A, Larbi EB, al-shedoki F. A double-blind, randomized, placebo-controlled trial of cisapride in Saudi Arabs with functional dyspepsia. Scand J Gastroenterol 1995; 30(6): Champion MC, MacCannell KL, Thomson ABR, et al. A doubleblind randomized study of cisapride in the treatment of nonulcer dyspepsia. Can J Gastroenterol 1997; 11(2): Chung JM. Cisapride in chronic dyspepsia: results of a doubleblind, placebo controlled trial. Scand J Gastroenterol 1993; 195(Suppl.): De Groot GH, De Both PSM. Cisapride in functional dyspepsia in general practice. A placebo controlled, randomized double blind study. Aliment Pharmacol Ther 1997; 11: De Nutte N, Van Ganse W, Witterhulghe M, Defrance P. Relief of epigastric pain in nonulcer dyspepsia: controlled trial of the promotility drug cisapride. Clin Ther 1989; 11(1): Francois I, De Nutte N. Nonulcer dyspepsia: effect of the gastrointestinal prokinetic drug cisapride. Curr Ther Res 1987; 41(6): Hannon R. Efficacy of cisapride in patients with nonulcer dyspepsia. A placebo-controlled study. Curr Ther Res 1987; 42(5): Kellow JE, Cowan H, Shuter B, et al. Efficacy of cisapride therapy in functional dyspepsia. Aliment Pharmacol Ther 1995; 9(2): Rosch W. Cisapride in non-ulcer dyspepsia. Results of a placebo-controlled trial. Scand J Gastroenterol 1987; 22(2): Wood SF, Penney SC, Cochran KM. Cisapride in functional dyspepsia: a double-blind, placebo-controlled randomized trial in general practice patients. Scand J Gastroenterol 1993; 28(Suppl. 195): Yeoh KG, Kang JY, Tay HH, et al. Effect of cisapride on functional dyspepsia in patients with and without histological gastritis: a double-blind placebo-controlled trial. J Gastroenterol Hepatol 1997; 12(1): Bruley des Varannes S, Flejou JF, Colin R, Zaim M, Meunier A, Bidaut-Mazel C. There are some benefits for eradicating Helicobacter pylori in patients with non ulcer dyspepsia. Aliment Pharmacol Ther 2001; 15: Gisbert JP, Cruzado AL, Garcia-Gravalos R, Pajares JM. Lack of benefit of treating Helicobacter pylori infection in patients with functional dyspepsia. Randomized one-year follow-up study. Hepato-gastroenterology 2003 in press. 55 Hsu PL, Lai KH, Tseng HH, et al. Eradication of Helicobacter pylori prevents ulcer development in patients with ulcer-like functional dyspepsia. Aliment Pharmacol Ther 2001; 15: Koskenpato J, Farkkila M, Sipponen P. Helicobacter pylori eradication and standardized 3-month omeprazole therapy in functional dyspepsia. Am J Gastroenterol 2001; 96: Miwa H, Hirai S, Nagahara A, et al. Cure of Helicobacter pylori infection does not improve symptoms in non-ulcer dyspepsia patients a double-blind placebo-controlled study. Aliment Pharmacol Ther 2000; 14: Koelz HR, Arnold R, Stolte M, Fischer M, Blum AL, the FROSCH Study Group. Treatment of Helicobacter pylori in functional dyspepsia resistant to conventional management: a double blind randomized trial with a six month follow up. Gut 2003; 52: McColl KE, el-nujumi A, Murray L, et al. The Helicobacter pylori breath test: a surrogate marker for peptic ulcer disease in dyspeptic patients. Gut 1998; 40(3): Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. J Am Med Assoc 1995; 273: Wong WM, Lam SK, Hui WM, et al. Long-term prospective follow-up of endoscopic oesophagitis in southern Chinese prevalence and spectrum of the disease. Aliment Pharmacol Ther 2002; 16: