Validation of a Four-Graded Scale for Severity of Heartburn in Patients with Symptoms of Gastroesophageal Reflux Disease

Volume 11 Number 4 2008 VALUE IN HEALTH Validation of a Four-Graded Scale for Severity of Heartburn in Patients with Symptoms of Gastroesophageal Reflux Disease Ola Junghard, PhD, 1 Ingela Wiklund, PhD 2 * 1 Biostatistics, AstraZeneca R&D, Mölndal, Sweden; 2 Outcomes Research, AstraZeneca R&D, Mölndal, Sweden ABSTRACT Objectives: To evaluate the reliability, responsiveness, and validity of a four-graded symptom severity scale (none, mild, moderate, severe) and an eight-graded frequency scale (number of days with symptoms during past week) for assessment of heartburn severity and frequency in patients with symptoms of heartburn but without esophagitis. Methods: Data were taken from two 4-week clinical trials comparing esomeprazole 20 mg and/or 40 mg to omeprazole 20 mg. Both scales were analyzed in terms of mean scores and treatment success rates, after dichotomization to treatment success variables. Results: Heartburn severity reliability was higher when assessed by patient diary cards than by the investigator (intraclass correlation coefficients were 0.79 and 0.59, respectively; corresponding figures for heartburn frequency were 0.77 and 0.78). There was good agreement between investigator and diary card assessments for the dichotomized variables complete resolution and adequate relief from heartburn. High correlation between investigator- and diary card-assessed heartburn severity and frequency was apparent. Responsiveness was high, as shown by comparing 4-week treatment success rates to the patients perception of treatment effect according to the Overall Treatment Effect questionnaire. Construct validity was good (kappa values approximately 0.70 for agreement between complete resolution and the dichotomized Gastrointestinal Symptom Rating Scale [GSRS] Heartburn item). Correlations between heartburn severity and frequency and the GSRS Heartburn item were similarly high. Conclusion: The four-graded heartburn severity and eightgraded frequency scales are reliable, responsive, and valid when used in clinical trials of patients with symptoms of gastroesophageal reflux disease, irrespective of the type of assessment (investigator assessment or patient diary cards). Keywords: four-graded scale, gastroesophageal reflux disease, heartburn, validation. Introduction Resolution of troublesome symptoms is one of the primary treatment goals in the management of patients with endoscopy-negative gastroesophageal reflux disease (GERD) [1]. The major symptoms of GERD assessed in clinical trials are heartburn and acid regurgitation [2]. Of these, heartburn is by far the most common and is generally regarded as being the hallmark symptom of GERD. For patients with endoscopy-negative GERD, heartburn is considered to be the most relevant outcome measure [1]. In order to assess heartburn severity in clinical trials, a fourgraded scale has often been used [3 5] both by investigators and by patients using daily diary cards. Treatment comparisons are sometimes based on a difference in mean severity score, but more often they are Address correspondence to: Ola Junghard, Biostatistics, Astra- Zeneca R&D, Pepparedsleden 1, 431 83 Mölndal, Sweden. E-mail: Ola.Junghard@astrazeneca.com 10.1111/j.1524-4733.2007.00313.x *At the time this study was completed, Professor Wiklund was affiliated with AstraZeneca R&D, Mölndal, Sweden. Professor Wiklund is currently affiliated with Global Health Outcomes, GlaxoSmithKline, Greenford, UK. based on a difference in treatment success rate, where the definition of treatment success is a dichotomization of the four-graded scale. For example, treatment success may be defined as complete resolution of heartburn (no heartburn during the last 7 days) before a certain clinical visit or time point, or as adequate relief of heartburn (at most 1 day of mild heartburn during the last 7 days), and may be based on either the investigator s assessment [4] or patient diary cards [5]. A difference in treatment success rate is easier to understand than a difference in mean scores, but may be less efficient in terms of statistical power. The aim of this article is to validate outcome variables derived from the four-graded severity scale and from an eight-graded frequency scale, in terms of reliability, responsiveness, and validity. For the assessment of both heartburn severity and frequency, a 7-day reference period was used. Methods Data presented here relate to two clinical studies [6] in patients with heartburn but without endoscopic findings of erosive esophagitis according to the Los 2008, International Society for Pharmacoeconomics and Outcomes Research (ISPOR) 1098-3015/08/765 765 770 765

766 Junghard and Wiklund Table 1 Baseline demographic and clinical characteristics of the study populations Characteristic Angeles classification (grades A D). These doubleblind, randomized, 4-week studies were of almost identical design; they differed only in terms of location and treatments. One study (study A), performed in the United Kingdom, Ireland, and Canada, included esomeprazole 40 mg, esomeprazole 20 mg, and omeprazole 20 mg, and the other, performed in Scandinavia (study B), included esomeprazole 20 mg and omeprazole 20 mg. Baseline demographics for the two studies are presented in Table 1. All patients had a history of heartburn as their main GERD symptom for at least 6 months, and had experienced heartburn for at least 4 days during the week before the baseline visit. They had all identified their main symptom as heartburn, defined as a burning feeling rising from the stomach or lower part of the chest up toward the neck. Exclusion criteria included a significant condition likely to affect the outcome of the study and use of a H 2-receptor antagonist or proton pump inhibitor in the 14 or 28 days, respectively, before the baseline endoscopy. Assessments Study A patients Study B patients All patients included Excluded patients N 871 539 1410 542 Males, n (%) 378 (43) 281 (52) 659 (47) 237 (44) Mean age (SD), year 49 (13) 49 (14) 49 (14) 47 (15) History of heartburn episodes, n (%) <6 months 1 (0) 0 1 (0) 1 (0) 6 12 months 113 (13) 55 (10) 168 (12) 69 (13) 1 5 years 336 (39) 151 (28) 487 (35) 216 (40) >5 years 421 (48) 333 (62) 754 (53) 256 (47) Overall heartburn severity in previous 7 days, n (%) None 1 (0) 0 1 (0) 0 Mild 197 (23) 100 (19) 297 (21) 120 (22) Moderate 526 (60) 347 (64) 873 (62) 321 (59) Severe 147 (17) 92 (17) 239 (17) 101 (19) Gastrointestinal Symptom Rating Scale (GSRS). At baseline and after 2 and 4 weeks of treatment, patients answered the validated GSRS questionnaire [7], with questions relating to the previous 7 days. The GSRS uses a seven-graded Likert scale to assess symptom severity: No discomfort at all, Minor, Mild, Moderate, Moderately severe, Severe, or Very severe discomfort. It consists of 15 items, one of which is related to heartburn and worded Have you been bothered by HEARTBURN during the past week? (By heartburn we mean an unpleasant stinging or burning sensation in the chest.) All items are phrased in the same way, grading the discomfort patients experience as a result of various gastrointestinal symptoms such as nausea, bloating, and diarrhea. The items are mapped into five dimensions: Diarrhea, Indigestion, Constipation, Abdominal pain, and Reflux, where the Reflux dimension consists of the Heartburn item and a Regurgitation item. The GSRS is used for the evaluation of validity of the examined heartburn scales. Overall Treatment Effect (OTE). After 2 and 4 weeks of treatment, patients answered the OTE questionnaire. They were asked whether their reflux symptoms (heartburn and/or regurgitation) had improved, worsened, or remained unchanged. Patients then rated the degree of the improvement or worsening since start of treatment. For patients whose symptoms had improved, the categories of response were as follows: OTE scale A very great deal A great deal A good deal Moderately Somewhat A little Almost the same, hardly at all Unchanged For analysis A very great deal A great deal A good deal Somewhat Unchanged The corresponding categories were used for patients with worsened symptoms. For the purpose of analyzing responsiveness, some categories were combined (as shown above) and all worse categories were combined into one (due to the small number of cases). Although the OTE refers to both heartburn and regurgitation, heartburn is the dominant symptom. At baseline, 2 weeks, and 4 weeks, 63%, 41%, and 28%, respectively, had more severe heartburn than regurgitation, and 32%, 51%, and 64%, respectively, had the same severity of heartburn and regurgitation. Thus, we assumed that the OTE primarily measures the perceived change in heartburn. Both the GSRS and the OTE questionnaires were completed during the visits to the clinic before any other assessments were performed. The four-graded scale. The investigator assessments of patients overall heartburn severity and heartburn frequency (number of days with heartburn) were made at each clinical visit (at baseline and after 2 and 4 weeks of treatment) with reference to the 7 days before the visit. Patients themselves assessed and recorded their heartburn severity on a diary card. Each day, patients recorded the severity of the most severe heartburn episode during the night (to be recorded in the morning) and in the daytime (to be recorded before bedtime), throughout the 4-week treatment period. In order to have diary recordings comparable to investigator assessments, only recordings of daytime heartburn were used in the present analysis. Both assessments used the four-graded severity scale:

Four-Graded Heartburn Severity Scale Validation 767 None = 0 Mild = 1 Moderate = 2 no symptoms awareness of symptom, but easily tolerated discomfort sufficient to cause interference with normal activities Severe = 3 incapacitating, with inability to perform normal activities. Two treatment success variables were defined using dichotomization of the severity and frequency assessments: complete resolution (no heartburn during the previous 7 days) and adequate relief (at most 1 day with heartburn rated mild during the previous 7 days). These definitions were applied to both investigator and patient diary card assessments. In total, eight variables were examined: the binary treatment success variables complete resolution and adequate relief, and the ordinal variables heartburn severity and heartburn frequency, based on investigator assessment and diary cards completed by the patient. The ordinal variable heartburn severity has four grades (with scores given above) when assessed by the investigator and is the mean score over the last 7 days when based on diary cards. Statistical Analysis Responsiveness. For investigator-assessed heartburn severity and frequency, standardized response means were calculated as the difference between the baseline and 4-week values divided by the standard deviation of the change. Effect sizes were calculated as the difference between baseline and 4-week values divided by the standard deviation at baseline [8]. Effect sizes and standardized response means were also calculated for each OTE category. There were no patient diary assessments before treatment and thus standardized response means and effect sizes could not be calculated for the corresponding diary-based ordinal variables. For diary-based ordinal variables and treatment success variables, responsiveness was evaluated at 4 weeks using the OTE questionnaire categories as an anchor. In this evaluation, treatment success rates for the variables diary-based mean heartburn severity and mean heartburn frequency were calculated for each OTE category and the results presented graphically. If the outcome measure is responsive to change, effect sizes for patients who were according to the OTE should be larger than for patients who were worse or unchanged. Consequently, patients a very great deal should have the largest effect size. Similarly, mean heartburn severity and frequency at 4 weeks should be highest for patients who were worse or unchanged and lowest for those being a very great deal. In the same way, for dichotomized variables, the treatment success rate should be higher for those who are than for those who are unchanged or worse. Reliability. The ability of the heartburn variables to yield reproducible results was assessed using the test retest approach, where patients in a stable condition repeatedly assess their symptoms. In this analysis of reliability, we used assessments made at 2 and 4 weeks in patients who had the same answer to the OTE questionnaire at these time points. Reliability for the ordinal variables was evaluated by estimating intraclass correlation coefficients (ICCs). For the dichotomized outcome variables complete resolution and adequate relief, reliability was evaluated in terms of kappa values. The ICCs were calculated from an anova with visit and subject as factors [9]. The correlation between investigators assessment of heartburn and the assessment made by patient diary cards may also serve as a measure of reliability. These two assessments measure the same symptom during the same time period and thus fluctuation in the patient s health status has no impact. Validity. Construct validity is an assessment of the degree to which an instrument or scale measures what it is designed to measure [8]. In this case, the heartburn assessments under investigation were correlated with some other assessments of heartburn. Thus, when using GSRS as the comparator, the Heartburn item of the GSRS questionnaire was preferred to the reflux dimension scale. Construct validity of the ordinal variables was assessed by correlating the heartburn severity and frequency variables with the Heartburn item on the GSRS questionnaire, using Pearson correlation analysis. The validity of the dichotomized treatment success variable complete resolution of heartburn was assessed in terms of kappa values. For the latter evaluation the GSRS Heartburn item was dichotomized with treatment success defined as No discomfort at all. As the GSRS item does not separate severity from frequency, the dichotomized variable adequate relief has no direct dichotomized GSRS comparator. Instead, the validity of adequate relief was examined by correlating it with the original (not dichotomized) GSRS Heartburn item. In this examination, Pearson correlation coefficients were used as a substitute for kappa values. Results Of the 1952 patients in the intention-to-treat population, complete data for the OTE questionnaire and investigator- and diary card-assessed heartburn were available for 1410 patients. Baseline characteristics did not reveal any relevant differences between the two studies included in the analysis (Table 1). Most exclusion from the analysis was due to incomplete diary

768 Junghard and Wiklund Patients (%) 100 80 60 40 Inv. ass. complete resolution Inv. ass. adequate relief Diary ass. complete resolution Diary ass. adequate relief 91.6 90.6 87.2 84.1 20 0 Worse 18.2 11.7 16.2 13.0 Unchanged Somewhat A good deal A great deal A very great deal Figure 1 Proportion of patients achieving complete resolution or adequate relief of investigator- and diary-assessed heartburn (n = 1410) by the Overall Treatment Effect classification after 4 weeks of treatment. card recordings, and the key demographic variables for the excluded patients were essentially the same as for those included in this analysis (Table 1). Responsiveness Both complete resolution and adequate relief were responsive, as shown by the comparison of 4-week treatment success rates with the patients perception of treatment effect using the OTE questionnaire (Fig. 1). The responsiveness of diary card-assessed heartburn severity and frequency is illustrated graphically in Figure 2. Table 2 presents effect sizes and standardized response means by OTE classification for the investigator-assessed heartburn severity and frequency scales. Reliability. Out of the 1410 evaluable patients, 707 had the same answer to the OTE questionnaire at weeks 2 and 4 and were included in the evaluation of test retest reliability. The reliability of heartburn severity was higher when assessed by patient diary cards than when assessed by the investigator (ICC values of 0.79 and 0.59, respectively). The corresponding figures for heartburn frequency were 0.77 and 0.78. For investigator- and diary card-assessed heartburn, kappa values for the dichotomized variable complete resolution were 0.52 and 0.57, respectively, and for adequate relief kappa values were 0.59 for both. Measuring agreement between investigator- and diary card-assessments, kappa values were 0.76 with regard to both complete resolution and adequate relief from heartburn. Investigator assessment gave a slightly higher overall proportion with complete resolution and adequate relief than patient-recorded diaries (Table 3). The correlation between the investigator- and diary card-assessed heartburn severity was 0.71; for heartburn frequency the correlation was 0.85. Validity. The dichotomized variable complete resolution showed good agreement with the dichotomized GSRS Heartburn item after 4-week treatment (kappa values for investigator- and diary card-assessed heartburn were 0.70 and 0.69, respectively). Correlations between the ordinal heartburn severity variable and the GSRS Heartburn item were 0.72 for investigator- Score and number of days 6 5 4 3 2 1 0 5.31 1.29 Worse 4.88 1.02 3.78 Unchanged Somewhat 2.61 0.67 0.45 A good deal 1.68 Mean severity Mean frequency 0.28 A great deal 0.32 0.05 A very great deal Figure 2 Diary-assessed mean heartburn severity score and mean number of days with heartburn during the last week, according to the Overall Treatment Effect classification after 4 weeks of treatment (n = 1410).

Four-Graded Heartburn Severity Scale Validation 769 Table 2 Change, SRM, and effect size for investigator-assessed heartburn severity during the previous 7 days from baseline to after 4 weeks of treatment, according to the OTE classification OTE classification n Investigator-assessed overall severity change Investigator-assessed frequency change Change SRM Effect size Change SRM Effect size Worse 29 0.17 0.19 0.25 0.83 0.38 0.70 Unchanged 154 0.67 0.76 0.96 1.31 0.51 1.06 Somewhat 67 0.96 1.19 1.36 2.55 0.98 1.88 A good deal 203 1.19 1.42 1.90 3.75 1.47 2.94 A great deal 276 1.44 1.75 2.41 4.62 2.07 3.77 A very great deal 681 1.87 2.68 3.19 5.84 3.93 4.83 All 1410 1.48 1.65 2.40 4.55 1.77 3.69 OTE, Overall Treatment Effect; SRM, standardized response mean. assessed heartburn and 0.71 for diary card-assessed heartburn at 4 weeks. The correlation between heartburn frequency and the GSRS Heartburn item was similar: 0.68 for investigator-assessed heartburn and 0.67 for diary card-assessed heartburn. The correlation between diary-based adequate relief and the GSRS Heartburn item was 0.62; the corresponding value for investigator-based adequate relief was 0.64. Discussion The development of the GSRS started in 1988, when a rating scale comprising 15 interview-based questions was proposed for a comprehensive evaluation of treatment effects [10]. This rating scale was later modified to become a self-administered questionnaire with three dimensions. The current subgrouping of the items into five dimensions [11] has been validated with regard to validity, reliability, and responsiveness [7]. Nevertheless, the GSRS does not distinguish between severity and frequency and thus the examined variables can give additional information about the symptomatology. The relationship between heartburn severity/ frequency and the perception that the medication gives sufficient control of heartburn was studied in a trial of omeprazole in patients with endoscopy-negative Table 3 Cross-tabulation of diary- and investigator-assessed complete resolution and adequate relief Investigator-assessed No Yes All n % n % N % Complete resolution Diary-assessed No 494 35.0 120 8.5 614 43.5 Yes 45 3.2 751 53.3 796 56.5 All 539 38.2 871 61.8 1410 100.0 Adequate relief Diary-assessed No 406 28.8 98 7.0 504 35.7 Yes 55 3.9 851 60.4 906 64.3 All 461 32.7 949 67.3 1410 100.0 GERD [12]. After 4 weeks of treatment, 99% of those with complete resolution and 92% of those with 1 day of mild heartburn stated that the trial medication gave sufficient control of their heartburn, compared to only 10% of those with mild heartburn 5 to 7 days in the past week. Only 1% of those with moderate or severe heartburn after 4 weeks of treatment had sufficient control of heartburn. Thus, both frequency and severity of heartburn are valuable for interpreting the outcome in clinical terms, and were included in the present analysis. In studies where treatments are compared in terms of mean severity score, the scale should preferably have more than four categories [13]. Nevertheless, the results from the present study indicate that even a four-graded severity scale could be used satisfactorily in such an analysis of acid-suppressive therapies for GERD. For scales with seven grades, a change in mean score of 0.5 grades is often regarded as being clinically relevant [14]. In some studies evaluating a clinically relevant change the OTE has been used as an anchor [14], and the change is defined as the difference in mean score between patients who are unchanged and patients being somewhat [15]. If this definition is applied to the diary-assessed heartburn severity variable, then a clinically relevant change would be 0.35 (1.02 minus 0.67; see Fig. 2). Similarly, using data from Table 2, one can calculate that the clinically relevant change for the investigator-assessed four-graded scale for heartburn severity is 0.29 (0.96 minus 0.67). Table 2 shows that 13% (183/1410) of the patients were unchanged or worse after 4 weeks of treatment. One reason for this lack of effect may be that the cause of heartburn was not acid-related. The test retest reliabilities for the binary treatment success variables lie in the range of 0.5 to 0.6, which may seem somewhat low but can be regarded as being of fair to good according to Fleiss [9]. Reliability was when assessed as the agreement between diary- and investigator-based treatment success variables, with kappa values of 0.76. However, it is possible that, when answering the investigator s question, patients recalled what they had written in the diary during the preceding week.

770 Junghard and Wiklund Patient diary-recorded heartburn correlated well with investigator-assessed heartburn at 4 weeks. The investigator assessment gave a slightly higher proportion of patients with complete resolution and adequate relief than the patient diary recordings. These findings are in agreement with previous studies [16,17]. When using ordinal variables, frequency of heartburn seems to be an alternative to severity of heartburn. Frequency of heartburn shows good agreement between patient diary recordings and investigator assessment. The reliability is high and the responsiveness, measured as the difference in effect size between unchanged patients and patients a very great deal, is higher for heartburn frequency than for heartburn severity. An explanation might be that the number of days with heartburn during a 7-day period is a more objective and precise measure than the overall assessment of heartburn severity during the same period. Thus, we would recommend choosing mean heartburn frequency rather than mean heartburn severity as an outcome measure in clinical trials. Overall, our results indicate that all the examined variables are appropriate for use as outcome measures in clinical trials of endoscopy-negative patients with symptoms of heartburn. The results are only valid in such a context and with patients having a distribution of baseline characteristics similar to those in the clinical trials examined in this study. One limitation therefore is that generalization of the results to other patient populations may not be appropriate. Furthermore, the symptom studied was heartburn with a specific scale, and the results may not be applicable to other GERD symptoms such as acid regurgitation or dysphagia, or to other four-graded scales with different categories and scales with a different number of grades. Conclusion To summarize, the heartburn frequency and the fourgraded scale for assessing heartburn severity are reliable, responsive, and have good construct validity in patients with GERD. This applies to both assessments made by the investigator and assessments made by the patient using daily diary cards, and also when the scale is dichotomized into a success variable. We thank Claire Byrne and Steve Winter, from Wolters Kluwer Health, who provided editing assistance funded by AstraZeneca. Source of financial support: This study was supported by AstraZeneca. References 1 Tytgat GN. Review article: management of mild and severe gastro-oesophageal reflux disease. Aliment Pharmacol Ther 2003;17(Suppl. 2):S52 6. 2 Bytzer P. Assessment of reflux symptom severity: methodological options and their attributes. Gut 2004;53(Suppl. 4):S28 34. 3 Kahrilas PJ, Falk GW, Johnson DA, et al. Esomeprazole improves healing and symptom resolution as compared with omeprazole in reflux oesophagitis patients: a randomized controlled trial. The Esomeprazole Study Investigators. Aliment Pharmacol Ther 2000;14:1249 58. 4 Venables TL, Newland RD, Patel AC, et al. Omeprazole 10 milligrams once daily, omeprazole 20 milligrams once daily, or ranitidine 150 milligrams twice daily, evaluated as initial therapy for the relief of symptoms of gastro-oesophageal reflux disease in general practice. Scand J Gastroenterol 1997;32:965 73. 5 Katz PO, Castell DO, Levine D. Esomeprazole resolves chronic heartburn in patients without erosive oesophagitis. Aliment Pharmacol Ther 2003;18:875 82. 6 Armstrong D, Talley NJ, Lauritsen K, et al. The role of acid suppression in patients with endoscopynegative reflux disease: the effect of treatment with esomeprazole or omeprazole. Aliment Pharmacol Ther 2004;20:413 21. 7 Revicki DA, Wood M, Wiklund I, Crawley J. Reliability and validity of the Gastrointestinal Symptom Rating Scale in patients with gastroesophageal reflux disease. Qual Life Res 1998;7:75 83. 8 Fayers P, Machin D. Quality of Life: Assessment, Analysis and Interpretation. Chichester: Wiley, 2000. 9 Fleiss JL. Design and Analysis of Clinical Experiments. New York: Wiley, 1999. 10 Svedlund J, Sjodin I, Dotevall G. GSRS a clinical rating scale for gastrointestinal symptoms in patients with irritable bowel syndrome and peptic ulcer disease. Dig Dis Sci 1988;33:129 34. 11 Dimenas E, Glise H, Hallerback B, et al. Well-being and gastrointestinal symptoms among patients referred to endoscopy owing to suspected duodenal ulcer. Scand J Gastroenterol 1995;30:1046 52. 12 Junghard O, Carlsson R, Lind T. Sufficient control of heartburn in endoscopy-negative gastro-oesophageal reflux disease trials. Scand J Gastroenterol 2003;38:1197 9. 13 Wyrwich KW, Tardino VM. A blueprint for symptom scales and responses: measurement and reporting. Gut 2004;53(Suppl. 4):S45 8. 14 Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life. Med Care 2003;41:582 92. 15 Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a diseasespecific quality of life questionnaire. J Clin Epidemiol 1994;47:81 7. 16 McColl E. Best practice in symptom assessment: a review. Gut 2004;53(Suppl. 4):S49 54. 17 McColl E, Junghard O, Wiklund I, et al. Assessing symptoms in gastroesophageal reflux disease: how well do clinicians assessments agree with those of their patients. Am J Gastroenterol 2005;100:11 8.