Development and Validation of an Improved, COPD-Specific Version of the St. George Respiratory Questionnaire*

Transcription

1 Original Research COPD Development and Validation of an Improved, COPD-Specific Version of the St. George Respiratory Questionnaire* Makiko Meguro, Mphil; Elizabeth A. Barley, PhD, CPsychol; Sally Spencer, PhD; Paul W. Jones, PhD, FRCP Objective: To produce an improved, COPD-specific version of the St. George respiratory questionnaire (SGRQ-C). Methods: Five different steps were required: (1) Rasch analysis of the responses of 893 COPD patients to the St. George respiratory questionnaire (SGRQ) identified weaker items to be removed; (2) a scoring algorithm was produced using data from 1,036 patients; (3) validity of the new and original SGRQ was tested using data from the original validation study; (4) responsiveness was tested using data from a previously published trial; and (5) a reworded version (SGRQ-C) that no longer specified the recall period was administered to 63 pulmonary rehabilitation participants. Results: Items were removed due to lack of response (n 1), misfit to the Rasch model (n 8), and disordered responses (n 1). Another six items had disordered responses; this was corrected. Scores from the two versions differed slightly, so the scoring algorithm was revised to produce scores equivalent to the original. Intraclass correlation coefficient (ICCC) for the scores for original and new versions was Correlations with other measures of disease were very similar to those obtained with the original. New and original scores for treatment effects were similar: difference, U( SD). Baseline SGRQ and SGRQ-C scores were similar (ICCC, 0.95; 95% confidence interval, 0.92 to 0.97; mean difference, U). Change scores were similar (difference, U). Conclusions: The SGRQ-C contains the best of the original items, no longer specifies a recall period, and produces scores equivalent to the original. (CHEST 2007; 132: ) Key words: COPD; health-related quality of life; health status; St. George respiratory questionnaire; questionnaires Abbreviations: ICC item characteristic curve; ICCC intraclass correlation coefficient; PSI person separation index; SGRQ St. George respiratory questionnaire; SGRQ-C COPD-specific version of the St. George respiratory questionnaire *From Respiratory Medicine, Cardiac and Vascular Sciences (Miss Meguro, Dr. Barley, and Dr. Jones), St. George s, University of London, London; and Brunel University (Dr. Spencer), Osterley Campus, Isleworth, Middlesex, UK. The authors have no financial or other potential conflicts of interest to disclose. The St. George respiratory questionnaire (SGRQ) 1 is a measure of health status for COPD. It has been in existence for 15 years, and there is a large body of evidence concerning its validity. Rigorous methodology was used in its development, but it is inevitable that some items have weaker measurement properties than others. Removal of these should improve reliability and responsiveness. This study was designed to identify such items and test the effect of their removal on the performance of the questionnaire. Shortening the questionnaire was not a specific aim. Short instruments for measuring health status in Manuscript received March 16, 2006; revision accepted April 19, Reproduction of this article is prohibited without written permission from the American College of Chest Physicians ( org/misc/reprints.shtml). Correspondence to: Paul W. Jones, PhD, FRCP, Respiratory Medicine, Cardiac and Vascular Sciences, St. George s, University of London, Cranmer Terrace, London, SW17 0RE; p.jones@sgul.ac.uk DOI: /chest Original Research

2 respiratory disease are available, 2 4 but they may perform slightly less well than longer instruments. 2,3,5 From experience, we knew that up to 10 randomly missed items did not affect scores significantly, so we limited item removal to 10 items. We were concerned to ensure that the revised instrument would produce scores directly comparable to those from the original, to enable direct comparison between studies using different versions. The SGRQ was developed initially for use in asthma and COPD. It has found widespread use in COPD but less application in asthma, so we restricted this analysis to data from COPD patients. We describe a series of studies that reanalyzed the SGRQ items, produced a revised version and scoring system, and tested validity and responsiveness. A final study reports a preliminary test of the crosssectional and longitudinal validity of the new revised and reworded measure. The SGRQ Materials and Methods A total and three component scores are provided: symptoms (8 items), activity (16 items), and impacts (26 items). The 50 items vary in form between polytomous (eg, Likert-type scale) and dichotomous (eg, true/false ). In the symptoms component, patients are asked to recall symptoms over a specified time frame: 1 month, 3 months, or 1 year. Other items are not time related. Each item has an empirically derived weight. 6 Scores range between 0 (no impairment) and 100 (worst possible health). Study Protocols Study 1: Development of the Revised Version: Baseline data were available from the Inhaled Steroids in Obstructive Lung Disease in Europe study 7 and the placebo limb of a study 8 of tiotropium in COPD. Only data from patients with no missing responses were used (n 893; 371 women; mean age, 64 7 years [ SD]; mean prebronchodilator FEV 1, L). The performance of each item was tested using Rasch analysis. Study 2: Development of a New Scoring System: Rasch analysis showed that responses to some polytomous items should be amalgamated. For the revised version, we averaged the item weights from the response categories that were to be collapsed to produce weights for the new categories. Study 3: Cross-sectional Validity: Scores for the revised version were compared with scores from the original using data from the original validation study 1 (n 152; 53 women; mean age, 63 8 years; mean FEV 1, % of predicted). Study 4: Responsiveness: Sensitivity to change was compared with that of the original using data from a trial 9 of salmeterol. We only used patients who provided SGRQ data with no missed responses (n 169; 25 women; mean age, 62 8 years; mean FEV 1,46 15% of predicted). Study 5: Cross-sectional and Longitudinal Validity: The analyses described in studies 3 and 4 were performed using a revised scoring algorithm developed from studies 1 and 2. Since this involved collapsing response categories for a few items, we revised the wording (details in on-line Supplement). We also removed the recall period used in the symptoms component (details in on-line Supplement) since previous work had shown that the 1-month version had only marginally inferior characteristics to the 1-year version. 10 The new SGRQ is termed the COPD-specific version of the St. George respiratory questionnaire (SGRQ-C). The SGRQ-C was administered with the SGRQ (1-year version) at the same time using a randomized order to 63 COPD patients (23 women; mean age, 70 9 years; mean FEV 1, 47 14% of predicted) participating in three pulmonary rehabilitation programs throughout the United Kingdom. The SGRQ-C was readministered after 4 weeks. Analysis Study 1: Prior to the Rasch analysis, we removed those items that had a high proportion of missed responses relative to the others. Rasch Analysis: The Rasch model 11,12 provides a template for measurement in which all the items are assumed to conform to a unidimensional structure. It enables tests of how well each item fits that structure. The first step examines the ordering of responses to polytomous items. When correctly ordered, the probability of responding to items of different severities should follow the level of health status impairment of the respondents. Item responses that do not follow a logical order are disordered, but this may be corrected by combining response categories. We used a range of tests to identify the least well-performing items that could be removed without altering the basic properties of the SGRQ. Analyses were performed using a Rasch unidimensional measurement model (RUMM 2020; RUMM Laboratory; Perth, Western Australia). 13 The capacity of the scale to discriminate between patients was tested using the person separation index (PSI) from the Rasch analysis. It is analogous to Cronbach. 14 The PSI for each component was calculated before and after removing an item. If it worsened, the item was retained. Finally, the PSI was calculated for all items used in the revised version and compared with that for the original 50 items. We examined the item characteristic curve (ICC) and 2 fit statistic for each item to test its ability to discriminate between patients with different levels of health. ICCs show whether persons with a similar level of health respond similarly to an item. The 2 was used to test the quality of fit of each ICC to the unidimensional model. To maintain the balance of symptom, activity, and impact items present in the original version, we examined the fit of each item only in relation to the others within its component. Items for which ordered thresholds could not be achieved and items showing misfit were removed. Where several items met the removal criteria, item content was also considered in order to preserve content validity. Other Analysis: Differences between original and revised versions of the SGRQ were tested using paired t tests and intraclass correlation coefficients (ICCCs). Agreement was tested by plotting the difference between original and revised scores against the mean of the two. 15 Comparative validity was tested by examining the correlations (r) between the two versions of the SGRQ and other measures of disease severity. Differences between patient groups were tested using analysis of variance. Data are reported as mean SD. Study 1 Results The PSI of the original SGRQ indicated excellent discriminatory function (PSI, 0.90). One item referwww.chestjournal.org CHEST / 132 / 2/ AUGUST,

3 ring to ability to work was found to have a large number of missing responses (19% compared with a mean of 1% for the other items). It was removed. Symptoms Items: The PSI for this component was 0.74 ( good ). Six of seven Likert items had disordered thresholds. In five, ordered response options were achieved by combining two or more categories (details in the on-line Supplement). One item, How long did the worst attack of chest trouble last? could not be rescored and was removed. Following these changes, the PSI for this component was Activity Items: The initial PSI was The two worst-fitting (p ) items I get breathless lying still and I get breathless playing sports or games were removed. An item concerning very strenuous activities was also removed. It fitted the model, but its location ( 2.32 logits) showed it was less severe than an item about moderately strenuous activity ( 2.49 logits), which is counter intuitive. After removal of these items, the PSI was Impact Items: The initial PSI was One item showed disordered thresholds; this was resolved by combining two response categories. The item with the worst fit ( I do not expect my chest to get any better ; p ) was removed. Removal of the next worse-fitting item caused another previously good item to misfit. Both were retained and other misfitting items examined. Three referred to medication. A fourth medication item had quite a good fit, but as a single item it now appeared incongruous with the other items that reflect quite different psychosocial effects. It was removed. Removal of these items reduced the PSI by 0.005, which was considered acceptable. The PSI for the 40 retained items showed that the excellent discriminatory function had been maintained (PSI, 0.90). Study 2 A Bland and Altman plot 15 for the total score (not shown) showed that the difference between the new and original scores also increased very slightly with higher scores (linear regression slope, 0.10; R ; p ). This was also the case for the component scores (symptoms: slope, 0.02; R , p 0.01; activity: slope, 0.13; R , p ; and impacts: slope, 0.10; R , p ). This appeared to be due to the removal of items that had a high response rate in severe disease but had a relatively low item weight. To achieve equivalence between the original and revised scores, the revised scores were adjusted. We calculated regressions between the original (dependent variable) and the revised (independent variable) scores (total: Y [0.90 X], R ; symptoms: Y 0.94 [0.99 X], R ; activity: Y 7.01 [0.87 X], R ; and impacts: Y 2.18 [0.88 X], R ) [p for all scores]. From this, we created algorithms that would produce original SGRQ scores from the 40-item set. Subsequent analyses of the revised SGRQ reported in this article use scores adjusted in this way. Table 1 shows the regressions between the original and revised total scores before and after using these algorithms. Study 3 Differences between the two scores were minimal: mean total, U (minimum, 6.4 U; maximum, 5.7 U; median, 0.1 U); mean symptoms, U (minimum, 16.2 U; maximum, 13.1 U; median, 0.5 U); mean activity, U (minimum, 9.3 U; maximum, 10.6 U; median, 0.8 U); mean impacts, U (minimum, 9.4 U; maximum, 11.0 U; median, 0.6 U; all p 0.05) [paired t test, degree of freedom 151; t 0.08, total; t 0.003, symptoms; t 0.02, activity; t 0.04, impacts]. The IC- CCs showed excellent reliability: total, 0.99; symptoms, 0.96; activity, 0.99; and impacts, Bland and Altman plots indicated good agreement (Fig 1). The difference between the two total scores was less than the minimum clinically important score of 4 Table 1 Step 2, Regression Estimates Between Original SGRQ Total Scores (Dependent Variable) and the Revised Total Scores (Independent Variable) Before and After the Correction Algorithm Was Applied* Before Algorithm Applied After Algorithm Applied Variables No. Intercept Slope R 2 Intercept Slope R 2 Step Step Step Prospective study *The intercept is measured in SGRQ units; the slope is a ratio (original score divided by revised score). For steps 2 to 4 the revised and original SGRQ scores were calculated from the same set of item responses, but the scores were produced using different items. In the prospective study, two different questionnaires were administered to the patients: the original version of the questionnaire and the reworded revised questionnaire (the SGRQ-C). 458 Original Research

4 Figure 1. Difference between original and revised SGRQ scores plotted against the mean of the original and revised SGRQ scores. points for all but 13 9% of patients (Fig 1). Scores from the new and original versions correlated very similarly with other measures of disease (Table 2). Both were independent of age and sex (p 0.05). Study 4 Mean change scores (follow-up baseline) for the original and revised versions were very similar: mean difference, total: U (minimum, 6.6 U; maximum, 8.2 U; median, 0.4 U); symptoms: U (minimum, 16.2 U; maximum, 14.5 U; median, U); activity: U (minimum, 7.5 U; maximum, 8.1 U; median, U); and impacts, (minimum, 10.6; maximum, 12.1 U; median, 0.01 U) [all scores, p 0.05]. The ICCC between the original and revised scores (n 169) showed excellent reliability: total, 0.98; symptoms, 0.93; activity, 0.98; and impacts, Original and revised SGRQ change scores were similarly correlated with change in the other variables (Table 3). According to both SGRQ scores, patients receiving salmeterol showed greater improvement than those receiving placebo (Table 4). Differences between treatment groups were significant for impacts and total scores (original and revised) only (analysis of variance, degrees of freedon 1,168; impacts: original, F 7.5, p 0.007; new, F 6.1, p 0.02; total: original, F 6.0, p 0.02; new, F 4.3, p 0.04). In the placebo-treated arm of the study, repeatability measured as ICCC was the same for both versions (Table 4). Study 5 At baseline, SGRQ and SGRQ-C scores were very similar (mean difference: total, U; symp- CHEST / 132 / 2/ AUGUST,

5 Table 2 Step 3, Correlations (r) Between Original and Revised SGRQ Scores and Concurrent Measures of Disease Severity (n 152)* Total Symptoms Activity Impacts Variables Original Revised Original Revised Original Revised Original Revised FEV FVC min walk distance MRC dyspnea grade HADS anxiety SIP total Global health Cough/phlegm Daily wheeze *Data are from Jones et al. 1 MRC Medical Research Council; HADS hospital anxiety and depression scale; SIP sickness impact profile; Global health 5-point global health rating (very poor, poor, satisfactory, good, very good); Cough/phlegm and daily wheeze were measured using the Medical Research Council Respiratory Questionnaire; p unless otherwise indicated. p 0.05, not significant. p p toms, U; activity, U; impacts, U) [all p 0.05]. The ICCCs showed excellent reliability: total, 0.95; activity, 0.93; and impacts, The symptoms component was slightly less reliable (ICCC, 0.80). The Bland and Altman plots showed that the activity score differed slightly depending on severity (linear regression slope, R , p 0.01), but this was not seen with the other components or total score (Fig 2). The two questionnaires were correlated similarly with the other measures of disease severity (Table 5). Both were independent of age and sex (p 0.05). Thirty-six patients were available for follow-up (mean age, 70 9 years; mean FEV 1,49 13% of predicted). Their global health rating did not change. SGRQ and SGRQ-C total scores did not change: SGRQ, U; SGRQ-C, U. The SGRQ-C changed slightly more than the SGRQ, but this was not significant (paired t test, p 0.05). Discussion This analysis of responses to the SGRQ from a large number of COPD patients identified 10 weaker items that could be removed without altering the performance of the instrument. Initial testing found that the SGRQ already had good measurement properties. Its internal reliability has now been improved by removing an item with a low response rate and by using Rasch analysis to identify items that did not fit a unidimensional model so well. The content balance of the SGRQ Table 3 Step 4, Correlations (r) Between Changes in Original and Revised SGRQ Scores and Changes in Other Measures of Disease Severity From a Randomized Trial of Salmeterol (n 169)* Total Symptoms Activity Impacts Variables Original Revised Original Revised Original Revised Original Revised FEV 1 % predicted min walk distance SF-36, physical functioning SF-36, social functioning SF-36, physical role limitation SF-36, emotional role limitation SF-36, mental health SF-36, vitality SF-36, health perception *From Jones and Bosh. 9 SF-36 Medical Outcomes Study Short Form-36 health scale; p 0.05 unless otherwise indicated. p p p p Original Research

6 Table 4 Step 4, Change in Score Over 16 Weeks for Placebo- (n 81) and Salmeterol- (n 88) Treated Patients for the Original and Revised SGRQ Scores. The ICCC Shows the Degree of Association Between Baseline and Follow-up Scores in Placebo-Treated Patients* Original SGRQ Revised SGRQ Variables Mean (SD) ICCC (95% CI) Mean (SD) ICCC (95% CI) Total Placebo 1.4 (11.5) 0.81 ( ) 1.6 (11.6) 0.80 ( ) Salmeterol 5.9 (12.7) 5.5 (13.1) Symptoms Placebo 4.9 (15.0) 0.73 ( ) 5.1 (15.7) 0.66 ( ) Salmeterol 6.7 (16.6) 6.4 (16.9) Activity Placebo 1.8 (15.1) 0.78 ( ) 2.3 (14.8) 0.77 ( ) Salmeterol 3.4 (14.6) 3.3 (14.4) Impacts Placebo 0.2 (15.8) 0.70 ( ) 0.1 (15.8) 0.69 ( ) Salmeterol 7.0 (16.3) 6.4 (17.0) *Data are from Jones and Bosh. 9 Mean change was calculated as follow-up baseline score. Differences between placebo and treatment groups were significant for impacts (original, p 0.007; revised, p 0.02) and total (original, p 0.02; revised, p 0.04) scores. Changes in score calculated in this analysis differ from those in the original publication because that analysis included patients for whom there were missed responses in the questionnaires and the SGRQ was scored using preset rules for including/excluding questionnaires based on the number of missing items. This analysis used only those questionnaires for which there were no missed items at baseline or follow up. CI confidence interval. has been maintained. The item with a low response rate referred to employment; since most of these patients were beyond retirement age, this finding was unsurprising. The most important finding was that the response options for some polytomous items did not operate in an ordered manner. This could be resolved by combining two or more options for all except one item, which was removed. Eight items were removed because they showed a poor fit relative to the rest of the items in the same component of the questionnaire. Rasch analysis is a useful tool for examining the performance of individual items. The properties of each item are defined in relation to one another. No single test of fit is both necessary and sufficient for confirming the model; rather, evidence of statistical misfit needs to be seen as giving a clue to a substantive anomaly to be studied further. 16 Removal of the worst-fitting impact item caused another previously well-fitting item to fit less well. When this occurs, the effect of removing items that show less misfit can be tested. Item content should guide which items to examine because this will help maintain content validity. This reasoning led us to remove four impact items. Only three showed misfit, but all four referred to medication use. A single medication item would have been incongruous with the remaining items in that component. Initial testing of the revised SGRQ showed its scores to be a little different from the original, so we derived a scoring algorithm to ensure that scores from the new version are equivalent to the original. Cross-sectional and longitudinal tests of validity us- Table 5 Prospective Study of Reworded, Revised Version (SGRQ-C) vs Original SGRQ in Patients Recruited to a Rehabilitation Program: Baseline Correlations (r) Between SGRQ and SGRQ-C Scores and Concurrent Measures of Disease Severity* Total Symptoms Activity Impacts Variables SGRQ SGRQ-C SGRQ SGRQ-C SGRQ SGRQ-C SGRQ SGRQ-C FEV HADS anxiety HADS depression MRC dyspnea grade Global health *See Table 2 for expansion of abbreviations. p 0.05, not significant. p p p CHEST / 132 / 2/ AUGUST,

7 Figure 2. Difference between the original SGRQ and the SGRQ-C scores plotted against the mean of the original SGRQ and the SGRQ-C scores. ing existing datasets showed this algorithm to be reliable. Finally, the correlation between the SGRQ-C and the original was slightly lower than that found in studies 3 and 4. This was to be expected because in those studies patients completed only one questionnaire and the comparison was between scores calculated in different ways. By contrast, in the final study, two different questionnaires were completed, so the correlation between them contains measurement error due to between-test repeatability and the effects of differences between the two. Overall, good agreement was found between the SGRQ and the SGRQ-C. Larger studies comparing the two questionnaires are needed to confirm these findings. In summary, the new COPD-only version of the SGRQ, the SGRQ-C, is shorter, contains the best of the original items, no longer specifies a recall period, and produces scores equivalent to the existing instrument. References 1 Jones PW, Quirk FH, Baveystock CM, et al. A self-complete measure of health status for chronic airflow limitation. Am Rev Respir Dis 1992; 145: Barley EA, Quirk FH, Jones PW. Asthma health status measurement in clinical practice: validity of a new short and simple instrument. Respir Med 1998; 92: Juniper EF, Guyatt GH, Cox FM, et al. Development and validation of the mini asthma quality of life questionnaire. Eur Respir J 2001; 14: Katsura H, Yamada K, Kida K. Usefulness of a linear analog scale questionnaire to measure health-related quality of life in elderly patients with chronic obstructive pulmonary disease. J Am Geriatr Soc 2003; 51: Oga T, Nishimura K, Tsukino M, et al. Comparison of the responsiveness of different disease-specific health status measures in patients with asthma. Chest 2002; 122: Quirk F, Baveystock C, Wilson R, et al. Influence of demographic and disease related factors on the degree of distress associated with symptoms and restrictions on daily living due 462 Original Research

8 to asthma in six countries. Eur Respir J 1991; 4: Burge PS, Calverley PM, Jones PW, et al. Prednisolone response in patients with chronic obstructive pulmonary disease: results from the ISOLDE study. Thorax 2003; 58: Casaburi R, Mahler DA, Jones PW, et al. A long-term evaluation of once-daily inhaled tiotropium in chronic obstructive pulmonary disease. Eur Respir J 2002; 19: Jones PW, Bosh TK. Quality of life changes in COPD patients treated with salmeterol. Am J Respir Crit Care Med 1997; 155: Barr JT, Schumacher GE, Freeman S, et al. American translation, modification and validation of the St George s Respiratory Questionnaire. Clin Ther 2000; 22: Rasch G. Probabilistic models for some intelligence and attainment tests. Chicago, IL: University of Chicago Press, Barley EA, Jones PW. Repeatability of a Rasch model of the AQ20 over five assessments. Qual Life Res 2006; 15: Andrich D, Sheridon BE, Luo G. RUMM2020 (Rasch unidimensional measurement models). Perth, Western Australia: RUMM Laboratory, Andrich D. Rasch models for measurement. Newbury Park, CA: Sage Publications, Bland JM, Altman, DG Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: Hagquist C, Andrich D. Is the sense of coherence-instrument applicable on adolescents? A latent trait analysis using Raschmodelling. Person Indiv Diff 2004; 36: CHEST / 132 / 2/ AUGUST,