Criterion validity, test-retest reliability and sensitivity to change of the St George urinary incontinence score

Transcription

1 Original Article THE ST GEORGE URINARY INCONTINENCE SCORE A.L. BLACKWELL et al. Criterion validity, test-retest reliability and sensitivity to change of the St George urinary incontinence score A.L. BLACKWELL, W. YOONG and K.H. MOORE Pelvic Floor Unit, St George Hospital, University of New South Wales, Sydney, Australia Accepted for publication 2 October 23 OBJECTIVE To assess the criterion validity, test-retest reliability and the sensitivity to change after treatment of the St George Urinary Incontinence Score (SGUIS). PATIENTS AND METHODS Women presenting with urinary incontinence completed two SGUIS tests one week apart. A frequency-volume chart (FVC) was completed before the first attendance and the women had a 1-h pad test before treatment. Patients were treated conservatively under the care of a urogynaecologist or nurse continence advisor, or surgically with an open or laparoscopic colposuspension. After treatment, the SGUIS, FVC and 1-h pad test were repeated. RESULTS The SGUIS correlated moderately well with the number of leaks per week (Spearman s r =.61, 95% confidence interval , P <.1) but less well with the 1-h pad test loss (r =.257, , P =.2). The test-retest reliability was acceptable, as the mean (SD) difference between the first and second SGUIS was.337 (2.675), with limits of agreement of to The change in the SGUIS after treatment correlated well with the improvement in the number of leaks per week (r =.742, , 156 samples, P <.1) but not as well with the change in 1-h pad test loss (r =.531, , 151, P <.1), although the trend was similar. CONCLUSION The criterion validity of the SGUIS appeared adequate for both the number of leaks per week on the FVC and leakage on the 1-h pad test. The statistical reproducibility of the test was adequate and appeared more responsive to change after treatment than the other measures. The score is suitable for rapid selfadministration by patients with a range of incontinence types, unlike most other currently available test instruments. KEYWORDS urinary incontinence, validity, reproducibility INTRODUCTION The search for a robust quantitative scoring system for the severity of urinary incontinence which can be used to document baseline status and outcome after treatment has been in progress for many years. Several authors have devised incontinence scoring systems but they have not been adopted by the research community in general. As long ago as 1953, the Ingelman Sundberg Incontinence Severity Grading was developed, which comprised three degrees of severity (I, leak only with cough or heavy lifting; II, leak with daily activities; III, dripping incontinence whenever upright) [1]. In 1992, Lagro-Jansen et al. [2] devised a 12-point score representing frequency and amount of leakage, pad usage and restriction of daily activity. More recently a range of in-depth scores was developed [3 1] but many are lengthy and/or need to be administered by a clinician. Nevertheless, the development of a simple scoring system which may be applied either as an adjunct to or in place of the frequencyvolume chart (FVC) or the 1-h pad test (both of which require considerable mental and physical capacity, as well as visual acuity) would be worthwhile. For example, the FVC can be difficult for patients who are incontinent before they reach the toilet, those who are physically unable to place a collecting device underneath their urethra (e.g. with severe arthritis, after a cerebrovascular accident, Parkinson s disease, etc.) and those who have trouble reading and recording their volumes. Office-based pad tests may also be troublesome for individuals with poor mobility and may not replicate the factors which provoke incontinence. In our Pelvic Floor Unit urogynaecologists collaborate with a colorectal team in the joint evaluation and treatment of patients with both urinary and fecal incontinence. We therefore became aware that a fecal incontinence grading score had been widely used for several years [11,12]. This scoring system comprises a Likert-type scale which measures the severity and frequency of fecal, liquid and flatal incontinence, and the effect of these on lifestyle as perceived by the patient (maximum composite score 2). We modified this four-item scoring system to depict the severity of stress and urge urinary incontinence and effect on lifestyle, with a maximum score also of 2 (Table 1), terming it the St George urinary incontinence score (SGUIS). The quantity of urine leakage is assessed by the frequency of incontinent episodes and the degree of wetness of perineal pads. The aim of the current study was to: (a) measure the criterion validity of the SGUIS, i.e. its correlation with the number of episodes of urinary leakage on the baseline FVC and urinary loss on the baseline 1-h pad test; (b) assess the test-retest reliability of the score before treatment; and (c) evaluate the sensitivity to change in leakage of the new score after conservative or surgical treatment (in relation to changes in the FVC and 1-h pad test). 24 BJU INTERNATIONAL 93, doi:1.1111/j x x 331

2 A.L. BLACKWELL ET AL. PATIENTS AND METHODS The study was conducted in three parts; first, a consecutive series of 27 women who understood English and had no cognitive impairment were invited to complete the SGUIS at their initial visit to the Pelvic Floor Unit. During history-taking the nature of stress and urge incontinence were explained to patients as part of routine education in the Unit. After a physical examination the score sheet (Table 1) was given to the patient to answer according to their present symptoms and whether their leakage was affecting their lifestyle. The items on the score were shown to the patient by the examining doctor (one consultant, two urogynaecology research fellows, one gynaecology registrar) or the clinical nurse specialist who undertook the routine first clinical assessment. If the patient was not clear about the score sheet after reading it, a simple sentence of clarification was given ( Please tell us whether you leak with cough or with urge, how wet you are and whether your lifestyle is affected ). These women had recorded a 3-day FVC [13] during the preceding week, with written instruction to measure all voided volumes and document all incontinence episodes. The number of incontinence episodes per week was calculated from the FVC. Within 2 weeks of their first visit, a standard International Continence Society (ICS) 1-h pad test [14] was performed and urodynamic studies undertaken. The criterion validity of the SGUIS was assessed using the FVC and the 1-h pad test. In the second part of the study (83 women) a second SGUIS test was administered on the day of the 1-h pad test, 7 14 days after the initial visit and before starting any treatment. The test was administered under identical conditions, with the same members of the urogynaecology team. The test-retest reliability was then assessed for the total sample and for a subset analysis of those with and without detrusor overactivity. In the third part of the study patients were reviewed at 3 months after conservative (e.g. weight loss, pelvic floor exercises; 15 women) or surgical (i.e. open or laparoscopic colposuspension; 52 women) treatment. At this time they completed another SGUIS test, completed a repeat FVC and another 1-h pad test. The sensitivity to change after treatment of the three outcome measures was compared. All three components of this study Difference in test result (1 2) TABLE 1 The SGUIS scheme; the total is 2 Incontinence were approved by the local hospital ethical committee. STATISTICS Never () Rarely (> weekly < daily) Mean test score ((1+2)/2) The data were not normally distributed, as assessed by Shapiro-Wilks testing and thus are summarized using the median and interquartile range (IQR). Criterion validity was assessed by calculating Spearman s rank correlation coefficient with 1-h pad test loss and leaks/week on the FVC. The test-retest reliability of the SGUIS was assessed by determining the agreement between the first and second scores, as described by Altman and Bland [15]. As shown in Fig. 1, there is no tendency for the amount of variation to change with the magnitude of the measurements, which means the reliability of the measure is not related to the value of the score. Adequate repeatability is assumed when in 95% of cases the difference between the scores lies within two SDs of the mean difference between the tests, even though the data may not be normally distributed [16]. As recommended by Bland and Altman [17] the mean difference between test 1 and test 2 should be close to zero, and the mean difference ± 2 SD of that difference are termed the limits of agreement. The responsiveness of the SGUIS to change was assessed by correlating the percentage change in SGUIS with the percentage changes in the 1-h pad test loss and the number of incontinence episodes per week (Spearman s rank correlation). RESULTS Sometimes (daily) Often (2/day) Stress Urge Pads - damp soaked Lifestyle Always ( 3/day) FIG. 1. The Bland-Altman test for testretest reliability of SGUIS in 83 women complaining of urinary incontinence. The 27 women who participated in the criterion validity investigation had a median (range, IQR) age of 57 (17 84, 47 68) years and median (IQR) parity of 2 (2 3). Their median (IQR) loss on the 1-h pad test was 12 (4 28) g, indicating that their incontinence tended to be moderate [18]. The median score was 9.5/2 (2 2, 7 12). Of the 27 women, one did not undergo a 1-h pad test, two did not complete a FVC and another could not estimate the frequency of her incontinent episodes because of constant leakage. Urodynamic testing showed that 68% had genuine stress incontinence, 21.7% had mixed incontinence and 6.8% had sensory urgency or detrusor overactivity (urodynamic testing did not detect any of these diagnoses in seven incontinent patients BJU INTERNATIONAL

3 THE ST GEORGE URINARY INCONTINENCE SCORE TABLE 2 The sensitivity of the SGUIS to change Variable SGUIS, points FVC, leaks/week Pad test, g/h Median (IQR): change* -6 (-3 to -8) -1 (-3 to -21) -7 (-2 to -24) % improvement 71.4 (4 1) 85.7 (5 1) 1 (5 1) Sample size Correlation, r % CI P <.1 <.1 *A negative number indicates improvement. FIG. 2. Box and whisker plots of the baseline and values at 3-month after treatment in a, SGIUS, b, the FVC and c, the pad test. SGUIS score a 5 and they were excluded from the subset analysis of urodynamic categories). The SGUIS took ª3 s to complete, and correlated moderately well with the number of leaks/week on the FVC (Spearman s rank correlation coefficient.61, 95% CI , 24, P <.1). The SGUIS correlated significantly but much less strongly with the 1-h pad test loss (Spearman s rank correlation coefficient.257, 95% CI , 26, P =.2). Analysis of test-retest reliability amongst 83 patients using the Bland-Altman test for agreement is shown in Fig. 1. Urodynamic testing in these patients showed that 54.2% had genuine stress incontinence, 22.9% had mixed incontinence, and 8.4% had detrusor overactivity or sensory urgency (urodynamic testing did not detect any of these diagnoses in 12 incontinent patients and they were excluded from the subset analysis). The data fulfil the statistical criteria required by the British Standards Institution of acceptable repeatability [16]. The mean (SD) difference between the first and second SGUIS was.337 (2.675), which differed little from zero. The limits of agreement were to (mean difference ± 2 SDs). To further assess repeatability according to the presence or absence of symptoms of detrusor overactivity, the following subsets were analysed. First, patients with pure genuine stress incontinence were analysed and the mean (95% CI) difference in scores for these 45 women was.489 (-.367 to 1.345). Second, patients with any genuine stress incontinence were analysed and for these 64 women the mean difference in scores was.234 (-.459 to.928). Analysis of the seven women with only sensory urgency or detrusor overactivity showed a mean difference of -.14 ( to 2.442), and finally analysis of the 26 women with any detrusor overactivity or sensory urgency gave a mean difference of -.37 (-1.35 to.732). This suggests that the SGUIS is equally repeatable regardless of the underlying urodynamic diagnosis. As noted, about a third of patients had surgical treatment and at the 6-week assessment they were predominantly dry. The remainder underwent a combination of conservative treatments, with a wide range of outcomes. After therapy, results shown in Table 2 indicate that the median change and median percentage improvement in all three tests showed similar trends (Fig. 2a c). The pad test data were clearly skewed, as the 1-h pad test showed a median improvement of 7 g (Table 2), but because many patients became continent on this test the median percentage improvement was 1%. Therefore the mean (SD) percentage changes in each score were compared and showed similar trends; SGUIS, 61.5 (36.5)%; FVC, 54.6 (129.5)%; and the pad test 3.4 (192.9)%. DISCUSSION The development of the SGUIS as an outcome measure began in 1996, before the introduction of more recent urinary symptom and quality-of-life questionnaires [3 1]. However, despite the existence of other tests, none has provided the simple scoring system encompassing leakage severity and lifestyle impact that we obtain from this test, and furthermore, none directly parallel a fecal incontinence score. Leaks/week Urine lost, g b c 4 Baseline 3 months Bernstein et al. [3] described a pilot study of 56 female patients completing a selfadministered questionnaire which assessed LUTS and bother, but it included sexual function, pain, use of appliances and quality of life. Jackson et al. [4] developed the Bristol Female LUTS questionnaire to assess all LUTS, with an emphasis on incontinence and effect on quality of life. It was validated in 85 women but is much longer than the SGUIS (33 items) and has not been assessed for 24 BJU INTERNATIONAL 333

4 A.L. BLACKWELL ET AL. change after treatment. The King s Health Questionnaire [5], which involves eight domains and 21 questions focusing on the impairment of quality of life caused by urinary symptoms, has been validated in 293 women, assessed for test-retest reliability, but the sensitivity to change after treatment has not been published. Gunthorpe et al. [6] devised a 33-item questionnaire for use as a screening tool in the primary-care setting to identify patients with urinary incontinence. This was then reduced to five items, focusing on urge symptoms and bothersomeness. It was reported to predict almost 7% of women who would have leakage on a 48-h pad test but sensitivity to change was not assessed. Sandvik et al. [7] designed a severity index that was validated against the 48-h pad test but test-retest reliability and sensitivity to change were not examined. Most recently, Shaw et al. [8] described a measure of incontinence severity which was validated in 93 subjects. However, it is longer (11 questions) and provides a rather more subjective/qualitative assessment of leakage. In the same year, Hillier et al. [9,1] described a questionnaire and scoring system validated in 63 women, which includes urinary dysfunction and incontinence. This test is similar to the Bristol Female LUTS questionnaire but also includes 1 items about bowel evacuation and constipation. Sensitivity to change after hysterectomy was described, but sensitivity to change after treating incontinence symptoms was not measured. Many of the above scores require a substantial time for completion; in our experience, administering the SGUIS was quick (<3 s) and patient compliance was good, in that data were seldom missing. Only one patient, who had constant dripping loss, was unable to complete all the questions. The SGUIS, as a simple and quick office test, is useful as an adjunct to currently available objective outcome measures of the severity of incontinence, particularly because it includes a simple assessment of effect on lifestyle. Another advantage of the Likert-type scale is that it may be conducted by telephone interview, which facilitates the long-term surveillance of patients in clinical trials. We have recently used the SGUIS for a 3-year follow-up of patients in a randomized trial who had already completed the score at baseline and 6-week review, and found it easy to administer over the telephone [19]. The fecal incontinence grading scale [11], which parallels the SGUIS, is a widely used office tool. It has recently been extended to include 24-point scores so as to assess fecal urgency and the use of antidiarrhoeal medication. The validity, test-retest reliability and sensitivity to change of this score have been published [2]. Thus the combination of the amended Wexner scale with the SGUIS allows a rapid reliable assessment of both fecal and urinary incontinence. For criterion validity the SGUIS correlated significantly at baseline with the number of leaks/week on the FVC. This was not surprising considering that the frequency of incontinence episodes (per day or week) constitutes an item within the SGUIS. The correlation with the ICS 1-h pad test was significant but less strong. However, the 1-h pad test has poor reproducibility even when the bladder volumes are standardized by retrograde urethral catheter filling [21]. Several authors have reported that when two 1-h pad tests were conducted under identical conditions the first pad loss is often significantly smaller than the second [22 24]. That the first pad loss may not be representative of true incontinence may explain the relatively poor correlation between the SGUIS and a single 1-h pad test. This study could be criticised in that there was no formal validation of lifestyle impact, relative to other quality-of-life tests. When the SGUIS was first introduced quality-of-life testing was not common practice. However, the main focus of the test is to provide an overall severity index rather than a specific indication of quality of life. Furthermore, the score does not aim to measure the bothersomeness of urge symptoms, merely to assess the severity of leakage. This could be considered a weakness of the test, but we considered that the primary need was for a simple severity index, particularly for use as measure of outcome after treatment. For test-retest reliability the SGUIS conforms to the British Standards Institution criteria [16]. As with all incontinence questionnaires and scores, the SGUIS depends on the patient s memory of recent events, which may be a confounding variable. The relatively wide limits of agreement for two SGUIS administered 7 14 days apart may also reflect the varying nature of the urinary symptoms rather than poor reliability of the scoring system. This may be a result in part to the variation of the clinical condition over time, with comorbidity such as respiratory tract infection, or with other environmental influences such as cold weather and emotional stress affecting patients with stress and urge incontinence, respectively. Nevertheless, subset analysis of patients with pure genuine stress incontinence, pure urge leak or mixed incontinence showed no apparent trend in the magnitude of differences between the first and second test in relation to diagnosis. Most importantly, the SGUIS was shown to be sensitive to change after both conservative and surgical treatment. Many other questionnaires have not been validated in this way [3 7,9]. Although some difficulties were encountered with heavily skewed data, the overall picture suggests that the SGUIS follows a similar trend to other outcome measures, but gives an added dimension of lifestyle effect. In summary, the SGUIS is a brief, simple and validated outcome measure that is readily completed by patients with minimal supervision. In keeping with the recommendations of the ICS Standardization Committee on outcome measures [25], we suggest that validated scoring systems such as this should be more widely used to measure the domain of patient symptoms when assessing the efficacy of any new treatment. ACKNOWLEDGEMENTS We thank A. Simons, A. Foote and S. Prashar for recruiting subjects and collecting data, and S, Buckland for performing the pad testing. REFERENCES 1 Ingelmann-Sundberg A. Urininkontinens hos kvinnan. Nordisk Med 1953; 5: Lagro Janssen ALM, Debruyne FMJ, Smits AJA, Van Weel C. The effects of treatment of urinary incontinence in general practice. Family Prac 1992; 9: Bernstein I, Sejr T, Able I et al. Assessment of lower urinary tract symptoms in women by a self BJU INTERNATIONAL

5 THE ST GEORGE URINARY INCONTINENCE SCORE administered questionnaire: test-retest reliability. Int Urogynecol J 1996; 7: Jackson S, Donovan J, Brookes S, Eckford S, Swithinbank L, Abrams P. The Bristol Female Lower Urinary Tract Symptoms Questionnaire: development and psychometric testing. Br J Urol 1996; 77: Kelleher C, Cardozo LD, Khullar V, Salvatore S. A new questionnaire to assess the quality of life of urinary incontinent women. Br J Obstet Gynaecol 1997; 14: Gunthorpe W, Brown W, Redman S. The development and evaluation of an incontinence screening questionnaire for female primary care. Neurourol Urodynam 2; 19: Sandvik H, Seim A, Vanvik A, Hunskaae S. A severity index for epidemiological surveys of female urinary incontinenc: comparison with 48-hour pad-weighing tests. Neurourol Urodyn 2; 19: Shaw C, Matthews RJ, Perry SI et al. Validity and reliability of an intervieweradministered questionnaire to measure the severity of lower urinary tract symptoms of storage abnormality: the Leicester Urinary Symptom Questionnaire. BJU Int 22; 9: Hiller L, Radley S, Mann CH et al. Development and validation of a questionnaire for the assessment of bowel and lower urinary tract symptoms in women. Br J Obstet Gynaecol 22; 19: Hiller L, Bradshaw HD, Radley SC, Radley S. A scoring system for the assessment of bowel and lower urinary tract symptoms in women. Br J Obstet Gynaecol 22; 19: Jorge JMN, Wexner SD. Aetiology and management of faecal incontinence. Dis Colon Rectum 1993; 36: Rothbarth J, Bemelman WA, Meijerink WJHJ et al. What is the impact of faecal incontinence on quality of life? Dis Colon Rectum 21; 44: Wyman JF, Choi SC, Harkins SW, Wilson MS, Fantl JA. The urinary diary in evaluation of incontinent women: a testretest analysis. Obstet Gynecol 1988; 71: Abrams P, Blavais JG, Stanton SL, Andersen JL. Standardisation of Terminology of lower Urinary Tract Function. Quantification of urinary loss. Br J Obstet Gynaecol Suppl 199; 6: Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. Statistician 1983; 32: British Standards Institution. Precision of test methods I. Guide for the determination and reproducibility for a standard test method (BS 5497, part 1). London: BSI, Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: Klarskov P, Hald T. Reproducibility and reliability of urinary incontinence assessment with a 6 minute test. Scand J Urol Nephrol 1984; 18: Moore KH, O Sullivan RJ, Simons A, Prashar S, Anderson P, Louey M. Randomized controlled trial of nurse continence advisor therapy versus standard urogynaecology regime for conservative incontinence treatment: Efficacy, costs and two year follow up. Br J Obstet Gynaecol 23 2 Vaizey CJ, Carapeti E, Cahill JA, Kamm MA. Prospective comparison of faecal incontinence grading systems. Gut 1999; 44: Lose G, Rosenkilde P, Gammelgaard J, Schroeder T. Pad weighing test performed with standardised bladder volumes. Urology 1988; 32: Fantl JA, Harkins SW, Wyman JF, Choi SC, Taylor JR. Fluid loss quantification test in women with urinary incontinence: a test retest analysis. Obstet Gynecol 1987; 7: Jakobsen H, Kromann-Andersen B, Nielson KK, Maegaard E. Pad weighing tests with 5% or 75% bladder filling. Does it matter? Acta Obstet Gynecol Scand 1993; 72: Simons AM, Yoong WC, Buckland S, Moore KH. Inadequate repeatability of the one hour pad test: the need for a new incontinence outcome measure. Br J Obstet Gynaecol 21; 18: Lose G, Fantl JA, Victor A et al. Outcome measures for research in adult women with symptoms of lower urinary tract dysfunction. Neurourol Urodyn 1998; 17: Correspondence: K Moore, First Floor, Pitney Clinical Sciences Building, St George Hospital, Kogarah, Sydney NSW 2217, Australia. k.moore@unsw.edu.au Abbreviations: FVC, frequency-volume chart; SGUIS, St George urinary incontinence score; ICS, International Continence Society; IQR, interquartile range. 24 BJU INTERNATIONAL 335