Voiding Dysfunction. The Validity and Reliability of the Neurogenic Bladder Symptom Score

Transcription

1 Voiding Dysfunction The Validity and Reliability of the Neurogenic Bladder Symptom Score Blayne Welk,*, Sarah Morrow, Wendy Madarasz, Richard Baverstock, Jennifer Macnab and Keith Sequeira From the Division of Urology, Department of Surgery (BW) and Departments of Clinical Neurosciences (SM), Physical Medicine and Rehabilitation (KS) and Epidemiology and Biostatistics (JM), Western University and St. Joseph s Health Care (WM), London, Ontario and Vesia (Alberta Bladder Centre), Division of Urology, Department of Surgery, University of Calgary (RB), Calgary, Alberta, Canada Abbreviations and Acronyms AUASS ¼ American Urological Association symptom score ICIQ-UI ¼ International Consultation on Incontinence- Urinary Incontinence I-QOL ¼ incontinence quality of life MS ¼ multiple sclerosis NBSS ¼ neurogenic bladder symptom score PROM ¼ patient reported outcome measure QOL ¼ quality of life SB ¼ spina bifida SCI ¼ spinal cord injury SF ¼ Short Form Accepted for publication January 8, Study received institutional review board approval. Supported by the CUA Astellas Research Grant Program funded by Astellas Pharma Canada, Inc. and jointly established by Astellas Pharma Canada, Inc. and the Canadian Urological Association. There is no charge for use of the NBSS for academic research, and permission may be obtained by contacting blayne.welk@sjhc.london.on.ca. * Correspondence: Division of Urology, Western University, Room B4-667, St. Joseph s Health Care, 268 Grosvenor St., London, Ontario N6A 4V2, Canada (telephone: ; FAX: ; blayne.welk@sjhc.london. on.ca). Financial interest and/or other relationship with Astellas, Allergan and Pfizer. Financial interest and/or other relationship with AMS, Astellas, Pfizer and Allergan. Purpose: The neurogenic bladder symptom score is a tool to measure urinary symptoms and consequences in patients with acquired or congenital neurogenic bladder. We describe score validity and reliability. Materials and Methods: Exploratory factor analysis was used to assess item variability and subscale structure. Reliability was assessed by the Cronbach a and correlation with retest data. Validity was assessed with a priori hypotheses specifying relationships with the AUASS (American Urological Association symptom score), ICIQ-UI (International Consultation on Incontinence-Urinary Incontinence) and urinary specific quality of life SF-Qualiveen questionnaires, and a self-assessed global bladder problem score. Known groups analysis was used to further assess construct validity. Results: A cohort of 230 patients with spinal cord injury (35%), multiple sclerosis (59%) and congenital neurogenic bladder (6%) were included in study. Factor analysis suggested 3 neurogenic bladder symptom score domains, including incontinence, storage and voiding symptoms, and consequences. Overall internal consistency was high (Cronbach a ¼ 0.89). Test-rest reliability was also excellent with an ICC 2,1 of Validity was demonstrated by the confirmation of hypothesized correlations with the AUASS, ICIQ-UI and SF-Qualiveen, and significant differences in neurogenic bladder symptom score scores among known groups. Patients with a history of seeing a urologist had a significantly higher mean score, as did those with a higher global bladder problem score (22.1 vs 17.1 and 22.1 vs 12.6, respectively, each p <0.001). Conclusions: The neurogenic bladder symptom score, developed specifically to assess symptoms and consequences associated with neurogenic bladder dysfunction, has appropriate psychometric properties. Depending on the measurement need individual domains may be selected or it can be used as a comprehensive score. Key Words: urinary bladder, neurogenic; questionnaires; quality of life; lower urinary tract symptoms; outcome assessment (health care) NEUROGENIC conditions such as SCI, MS and SB are usually associated with significant bladder dysfunction. Signs and symptoms of this dysfunction are variable and can change. 1 However, there is no standard PROM for assessing urological symptoms and consequences. Population specific QOL instruments have been developed for patients with SCI 2 and 452 j /14/ /0 THE JOURNAL OF UROLOGY 2014 by AMERICAN UROLOGICAL ASSOCIATION EDUCATION AND RESEARCH,INC. Vol. 192, , August 2014 Printed in U.S.A.

2 VALIDITY AND RELIABILITY OF NEUROGENIC BLADDER SYMPTOM SCORE 453 MS 3 but urological issues are not reflected in the scores due to the broad nature of the questions. A urinary specific QOL instrument was developed for patients with SCI but the focus of this instrument is feelings rather than symptoms. 4 Other PROMs have been cross-validated, such as the I-QOL (Incontinence QOL) scale for SCI. 5 However, because the items were not developed for patients with neurogenic bladder dysfunction, they may be incomplete or inappropriate. There is a need for a symptom score developed specifically for a wide range of patients with neurogenic bladder dysfunction. NBSS is a discriminative tool developed for patients with SCI, MS and SB. Its 22 questions cover incontinence, storage and voiding symptoms, and urinary complications associated with neurogenic bladder dysfunction (supplementary Appendix, Two additional questions address bladder management and urinary specific QOL. Items use an adjectival scale with 4 or 5 ordinal responses. This is a symptom scale, which attempts to capture objective signs and symptoms, in contrast to a QOL instrument, which would measure subjective social, emotional and physical functioning. 6 QOL and symptom scales measure different concepts and are not necessarily correlated. We report NBSS validity and reliability in patients with neurogenic bladder dysfunction. METHODS Preliminary development of NBSS involved a multidisciplinary group that reviewed items from disease specific QOL instruments, urological symptom specific instruments and the neurogenic bladder literature. Patients were interviewed qualitatively to add additional items. Proposed items were reviewed by external experts and modified based on their feedback. Items were then pretested in patients to ensure readability and interpretability. 7 Patient Recruitment The study cohort was assembled from consecutive patients who attended clinic visits with urologists, physiatrists and neurologists from February to June Study inclusion criteria were adults with SCI, MS or a congenital condition associated with bladder dysfunction, ie SB. Exclusion criteria were recent urological surgery or a change in overall health, recent urinary tract infection or recent change to bladder management or medications. Relevant demographic information was collected. Patients were asked to complete the NBSS, SF-Qualiveen, 8 AUASS 9 and ICIQ-UI SF 10 questionnaires at baseline. They were asked to answer the global bladder assessment question, How much of a problem is your bladder or urinary function? using a Likert scale of 1dno problem to 10dvery significant problem. Patients were also asked to complete NBSS again in approximately 3 weeks to obtain a retest score. Those who indicated that bladder function had changed during that interval were excluded from reliability analysis. Cohort sample size was based on the requirement for at least 10 patients per item (minimum 220 patients) for factor analysis. COSMIN (Consensus-based Standards for the Selection of Health Status Measurement Instruments) guidelines for measurement reporting, specifically internal consistency, reliability and construct validity, were used to guide analysis and reporting. 11 Factor Analysis Factor analysis was performed on 22 of the 24 NBSS items, excluding bladder management and QOL questions. The purpose of factor analysis is to describe the variability of scale items and suggest items that measure a similar conceptual domain. 12 Only patients with complete data were included (226 of 230) since imputed data can artificially inflate factor loadings. Principal components extraction was used to estimate the number of factors. Principal factors extraction with varimax (orthogonal) and oblique rotation was then performed. The goal was to achieve a parsimonious solution with the fewest factors. Factor structure was confirmed using multiple extraction techniques and replicated with retest data. Factor based provisional scales were developed from the factor pattern matrix of the varimax solution. Items with meaningful loadings (greater than 0.30) were included in the provisional scale. If an item had meaningful loading on more than 1 factors, the item was included on the factor with the highest loading. Reliability We used the Cronbach a 13 to measure internal consistency of the subscales and overall reliability of the total scale score, which was calculated from linear combination of the subscales. Test-retest reliability was measured using the correlation between baseline and retest scores. Validity Construct validity was assessed by testing a priori hypothesized correlations between NBSS and other symptom scales (AUASS and ICIQ-UI), a QOL scale (SF- Qualiveen) and the global bladder assessment question. Certain correlations were defined a priori, including a strong positive correlation of AUASS and NBSS storage with voiding, ICIQ-UI with NBSS incontinence and SF- Qualiveen with the NBSS QOL question. A moderate positive correlation was hypothesized between the global bladder assessment score and NBSS consequences, and between SF-Qualiveen and NBSS. Validity was also assessed using differences between known groups. Two a priori hypotheses were tested using the unpaired Student t-test. 1) Patients who had seen a urologist in the prior year would have a higher NBSS score than those who had not seen a urologist. 2) Those with a global bladder assessment score of 5 or greater would have a significantly higher NBSS score than those with a score of less than 5. All statistical analysis was performed using SASÒ 9.2. For reliability and construct validity analyses missing items from questionnaires were replaced using multiple imputation but less than 1% of data were missing for any question. The Pearson correlation coefficient was used at p <0.001 unless otherwise specified with r >0.70

3 454 VALIDITY AND RELIABILITY OF NEUROGENIC BLADDER SYMPTOM SCORE considered strong, r ¼ 0.70 to 0.30 considered moderate and r <0.30 considered weak correlation. 14 The Student t-test was used to assess differences between continuous measures. The ICC 2,1 was used to assess agreement between baseline and retest scores. 15 IQR served as the measure of range. RESULTS A total of 230 patients completed the baseline visit, including 80 (35%) with SCI, 136 (59%) with MS and 14 (6%) with congenital neurogenic bladder (table 1). Of the 230 patients 147 (64%) had not seen a urologist in the last year and 152 (66%) did not use a catheter or urostomy bag. Patients completed the NBSS in a median of 6 minutes (IQR 5e8). The median total score on the 22 items was 19 (IQR 11e26) on a possible scale range of 0 to 74. Median urinary related QOL was 3dmixed with an IQR of 2dmostly satisfied to 4dmostly unsatisfied. Factor analysis suggested that a 3-factor solution would be a parsimonious solution with the fewest reliable factors (table 2). The factors were designated incontinence, storage and voiding symptoms, and consequences. Oblique rotation revealed low correlation among the factors. Therefore, orthogonal rotation was selected, which assumes uncorrelated factors. Table 2 shows the final solution with items grouped by highest factor loading. The factors were well defined by the items in that all squared multiple correlations of the factors were greater than 0.60 (range 0.73 to 0.90). The communality of a variable (the percent of variance accounted for by factors) suggests how well the variable is explained by the factor model. Communality values ranged from 0.14 to 0.71 and only 3 of the 22 items had a communality of less than Simple structure was achieved in that only 1 variable (item 9) had meaningful loading on more than 1 factor (table 2). The 3 factors accounted for 42% of the total variance of the item pool. Factor 1 accounted for 53% of covariance in the solution and the remainder was shared equally by factors 2 and 3. Table 2 also shows the median score, IQR and skewness of each item. Individual item responses were evaluated graphically. Item 14 (pain with voiding or catheters), and items 20 and 21 (frequency of kidney/bladder stones) showed graphic evidence of floor effects. NBSS internal consistency was assessed using the Cronbach a. Values range from 0 to 1 with a score close to 1 indicating a reliable, homogenous scale and values of 0.70 or greater considered indicators of good internal reliability. 12 The overall Cronbach a for NBSS was Individual factors had an a of 0.90 (incontinence), 0.74 (storage and voiding) and 0.69 (consequences) (table 2). The Cronbach a of each factor with each item deleted did not increase significantly, suggesting that each item contributed in a positive way to the internal consistency of the factor. This was replicated in retest data. Almost all item-to-domain correlations were moderate to strong (r 0.30 with r ¼ 0.29 for items 14 and 21) and all were positive. NBSS reliability was also assessed by a testretest method in 131 patients with baseline and retest data available. Three patients who indicated that there had been a significant change in urinary symptoms were excluded from analysis. Mean time between baseline and retest data was 20 days (IQR 16e30). There was no significant difference between baseline and retest scores (mean difference 0.04, p ¼ 0.93). The ICC 2,1 of the total score was 0.91 and for subscales it was 0.87 (incontinence), 0.86 (storage and voiding) and 0.86 (consequences). NBSS cross-sectional concurrent validity was assessed using a priori hypotheses (table 3). All a priori hypotheses were demonstrated except the strong correlation hypothesized between the NBSS QOL question and SF-Qualiveen (r ¼ 0.67, which was just below the required r >0.70). As hypothesized, the mean NBSS score of the 82 patients who had seen a urologist in the previous year (22.1, 95% CI 19.7e24.4) was significantly higher than that of the 147 who had not seen a urologist in the last year Table 1. Characteristics of 230 patients who completed baseline visit based on neurogenic bladder etiology SCI MS Congenital Neurogenic Bladder No. pts (% male) 80 (64) 136 (24) 14 (25) Median age (IQR) 52 (39e63) 47 (38e56) 47 (28e54) Neurogenic details (%) Cervical spine (42), thoracic spine (39), lumbar spine (16), not specified (3) Relapsing remitting (68), secondary progressive (23), primary progressive (4), not specified (5) SB (64), exstrophy (7), sacral agenesis (14), unspecified (14) Median yrs neurological disease history (IQR) 10 (3e22) 13 (7e20) 47 (28e54) % Bladder management: Indwelling catheter/urostomy bag Condom catheter Intermittent catheter Voiding % Seen by urologist in last yr % Previous urological operations

4 VALIDITY AND RELIABILITY OF NEUROGENIC BLADDER SYMPTOM SCORE 455 Table 2. NBSS squared multiple correlations, final communalities and rotated factor pattern matrix, and internal consistency using baseline scores of 230 patients Item No. (description) Factor 1 (incontinence) Factor 2 (storage þ voiding) Factor 3 (consequences) Final Communality Median Score* Skewness Coefficient Correlation Cronbach a Factor 1eincontinence (a ¼ 0.90): 4eDaytime urine leakage quantity (No. pads) (0e2) eDaytime urine leakage quantity (0e2) (pad saturation) 2eDaytime urine leakage frequency (0e3) eLongest no incontinence daytime interval (0e2) eUrine leakage limits activities (0e2) eUrine leakage changed amount of (0e2) liquid consumed 7eUrine leakage caused skin problems (0e1) eUrine leakage nighttime frequency (0e1) Factor 2estorage þ voiding (a ¼ 0.74): 9eFrequency of sudden urge to urinate (1e2) (or bladder spasm) 11eNocturia/nocturnal awakenings (0e3) related to bladder 16eUrinary stream strength (0e1) eUrination or clean intermittent (0e1) catheterization urgency 15ePost-void/post-clean intermittent (0e1) catheterization fullness 12eLongest urination or clean intermittent (1e2) catheterization daytime interval 17eStraining (0e1) Factor 3econsequences (a ¼ 0.69): 18eUrinary tract infection frequency (0e2) eUrinary tract infection severity (0e2) eEffectiveness of medications related (0e1) to bladder 14ePain associated with urinating or (0e1) using urinary catheters 22eNeed for medications related to bladder (0e1) eKidney stone frequency (0e0) eBladder stone frequency (0e0) Squared multiple correlation e e e e e Eigenvalue (l) e e e e e % Variance e e e e e % Covariance e e e e e * Measure of floor and ceiling effects. With subscale total score to assess factor internal consistency. For factor with item deleted to measure item importance to subscale and whether reliability would improve significantly by removing item. Highest item squared multiple correlation. (17.1, 95% CI 15.5e18.6, p <0.001). Similarly those with a global bladder assessment score of 5 or greater had a mean NBSS score that was significantly higher than the score of those with a lower global bladder assessment score (22.1, 95% CI 20.6e23.7 vs 12.6, 95% CI 10.8e14.4, p <0.001). DISCUSSION NBSS is a PROM designed specifically to assess signs and symptoms related to neurogenic bladder dysfunction. Development of a PROM is an iterative process that requires qualitative and quantitative study. 16 Before using a PROM it is important to Table 3. NBSS construct validity based on a prior hypotheses NBSS Relationship Hypothesized Strength* Pearson Correlation AUASS Total ICIQ-UI Total Global Bladder Assessment SF-Qualiveen Total Total Moderate Incontinence subscale Strong Storage þ voiding subscale Strong Consequences subscale Moderate QOL item Strong * Hypothesized direction was positive. Greater than 0.70 considered strong, 0.70 to 0.30 considered moderate and less than 0.30 considered weak, and all correlations were significant (p <0.001). Result of hypothesized relationship.

5 456 VALIDITY AND RELIABILITY OF NEUROGENIC BLADDER SYMPTOM SCORE review the development process and ensure that the scale is adequate based on the measurement purpose, conceptual framework, intended population and completeness of items and responses. NBSS feasibility and sensibility 17 were demonstrated in this study. It was successfully self-administered, there was a low and random proportion of missing responses, questions required minimal specific recall of temporal events and the time requirement was modest. Exploratory factor analysis allows items that share a similar variance to be grouped into factors. A 3-factor solution matched well with the clinical interpretation of the items. Each factor is unipolar, appropriately populated with items and contributes an acceptable amount of variance to the scale. The results of factor analysis, and the subsequent domain specific validity and reliability analysis supports the use of these 3 domains as reliable independent subscales. Items were evaluated for poor psychometric performance and potential removal. Items 14, 20 and 21 (pain with voiding and bladder/renal stones) had significant floor effects and were weakly explained by the factor structure as evidenced by the low communalities. However, they do not adversely affect the internal consistency of the scale since the deleted Cronbach a remained stable. During NBSS development they were believed to be highly relevant and so they were retained. Reliability is the amount of variation of a scale attributable to the true score as opposed to measurement error. 16 A highly reliable scale is interpreted as being precise. Two measures of reliability were assessed. The Cronbach a for internal consistency (0.89) and intrarater reliability or ICC 2,1 based on baseline and retest measurements (0.91) were excellent. 16 Our retest interval was appropriate and our patient population seemed to be stable since there was no significant difference in mean scores. Individual domains showed good internal consistency (Cronbach a 0.69 and intrarater reliability 0.86 or greater). The Cronbach a is a measure of the intercorrelation of items. Therefore, it is not surprising that the consequences domain had the lowest a because items such as urinary tract infection and stone disease may not be correlated within patients. Validity refers to the ability of a scale to measure what it intends to measure. NBSS face validity is subjective ability of the items to represent neurogenic bladder dysfunction symptoms and consequences. Content validity is the extent to which NBSS can thoroughly assess neurogenic bladder dysfunction. A thorough item generation process with expert and patient input and feedback during the development process maximizes these validities. 7 Construct validity is the ability of NBSS to measure the symptoms and consequences that it claims to measure. This was demonstrated in this study by hypothesis testing based on established measurement instruments in the field of urological symptoms and neurogenic bladder QOL. AUASS is a well studied tool for assessing storage and voiding symptoms. 18 I-QOL-Urinary Incontinence is an incontinence symptom scale that assesses incontinence frequency, amount and impact. 5 SF- Qualiveen is a urinary specific QOL instrument that was developed for patients with SCI 4 and validated for use in patients with MS. 19 We successfully noted moderate or strong correlations of NBSS and its subscales with the independent instruments. In addition, we used known groups to determine significant differences in NBSS scores between patients with vs without a reason to have more significant symptoms. It is important to acknowledge NBSS limitations. During its creation we tried to make questions applicable to patients using a wide range of bladder management techniques. We avoided any judgment on the appropriateness or superiority of one bladder management technique over another and instead focused only on symptoms. For example, a patient who voids spontaneously but has significant incontinence would have a worse score but a patient with an indwelling catheter may not have incontinence or catheter bypassing and, therefore, have a better score. The influence of specific bladder management techniques and the comparability of scores between these groups requires further study. Due to the relative rarity of adults with congenital neurogenic bladder these patients had limited representation in our cohort. Finally, the interpretability of NBSS scores and scale responsiveness will be evaluated in a future study. CONCLUSIONS NBSS is a reliable, valid tool to measure the symptoms and consequences of neurogenic bladder dysfunction. Many items cover questions that are part of the urological functional inquiry. However, NBSS allows these concepts to be measured with a continuous scale using standard language and responses, and in a self-reported manner. Previous studies generally focused on incontinence but the 3 NBSS domains better represent the spectrum of neurogenic bladder dysfunction and associated complications. Each NBSS domain is valid and reliable, and may be used individually depending on clinical need. ACKNOWLEDGMENTS Drs. Patrick Potter, Eldon Loh and Marcelo Kremenchutzy assisted with patient recruitment.

6 VALIDITY AND RELIABILITY OF NEUROGENIC BLADDER SYMPTOM SCORE 457 REFERENCES 1. Wyndaele JJ, Kovindha A, Madersbacher H et al: Neurologic urinary incontinence. Neurourol Urodyn 2010; 29: Renwick R, Nourhaghighi N, Manns PJ et al: Quality of life for people with physical disabilities: a new instrument. Int J Rehabil Res 2003; 26: Vickrey BG, Hays RD, Harooni R et al: A healthrelated quality of life measure for multiple sclerosis. Qual Life Res 1995; 4: Costa P, Perrouin-Verbe B, Colvez A et al: Quality of life in spinal cord injury patients with urinary difficulties. Development and validation of Qualiveen. Eur Urol 2001; 39: Schurch B, Denys P, Kozma CM et al: Reliability and validity of the Incontinence Quality of Life questionnaire in patients with neurogenic urinary incontinence. Arch Phys Med Rehabil 2007; 88: Leplege A and Hunt S: The problem of quality of life in medicine. JAMA 1997; 278: Welk B, Morrow S, Madarasz W et al: The conceptualization and development of a patientreported neurogenic bladder symptom score. Res Rep Urol 2013; 5: Bonniaud V, Bryant D, Parratte B et al: Development and validation of the short form of a urinary quality of life questionnaire: SF- Qualiveen. J Urol 2008; 180: Barry MJ, Fowler FJ, O Leary MP et al: The American Urological Association symptom index for benign prostatic hyperplasia. The Measurement Committee of the American Urological Association. J Urol 1992; 148: Avery K, Donovan J, Peters TJ et al: ICIQ: a brief and robust measure for evaluating the symptoms and impact of urinary incontinence. Neurourol Urodyn 2004; 23: Mokkink LB, Terwee CB, Patrick DL et al: The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010; 19: Norman GR and Streiner DL: Biostatistics: The Bare Essentials. Beijing: PMPH-USA Streiner DL: Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess 2003; 80: McHorney CA, Ware JE and Raczek AE: The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care 1993; 31: Bravo G and Potvin L: Estimating the reliability of continuous measures with Cronbach s alpha or the intraclass correlation coefficient: toward the integration of two traditions. J Clin Epidemiol 1991; 44: Streiner DL and Norman GR: Health Measurement Scales: A Practical Guide to Their Development and Use, 4th ed. Oxford, United Kingdom: Oxford University Press Rowe BH and Oxman AD: An assessment of the sensibility of a quality-of-life instrument. Am J Emerg Med 1993; 11: Barry MJ, Williford WO, Chang Y et al: Benign prostatic hyperplasia specific health status measures in clinical research: how much change in the American Urological Association symptom index and the benign prostatic hyperplasia impact index is perceptible to patients? J Urol 1995; 154: Bonniaud V, Bryant D, Parratte B et al: Qualiveen: a urinary disorder-specific instrument for use in clinical trials in multiple sclerosis. Arch Phys Med Rehabil 2006; 87: 1661.