The Patient-Specific Functional Scale (PSFS) was designed

Transcription

1 [ research report ] CHERYL HEFFORD, MPhty 1 J. HAXBY ABBOTT, PT, PhD, FNZCP 2 RICHARD ARNOLD, PhD 3 G. DAVID BAXTER, DPhil 4 The Patient-Specific Functional Scale: Validity, Reliability, and Responsiveness in Patients With Upper Extremity Musculoskeletal Problems The Patient-Specific Functional Scale (PSFS) was designed to provide clinicians with a valid, reliable, responsive, and efficient outcome measure that would be easy to use and applicable to a large number of clinical presentations. 29 As a clinical outcome measure, the PSFS allows patients to report on their functional status at baseline and at a follow-up session, to determine if a meaningful change in functional status has occurred. Since its TTSTUDY DESIGN: Clinical measurement, longitudinal; multicenter prospective cohort study. TTOBJECTIVES: To examine the validity, reliability, and responsiveness of the Patient-Specific Functional Scale (PSFS) in patients with musculoskeletal upper extremity problems being treated in physical therapy. TTBACKGROUND: The clinimetric properties of the PSFS have not been established nor compared with region-specific outcome measures in patients with upper extremity problems. TTMETHODS: Patients completed the PSFS, Upper Extremity Functional Index (UEFI), and numeric pain rating scale (NPRS) at baseline and follow-up, and were categorized as improved, stable, or worsened, using the global rating of change. Construct validity was assessed by comparing the change scores of the stable and improved groups, using independent-samples t tests. Reliability was evaluated using intraclass correlation coefficient (ICC 2,1 ) with 95% confidence intervals. Bland-Altman plots determined limits of agreement. Responsiveness and minimal important difference (MID) were determined with receiver operator characteristic (ROC) curves. TTRESULTS: One hundred eighty patients met the inclusion criteria. Construct validity was supported for the PSFS and the UEFI (P<.001). Reliability was moderate to good for the PSFS (ICC 2,1 = 0.713) and UEFI (ICC 2,1 = 0.848). Reported estimates of reliability may be lower than true values because the group of stable patients from this cohort had, on average, a small positive change. Bland-Altman plots indicated good agreement. The area under the ROC curve (AUC) was significantly different from the null value of 0.5 for the PSFS (0.887) and the UEFI (0.877), indicating good accuracy in distinguishing improved patients from stable patients. MID was 1.2 for the PSFS (scale, 0-10) and 8.5 for the UEFI (scale, 0-80). TTCONCLUSION: The PSFS is a valid, reliable, and responsive outcome measure for patients with upper extremity problems. J Orthop Sports Phys Ther 2012;42(2): doi: /jospt TTKEY WORDS: clinical measurement, instrument validation, outcome measure, upper limb development in 1995 for patients with low back pain, the PSFS has been compared to the Medical Outcomes Study 36-Item Short-Form Health Survey (SF- 36) for patients with knee dysfunction 5 and chronic whiplash 28 and for workers compensation claimants for a variety of conditions, 12 the Neck Disability Index (NDI) for neck dysfunction, 33 cervical radiculopathy, 6 and chronic whiplash, 28 and the Roland-Morris Disability Questionnaire (RMQ) for low back pain. 21 No study to date has compared the PSFS to an upper extremity outcome measure nor examined the psychometric properties of the PSFS for patients with musculoskeletal problems of the upper extremity. Region-specific outcome measures for the entire upper extremity include, in order of frequency of citation, the Disability of the Arm, Shoulder and Hand (DASH) and its shorter version, the Quick DASH, 2,3,16 the Upper Extremity Functional Scale (UEFS), 22 the Upper Extremity Functional Index (UEFI), 30 and the Upper Limb Functional Index (ULFI). 11 There are also a number of other upper extremity outcome measurement tools pertaining to specific joints or conditions. The purpose of this study was to de- 1 Professional Practice Fellow, Center for Physiotherapy Research, School of Physiotherapy, University of Otago, Dunedin, New Zealand. 2 Sir Charles Hercus Health Research Fellow, Department of Surgical Sciences, Orthopedic Surgery Section, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand. 3 Senior Lecturer, School of Mathematics, Statistics and Operations Research, Victoria University of Wellington, New Zealand. 4 Professor, Center for Physiotherapy Research, School of Physiotherapy, University of Otago, Dunedin, New Zealand. The protocol of this study was approved by the Multi-region Ethics Committee of the New Zealand Ministry of Health. Address correspondence to Cheryl Hefford, Center for Physiotherapy Research, School of Physiotherapy, University of Otago, PO Box 56, Dunedin, New Zealand. cheryl.hefford@otago.ac.nz 56 february 2012 volume 42 number 2 journal of orthopaedic & sports physical therapy

2 TABLE 1 termine whether the PSFS could be used instead of a region-specific outcome measure for determining change in function for patients with musculoskeletal problems of the upper extremity. METHODS This was a multicenter cohort study of patients with musculoskeletal disorders who underwent an episode of physical therapy for disorders of the upper limb. National Ethics approval for an observational study was granted. Patients attending any of 5 School of Physiotherapy clinics (located in 3 New Zealand cities) for musculoskeletal problems of the upper extremity over a 6-month period were eligible. As A Description of the Outcome Measures (OMs) Used in This Study OM Type of OM Description Scoring PSFS Patient-specific Patients nominate 3 to 5 activities that they are unable to do or having difficulty doing because of their injury or problem. These activities are rated on an 11-point scale, where 0 is unable to perform the activity and 10 is able to perform the activity at preinjury level. UEFI Region-specific Twenty prescribed upper extremity activities are rated by patients according to a 5-point scale, where 0 is extreme difficulty or unable to perform the activity, 1 is quite a bit of difficulty, 2 is moderate difficulty, 3 is a little bit of difficulty, and 4 is no difficulty. NPRS Dimension-specific (pain) Patients are asked to rate their current pain and their worst and best in the last 24 hours on an 11-point scale, where 0 is no pain and 10 is the worst pain imaginable. GROC Generic Patients rate their degree of improvement (or worsening) at follow-up or end of therapy, ranging from a very great deal worse to a very great deal better, compared retrospectively to their initial visit. An average score for the activities gives an ability score out of 10. A sum of the scores gives an ability score out of 80. A sum of the current, worst, and best pain scores gives a deficit score out of 30. Each statement relates to a number on a 15-point scale. A rating of no change (8) is the midpoint. Abbreviations: GROC, Global Rating of Change; NPRS, numeric pain rating scale; PSFS, Patient- Specific Functional Scale; UEFI, Upper Extremity Functional Index. patients in New Zealand have direct access to physical therapy, the patients were either self-referred or referred by their doctors. Two of the centers (comprising 4 clinics) were within a University Student Health setting, and 1 was in a suburban community. To meet the study inclusion criteria, patients had to be over 18 years of age and to have reported a problem in their upper extremity, which included the area distal to the acromioclavicular joint. A body diagram was used to help patients indicate the area of their symptoms. All eligible patients were asked to complete the PSFS and the UEFI, as well as a numeric pain rating scale 10 (NPRS), at their baseline visit. The UEFI was given to patients by nonclinical administrative staff, who instructed them to fill in the instrument prior to seeing their therapist. The PSFS and NPRS were completed during the patient assessment and administered by the therapist. The UEFI was chosen as a comparison measure for several reasons: (1) preliminary usage of the DASH within our clinics revealed that patients frequently failed to notice and complete the second page, causing administrative delays and missing data; (2) patients frequently questioned the relevance of the DASH question about sexual activity, which nonclinical staff who administered the questionnaire found awkward to explain; and (3) the format of the UEFI is consistent with that of the Lower Extremity Functional Scale (LEFS), which was being used in a concurrent study. These outcome measures are described in TABLE 1. At followup visits (second, sixth, and/or discharge visit), patients again completed these outcome measures (UEFI, PSFS, and NPRS), as well as the 15-point global rating of change (GROC) instrument. 18 Prior to the start of the study, physical therapists and administrators underwent training to ensure uniformity of data collection across the centers. They were each provided with manuals to guide them through the process of collecting, entering, and retrieving data. Data Analysis At the time of study conception, guidelines for estimating sample size in observational studies of outcome measure responsiveness using anchor-based analysis methods were not available; thus a minimum target of 100 sets of complete data in the final sample was established. In the sampling time frame set for this observational cohort study (May 2007 to November 2007), we anticipated recruiting at least 130 participants. As the study design was a pragmatic and not an experimental one, there was no control group and all patients received usual physical therapy treatment. All outcome data were entered into the clinics patient management database (Medtech32; Medtech Global Ltd, Auck- journal of orthopaedic & sports physical therapy volume 42 number 2 february

3 [ research report ] land, New Zealand), along with the usual patient data. Data relevant to the study were extracted and exported to Microsoft Excel. This information was sorted into patients with complete data and those with incomplete data. Data were considered complete if there were both initial and follow-up outcome measure scores for the UEFI, PSFS, and NPRS, plus a GROC for at least 1 of the second, sixth, or discharge visits. Data were considered incomplete if any outcome measures or GROC summary scores were missing. These incomplete data were not included in the statistical analyses; they were, however, included in the demographic analysis of the study, to assess for possible nonresponse bias by comparing patients with complete and incomplete outcome measures. To assess the capability of the PSFS to distinguish between patients who improved or did not improve, the GROC was used as an anchor measure to establish 3 groups: those who improved, remained stable, or became worse. Therefore, patients in the group with complete data were trichotomized into an improved group (GROC, 12), a stable group (GROC, 6-11), and a worsened group (GROC, 5). These cut-off points were described by Jaeschke et al 18 and have been used in previous studies. 6 These data were then exported to SPSS Version (SPSS, Inc, Chicago, IL) for further analysis of validity, reliability, and responsiveness. Changes in outcome measure scores were calculated by determining the difference between the baseline and followup scores for each outcome measure. The mean and standard deviation were determined for changes in each outcome measure. Validity Construct validity of the PSFS was examined by comparing the mean change scores within and between the groups classified as stable or improved by the GROC. This was performed as follows: 1. The hypothesis of a zero mean change within each group was tested by using a single sample t test separately for each group. No change (mean of zero) in the stable group and a significant (nonzero) mean change in the improved group were expected. 2. The hypothesis of different means between the 2 groups was tested by using an independent-samples t test. 6,8,9 A significant difference (P<.01) between the stable group and the improved group was expected. Concurrent validity was examined by correlating the score changes of the PSFS, NPRS, UEFI, and GROC. We considered correlation coefficients of 0.7 or better to be strong, and 0.5 to 0.7 to be moderate. Previous studies of the UEFI have supported the responsiveness of the UEFI 30 ; therefore, it was expected that the PSFS would correlate moderately well with the UEFI and the GROC, and less strongly with the NPRS, which is a measurement of pain rather than function. Reliability Reliability was examined for the PSFS, UEFI, and NPRS in the group of stable patients (GROC, 6-11), using an intraclass correlation coefficient (ICC 2,1 ) with 95% confidence intervals (CIs). 1,8,9,27,31 The reliability of an outcome measure is the extent to which the outcome measure scores remain consistent over repeated tests of the same patient under identical conditions. Therefore, a test for reliability ideally involves repeated measures within a short period in an untreated group. However, due to the pragmatic, observational cohort design of the present study, such data were not available. We therefore calculated the ICC for the stable group, based on GROC scores that indicated minimal change between measurements. An high ICC value (close to +1) indicated the consistency of measures in this unchanged stable group. 27 The minimum detectable change (MDC) is the measure of the statistically reliable change of an outcome measure. 24 This was calculated for the PSFS, UEFI, and NPRS using the following formula 24 : MDC = z 2 MSE, where z = 1.64 and is the score associated with a 90% CI, 6,24 2 reflects the uncertainty introduced by using scores at 2 different points in time, and the square root of the meansquare error (MSE) term represents the standard error of measurement (SEM). The MSE was found by constructing a 1-way analysis of variance (ANOVA) table of the baseline and follow-up scores of the stable group. 24 In addition, Bland-Altman plots were created for the PSFS, UEFI, and NPRS. Sole reliance on ICC has been questioned by Bland and Altman, 4 who maintained that the magnitude of an acceptable difference is a clinical decision rather than a statistical one. Bland and Altman 4 offered an alternative, which estimated the limits of agreement, wherein the differences in agreement for each subject were considered by plotting the difference of each measurement against the mean difference of the measurements in a Bland- Altman plot. In such plots, the data from a reliable measurement instrument will scatter evenly around the mean difference, with the variability independent of the size of the score (ie, there should be no funneling in the plot). Responsiveness External responsiveness was determined, with an individual-focused or longitudinal approach, using the GROC as an anchor. Receiver operator characteristic (ROC) curves were plotted for each outcome measure. The area under the ROC curve (AUC) indicates the degree of accuracy of each outcome measure in classifying improved and stable patients, whereby an AUC of 0.5 denotes no accuracy beyond chance and an AUC of 1.0 denotes perfect agreement with a gold standard classification. 8 DeLong s test 7 was used to compare paired AUC values and performed using the proc package in the R statistical program ( The minimal important difference (MID) is defined as the smallest difference in score in the outcome of inter- 58 february 2012 volume 42 number 2 journal of orthopaedic & sports physical therapy

4 Dunedin, n = 873 Christchurch, n = 508 Wellington, n = 475 Combined total, n = 1856 UEFI completed at initial visit: Dunedin, n = 129 Christchurch, n = 108 Wellington, n = 49 Total, n = 286 Complete data set: Dunedin, n = 72 Christchurch, n = 77 Wellington, n = 31 Total, n = 180 Worse group (GROC, 5), n = 0 est that informed patients or informed proxies perceive as important, either beneficial or harmful, and which would lead the patient or clinician to consider a change in the management. 25 MID and minimal clinically important difference (MCID) are often used interchangeably; however, there is a subtle difference in the semantic emphasis of these terms. The reason for removing the word clinical was to shift the emphasis away from clinician rating toward patient rating. 26 For this study, MID was used, which was calculated by identifying the optimal cut-offs in the change scores of the PSFS, UEFI, and NPRS for distinguishing improved and stable patients according to self-reported GROC. This was determined by the point on the ROC curve nearest the upper left-hand corner. 6,8 Sensitivity and specificity values and 95% CIs were calculated for the selected cut-off scores. RESULTS Under 18 years, n = 4 Data form invalid, n = 4* Outlier, n = 1 Total exclusions, n = 9 Incomplete data: Dunedin, n = 55 Christchurch, n = 26 Wellington, n = 16 Total, n = 97 Stable group (GROC, 6-11) at: Discharge, n = 19 2nd visit, n = 8 6th visit, n = 11 Total, n = 38 A total of 286 patients were identified as having upper extremity problems at baseline (FIGURE 1). This represented 15% of the 1856 patients Exclusions: Regions other than upper extremity: Dunedin, n = 694 Christchurch, n = 331 Wellington, n = 397 Total, n = 1422 No regional outcome measure: Total, n = 148 Total exclusions, n = 1579 Improved group (GROC, 12) at: Discharge, n = 128 2nd visit, n = 5 6th visit, n = 9 Total, n = 142 FIGURE 1. Participant and data flow chart. Abbreviations: GROC, global rating of change; UEFI, Upper Extremity Functional Index. *The UEFI was invalid if more than 1 item was missing on the form. Outlier: 1 patient completed the UEFI at baseline but had a condition code for a lower limb problem and had PSFS and UEFI scores that were 10/10 and 80/80, respectively, at baseline and follow-up, with an NPRS score of 1/30 at baseline and 0/30 at follow-up. This patient was, therefore, excluded because the integrity of this data set was questionable. with musculoskeletal problems who attended the clinics over the sampling period. Nine patients were excluded, and 277 patients completed the UEFI at baseline. Of these 277 patients, 180 completed outcome measures at a follow-up visit, surpassing our intended minimum sample size of 100. Of the 97 patients with incomplete outcome measures, the most common reason was having only baseline data and no GROC (74 patients [75%]); the other 23 patients (24%) attended the clinic for a single visit. The mean ages and proportions of males and females were similar in the complete and incom- journal of orthopaedic & sports physical therapy volume 42 number 2 february

5 [ research report ] plete outcome measure groups overall and for each individual center (TABLE 2). Patients with completed outcome measures were further classified, according to their GROC score, into those who improved (142) and those who were stable (38); there were no patients who became worse (FIGURE 1). Area of Injury Injury codes were aggregated into the following 4 regions: (1) shoulder and upper arm, (2) elbow and forearm, (3) wrist and hand, and (4) spine. Spine indicated that patients who had originally presented with arm symptoms and completed the UEFI outcome measure but were subsequently diagnosed as having a problem that originated in the spine. The most common area of injury was the shoulder and upper arm (46%), followed by the wrist and hand (24%), elbow and forearm (16%), and spine (8%). There was a small group (6%) for whom the injury code was not recorded but who completed the UEFI. This pattern was similar for each of the centers, and patients in the complete group were similar in injury site distribution to those in the incomplete group. The final data set consisted of 180 patients who completed the UEFI, PSFS, and NPRS at the initial visit (baseline) and at a follow-up visit, as well as the GROC at follow-up. There were no significant differences in baseline characteristics between the stable and improved groups other than the time between baseline and follow-up (TABLE 2). The data set included the change in score from baseline to follow-up for each outcome measure. Across all 3 outcome measures, mean baseline scores for both the stable and improved groups were similar, whereas follow-up scores differed strongly in the expected direction (TABLE 3). TABLE 2 Construct Validity Paired-samples t tests demonstrated that the mean change score was consistent, with zero in the stable group and significantly different from zero (in the appropriate direction) in the improved group (TABLE 3). Independent-samples t tests demonstrated a significant difference in mean change between the stable and improved groups (TABLE 4). Differences for all outcome measures were highly significant (P<.001), providing evidence of the construct validity of the PSFS. Concurrent Validity The concurrent validity of the PSFS was supported by moderate correlation with the UEFI (0.59; 95% CI: 0.48, 0.67) and the NPRS (0.51; 95% CI: 0.39, 0.61) (TABLE 4). Reliability The ICC 2.1 of the stable group is reported for each outcome measure in TABLE 4. The moderate to high values ( ) indicated good reliability for all 3 measures. Bland-Altman plots for the PSFS, UEFI, and NPRS were constructed (FIG- URE 2). There was an even scatter of points in these plots and no funneling; thus there was no evidence of systematic change in variability with the level of the score. Within the stable group, the mean values of the change scores were not significantly different from zero (P =.055, P =.304, and P =.676 for the PSFS, UEFI, and NPRS, respectively), indicating that there was an absence of additive bias and the GROC 6-to-11 criterion identified a stable Baseline Characteristics for Stable and Improved Patients* Characteristics Stable Patients (n = 38) Improved Patients (n = 142) P Value Age, y Female, % Initial PSFS score Initial UEFI score Initial NPRS score Number of visits, n Days between evaluations, d Abbreviations: NPRS, numeric pain rating scale; PSFS, Patient-Specific Functional Scale; UEFI, Upper Extremity Functional Index. *Values are means SD unless otherwise indicated. 0 to 10, with higher scores indicating better function. 0 to 80, with higher scores indicating better function. 0 to 30, with higher scores indicating more pain. group. There was no evidence of multiplicative bias (ie, no sloping pattern of points in the graph); however, none was expected, due to use of the same instrument for both baseline and follow-up scores. Minimum Detectable Change (MDC) The MDC for each outcome measure (with 95% bootstrap CIs), based on the variation in repeated measurements in the stable group, is reported in TABLE 4. Responsiveness The AUCs of the ROC curves were all significantly different from the null value of 0.5, indicating that the PSFS, NPRS, and UEFI outcome measures are all effective in distinguishing improved patients from stable patients (TABLE 4). DeLong s test 7 for paired AUC values did not find any significant difference between the AUC values for the 3 outcome measures (PSFS and UEFI, P =.73; PSFS and NPRS, P =.57; UEFI and NPRS, P =.38). The MID for the PSFS was a change in score of 1.2 points (TABLE 4). DISCUSSION This study presents evidence supporting the validity, reliability, and responsiveness of the PSFS for determining a meaningful change in function for patients with musculoskeletal 60 february 2012 volume 42 number 2 journal of orthopaedic & sports physical therapy

6 TABLE 3 Change in Outcome Measure Scores, With Baseline and Follow-up Scores, for the Stable and Improved Groups of Patients* Abbreviations: NPRS, numeric pain rating scale; PSFS, Patient-Specific Functional Scale; UEFI, Upper Extremity Functional Index. *Values are mean SD. P values test the null hypothesis that mean values for the change scores do not differ from zero. 0 to 10, with higher scores indicating better function. 0 to 80, with higher scores indicating better function. 0 to 30, with higher scores indicating more pain. Baseline Score Follow-up Score Change Score (P Value) Outcome Measure Stable Improved Stable Improved Stable Improved PSFS (.0548) (<.0001) UEFI (.3042) (<.0001) NPRS (.6755) (<.0001) TABLE 4 problems of the upper extremity. Our results indicate that the PSFS is comparable to the UEFI for this purpose. The MID is considered the single most important clinimetric value associated with a clinical outcome measure, as it represents the amount of change that patients perceive as meaningful. 32 In this study, the MID for the PSFS (1.2) was found to be similar to the only other report of a MID for the PSFS, for patients with cervical radiculopathy (MID, 2.0) 6 (TABLE 5). Validity The construct validity of the PSFS, UEFI, and NPRS was established with high significance (P<.001). Construct validity can be measured in many different Construct Validity, Concurrent Validity, and Clinimetric Properties for the PSFS, UEFI, and NPRS Outcome Measure Construct Validity* Concurrent Validity Reliability MDC (95% CI) Abbreviations: AUC, area under the curve; CI, confidence interval; ICC, intraclass correlation coefficient; MDC, minimum detectable change; MID, minimal important difference; NPRS, numeric pain rating scale; PSFS, Patient-Specific Functional Scale; SEM, standard error of the mean; UEFI, Upper Extremity Functional Index. *Values are P value, mean difference, SEM. Values are correlation with PSFS change scores; 95% CI; P value. ICC 2,1 (95% CI). 0 to 10, with higher scores indicating better function. 0 to 80, with higher scores indicating better function. 0 to 30, with higher scores indicating more pain. Responsiveness, AUC (95% CI) MID (Sensitivity, Specificity) PSFS <.001, 3.4, (0.51, 0.84) 3.0 (1.7, 4.2) 0.89 (0.83, 0.95) 1.16 (0.88, 0.79) UEFI <.001, 17.9, ; 0.48, 0.67; < (0.73, 0.92) 17.6 (7.3, 25.4) 0.88 (0.81, 0.94) 8.50 (0.73, 0.92) NPRS <.001, 6.6, ; 0.39, 0.61; < (0.55, 0.86) 5.7 (3.8, 7.2) 0.90 (0.85, 0.96) 2.50 (0.80, 0.87) ways, using different constructs. In this study, we hypothesized that the improved group (GROC, 12) would exhibit a significantly larger change in score on the PSFS than the stable group (GROC score, 6-11). This hypothesis was confirmed, satisfying the construct that the PSFS can distinguish improved patients from patients without improvement. For concurrent validity, the change in score on the PSFS was moderately correlated with the change in score on the UEFI. Because these outcome measures do not evaluate identical constructs but, rather, related constructs, this relationship was expected. Correlation of the PSFS with the NPRS was, however, only slightly less than with the UEFI, indicating a relationship between pain score and function score changes. One explanation for this may be that patients perceived pain and function together (eg, a number of participants reported activities in the PSFS such as to be able to wash my hair without pain ). Reliability The reliability of the PSFS in this study, as determined by ICC 2,1, was less than expected (0.71; 95% CI: 0.51, 0.84), and similar to that of both the NPRS (0.74; 95% CI: 0.55, 0.89) and the UEFI (0.85; 95% CI: 0.73, 0.92), the 3 confidence intervals covering similar ranges. However, these may be biased downward. As noted above, due to the pragmatic observation- journal of orthopaedic & sports physical therapy volume 42 number 2 february

7 [ research report ] al cohort design of this clinical study, we do not have measurements for untreated individuals within a short period, and the interval between baseline and follow-up scores for the stable group ranged from 2 to 53 days (mean SD, days). Patients were identified as stable if their GROC was between 6 and 11; but most of the stable group reported small improvement. This potentially poses a problem for the stability of the stable group and would likely adversely affect the reliability and MDC scores. Therefore, we consider these a conservative estimate and the true values as likely being better. The reliability of the PSFS, as determined by the Bland-Altman plot, indicates consistent accuracy over the full range of measurement. There are no other studies of the PSFS reporting Bland-Altman plots. Responsiveness The high AUC of the PSFS (0.89) was similar to that of the UEFI and the NPRS, indicating a high level of accuracy in classifying stable and improved patients. DeLong s test 7 for paired AUC values found no significant difference in responsiveness between the outcome measures used. This multicenter observational study was part of a large consecutive cohort involving all patients with musculoskeletal problems presenting to participating clinics. The sample size of this study exceeds all but 1 of earlier comparable studies of the PSFS (TABLE 5). Our estimates of reliability and agreement for 3 outcome measures might have underestimated the true values, because the stable group predominantly contained patients who had improved somewhat (GROC, >8), but not enough to pass the defined threshold value for improved (GROC, 12). As a consequence, the MDC reported here is also likely to be biased upward, resulting in a value slightly higher than that necessary to confirm that a change in score occurred. Notwithstanding the limitations of determining a stable group of patients in a primarily acute and subacute clinical setting, the PSFS Change Score UEFI Change Score NPRS Change Score Mean PSFS Mean UEFI Mean NPRS above-reported results establish the reliability of the PSFS. In this study, the number of items patients nominated in the PSFS ranged from 1 to 5. The PSFS asks for 3 to 5 activities 29 ; in our data, the vast majority of patients (71%) nominated 3 activities. Also, there have been different approaches regarding whether patients should be informed of their previous scores when Descriptive statistics for PSFS change score in the stable group n Minimum Maximum Mean SD Descriptive statistics for UEFI change score in the stable group n Minimum Maximum Mean SD Descriptive statistics for NPRS change score in the stable group n Minimum Maximum Mean SD FIGURE 2. Bland-Altman plots and descriptive statistics for the changes in score in the stable group for the PSFS (0-10, with higher values indicating better function), UEFI (0-80, with higher values indicating better function), and NPRS (0-10, with higher values indicating more pain). Abbreviations: NPRS, numeric pain rating scale; PSFS, Patient-Specific Functional Scale; SD, standard deviation; UEFI, Upper Extremity Functional Index. completing the PSFS at follow-up; some inform patients 29,33 and some do not. 5,6 In a study that examined the data of a randomized controlled trial in terms of whether participants were informed of the previous scores of an outcome measure, the authors concluded that being informed of previous scores had a calibration effect that resulted in more valid responses rather than bias. 13 In this study, 62 february 2012 volume 42 number 2 journal of orthopaedic & sports physical therapy

8 TABLE 5 Comparison of PSFS Responsiveness, Reliability, and Validity Study Comparison Measures n Responsiveness MID SEM Reliability MDC Validity Stratford et al (1995) 29 RMQ for low back pain (not chronic) Abbreviations: AUC, area under the curve; ES, effect size; GRI, Guyatt Responsiveness Index; ICC, intraclass correlation coefficient; MDC 90, minimum detectable change to 90% confidence; MID, minimal important difference; NDI, Neck Disability Index; NR, not reported; PSFS, Patient-Specific Functional Scale; RMQ, Roland-Morris Disability Questionnaire; SEM, standard error of measurement; SF-36, 36-Item Short-Form Survey; UEFI, Upper Extremity Functional Index. we did not specify whether patients were to be informed of their previous scores when completing the PSFS at followup; physical therapists were, therefore, able to remind patients of their previous scores when necessary, although in most cases they did not. Though we did not investigate this question experimentally, anecdotally it appeared to us that the patients having been informed of their previous scores did not influence the results in any identifiable way, despite considerable variation in time between visits. It is not clear how this might have affected the results of this study. There has been criticism of the GROC in terms of the ability of patients to recall their original (baseline) condition and to make a reliable judgment about whether they improved ,23 A recent study of global perceived effect scales for people with musculoskeletal problems found that ratings were strongly influenced by current status, especially when the time between evaluations extended into 63 NR NR 0.41 ICC 1,1 = 0.97 NR Construct validity: follow-up scores months. 19 In this study, the time between visits was variable (mean SD, days). Despite these limitations, we considered the GROC to be the best available external anchor for trichotomizing patients on the basis of self-perceived improvement, and for determining external responsiveness. Jaeschke et al 18 described the GROC as a credible alternative in the absence of a standard criterion for change in functional status. The clinimetric and psychometric properties of the PSFS for patients with upper extremity problems were similar to those reported by other studies of the PSFS. 5,6,21,28,29,33 Only 1 other study 6 comparably investigated construct validity, which was assessed in the manner of the present study, and the authors also found the PSFS to be valid (P<.001) (TABLE 5). The reliability of the PSFS, as determined by ICC 2,1 (0.71; 95% CI: 0.51, 0.84), was less than that reported in previous studies (TABLE 5), which, as discussed above, might have been due to the correlated with the RMQ ( 0.67); Westaway et al (1998) 33 NDI for neck dysfunction 31 NR NR 0.43 ICC 2,1 = Construct validity: correlated with NDI P<.001 initial (0.82), follow-up (0.83), and change (0.83) scores; P<.001 Chatman et al (1997) 5 SF-36 for knee dysfunction 38 NR NR 1.0 ICC 2,1 = Construct validity: correlated with Pengel et al (2004) 21 RMQ for low back pain 155 GRI; PSFS ES = 1.6 NR NR NR NR NR functional items of the SF-36; initial (0.34) and follow-up (0.49) scores Cleland et al (2006) 6 NDI for cervical radiculopathy 38 AUC for PSFS = ICC 2,1 = Construct validity of the change in Stewart et al (2007) 28 NDI, SF-36 for chronic whiplash 132 AUC for PSFS = 0.71 NR NR NR NR NR Hall et al (2010) 14 RMQ for low back pain 831 AUC for PSFS = 0.85 NR NR NR NR NR scores of the stable and improved groups; P<.001 This study UEFI for upper extremity 180 AUC for PSFS = ICC 2,1 = Construct validity of the change in scores of the stable and improved groups; P<.001 difficulties of determining a stable group in the clinical setting. Current findings of responsiveness are in keeping with similar studies (TABLE 5). 6,21,28 In this study, we found no significant difference in responsiveness for the PSFS compared to the UEFI, whereas other studies have reported the PSFS to be more responsive than comparison outcome measures. 14,28 The paradoxical findings in this study of MDC values larger than MID values may be related to, as discussed above in the Reliability section, the stable group in this study actually improving slightly rather than remaining truly unchanged. This might have adversely affected the estimate of reliability and SEM, resulting in reliability values biased lower and MDC values biased higher than would be expected in the population, with less impact on the estimate of MID. A criticism of the PSFS is that it may not be applicable to comparing change between patients or groups of patients journal of orthopaedic & sports physical therapy volume 42 number 2 february

9 [ research report ] given its patient-specific nature (ie, compared to region-specific outcome measures such as the UEFI). 33 It has been argued that, due to the heterogeneous nature of the activities nominated by each individual patient, that patients PSFS scores cannot be aggregated at the group level, changes in scores between individual patients cannot be validly compared, and content validation of the PSFS is not possible. 15 Because there is no evidence yet that PSFS data can validly be aggregated at the group level and compared between patients or groups of patients, we cannot recommend the PSFS for such purposes. 15 Further research into the types of activities identified by patients in the PSFS and how they compare to those identified with other outcome measures is required to explore this issue. Data from the current study suggest that the PSFS is useful for patients and clinicians to determine whether a clinically important change in score has occurred for an individual patient over the time between visits. An advantage of the PSFS is that, at face value, its patient-specific nature ensures that it is highly relevant to individual patients and may include activities that are not listed on any other self-report outcome measure 17 (eg, serving at tennis or lifting baby out from the car seat ). These are clearly highly important to the patient and would not necessarily be captured by a non patient-specific outcome measure. CONCLUSION This study has demonstrated that the PSFS is as valid, reliable, and responsive as the UEFI for individual patients presenting with upper extremity problems in an outpatient, musculoskeletal physical therapy setting. The MID, or the smallest change in score perceived by informed patients or informed proxies as important, for the PSFS was 1.2 on a 0-to-10 point scale. t KEY POINTS FINDINGS: The results suggest that the PSFS is clinimetrically comparable to the UEFI as a valid, reliable, and responsive outcome measure for patients with upper extremity problems in an outpatient, musculoskeletal physical therapy setting. The MID was found to be a change in score of 1.2 points. IMPLICATION: Use of the PSFS has been validated previously for patients with back, neck, and knee problems. The present findings indicate that it can now be used confidently to measure progress over time in patients with upper extremity problems. CAUTION: The PSFS is an outcome measure that is uniquely relevant to individual patients. Using it for group-level data has not been established. ACKNOWLEDGEMENTS: We are grateful for research support from the New Zealand Lottery Grants Board (Health Research Grant AP237485), The New Zealand Society of Physiotherapists (NZSP) Scholarship Trust, and the University of Otago Centre for Physiotherapy Research. REFERENCES 1. Bartko JJ. Measurement and reliability: statistical thinking considerations. Schizophr Bull. 1991;17: Beaton DE, Katz JN, Fossel AH, Wright JG, Tarasuk V, Bombardier C. Measuring the whole or the parts? Validity, reliability, and responsiveness of the Disabilities of the Arm, Shoulder and Hand outcome measure in different regions of the upper extremity. J Hand Ther. 2001;14: Beaton DE, Wright JG, Katz JN. Development of the QuickDASH: comparison of three itemreduction approaches. J Bone Joint Surg Am. 2005;87: JBJS.D Bland JM, Altman DG. A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement. Comput Biol Med. 1990;20: Chatman AB, Hyams SP, Neel JM, et al. The Patient-Specific Functional Scale: measurement properties in patients with knee dysfunction. Phys Ther. 1997;77: Cleland JA, Fritz JM, Whitman JM, Palmer JA. The reliability and construct validity of the Neck Disability Index and patient specific functional scale in patients with cervical radiculopathy. Spine (Phila Pa 1976). 2006;31: dx.doi.org/ /01.brs DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44: Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Control Clin Trials. 1991;12:142S-158S. 9. Domholdt E. Rehabilitation Research: Principles and Applications. St Louis, MO: Elsevier Saunders; Downie WW, Leatham PA, Rhind VM, Wright V, Branco JA, Anderson JA. Studies with pain rating scales. Ann Rheum Dis. 1978;37: Gabel CP, Michener LA, Burkett B, Neller A. The Upper Limb Functional Index: development and determination of reliability, validity, and responsiveness. J Hand Ther. 2006;19: ; quiz Gross DP, Battie MC, Asante AK. The Patient- Specific Functional Scale: validity in workers compensation claimants. Arch Phys Med Rehabil. 2008;89: org/ /j.apmr Guyatt GH, Townsend M, Keller JL, Singer J. Should study subjects see their previous responses: data from a randomized control trial. J Clin Epidemiol. 1989;42: Hall AM, Maher CG, Latimer J, Ferreira ML, Costa LO. The patient-specific functional scale is more responsive than the Roland-Morris Disability Questionnaire when activity limitation is low. Eur Spine J. 2010;20: org/ /s Hart DL, Werneke MW. Re: Pengel LHM, Refshauge KM, Maher CG. Responsiveness of pain, disability, and physical impairment outcomes in patients with low back pain. Spine. 2004;29: Spine (Phila Pa 1976). 2004;29: Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder and hand) [corrected]. The Upper Extremity Collaborative Group (UECG). Am J Ind Med. 1996;29: (SICI) (199606)29:6<602::AID- AJIM4>3.0.CO;2-L 17. Hush JM, Refshauge KM, Sullivan G, De Souza L, McAuley JH. Do numerical rating scales and the Roland-Morris Disability Questionnaire capture changes that are meaningful to patients with persistent back pain? Clin Rehabil. 2010;24: org/ / Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10: Kamper SJ, Ostelo RW, Knol DL, Maher CG, de Vet HC, Hancock MJ. Global Perceived 64 february 2012 volume 42 number 2 journal of orthopaedic & sports physical therapy

10 Effect scales provided reliable assessments of health transition in people with musculoskeletal disorders, but ratings are strongly influenced by current status. J Clin Epidemiol. 2010;63: e1. jclinepi Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol. 1997;50: Pengel LH, Refshauge KM, Maher CG. Responsiveness of pain, disability, and physical impairment outcomes in patients with low back pain. Spine (Phila Pa 1976). 2004;29: Pransky G, Feuerstein M, Himmelstein J, Katz JN, Vickers-Lahti M. Measuring functional outcomes in work-related upper extremity disorders. Development and validation of the Upper Extremity Function Scale. J Occup Environ Med. 1997;39: Schmitt J, Di Fabio RP. The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil. 2005;86: org/ /j.apmr Schmitt JS, Di Fabio RP. Reliable change and minimum important difference (MID) proportions facilitated group responsiveness comparisons using individual threshold criteria. J Clin Epidemiol. 2004;57: org/ /j.jclinepi Schunemann HJ, Akl EA, Guyatt GH. Interpreting the results of patient reported outcome measures in clinical trials: the clinician s perspective. Health Qual Life Outcomes. 2006;4:62. org/ / Schunemann HJ, Guyatt GH. Commentary-- goodbye M(C)ID! Hello MID, where do you come from? Health Serv Res. 2005;40: dx.doi.org/ /j x 27. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86: Stewart M, Maher CG, Refshauge KM, Bogduk N, Nicholas M. Responsiveness of pain and disability measures for chronic whiplash. Spine (Phila Pa 1976). 2007;32: org/ /01.brs d 29. Stratford P, Gill C, Westaway M, Binkley J. Assessing disability and change on individual patients: a report of a patient specific EARN CEUs With JOSPT s Read for Credit Program JOSPT s Read for Credit (RFC) program invites Journal readers to study and analyze selected JOSPT articles and successfully complete online quizzes about them for continuing education credit. To participate in the program: 1. Go to and click on Read for Credit in the left-hand navigation column that runs throughout the site or on the link in the Read for Credit box in the right-hand column of the home page. 2. Choose an article to study and when ready, click Take Exam for that article. 3. Login and pay for the quiz by credit card. 4. Take the quiz. 5. Evaluate the RFC experience and receive a personalized certificate of continuing education credits. Physiother Can. 1995;47: org/ /ptc Stratford PW, Binkley JM, Stratford DM. Development and initial validation of the Upper Extremity Functional Index. Physiother Can. 2001;53: Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 4th ed. New York, NY: Oxford University Press; Turner D, Schunemann HJ, Griffith LE, et al. The minimal detectable change cannot reliably replace the minimal important difference. J Clin Epidemiol. 2010;63: org/ /j.jclinepi Westaway MD, Stratford PW, Binkley JM. The patient-specific functional scale: validation of its use in persons with neck dysfunction. J Orthop Sports Phys Ther. 1998;27: MORE INFORMATION The RFC program offers you 2 opportunities to pass the quiz. You may review all of your answers including the questions you missed. You receive 0.2 CEUs, or 2 contact hours, for each quiz passed. The Journal website maintains a history of the quizzes you have taken and the credits and certificates you have been awarded in the My CEUs section of your My JOSPT account. journal of orthopaedic & sports physical therapy volume 42 number 2 february