Measuring the Whole or the Parts?

Transcription

1 Measuring the Whole or the Parts? Validity, Reliability, and Responsiveness of the Disabilities of the Arm, Shoulder and Hand Outcome Measure in Different Regions of the Upper Extremity Dorcas E. Beaton, BScOT, MSc, PhD Institute for Workand Health Toronto, Ontario, Canada; Department of Occupational Therapy, Graduate Department of Rehabilitation. Sciences, and Clinical Epidemiology and Health Care Research Program University of Toronto; St. Michael's Hospital, Toronto Jeffrey N. Katz, MD, MS Institute for Workand Health, Toronto; Graduate Department of Rehabilitation Sciences University of Toronto; Brigham and Women's Hospital and Harvard Medical School Boston, Massachusetts Anne H. Fossel Brigham and Women's Hospital Boston, Massachusetts James G. Wright, MD, FRCSC, MPH R. B. SalterChair in Surgical Research, Department of Surgery, University of Toronto; Graduate Department of Rehabilitation Sciences, Clinical Epidemiology and Health Care Research Program, and Department of Public Health Sciences University of Toronto; The Hospital for Sick Children, Toronto Valerie Tarasuk, PhD Department of Nutritional Sciences University of Toronto Claire Bombardier, MD, FRCP Institute for Workand Health, Toronto; Graduate Department of Rehabilitation Sciences, Clinical Epidemiology and Health Care Research Program, Department of Medicine, and Department of PublicHealth Sciences, University of Toronto; The University Health Network Toronto General Hospital; Mt. Sinai Hospital, Toronto ABSTRACT: The Disabilities of the Ann, Shoulder and Hand (DASH) outcome measure was developed to evaluate disability and symptoms in single or multiple disorders of the upper limb at one point or at many points in time. Purpose: The purpose of this study was to evaluate the reliability, validity, and responsiveness of the DASH in a group of diverse patients and to compare the results with those obtained with joint-specific measures. Methods: Two hundred patients with either wrist/hand or shoulder problems were evaluated by use of questionnaires before treatment, and 172 (86%) were re-evaluated 12 weeks after treatment. Eighty-six patients also completed a test-retest questionnaire three to five days after the initial (baseline) evaluation. The questionnaire package included the DASH, the Brigham (carpal tunnel) questionnaire, the SPADI (Shoulder Pain and Disability Index), and other markers of pain and function. Correlations or t-tests between the DASH and the other measures were used to assess construct validity. Test-retest reliability was assessed using the intraclass correlation coefficient and other summary statistics. Responsiveness was described using standardized response means, receiver operating characteristics curves, and correlations between change in DASH score and change in scores of other measures. Standard response means wereused to compare DASH responsiveness with that of the Brigham questionnaire and the SPADI in each region. Results: The DASH was found to correlate with other measures (r> 0.69) and to discriminate well, for example, between patients who were working and those who were not (p < ). Test-retest reliability (ICC=0.96) exceeded guidelines. The responsiveness of the DASH (to self-rated or expected change) was comparable with or better than that of the joint-specific measures in the whole group and in each region. Conclusions: Evidence was provided of the validity, test-retest reliability, and responsiveness of the DASH. This study also demonstrated that the DASH had validity and responsiveness in both proximal and distal disorders, confirming its usefulness across the whole extremity. J HAND THER. 2001;14: This work was supported by research grants from the American Society for Surgery of the Hand, Rosemont, Illinois, and the Institute for Work and Health, Toronto, Ontario, Canada; by a PhD fellowship in health research from the Medical Research Council of Canada (Dr. Beaton); by a scientist award from the Medical Research Council of Canada (Dr. Wright); and by grant AR36308 from the U'S. National Institutes of Health and the u.s. National Arthritis Foundation (Dr. Katz). Address correspondence and reprint requests to Dorcas Beaton, BScOT,MSc, PhD, Institute for Work and Health, 250 Bloor Street East, Suite 702, Toronto, Ontario, Canada M4W 1E6; <dbeaton@iwh.on.ca>. 128 JOURNAL OF HAND THERAPY

2 The measurement of disabilityv ' or capacity to function" is critical to a comprehensive assessment of outcome following an injury in the upper limb. The fluid motion of a swimmer, artist, or musician attests to the coordinated kinetic chain along the extremity which allows for such expression and function.y' However, measuring disability in patients with upper-limb disorders poses practical challenges. For example, distinct questionnaires have been developed for the different regions of the upfer limb7~11 and for various disorders in the limb.12-1 Given that many patients could have multiple disorders or multiple affected regions, the choice between available measures is difficult. The Disabilities of the Arm, Shoulder and Hand outcome measure (the DASH) provides one possible solution. It is a questionnaire designed to be used for single or multiple disorders in the upper limb, providing the possibility of a single questionnaire for measuring disability for any upperlimb region. 5,15,16 The intent is that the DASH be used no matter what region or regions are affected. The development of the DASH has been documented elsewhere. I5-17 The DASH is a 30-item questionnaire that evaluates symptoms and physical function (at the level of disabilityl,3,18), with a fiveresponse option for each item. Scoring is done by summing up the circled responses and subtracting 30. (Subtracting 30 anchors the score with a base of 0, a correction required because the response scale is 1 to 5 and needs to be changed to a 0 to 4 equivalent). This figure is then divided by 1.2 to get a DASH function/symptom score out of a possible 100. A higher score on the DASH reflects greater disability. Missing responses to items (up to three items, or 10% of items) are replaced by the mean value of the responses to the other items before summing. If responses to more than three items are missing, the overall score cannot be calculated.p Preliminary work on the validity (against constructs of function and pain) and reliability (alpha, ; test-retest reliability, ) has been carried out directly by those involved in the development l5-17 as well as indirectly by others who used the DASH for comparison with another instrument. 14,20 Kirkley et al 14 and MacDermid et al 20 also demonstrated that the DASH was responsive, although slightly less so, in comparison with more joint- or disorder-specific measures-specifically, a shoulder instability and a wrist-specific instrument in these studies respectively. These studies provided initial evidence of the validity and reliability of the DASH scores; however, additional work was needed to compare the DASH in patients with disorders in different upper-limb regions. Of particular interest was how the DASH would do in evaluating change over time (its intended role) in patients with different affected parts of the extremity, a role that requires evidence of construct validity, test-retest reliability and responsiveness Of the articles mentioned above that used the DASH, only three collected data over time for the study,14,19,20 none focused on the DASH per se, and none provided information on all three attributes. The purpose of this study was to evaluate the validity, test-retest reliability, and responsiveness to change of the DASH in a longitudinal study of patients with various upper-limb disorders. METHODS A convenience sample of patients waiting for treatment of upper-limb conditions at one of two teaching centers (St. Michael's Hospital in Toronto and Brigham and Women's Hospital in Boston) were invited to participate in this prospective cohort study. The study did not affect their care but rather noted their progress over a three-month period by taking measures (through questionnaires only) before and after treatment. Four groups of patients, two with proximal disorders and two with distal disorders, were targeted. The patients with proximal disorders included those with glenohumeral arthritis who were undergoing joint replacement and those with soft tissue disorders around the shoulder (predominantly rotator cuff tendinitis). Patients with distal disorders included those undergoing carpal tunnel release and those receiving treatment for a tendon disorder in the wrist or hand (predominantly trigger finger and tendinitis). Acute injuries such as tendon lacerations or fractures were not included, because pre-treatment measures of disability cannot be obtained in these conditions. No guidelines were available for calculating sample sizes for studies of responsiveness. A traditional paired sample calculation (alpha at 0.025, a Bonferroni correction to allow for multiple comparisons." and beta at 0.10) was used. 27 The amount of change we wanted to be able to detect was calculated using data from a worksite study and was defined as the difference in improvement on the DASH scores between those who said they were much better and those who said they were somewhat better between testings one year apart. 28,29 The former group had change scores of 5.79 out of 100 (SD 11.2) on average, and the latter had change scores of 2.42 out of 100 (SD 12.3). The difference between them was 3.37 out of 100. The average change in those who said they had not changed between testings was 0.89 out of 100 (SD 9.9). The SD in this group was used as the variance in the sample size calculation, as suggested by Guyatt et al 30 and Rossner." A sample of 113 patients was required. A 20% correction was added to allow for missing or unusable questionnaires, which raised the requirement to 142 patients. (The target of 113 patients was assumed to April-June 200 I 129

3 equal80% of the data collected.) We anticipated thatup to 20% of participants would not complete the followup package; hence, recruitment was targeted for 178 patients. We continued recruitment until it was apparent that we would have follow at least this number and that it represented at least 80% of the baseline sample. Patients at the St. Michael's Hospital site were approached in person by study personnel (either at a surgeon's clinic or in a pre-admission clinic). The study was explained to them, and they were asked to sign a consent form if they wished to participate. At that time they were given the baseline package to complete and return as well as a second package to be completed three to five days later (depending on the date of surgery). Two follow-up packages were mailed out to the subjects with stamped return envelopes. This was done 4 and 12 weeks after treatment. (Only 12-week data are presented in this paper.) Up to two reminder packages and phone calls were made to encourage response. In Boston, study personnel identified patients by reviewing ICD-9-CM diagnostic codes in billing data and looking for patients receiving or awaiting surgical or nonsurgical care for the target disorders. These potential subjects were sent detailed letters explaining the study and inviting them to participate. The baseline questionnaire package and a stamped return envelope were also included. Participation (returning the baseline questionnaire package) was considered consent. Subjects from Boston were sent their second package by mail at 12 weeks. Again, phone calls and reminder packages (up to two) were sent as necessary. The study was approved by the Research Ethics Board at both sites. MEASURES AND ANALYSIS Each completed questionnaire package was checked by research staff for such things as missing items and duplicate responses and was then entered into a customized database at each site. These data were converted into SAS (version 6.12) data sets (SAS Statistical Analysis Systems, Cary, North Carolina) and merged. All analysis was done in SAS. Sample Description Baseline demographics for the whole cohort were analyzed descriptively. This was repeated separately for subjects from each of the two data collection sites. The variables described included age, gender, education, and clinical variables (comorbidity, pain medication use, duration of symptoms, etc.). The SF-36 generic health status measure 32,33 was also used to describe baseline overall health in the cohort. Means and medians for each dimension of the SF- 36 were calculated for the entire cohort and plotted against mean values for the general Ll.S. population. Construct Validity Several different comparisons were done, using recommended methods, to evaluate the construct validity of the DASH scores. 25,34 We hypothesized that the DASH scores would be sensitive to the range of disability in our sample. This was verified by looking at the distributions of baseline scores (whole sample, proximal, distal, surgical, nonsurgical) and specifically looking for floor or ceiling effects (patients with scores at either extreme of the scale), which would indicate a lack of sensitivity to the disability experience in this sample. Floor and ceiling effects would also lead to difficulties in trying to measure change (for instance, if everyone scores at the maximum score--a ceiling effect-there is no place to move to on the scale if they improve).35 We also felt that the DASH scores should be lower (indicating less disability) in the following groups: those working full duty rather than not and those able to cope and do what they want rather than not. These contrasts were tested with an unpaired Student's t-test, at a 0.05 level of error. We posited that the DASH should also correlate at least moderately (Pearson correlations greater than 0.5) with visual analog scales of function, pain, and ability to work as well as with established joint-specific measures, specifically the Shoulder Pain and Disability Index (SPADD for patients with shoulder conditions7,36,37 and the Brigham questionnaire (the Brigham) 12,38 for patients with wrist and hand conditions). Furthermore, if the joint-specific measures are indeed specific to a particular joint, we should see lower correlations between the DASH and the Brigham in the patients with shoulder conditions as well as between the DASH and the SPADI in the patients with wrist conditions. The disability or function scores of the SPADI and the Brigham were the focus of this analysis. Test-Retest Reliability Test-retest reliability was analyzed using data from patients who had completed two measures before treatment began (three to five days apart) and said that their ann problem had not changed (in response to the question, "How is your problem now compared to before your treatment/surgery?").mean change scores and associated paired t statistics (and p values) were calculated. Correlation coefficients were obtained using both the Pearson method (parametric, for normally distributed data) and the Spearman method (non-parametric, using ranks). These correlations indicate whether scores for a given patient are high at baseline and also high at followup but not whether the scores are identical. Intraclass correlation coefficients (ICCs) provide an 130 JOURNAL OF HAND THERAPY

4 estimate of how closely the numeric scores for each patient were to each other (called concordance,and are therefore considered a stronger statistic for describing reliability,z,39,4o Specifically, we used a Shrout and Fleiss (2,1) model derived from a twoway analysis of variance.t" By adopting this particular model, we are saying that the testing framework in this particular study (three to five days apart, pretreatment) is assumed to be only one of many possible ways the test-retest reliability could have been assessed. We considered a coefficient between 0.90 and 0.95 a minimum standard for reliability, based on the guidelines of Lohr et al. 25 and others. 34,4l for the ability to interpret questionnaire scores in individual patients. The final estimate of reliability, the minimally detectable change (MOC),42-46 was calculated using the test-retest reliability coefficient to estimate the standard error of measurement (SEM) for the difference score.t' The SEM was therefore the SO at baseline (abase) times the square root of (1- RxX>, where Rxx is the test-retest reliability.t! Christensen and Mendoza'f and others 4l,47 suggest multiplying the (1-R xx> by 2, to adjust for the fact that when looking at a change score two samples are being used (time-i and time-2), each with measurement error. The SO is therefore increased by this factor to account for this. From this calculation of the SEM, it is possible to describe the amount of change that would need to be observed on the questionnaire to exceed this measurement error. In health measurement, the terms "minimally detectable change,,45,48 and "smallest detectable difference or change,,47 have been used for this change score. It is calculated by multiplying the SEM by the appropriate z value (depending on the level of confidence desired). Thus, for a minimal detectable change at the 95% confidence level, the formula would be The subscript 95 is used with the label because it is possible to calculate this change score at different levels, including 90%48 and 67%.46 The MOC 95 yields a threshold-a minimum change score-that allows you to be 95% confident that when you observe a change score that is greater than this value, it is likely to indicate a real change in your patient, rather than measurement error (for that instrument in that population) alone. The MOC provides a unique opportunity to translate the test-retest reliability coefficient (ICC) into units of change in the instrument. It should be noted that the ways others calculate these values vary. In early work, Jacobson et a1. 43 did not double the error term; however, Jacobson has since chan ed this position and adopted the adjusted formula.t'" 1 Wyrwich et al., in their work on bounds of relevant change,46,52 do not use the adjustment-tothe-error term and replace the test-retest reliability coefficient with the Cronbach alpha coefficient. The use of the alpha moves away from longitudinal stability as the source of the variance and favors instead an instrument with high correlations between items, a cross-sectional strength." Wyrwich et al. argue, and we agree, that in cases of very high sample sizes, the Cronbach alpha and test-retest coefficients will be almost the same 34,46,52; however, in our experience in clinical research, such large samples (N - 300)34 samples are rarely found. Test-retest coefficients derived from similar patients are preferable. Caution should be used to avoid confusing the Wyrwich coefficient with what appear to be similar coefficients under the rubric of minimal detectable change or reliability change indexes. Responsiveness to Change The ability to detect change when it has occurred,53,54 or responsiveness, is often incorrectly felt to be a fixed property of an instrument (e.g., the XXX questionnaire is "responsive"). However, it is more like construct validity, in which we are validating the application of the instrument in a specific test situation, not the instrument in and of itself. Evaluation of responsiveness requires that some sort of change has occurred (and that it can be verified in some way), and then the questionnaire's scores are tested against that change. It is possible that a given questionnaire could be responsive or sensitive to a particular type of change but not to another type. 55,56 Responsiveness needs to be described in relation to the relevant type of change. In our study, we looked at three types of change known to have occurred between baseline and 12 weeks after the beginning of treatment. First, we assumed that patients would likely begin to show improvement with each of the treatments in the study (i.e., total shoulder replacement, carpal tunnel release), and therefore we compared pre-treatment and 12 week post-treatment scores in the whole group. We recognize that this does not reflect full recovery, but patients are likely to have had small improvements from their pre-treatment states by this time. Second, we looked at patients who said that their upper-limb problem was better. We determined this by their response to the Tl-point scale asking, "Compared to before your treatment/surgery, how much has your arm (either shoulder or wrist/hand) problem changed?" Those indicating more than 6 on the scale (where 5 indicates no change and 10 indicates much better) were considered to have improved. Different cut-off points could have been used; however, we chose this point on the basis of an a priori consensus of four of the authors (O.B., C.B., J.G.W., and J.N.K.). Similarly, an indication of improvement of more than 6 in ability to function in April-June 200 I 131

5 TABLE 1. Sociodemographic Data for the Study Participants Whole Cohort (n =200) Boston (n=91) Toronto (n=109) Follow-up rate-no. (%) completing baseline and wk follow-up (86%) (86%) (86%) Age (mean Gender: Male Female Marital status: Married/living Divorced/separated Widowed Single Schooling: Less than grade Some high school (14%) (3%) (23%) High school Some college or university Graduated from college or university (50%) (67%) (34%) Work status: Full time Part time Disabled because of DE (12%) (2%) (19%) Disabledfor other reason Homemaker Retired Student Worker's compensation: No, not on worker's compensation Yes, but not yet Yes, receiving it Yes, but no longer Lawyer for DE? (% yes) (6%) (7%) (6%) NOTE: The first column represents the whole cohort, the second and third that portion of the cohort coming from each of the sites. The differences between the Boston and Toronto cohorts are noted, although they are not likely to affect analysis of the questionnaires in the whole cohort. Analysis across study sites was not done. daily activities was used as an external indication that change had occurred, and responsiveness analysis was carried out on that subsample of patients who indicated more than 6 of 10 on the scale. In all cases, responsiveness was summarized using the following statistics-change scores (the mathematical difference between baseline and follow-up scores), effect size 57 (mean change divided by the SD of baseline scores) and the standardized response mean 5 8--{jO (mean change divided by the SD of change scores, or SRM). A comparison was then carried out contrasting the responsiveness (using the SRM) of the DASH, the Brigham, and the SPADI in each of the subcohorts-patients with shoulder conditions and patients with wrist or hand conditions. We thus evaluated the responsiveness of a shoulder questionnaire to improvements in wrist patients and vice versa. It was hypothesized that the responsiveness of the DASH should be comparable with that of the jointspecific measures but that the highest responsiveness would be found for the joint-specific measures for patients in whom that joint was involved. Responsiveness was also described by correlating change scores on the DASH with changes in pain intensity, function, and severity of the problem. Two approaches were used to measure these attributes. First, numeric rating scales were used to gather these ratings at each testing time (we called these "status measures"), and the differences between the status measures at 12 weeks and at baseline (we called these "differences in status measures") were correlated with the difference in DASH scores. Second, we also asked patients (at 12 weeks) to rate the amount these same attributes had changed since baseline (pre-treatment). We called this "a transition approach." It would be hypothesized that both approaches should lead to at least moderate correlations with changes in the DASH scores. Slightly lower correlations were expected because the comparison measures are made up of only one or two items and hence are prone to more measurement error, which would lower the correlation coefficient (attenuation of correlatiorr'"). The correlations are also reduced because the change scores are derived from two samples (pre- and post-treatment), which also adds to the error in the measure (as discussed above) and causes further attenuation of the correlation. We therefore considered an r value of about 0.4 indicative of a moderate correlation. We also constructed receiver operating characteristics (ROC) curves, as described by others. 61,62 These curves demonstrate how accurately different change scores on the questionnaire distinguish those who are better from those who are not (as defined by some other criterion). In our study we used an affirmative answer to the question "Can you cope with your problem and do what you would like to do?" at follow-up, given an inability to cope at baseline, as the criterion of improvement. We selected this question \32 JOURNAL OF HAND THERAPY

6 on the basis of a qualitative study in which this was described as a threshold type of indicator of being "better.,,63 This seemed to be a reasonable marker for change; however, it is not a "gold standard," and other markers could also have been used. Changes in DASH scores of -1,-5, -7, -10, -IS, and -20 were considered" and were compared with the externalmarkerto see how well eachchangescore correctly corresponds to the external marker. The truepositive rate (the percentage of people who had a change score of at least that amount and were also now able to cope) and the false-positive rate (l minus specificity, or the percentage of people who had a change score of at least that amount but had not shifted from being unable to being able to cope) were calculated and plotted on a graph-a ROC curve.66 The area under the curve represents the responsiveness; the larger the area, the more responsive the instrument, because the different change scores accurately discriminate between improved and non-improved patients as defined by our external marker. In a ROC curve for one instrument, the point highest to the upper left might be considered the change score most able to discriminate between those who have shifted to coping with their condition and those who have not. RESULTS Sample Description Two hundred patients were enrolled and completed the baseline portion of this study. One hundred and seventy-two completed the 12-week follow-up questionnaire (86% follow-up rate). The description of the samples differed between sampling sites. However, because we are making within-person, and not between-site, analyses, the differences should not affect the results. The description of the sample is shown in Table 1. The first column reflects the findings in the whole cohort, the second and third columns the Boston (n = 91) and Toronto (n = 109) subsamples, respectively. The mean age and marital status were similar across sites. The average age was 42 years, and the majority of people were married. The split between men and women was fairly even in the whole sample, but most of the male patients came from Toronto. *The reasons these change scores were selected for the cut-offs are as follows: 1 is closest to the smallest change detectable on the DASH (0.83).63 A change of 5 is equivalent to one standard error of measurement, which Wyrwich et al 46 suggest is close to a minimally clinically important difference; the cut-off suggested by Redelmeier and Lorig 64 is 7 (or 7% change). A cut-off of 10 is selected for convenience only. A cut-off of 15 corresponds to the minimally detectable change derived from the work of Turchin et al. 19 but also corresponds to the suggestion of Redelmeier et a1. 65 that the criterion for an important change is 0.5 points per item in a questionnaire (therefore, 30 x 0.5). Finally, we selected a cut-off of 20 to represent an extreme change score. TABLE 2. Clinical Characteristics of the Study Participants Characteristic Region affected: Whole Sample Boston Toronto (n=200) (n=91) (n=109) Shoulder Wrist/hand Duration of symptoms (mean weeks): (SD 374) Medication use: Aspirin, NSAID Tylenol (OTC) Narcotics Other Not taking medication Comorbidity: Hypertension Asthma Diabetes Ulcers Depression (26%) (23%) (27%) Cancer Arthritis Low back pain (62%) (65%) (60%) No. of comorbid conditions: None One Two Three Four Five or more The level of education of patients differed between sites. In Boston 67% had graduated from university or college, whereas in Toronto only 34% had done so. In contrast, a larger proportion of the Toronto sample indicated that their highest educational level was "completed some high school" (25%) than the Boston sample (3%). Level of employment differed, with more people being off work due to their upper- limb problem in Toronto (19%) than in Boston (2%). Likewise, the Toronto group had a higher proportion of persons on worker's compensation. Table 2 summarizes some of the clinical findings for the whole cohort as well as for each site. The majority of the patients with shoulder conditions were from Toronto, especially those with osteoarthritis (undergoing shoulder replacement). The mean duration of April-June 200 I 133

7 100, ,----, _. _ _. - _. -_. - _. - _. _ _. - _. _. - _. o '-- PF SF RP RE MH VT BP GH PCS MCS J FIGURE 1. SF-36 health status: Results for entire cohort compared with U.S. general population data (solid squares),32 expressed as baseline mean scores (gray line) and median scores (solid triangles) in the eight SF-36 dimensions, and as two component scores. The eight SF 36 dimensions are physical function (PF), social function (SF), role-physical (RP), role-emotional (RE), mental health (MH), vitality (VT), bodily pain (BP), and general health perceptions (GH); the component scores are the physical component score (PCS) and the mental component score (MCS). A score of 100 indicates good health. Population norms for the United States were done on SF 36-US with a 4-week window. The study data were gathered on the Canadian version of SF-36 Acute. Number whole cohort mean= s.dev'n= median=44.6 _ o ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ",,<::I '),<::1 ~<::I ",,<::I,,<::I '0<::1 ~<::I '0<::1 0,<::1 -c DASH Score shoulder patients wrisuhand patients 40, , ' , mean=48.4 s.dev'n=21.2 mean=34.2 s.dev'n= median= medlan= o ~.!',,(:/,,0, -5'~ 4f} tl'o, <,ff<f> ~ '\(:/~.a!' cfirf',,<f' DASH Score ~ -?o,,,(:/~ -5'~ - "?...,:l <,ff<f> ~ '\(:/'\0,.a!' cfirf',,<f' DASH Score FIGURE 2. Baseline distribution of DASH scores (out of 100) for the entire sample (top) and for the subsamples defined by the location of patients' problems-shoulder (bottom left) or wrist/hand, (bottom right). A DASH score of 100 indicates greater disability. TABLE 3. Construct Validity; Pearson/Spearman Correlations Between the DASH and Other Measures of Upper Extremity Function Whole Cohort (n=200) Shoulder (n = 138) Wrist/Hand (n = 62) Overall rating of problem Pain severity Ability to function Ability to work SPADIpain SPADI function Brigham symptoms Brigham function 0.71 / / / / / / / / / / / / / / / / / / / / / / / / JOURNAL OF HAND THERAPY

8 symptoms was 193 weeks, more than three years, and both sites offered secondary or tertiary levels of care. The majority of the sample were taking some sort of medication to manage symptoms preoperatively, including 59 patients (of the 200 total) who stated that they were taking narcotics for their pain. Of interest is the high proportion of patients with histories of depression (23% in Boston and 27% in Toronto) and low back pain (65% in Boston and 60% in Toronto). The general health of the cohort is shown in Figure 1, where the mean and median scores in each dimension of the SF-36health status measure are plotted with data from the general population ofthe United States. Construct Validity The distribution of the baseline scores on the DASH appear to be normally distributed (Figure 2), with mean of 43.9 and median of Only one patient was at the "ceiling" (with a score of 0), indicating perfect health, and no one was at the floor (with a score of 100, indicating maximum disability on the scale). The distribution for the patients with shoulder conditions and patients with wrist or hand conditions, shown in the same figure, demonstrates the less severe disability in the patients with wrist and hand conditions, described by the DASH scores. Discriminative validity was confirmed. Those currently working with their upper limb condition and able to continue doing so had significantly lower disability than those who were not able to work (26.8 vs. 50.7, t=-7.51, p<o.oool). (This analysis contrasted only these two subgroups and did not analyze the responses from those who were retired or not working for reasons other than their upper limb condition.) Statistically significant differences in DASH scores were also found between those who were able to do all they want to do as opposed to those who were not able to do so (23.6 vs. 47.1, t = -5.81, P< ). Similar discrimination was found within the patients with shoulder conditions and those with wrist or hand conditions when these groups were analyzed separately. Thus, the difference was in the anticipated direction (patients who were unable to do what they wanted and those who were unable to work had more disability and higher DASH scores), and the difference was statistically significant. Convergent construct validity of the DASH was demonstrated by finding correlations in the expected direction and of the expected magnitude with other measures of upper limb function and symptoms. Table 3 summarizes the results. In the whole cohort, all correlations exceeded 0.70 (Pearson). Correlations were highest with the measure of function as well as with the function scores on the Brigham and the SPAD!. Correlations between the DASH and these joint-specific instruments were found even in the opposite joint. Test-Retest Reliability Fifty-six of the 86 people completing the test-retest reliability package (three to five days after baseline) indicated that they had no change in their problem (no change ± 1 response category on an ll-point transitional scale). The mean change score in this group was (median, 0), with an SO of The difference was not significant (paired t statistic, ; p value, 0.86). The correlation between baseline and retest was 0.96 (Pearson correlation) and 0.95 (Spearman ranked correlation). The ICC(2,1) on this sample (n = 56) was 0.96 (95% confidence interval, ) for the DASH, indicating excellent agreement. 25,34,41,67 The SEM is 4.6 DASH points, which led to a minimal detectable change (MDC 95 ) of (SO at baseline, 23.02) on a 100-point scale. A 90% MDC (MDC 90 ) would be 10.7 of 100 DASH points. Responsiveness The DASH questionnaire was able to demonstrate change in all situations in which change was presumed to have occurred-before and after treatment (SRM, ) and in those patients who either said that their problem was better overall or that their ability to function had improved (SRM, ) (Table 4). Standards for a "good" or "large" responsiveness statistic have little meaning, because they are dependent on the type of change being examined. The distributions of the change scores are shown in Figure 3. The large histogram shows the change for all patients before and after treatment (whether they got better or worse) as well as the change for the subgroups of patients who said their problem was better and those who said their function was better. The DASH was found to have comparable or slightly better responsiveness than the joint-specific measures; Table 5 and Figure 4 summarize these comparisons using the SRM statistic. When comparisons were made at the subgroup level (by region of injury), the DASH remained comparable with (only lower, by 0.04 to 0.08, in patients with wrist or hand conditions) or better than the disease-specific measures. Correlations between differences in the status measures (self-ratings at follow-up minus self-ratings at baseline) and the change scores on the DASH were moderately high (Pearson r>0.65) (Table 6). Those with the transitional indexes ("how are you now compared to before") failed to meet our modest standard of 0.4 (correlations were in the range of ) except in the overall rating of change in their problem (Pearson r=0.40, Spearman r=0.43). The correlations between the DASH scores and the differences in state measures were of the hypothesized magnitude; however, those between the transi- Text continues on p. 138 April-June 200 I 135

9 All patients completing baseline and follow-up (n =172): TABLE 4. Responsiveness of the DASH to Clinical Changes, Expressed as Mean (SD) Baseline Follow-up Change Score Effect Size SRM Observed change 44.5 (22.7) 30.9 (22.8) (16.9) Those rating problem as better (>6/10) 42.9 (22.9) 24.9 (20.2) (16.4) Those rating function as better (> 6/10) 40.7 (23.4) 20.2 (19.2) (16.5) Shoulder patients: Observed change 48.3 (21.0) 35.3 (21.3) (16.6) Those rating problem as better (>6/10) 49.0 (20.9) 30.0 (19.5) (15.7) Those rating function as better (» 6/10) 49.1 (21.9) 24.3 (17.6) (16.4) Wrist/hand patients: Observed change 33.8 (22.8) 20.5 (22.9) (17.5) Those rating problem as better (>6/10) 30.5 (22.2) 13.8 (17.2) (17.8) Those rating function as better (>6/10) 27.4 (19.4) 13.5 (20.3) (14.5) NOTE: The mean [and SDI for the DASH score at baseline and follow-up are shown, with the change score, effect size (mean change divided by SD of baseline), and standardized response mean (mean change divided by SD of change). Data are shown only for patients with data for both baseline and 12-week follow-up. SRM indicates standardized response mean. Number, , , eve change score = less disability g. +ve change score = III ~ more disability _ 10 o < Change in DASH (observed change) in those saying arm problem better (>6) (n=112) in those saying ability to function better (>6) (n=79) 35 35, t , o «so Change in DASH 15 II < Change in DASH FIGURE 3. Number of subjects falling into lo-point score ranges for change in DASH scores, using 12-week follow-up data, for all patients (top), patients who said their arm problem was better(bottom left), and patients who said they wereable to function better (bottom right). A score of 0 (no change) is shown to provide an anchor. 136 JOURNAL OF HAND THERAPY

10 TABLE 5. Comparison of Responsiveness of the DASH and Two Joint-specific Measures, the SPADI and the Brigham, Expressed as Standardized Response Means (SRMs) All patients: DASH SPADI Function Score BrighamFLs Observed change Those rating problem as better (>6/10) Those rating their function as better (>6/10) Shoulder patients: Observed change Those rating problem as better (>6/10) Those rating function as better (>6/10) Wrist/hand patients: Observed change Those rating problem as better (>6/10) Those rating function as better (>6/10) NOTE: The SRM (mean change score divided by SD of the difference) is used as the summary statistic; the SRM values are the same as those shown in Table 4. SPADI indicates Shoulder Pain and Disability Index; FLs, functional limitations. whole cohort Observed change _DASH DSPADI I:lBrigham "Problem better" "Function better"......,.,.",, " "...,,,,,,, o Sh0 UIder patienr-:-t-=-:s-:;-:-:----, _ DASH C SPADI IUlBrigham 0.6 SRM wrisuhand patients _ DASH C SPADI IUlBrigham ",'",q\# ~ ~ ~ ~0 -.;f 0~,q\# ~,-#, ~ ~ ~0 ~0 -.;f -.;f !#' ,</:1.«," SRM SRM FIGURE 4. Responsiveness of the DASH compared with joint-specific measures (the SPADI and the Brigham function scores) for the entire sample(top) and for patients by affected region-shoulder (bottom left) and wrist/hand (bottom right). Standardized response meansareshown for the wholecohort (observed change), thosewho said their problem was better(n = 112), and thosewho said they were ableto function better (n = 79). April-June 200I 137

11 Correlation Table 6. Correlation Between Change in DASH Score and Change in Self-rated Pain, Function, and Severity of Upper-limb Problem Transition Scale Status Ratings Difference Change in Change in Change in Change in Change in Change in Problem Pain Function Problem Status Pain Status Function Status Pearson OAO Spearman NOTE: "Correlation" refers to the correlation with the change in DASH scores. Correlation was measured in two ways-first on an ll-point transition scale (from "much worse" to "much better") at 12 weeks and then as the difference in status ratings (pain at 12 weeks minus pain at baseline, each using a 7-point rating scale on both testing occasions). Pearson (parametric) and Spearman (ranked) correlations coefficients are shown. TABLE 7. Sensitivity and Specificity of Different Levels of Change x less than: Sensitivity Specificity Accuracy (%) < < < < < < NOTES: The basis for comparison at different levels of change was a "yes" answer to the question, "Are you able to cope with your problem and do what you would like to do?" at follow-up after a "no" answer to the same question before treatment. x indicates the change in DASH score. Sensitivity is the probability of having a change in DASH score of x or less, given that the patient is better. Specificity is the probability of not having a change in DASH of x or less, given that the patient is not better. Accuracy is the percentage of patients whose change score of x or less correctly classifies them as better or not better. A change score of -15 or -20 was best able to discriminate between those who were better and those who were not better, according to the selected criteria (see footnote on p. 133). It is important to remember that a negative change score on the DASH means less disability and, therefore, an improvement. Thus, a change of less than -1 means just a little bit of improvement, and a change of less than -20 means a much greater improvement. Text continued from p. 135 tional indexes and the change in DASH scores were at or just below the expected level of The ROC curves, as shown in Table 7 and Figure 5, also demonstrated that the change scores in the DASH were more sensitive to ability to cope than to just chance alone. In ROC curves, "chance" is equivalent to the diagonal on the graph, where sensitivity equals 1 minus the specificity. Along this line the change score gives no more information about who is better and who is not than would chance alone (e.g., flipping a coin). The point farthest up and off to the left from the diagonal is often considered the most discriminating. In our graphs, a change score of at least -15 or -20 appears to be the most discriminative for the criterion of the patient's becoming able to cope; a score of -15 correctly rated 68% of the sample, and a score of -20 correctly rated 72% (Table 7). Using the self-rated scores (in which a score of more than 6 of 10 indicated improvement), as used in the statistical summary of responsiveness, much lower change scores were found to be the most discriminating (for a change of -lor a greater negative value, accuracy was 75%, sensitivity 0.87, and specificity 0.44). This highlights the effect of the external marker in this type of analysis. FIGURE 5. Receiver operating characteristics (ROC) curve describing the ability of different amounts of change to differentiate between patients who went from not coping to coping with their disorder (an improvement) and those who did not. The "best" cut-off, in terms of accuracy, would be that shown at the upper left of the curve. Changes of less than -15 or -20 appeared most accurate on this curve. Higher change scores were not assessed Sensitivity <-1 <-1 o~_----<..- < < < < -20 o o Specificity 138 JOURNAL OF HAND THERAPY

12 DISCUSSION This study has provided evidence of the construct validity, test-retest reliability and responsiveness of the DASH Outcome Measure in patients undergoing treatment for either proximal or distal disorders in the upper limb. Reliability, Validity, and Responsiveness of the DASH The DASH outcome measure exceeded recommended standards for test-retest reliabilitr,25,34,41,67 for both individual- and group-level interpretation of the scores." Generally, test-retest coefficients need to exceed 0.90 or 0.95 before their interpretation on an individual level can be considered. For group-level interpretation, lower coefficients (approximately 0.75) are acceptable. 34,40 Our results provided a coefficient of 0.96, which is only slightly higher than that of Turchin et al. 19 (0.92 in patients with stable elbow conditions), and these are not likely to be significantly different in magnitude. Evidence of both convergent and known-groups validity of the DASH was also found. Convergent validity was shown by demonstrating moderate to high correlations with other markers of disability and symptoms, and known-groups validity by showing differences between the DASH scores of patients who were working or functioning and the scores of those who were not. The DASH validity was comparable with previous results using this questionnaire in other populations. The findings of Hudak et al. 16 and Turchin et al. 19 and the results of the field-testing'' produced similar findings, although against a smaller number of constructs. The DASH was also responsive to the different types of change designed into this study-specifically, change observed before and after treatment of the target conditions and change in those patients who said they were better. Our results also highlighted two important issues that concern responsiveness. First, the size of the responsiveness statistic varied with the type of change that was being quantified. The SRMs for observed change were lower in magnitude than those for the change in patients who said they were better. This finding supports the taxonomy for responsiveness that we have presented elsewhere,55 which suggests that instruments are "responsive to" different types of change to different degrees. They do not inherently possess a trait of being "responsive." Comparisons of the responsiveness of different instruments should only be conducted when similar types of change are being tested or, ideally, when the instruments are placed in a head-to-head comparison. Second, our results also showed, as did Wright and Young/" that the choice of statistic will affect the description of responsiveness. For the same change in the same patients, we demonstrated a variation up to 1.33-fold in the responsiveness described by the effect size statistic compared with the SRM (0.68 and 0.91, respectively). This difference would also span the often used (or misused) guideline for what Cohen 68 calls a moderate vs. a large effect. However, the difference in the numeric estimates we were comparing was attributable to the statistic chosen alone. Responsiveness was also described by correlating changes 14,61 in the DASH with changes in three attributes (pain, function, and problem), each measured in two ways-with transitional scales and with difference-in-status measures performed at baseline and at 12 weeks. Our results suggested a distinct difference between the transitional approach (correlations of 0.32 to 0.43) and the difference-in-status approach (correlations of 0.60 to 0.69). Given that the concepts (pain and function) being measured using the two approaches were the same, the differences might be attributed to the way the questions were asked (transition vs. difference in status), leading to several possible explanations. Differences in a transitional approach vs. differences in status over time could be due to recall bias 69,7o or to a change in how people cognitively formulate a response when asked to describe a current state as 0fJosed to recalling a change in that state over time. 6, -74 Changes in how people calibrate or define pain, health, and quality of life over time have been described in the literature (e.g., "response shift phenomenon,,75-77) and could have influenced our results.?5-80 All these things could be possible reasons for the difference in the correlations between the change in DASH score and the two approaches to determining whether change had occurred (difference in serial state measures vs. the transition style of constructs). We are not suggesting which is better; arguments can be made in both directions. 69,77,81-83 Like Fischeret al.,84 we have demonstrated the difference in the two approaches and, like them, we do not have evidence to suggest which is better. The DASH in Comparison with Joint-specific Measures The DASH had high correlations with the two joint-specific measures, the Brigham and the SPADI-a pattern that persisted when the joint-specific measure was applied in the other region (Brigham vs. DASH in shoulder patients, 0.90; SPADI vs. DASH in wrist patients, 0.92). Therefore, our hypothesis that there would be a difference in the correlations when the joint-specific measures were applied in the wrong joint was not supported. On a cross-sectional basis, the joint-specific measures performed well in the "wrong" joint. To our knowledge, this is the first time joint-specific meas- April-June 200 I 139