The development and validation of a patientreported questionnaire to assess outcomes of elbow surgery

Transcription

1 The development and validation of a patientreported questionnaire to assess outcomes of elbow surgery J. Dawson, H. Doll, I. Boller, R. Fitzpatrick, C. Little, J. Rees, C. Jenkinson, A. J. Carr From University of Oxford, Oxford, England J. Dawson, DPhil, Senior Research Scientist H. Doll, DPhil, Senior Medical Statistician R. Fitzpatrick, PhD, Professor of Public Health C. Jenkinson, DPhil, Professor of Health Services Research Department of Public Health University of Oxford, Old Road Campus, Headington, Oxford OX3 7LF, UK. I. Boller, BSc, Research Officer School of Health and Social Care Oxford Brookes University, Marston Road Campus, Jack Straws Lane, Oxford OX3 0FL, UK. C. Little, FRCS(Tr & Orth), Consultant Orthopaedic Surgeon J. Rees, MD, FRCS(Tr & Orth), University Lecturer in Orthopaedics and Honorary Consultant Orthopaedic Surgeon A. J. Carr, FRCS, Nuffield Professor of Orthopaedic Surgery Nuffield Department of Orthopaedic Surgery Nuffield Orthopaedic Centre, Windmill Road, Oxford OX3 7LD, UK. Correspondence should be sent to Dr J. Dawson; jill.dawson@dphpc.ox.ac.uk 2008 British Editorial Society of Bone and Joint Surgery doi: / x.90b $2.00 J Bone Joint Surg [Br] 2008;90-B: Received 3 October 2007; Accepted 21 November 2007 We developed a questionnaire to assess patient-reported outcome after surgery of the elbow from interviews with patients. Initially, 17 possible items with five response options were included. A prospective study of 104 patients (107 elbow operations) was carried out to analyse the underlying factor structure, dimensionality, internal and test-retest reliability, construct validity and responsiveness of the questionnaire items. This was compared with the Mayo Elbow performance score clinical scale, the Disabilities of the Arm, Shoulder and Hand questionnaire, and the Short-Form (SF-36) General Health Survey. In total, five questions were considered inappropriate, which resulted in the final 12-item questionnaire, which has been referred to as the Oxford elbow score. This comprises three unidimensional domains, elbow function, pain and social-psychological ; with each domain comprising four items with good measurement properties. This new 12-item Oxford elbow score is a valid measure of the outcome of surgery of the elbow. The prevalence of symptomatic elbow conditions in the population of the United Kingdom is estimated to be approximately 8% both for men and women, with approximately 40% consulting a health-care professional about their problem. 1 It is not known how many of these proceed to surgery. The most common elective orthopaedic elbow operations include arthroscopy, nerve decompression, release for tendonitis, procedures to correct instability, excision of the radial head and joint replacement. Rheumatoid arthritis (RA) is the single most common disorder responsible for many of these interventions; rheumatoid patients are increasingly treated by joint replacement. 2,3 The lack of an appropriate standardised method of outcome assessment has led to inadequate evaluation of many elbow operations; 4 particularly lacking are patient-focused measures. 2 Well-developed patient-reported outcome measures provide an objective measure of a subjective construct 5 and, being independent of the surgical team, minimise the risk of bias. 6 It has been shown that assessments of health status made by health-care professionals can differ from those made by the patient, 7 whose perspective might be expected to match their degree of satisfaction with a given treatment more closely than an objective measure. A number of patient-reported outcome measures are now widely used to assess the results of orthopaedic operations. These include the Oxford hip, 8 knee 9 and shoulder 10,11 scores. However, only a limited number of questionnaires have been produced to measure a patients subjective experience of elbow surgery, and these have either not been specific to the elbow, 12 have been developed without patients input thereby tending to represent the clinician s perspective, 13 or, where they have involved patients input, have not included preand post-operative assessments. 14 This paper describes the development and assessment of a short questionnaire intended for use as an outcome measure of elbow surgery. The measure was modified and tested on a prospective series of patients undergoing elbow surgery. We report the stages of development, item selection and reduction, and assessment of the baseline measurement properties, as well as the responsiveness of what is now called the Oxford elbow score. Patients and Methods Approval was obtained from the local ethics committee and all patients consented to participate in the study. Development of the instrument. An initial series of exploratory 30-minute interviews was conducted by a research officer (IB) with 18 patients attending orthopaedic clinics for problems with their elbow, including six who had already 466 THE JOURNAL OF BONE AND JOINT SURGERY

2 THE DEVELOPMENT AND VALIDATION OF A PATIENT-REPORTED QUESTIONNAIRE TO ASSESS OUTCOMES OF ELBOW SURGERY 467 undergone surgery. The interviews were semi-structured and used prompts regarding patients perceived problems, such as pain, elbow movement, effects on employment/domestic work, social life (including sports activities), in addition to a general exploration of their feelings about their elbow problem. Interviews continued with additional patients until no new common themes emerged. From these interviews a questionnaire, initially containing 21 potential items, was drafted and piloted in 21 new patients. These patients were also asked to comment at the end of the questionnaire on any elbow-related problems not addressed by the questionnaire. The resulting modifications reduced the number of potential items to 17, with five response options per item. This was piloted on a further 20 patients and did not require further modification. A study was then conducted to examine whether the new questionnaire had the properties necessary for a health status instrument by being internally consistent, reproducible, valid and sensitive to clinical change. Internal consistency examines whether items of an instrument measure a single underlying concept. Reproducibility determines whether an instrument yields the same results on repeated trials under the same conditions. Validity assesses whether an instrument measures what it purports to measure, and can be examined in a number of ways. Construct validity is assessed by quantitatively examining the relationships of a construct, such as functional impairment of the elbow, to see whether the instrument produces a set of relationships with other variables that might be expected (for example clinical evidence or other health status instruments). Sensitivity to change, which is also referred to as responsiveness, reflects an instrument s ability to detect clinically significant changes over time; a particularly important property for an outcome measure. 15 Study to test the questionnaire. Between March 2004 and October 2006 we recruited 104 consecutive patients (107 elbows), who were due to have an operation on their elbow within two weeks at the Nuffield Orthopaedic Centre, Oxford. They completed a variety of assessments preoperatively, and again in the outpatient clinic six months later. The sample size for development of a questionnaire is typically based on the assumption that the number of respondents should exceed the number of items in the questionnaire by at least a factor of three. 16 So, with 21 potential items, a sample size of 100 was considered sufficient. Patient-reported assessments. Patients completed a general questionnaire while attending the pre-admission clinic. This provided demographic information about age, gender, and highest qualification (none, some qualifications/no degree, degree or higher). Next, they completed the new elbow questionnaire (containing 17 items at this stage) for each elbow receiving treatment. Each item scored 0 to 4, with lower scores denoting greater severity. They also completed the Disabilities of the Arm, Shoulder and Hand (DASH) outcome measure disability/symptom scale. 12 This is a 30-item, selfreporting questionnaire designed to measure physical function and symptoms in people with any of several musculoskeletal disorders of the upper limb. Items are scored between 1 and 5, with higher scores indicating greater severity. Finally, patients were asked to complete the Short-Form (SF)-36 general health questionnaire version II. 17,18 This contains 36 items and is widely used as a generic health status instrument. It provides scores on eight dimensions: physical functioning, social functioning, role limitations due to physical problems, role limitations due to emotional problems, mental health, energy/vitality, bodily pain and general health perceptions over the previous four weeks. Scores for each dimension range between 0 (poor health) and 100 (good health). An extra item addresses health change during the last 12 months. Summary scores can be calculated from the eight dimensions to represent limitations related to physical (physical component summary) and mental (mental component summary) factors, 19 which are standardised to general population scales with a mean of 50 (SD 10). At the end of the pre-operative assessment, patients were given a second copy of the Oxford elbow score questionnaire with a prepaid return envelope and asked to complete it again, 48 hours later. Clinical assessment. An upper limb surgeon (CL, JR, AJC) completed a clinical examination using the standard Mayo elbow performance score. 20 In this scale, pain, movement, stability and function account for 45%, 20%, 10% and 25% of the overall score, respectively, with low scores representing greater severity. The surgeon was blinded to patients responses to the other health status instruments. The same assessments were repeated in an outpatient clinic six months after surgery. Statistical methods. Data were analysed using SPSS version 14 (SPSS Inc., Chicago, Illinois). Non-parametric tests were used in the analysis where data were not normally distributed. To make comparison with the general population easier, the SF-36 scores were adjusted for age and gender. Data are presented using the mean and SD, median and range, or n (%), as appropriate. To examine the dimensionality of the Oxford elbow score, and the functioning and fit of individual items, we used a Rasch unidimensional measurement model in RUMM ,22 (RUMM Laboratory Pty Ltd., Duncraig, Australia). The significance level for all analyses was twosided p < Item exclusion. Items were excluded from the list of potential questionnaire items on the basis of two predetermined criteria. First, if in terms of their response distribution, they showed particularly high ceiling or floor effects, where at least 50% of responses to an item took either of the two most extreme response categories, and secondly, if on fitting to a Rasch unidimensional model to any identified domains, they showed a particularly poor fit to the model. Items were also considered for revision or exclusion if, dur- VOL. 90-B, No. 4, APRIL 2008

3 468 J. DAWSON, H. DOLL, I. BOLLER, R. FITZPATRICK, C. LITTLE, J. REES, C. JENKINSON, A. J. CARR ing factor analysis, they correlated > 0.40 on more than one factor, indicating that an item was not unique to one factor ( cross loaded ). Factor structure. Exploratory factor analysis with principal components extractions and varimax rotation was undertaken to group variables into uncorrelated factors, or components, on the basis of their correlation matrix, without pre-defining any particular questionnaire structure, and retaining all the variability in each item. 23 Factors were extracted if their eigenvalue was > 1. An eigenvalue indicates the relative proportion of total variance explained by the factor. A value > 1 is a criterion 24 which is commonlyused to determine how many factors are present, indicating a factor that extracts at least as much variance as the equivalent of one original variable. 23 Domain scores of the resulting factors were calculated as the sum of the component item scores. Individual item functioning. The Oxford elbow score and its component dimensions were assessed using the one-parameter Rasch model. 25,26 The Rasch model is equivalent to a test of the theoretical construct validity and adequacy of a scale. 27 It assumes that as a person s disability or symptoms increase, the probability of a maximum score on the item increases. The Rasch model assesses the unidimensionality of items in a scale. Whereas a composite score obtained from all items in the questionnaire was thought unlikely to fit a unidimensional model, any domains identified by factor analysis were expected to fit such a model and thus to confirm the structure of the Oxford elbow score. Internal consistency. Cronbach s α coefficients were calculated to assess the internal consistency of the questionnaire domains. Values of α in the range 0.80 to 0.90 are considered optimal, 28 with a minimum α of 0.70 necessary to claim internal consistency. 29,30 Test-retest reliability (repeatability) was assessed with intraclass correlation coefficients comparing Oxford elbow score domain scores obtained at patients preoperative assessment with measures completed at home 48 hours later and returned for analysis. Convergent validity. An a priori hypothesis was that at least a moderate correlation would be obtained between the new elbow measure, the Mayo elbow performance score clinical assessment and the DASH, each of which purport to measure something similar. It was considered likely that SF-36 domains would be less highly correlated with the new measure. To test the convergent validity of the Oxford elbow score, Spearman s correlation coefficients between any individual domain scores that emerged (i.e. pain) and similar domain scores of the DASH, SF-36 and the Mayo elbow performance score clinical scale was performed. Similar domains from different instruments were expected to be highly correlated (r > 0.5) with each other. Divergent validity. It was considered likely that SF-36 domains would be less highly correlated with the new measure. Again using Spearman s correlation, Oxford elbow score domain scores and dissimilar individual domain scores on the SF-36 were examined. The Oxford elbow score pain domain would not be expected to be highly correlated with the SF-36 mental health, and general health perception domains. Responsiveness was assessed in the patients who underwent surgery and who provided relevant outcome data at both pre- and post-operative assessments. Mean preoperative and six-month post-operative scores, change scores and effect sizes between Oxford elbow score, DASH, Mayo elbow performance score clinical scale and SF-36 domains were compared. Change scores were calculated as the pre-operative score minus the post-operative score for each instrument. Effect size is a method of calculating the extent of change measured by an instrument in a standardised way that allows comparison between instruments. 31 Here, it was calculated as the difference between the sample s mean pre- and post-operative scores, divided by the SD of the pre-operative score. An effect size of 1.0 is equivalent to a change of one SD in the sample. Effect sizes of 0.2, 0.5 and 0.8 are typically regarded as indicating small, medium and large degrees of change, respectively. We hypothesised that at least moderate effect sizes ( 0.5) would be obtained between the elbow or upperlimb-specific measures, and relevant domains of the SF-36 (i.e. pain and physical functioning), and that the elbowspecific measures would be the most responsive. Results Study sample and characteristics. Of the 107 elbows in 104 patients, 62 (58%) were right elbows for which surgery was intended. The mean age of the patients at the time of questionnaire was years (SD 15.14, median 47, 18 to 81). A total of 59 patients (57%) were male, 51 (49%) were in full-time paid employment, nine (9%) were part-time employed, and 22 (21%) were retired. The remaining 22 patients (21%) were either students, unemployed or a homemaker/carer. The diagnosis affecting the elbows was primary osteoarthritis (OA) in ten (9%), secondary OA in 23 (22%), RA in 24 (23%), post-traumatic stiffness in 12 (11%), epicondylitis in 12 (11%), and other conditions in 26 (24%) patients. Two of the original 107 planned elbow operations were cancelled, one required revision and two others were postponed beyond the study period. Of the remaining 102 patients, 62 (61%) attended an outpatient appointment and were fully assessed six months after surgery. A further 13 (13%) completed their follow-up assessment with questionnaires only, which they returned by post. Therefore, in total, 75 elbows (74%) received at least one form of follow-up assessment. Of the remaining 27 (26%) either the patient did not attend their six-month follow-up appointment, or did not respond to the follow-up postal questionnaire or the reminder that followed. The characteristics of those who did and did not complete a follow- THE JOURNAL OF BONE AND JOINT SURGERY

4 THE DEVELOPMENT AND VALIDATION OF A PATIENT-REPORTED QUESTIONNAIRE TO ASSESS OUTCOMES OF ELBOW SURGERY 469 Table I. Pre-operative short-form (SF)-36 scores in our study sample, in the study sample adjusted for age and gender, and in a general population sample 35 SF-36 domain Study sample (n = 104) General population (n = 9332) 35 Mean (SD) Adjusted for age and gender (mean) Mean (SD) Significance t-test p-value Physical functioning (28.16) (17.98) 9.14 < Pain (27.27) (21.69) < Role physical (28.16) (29.93) < Role emotional (30.35) (31.76) 3.24 < Social functioning (29.63) (19.58) 4.45 < Mental health (20.36) (17.24) Energy/vitality (22.60) (19.67) < General health perception (25.01) (19.90) 2.10 < 0.05 up assessment were compared, but no significant differences were found in relation to age (t = 1.43, p = 0.16), employment status (χ , p = 0.43) or having a diagnosis of RA (against any other diagnosis) (χ , p = 0.92). However, females were more likely to respond than males (females 38 of 44, 86%; males 37 of 58, 64%; χ , p = 0.01). Pre-operative scores for clinical assessment, DASH and SF- 36. The mean pre-operative score on the Mayo elbow performance score clinical scale was 63.6 (SD 19.2, 5 to 100) and on the DASH, 42.6 (SD 21.7, 2.50 to 92.5). There was no statistically significant difference between scores from the right and left elbows. Patients s scores on the SF-36 were compared with population norms (Table I), 8 and showed that patients in the study had significantly lower (poorer) scores in all domains except mental health. Item response distribution: potential Oxford elbow score items. The Oxford elbow score was completed preoperatively for 106 elbows. The data was incomplete for one elbow. Two of the 17 potential items showed a particularly strong floor effect, with 70 (66%) and 60 (57%) cases, respectively, giving the response No, not at all to items concerned with difficulty using the lavatory and trouble with transport. Consequently, these two items were excluded. Initial assessment of dimensionality and factor structure of the Oxford elbow score. Dimensionality. The remaining 15 items were found not to fit a unidimensional model on Rasch analysis. Factor structure. An exploratory factor analysis of the remaining 15 items extracted three factors with an eigenvalue > 1, explaining 69.1% of the variance. However, one item, turning a key in a lock, cross-loaded on two factors and a number of patients observed that they could use their unaffected arm for this task. This latter consideration also applied to another item, using a telephone/mobile telephone, which on further preliminary Rasch analysis had disordered response categories. In addition, the item being in low spirits had disordered response categories. Accordingly, all three of these items were removed leaving 12 items in the questionnaire. The Oxford elbow score 12-item questionnaire (final version, Table II). Completion rates. The research officer (IB) checked questionnaire responses and occasionally assisted patients at the pre-operative stage, so completion rates were analysed at the follow-up stage, including the 13 (13%) postal responses. The response rate for each of the 12 individual items was 100%. Factor structure. Factor analysis of the remaining 12 items produced three components with an eigenvalue < 1.0, which explained 75.1% of the variance. The loadings on these three components are shown in Table III. Component 1, comprising four items, measures the level of elbow pain ( pain domain ). Component 2, comprising four items, measures perceived functional impairment related to the elbow ( elbow function domain ). Component 3, comprising four items, includes two activities concerned with participation (limitations to work, leisure and everyday activities) and two items about psychological aspects of the condition (elbow problem on your mind or controlling your life ) ( social-psychological domain ). Dimensionality. Application of a unidimensional Rasch model confirmed that the 12-item Oxford elbow score questionnaire was not unidimensional. However, each of the three domains identified by factor analysis was found to be unidimensional. Scores for each domain are calculated as the sum of each individual item score and in each case, expressed on a scale of 0 to (lower score representing greater severity) using the following formula: Conversion to metric score: 100 Actual score Maximum possible domain score Descriptive statistics from the three domains (scored prior to 106 operations) were: elbow function comprising four items: mean 58.7 (SD 26.2), median 62.5 (0 to 100), pain domain comprising four items: mean 45.0 (SD 25.1), median 37.5 (0 to 100), social-psychological domain comprising four items: mean 43.7(SD 25.5), median 43.8 (0 to 100). VOL. 90-B, No. 4, APRIL 2008

5 470 J. DAWSON, H. DOLL, I. BOLLER, R. FITZPATRICK, C. LITTLE, J. REES, C. JENKINSON, A. J. CARR Table II. The final 12-item Oxford elbow score During the past 4 weeks Have you had difficulty lifting things in your home, such as putting out the rubbish, because of your elbow problem? No difficulty A little bit of difficulty Moderate difficulty Extreme difficulty Impossible to do 2. Have you had difficulty carrying bags of shopping, because of your elbow problem? No difficulty A little bit of difficulty Moderate difficulty Extreme difficulty Impossible to do 3. Have you had any difficulty washing yourself all over, because of your elbow problem? No difficulty A little bit of difficulty Moderate difficulty Extreme difficulty Impossible to do 4. Have you had any difficulty dressing yourself, because of your elbow problem? No difficulty A little bit of difficulty Moderate difficulty Extreme difficulty Impossible to do 5. Have you felt that your elbow problem is controlling your life? No, not at all Occasionally Some days Most days Every day 6. How much has your elbow problem been on your mind? Not at all A little of the time Some of the time Most of the time All of the time 7. Have you been troubled by pain from your elbow in bed at night? Not at all 1 or 2 nights Some nights Most nights Every night 8. How often has your elbow pain interfered with your sleeping? No, not at all Occasionally Some days Most days All of the time 9. How much has your elbow problem interfered with your usual work or everyday activities? Not at all A little bit Moderately Greatly Totally 10. Has your elbow problem limited your ability to take part in leisure activities that you enjoy doing? No, not at all Occasionally Some days Most days All of the time 11. How would you describe the worst pain you had from your elbow? No pain Mild pain Moderate pain Severe pain Unbearable 12. How would you describe the pain you usually had from your elbow? No pain Mild pain Moderate pain Severe pain Unbearable THE JOURNAL OF BONE AND JOINT SURGERY

6 THE DEVELOPMENT AND VALIDATION OF A PATIENT-REPORTED QUESTIONNAIRE TO ASSESS OUTCOMES OF ELBOW SURGERY 471 Table III. Factor analysis: rotated loadings of each of the 12 items on the three factors (items ordered by loading size) representing the final version of the Oxford elbow score (OES) Factor 1 Factor 2 Factor 3 Item Number OES item * Pain Elbow function Social/psychological 7 Pain in bed at night Pain interfered with sleeping Usual pain Worst pain Difficulty dressing Difficulty washing all over Difficulty with lifting Difficulty carrying bags Limited leisure activities Elbow problem on your mind Elbow problem controlling your life Interfered with usual work/everyday activities % of variance explained by this factor * see Table II for precise wording of items Table IV. Pre-operative Spearman s correlation coefficients between the three Oxford elbow score (OES) domains, the Mayo elbow performance score (MEPS) clinical scale, Disabilities of the Arm, Shoulder and Hand outcome measure (DASH), Short-Form (SF)-36 domains, and SF-36 physical component summary (PCS) and mental component summary (MCS) SF-36 domains SF-36 components Physical Role Role Social Mental Energy/ General OES domains MEPS DASH functioning Physical Emotional functioning health vitality Pain health PCS MCS Elbow function * * * * * * * * Pain * * * * * * * * * Social-psychological * * * * * * * * * * * High correlation values (r > 0.5) are shown in bold typeface * p < p < 0.01 p < 0.05 Internal reliability (consistency). The Cronbach s α coefficients for the three Oxford elbow score domains were pain 0.89, elbow function 0.90, social-psychological 0.84, and following deletion of each item the coefficients revealed that no one item had a particularly large effect on the reliability of each domain.thus each factor had a high level of internal reliability and none exceeded 0.90, which would indicate item redundancy. 29,30 Test-retest reliability (repeatability). A total of 52 patients provided repeat measures of the Oxford elbow score preoperative measurements. The intraclass correlation coefficient comparing these with measures of the Oxford elbow score scales obtained at least 48 hours earlier were pain 0.98, elbow function 0.89 and social-psychological 0.87 (all statistically significant p < 0.001). Convergent and divergent validity. The correlations applying Spearman s ρ between the Oxford elbow score domains, the Mayo elbow performance score clinical scale, the DASH and SF-36 scores are shown in Table IV. All of these were positive, except in relation to the DASH (which is scored in an opposite direction to the other scales), demonstrating that the lower the Oxford elbow scores, (indicating worse elbow problems), the poorer were the Mayo elbow performance score clinical scale, DASH and SF-36 scores. The convergent validity of the Oxford elbow score was demonstrated by high correlations between the Mayo elbow performance score clinical scale, the DASH and all three Oxford elbow score domains, with one exception: the Oxford elbow score social-psychological scale was only moderately correlated with the Mayo elbow performance score (Table IV). The Oxford elbow score elbow function domain was strongly associated with the SF-36 physical functioning, role physical, social functioning, pain and energy/vitality domains, and with the SF-36 physical component summary score. The Oxford elbow score pain domain was most strongly related to the SF-36 pain and role physical domains, and to the physical component summary score. The Oxford elbow score social-psychological domain was most strongly related to the SF-36 role physical and social functioning domains and to the physical component summary score. VOL. 90-B, No. 4, APRIL 2008

7 472 J. DAWSON, H. DOLL, I. BOLLER, R. FITZPATRICK, C. LITTLE, J. REES, C. JENKINSON, A. J. CARR Table V. Instrument responsiveness: mean pre- and six-month post-operative scores, change scores and effect sizes comparing the Oxford elbow score (OES), Mayo elbow performance score (MEPS) clinical assessment, Disabilities of the Arm, Shoulder and Hand outcome measure (DASH) and Short-Form (SF)-36 domains Number * Mean (SD) Pre-op 12 mth post-op Mean change (SD) p-value Effect size OES Elbow function (24.80) (22.21) (21.03) < Pain (25.33) (22.92) (25.01) < Social-psychological (23.62) (25.04) (23.70) < MEPS Elbow scale (19.50) (16.06) (17.58) < DASH Disability score (20.19) (20.49) (15.26) < SF-36 Bodily pain (27.95) (25.10) (24.33) < Physical function (26.35) (30.32) (16.76) Role physical (32.03) (30.27) (26.72) < Mental health (20.14) (18.82) (18.47) Role emotional (29.38) (28.10) (30.95) Vitality/energy (22.62) (23.48) (23.13) Social function (28.84) (26.78) (30.63) Health perceptions (24.33) (24.67) 2.42 (16.70) * the number of cases with complete data provided at both pre- and post-operative assessments to permit the evaluation of scales and hence change in pain, function, overall health-related quality of life the positive/negative direction of the sign relates to the direction in which each scale is scored to denote increasing severity, is irrelevant to the size of the effect paired t-tests calculated as mean change/sd of pre-operative score Divergent validity of the Oxford elbow score was shown by the moderately low correlations between all three Oxford elbow score domains and the SF-36 mental health and general health perception domains. Responsiveness. The mean pre-operative and 12-month postoperative scores, change scores and effect sizes of the Oxford elbow score, Mayo elbow performance score, DASH and SF- 36 domains for patients with scores at both assessments are presented in Table V. It can be seen that each of the elbow and upper-limb-specific assessments and two of the SF-36 domains (pain and role physical) exhibited highly statistically significant improvements in scores at 12 months post-operatively. Effect sizes for the pain and social-psychological scales of the Oxford elbow score and the Mayo elbow performance score clinical scale were both > 1.0, indicating a very large degree of improvement. The Oxford elbow score elbow function scale and the DASH showed effect sizes of 0.84 and -0.74, respectively, still indicating substantial improvement. Conversely the generic scales of the SF-36 had lower effect sizes than the elbow or upper-limb-specific measures, indicating that these condition-specific scales were more responsive in this context. The two most responsive of the generic SF-36 domains, pain and role physical, only registered moderate effect sizes. Discussion The 12-item Oxford elbow score questionnaire developed in this study addresses themes identified from interviews conducted with patients with various elbow problems who were about to undergo operative treatment. The resulting questionnaire was tested prospectively on surgical patients using accepted methods. 23,32-34 Having excluded five items that were unreliable the final 12-item version of the questionnaire appeared acceptable to patients, all of whom completed the questionnaire preoperatively and 73% of patients (74% of elbows) after the 12 months follow-up. Although the final 12-item questionnaire was not unidimensional, and hence not suitable for the calculation of an overall score, factor analysis found three subscales (or domains) that underlay the 12 items representing pain, elbow function, and a social-psychological domain, and were internally consistent, unidimensional, and had no item redundancy. The assessment of convergent and divergent validity showed that the new questionnaire performed well in this regard. The Oxford elbow score elbow function and pain domains correlated highly with the Mayo elbow performance scale clinical scale and the DASH, as well as with relevant domains of the generic SF-36. These correlations were in the directions that we had anticipated. Correlations between the Oxford elbow score social-psychological domain and the DASH, as well as with the relevant domains of the SF-36, were also high, but the correlation with the Mayo elbow performance score was barely moderate. This latter finding was unsurprising, as the Oxford THE JOURNAL OF BONE AND JOINT SURGERY

8 THE DEVELOPMENT AND VALIDATION OF A PATIENT-REPORTED QUESTIONNAIRE TO ASSESS OUTCOMES OF ELBOW SURGERY 473 elbow score places some emphasis on patients opinions in relation to elbow problems dominating their lives, which may well not mirror the more objective perspective assessed by the Mayo elbow performance score. Responsiveness is considered to be a particularly important property of a health outcome measure. 28 Tested in the context of patients undergoing elbow surgery, the Oxford elbow score was also demonstrated to be highly responsive. Each domain showed a large effect size, indicating a very large degree of improvement post-operatively. This was also the case for the upper-limb DASH scale, whereas the SF-36 was much less responsive to changes following surgery. The results of this study demonstrate that the newlydeveloped Oxford elbow score is reliable, valid and acceptable to patients, with a high rate of completion. It is a useful method of assessing the outcome of elbow surgery from the patient s perspective. Supplementary Material Tables showing the development of the Oxford elbow score and Rasch analysis are available with the electronic version of this article at We wish to thank all the patients who contributed their views and valuable time to assist us with this study. The study received no external funding and none of the authors have any conflict of interest in relation to the study/paper. No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article. References 1. Roquelaure Y, Ha C, Leclerc A, et al. Epidemiologic surveillance of upper-extremity musculoskeletal disorders in the working population. Arthritis Care and Research 2006;55: MacDonald D. The shoulder and elbow. In: Pynsent P, Fairbank J, Carr A, eds. Outcome measures in orthopaedics. Oxford: Butterworth-Heinemann Ltd., van der Lugt JCT, Rozing PM. Systemic review of primary total elbow prostheses used for the rheumatoid elbow. Clin Rheumatol 2004;23: Buchbinder R, Green S, Bell S, et al. Surgery for lateral elbow pain (Cochrane Database of Systematic Reviews). The Cochrane Library, Issue 1. CD003525, Oxford: Update Software: Haywood K. Patient-reported outcome: measuring what matters or just another paper exercise? Musculoskeletal Care 2006;4: Pynsent P, Fairbank JTC, Carr A. Outcome measures in orthopaedics. Oxford: Butterworth-Heinemann, Jenkinson C. Measuring health and medical outcomes: an overview. In: Jenkinson C, ed. Measuring health and medical outcomes. London: UCL Press, 1994: Dawson J, Fitzpatrick R, Carr A, Murray D. Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg [Br] 1996;78-B: Dawson J, Fitzpatrick R, Murray D, Carr A. Questionnaire on the perceptions of patients about total knee replacement. J Bone Joint Surg [Br] 1998;80-B: Dawson J, Fitzpatrick R, Carr A. Questionnaire on the perceptions of patients about shoulder surgery. J Bone Joint Surg [Br] 1996;78-B: Dawson J, Fitzpatrick R, Carr A. The assessment of shoulder instability: the development and validation of a questionnaire. J Bone Joint Surg [Br] 1999;81-B: Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder and hand) [corrected]: The Upper Extremity Collaborative Group (UECG). Am J Ind Med 1996;29: King GJW, Richards RR, Zuckerman J, et al. A standard method for assessment of elbow function. J Shoulder Elbow Surg 1999;8: Sathyamoorthy P, Kemp GJ, Rawal V, Rayner V, Frostick SP. Development and validation of an elbow score. Rheumatology (Oxford) 2004;43: Guyatt G, Walter S, Normal G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis 1987;40: Barrett P, Kline P. The observation to variable ratio in factor analysis. J Personality Group Behaviour 1981;1: Ware JEJ, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). 1: conceptual framework and item selection. Med Care 1992;30: Jenkinson C, Stewart-Brown S, Petersen S, Paice C. Evaluation of the SF-36 version 2 in the United Kingdom. J Epidemiol Community Health 1999;53: Ware JE, Kosinski M, Keller SD. SF-36 physical and mental summary scales: a user s manual. Boston: The Health Institute, New England Medical Center, Morrey BF, An KN. Functional evaluation of the elbow. In: Morrey BF, ed. The elbow and its disorders. Third ed. Philadelphia: WB Saunders, 1993: Prieto L, Alonso J, Lamarca R. Classical test theory versus Rasch analysis for quality of life questionnaire reduction. Health and Quality of Life Outcomes 2003;1: Wolfe F, Kong SX. Rasch analysis of the Western Ontario MacMaster Questionnaire (WOMAC) in 2205 patients with osteoarthritis, rheumatoid arthritis, and fibromyalgia. Ann Rheum Dis 1999;58: Fayers P, Machin D. Quality of life: assessment, analysis and interpretation. Oxford: John Wiley & Sons Ltd., Kaiser HF. The application of electronic computers to factor analysis. Education and Physiological Measurement 1960;20: Hambleton RK, Jones RW. Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement Issues and Practice 1993;12: Andrich D. Rasch models for measurement. London: Stage Publications, Avlund K, Kreiner S, Schultz-Larsen K. Construct validation and the Rasch model: functional ability of healthy elderly people. Scand J Soc Med 1993;21: Guyatt GH, Feeny DH, Patrick DL. Measuring health-related quality of life. Ann Intern Med 1993;118: Nunnally JC, Bernstein IH. Psychometric theory. Third ed. New York: McGraw-Hill, Kline P. A handbook of psychological testing. London: Routledge, Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care 1989;27(Suppl): No authors listed. Royal College of Surgeons of England. Background to outcomes database (date last accessed 27 November 2007). 33. Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2: Streiner DL, Normal GR. Health measurement scales: a practical guide to their development and use. New York: Oxford University Press, Jenkinson C, Coulter A, Wright L. Short form 36 (SF36) health survey questionnaire: normative data for adults of working age. BMJ 1993;306: VOL. 90-B, No. 4, APRIL 2008