C. B. Terwee 1, L. B. Mokkink 1, M. P. M. Steultjens 1,2 and J. Dekker 1,2

Transcription

1 Rheumatology 2006;45: Advance Access publication 3 February 2006 Performance-based methods for measuring the physical function of patients with osteoarthritis of the hip or knee: a systematic review of measurement properties C. B. Terwee 1, L. B. Mokkink 1, M. P. M. Steultjens 1,2 and J. Dekker 1,2 doi: /rheumatology/kei267 Objective. To systematically review the measurement properties (i.e. internal consistency, reproducibility, validity, responsiveness and interpretability) of all performance-based methods which have been used to measure the physical function of patients with osteoarthritis of the hip or knee. Methods. A systematic search was conducted in Medline, CINAHL, PsychINFO and Embase. Standardized criteria were applied to assess the quality of the clinimetric studies and the measurement properties. Results. Twenty-six performance-based methods were included: 13 walking tests, two stair-climb tests, one chair test and ten multi-item tests. Three out of seven multi-activity tests were tested for internal consistency and two were rated positively. Fourteen tests were tested for reliability and five were rated positively. The absolute measurement error (agreement) was assessed for 10 tests. Only one test received a positive rating. Fourteen tests were tested for construct validity. Only two tests received positive ratings. Responsiveness was assessed for 12 tests, but none of them received a positive rating. A lot of indeterminate ratings were given, mostly for small studies or non-optimal analyses. Conclusion. Many more well-designed studies are needed to assess the measurement properties of performance-based methods. More importantly, however, before one can make a justified choice of a particular performance-based method, consensus is needed on what activities should be included in a performance-based test for patients with hip or knee osteoarthritis and which aspects of function should be measured. KEY WORDS: Osteoarthritis, Systematic review, Performance-based method, Reproducibility of results, Validation study. Physical function, i.e. the ability to perform daily activities, is generally considered one of the most important outcome measures for patients with osteoarthritis (OA) of the hip or knee [1]. In addition to self-reports, a variety of performance-based methods are used to assess the physical function of patients with OA of the hip or knee, e.g. the 6-min walk test [2], self-paced walking time tests [3], the timed get up and go test [4], different stair-climb tests, etc. Most tests include the recording of the time it takes a patient to perform the requested activity. In addition, several observational methods have been described that use ratings from observers to assess the quality of physical function [5, 6]. There is no overview of the measurement properties of these performance-based methods. Such an overview is necessary to decide which test to choose for routine clinical practice or in designing a study. The aim of this study was to systematically review the literature on the measurement properties (i.e. internal consistency, reproducibility, validity, responsiveness and interpretability) of all performance-based methods that have been used to measure the physical function of patients with OA of the hip or knee. Studies in patients undergoing hip or knee replacement (with or without a further specification of the patient population) were also included because in general the majority of these patients are diagnosed with OA [7]. A standardized set of criteria was applied to assess the quality of the clinimetric studies and the measurement properties. This review aims to offer investigators and clinicians a basis for choosing a performance-based method for clinical practice or for a study. Methods Literature search We searched the following databases: Medline (1966 to April 2004), CINAHL (1982 to September 2003), PsycINFO (1966 to September 2003) and Embase (1980 to week 40, 2003). The full search strategy is listed in Appendix 1. In short, a combination of different variations of the following text words was used: hip or knee, and osteoarthritis or replacement, and performance test or objective test or observational test or gait or stair or chair or walk or gait analysis or kinematic analysis. Additional articles were identified by manually searching references of the retrieved articles and the authors own literature databases. Inclusion and exclusion criteria We used the following inclusion criteria: (i) The study population should include patients with OA of the hip or knee, or patients undergoing hip or knee replacement. Down 1 EMGO Institute, 2 Department of Rehabilitation Medicine, VU University Medical Center; Amsterdam, The Netherlands Submitted 22 April 2005; revised version accepted 22 November Correspondence to: C. B. Terwee, EMGO Institute, VU University Medical Center, Van der Boechorststraat 7, 1081 BT Amsterdam, The Netherlands. cb.terwee@vumc.nl 890 ß Published by Oxford University Press on behalf of the British Society for Rheumatology 2006.

2 Performance-based methods of physical function 891 (ii) A performance-based method should have been used to assess physical function, which was defined as a test in which patients had to perform one or more activities, while aspects related to the quality of physical function, i.e. either the (dis)ability to perform activities or the quality of the performance (e.g. time to perform the activity or kinematic parameters), were measured by an observer or a measurement instrument. (iii) The article should have been published in the English language. (iv) Information on the measurement properties of the performance-based method in a sample of patients with OA of the hip or knee, or patients undergoing hip or knee replacement should be provided. Information on measurement properties was only included if it was intentionally collected or calculated to assess the measurement properties of the particular performance-based method. If, for example, correlations between a performance-based method and a self-report questionnaire were presented to assess the validity of the questionnaire (while the performance-based method was used as an external criterion), the data were not included in this review. We also excluded studies with fewer than 10 patients. Finally, we only included full-text articles; abstracts, books, theses or conference proceedings were excluded. Selection of articles Article selection, data extraction and quality assessment was performed by two independent reviewers (CBT and either LBM or MPMS). Disagreements were discussed and resolved. All abstracts were scanned against the inclusion criteria. The full-text article was retrieved of all abstracts that fulfilled the inclusion criteria and of abstracts that did not contain measurement properties but where indications were found that these properties were presented in the full-text article. The retrieved articles were reviewed again against the inclusion criteria. Data extraction A description of the performance-based method was extracted from the included articles, including the activities that were performed by the patients, the aspects of function that were measured, and the equipment used to measure these aspects. Information on the measurement properties of the instruments and the quality of the clinimetric studies was extracted, using a recently developed standardized checklist, described below. Quality assessment of the clinimetric studies and measurement properties Bot et al. [8] developed a checklist to evaluate the quality and outcomes of clinimetric studies on self-report questionnaires. The checklist was partly based on the review criteria developed by the Scientific Advisory Committee of the Medical Outcome Trust [9] and a checklist developed by Bombardier and Tugwell [10]. We slightly modified this checklist for the evaluation of performancebased methods (Appendix 2). All measurement properties were rated as positive (þ), negative ( ) or indeterminate (?) depending on the methods and results of the clinimetric studies (see below). If no information was available, a 0 was recorded. For all measurement properties a sample size of at least 50 patients was considered necessary to receive a positive rating [11]. The following measurement properties were evaluated. Reproducibility. A distinction was made between reliability and absolute measurement error (agreement). Reliability refers to the ability to differentiate among patients, despite measurement errors [12]. The intraclass correlation coefficient (ICC), or kappa for dichotomous or ordinal data, was considered as an adequate measure of reliability [12]. The use of Pearson correlation coefficients was rated as indeterminate, as it neglects systematic errors [13]. An ICC >0.70, with the lower limit of the confidence interval >0.60 or a sample size of at least 50 patients, was considered as acceptable [9, 14]. Absolute measurement error, sometimes called agreement [12], refers to the precision of the instrument and is expressed in the units of measurement of the instrument. The limits of agreement [15], the standard error of measurement (SEM), or smallest detectable change (SDC) [16], were regarded as adequate measures of absolute measurement error. The rating of the absolute measurement error depends on what is considered a minimal important difference (MID) in scores between or within persons. Absolute measurement error was rated positively if the limits of agreement or SDC were smaller than the MID. If the MID was not known, a value of 0.5 S.D. was considered as a general guideline for the MID [17]. Internal consistency. Factor analysis should have been applied to determine if scores to be summarized measure the same concept [18]. In addition, a Cronbach s alpha should have been calculated for each set of scores to be summarized, as a measure of internal consistency. A Cronbach s alpha of at least 0.70 was considered acceptable [14]. Construct validity. Validity refers to the ability of an instrument to measure the concept that it is intended to measure [9]. Construct validity was considered to be adequate if specific hypotheses were defined about expected relationships with other measures of physical function or about expected differences in scores between specific subgroups, and if at least 75% of the results of the clinimetric study were in correspondence with these hypotheses. Floor and ceiling effects were considered present if more than 15% of respondents achieved the highest or lowest possible score, respectively [19]. Responsiveness. Responsiveness refers to an instrument s ability to detect important change over time in the concept being measured [20, 21]. It should be considered an aspect of validity in a longitudinal setting [21, 22]. Responsiveness was therefore rated similarly to construct validity. Reproducibility, validity and responsiveness depend on the setting and the population in which it is assessed. Therefore, we considered a clear description of the design of each individual clinimetric study including characteristics of the study population (diagnosis and clinical features), measurements, testing conditions and data analysis required to receive a positive rating. Furthermore, if any methodological weakness in the design or execution of the clinimetric study was found, the evaluated measurement property was rated as indeterminate. Interpretability. Interpretability was defined as the degree to which one can assign qualitative meaning to quantitative scores [9]. We recorded whether the following information was presented that could aid in interpreting the scores of the performance-based test: (i) mean scores (and S.D.) of the study population before and after treatment; (ii) mean scores (and S.D.) in relevant subgroups of the study population; (iii) mean scores (and S.D.) of patients and mean scores of healthy controls; (iv) mean scores (and S.D.) per category of other well-known measures; and (v) mean changes in scores (and S.D.) of patients who consider themselves to be improved and of patients who consider themselves to be unchanged. Down

3 892 C. B. Terwee et al. Investigators had to provide at least two of the previous described types of information for a positive rating of interpretability. Practical issues. The time needed to complete the test was recorded as a measure of patient burden. In addition, the requirements to perform the test were recorded, in terms of apparatus and space required. These practical issues were not rated because their importance depends on the application of the test, e.g. in clinical practice or in a large study. Results We obtained the following number of abstracts from the searches: Medline, 1834 abstracts; Embase, 221 additional abstracts; CINAHL, 64 additional abstracts; PsycINFO 17 additional abstracts. Seventy-three abstracts were selected for further inspection of the full-text article. The most common reasons for excluding an abstract were a lack of a performance-based method or a lack of information on measurement properties. In addition, 26 articles were considered from our own literature databases or by reference tracking of the retrieved articles. Seventy-one articles did not meet the inclusion criteria, mostly because of a lack of information on measurement properties. Finally, we included 28 studies, referring to 26 performance-based methods. Description of the 26 performance-based methods Thirteen walking tests were found representing minor variations on the same theme (Table 1), two different stair-climb tests and one chair test. Ten multi-item tests were found, in which different activities are being performed, such as walking, stair climbing, rising from a chair and sitting down. In most of these tests, the time to perform the activities was measured, one test measured level of assistance [the Iowa Level of Assistance Scale (ILAS)], two tests measured observed disability [Steultjens test and the Functional Assessment System (FAS)], and two tests measured functional parameters from accelerometer signals (Numact monitor and DynaPort KneeTest). Internal consistency In seven multi-activity tests, scores for different activities are summarized into one total score (Table 2). For three of these tests [Physical Activity Restrictions test (PAR), Steultjens test, and FAS], the dimensionality was examined by means of factor analysis or Mokken scale analysis. Only for the PAR and Steultjens test were Cronbach s alphas calculated for each subscale. Both tests were rated positively for internal consistency. Reproducibility Fourteen tests (54%) have been tested for reliability (Table 2). Most studies calculated ICCs and all ICCs were >0.70. Nine tests received an indeterminate rating for reliability, six times due to a small sample size (self-paced walk 1, 2 and 3, footprint analysis, Marks s test, and FAS) and three times due to an inadequate method or unclear description of the design (Lin s test, PAR and Steultjens test). Only five tests received a positive rating for reliability [5-min walking field test, gait analysis 2, get up and go, Aggregated Locomotor Function score (ALF) and ILAS]. Absolute measurement error (agreement) was assessed for 10 tests (38%) (Table 2). A MID was defined only for the ILAS test. The SDC of the ILAS (6 points) was smaller than the MID (7 points). For the self-paced walk 3 test, the get up and go test, the ALF and the Lin test, the SDC was less than 0.5 S.D. of the baseline mean score. These tests, however, received an indeterminate rating for absolute measurement error due to small sample sizes (selfpaced walk 3 test, get up and go test and ALF) or flaws in the study design (Lin s test). The ILAS was therefore the only test that received a positive rating for absolute measurement error. Construct validity Fourteen of the 26 tests (54%) have been tested for construct validity. Nine studies received an indeterminate rating because no hypotheses were specified. For the footprint analysis test, the get up and go test and the PAR, at least 75% of the results of the clinimetric study were in correspondence with the hypotheses. The footprint analysis test, however, received an indeterminate rating due to a small sample size. Therefore, only the get up and go test and the PAR received positive ratings for construct validity. Responsiveness Responsiveness was assessed for 12 tests (46%). Ten studies received an indeterminate rating because no hypotheses were specified. For two tests (gait analysis 3 and Steultjens test), less than 75% of the results were in correspondence with the hypotheses. Steultjens test therefore received a negative rating for responsiveness; gait analysis 3 received an indeterminate rating due to a small sample size. Interpretability For eight tests, at least one type of information was presented. No information found was recorded for interpretability of the PAR because three different interventions were used, which makes the mean scores uninterpretable. For five tests (the 6-min gait test, Locome trix, gait analysis 1, gait analysis 4 and the stair ascent test), two types of information were presented, but all received an indeterminate rating due to small sample sizes. Practical issues In 11 studies, the amount of time needed to perform the test was mentioned, varying from quick to 30 min. From the description of the tests, we estimate that all tests can be performed within 30 min. The equipment needed to perform the test varies from 3 m of indoor space and a stopwatch to standardized equipment (e.g. stair or chair), accelerometers with special software, or a two-dimensional biomechanical gait analysis system. Quality assessment The quality assessment of the 26 performance-based methods is summarized in Table 4. Only 11 of the 169 ratings were positive (7%). Forty-six (27%) indeterminate ratings were given, mostly for small studies or non-optimal analyses. Four tests (self-paced walk 3, gait analysis 2, ALF and Steultjens test) received one positive rating, two tests (get up and go and PAR) received two positive ratings and one test (ILAS) received three positive ratings. Discussion We identified 26 performance-based methods that have been used to measure the physical function of patients with OA of the hip or knee or patients undergoing hip or knee replacement and for which measurement properties have been evaluated. None of the 26 performance-based methods included has been tested for all measurement properties and only 7% of the ratings were positive. Forty-six (27%) indeterminate ratings were given, mostly for small Down

4 TABLE 1. Description of the performance-based methods Performance-based method Target population Study population Walking tests Self-paced walk 1 Knee OA Knee OA, women [3] Self-paced walk 2 Knee OA Knee OA, women [25] Self-paced walk 3 Knee OA Knee-OA, women [26] Five-min walking field test Six-min gait test Measurement concept Number of activities Activities Functional measures Number of scores Time to administer Requirements Walking time 1 3 m walking Time 1 3 m indoor level ground, stopwatch Walking speed 1 13 m walking Gait velocity 1 13 m walkway, stopwatch Knee function 1 13 m walking Time 1 13 m carpeted walkway, two photocells Arthritis [27] Knee OA [28] Aerobic fitness 1 5 min walking Distance 1 5 min 26 m rectangular area, stopwatch, measuring wheel Chronic heart failure [29] Knee OA, undergoing TKR [2, 30] Pedometer General Patients after Walking activity 1 Walking (continuous population [31] THR [32] registration) Footprint analysis Arthritis Knee OA [33] Gait 2 Six-step walking, Exercise capacity 1 6 min walking Distance 1 6 min 50 m corridor [30], 30 m carpeted level surface [2], stopwatch, measuring wheel 50 ft (17 m) walking Number of walking cycles per day Step length, stride width, stride length, Cadence (steps/min), gait velocity Accelerometry Hip arthroplasty Hip OA [34] Gait improvement 1 5 m walking Duration gait cycle, stance time (L/R), double support time (L/R) Locome trix Hospital out-patients Mechanical musculoskeletal disorders [35] a Gait analysis 1 OA patients listed for TKR [30] Gait-disorder related disability 1 40 m walking Cycle frequency, stride symmetry, stride regularity 1 4 days Pedometer 5 20 min Runway with carbon-paper, 50 ft (17 m) space 5 A few min 35 m walkway, two accelerometers, data logger, PC, software 3 Little time 40 m straight corridor, two accelerometers, recording device, PC, software Gait speed 1 10 m walking Gait speed 1 10 m walkway, two-dimensional biomechanical gait analysis system, footswitches Gait analysis 2 Knee OA Knee OA [36] Gait 1 8 m walking Walking speed, cadence (steps/min), mean stride length (all at normal and fast speed) 6 Quick 8 m electronic footswitch walkway, two micro-optical sensors, conditioning box, PC, software (Continued) Performance-based methods of physical function 893

5 TABLE 1. Continued Performance-based method Target population Study population Measurement concept Number of activities Activities Functional measures Gait analysis 3 Rehabilitation Hip/knee OA [37] Gait deviation 1 12 m walking Temporal asymmetry (stance difference), load asymmetry (difference in vertical impulse), force variability (S.D.), force discrepancy, extreme force discrepancy, stance time Gait analysis 4 TKR, Knee OA, after post-operative TKR [38] Stair climb and chair tests Stair ascent Hip OA, listed for TKR [30] 30 s stair climb Knee OA, undergoing TKR [2] Post-operative function Stair ascent cycle duration Functional capacity m walking (12 gait runs, 43 gait cycles) Velocity, cycle duration, step rate, stride length, stance phase, step length, weight acceptance time, max. vertical force, stance knee flexion, swing knee flexion, total knee flexion, single support 1 Four-step stair climb Stair ascent cycle duration 12-step stair climb for 30 s Get up and go Elderly [39] Knee OA [4] Physical function 1 Chair-rise with 50 ft (17 m) walking Multi-activity tests Madsen Knee OA Knee OA, listed for TKR [40] Disability 2 50 m walking 30-step stair climb Number of scores Time to administer Requirements 6 12 m corridor, two force plates, two photoelectric barriers, chronograph 12 Two 5 m force plates, optical switches 1 Four-step stair, two-dimensional biomechanical gait analysis system, six footswitches Number of steps 1 30 s 12-step stair Time 1 11 s Standard-height chair with 20 m corridor, stopwatch Time Time (up and down) 2 50 m space, stair 894 C. B. Terwee et al. Marks Knee OA Knee OA [41] Locomotor function 2 13 m walking 4-step stair climb ALF Knee OA Knee OA [42] Locomotor function 3 8 m walking 7-step stair climb Chair rise Time Time (up and down) Time (three times) Time (four times) Time (three times) 2 16 m carpeted walkway, four-step carpeted stair 3 and 1 total score 10 m space, seven-step stair, chair, stopwatch

6 Lin Hip/knee OA Symptomatic Physical function 3 8 ft (2.5 m) walking hip/knee OA [43] Four-step stair climb PAR Knee OA Knee OA [23] Physical activity restrictions ILAS Steultjens FAS Numact monitor DynaPort KneeTest Acute care, physical therapy after THR/TKR Hip/knee OA Lower extremities OA General population [48] Knee problems Patients undergoing THR/TKR [44] Hip/knee OA [5, 45] Hip OA, listed for THR [46], hip/knee OA, listed for THR/TKR [47] Knee OA [49, 50] TKR [51] Chair rise Joint flexibility Quadriceps strength 3 or 4 6 min walking 5- or 9-step stair climb Lift and carry 22 kg Get in/out car Function 5 15 ft (4.5 m) walking 3-step stair climb Stand from bed Get out of bed 13.4 m walking Disability in mobility Lower extremity dysfunction Spontaneous ambulatory activity Knee-related functional ability 5 1 min walking Sit down in chair Lie down on bench Bend and lift 2 kg 5 m walk 7 (subscale Raising from physical half-standing, disability) raising up/sitting down, step height, standing on one leg, stair climbing, gait speed, walking aid 4 Sitting, standing, lying, walking 29 Locomotion (8 items), rise and descend (10), lift and move (7), transfers (4) Time (faster of 2 trials) Time (up/down) (faster of 2 trials) Time (5 times) Degrees Nm Distance Time Time Time Level of assistance (0 6) Time (0 6) Guarding while walking (0 2) Rigidity while bending, reclining, sitting down (0 5) Stand-to-sit time Stand-to-recline time 5 m walking time Observed disability (5-point Likert scale) Time, number of steps, average amplitude of steps, longest continuous walk, peak 5-min energy, energy expenditure Accelerations (10 variables), angles (4), durations (11), other (5) 10 and 1 total score 3 or 4 and 1 total score 5 and 1 total score 5 and 1 total score 3 m space, four-step stair, chair, stopwatch, goniometer, myometer 25 min (four tasks) Walking space, five- or ninestep stair, movable shelves and 22 kg weight (mock-up car) min 13.4 m space, three-step stair, bed, trained observers 8 m space, chair, bench, 2 kg weight, video camera, stopwatch, trained observers 7 (subscale physical 65 m walkway, disability) or chair, platform 20 (total FAS) b with different step heights, stopwatch 6 24 h Numact monitor, data recorder, PC, software 29 þ 4 cluster scores þ1 total score 30 min Standardized equipment c, six accelerometers, data recorder, PC, software TKR, total knee replacement; THR, total hip replacement; ALF, aggregated locomotor function; PAR, physical activity restrictions; ILAS, Iowa Level of Assistance Scale; FAS, functional assessment system., no data presented. a 66% hip/knee OA. b Including five subscales: hip impairment, knee impairment, physical disability, social disability and pain. c Wooden stair, mock-up shopping trolley, wooden steps (three heights) and a slope, to be obtained from McRoberts BV. Performance-based methods of physical function 895

7 TABLE 2. Internal consistency and reproducibility Performance-based method Dimensions studied? Internal consistency Reproducibility Absolute measurement error Alpha Study size Reliability Design Time interval Study size Absolute measurement error Walking tests Self-paced walk 1 [3] NA ICC ¼ 0.83 Intra-rater 6 weeks 10 SEM ¼ No Self-paced walk 2 [25] NA ICC ¼ 0.78 Inter-session 1 week 10 ICC ¼ 0.97 Intra-session same day Self-paced walk 3 [26] NA ICC ¼ 0.98, 0.99 i Intra-session 2 minutes 15 SEM ¼ i 15 Yes ICC ¼ 0.80, 0.88 i Inter-session 1 week SEM ¼ i No Five-min walking field test [28] NA ICC ¼ 0.92 ( ) Intra-rater Max. 11 days 60 Six-min gait test [2, 30] NA Pedometer [32] NA Footprint analysis [33] NA ICC ¼ 0.70 a 42 SEM ¼ 1.85 a 42 No ICC ¼ 0.95 b SEM ¼ 2.68 b No Accelerometry [34] NA Locome trix [35] NA Gait analysis 1 [30] NA Gait analysis 2 [36] NA ICC ¼ g Inter-session 1 week 41 SEM ¼ g 41 3/6 yes ICC ¼ g Intra-session same day Gait analysis 3 [37] NA Gait analysis 4 [38] NA Stair climb and chair tests Stair ascent [30] NA 30 s stair climb [2] NA Get up and go [4] NA ICC ¼ 0.95 ( ) Intra-rater 2 min 25 SEM ¼ Yes ICC ¼ 0.98 ( ) Inter-rater 2 min 25 SEM ¼ Yes Multi-activity tests Madsen [40] NA CV ¼ 7 c 21 CV ¼ 4.9 d Marks [41] NA ICC ¼ 0.73 c, 0.89 d Intra-rater 1 week 12 SEM ¼ 1.05 c 12 No ICC ¼ 0.74 c, 0.85 d 6 weeks SEM ¼ 1.67 d No ALF [42] No ICC ¼ 0.99 ( ) Intra-rater 1 week 15 SEM ¼ Yes Lin [43] No ICC ¼ g Intra-rater SEM ¼ g Yes PAR [23] Yes 0.82 e 426 r ¼ g 2 weeks f 203 r ¼ g 3 months 148 ILAS [44] No K ¼ g Inter-rater Same day 86 SEM ¼ 2.15 h 55 7 Yes ICC ¼ 0.82 h Inter-rater Same day K ¼ g Intra-rater Video rating Steultjens [5, 45] Yes r ¼ 0.78 Inter-rater 186 r ¼ 0.90 Intra-rater FAS [47] Yes 105 G ¼ g Inter-rater 42 Numact monitor [49] NA DynaPort [51] KneeTest No Study size MID SDC<MID or 0.5 S.D. 896 C. B. Terwee et al. ALF, aggregated locomotor function; PAR, physical activity restrictions; ILAS, Iowa Level of Assistance Scale; FAS, Functional Assessment System; NA, not applicable; r ¼ correlation coefficient; K ¼ kappa; G ¼ Goodman Kruskal gamma coefficient; ICC, intraclass correlation coefficient; MID, minimal important difference; SDC, smallest detectable difference; SEM, standard error of measurement; S.D., standard deviation; CV, coefficient of variation., no data presented. a Stride width, b step length, c walking, d stair climb, e three activities, f four activities, g range, h total score, i measured twice.

8 TABLE 3. Construct validity and responsiveness Construct validity Performance-based method Hypotheses Main results Study size Hypotheses Treatment Walking tests Self-paced walk 1 [3] Self-paced walk 2 [25] Self-paced No r ¼ 0.66 with Lequesne s walk 3 [26] functional index Five-min [28] walking field test Six-min [2, 30] gait test Responsiveness Follow-up time Main results Study size No No No No 15 No No r ¼ 0.80 with VO 2 peak 46 No No No 4 months 4 months 4 months SRM ¼ 0.58 [30] ES ¼ 0.35 [30] Paired t-statistic ¼ 4.66 [30] 3 months Paired t-statistic ¼ 11.5 (P<0.001) [2] 65; 66 Pedometer [32] No r ¼ 0.66 with step 33 No TKR; TKR activity monitor Footprint [33] Yes r ¼ 0.43 gait 42 No analysis velocity height r ¼ 0.49 step length height R ¼ 0.02 stride width height r ¼ 0.51 stride width step length r ¼ 0.43 stride width gait velocity Accelerometry [34] No No THR 3, 6, 9 months Significant changes 12 Locome trix [35] No r ¼ with Lequesne s functional index; AUC for discriminating 20 22; 202 patients controls a No Gait analysis 1 [30] No No TKR 4 months SRM ¼ 0.28; ES ¼ 0.16; 65 paired t-statistic ¼ 2.23 Gait analysis 2 [36] No No Gait analysis 3 [37] Yes % difference in gait parameters from controls; s difference in stance time from controls 41 knee, 14 hip, 38 controls Yes Free to fast velocity Walking shod to barefoot Same day Same dayc % change in gait parameters % change in gait parameters Gait analysis 4 [38] No Significant correlations with clinical examination variables 21 No Stair climb and chair tests Stair ascent [30] No No TKR 4 months SRM ¼ 0.08, ES ¼ 0.05, paired t-statistic ¼ s stair climb [2] No No TKR 3 months Paired t-statistic ¼ 5.8 (P<0.001) Performance-based methods of physical function 897

9 Get up and go [4] Yes Significant difference between patients and controls; higher correlations with patient-reported measures of physical function than with other measures 50; 105 No Multi-activity tests Madsen [40] No No Marks [41] No No ALF [42] No r ¼ 0.59 with WOMAC 214 No Exercise provision 12-months SRM ¼ r ¼ 0.53 with SF-36 phys. fun. Lin [43] No r ¼ with WOMAC PAR [23] Yes r ¼ of individual tasks with treadmill time, VO 2peak, knee strength r ¼ of individual tasks with self-reported dysfunction ILAS [44] No r ¼ 0.86 total score with Harris hip score No No 65 No Acute post-ok 2 6 days Guyatt s responsiveness 55 care post operative index ¼ 0.75 Steultjens [5,45] Yes r ¼ with 198 Yes Exercise therapy 12 weeks r ¼ of change 186 self-reported mobility; two-factor structure (instead of expected three) scores with changes in self-reported mobility; different factor structure than expected FAS [46] No No THR 3 months SRM of mean score ¼ months SRM of mean score ¼ 0.7 Numact No r ¼ 0.15 with 57 No TKR 6 months ES ¼ monitor [49, 50] NHP mobility r ¼ 0.2 with patient s opinion 57 r ¼ 0.4 with physician s opinion 57 Significant difference between patients taking NSAIDs and patients awaiting TKR 28 DynaPort KneeTest [51] No No TKR 3, 6, 12, 24 months ES ¼ 0.3, 0.9, 1.4, b THR, total hip replacement; TKR, total knee replacement; ALF, Aggregated Locomotor Function; PAR, physical activity restrictions; ILAS, Iowa Level of Assistance Scale; FAS, Functional Assessment System; WOMAC, Western Ontario and McMaster Universities Osteoarthritis Index (physical functioning subscale); NHP, Nottingham Health Profile; r ¼correlation coefficient; AUC, area under the (ROC) curve; SRM, standardized response mean; ES, effect size; NSAIDs, non-steroidal anti-inflammatory drugs., no data presented. a 42/63 ¼ 66% hip/knee OA. b Not all patients were measured at all time points. 898 C. B. Terwee et al.

10 TABLE 4. Summary of the evaluation of the quality and outcomes of the clinimetric studies Performance-based method Internal consistency Reliability Absolute measurement error MID Construct validity Floor/ceiling effect Responsiveness Interpretability Walking tests Self-paced walk 1 [3] NA?? 0 0 NA 0 0 Self-paced walk 2 [25] NA? NA 0 0 Self-paced walk 3 [26] NA?? 0? NA 0 0 Five-minute walking field test [28] NA þ 0 0? NA 0 0 Six-minute [2, 30] gait test NA NA?? Pedometer [32] NA 0 0 0? NA 0 0 Footprint analysis [33] NA?? 0? NA 0 0 Accelerometry [34] NA NA? 0 Locome trix [35] NA 0 0 0? NA 0? Gait analysis 1 [30] NA NA?? Gait analysis 2 [36] NA þ? 0 0 NA 0 0 Gait analysis 3 [37] NA 0 0 0? NA? 0 Gait analysis 4 [38] NA 0 0 0? NA 0? Stair climb and chair tests Stair ascent [30] NA NA?? 30 s stair climb [2] NA NA? 0 Get up and go [4] NA þ? 0 þ NA 0 0 Multi-activities tests Madsen [40] NA 0? 0 0 NA 0 0 Marks [41] NA?? 0 0 NA 0 0 ALF [42] 0 þ? 0? NA? 0 Lin [43]??? 0? PAR [23] þ? 0 0 þ ILAS [44] 0 þ a þ þ? 0? 0 Steultjens [5, 45] þ? 0 0 0? FAS [47]?? ? 0 Numact monitor [49, 50] NA 0 0 0? NA? 0 DynaPort [51] KneeTest ? 0 þ, positive rating;, negative rating;?, indeterminate rating; 0, no information available. a For inter-rater reliability of the total score (inter- and intra-rater reliability of the individual items score?). Performance-based methods of physical function 899

11 900 C. B. Terwee et al. studies or non-optimal analyses. The multi-activity tests were most extensively evaluated. The ILAS received the best ratings. There are no standardized criteria to evaluate the quality of performance-based methods. Some of our criteria are arbitrary and may be interpreted as strict, but we did not want to base our conclusions on small studies with unreliable results. Furthermore, when hypotheses are not specified in validity or responsiveness studies, the risk of bias is large, because often only the positive results will be presented. If authors disagree with our criteria, they can make their own judgement based on the results presented in Tables 2 and 3. Some limitations of our study should be acknowledged. We only included information on measurement properties that was intentionally calculated to assess the measurement properties of the particular performance-based method. More evidence is probably available in the literature that could be used to determine the validity or responsiveness of the tests, e.g. in studies that compared self-reported questionnaires and performance-based tests or in studies that evaluated self-report questionnaires by using a performance-based method as an external standard. We did not include these articles because without specified hypotheses about the expected relationship of the performance-based test with the other measures used, the results of these studies are difficult to interpret. Furthermore, we included only English language publications, and we therefore might have missed some publications on additional performance-based methods or measurement properties. Several performance-based tests have been evaluated for their measurement properties in a general ( healthy ) population or in other patient populations. These studies were excluded because the measurement properties of a given instrument depend on the setting and the population in which it is assessed. Therefore, the results found in these studies may not be generalizable to patients with OA of the hip or knee. With the currently available evidence it is extremely difficult to formulate recommendations for which instrument to choose. Many performance-based tests, especially the walking tests, represent minor variations on the same theme, but we have no idea which might be the most useful because a proper justification of the choice of the activities included in the test and the aspects of function that were measured is mostly lacking. The PAR is the only test for which a proper justification for the choice of the activities was provided [23]. The PAR also received a positive rating for internal consistency. We therefore consider this test to be a good potential candidate for further testing. The reproducibility has not been studied adequately yet, and the responsiveness of the test has not been studied at all. The ILAS received the best ratings. However, a proper justification of the content of the test is lacking and internal consistency of this test has not been evaluated. In our opinion, multi-activity tests are more valid for measuring the physical function of patients with OA of the hip or knee than single-activity tests, such as walking tests, because patients with OA of the hip or knee experience functional problems in more activities than just walking. For example, in patients with mild OA walking may be unaffected, but these patients may have problems with climbing stairs or standing up from a chair. In addition, time alone may be inadequate to represent the concept of physical function [24]. Tests that combine aspects of physical function with aspects of impairment, e.g. joint flexibility or muscle strength (e.g. Lin s test), are difficult to interpret because they measure multiple different underlying constructs [activity limitations and impairments according to the International Classification of Functioning (ICF)]. As the content of the multi-activity test are much alike, further testing of ALF, ILAS, Steultjens test, FAS or the DynaPort KneeTest could also be considered. More importantly, however, before one can make a justified choice of a particular performance-based method, consensus is needed on what activities should be included in a performancebased test for patients with OA of the hip or knee and which aspects of function (e.g. time and other aspects of quality of movement) should be measured. Rheumatology The authors have declared no conflicts of interest. References Key messages We identified 26 performance-based tests; many represent variations on the same theme. We do not know which might be the most useful because consensus is lacking on what activities and functional parameters should be included. Data on reproducibility and validity are mostly lacking. Many more welldesigned clinimetric studies are needed. 1. Bellamy N, Kirwan J, Boers M et al. Recommendations for a core set of outcome measures for future phase III trials in knee, hip, and hand osteoarthritis. Consensus development at OMERACT III. J Rheumatol 1997;24: Kreibich DN, Vaz M, Bourne RB et al. What is the best way of assessing outcome after total knee replacement? Clin Orthop 1996: Marks R. Walking time measures for evaluating OA of the knee. Physiotherapy 1994;50: Piva SR, Fitzgerald GK, Irrgang JJ, Bouzubar F, Starz TW. Get up and go test in patients with knee osteoarthritis. Arch Phys Med Rehabil 2004;85: Steultjens MPM, Dekker J, van Baar ME, Oostendorp RAB, Bijlsma JWJ. Internal consistency and validity of an observational method for assessing disability in mobility in patients with osteoarthritis. Arthritis Care Res 1999;12: Odding E, Valkenburg HA, Stam HJ, Hofman A. Assessing joint pain complaints and locomotor disability in the Rotterdam Study: effect of population selection and assessment mode. Arch Phys Med Rehabil 2000;81: Birrell F, Johnell O, Silman A. Projecting the need for hip replacement over the next three decades: influence of changing demography and threshold for surgery. Ann Rheum Dis 1999;58: Bot SDM, Terwee CB, van der Windt DAWM, Bouter LM, Dekker J, de Vet HCW. Psychometric evaluation of self-report questionnaires the development of a checklist. In: Ader HJ, Mellenbergh GJ, ed. Second workshop on research methodology, June, Amsterdam: VU University Amsterdam 2003: Lohr KN, Aaronson NK, Alonso J et al. Evaluating quality of life and health status instruments: development of scientific review criteria. Clin Ther 1996;18: Bombardier C, Tugwell P. Methodological considerations in functional assessment. J Rheumatol 1987;14(Suppl. 15): Altman DG. Practical statistics for medical research. London: Chapman and Hall, de Vet HCW. Observer reliability and agreement. In: Armitage P, Colton T, eds. Encyclopedia of biostatistics. New York: John Wiley, 1998: Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Controlled Clinical Trials 1991;12: 142S 58S. 14. Nunnally JC, Bernstein IH. Psychometric theory, 3rd edn. New York: McGraw-Hill, Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986: Down

12 Performance-based methods of physical function Ravaud P, Giraudeau B, Auleley GR, Edouard-Noël R, Dougados M, Chastang Cl. Assessing smallest detectable change over time in continuous structural outcome measures: application to radiological change in knee osteoarthritis. J Clin Epidemiol 1999;52: Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life. The remarkable universality of half a standard deviation. Med Care 2003;41: de Vet HCW, Ader HJ, Terwee CB, Pouwer F. Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36. Qual Life Res 2005;14: McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 1995;4: Testa MA, Simonson DC. Assessment of quality of life outcomes. N Engl J Med 1996;334: Terwee CB, Dekker FW, Wiersinga WM, Prummel MF, Bossuyt PMM. On assessing responsiveness of health-related quality of life instruments: guidelines for instrument evaluation. Qual Life Res 2003;12: Hays RD, Hadorn D. Responsiveness to change: an aspect of validity, not a separate dimension. Qual Life Res 1992;1: Rejeski WJ, Ettinger WH Jr, Schumaker S, James P, Burns R, Elam JT. Assessing performance-related disability in patients with knee osteoarthritis. Osteoarthritis Cartilage 1995;3: Stratford PW, Kennedy D, Pagura SMC, Gollish JD. The relationship between self-report and performance-related measures: questioning the content validity of timed tests. Arthritis Rheum 2003; 49: Marks R, Quinney HA, Wessel J. Proprioceptive sensibility in women with normal and osteoarthritic knee joints. Clin Rheumatol 1993; 12: Marks R. Reliability and validity of self-paced walking time measures for knee osteoarthritis. Arthritis Care Res 1994;7: Price LG, Hewett JE, Kay DR, Minor MA. Five-minute walking test of aerobic fitness for people with arthritis. Arthritis Care Res 1988;1: Péloquin L, Gauthier P, Bravo G, Lacombe G, Billiard JS. Reliability and validity of the five-minute walking field test for estimating VO2 peak in elderly subjects with knee osteoarthritis. J Aging Phys Act 1998;6: Guyatt GH, Sullivan MJ, Thompson PJ et al. The 6-minute walk: a new measure of exercise capacity in patients with chronic heart failure. J Can Med Assoc 1985;132: Parent E, Moffet H. Comparative responsiveness of locomotor tests and questionnaires used to follow early recovery after total knee replacement. Arch Phys Med Rehabil 2002;83: Bassey EJ, Dallosso HM, Fentem PH, Irving JM, Patrick JM. Validation of a simple mechanical accelerometer (pedometer) for the estimation of walking activity. Eur J Appl Physiol Occup Physiol 1987;56: Silva M, Shepherd EF, Jackson WO, Dorey FJ, Schmalzried TP. Average patient walking activity approaches 2 million cycles per year. J Arthroplasty 2002;17: Falconer J, Hayes KW. A simple method to measure gait for use in arthritis clinical research. Arthritis Care Res 1991;4: Aminian K, Rezakhanlou K, De Andres E, Fritsch C, Leyvraz PF, Robert P. Temporal feature estimation during walking using miniature accelerometers: an analysis of gait improvement after hip arthroplasty. Med Biol Eng Comput 1999;37: Auvinet B, Chaleil D, Barrey E. Accelerometric gait analysis for use in hospital outpatients. Rev Rhum Engl Ed 1999;66: Fransen M, Crosbie J, Edmonds J. Reliability of gait measurements in people with osteoarthritis of the knee. Phys Ther 1997;77: Lafuente R, Belda JM, Sanchez-Lacuesta J, Soler C, Poveda R, Prat J. Quantitative assessment of gait deviation: contribution to the objective measurement of disability. Gait Posture 2000;11: Olsson E, Barck A. Correlation between clinical examination and quantitative gait analysis in patients operated upon with the Gunston-Hult knee prosthesis. Scand J Rehabil Med 1986;18: Mathias S, Nayak USL, Isaacs B. Balance in elderly patients: the Get-up and Go test. Arch Phys Med Rehabil 1986;67: Madsen OR, Brot C. Assessment of extensor and flexor strength in the individual gonarthrotic patient: interpretation of performance changes. Clin Rheumatol 1996;15: Marks R. Pilot study of the reproducibility of measures of walking performance variables in persons with osteoarthritis of the knee. Physiother Can 1995;47: McCarthy CJ, Oldham JA. The reliability, validity and responsiveness of an aggregated locomotor function (ALF) score in patients with osteoarthritis of the knee. Rheumatology 2004;43: Lin YC, Davey RC, Cochrane T. Tests for physical function of the elderly with knee and hip osteoarthritis. Scand J Med Sci Sports 2001;11: Shields RK, Enloe LJ, Evans RE, Smith KB, Steckel SD. Reliability, validity, and responsiveness of functional tests in patients with total joint replacement. Phys Ther 1995;75: (discussion 76 9). 45. Steultjens MPM, Roorda LD, Dekker J, Bijlsma JWJ. Responsiveness of observational and self-report methods for assessing disability in mobility in patients with osteoarthritis. Arthritis Care Res 2001; 45: Nilsdotter AK, Roos EM, Westerlund JP, Roos HP, Lohmander LS. Comparative responsiveness of measures of pain and function after total hip replacement. Arthritis Care Res 2001;45: Oberg U, Oberg B, Oberg T. Validity and reliability of a new assessment of lower-extremity dysfunction. Phys Ther 1994; 74: Walker DJ, Heslop PS, Plummer CJ, Essex T, Chandler S. A continuous patient activity monitor: validation and relation to disability. Physiol Meas 1997;18: Walker DJ, Kidd E, Heslop PS, Chandler C. Spontaneous ambulatory activity as a quantifiable outcome measure for rheumatoid arthritis. Rheumatology 1999;38: Walker DJ, Heslop PS, Chandler C, Pinder IM. Measured ambulation and self-reported health status following total joint replacement for the osteoarthritic knee. Rheumatology 2002;41: van den Dikkenberg N, Meijer OG, van der Slikke RMA et al. Measuring quality of movement in patients with knee problems: rationale and construction of the DynaPort KneeTest. Knee Surg Sports Traumatol Arthrosc 2002;10: Appendix 1. Full search strategy (((knee OR hip) AND (osteoarthritis OR replacement)) OR TKR OR TKA OR THR OR THA) AND ( performance test OR performance tests OR performancebased test OR performance-based tests OR performance instrument OR performance instruments OR performancebased instrument OR performance-based instruments OR performance-based method OR performance-based methods OR performance measure OR performance measures OR performance-based measure OR performance-based measures OR performance index OR performance indices OR performance-based index OR performance-based indices OR performance-based assessment OR objective test OR objective tests OR objective testing OR objective instrument OR objective instruments OR objective method OR objective methods OR objective measure OR objective measures OR objective evaluation OR objective function OR objective functional OR objective functioning OR objective disability OR objective assessment OR objective assessments OR observational test OR observational tests OR observation-based test OR observationbased tests OR observation-based testing OR observational testing OR observational instrument OR observational Down

13 902 C. B. Terwee et al. instruments OR observation-based instrument OR observationbased instruments OR observational method OR observational methods OR observation-based method OR observation-based methods OR observational measure OR observational measures OR observation-based measure OR observation-based measures OR observational index OR observational indices OR observation-based index OR observation-based indices OR observed disability OR observed functioning OR gait OR stair OR chair OR walk OR walking OR gait analysis OR gait analyses OR kinetic parameter OR kinematic parameter OR kinematic analysis OR kinematic analyses OR gait evaluation ) Limit: English, human Search date: 13 April 2004 Hits: Appendix 2. Checklist for the evaluation of performance-based methods Property Definition Quality criteria a,b Internal consistency Agreement Reliability Construct validity Responsiveness Interpretability The extent to which items in a (sub)scale are intercorrelated, thus measuring the same construct The extent to which the scores on repeated measures are close to each other (absolute measurement error) The extent to which patients can be distinguished from each other, despite measurement errors (relative measurement error) The extent to which scores on a particular instrument relate to other measures in a manner that is consistent with theoretically derived hypotheses concerning the concepts that are being measured The instrument s ability to detect important change over time in the concept being measured The degree to which one can assign qualitative meaning to quantitative scores þ, factor analyses performed on adequate sample size (7 no of items) AND Cronbach s alpha(s) calculated per dimension in a sample size of at least 50 patients AND Cronbach s alpha(s) 0.70?, no factor analysis OR doubtful design or method OR sample size too small, Cronbach s alpha(s) <0.70, despite adequate design and method 0, no information found on internal consistency þ, (MIC OR 0.5 S.D.) >SDC OR (MIC OR 0.5 S.D.) outside the LOA AND SDC and MID both determined in a sample size of at least 50 patients?, doubtful design or method or sample size <50, (MIC OR 0.5 S.D.) SDC OR (MIC OR 0.5 S.D.) inside LOA, despite adequate design and method 0, no information found on agreement þ, ICC or kappa >0.70 with the lower limit of the confidence interval >0.60 or a sample size of at least 50 patients?, doubtful design or method (e.g. time interval not mentioned, Pearson correlation) OR ICC or kappa >0.70 with the lower limit of the confidence interval 0.60 or sample size <50, ICC or kappa <0.70, despite adequate design and method 0, no information found on reliability þ, specific hypotheses were formulated AND at least 75% of the results are in accordance with these hypotheses in a sample size of at least 50 patients?, doubtful design or method (e.g. no hypotheses) OR sample size <50, less than 75% of hypotheses were confirmed, despite adequate design and methods 0, no information found on construct validity þ, specific hypotheses were formulated AND at least 75% of the result are in accordance with these hypotheses in a sample size of at least 50 patients?, doubtful design or method (e.g. no hypotheses) OR sample size <50, less than 75% of hypotheses were confirmed, despite adequate design and methods 0, no information found on responsiveness þ, mean and S.D. scores presented of at least two relevant subgroups of patients in a sample size of at least 50 patients?, doubtful design or method OR less than two subgroups OR sample size <50 0, no information found on interpretation Down ICC, intraclass correlation coefficient; MIC, minimal important change; SDC, smallest detectable change; LOA, limits of agreement; S.D., standard deviation. a þ, positive rating;, negative rating;?, indeterminate rating; 0, no information available. b Doubtful design or method ¼ lacking a clear description of the design or methods of the clinimetric study, or any important methodological weakness in the design or execution of the study.