IT IS WELL DOCUMENTED that long-term use of wheelchairs

Transcription

1 ORIGINAL ARTICLE Test-Retest Reliability and Minimal Detectable Change of the SmartWheel Clinical Protocol Jordon Lui, BKin, Megan K. MacGillivray, MSc, Bonita J. Sawatzky, PhD 2367 ABSTRACT. Lui J, MacGillivray MK, Sawatzky BJ. Test-retest reliability and minimal detectable change of the SmartWheel clinical protocol. Arch Phys Med Rehabil 2012;93: Objectives: (1) To quantify the intra- and intersession reliability and minimal detectable change (MDC) of the SmartWheel clinical protocol (SCP) and (2) to compare the reliability of the SCP between experienced and naïve wheelchair users. Design: Test-retest study. Setting: Biomechanics laboratory. Participants: Manual wheelchair users (WCUs) (n 10) with 1 to 32 years of wheeling experience and able-bodied (AB) naïve users (n 15). Interventions: Not applicable. Main Outcome Measures: Wheelchair propulsion parameters including average peak force, push frequency, push length, and velocity were measured according to the SCP but with 5 trials per session, for 2 sessions. WCUs and AB users were analyzed separately. Intraclass correlation coefficients ([ICC] 2,1 and ICC 2,5 ) were calculated to assess intrasession reliability. ICC 2,1 and MDC (with 95% confidence) were calculated for each SCP parameter using a single trial from each session and with the mean of 5 repeated measures to evaluate intersession reliability. Results: Intra- and intersession reliability for WCU parameters ranged from high to very high correlation (ICC range,.70.99). For AB parameters, intrasession ICC 2,1 ranged from moderate to very high (ICC range,.50.92), while intersession ICC 2,1 indicated low to very high correlation (ICC range,.25.90). Estimates of standard error of measurement and MDC were provided for each parameter. For both WCUs and AB users, using the means from 5 trials increased intra- and intersession ICC and decreased MDC values. All MDC values were lower for WCUs compared with AB users. Conclusions: The SCP is a reliable method for assessing propulsion parameters in WCUs, even if just 1 trial is taken per session. AB users showed lower intra- and intersession reliability compared with WCUs. Therefore, for AB users or individuals with minimal wheeling experience, averaging multiple trials is recommended for the SCP. Key Words: Rehabilitation; Reproducibility of results; Wheelchairs. From the School of Kinesiology (Lui) and the Departments of Rehabilitation Sciences (MacGillivray) and Orthopaedics (Sawatzky), University of British Columbia, Vancouver; and the International Collaboration on Repair Discoveries, Vancouver (Lui, MacGillivray, Sawatzky), Canada. Supported by the Natural Sciences and Engineering Research Council (grant no. RGPIN ). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated. Correspondence to Bonita J. Sawatzky, PhD, International Collaboration on Repair Discoveries, 818 W 10th Ave, Vancouver, BC, Canada, bonitas@mail.ubc.ca. Reprints are not available from the author. In-press corrected proof published online on Sep 14, 2012, at /12/ $36.00/ by the American Congress of Rehabilitation Medicine IT IS WELL DOCUMENTED that long-term use of wheelchairs is associated with repetitive strain injuries to the upper extremities. 1-3 Wheelchair configurations and interventions designed to modify propulsion strategies can effectively reduce the forces needed to propel a wheelchair and therefore reduce the load on the shoulders and wrists. 3-6 To better understand manual wheelchair propulsion, the SmartWheel a was developed to allow clinicians and researchers to objectively assess the forces and moments applied to the pushrim. 7 The Smart- Wheel is a specialized wheelchair wheel that measures propulsion parameters, such as the three-dimensional forces applied to the pushrim, push length, push frequency, and velocity during wheeling. 7 The SmartWheel is used in conjunction with the SmartWheel clinical software to assist clinicians in assessing these propulsion parameters. The SmartWheel can provide clinicians with objective data needed to prescribe the most appropriate wheelchair configurations or to implement and track the effectiveness of their interventions. The SmartWheel users group, an international network of researchers and clinicians, developed a SmartWheel clinical protocol (SCP), which was designed to accommodate a clinic s availability of space and time. 8 In the SCP, patients are instructed to wheel at a self-selected speed for a maximum of 10 seconds (or 10m) on 3 different surfaces (tile, 5% grade ramp, and carpet) while the SmartWheel software generates data on 4 key measurements: velocity, average peak force, push frequency, and push length. 8 This provides 12 SCP parameters (3 surfaces 4 measurements), which are considered the most clinically relevant and are used to help guide wheelchair prescription and/or training. 8 Accordingly, velocity, average peak force, push frequency, and push length are important measures that clinicians can modify in order to improve the performance or efficiency of manual wheelchair propulsion. 9 Therefore, it is important that these measurements are reliably determined by the SCP. It is also important to estimate the minimal detectable change (MDC) of the protocol. This would allow clinicians to distinguish true variability from measurement error and to estimate the minimal difference that can be detected confidently between sessions. 10 In addition, the concept of reliability depends highly on the population being measured. 11 Therefore, reliability should be evaluated on a sample that most closely resembles the intended AB ICC MDC MSE SCP WCU List of Abbreviations able-bodied intraclass correlation coefficient minimal detectable change mean squared error SmartWheel clinical protocol wheelchair user

2 2368 RELIABILITY OF THE SMARTWHEEL CLINICAL PROTOCOL, Lui target population. 11 In wheelchair users (WCUs), experience has been shown to affect propulsion technique and consistency This could mean that the reliability of the SCP may be dependent on general wheelchair experience and therefore should be evaluated accordingly. To date, no studies have reported the test-retest reliability or the MDC of the SCP. Therefore, the aims of the study were to: (1) examine the intra- and intersession reliability of the SCP; (2) quantify the MDC for each of the SCP parameters; and (3) compare the intra- and intersession reliability between experienced and naïve WCUs. METHODS Participants Based on the methodology of Walter et al 16 for calculating sample size for reliability studies, it was determined that a minimum sample size of 9 was needed. Ten spinal cord injured manual WCUs with at least 1 year of wheeling experience were recruited through advertisements posted at an outpatient rehabilitation clinic. Only WCUs whose wheelchairs were equipped with 61.0 or 63.5cm (24 or 25in). wheels, quick release axles, and pneumatic high-pressure tires were included in the study. Fifteen able-bodied (AB) individuals were also included and analyzed separately from the WCUs. The AB individuals were comprised of a convenience sample to represent individuals who are naïve to manual wheeling. Therefore, they are to some extent comparable with newly injured individuals with intact upper body function. Inclusion criteria for all participants included being between 18 to 65 years of age, and being able to use a manual wheelchair without pain. Individuals who had muscular or cardiorespiratory illnesses were excluded from this study. The study received approval from the university s clinical research ethics board. All participants provided written informed consent. Study Design The study used a test-retest design. All participants were evaluated with the same protocol twice, with approximately 7 days between testing sessions. Procedure WCUs used their own wheelchairs for the study, and AB individuals were provided with appropriately sized (either 40.6 or 43.2cm [16 or 17in]) Elevation b wheelchairs that allowed for seat height and dump adjustments via a built in hydraulic spring system. Individual wheelchair configurations were kept consistent between sessions. The SmartWheel was affixed to the wheelchair on the side of the participant s dominant arm while the wheelchair s standard wheel remained on the nondominant side. Pneumatic high-pressure tires were used on all wheels. All tires were inflated to manufacturer s specifications. Once the SmartWheel was attached, all participants were given time to familiarize themselves with their wheelchairs and the different test surfaces prior to testing. The testing orders of the 3 surfaces were randomized. Adequate rest was provided at any time (at participant s request) during testing in order to minimize fatigue. Protocol Following the SCP, participants were instructed to wheel across 3 different surfaces: tile, ramp, and carpet. Data were collected in accordance with the SCP, but with additional trials for each surface. The current SCP does not specify the number of trials that should be collected. However, a total of 5 trials were collected for each surface in order to (1) evaluate the intrasession reliability and (2) evaluate the effect of averaging multiple trials on the intersession reliability. As defined by the SCP for the tile surface, wheeling was done on smooth, level tile. Participants accelerated from a stationary position up to a comfortable, self-selected speed and wheeled continuously for 10 seconds or 10 meters, whichever occurred first. Collection started as soon as the SmartWheel pushrim was contacted and participants were told to keep wheeling at a normal speed until instructed to stop. The SCP was administered for 5 consecutive trials. The ramp and carpet protocols followed the same procedure as the tile protocol, but with a 5% grade ramp and a low-pile carpet, respectively. Measures Four main measurements were computed with the SmartWheel clinical software for each of the 3 surfaces: average peak force (N/kg), average push frequency (Hz), average push length ( ), and average velocity (m/s). These are the 12 SCP parameters currently used by clinicians to assess their patients. Calculations for each parameter were based on the data collected from all pushes except for the first 3. This was done to capture the participants propulsion when they are assumed to have completed the bulk of their acceleration. 8 Statistical Analysis For each parameter, the intrasession reliability was calculated using intraclass correlation coefficients (ICCs): ICC 2,1 (single measures) and ICC 2,5 (average measures of 5 trials). 17 Both were based on a 2-way (random effects) repeated-measures analysis of variance model with absolute agreement. For each session, intrasession paired t tests were calculated between the first and fifth trials for each parameter in order to detect potential learning effects. The alpha level of.05 was adjusted using a Bonferroni correction for 12 comparisons. The intersession reliability was calculated using ICC 2,1 with the first trial of each session for each parameter. In addition, to ensure that stable measures were obtained for each wheeling parameter, the mean of 5 trials was also calculated for each parameter of each session. ICC 2,1 for intersession reliability was then calculated using the mean parameter values of each session. Although there are many ways to interpret ICC values, we chose Munro s classification of reliability coefficients:.26 to.49 reflects low correlation;.50 to.69 reflects moderate correlation;.70 to.89 reflects high correlation; and.90 to 1.00 indicates very high correlation. 18 Standard error of measurement was calculated to assess the measurement error across sessions because of variability. Standard error of measurement is an index of absolute reliability; therefore, it estimates the precision of individual scores. 17 The standard error of measurement was estimated using the mean squared error (MSE) from a 2-way analysis of variance, where standard error of measurement MSE. This formulation has the advantage of being independent of the ICC values, and therefore is unaffected by the range of measurement values. 17 To estimate the minimal amount of change needed to exceed the measurement error, the MDC for each SCP parameter was estimated based on a 95% confidence interval, where MDC standard error of measurement Bland-Altman plots were constructed for each parameter by plotting the mean difference between session measurements (using the mean values of 5 trials) against the mean of the 2 test sessions for each participant. 19 Ninety-five percent confidence intervals (limits of agreement) were calculated to determine whether there were any systematic biases between sessions. All analyses were done using SPSS c and Microsoft Excel. d

3 RELIABILITY OF THE SMARTWHEEL CLINICAL PROTOCOL, Lui 2369 Table 1: Means SDs for SCP Parameters AB Individuals WCU SCP Parameter Session 1 Session 2 Session 1 Session 2 Tile Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Ramp Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Carpet Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) NOTE. Values are mean SD. RESULTS WCU Group A total of 7 men and 3 women WCUs were recruited with a mean age of years. Eight of the participants were persons with paraplegia and 2 were persons with tetraplegia. Wheeling experience ranged from 1 to 32 years, with a mean of 16.5 years. Table 1 shows the means (across all trials) and SDs of all parameters for each session. ICC values for intra- and intersession reliability are shown in table 2. For WCUs, intrasession correlation was high for 7 of 12 SCP parameters and very high for the other 5 parameters. As expected, using ICC 2,5 produced higher ratings; all parameters indicated very high correlation. Paired t tests showed no significant differences between the first and fifth trial for all parameters. For intersession reliability, based on the first trials of each session, most parameters had ICC 2,1 values that reflected high correlation (see table 2). Seven of 12 SCP parameters had high correlations and 5 had very high correlations (frequency, push length, and velocity on ramp, as well as force and velocity on carpet). ICC 2,1 based on the mean of 5 trials resulted in very high correlations for 10 of 12 SCP parameters, while 2 parameters had high correlations (tile force and frequency). Standard error of measurement and MDC values at the 95% confidence level are provided (see table 2). Standard error of measurement and MDC values based on the mean of 5 repeated measures were lower for all parameters compared with using only the first trials. Bland-Altman plots revealed no systematic biases between sessions, for all parameters (fig 1). AB Group Ten men and 5 women were recruited, with a mean age of years. All 15 AB individuals had no prior wheeling experience. Table 1 shows the test and retest means and SD for all parameters. Using ICC 2,1, intrasession correlation was moderate for 4 of 12 SCP parameters; 6 of 12 parameters reflected high correlation; and 2 of 12 showed very high correlations (table 3). Table 2: Intrasession and ICC, Standard Error of Measurement, and MDC Values for the SCP in WCU Intrasession (1st trials from each session) (mean of 5 trials) SCP Parameter ICC 2,1 ICC 2,5 ICC 2,1 SEM MDC ICC 2,1 SEM MDC Tile Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Ramp Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Carpet Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Abbreviation: SEM, standard error of measurement.

4 2370 RELIABILITY OF THE SMARTWHEEL CLINICAL PROTOCOL, Lui Fig 1. Sample Bland-Altman plots for WCUs: average peak force, push length, push frequency, and velocity. Mean differences (using the mean of 5 trials) between sessions are plotted against the mean of the 2 test sessions for each participant. ICC 2,5 values reflected much higher intrasession reliability: 11 of 12 SCP parameters reflected very high intrasession correlation, and 1 of 12 indicated high correlation (see table 3). Paired t tests showed no significant differences between the first and fifth trial for all parameters. Based on the first trials of each session, intersession ICC 2,1 indicated that 6 of 12 SCP parameters had low correlation; 2 of 12 indicated moderate correlation; 3 of 12 had high correlation; and 1 had very high correlation. ICC 2,1 values based on the mean of 5 repeated measures increased reliability for most parameters. Six of 12 showed high correlation; 3 of 12 showed moderate correlation; and 3 of 12 showed low correlation. Standard error of measurement and MDC values at the 95% confidence level are provided in table 3. Similar to WCUs, standard error of measurement and MDC values based on the average of 5 repeated measures were lower for all SCP parameters. Bland-Altman plots revealed no systematic biases between sessions, for all parameters (see fig 1). DISCUSSION Previous studies have examined the intra- and intersession reliability of clinical protocols and measurements in gait and manual wheeling Not all of the protocols in these studies have been found to be reliable. 20,22 Therefore, characterizing the reliability of protocols is well advised. Using similar methodology, to the best of our knowledge this was the first study to examine the reliability of the SCP. The observed ICC 2,1 values in this study show excellent intrasession reliability for all SCP parameters in the WCU group. As expected, averaging 5 trials further increased the intrasession reliability for all parameters. This occurs because averaging the results of multiple trials decreases the influence of the error variance compared with the true score. 26 The significance of using averages of multiple trials is especially illustrated in the intrasession reliability for the AB group. Using ICC 2,1, only 8 parameters had values that indicated high or very high intrasession reliability. When using the average of 5 trials (ICC 2,5 ), all SCP parameters showed very high or high intrasession reliability. This highlights the importance of using multiple trials to obtain stable intrasession measures, especially in individuals with no prior wheeling experience (eg, injured individuals getting their first chair). In both AB individuals and WCUs, paired t tests showed no significant differences between the first and fifth trials for all

5 RELIABILITY OF THE SMARTWHEEL CLINICAL PROTOCOL, Lui 2371 Table 3: Intra- and ICC, Standard Error of Measurement, and MDC Values for the SCP in AB Individuals Intrasession (first trials from each session) (mean of 5 trials) SCP Parameter ICC 2,1 ICC 2,5 ICC 2,1 SEM MDC ICC 2,1 SEM MDC Tile Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Ramp Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Carpet Force (N/kg) Frequency (Hz) Push length ( ) Velocity (m/s) Abbreviation: SEM, standard error of measurement. parameters. This means that a learning effect or fatigue was unlikely within each session. WCUs showed excellent intersession reliability between the first trials of each session and between the mean of 5 trials, whereas AB individuals showed poor intersession reliability when comparing just the first trials of each session. This was expected because of the large intrasession variability in AB individuals, as indicated by the low intrasession ICC 2,1 values when averages of trials were not used. By taking the mean of 5 trials, ICC 2,1 values for intersession reliability increased. For AB individuals, however, ICC 2,1 based on mean session values for push force and push length on tile, force on ramp, and frequency on carpet still remained low, with unacceptable values that indicated low intersession reliability. As shown by the Bland-Altman plots, no learning effects or systematic differences were observed between sessions for either WCUs or AB individuals when the means of 5 trials were used. Differences in wheeling variability between AB individuals and WCUs illustrate the importance of manual wheeling experience. The effect of practice has been shown to improve propulsion technique and coordination. 27 However, for individuals with no prior wheeling experience, de Groot et al 15 showed that a 12-minute practice session is inadequate for familiarization and developing consistency in timing variables such as push frequency. They observed no improvements in interstroke consistency after a 12-minute practice period or during 3 weeks of practice. 14,15 This could be indicative of inexperienced users exploring various propulsion strategies because of the novel task. 14,15 In our study a typical familiarization period ranged from 10 to 15 minutes, similar to the de Groot study. 15 This shows the importance of sufficient wheeling experience in order to decrease variability in wheeling and may partly explain the large variability observed in AB individuals compared with WCUs in the current study. In general, WCUs showed higher intra- and intersession reliability compared with AB individuals. For WCUs, if there is only time to do 1 trial per surface within a given clinical situation, the ICC values for intra- and intersession reliability showed acceptable values for all parameters of the SCP using only the first trial of each session. Thus, it may be clinically acceptable to conduct only 1 trial per surface for experienced WCUs. However, for AB individuals, averaging the 5 trials of each session is needed to increase both the intra- and intersession reliability of the SCP to a more acceptable level. Therefore, for inexperienced or new WCUs, averaging multiple trials is highly recommended. For the parameters that still indicate poor reliability despite using means, MDC values may provide a more helpful guide for clinical judgments. ICC is a measure of relative reliability. Therefore, low levels of intersubject variability will lower ICC values even if differences between test sessions are small. 18 Unlike ICCs, standard error of measurement and MDC are measures of absolute reliability. MDC values could be more useful to clinicians, because they are quantified in the same units as the parameters of interest. 18 MDC values are especially useful in determining whether changes in values for wheeling parameters are indicative of real change and not measurement error. The MDC values of this study could allow clinicians to estimate with 95% confidence the minimal amount of change in patient status that can be assessed confidently between test sessions. For instance, if a WCU undergoes propulsion training and increases his push length on tile by 12 (as measured by the SCP), based on an estimated MDC of (see table 2), we could consider with 95% confidence that the change is because of the intervention and not because of random variation in measurement. This is because the observed change of 12 exceeds the estimated MDC of In both WCUs and AB individuals, the responsiveness of the SCP is increased (lower MDC) when the mean of 5 trials is used. The AB group showed higher MDC values for all parameters. Therefore, if our AB group were to reflect newly injured individuals who are prescribed with wheelchairs for the first time, the threshold for real change that can be assessed with 95% confidence would be higher than in individuals with manual wheeling experience. Again, this highlights the benefits (especially in individuals with minimal experience) of averaging multiple trials to increase the intersession reliability of the SCP. Study Limitations The most significant limitation for this study was the use of AB individuals to represent naïve, newly injured WCUs. This comparison may not be entirely valid because of differences in motor function (ie, trunk control). It would have been difficult

6 2372 RELIABILITY OF THE SMARTWHEEL CLINICAL PROTOCOL, Lui to include naïve WCUs with acute injuries, because patients during this stage are in the midst of rehabilitation. However, we and other authors believe that AB individuals naïve to manual wheeling and newly injured WCUs share similar aspects of inexperienced wheeling. 28,29 For future directions, the SCP may be modified to include the collection of multiple trials. The optimal number of trials to collect should also be determined. The current SmartWheel clinical software does not include a built-in function to allow for the averaging of multiple trials. Future software updates could address this. CONCLUSIONS In this study, the test-retest reliability was quantified for each parameter of the SCP. In general, the SCP showed excellent intra- and intersession reliability in individuals with prior wheelchair experience (at least 1y), even when only the first trial is used from a session. However, for individuals with no wheeling experience, collecting and averaging multiple trials is strongly recommended in order to increase the reliability of the SCP. MDC estimates were presented to provide insight into the sensitivity of the SCP in detecting statistically significant changes in wheelchair propulsion. Acknowledgment: We thank Rachel E. Cowan, PhD, for providing valuable comments on the manuscript. References 1. Silfverskiold J, Waters RL. Shoulder pain and functional disability in spinal cord injury patients. Clin Orthop Relat Res 1991;(272): Boninger ML, Souza LS, Cooper RA, Fitzgerald SG, Koontz AM, Fay BT. Propulsion patterns and pushrim biomechanics in manual wheelchair propulsion. Arch Phys Med Rehabil 2002;83: Mercer JL, Boninger M, Koontz AM, Ren D, Dyson-Hudson T, Cooper R. Shoulder joint kinetics and pathology in manual wheelchair users. Clin Biomech 2006;21: Shimada SD, Robertson RN, Boninger ML, Cooper RA. Kinematic characterization of wheelchair propulsion. J Rehabil Res Dev 1998;35: Richter WM, Rodriguez R, Woods KR, Axelson PW. Stroke pattern and handrim biomechanics for level and uphill wheelchair propulsion at self-selected speeds. Arch Phys Med Rehabil 2007; 88: Requejo PS, Mulroy SJ, Haubert LL, Newsam CJ, Gronley JK, Perry J. Evidence-based strategies to preserve shoulder function in manual wheelchair users with spinal cord injury. Top Spinal Cord Inj 2008;13: Cooper RA. Smartwheel: from concept to clinical practice. Prosthet Orthot Int 2009;33: Cowan RE, Boninger ML, Sawatzky BJ, Mazoyer BD, Cooper RA. Preliminary outcomes of the SmartWheel User s Group database: a proposed framework for clinicians to objectively evaluate manual wheelchair propulsion. Arch Phys Med Rehabil 2008; 89: Consortium for Spinal Cord Medicine. Preservation of upper limb function following spinal cord injury: a clinical practice guideline for health-care professionals. Washington (DC): Paralyzed Veterans of America; Lexell JE, Downham DY. How to assess the reliability of measurements in rehabilitation. Am J Phys Med Rehabil 2005; 84: Dawis RV. Scale construction. J Couns Psychol 1987;34: Brown DD, Knowlton JH, Schneider TL, Hetzler RK. Physiological and biomechanical differences between wheelchair-dependent and able-bodied subjects during wheelchair ergometry. Eur J Appl Physiol 1990;60: Robertson RN, Boninger ML, Cooper RA, Shimada SD. Pushrim forces and joint kinetics during wheelchair propulsion. Arch Phys Med Rehabil 1996;77: De Groot S, Veeger HE, Hollander PA, Van der Woude LH. Wheelchair propulsion technique and mechanical efficiency after 3 wk of practice. Med Sci Sport Exer 2002;34: De Groot S, Veeger HE, Hollander AP, van der Woude LH. Adaptations in physiology and propulsion techniques during the initial phase of learning manual wheelchair propulsion. Am J Phys Med Rehabil 2003;82: Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Statis Med 1998;17: Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res 2005; 19: Munro BH. Statistical methods for health care research. Philadelphia: Lippincott Williams & Wilkins; Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1; McGinley J, Baker R, Wolfe R, Morris ME. The reliability of three-dimensional kinematic gait measurements: a systematic review. Gait Posture 2009;29: Turcot K, Aissaoui R, Boivin K, Hagemeister N, Pelletier M, de Guise JA. Test-retest reliability and minimal detectable change determination for 3-dimensional tibial and femoral accelerations during treadmill walking in knee osteoarthritis patients. Arch Phys Med Rehabil 2008;89: Klejman S, Andrysek J, Dupuis A, Wright V. Test-retest reliability of discrete gait parameters in children with cerebral palsy. Arch Phys Med Rehabil 2010;91: Cowan RE, Nash MS, de Groot S, der Woude LH. Adapted manual wheelchair circuit: test-retest reliability and discriminative validity in persons with spinal cord injury. Arch Phys Med Rehabil 2011;92: Kirby RL, Dupuis DJ, MacPhee AH, et al. The wheelchair skills test (version 2.4): measurement properties. Arch Phys Med Rehabil 2014;85: Crane BA, Holm MB, Hobson D, Cooper RA, Reed MP, Stadelmeier S. Test-retest reliability, internal item consistency, and concurrent validity of the wheelchair seating discomfort assessment tool. Assist Technol 2005;17: Bravo G, Potvin L. Estimating the reliability of continuous measures with Cronbach s alpha or the intraclass correlation coefficient: toward the integration of two traditions. J Clin Epidemiol 1991;44: Lenton JP, Fowler NE, van der Woude LH, Goosey-Tolfrey VL. Wheelchair propulsion: effects of experience and push strategy on efficiency and perceived exertion. Appl Physiol Nutr Metab 2008; 33: De Groot S, Veeger HE, Hollander AP, van der Woude LH. Short-term adaptations in co-ordination during the initial phase of learning manual wheelchair propulsion. J Electromyogr Kines 2003;13: Van Den Berg R, De Groot S, Swart KM, van der Woude LH. Physical capacity after 7-weeks of low-intensity wheelchair training. Disabil Rehabil 2010;32: Suppliers a. Three Rivers Holdings, 1826 W Broadway Rd, Ste 43, Mesa, AZ b. Instinct Mobility, 818 W 10th Ave, Vancouver, BC, V5Z 1M9 Canada. c. Version 16.1; SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL d. Microsoft, One Microsoft Way, Redmond, WA