Issues for selection of outcome measures in stroke rehabilitation: ICF activity

Transcription

1 Disability and Rehabilitation, 2005; 27(6): CLINICAL COMMENTARY Issues for selection of outcome measures in stroke rehabilitation: ICF activity K. SALTER 1, J. W. JUTAI 1,2, R. TEASELL 1,2, N. C. FOLEY 1, J. BITENSKY 1, & M. BAYLEY 3 1 Department of Physical Medicine and Rehabilitation, St. Joseph s Health Care London, 2 University of Western Ontario, London, Ontario, Canada, 3 Neurorehabilitation Program, Toronto Rehabilitation Institute, Toronto, Ontario, Canada (Accepted date August 2004) Abstract Purpose. To evaluate the psychometric and administrative properties of outcome measures in the WHO International Classification of Functioning, Disability and Health (ICF) Activity category used in stroke rehabilitation research and reported in the published literature. Method. Critical review and synthesis of measurement properties for nine commonly reported instruments in the stroke rehabilitation literature. Each instrument was rated using the eight evaluation criteria proposed by the UK Health Technology Assessment (HTA) programme. The instruments were also assessed for the rigour with which their reliability, validity and responsiveness were reported in the published literature. Results. The reporting of specific measurement qualities for outcome instruments was relatively consistent across measures located within the same general ICF category. There was evidence to suggest that the measures were responsive to change as well as being valid and reliable tools. The best available instruments were associated with the assessment of activities of daily living, balance (static and dynamic), functional independence, and functional mobility. Conclusions. Given the diversity that exists among available measures, the reader is encouraged to examine carefully the nature and scope of outcome measurement used in reporting the strength of evidence for improved functional activity in stroke rehabilitation. However, there appears to be good consensus regarding the most important indicators of successful rehabilitation outcome, especially in the case of functional mobility. Introduction Measuring the effectiveness of rehabilitation interventions is widely accepted as essential to good practice. Van der Putten et al. [1] pointed out that measuring the outcome of health care is a central component of determining therapeutic effectiveness and, therefore, the provision of evidence-based healthcare. Reliability, validity, and administrative burden are properties of measurement instruments that affect the credibility of the measurement process [2,3] and the reporting of research findings [4 6]. Remarkably few published rehabilitation outcome studies appear to give adequate reports on these properties in defending their research designs and interpreting their results [2]. Recently, there have been important advances in compiling and publishing the best-available scientific evidence examining the effectiveness of stroke rehabilitation [7,8]. However, there are limitations to successfully transferring the research results to clinical practice and service delivery, in part due to a lack of consensus regarding the selection of appropriate measures to best address and balance the needs and values of stakeholders in stroke rehabilitation, including patients and their caregivers, practitioners, and healthcare decision makers. Ultimately, the comparison of size and direction of statistical results across areas of stroke rehabilitation covered within evidence-based reviews will be most meaningfully interpreted when comparable approaches to outcome measurement have been used Correspondence: K. Salter, Department of Physical Medicine and Rehabilitation, St. Joseph s Health Care London, 801 Commissioners Road East, London (Ontario) N6C 5J1, Canada. Katherine.Salter@sjhc.london.on.ca ISSN print/issn online ª 2005 Taylor & Francis Group Ltd DOI: /

2 316 K. Salter et al. [9]. To enhance the clinical meaningfulness of the current evidence, this paper presents the best available information on how outcome measures might be classified and selected for use, based upon their measurement qualities. For this purpose, we have selected for review only some of the more commonly used measures in stroke rehabilitation. This paper is not intended to be a comprehensive compendium of stroke outcome measures. This paper attempts to describe how the ICF [10, 11] conceptual framework can be used for classifying outcome measures in stroke rehabilitation, and summarize aspects of measurement theory that are pertinent for evaluating measures. It also gives a template presentation on the characteristics, application, reliability, validity, and other clinimetric qualities of commonly used measures in a format for easy reference. For a more extensive discussion of outcome measurement theory and properties in physical rehabilitation, the reader is referred to the book by Finch et al. [12] This paper will present only the information most relevant for the rehabilitation of stroke patients. Classification of stroke rehabilitation outcomes Outcomes research requires a systematic approach to describing outcomes and classifying them meaningfully. The study and assessment of stroke rehabilitation has sparked the development of numerous outcome measures applicable to one or more of its many dimensions. In attempting to discuss some of the commonly used measures available for use within the field of stroke rehabilitation, it is useful to have guidelines available for classifying these tools. The WHO International Classification of Functioning, Disability and Health (ICF)[10, 11] provides a multi-dimensional framework for health and disability suited to the classification of outcome instruments. Originally published in 1980, the WHO framework has undergone several revisions. In the most recent version, the ICF framework [10, 11] identifies three primary levels of human functioning the body or body part, the whole person, and the whole person in relation to his/her social context. Outcomes may be measured at any of these levels Body functions/ structure (impairment), Activities (refers to the whole person formerly conceived as disability in the old ICIDH framework) and Participation (formerly referred to as handicap). Activity and Participation are affected by environmental and personal factors (referred to as contextual factors within the ICF Table I). Outcome measures can also be conceived of as falling along a continuum of measurement moving from measurements at the level of body function or Old terminology Table I. New terminology Impairment Body function/ structure ICF Definitions. Definition Physiological functions of body systems including psychological. Structures are anatomical parts or regions of their bodies and their components. Impairments are problems in body function or structure. Disability Activity The execution of a task by an individual. Limitations in activity are defined as difficulties an individual might experience in completing a given activity. Handicap Participation Involvement of an individual in a life situation. Restrictions to participation describe difficulties experienced by the individual in a life situation or role. structure to those focused on participation and life satisfaction. It becomes more difficult to attribute outcomes to particular rehabilitation interventions as one moves away from body function toward participation, since many variables other than the interventions might account for changes observed [13, 14]. We reviewed the findings from a number of recent studies that have examined the patterns of scale use in various settings, both clinical [15 17] and research [14, 18 21]. As there is no authoritative published recommendation on what a complete list of stroke rehabilitation measures should include, we have focused our review on scales with which most stroke specialists would be familiar. Table II presents twenty of the most popular instruments from the stroke rehabilitation literature, classified by ICF category by the primary author, after consideration of the study author s stated purpose for the tool and the content of the instrument s items. This subjective component was introduced because there is no published consensus on how this kind of classification should proceed. The classification was reviewed independently by the co-authors. Table II reflects the consensus among the paper s authors. If a classification is to be useful for scientific research, the basic categories and concepts within it need to be measurable, and their boundaries clear and distinct. It is not yet clear from the research evidence that the three ICF categories completely fulfil these criteria. Nonetheless, when applied to outcome assessment in stroke rehabilitation the ICF conceptual framework can be used to place outcome measures into one of the three categories depending upon what it is they purport to measure.

3 Outcome measures in stroke rehabilitation 317 Table II. Classification of outcome measures. Body structure (impairments) Activities (limitations to activity disability) Participation (barriers to participation handicap) 1. Beck Depression Inventory 6. Barthel Index 15. Euroqol-5D 2. Fugl-Meyer Assessment 7. Berg Balance Scale 16. Medical Outcomes Study Short Form Mini Mental State 8. Chedoke McMaster Stroke Assessment Scale 17. Nottingham Health Profile Examination 9. Functional Independence Measure (FIM) 18. Sickness Impact Profile (stroke adapted version) 4. Modified Ashworth Scale 10. Frenchay Activities Index 19. Stroke Impact Scale 5. Motor-free Visual Perception 11. Modified Rankin Handicap Scale 20. Stroke Specific Quality of Life Test 12. Rivermead Motor Assessment 13. Rivermead Mobility Index 14. Timed-Up-and-Go (TUG) It should be noted that linking existing measures to the ICF is not a straightforward process [22 24]. Many existing measures include items that fall into several ICF dimensions in addition to items that may not be included in the ICF at all. Instruments appearing in the Participation domain, for instance, assess participation in life situations such as social functioning or roles, but include the assessment of elements of one or both of the Body Structure/ Function and Activities categories. While these measures have been used to assess health-related quality of life, it is not the intent of this paper to define this construct or its assessment. The present study was not intended as an attempt to provide item by item mapping for each of the identified measures. The ICF was used as a framework within which measures were classified according to the level of assessment they include furthest along a continuum from body function, through activity to participation. However, the process of developing systematic approaches to establishing linkages between existing measures and the ICF is an important one in the ongoing attempt to create an international language and standard for measurement. Evaluation criteria for outcome measures While it is useful to have the ICF framework within which to classify levels of outcomes measures, it is necessary to have a set of criteria to guide the selection of outcomes measures. Reliability, validity and responsiveness have widespread usage and are discussed as being essential to the evaluation of outcome measures [1, 14, 19, 25]. Finch et al. [12] provide a good tutorial on the general issues for outcome measure selection. The Health Technology Assessment (HTA) programme [26] examined 413 articles that focused on methodological aspects of the use and development of patient-based outcome measures. In their report, they recommended the use of eight evaluation criteria. Table III lists the criteria and gives a definition for each one. It also identifies a recommended standard for quantifying (rating) each criterion, where applicable, and how the ratings should be interpreted. The criteria, including some additional considerations described below, were applied to each of the outcome measures reviewed in this paper. Each measure reviewed in this paper was also assessed for the thoroughness with which its reliability, validity and responsiveness have been reported in the literature. Standards for evaluation of rigour were adapted from McDowell & Newell [27] and Andresen [4]. The authors assessed rigour in the manner described above for the other ratings, and scored each instrument on each of the three properties as follows: Excellent most major forms of testing reported; + + Adequate several studies and/or several types of testing reported; + Poor minimal information is reported and/or few studies (other than author s); N/a no information available. For example, a rating of (or excellent) for validity meant that evidence has been presented demonstrating excellent construct validity based on the standards provided and in various forms including convergent and discriminant validity. In addition to the criteria outlined above, three additional questions were considered. Has the measure been used in a stroke population? Has the measure been tested for use with proxy assessment? What is the recommended time frame for measurement? The primary author reviewed and rated each instrument using these evaluative criteria. The results were reviewed independently by the coauthors. There were very few instances of disagreement among raters, and they were never more than one level apart in their evaluations. The results presented in this paper reflect the authors consensus on ratings after discussing all discordant ratings. Has the measure been used in a stroke population? Reliability and validity are not fixed qualities of measures. They should be regarded as relative

4 Table III. Evaluation criteria and standards. Criterion Definition Standard (1) Appropriateness The match of the instrument to the purpose/question under study. One must determine what Depends upon the specific purpose for which the measurement is intended. information is required and what use will be made of the information gathered [74] (2) Reliability Refers to the reproducibility and internal consistency of the instrument. Test retest or interobserver reliability(icc; kappa statistics) [4, 81, 99] Reproducibility addresses the degree to which the score is free from random error. Test re test & interobserver Excellent: ; Adequate: ; Poor: reliability both focus on this aspect of reliability and are commonly evaluated using correlation statistics including ICC, Pearson s or Spearman s coefficients and kappa coefficients (weighted or unweighted). Internal consistency assesses the homogeneity of the scale items. It is generally examined using split-half reliability or Cronbach s alpha statistics. Item-to-item and item-to scale correlations are also accepted methods. Note: Fitzpatrick et al. [26] recommended a minimum test-retest reliability of 0.90 if the measure is to be used to evaluate the ongoing progress of an individual in a treatment situation. Internal consistency (split-half or Cronbach s a statistics): Excellent: ; Adequate: ; Poor [4] Note: Fitzpatrick et al. [26] cautioned a values in excess of 0.90 may indicate redundancy. Inter-item & item-to-scale correlation coefficients: Adequate levels inter-item: between 0.3 and 0.9; item-to-scale: between 0.2 and 0.9 [26, 64] (3) Validity Does the instrument measure what it purports to measure? Forms of validity include face, content, construct, and criterion. Concurrent, convergent or discriminative, and predictive validity are all considered to be forms of criterion validity. However, concurrent, convergent and discriminative validity all depend on the existence of a gold standard to provide a basis for comparison. If no gold Construct/convergent and concurrent correlations: Excellent: , Adequate: , Poor: [4, 26, 27, 100] ROC analysis AUC: Excellent: , Adequate: , Poor: [27] There are no agreed on standards by which to judge sensitivity and specificity as a validity index [101]. standard exists, they represent a form of construct validity in which the relationship to another measure is hypothesized [12]. (4) Responsiveness Sensitivity to changes within patients over time (which might be indicative of therapeutic effects). Sensitivity to change: Responsiveness is most commonly evaluated through correlation with other change scores, effect sizes, standardized response means, relative efficiency, sensitivity & specificity of change scores and ROC analysis. Assessment of possible floor and ceiling effects is included as they indicate limits to the range of detectable change beyond which no further improvement or deterioration can be noted. Excellent: Evidence of change in expected direction using methods such as standardized effect sizes: = small; = moderate; = large) Also, by way of standardized response means, ROC analysis of change scores (area under the curve see above) or relative efficiency. Adequate: Evidence of moderate/less change than expected; conflicting evidence. Poor: Weak evidence based solely on p-values (statistical significance) [4, 26, 27, 100] 318 K. Salter et al. (5) Precision Number of gradations or distinctions within the measurement. E.g. Yes/no response vs. a 7-point Likert response set. (6) Interpretability How meaningful are the scores? Are there consistent definitions and classifications for results? Are there norms available for comparison? (7) Acceptability How acceptable the scale is in terms of completion by the patient does it represent a burden? Can the assessment be completed by proxy, if necessary? (8) Feasibility Extent of effort, burden, expense & disruption to staff/clinical care arising from the administration of the instrument. Floor/Ceiling Effects: Excellent: No floor or ceiling effects. Adequate: floor and ceiling effects 4 20% of patients who attain either the minimum (floor) or maximum (ceiling) score. Poor: 4 20% [64]. Depends on the precision required for the purpose of the measurement (e.g., classification, evaluation, prediction). Jutai & Teasell [9] point out these practical issues should not be separated from consideration of the values that underscore the selection of outcome measures. A brief assessment of practicality will accompany each summary evaluation.

5 Outcome measures in stroke rehabilitation 319 indicators of how well the instrument might function within a given sample or for a given purpose [26, 28]. Responsiveness, too, may be condition or purpose specific. Van der Putten et al. [1] for example, in an evaluation of the Barthel Index and Functional Independence Measure, found both measures to be equally responsive in terms of effect sizes when used among stroke patients and patients with multiple sclerosis. Within the stroke group, floor and ceiling effects were within acceptable limits on both measures. However, the authors point out that within the MS patient group, there were larger ceiling effects associated with the BI scores and the scores from the FIM cognitive subscale. This, coupled with the much smaller effect sizes noted among MS patients, leads the authors to suggest that these two instruments are better suited for use among stroke patients. Therefore, it would seem important for a measure to have been tested for use in the population within which it will be applied. Has the measure been tested for use with proxy assessment? When assessment is conducted in such a way as to require a form of self-report (e.g. interview or questionnaire in person, by telephone or by mail), stroke survivors who have experienced significant cognitive or speech and language deficits may not be able to complete it and therefore, may be excluded from assessment. In such cases, the use of a proxy respondent becomes an important alternative source of information. However, the use of proxy respondents should be approached with caution. Studies of proxy assessments report a tendency for significant others, including family members, to assess patients as more disabled than they appear on other measures of functional disability, including self-reported methods. This discrepancy becomes more pronounced for patients with more impaired levels of functioning [29 31]. Hachisuka et al. [31] suggested that this discrepancy could be explained by a difference in interpretation. Proxy respondents may be rating actual, observable performance, while patients may rate their perceived capability what they think they are capable of doing rather than what they actually do. Unfortunately, using a healthcare professional as a substitute for the family member or significant other as proxy does not solve this problem. A similar discrepancy has been noted in ratings when using healthcare professionals as proxy respondents though in the opposite direction. They may tend to rate patients higher than the patients themselves would [30, 32]. It has been suggested that, in this case, the discrepancy is due to a difference in frame of reference. A healthcare professional may use a different, more disabled group as a reference norm, whereas the patient would only compare him/herself to pre-stroke conditions [32]. What is the recommended timeframe for measurement? The natural history of stroke presents problems in assessment in that the rate and extent of change in outcomes varies across the different levels of ICF classification [19]. The further one moves along the outcome continuum from body structure toward participation, the more time it may take to reach a measurement end point, that is, participation within a defined social context may take longer to stabilize than the impaired body structure [33]. Jorgensen et al. [34] demonstrated that recovery in Activities of Daily Living (ADL) occurred, in most patients, within the first 13 weeks following a stroke even though the time course of both neurological and functional recovery is strongly related to initial stroke severity. They suggest that a valid prognosis of functional recovery might be made within the first 6 months. According to Mayo et al. [35], by 6 months post-stroke, physical recovery is complete, for the most part, with additional gains being a function of learning, practice and confidence. Duncan et al. [19] support this suggested time frame for assessment of neurological impairment and disability outcomes but suggest that participation outcomes should not be measured sooner than 6 months post-stroke, to provide the opportunity for the patient s social situation to stabilize. They also suggest that assessments at the time of discharge not be used as endpoint measurements. They argue that variability in treatment interventions and length of stay practices decreases the comparative usefulness of this information. Review of activity outcome measures This paper is the second in a series of three, and deals with the second level or category of the ICF classification system; Activity. The necessity for clearly defined boundaries between categories of classification is most apparent when one considers the ICF dimensions of Activity and Participation. Given that the domains associated with activity and participation are presented as a single, neutral list with several classification options [10], it is not surprising that questions have arisen with regard to the validity of separating them into distinct dimensions. While exploration of this issue is ongoing, it is worth noting that Jette et al. [36] recently provided empirical evidence of distinctly differing concepts conforming to the dimensions of activity and participation as defined within the ICF

6 320 K. Salter et al. and suggested that the participation domain may represent more complex categories of life behaviours. While recognizing that the fit of a given instrument within a single category is rarely perfect, measures appearing in this section focus on the identification or assessment of Activity. As defined by the ICF [10, 11], Activity refers to the execution of a task or action by an individual. Activity limitations refer to difficulties an individual may have in executing activities. In contrast with the ICF Participation category, tasks subsumed within the Activity level are relatively simple, related to impairment of body function, performed alone at the individual level, less dependent upon environmental influence, assessed in hospital by clinicians, and the focus for post-acute care [37]. The reader should note that three additional measures listed in Table II include assessments in the ICF Activity category. They are the Medical Outcomes Study Short Form 36 (SF-36) [38], the Stroke Impact Scale (SIS) [39], and the Stroke Specific Quality of Life Scale (SSQOL) [40]. They are reviewed in another companion paper [41] because they all include measurements in the ICF Participation category. Barthel index The Barthel Index of Activities of Daily Living (BI) has been in use since 1955 [42]. It was originally intended as a simple index of independence with which to quantify the ability of a patient with a neuromuscular or musculoskeletal disorder to care for him/herself (regardless of particular diagnostic designations). It is, perhaps, the most widely used measure of functional disability. The BI is very simple, consisting of 10 common ADL activities, and is administered through direct observation. Each activity is assessed for independence/dependence and scored via an arbitrary weighting system (originally applied to reflect nursing care and social acceptability). Eight of the 10 items represent activities related to personal care; the remaining two are related to mobility. The index yields a total score out of 100 the higher the score, the greater the degree of functional independence [27]. The BI can take as little as 2 5 min to complete by self-report and up to 20 min by direct observation [12]. It does not require training to administer and has been shown to be equally reliable when administered by skilled and unskilled individuals [43]. The measurement properties of the Barthel Index are summarized in Table IV. Advantages The clearest advantage of the BI is its simplicity and ease of administration. Its reliance on information collected during functional examination enhances its convenience and cost effectiveness in longitudinal assessment. It has been used across a variety of settings without a significant decrease in reliability or validity. Its established, widespread use provides a high degree of familiarity and interpretability. Limitations Perhaps the most common criticism of the Barthel Index is its relative insensitivity to change and lack of comprehensiveness, which is reflected in large reported ceiling and floor effects. Duncan et al. [44] demonstrated that, among patients recovering from mild stroke or TIA who scored 100 on the BI, there were deficits in health status suggesting that the BI is not sufficiently sensitive to change among the least impaired stroke survivors. However, Collin & Wade [43] noted that the BI is able to discriminate between an individual who is independent, and one who requires assistance. This distinction, the authors point out, may have more significance to clinical practice than to research. Additional criticisms of the BI include its imprecision associated with the tendency for scale scores to be dichotomised [27]. Dichotomization reduces outcome information and may limit a scale s ability to detect a significant shift in disability [19] while also reducing statistical power. Although Granger [45] proposed a 60/61 split as the threshold of dependence/independence, this has not been adopted as a standardized cut-off and, indeed, there seems little agreement regarding classifications derived from the BI score. Since there is no apparent consensus for categorization of BI scores, even in terms of dichotomization for functional dependence, comparison of outcomes across trials is difficult and does not lend itself to meta-analytic approaches [14, 19, 46]. Summary Barthel Index The ratings of methodological rigour associated with evaluation of the measurement properties of the Barthel Index are presented in Table XIV. Practicality Interpretability The degree of familiarity of the BI contributes to its interpretability. However, there is a lack of agreement regarding a threshold for independence vs. dependence and several different scoring

7 Outcome measures in stroke rehabilitation 321 Table IV. Measurement properties of the Barthel Index. Reliability. Test retest: in both skilled and unskilled raters, K = 0.93 [102]; Wolfe et al. [81] reported test retest as 0.98; supported by Green et al. [76] reported 4 75% agreement on individual items assessed one week apart though the K statistic showed considerable variability; Cohen et al. [100] reported studies showing good test retest (r = 0.87) & intrarater reliability (r = ).. Interobserver reliability: D Olhaberriague et al. [103] found BI highest inter-observer reliability (r = 0.88) compared to the Rankin and the Stroke Severity Scale; Wolfe et al. [81] reported K w ; Hsueh et al. [99] reported K w values of on the item level with an overall ICC of 0.94; Cohen et al. [100] reported studies showing interobserver reliability of r = ).. Internal consistency: Shah et al. [104] reported Cronbach s a = for their version and for the original = ; supported by Hsueh et al. [99] Validity. Predictive validity: BI associated with level of support required to perform ADLs in the home [105]; D Olhaberriague et al. [103] BI scores correlated highly with immediate post stroke condition; Shah et al. [106] reported BI scores predicted dependency needs for elderly stroke patients in the community. Duncan et al. [44] found BI + symptom status most consistent predictor of health status (mild stroke); initial BI score shown to be predictive of cognitive impairment/mmse score [107] and of level of activity assessed by the FAI [99].. Concurrent validity: as reported in Cohen and Marino [100] high correlations shown with the BI and Katz index (k = 0.77) and the Kenny self-care evaluation (spearman r = 0.73); walking speed (r = 0.71) [108]; Berg Balance Scale (r = ) and the Fugl-Meyer motor assessment (r = ) [99]; with FAI (0.79) and the Rankin scale ( ) [109].. BI often used as the gold standard criterion for assessment of validity of other measures.. Construct validity (Known groups); Duncan et al. [44] reported BI able to distinguish between stroke survivors; patients with history of TIA and asymptomatic groups (p ).. Convergent/Discriminant validity: Hobart & Thompson [110] demonstrated convergence with the FIM (r = 0.93), less with the LHS & SF-36 physical component summary (r = 0.37 & 0.22). Discriminant validity was demonstrated with the SF-36 mental component summary r = Responsiveness. Wood-Dauphinee et al. [111] reported ES = 0.42 from admission to 5 weeks post stroke; Wood-Dauphinee et al. [112] reported ES = 0.71 from 0 6 weeks post-stroke and ES = 0.25 from 6 12 weeks; Stone et al. [113] found the BI to be sensitive to changes in a clinical rehabilitation setting especially among stroke patients (median change = 5; p ); Hocking et al. [108] reported AUC for change scores = 0.66; p for change).; BI demonstrated significant effect sizes in the expected direction from admission to discharge (ES = 0.37, p ) [1]; Hsueh et al. [99] reported significant effect sizes at all stages of stroke recovery except for the latest ( days post stroke); Wallace et al. [70] reported ES = 0.31 & SRM = 0.63 (0.48 and 0.86 for those known to change). Hsueh et al. [68] reported SRM = 1.2 (p ) indicating significant change; Dromerick et al. [114] reported SRM = 1.72 (change from admission to discharge from rehabilitation) and ROC/AUC = 0.82 (using FIM as the reference measure).. Hobart and Thompson [110] reported a 1% and 5% floor/ceiling effect for the BI; Duncan et al. [44] reported 66% of individuals with mild stroke scored 100 on BI while SF36 continued to demonstrate differences in function between groups (stroke, TIA, asymptomatic) when BI scores for individuals in these groups when BI = 100 [44]; van der Putten et al. [1] found BI scores showed a 7% ceiling effect compared to the motor FIM s 1%; A ceiling effect of 27% was reported at discharge from rehabilitation [114]. Tested for stroke patients?. Yes [1, 43, 44, 68, 70, 99, 104, 108, 113] Other formats. Note Modified Barthel-Index: Many versions of the BI have emerged. There appears to be no definitive version, though the 10-item, 100 point version may be most widely used [115]. The term modified-barthel may apply to a 10-item version scored out of 20 [43]; a 10-item version with a weighted 5-pt. Likert scale [104]; or a 15-item version with a 4-pt. weighted scale [105].. 5-item short form [110] excellent concurrent validity with 10-item BI (ICC = 0.90) and FIM (ICC = 0.87); effect sizes comparable to the original; a for internal consistency exceeded 0.80 contains less information; poor discriminative ability among stroke patients with severe disability due to a large floor effect [68]. Hobart and Thompson [110] reported a 4.5% and 8.9 % floor/ceiling effect for the 5-item short form.. Barthel administered by telephone interviews, mailed questionnaires and at-home interviews: found to produce equivalent results. Yeo et al. [116] reported r = 0.80 to 0.88 for agreement between methods of administration. Korner-Bitensky and Wood-Dauphinee [117] reported 4 90% agreement between telephone & face-to-face interviews by trained & untrained interviewers; supported earlier reports by Shinar et al. [118] and Shah et al. [106]. Self-report BI: McGinnis et al. [32] reported significant differences between patient ratings and ratings performed by healthcare professionals (p ). Dorevitch et al. [119] also reported a significant, though weaker, difference between self-report and direct observation. No systematic response bias was reported. Use by proxy?. Collin & Wade [43] tested 4 methods of rating patients (including asking a family member and a health professional) found a highly significant coefficient of concordance between them (W-0.93); Korner- Bitensky and Wood-Dauphinee [117] reported agreement between self-report interviews and proxy respondent interviews to be greater than 90%; ICC = reported for BI ratings completed by the patient and a family member & compared to BI administered by a healthcare professional [31].. Dorevitch et al. [119] compared modes of administration found informant (proxy) ratings to differ most from direct observation. Home carers provided information most different from direct observation (p ).

8 322 K. Salter et al. systems are used, making comparisons across groups/studies more difficult. Acceptability The BI has been evaluated for both selfreport and use with proxy respondents in addition to direct observation. Both self-report and interview formats take less time to complete than direct observation and may serve to reduce patient burden. Feasibility The BI is simple to administer and requires no training. It has been developed in many forms that can be administered in many situations and is suited for longitudinal assessment. Berg Balance Scale The Berg Balance Scale (BBS) provides a quantitative assessment of balance in older adults [47]. It was intended for use in monitoring the clinical status of patients or effectiveness of treatment interventions over time [48]. The scale consists of 14 items requiring subjects to maintain positions or complete movement tasks of varying levels of difficulty. All items are common to everyday life. Administration of the scale requires a ruler, a stopwatch, chair, step or stool, room to turn 3608 and min. It is administered via direct observation of task completion [48, 49]. Each of the 14 items receives a score of 0 4 based on the subject s ability to meet the specific time and distance requirements of the test. A score of zero represents an inability to complete the item and a score of 4 represents the ability to complete the task independently. Scores of less than 45 out of a possible 56 are generally accepted as indicative of balance impairment [50, 51]. The measurement properties of the Berg Balance Scale are summarized in Table V. Advantages The Berg Balance Scale measures a number of different aspects of balance, both static and dynamic, and does so with minimal space and equipment requirements [51 53]. No specialized training is required to administer the BBS [53] and in fact, high levels of reliability have been reported [48] when the test was administered by untrained individuals. Wee et al. [54] suggested that the BBS is particularly well suited for use in acute stroke rehabilitation, as the majority patients do not obtain maximum scores on admission to rehabilitation. Limitations The BBS takes somewhat longer to administer than other balance measures [52] and may not be suitable for the evaluation of active, elderly persons, as the items included are not sufficiently challenging for this group [47, 51, 53]. No common interpretation exists for BBS scores, their relationship to mobility status and the requirement for mobility aides [55]. The BBS may suffer from decreased sensitivity in early stages post stroke among severely affected patients as the scale includes only one item relating to balance in the sitting position [56]. Summary Berg Balance Scale The ratings of methodological rigour associated with evaluation of the measurement properties of the Berg Balance Scale are presented in Table XIV. Practicality Interpretability While the reliability and validity of the scale are excellent, there are no common standards for the interpretation of BBS scores. However, there is an accepted cut-off point for the presence of balance impairment. Acceptability This direct observation test would not be suited for severely affected patients as it assesses only one item relative to balance while sitting. Active individuals would find it too simple. The scale is not suited for use by proxy. Feasibility The BBS requires no specialized training to administer and relatively little equipment or space. Chedoke-McMaster Stroke Assessment Scale The Chedoke-McMaster Stroke Assessment Scale (CMSA) is a 2-part assessment consisting of a physical impairment inventory and a disability inventory. The impairment inventory is intended to classify patients according to stage of motor recovery while the disability inventory assesses change in physical function [57]. The scale s impairment inventory has 6 dimensions; shoulder pain, postural control, arm movements, hand movements, leg movements, and foot movements. Each dimension (with the exception of shoulder pain whose rating scale is unique) is rated on a 7-point scale corresponding to Brunnstrom s 7 stages of motor recovery (where 1 = flaccid paralysis & 7 = normal). The maximum total score for physical impairment is 42. The disability inventory consists of a gross motor index (10 items) and a walking index (5 items). With the exception of a 2-min walking test, items are scored using the same 7-point scale used in the Functional Independence Measure (FIM) where 1

9 Outcome measures in stroke rehabilitation 323 Table V. Measurement properties of the Berg Balance Scale. Reliability. Test retest: ICC s = 0.91 (general elderly) and 0.99 (stroke survivors) [48]; ICC = 0.88; [120]; ICC = 0.98 [121].. Interobserver reliability: ICC s = 0.92 (general elderly) and 0.98 (stroke) [48]; ICC = 0.98 [122] Mao et al. [56] reported an overall ICC = 0.95 and a range of K w for BBS items from Internal Consistency: Berg et al. [122] reported a = 0.96; In general elderly sample, a = 0.83 and 0.97 among stroke survivors. Item to total correlations ranged from (elderly) and (stroke group) [48]; Mao et al. [56] reported a = Validity. Concurrent validity: correlated with global ratings of balance provided by a carer ( ), by the patients themselves ( ) and TUG scores ( ) [50]; with TUG scores (r = ; p ), mobility items of the BI (r = 0.67; p ) and with speed and amplitude laboratory measures [122]; Liston and Brouwer [121] showed BBS scores related to dynamic Balance Master measures (LR-3sec LR 2-sec, FB 3 sec, FB 2 sec (all p , r ) and limit of stability movement time p l r ); Mao et al. [56] reported strong relationships between BBS scores and Fugl-Meyer-(B)alance (r = ), postural assessment scale for stroke patients ( ) at 4 assessment times (14, 30, 90 and 180 days post stroke).. Construct validity: scores significantly correlated in the expected direction, with BI scores (r = 0.80), Fugl- Meyer scale scores ( ) [50]; with BI, r = 0.86 to 0.91 [56]; BBS scores also reported to correlate with FIM r = 0.57 to 0.70; p [49]; r = 0.76; p [54].. Construct validity (known groups): Berg et al. [50, 122] found BBS scores differentiated groups based on use of mobility aides (p ) and location of evaluation (home, rehabilitation program, acute hospital) at the end of study follow-up (p ) [50]; Wee et al. [54, 55] also showed admission BBS able to discriminate groups based on discharge destination of home vs. institution (p ); BBS scores discriminative of functional subgroups, p [123]; and of groups based on ambulatory status p [124].. Predictive validity: handicap situation in stroke survivors 6 months post discharge (multiple regression r 2 = 0.66; p = 0.002) [125]; admission BBS moderately predictive of length of stay (LOS) in rehabilitation unit, r = , p ; r 2 = [49]; r = ; p when controlling for age [54]; r = controlling for age and r = when patients were admitted to rehabilitation within 14 days of stroke [55]; Wee et al. [54] demonstrated admission BBS, age and presence of social support to be predictors of discharge destination. Admission BBS score + presence/absence of family support increased prediction accuracy re: discharge destination [55]; BBS scores at 14, 30 and 90 days post stroke were predictive of motor assessment scale scores at 180 days post stroke event [56].. high specificity (96%) for predicting non-fallers in the elderly population, but 53% sensitivity in positive prediction of falls [120]; Shumway-Cook et al. [126] found BBS related to fall status, p and best predictor thereof (specificity 86%; sensitivity 77%). Responsiveness. At 14 days post stroke event, Mao et al. [56] report a 35% floor effect. A 28.8% ceiling effect was present at 90 days post stroke; greater relative efficiency reported for the BBS vs. BI (1.0 vs. 0.68) and larger effect size at 6 12 weeks post-stroke evaluation suggest less ceiling effect for BBS than BI [112]; Bogle Thorbahn et al. [120] reported 11% ceiling effect.. Wood-Dauphinee et al. [112] reported an effect size of 0.66 for initial 6-week, post-stroke evaluation period, 0.25 for 6 12 weeks and overall effect size of 0.97; Mao et al. [56] reported significant change (p ) between times of assessment (14, 30, 90, 180 days post stroke. Effect sizes were greatest in the interval between 14 and 30 days (0.80) and diminished the further one moved through time from the stroke event ( days effect size = 0.40) [56]; Significant change reported from pre to post intervention testing (p ) [123]. Minimum discernible amount of change calculated as5.8 (90% CI) or 6.9 (95% CI). Tested for stroke patients? Yes [48 50, 54 56, 112, 123, 124] Other formats N/a Use by proxy? N/a represents total assistance and 7 represents total independence. The walking test item receives a score of either 0 or 2. Overall, the disability inventory has a maximum score of 100; 70 points derived from the gross motor index, 30 from the walking index. Assessments are completed by direct observation. Instructions for administration, scoring and interpretation are required to perform the CMSA [58]. In addition to the manual, administration of the test requires a mat or bed and a chair. The assessment takes approximately 1 h to complete [59, 60]. The measurement properties of the Chedoke- McMaster scale are summarized in Table VI. Advantages The Chedoke-McMaster Stroke Assessment was designed for use in conjunction with the FIM and incorporates the same rating scale in its disability inventory. This may provide improved interpretability by using a consistent concept of independence, while improving sensitivity to small physical changes [57]. Limitations The assessor must be familiar with the training manual to effectively administer the CMSA. The

10 324 K. Salter et al. Table VI. Measurement properties of the Chedoke-McMaster Stroke Assessment Scale. Reliability. Gowland et al. [57] reported intra-rater ICC of for impairment inventory; disability test retest of Gowland et al. [57] reported inter-observer reliability ICC = for impairment inventory; disability inventory ICC = 0.99 (0.98 for individual items).. Internal consistency: Valach et al. [61] reported high a reliability coefficients of 0.98 (total scale), 0.94 (impairment) and (disability) Validity. Concurrent validity: Gowland et al. [57] reported impairment scale significantly correlated with Fugl-Meyer test of physical performance (r = 0.95; p ); disability inventory correlated with FIM (r = 0.79; p ); most disability items correlated with BI items ( ) except shoulder pain (r ) and BI items eating and bowel incontinence [61].. Construct validity: various impairment sumscores demonstrated convergence with corresponding subscales on Fugl-Meyer (r = ); similarly, disability inventory subscores correlated with corresponding FIM subscores (r = ); physical impairment items demonstrated divergence with items on FIM; similarly disability items did not correlate with Fugl-Meyer subscales [57, 60].. Predictive validity: Leg + postural control scores (physical impairment scale) of 5 9 showed 100% sensitivity and 80% specificity in the prediction of ambulatory status (independent ambulation Functional Ambulation Classification 5 3) [127]. Responsiveness. FIM and CMSA yielded significant variance ratios (p , one-tailed) from admission to discharge, however CMSA 1.92 times greater relative efficiency [57]. Tested for stroke patients? Yes stroke-specific scale. Other formats N/a Use by proxy? N/a relative complexity and length of administration may make the CMSA less useful for application in a clinical practice setting [60]. The upper extremity tasks included on the test are not related to the assessment of function and, except for items related to transfer and gait, the CMSA is primarily a measure of motor impairment. It is recommended that measures of motor impairment be accompanied by a measure of functional disability such as the BI or FIM.[60] The integrated disability index of the CMSA does not fulfil this role. A regression analysis performed by Valach et al. [61] revealed that although as few as three items of the CMSA disability inventory could be used to predict BI scores, there was still a large portion of unexplained variance. In addition, BI-derived factors of eating/drinking and bowel/ bladder incontinence were shown to add information not covered by the Chedoke-McMaster assessment [61]. Summary Chedoke McMaster Stroke Assessment The ratings of methodological rigour associated with evaluation of the measurement properties of the Chedoke McMaster Stroke Assessment are presented in Table XIV. Practicality Interpretability The use of Brunnstrom staging and FIM scoring increases interpretability and facilitates comparisons across groups of stoke patients. However, the assessment might best be regarded as a measure of motor impairment [60, 61]. Acceptability The CMSA is a long test and is not suited to proxy use. Feasibility The CMSA requires little equipment but is fairly lengthy and complex to administer. It has been tested for use in longitudinal assessment. Functional Independence Measure Developed in 1987, in part as a response to criticism of the Barthel Index, the Functional Independence Measure (FIM) was intended to address issues of sensitivity and comprehensiveness as well as provide a uniform measurement system for use in the medical remuneration system in the USA [27]. Rather than independence or dependence, the FIM assesses physical and cognitive disability in terms of burden of care. The FIM is a composite measure consisting of 18 items assessing six areas of function (self-care, sphincter control, mobility, locomotion, communication and social cognition). These fall into two basic domains; physical (13 items) and cognitive (five items). The 13 physical items are based on those found in the Barthel Index, while the cognitive items are intended to assess social interaction, problemsolving and memory. The physical items are referred to collectively as the motor-fim and the remaining five items are referred to as the cognitive-fim. Each item is scored on a 7-point Likert scale indicative of the amount of assistance required to

11 Outcome measures in stroke rehabilitation 325 perform the item (where 1 = total assistance, 7 = total independence). A simple summed score of is obtained where 18 represents complete dependence/total assistance and 126 represents complete independence. Subscale scores for the physical and cognitive domains may also be used and may yield more useful information than combining them into a single FIM score [62]. Administration of the FIM requires training and certification. The most common approach to administration is direct observation. The FIM takes approximately 30 min to administer and score. The developers of the FIM further recommend that the ratings be derived by consensus opinion of a multidisciplinary team following a period of observation. The measurement properties of the FIM are summarized in Table VII. Advantages The Functional Independence Measure has been found to be as effective as such lengthy measures as the Sickness Impact Profile (SIP) in predicting burden of care following stroke and therefore, just Table VII. Measurement properties of the Functional Independence Measure (FIM). Reliability. In a review of 11 studies, Ottenbacher et al. [69] reported a mean inter-observer reliability value of 0.95; a median test retest reliability of 0.95 and a median equivalence reliability (across versions) of Reliability was higher for items in the motor domain than for those in the social/cognitive domain.. Internal consistency: a of reported at admission vs. discharge [128]; Hsueh et al. [68] reported a of 0.88 to Validity. Predictive validity: FIM admission score predictive of placement after discharge [128, 129]; FIM scores + length of stay predictive of functional gain, p [130]; Granger et al. [63] reported FIM predictive of burden of care assessed in help in minutes/day (p = 0.01).. Concurrent validity: motor-fim showed strong concurrent validity in association with BI Spearman s correlation coefficient ranging from 0.74 (admission) 0.92 (discharge) [68].. Construct validity: Linacre et al. [62] reported 2 distinct aspects of disability within the FIM motor and cognitive function; Cavanagh et al. [65] suggested simple 2-factor model of the FIM not sufficient to describe disability following stroke (66% of variance) may not adequately measure within patient change whereas a 3- factor model (selfcare, cognition and elimination) accounted for more variance (74.2%); use of Rasch transformed scores for comparison of level of ability at the end of treatment show the motor FIM to be a discriminative, ordinal, outcome measure of disability [62, 131].. Construct validity (known groups): FIM scores discriminated between groups based on spinal cord injury severity (p ) and presence of comorbid illness (p ) and right or left-sided involvement in stroke patients both at admission (p ) and discharge (p ) most of this score difference occurred on the communication domain [128]; on admission and discharge, FIM scores discriminated groups with and without neglect (p ; p ) and with or without aphasia (p ; p ) [130]. Responsiveness. Changes in FIM scores from admission to discharge in the expected direction (p ) [128].. Neither floor nor ceiling effects were reported at admission to or discharge from rehabilitation post stroke [114]; 16% ceiling effect reported for motor FIM [131]; van der Putten et al. [1] no significant floor or ceiling effects when administering the FIM to stroke patients effect sizes of 0.30, 0.34 and 0 were reported for the total-fim, motor-fim and cognitive-fim respectively; Wallace et al. [70] reported ES = 0.31 (0.46 in known changers) and AUC ROC curve = 0.675; Dromerick et al. [39] reported SRM = 2.18 from admission to discharge from rehabilitation the FIM detected change in 91/95 individuals including change in 18 patients in whom the BI detected no change (p ) Tested for stroke patients? Yes [1, 29, 62, 63, 70, ] Other formats. Standardized Interview: Daving et al. [133] examined the FIM home interview for intrarater stability kappa values on 17/18 items; motor FIM reliability reported higher than social/cognitive items (K = 0.46 to 0.61). On sequentially separate interviews, self-care items showed K values of ; transfers, locomotion and social/cognitive items were below 0.4 (poor).. Telephone Interview: Smith et al. [132] reported total-fim ICC = 0.97, motor-fim ICC = 0.98, and cognitive- FIM ICC = 0.57 comparing telephone interview with direct observation in the home. Item level agreement was superior for items in the motor domain (Kappa values exceeded 0.45).. Petrella et al. [134] reported good predictive validity (discharge FIM vs. phone FIM at 8 weeks; r = 0.436; p = 0.02) though not as good as observed FIM scores (r = 0.699, p ). Phone FIM showed good concurrent validity with the observed FIM (r = 0.741, p ) and was sensitive to change over time (t = , p = 0.001).. Duncan et al. [135] reported a 46% ceiling effect on the motor FIM when administered by telephone at 6- month follow-up of stroke patients. Use by proxy?. Segal et al. [29] ICC s for patient assessment vs. proxy assessment for both in-person and telephone interviews of 0.90 and 0.91, respectively agreement was much higher for motor-fim than for cognitive- FIM. The authors speculate that, due to the more subjective nature of the cognitive dimension of the FIM, this portion of the scale may not be appropriate for proxy assessment.