RobRoy L. Martin, PT, PhD, CSCS 1 James J. Irrgang, PT, PhD, ATC 2 A Survey of Self-reported Outcome Instruments for the Foot and Ankle Self-reported outcome instruments, which are used to measure change in health status over time, have become increasingly popular. This trend may be related to the greater awareness of evidence-based practice and of outcome-related data collection outlined in the American Physical Therapy Association Vision Statement for Physical Therapy 2020 and in the Guide for Physical Therapist Practice. 68 Self- The information acquired from self-reported outcome instruments is useful only if there is evidence to support the interpretation of obtained scores. To properly interpret scores, there should be evidence for content validity, construct validity, reliability, and responsiveness. Evidence regarding score interpretation must also contain a description of the applicable test conditions, including information about the characteristics of subjects, timing of data collection, and construct of change. The objective of this review was to identify self-reported outcome instruments that have evidence to support their usefulness for assessing the effect of treatment directed at individuals with foot and ankle-related pathologic conditions in an orthopaedic physical therapy setting. In addition, we provide specific information that will allow clinicians and researchers to select an appropriate instrument and properly interpret the obtained scores. Fourteen self-reported outcome instruments that met the objective of this review were identified. Five instruments, the Foot and Ankle Ability Measure, Foot Function Index, Foot Health Status Questionnaire, Lower Extremity Function Scale, and Sports Ankle Rating System quality of life measure, satisfied all 4 categories of evidence (content validity, construct validity, reliability, and responsiveness) outlined herein. J Orthop Sports Phys Ther 2007;37(2):72-84. doi:10.2519/ jospt.2007.2403 t Key Words: ankle, foot, outcome instruments, reliability, responsiveness, validity reported outcome instruments have been used by clinicians and by researchers to assess the effect of treatment interventions directed at individuals with foot and ankle-related pathologic conditions and subsequent impairments. If treatment outcomes are to be appropriately measured, clinicians and researchers need to select a suitable instrument and properly interpret the obtained scores. There are several factors that must be considered when selecting and using an instrument to assess the outcome of treatment. These factors are not universally defined. However, a format to help in critically reviewing instruments for their potential usefulness has been outlined. 55 This format includes determining the instrument type, the psychometric evidence available to support score interpretation, and the characteristics of subjects included in the studies that offer this evidence. To properly interpret scores, there should be evidence for content validity, construct validity, reliability, and responsiveness. Instrument Type The 4 basic types of self-reported outcome instruments are generic, disease specific, region specific, and patient specific. Each type of instrument has a unique purpose and has advantages and disadvantages that affect the instrument s potential usefulness. A generic instrument may be used for different patients and conditions; however, it may not capture important aspects of a particular disease process. Conversely, a disease-specific instrument is designed to be sensitive to unique characteristics of a distinct disease state. 66 A region-specific instrument contains items specific to only 1 body part and can be used with several different disease states affecting this specific region. A patient-specific instrument attempts to make the scale as meaningful and potentially sensitive as possible to each particular individual, but comparisons cannot be made across patients or subjects. 83 Evidence for Content Validity The specific items on the instrument and what they measure determine the evidence that is based on content. The International Classification of Functioning, Disability and Health 86 model can be used to define item content. According to this model, items can potentially measure the domains of (1) body structure and function and (2) activity and participation. Example measures of body structure and function include symptoms (eg, pain) and signs (eg, strength and range of motion). Activities are tasks executed by an individual, and participation is involvement in life situations. There is no evidence that scores from items that assess body structure and function can 1 Assistant Professor, Department of Physical Therapy, Duquesne University, Pittsburgh, PA. 2 Director of Clinical Research, Department of Orthopaedic Surgery, University of Pittsburgh School of Medicine, Pittsburgh, PA. Address correspondence to Dr RobRoy Martin, Assistant Professor, Department of Physical Therapy, Duquesne University, 114 Rangos School of Health Sciences, Pittsburgh, PA 15282. E-mail: martinr280@duq.edu 72 february 2007 volume 37 number 2 journal of orthopaedic & sports physical therapy
be combined with scores from items that assess activity and participation, as relationships between (1) body structure and function and (2) activity and performance have not been reported in the literature to date. 4,21,22,42,45,65,67,73,75,85 Although issues of body structure and function may be important, an individual s ability to execute tasks and to participate in life situations may be more useful when describing functional outcomes. Overall, an instrument should contain items that assess issues that are important to the intended patient population without item irrelevance and without item underrepresentation. 6 Item underrepresentation occurs when meaningful aspects of the construct being measured are not included in the instrument. Item irrelevance occurs when something other than the intended construct is measured. Internal structure relates to how well the items relate to each other and to the construct being measured. Therefore, internal structure can be used to assess item irrelevance. 60 If only 1 construct influences an individual s response to items on the instrument, all items should relate to this construct, and the instrument should display evidence of unidimensionality using factor analysis. Assessment of internal consistency can be used to provide evidence for content. Internal consistency is the degree to which all items on the instrument measure the underlying condition. It is essentially a measure of how consistently an individual responds among items and is typically estimated using the a level. The a relationship can suffer if individual items on the instrument do not measure the same construct. Evidence for Construct Validity While evidence for content relates directly to what the items measure, evidence for construct validity assesses how the scores on the instrument relate to other measures of the construct. 60 A strong relationship between the scores on the instrument and other measures of the same or a related construct provides evidence for convergent validity; little or no relationship between the scores on the instrument and other measures of distinctly different constructs provides evidence for divergent validity. 60 Hypothesis testing to determine if the instrument performs as expected is another method to offer evidence for construct validity. For example, the hypothesis that there will be score differences between younger and older individuals can be tested to offer evidence of construct validity. Evidence for Reliability Assessment of the instrument s stability of scores offers evidence for reliability. The stability of a score with repeated measurements over time is defined as test-retest reliability. It is assessed by having an individual complete the instrument 2 or more times during a period when the individual s condition is expected to remain stable. Intraclass correlation coefficient and SEM values are commonly used to estimate test-retest reliability. While intraclass correlation coefficient values greater than 0.70 may often be considered acceptable, 47 an intraclass correlation coefficient value does not quantify the amount of error associated with repeated measurements. Because the SEM quantifies this error, the SEM may be more useful when interpreting the reliability of repeated measurements. The SEM can also be used to calculate the minimal amount of change that exceeds measurement error at a set confidence level. This value, defined as the minimum detectable change (MDC), can be calculated using different methods. 28,35,77,85 If intraclass correlation coefficient, SEM, and MDC values are to be properly interpreted, the study conditions, including the time between measurements, should be similar to those of the intended clinical environment. Evidence for Responsiveness Whereas evidence for reliability relates to stability of scores, evidence for responsiveness assesses the ability of the instrument to detect changes in an individual s status over time. The score on an outcome instrument should increase and decrease if the individual s condition improves and worsens, respectively. 24 A construct of change, such as a patient indicating that he or she has improved, a clinician indicating that a patient has improved, or improvements on established clinical measures, is used to demonstrate that change has in fact occurred. 11 The change in score is then linked to the construct of change in statistical analysis. The focus of the analysis associated with evidence for responsiveness can be at the group level or at the individual level. When the focus of the analysis is at the group level, statistics such as mean differences, effect size, and Guyatt responsiveness index are reported. 10 Group statistics provide useful information for a group of individuals. However, group-level analysis does not provide meaningful information for interpreting score changes of a particular individual. When the focus of the analysis is at the individual level, receiver operating characteristic curves are used to determine the cutoff value for the change score that has the highest sensitivity and specificity for change. 10 Sensitivity of change is defined as the proportion of subjects who improved according to a criterion measure of change and who have a change score above a cutoff point. Specificity of change is defined as the proportion of subjects who did not improve according to a criterion measure of change and who have a change score below a cutoff point. 24 The challenge is to select a cutoff point that best discriminates between those who improved and those who did not improve. This cutoff value is referred to as the minimum clinically important difference (MCID). 10 The results of a responsiveness study at the individual level can be used to determine if a particular individual has improved or not, provided that conditions (eg, patient group, type of treatment, and timing of data collection) are similar to those of the responsiveness study. 10 journal of orthopaedic & sports physical therapy volume 37 number 2 february 2007 73
Characteristics of Subjects In addition to identifying the type of instrument and the evidence available regarding score interpretation, the characteristics of subjects included in the studies to support the interpretation of scores need to be defined. The characteristics of the population among whom the instrument is intended to be used should match the characteristics of subjects included in these studies. The obtained scores may be better generalized to and interpreted among a range of clinical environments if the instrument has evidence regarding score interpretation among individuals with a range of foot and ankle-related pathologic conditions. The objective of this review was to identify self-reported outcome instruments that have evidence to support their usefulness for assessing the effect of treatment interventions directed at individuals with foot and ankle-related pathologic conditions in an orthopaedic physical therapy setting. In addition, this review provides specific information that will allow clinicians and researchers to select an appropriate instrument and properly interpret the obtained scores. METHODS A literature search was performed in MEDLINE, Health and Psychosocial Instruments, CI- NAHL, Healthstar, and SPORTDiscus. All references in the databases were included up to the first week of April 2006. The searches were performed using the following key words: foot, ankle, index, measure, instrument, scale, questionnaire, reliability, validity, and responsiveness. The results from the searches were then combined using the Boolean connectors or and and as follows: (1) foot or ankle, and (2) index or measure or instrument or scale or questionnaire, and (3) reliability or validity or responsiveness. From each database, titles, abstracts, and articles were reviewed from each hit to identify self-reported outcome instruments that have evidence to support their use for assessing the effect of treatment interventions directed at individuals with foot and ankle-related pathologic conditions in an orthopaedic physical therapy setting. Identified instruments were then reviewed, and evidence regarding score interpretation was retrieved. Specific information that was retrieved included the following: (1) type of instrument, (2) item content, (3) evidence for content validity, (4) evidence for construct validity, (5) evidence for reliability, (6) evidence for responsiveness, and (7) characteristics of subjects included in studies that offer evidence regarding score interpretation. RESULTS The numbers of hits for each electronic database were as follows: MEDLINE (275 hits), Health and Psychosocial Instruments (1 hit), CI- NAHL (285 hits), Healthstar (147 hits), and SPORTDiscus (79 hits). Instruments that did not fit the objective of the search were omitted from this review; these included instruments that were developed for individuals with diabetes mellitus, 8,16,39,49,74,80,81 vascular disorders, 19,23,59,61 and neurological disorders. 3,27 The Western Ontario and McMaster Universities Osteoarthritis Index was excluded because it pertains to individuals with hip and knee osteoarthritis 12 and is without evidence to support its use among individuals with foot and ankle-related pathologic conditions. 55 The Medical Outcomes Study 36-Item Short-Form Health Survey 79 (SF-36) was also excluded from this review. Although the SF-36 is one of the most widely accepted generic self-report instruments with evidence to support its use for diverse pathologic conditions, 15,32,56-58,76,79,82 information to interpret scores from individuals with foot and ankle-related impairments is lacking. 55 Also omitted were instruments related to the outcome of treatment for patients with nail and skin disorders 9,20,50,51 and a questionnaire to assess the usability of orthopaedic shoes. 38 The clinical scoring system of the American Orthopaedic Foot and Ankle Society, 25,44 the rating scale of the Japanese Orthopaedic Association, 62,63 the Foot and Ankle Severity Scale, 52 and the Ankle Activity Scale 30 were excluded because they are primarily completed by clinicians. In addition, the Foot Disability Questionnaire 29 was omitted because it was not developed to measure change but rather was developed to identify and describe disability caused by foot pain. This literature search identified the following 11 instruments that met the objective of this review: American Academy of Orthopaedic Surgeons lower limb outcomes assessment instruments Foot and Ankle Module (AAOS-FA), 40 Ankle Osteoarthritis Scale (AOS), 26 Foot and Ankle Ability Measure (FAAM), 54 Foot and Ankle Disability Index (FADI), 31 Foot Function Index (FFI), 1,17,46,72 Foot Health Status Questionnaire (FHSQ), 13 Juvenile Arthritis Foot Disability Index (JAFI), 7 Lower Extremity Function Scale (LEFS), 2,33,78 Maryland Foot Score (MFS), 36 Rowan Foot Pain Assessment Questionnaire (ROFPAQ), 71 and Sports Ankle Rating System quality of life (QOL) measure. 84 The reference lists in the articles describing these 11 instruments were reviewed to identify instruments missed by this search. An additional study 14 was identified to support the use of the LEFS. Our search identified 2 articles that conducted relevant literature searches on the topic of outcome rating scales. 18,34 These articles were reviewed to identify instruments that may have been missed in our search. Button and Pinney 18 identified 49 instruments, but only 5 of them had evidence to support their use among individuals with foot and ankle-related impairments. Two of these instruments not identified in our literature search, the Olerud Scoring Scale 64 (OSS) and the Foot and Ankle Outcome Score (FAOS), 69 were added to this review. Also added to this review was the Karlsson Ankle Function Score (KAFS), 43 which was identified by Haywood et al 34 as a self-report instrument with evidence to support its use. To ensure that this literature search 74 february 2007 volume 37 number 2 journal of orthopaedic & sports physical therapy
Table 1 Self-reported Outcome Instruments With Evidence to Support Their Use for Assessing the Effect of Treatment Directed at Individuals With Foot and Ankle- Related Pathologic Conditions in an Orthopaedic Physical Therapy Setting Instrument Evidence for score interpretation Content Construct Reliability Responsiveness AAOS-FA General foot and ankle-related diagnoses 40 t t t AOS Osteoarthritis and posttraumatic arthritis 26 t t FAAM General leg, foot, and ankle-related disorders 54 t t t t FADI Chronic ankle instability 31 t t t FAOS Lateral ankle ligament reconstruction 69 t t t FFI Rheumatoid arthritis 17,72 t t t t Nontraumatic foot or ankle problem 1,46 FHSQ Skin, nail, and musculoskeletal conditions 13 t t t Plantar fasciitis 48 JAFI Juvenile rheumatoid arthritis 7 t t t KAFS Lateral ankle ligament reconstruction 43 t LEFS Disorders of the hip, knee, leg, foot, and ankle 14 t t t t Acute ankle sprains 2,78 t t t MFS Calcaneal fractures 36 t OSS Ankle fractures 64 t was as complete as possible, members of the Foot and Ankle Special Interest Group of the Orthopaedic Section of the American Physical Therapy Association were given the results of our search. They suggested inclusion of a function scale by Kaikkonen et al 41 and a foot impact scale by Helliwell et al 37 in our review. The function scale by Kaikkonen et al 41 was not included because it is primarily completed by clinicians. Based on the description of its development, the foot impact scale by Helliwell et al 37 has potential to be a disease-specific instrument for individuals with rheumatoid arthritis. However, evidence to support the use of the final version of the 51-item scale is limited, and it was not included in this review. An additional study 48 was identified that offered evidence to support the use of the FHSQ. Fourteen instruments were identified for review. Table 1 lists these instruments, the evidence to support their use, and the conditions of subjects included in the analyses. Detailed information regarding the available evidence for content validity, construct validity, reliability, and responsiveness is given in the subsections that follow and in Table 2. AAOS Lower Limb Outcomes Assessment Instruments Foot and Ankle Module The AAOS has developed outcome assessment tools designed for patients of all ages with musculoskeletal conditions in all body regions. 40 One of these tools is a region-specific module that was developed for individuals with foot and ankle-related diagnoses. The Foot and Ankle Module consists of a Lower Limb Core Scale, a Global Foot and Ankle Scale, and a Shoe Comfort Scale. The Lower Limb Core Scale contains 9 items that assess symptoms and functional status. The Global Foot and Ankle Scale combines 20 items available evidence ROFPAQ Chronic foot and ankle pain 71 t t t QOL Acute grade II ankle sprains 84 t t t t Abbreviations: AAOS-FA, American Academy of Orthopaedic Surgeons lower limb outcomes assessment instruments Foot and Ankle module; AOS, Ankle Osteoarthritis Scale; FAAM, Foot and Ankle Ability Measure; FADI, Foot and Ankle Disability Index; FAOS, Foot and Ankle Outcome Score; FFI, Foot Function Index; FHSQ, Foot Health Status Questionnaire; JAFI, Juvenile Arthritis Foot Disability Index; KAFS, Karlsson Ankle Function Score; LEFS, Lower Extremity Function Scale; MFS, Maryland Foot Score; OSS, Olerud Scoring Scale; ROFPAQ, Rowan Foot Pain Assessment Questionnaire; QOL, Sports Ankle Rating System quality of life measure. t pertaining to pain, function, stiffness, swelling, and giving way. The Shoe Comfort Scale contains 5 items assessing the ability to comfortably wear different shoe types. 40 The AAOS assessment tools and scoring algorithms and a description of normative data can be found on the Web. 5 Evidence for content validity, construct validity, and reliability is available for the Foot and Ankle Module as part of a larger study 40 that also investigated the Hip and Knee Core Scale and the Sports/Knee Module. Two hundred five subjects participated in the study. Their mean age was 48 years (age range, 21-85 years). Seventy subjects were identified as having a diagnosis in 1 of 23 foot and ankle categories. These diagnosis categories included the following: bunions, plantar fasciitis, hindfoot arthrodesis, ankle arthrodesis, rheumatoid foot deformities, claw toe, hammer toe, and Morton neuroma. Twenty-nine subjects participated journal of orthopaedic & sports physical therapy volume 37 number 2 february 2007 75
Table 2 Description of The Evidence Available To Support The Use of Self-reported Outcome Instruments Directed at Individuals With Foot And Ankle-related Pathologic Conditions in an Orthopaedic Physical Therapy Setting Instrument Content Validity Construct Validity Reliability Responsiveness AAOS-FA 40 AOS 26 FAAM 54 FADI 31 High internal consistency (a range,.83 to.93) 1-factor structure and high internal consistency (a,.96 and.98, respectively) FAOS 69 High internal consistency (a range,.88 to.94) FFI 17 High internal consistency (a range,.73 to.96) FHSQ 13 Appropriate factorial structure and high internal consistency (a,.85 and.88, respectively) Correlation to physician assessment and SF-36 score (r, 0.39 and 0.65, respectively) Correlation to heel lift ratio, Western Ontario and McMaster Universities Osteoarthritis Index, and SF-36 physical function subscale (r, 0.65, 0.90, and 0.66, respectively) No correlation to SF-36 physical function subscale and physical component summary score (r, 0.78 and 0.84, respectively)* Low correlation to SF-36 mental health subscale and mental component summary score* Significantly lower scores on the involved versus uninvolved side (P.001)* Correlation to score on the Ankle Function Scoring Scale by Karlsson and Peterson 43 (r range, 0.58 to 0.68) Correlation to the number of painful foot joints and the time to walk 15.24 m (r, 0.33 and 0.54, respectively) Discrimination among individuals with minor pathologic conditions, acute disease, and footwear problems JAFI 7 High internal consistency Consistent pattern of increasing Juvenile Arthritis Foot Disability Index scores with increasing joint impairment, Childhood Self Assessment Questionnaire scores, and self-rated foot problems* Lower item disability scores among control subjects* KAFS 43 Correlation to St Pierre Functional Score (r, 0.95)* Correlation to radiographic ankle evaluation (r range, 0.72 to 0.84)* Correlation to self-rating (r, 0.98)* ICC range, 0.79 to 0.99 during 24 h ICC range, 0.94 to 0.97 during 1 wk ADL subscale ICC, 0.87 (SEM, 2.1) Sports subscale ICC, 0.89 (SEM 4.5) MDC 95 of 6 points on the ADL subscale and 12 points on the sports subscale during 9 wk Treatment group during 1 wk: ADL subscale ICC, 0.89 (SEM, 2.6)* Sports subscale ICC, 0.84 (SEM, 5.3)* Nontreatment group during 6 wk: ADL subscale ICC, 0.93 (SEM 1.3)* Sports subscale ICC, 0.92 (SEM, 1.3)* ICC range, 0.70 to 0.92 during 29 d ICC range, 0.70 to 0.87 during 1 wk ICC range, 0.74 to 0.92 during 1 wk Good to very good during 1 wk Minimum clinically important differences of 8 and 9 points for the ADL and sports subscales, respectively, distinguishing between those improved versus not improved after 4 wk of physical therapy* Significantly different change in scores during 4 wk in the group expected to change (P.001)* Significantly increased scores after 4 wk of rehabilitation (P.007) Except for the disability subscale, correlation between score changes and changes in the number of painful foot joints during 6 mo (r range, 0.45 to 0.33) Significantly changed scores during 4 wk among subjects with plantar fasciitis who received orthotics 48 continued on next page in the test-retest reliability analysis. The physician assessment that was completed as part of the evidence for construct validity involved a summative score based on physicians ratings of the ability of subjects to perform the following tasks: (1) work, school, or homemaking activities; (2) selfcare; and (3) recreational activities. The scores from each of the 3 task ratings were based on 6-point scales that ranged from not limited at all to extremely limited or unable. The SF-36 score that was used represented the unweighted mean of the bodily pain, physical function, and role physical subscales. 40 76 february 2007 volume 37 number 2 journal of orthopaedic & sports physical therapy
Description of The Evidence Available To Support The Use of Self-reported Outcome Instruments Directed at Individuals With Foot And Ankle-related Pathologic Conditions in an Orthopaedic Physical Therapy Setting (continued) Instrument Content Validity Construct Validity Reliability Responsiveness LEFS General disorders 14 Acute ankle sprains 2 MFS 36 OSS 64 ROFPAQ 71 QOL 84 High internal consistency (a =.96) High internal consistency (a range,.81 to.89) High internal consistency (a range,.85 to.91) Correlation to SF-36 physical function subscale and physical component summary score (r, 0.80 and 0.64, respectively)* No correlation to SF-36 mental health subscale or mental component summary score* Higher scores for chronic versus acute conditions (P =.03)* Lower scores for those with versus without recent surgery (P =.006)* Significantly lower scores among individuals requiring gait aid and those with more acute injuries (P.009)* Correlation to weight-bearing status and range of motion measures (r, 0.36 and 0.48, respectively)* Correlation to SF-36 pain and physical function subscales (r, 0.64 and 0.78, respectively) Significant difference in scores based on the patient s rating of function, range of motion, dislocation, and osteoarthritis (range, P.05 to P.001) Correlation to pain subscale (Spearman rank correlation coefficient range, 0.69 to 0.88)* Correlation to Headache Scale (Spearman rank correlation coefficient range, 0.15 to 0.49)* Correlation to anxiety subscale (Spearman rank correlation coefficient, 0.49)* Significantly different initial scores among the group with ankle sprain versus control subjects (P.001)* Correlation to objective measures on the clinical rating score (r range, 0.31 to 0.87) and on the single numeric evaluation across measurement intervals (r range, 0.68 to 0.81)* ICC, 0.86 MDC 90 of 9 points during 24-48 h ICC, 0.87 (SEM, 4) MDC 90 of 9 points during 1 wk Spearman rank correlation coefficient range, 0.82 to 0.93 during 24 h Coefficients of variation 1% during 1 wk among control subjects Minimum clinically important difference of 9 points distinguishing those expected to improve versus not to improve Significant difference between score changes at 1 versus 2 wk when comparing subjects 6 versus 6 d after injury (P =.002) Significantly different scores from the group with ankle sprain across the 4-wk evaluation interval (P.001) Abbreviations: AAOS-FA, American Academy of Orthopaedic Surgeons lower limb outcomes assessment instruments Foot and Ankle module; ADL, activities of daily living, ICC, intraclass correlation coefficient; AOS, Ankle Osteoarthritis Scale; FAAM, Foot and Ankle Ability Measure; FADI, Foot and Ankle Disability Index; FAOS, Foot and Ankle Outcome Score; FFI, Foot Function Index; FHSQ, Foot Health Status Questionnaire; JAFI, Juvenile Arthritis Foot Disability Index; KAFS, Karlsson Ankle Function Score; LEFS, Lower Extremity Function Scale; MDC, minimum detectable change; MFS, Maryland Foot Score; OSS, Olerud Scoring Scale; QOL, Sports Ankle Rating System quality of life measure; ROFPAQ, Rowan Foot Pain Assessment Questionnaire; SF-36, Medical Outcomes Study 36-Item Short-Form Health Survey. *P,.05. AOS The AOS was developed as a disease-specific measure for individuals with ankle osteoarthritis. 26 The AOS was created from the FFI by adapting the responses so that they related to the ankle and by eliminating the FFI activity limitation subscale. Therefore, the AOS consists of two 9-item subscales for pain and for disability. As with the FFI, visual analog scales are used, with a lower score representing a higher level of function. 26 Evidence for reliability and construct validity has been reported for the AOS. 26 Twenty-eight subjects, 17 with posttrau- journal of orthopaedic & sports physical therapy volume 37 number 2 february 2007 77
matic ankle arthritis and 11 with primary ankle osteoarthritis, participated in this research. These subjects had a mean age of 52.7 years (age range, 16-79 years). 26 The heel lift ratio, used to provide evidence for convergent validity evidence, was calculated as the number of completed heel lifts on the uninvolved side divided by the number of heel lifts completed on the involved side. The Western Ontario and McMaster Universities Osteoarthritis Index was also used to provide evidence for convergent validity. However, this index was not developed for individuals with foot and ankle problems; therefore, these results should be interpreted cautiously. The negative correlation between the AOS and the SF-36 results from the fact that a lower score represents a higher level of function for the AOS, while a higher score represents a higher level of function for the SF-36. The correlation (r = 0.34) between the AOS and the SF- 36 pain subscales was not significant 26 ; therefore, this information is not listed as supporting evidence in Table 2. FAAM The FAAM was developed as a regionspecific instrument to comprehensively assess physical performance among individuals with a range of leg, foot, and ankle musculoskeletal disorders. 54 The instrument is divided into 2 separately scored subscales, the 21-item activities of daily living and the 8-item sports subscales. Items are scored using a Likert response format, with a higher score representing a higher level of ability. 54 Evidence for content validity, construct validity, reliability, and responsiveness was obtained among 243 subjects who were separated into a group expected to change and a group expected to remain stable. 54 The 164 subjects in the group expected to change were receiving formal physical therapy for 1 of the following diagnoses: joint or limb pain (n = 55), sprain or strain (n = 47), fractures (n = 28), plantar fasciitis (n = 22), bunion (n = 3), Achilles rupture (n = 2), and other or missing diagnoses (n = 7). Their mean age was 41.1 years (age range, 9-75 years). The 79 subjects in the group expected to remain stable had not received formal physical therapy for their leg, foot, or ankle disorder in more than 1 year. Their mean age was 45.2 years (age range, 19-86 years), and they had the following diagnoses: joint or limb pain (n = 39), sprain or strain (n = 24), fractures (n = 5), plantar fasciitis (n = 5), bunion (n = 1), and other or missing diagnoses (n = 5). Evidence for reliability was obtained using the group that was expected to remain stable. Group-level assessment of responsiveness was performed using the changes in scores from the group expected to change and from the group expected to remain stable. Individuallevel assessment of responsiveness was performed using the group expected to change, which was dichotomized into those who improved and those who did not improve after 4 weeks of physical therapy. Decisions regarding changes in individuals scores and status can be interpreted by using the MDC that was calculated at the 95% confidence level and the MCID values. 54 FADI The FADI is a former version of the FAAM. The 2 instruments are identical except for an additional 5 items found on the FADI. Four of these items assess pain, and the fifth item evaluates an individual s ability to sleep. 53 These 5 items were subsequently removed because of factor analysis and item response theory results. 54 Therefore, the FADI is composed of a 26-item activities of daily living subscale and an 8-item sports subscale. 53 Evidence for construct validity, reliability, and responsiveness was obtained among 31 subjects with chronic ankle instability and among 19 healthy subjects. 31 The mean SD age among these 50 subjects was 21.5 3.6 years. All subjects completed the FADI 3 times (at weeks 1, 2, and 7), with the subjects with chronic ankle instability completing the FADI for their involved and uninvolved sides. Sixteen subjects with chronic ankle instability completed a 4- week rehabilitation program; test-retest reliability was completed among this subgroup between weeks 1 and 2. Data from the 15 subjects with chronic ankle instability who did not receive treatment were used to determine test-retest reliability during a 6-week period. 31 FAOS The FAOS is a region-specific instrument that is intended to evaluate symptoms and functional limitation in individuals with generalized foot and ankle disorders. 69 Items for the Foot and Ankle Outcome Score were adapted from the Knee Injury and Osteoarthritis Outcome Score. 70 The FAOS is composed of the following 5 subscales: pain (9 items), other symptoms (7 items), activities of daily living (17 items), sports and recreational activities (5 items), and foot and ankle-related QOL (4 items). Subscales are scored separately using a Likert response format, with higher scores indicating higher levels of function. 69 Evidence for content validity, construct validity, and reliability is available from a study 69 involving 213 subjects who underwent lateral ankle ligament reconstruction. These subjects had a mean age of 39.6 years (age range, 20-60 years) and a mean of 12 years (range, 3-24 years) between surgery and data collection. Among 38 randomly selected subjects, test-retest reliability during a mean of 29 days (range, 12-68 days) was assessed. 69 FFI The FFI was developed as a region-specific instrument for older persons with foot-related pathologic conditions. 17 Although it is commonly considered disease specific for rheumatoid arthritis, the authors note that there is nothing in the design of the FFI that is specific to rheumatoid arthritis. The FFI consists of 23 items grouped into 3 subscales, including activity limitation (5 items), disability (9 items), and pain (9 items) subscales. Visual analog scales for each item are used to score the subscales sepa- 78 february 2007 volume 37 number 2 journal of orthopaedic & sports physical therapy
rately. The subscale scores are averaged together to obtain a total mean score. A lower FFI score represents a higher level of function. Evidence regarding score interpretation includes content validity, construct validity, reliability, and responsiveness. 17 This evidence was obtained among 87 subjects with the diagnosis of stage I or stage II rheumatoid arthritis and with a mean age of 61 years (age range, 24-79 years). Factor analysis provided evidence that the FFI measured 2 unique aspects of foot function; the pain and disability subscales each loaded on a separate factor. Factor analysis demonstrated that the items from the activity limitation subscale loaded on 2 unique factors, making the contribution of this subscale difficult to interpret. In addition, this subscale had low internal consistency values. Cronbach a values for the total mean score, disability subscale, and pain subscale were 0.96, 0.93, and 0.95, respectively, compared with 0.73 for the activity limitation subscale. Correlations between (1) the number of painful hand joints and (2) the grip strength and FFI scores were used for evidence of divergent validity. 17 These correlations, with r values ranging between 0.37 and 0.48, were statistically significant (P,.001). Related to this finding, one might anticipate that an individual with rheumatoid arthritis would have involvement of the joints of the hand and feet. With respect to responsiveness, the correlation between the change in the disability subscale score and the change in the number of painful foot joints during 6 months was not significant (r = 0.11) 17 ; because this information was contradictory to supporting evidence, it was not included in Table 2. Evidence for reliability using a Likert response format version of the FFI is available from a study 1 that included 96 individuals with nontraumatic foot or ankle problems. The mean age of these subjects was 52.8 years (age range, 19-74 years). The reliability analysis found acceptable reliability, with a mean of 68.8% of subjects providing retest answers within 1 response choice of their initial answer. Ceiling effects were associated with many of the items, which may prove problematic when the FFI is used among individuals who are functioning at a high level of ability. A Dutch version of the FFI with a Likert response format showed evidence regarding score interpretation. 46 Evidence for content validity, construct validity, reliability, and responsiveness for the pain and disability subscales and for the total mean score was obtained among 206 subjects with a mean SD age of 61 10 years. All subjects had nontraumatic forefoot pain. 46 Other investigations of the FFI demonstrated that subjects with rheumatoid arthritis could differentiate the effects of pathologic conditions on their right and left feet using separate questionnaires 72 ; that study showed that internal consistency, factor analysis, and test-retest reliability did not significantly change when subjects completed the FFI for each foot on 2 occasions 8 days apart. FHSQ The FHSQ is a region-specific instrument that was developed for individuals undergoing surgical treatment in a podiatry practice for common foot conditions. 13 The FHSQ consists of 4 subscales with a total of 13 items scored using a Likert response format. These 4 subscales represent the following 4 domains: pain (4 items), function (4 items), footwear (3 items), and general foot health (2 items). A computer program takes the score from each item and transforms the scores into separate scores for the relevant domains. Scores from each subscale range from 0 to 100, with a higher score representing a better scenario. 48 Evidence for content validity, construct validity, and reliability is available. 13 This evidence was obtained among 111 subjects with a mean SD age of 54 20 years. Subjects had conditions that represented those seen in a podiatry practice, including skin (47.7%), nail (8.1%), musculoskeletal (36.0%), and miscellaneous and other (8.2%) disorders. Evidence for construct validity was obtained, as the FHSQ was able to discriminate among individuals with minimal pathologic conditions (eg, skin conditions), acute inflammatory conditions, and footwear problems. A separate study 48 compared the responsiveness of the FHSQ with that of the FFI among 17 subjects with plantar fasciitis. These subjects had a mean SD age of 45 10 years. The 2 instruments were completed before and at 4 weeks after receiving foot orthotics. The results of the study demonstrated that the changes in the FHSQ scores were greater than the changes in the FFI scores. JAFI The JAFI was designed to be a diseasespecific instrument assessing foot-related disability among children and adolescents with juvenile rheumatoid arthritis. 7 The JAFI consists of 27 items graded using a 5-point Likert scale. These 27 items are divided into 3 separately scored dimensions of impairment (9 items), activity limitation (14 items), and participation restriction (4 items). A lower JAFI score represent a higher level of function. 7 Evidence is available for content validity, construct validity, reliability, and responsiveness. 7 This evidence was obtained among 36 children and adolescents diagnosed as having juvenile rheumatoid arthritis that involved the foot. Twentynine healthy subjects were also evaluated. Lower extremity joint impairment in the hips, knees, ankle, hindfoot, midfoot, and forefoot was used as evidence for convergent validity. The presence of capsular swelling, tenderness or pain, and loss of range of motion was used to assess joint impairment. A total joint impairment score was derived by combining the scores from each joint. Also used to provide evidence for validity were the lower extremity section of the Childhood Self Assessment Questionnaire and a 5-point self-rating regarding foot-related restriction in activity participation. Spearman rank correlation and weighted a coefficients were used to assess internal consistency and test-retest reliability, respectively. journal of orthopaedic & sports physical therapy volume 37 number 2 february 2007 79
KAFS The KAFS is a disease-specific measure that was developed for individuals who have sustained a lateral ankle injury. This scale contains 8 items covering content related to instability, pain, swelling, functional activities, and need for support. The items are scored using a Likert response format with the following point value ranges for each item: instability (0-25 points), pain (0-20 points), swelling (0-10 points), stiffness (0-5 points), stair-climbing (0-10 points), running (0-10 points), work activities (0-15 points), and use of ankle support (0-5 points). Values for each of these items are summed, with a higher score representing a higher level of function. The KAFS has evidence related to construct validity from among 148 individuals who underwent surgery for chronic ankle instability. 43 The mean length of follow-up after surgery was 6 years (range, 2-12 years). Information regarding the age of patients was not provided. Convergent evidence of validity was obtained using the St Pierre Function Scale, by a self-rating of function on a visual analog scale, and by means of radiographically measured anterior talar translation and talar tilt. The St Pierre Functional Scale is untested, which reduces the confidence in these results. The negative correlation between the KAFS scores and the radiographic measures indicated that less movement as shown on radiography was associated with a higher KAFS score. LEFS The LEFS was created to be a broad region-specific measure appropriate for individuals with musculoskeletal disorders of the hip, knee, ankle, or foot. 14 The LEFS consists of 20 items that specifically address the domains of activity and participation. The scale uses a Likert response format, with a higher score representing a higher level of ability. 14 Evidence for content validity, construct validity, reliability, and responsiveness was obtained among 107 subjects with a mean SD age of 44.0 16.2 years. 14 Subjects had impairments in the following regions: hip (n = 2), knee (n = 71), ankle (n = 14), foot (n = 8), thigh (n = 1), leg (n = 8), and missing information (n = 3). 14 Relative to responsiveness, the MCID was determined using the clinicians prognostic ratings, dichotomized based on which subjects the clinicians thought would and would not undergo an important change during a 3-week rehabilitation period. The MDC at the 90% confidence level and the MCID are based on an 80-point score and allow decisions to be made regarding changes in individuals scores and status. Evidence to support the use of the LEFS was obtained in a study 2 of individuals with ankle sprain. This study included 55 athletic subjects with a mean age of 20 years who sustained an ankle sprain less than 14 days before study enrollment. The LEFS was administered at weekly intervals during a 6-week period in which the subjects were receiving rehabilitation. Test-retest reliability during 1 week was calculated using the scores obtained between the fourth and fifth weeks of treatment. Evidence for construct validity was obtained using the weight-bearing status as defined on a 5-point ordinal scale (normal, antalgic, able to walk 4 steps, able to walk less than 4 steps, and non-weight bearing). Dorsiflexion and plantar flexion range of motion also provided evidence for convergent validity. The MDC was calculated at the 90% confidence level based on the 80-point score. 2 The LEFS has undergone advanced analysis using item response theory 78 and computerized adaptive testing. 33 Item response theory allowed generation of an item hierarchy and growth curves for individuals with ankle sprains. 78 Three body part-specific computerized adaptive testing models produced precise measures of functional status with good discriminative ability for individuals with hip, knee, ankle, or foot impairments. 33 MFS The Maryland Foot Score is a regionspecific measure that assesses foot pain and function in relation to pain, gait, stability, limp, range of motion, ability to climb stairs, and need for support. Scores are based on a 100-point scale and are commonly graded as follows: excellent (90-100 points), good (75-89 points), fair (51-74 points), and failure (<50 points). 36 Evidence for construct validity is available based on data from 25 subjects with intra-articular calcaneal fractures. The mean age at the time of injury was 44 years (age range, 22-65 years), and the mean follow-up was 2.5 years (range, 1-4 years). 36 OSS The OSS was developed as a disease-specific instrument to assess the outcome of treatment for individuals with ankle fractures. The scale contains 9 items scored in a Likert response format with the following point ranges for each item: pain (0-25 points), stiffness (0-10 points), swelling (0-10 points), stair-climbing (0-10 points), running (0-5 points), jumping (0-5 points), squatting (0-5 points), use of support (0-10 points), and work and activities of daily life (0-20 points). Values for each of these items are combined, with a higher score representing a higher level of function. 64 Evidence for construct validity was provided by determining the ability of the OSS to discriminate among groups of subjects. These groups were formed based on a self-rating of function (poor, fair, good, or excellent), dorsiflexion range of motion (symmetrical, limitation less than or equal to 15, or limitation greater than 15 ), radiographic evidence of dislocations (absent or present), and radiographic evidence of osteoarthritis (none, grade I, or grades II-IV). ROFPAQ The ROFPAQ was developed as a disease-specific instrument for individuals with chronic foot pain. 71 The ROFPAQ contains 39 items in the following 4 subscales: sensory (16 items), affective (10 items), cognitive (10 items), and comprehension (3 items). The sensory subscale assesses pain, including its temporal, spa- 80 february 2007 volume 37 number 2 journal of orthopaedic & sports physical therapy
tial, and thermal properties. The affective subscale relates to the emotional effects of pain, while the cognitive subscale evaluates the awareness, expectations, and comparisons of pain. The 3 items in the comprehension subscale indicate whether the subject is properly interpreting and responding to the items on the questionnaire. The items from each subscale are not grouped together but rather are distributed throughout the questionnaire. Each subscale is scored separately from 1 through 5 using a Likert response format. The item responses are averaged together to produce a subscale score ranging from 1 to 5, with a higher score representing a greater effect of pain. Evidence for content validity, construct validity, and reliability is derived from data obtained among 39 subjects with chronic foot pain defined as pain with no foreseeable end. 71 The mean age of the subjects was 58 years (age range, 40-83 years), with all reporting pain for more than 1 year. The Headache Scale and the Hospital Anxiety and Depression Scale were used as evidence for divergent validity. The low to moderate correlations to the Headache Scale indicated that the ROFPAQ was measuring foot pain as opposed to headache-related pain. The anxiety subscale and the depression subscale of the Hospital Anxiety and Depression Scale were compared only with the affective subscale of the ROFPAQ. The strong correlation (Spearman rank correlation coefficient, 0.87) to the depression subscale indicates that the affective subscale is measuring a component of depression as opposed to the emotional effect of foot pain. The FFI pain subscale was used as evidence for convergent validity. The Spearman rank correlation coefficient was used in the test-retest reliability analysis and in the assessment for convergent and divergent validity. Sports Ankle Rating System QOL Measure The Sports Ankle Rating System was developed as a region-specific measure comprising self-reported and cliniciancompleted outcome measures. 84 This system consists of the QOL measure, clinical rating score, and single numeric evaluation. The system was developed so that the 3 outcome measures could be used together or independently. The QOL measure is a self-reported questionnaire designed to assess an athlete s QOL after an ankle injury. This questionnaire contains 5 items in each of 5 subscales that pertain to symptoms, work and school activities, recreation and sports activities, activities of daily living, and lifestyle. The questionnaire is scored using a Likert response format, with a higher score representing a higher level of function. The QOL measure has evidence for content validity, construct validity, reliability, and responsiveness from among 30 subjects with a mean age of 19.7 years (age range, 18-22 years). 84 Fifteen subjects who sustained a grade II ankle sprain less than 48 hours before study enrollment were administered the complete Sports Ankle Rating System on 3 occasions. These included once at the time of study enrollment and then at 2 weeks and 4 weeks later. Fifteen subjects with normal ankle function were administered the Sports Ankle Rating System initially after study enrollment and at 1 week later; the coefficient of variation was calculated among this healthy group for the test-retest reliability analysis during a 1-week period. DISCUSSION The information acquired from self-reported outcome instruments is useful only if there is evidence to support the interpretation of the obtained scores. As noted in the review by Button and Pinney, 18 many instruments have been implemented in outcome-related research with little or no evidence to support their use. In terms of the test conditions under which the research was conducted that offers evidence regarding an instrument s score interpretation, the instrument type must be relevant to the clinical conditions in which the instrument will be implemented. Evidence regarding score interpretation must contain a description of the applicable test conditions, including information about the characteristics of subjects, the timing of data collection, and the construct of change. For appropriate score interpretation, the clinician or researcher must verify that the clinical conditions under which the instrument will be used are similar to the test conditions. This review identified 14 self-reported outcome instruments that had evidence to support their use when assessing the effect of interventions directed at individuals with foot and ankle-related pathologic conditions in an orthopaedic physical therapy setting. Five of 14 instruments had evidence to support their use among individuals with ankle ligament injuries, 2,31,43,69,84 2 instruments had evidence to support their use among patients with rheumatoid arthritis, 7,17,46,72 and 2 instruments had evidence to support their use among subjects with generalized orthopaedic leg, foot, and ankle disorders. 14,54 Evidence is also available for instruments used among individuals who have osteoarthritis or posttraumatic arthritis, 26 nontraumatic foot or ankle problems, 1 calcaneal fractures, 36 ankle fractures, 64 chronic foot pain, 71 plantar fasciitis, 48 and conditions commonly seen in a podiatry practice. 13 All 14 instruments offered evidence for construct validity, 11 offered evidence for reliability, and 6 offered evidence for responsiveness. Nine instruments had evidence for content validity as demonstrated by an assessment of internal consistency or by factor analysis. Evidence for content validity can be obtained not only by an assessment of internal consistency or by factor analysis but also by input from patients or expert clinicians. This type of evidence for content validity is somewhat subjective in nature. Therefore, a description related to justification of content validity through patient or expert input was not included in this review but can be obtained in the original articles that describe the instruments. Response burden related to the patient s completion of the instrument and journal of orthopaedic & sports physical therapy volume 37 number 2 february 2007 81