Validity and Reliability of the FIM Instrument in the Inpatient Burn Rehabilitation Population

Transcription

1 Archives of Physical Medicine and Rehabilitation journal homepage: pmr.org Archives of Physical Medicine and Rehabilitation 2013;94: ORIGINAL ARTICLE Validity and Reliability of the FIM Instrument in the Inpatient Burn Rehabilitation Population Paul Gerrard, MD, a Richard Goldstein, PhD, a Margaret A. DiVita, MS, b Colleen M. Ryan, MD, c,d Jacqueline Mix, MPH, b Paulette Niewczyk, MPH, PhD, b,e Lewis Kazis, ScD, f Karen Kowalske, MD, g Ross Zafonte, DO, a Jeffrey C. Schneider, MD a From the a Department of Physical Medicine and Rehabilitation, Spaulding Rehabilitation Hospital, Harvard Medical School, Boston, MA; b Uniform Data System for Medical Rehabilitation, Amherst, NY; c Sumner Redstone Burn Center, Surgical Services, Massachusetts General Hospital, Harvard Medical School, Boston, MA; d Shriners Hospitals for Children-Boston, Boston, MA; e Health Care Studies Department, Daemen College, Amherst, NY; f Department of Health Policy and Management, Boston University School of Public Health, Boston, MA; and g Department of Physical Medicine and Rehabilitation, University of Texas Southwestern Medical Center at Dallas, Dallas, TX. Abstract Objective: To provide evidence of construct validity for the FIM instrument in the inpatient rehabilitation burn population. Design: Confirmatory factor analysis and item response theory were used to assess construct validity. Confirmatory factor analysis was performed on a 2 factor model of the FIM instrument and on a 6 subfactor model. Mokken scale analysis, a nonparametric item response theory, was performed on each of the FIM instrument s 2 major factors, motor and cognitive domains. Internal consistency using Cronbach alpha and Molenaar and Sijtsma s statistic was also examined. Setting: Inpatient rehabilitation facilities. Participants: Data from the Uniform Data System for Medical Rehabilitation for patients with an impairment code of burn injury from the years 2002 to 2011 were used for this analysis. A total of 7569 subjects were included in the study. Interventions: Not applicable. Main Outcome Measures: Comparative fit index results for the confirmatory factor analyses and adherence to assumptions of the Mokken scale model. Results: Confirmatory factor analysis provided a comparative fit index of.862 for the 2 factor model and.941 for the 6 subfactor model. Mokken scale analysis showed scalability coefficients of.681 and.891 for the motor and cognitive domains, respectively. Measures of internal consistency statistic gave values of >.95 for each major domain of the FIM instrument. Conclusions: The FIM instrument has evidence of validity and reliability as an outcome measure for patients with burn injuries in the inpatient rehabilitation setting. The 6 subfactor model provides a better fit than the 2 factor model by confirmatory factor analysis. There is evidence that the motor and cognitive domains each form valid unidimensional metrics based on nonparametric item response theory. Archives of Physical Medicine and Rehabilitation 2013;94: ª 2013 by the American Congress of Rehabilitation Medicine The FIM instrument (FIM) is a commonly used functional measure in inpatient rehabilitation that has been extensively studied and validated in the inpatient rehabilitation setting. 1-5 However, none of these studies explicitly examine the validity of the FIM instrument in the burn population. Importantly, burn injuries can No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated. happen to any part of the body and in any combination of body parts resulting in injury patterns not seen in other trauma or rehabilitation populations, for example, isolated hand injuries with distinct functional consequences. 6 As such, the generalizability of functional measure validity studies in other rehabilitation pop ulations to the burn population is uncertain. Further background information regarding the FIM instrument and the importance of validation is provided in supplemental material 1, available online only at the Archives website: pmr.org/. This /13/$36 see front matter ª 2013 by the American Congress of Rehabilitation Medicine

2 1522 P. Gerrard et al study seeks to study the validity of the FIM instrument in the inpatient burn rehabilitation population. Methods Data source Data were obtained from the Uniform Data System for Medical Rehabilitation for patients with an impairment code of burn injury for the years 2002 to This data set includes demographic, medical, and functional data including admission and discharge FIM scores. A more thorough description of the data source has been described previously. 7 Discharge FIM data were used for validation and reliability calculations, because the use of multiple time points would violate the local independence assumption of the item response theory (IRT), and the time point at which the FIM instru ment most reflects an outcome is at rehabilitation discharge. Data analysis This study examines the construct validity and internal consistency of FIM ratings among burn patients admitted to an inpatient reha bilitation facility. This study validated 3 scales: the FIM instrument as a whole, the FIM instrument s 13 motor items as a free standing scale, and the FIM instrument s 5 cognitive items as a free standing scale. The FIM instrument as a whole was analyzed using confir matory factor analysis (CFA) to study the validity of 2 different hypothesized factor structures. CFA was performed using the statistical program R 8 and the package lavaan. 9,a The motor and cognitive scales were each analyzed using the IRT as free standing outcome measures independent of each other. We performed IRT based analyses with R 8 and the mokken package. 10,11,b,c We also determined the reliability for each of these 3 scales as measured by internal consistency using the mokken package. 10,11 CFA is a hypothesis driven technique for testing a proposed factor structure of a given scale or index. 12 It differs from exploratory factor analysis in that it is used to confirm the validity of an a priori determined factor structure rather than seeking a new factor structure. The 2 a priori factor structures that we examined were a 2 factor model composed of a motor and a cognitive subdomain that comes from prior research supporting such a factor structure in other rehabilitation populations 4,5 and a modification to this factor structure, which further subdivides the motor and cognitive domains into a total of 6 subfactors. 2,4 The 6 subfactor structure further divides the motor domain into self care, transfers, bowel and bladder management, and locomotion; it divides the cognitive domain into communication and social cognition. The weight with which each item relates to its respective factor is reported as both an unstandardized factor loading and a standardized regression weight. Both of these values are designed to measure the same thing, but they differ in the way they are computed; unstandardized loadings are calculated by constraining the first item s loading to 1, and standardized regression weights are calculated by constraining item and factor variances. (CFA requires constraining parameters to have few List of abbreviations: CFA confirmatory factor analysis IIO invariant item ordering IRT item response theory MSA Mokken scale analysis enough variables such that a unique solution exists.) Fit results from the 2 different factor structures were obtained for model comparison. Further information about these factor structures is provided in supplemental material 1, available online only at the Archives website: pmr.org/, and a graphical presentation of this is shown in supplemental material 2, available online only at pmr.org/. We separately sought evidence of construct validity for the motor scale and the cognitive scale as independent outcome measures using IRT, which will be explained in further detail in the next paragraph. The choice to study the validity of these subscales as free standing outcome measures comes from prior research indicating that they are separate factors of the FIM instrument. 4,5 The difference between the CFA and the IRT is that the CFA does not study the validity of the motor and cognitive scales as independent measures but only as components of the FIM instrument as a whole. As such, this article in a sense describes 2 different studies with complementary results. The IRT analyses were performed specifically using Mokken scale analysis (MSA), for which a number of articles provide an overview This type of nonparamteric IRT, like Rasch analysis, assumes the presence of unidimensionality, monotonicity, and local independence. 10 Within Mokken scales, there is the weaker monotone homogeneity model and the stronger double mono tonicity model. The importance of MSA is that it makes minimal assumptions about the distribution of the relation between person ability and item difficulty; it tests whether or not the assumptions listed above are true, and if they are, a test is considered valid. This is in contrast to logistic IRT models (eg, the Rasch model), which are more commonly seen in the rehabilitation literature, that assume the data generated by a test are characterized by the logistic function of the difference between a person s ability and the item s difficulty. 19 Unidimensionality refers to the notion that the scale measures a single construct and was assessed by calculating the scalability coefficient for each subdomain. 11 Monotonicity, which is the requirement that as a person s ability level increases, his or her chance of meeting any minimum score on a single item increases, was also tested for each item. The item difficulty was determined by the mean score on each item. 11 The double monotonicity model makes the added assumption that item difficulty ordering is the same for persons at all ability levels, a concept known as invariant item ordering (IIO). 14 Manifest IIO is measured for the scale as a whole with the scalability coefficient of the transposed matrix. 20 Testing for local independence was based on the methods described for the Patient Reported Outcomes Measurement Information System, 21 which has since been applied as the basis for recommendations in a published framework for scale devel opment in the field of rehabilitation. 22 Local independence was examined using a residual correlation matrix after a single factor CFA on the motor domain and on the cognitive domain using a cutoff of 0.2 or less as an indicator of local independence. Reliability statistics were calculated using 2 internal consis tency measures: Molenaar and Sijtsma s statistic 23 and Cron bach alpha. 24 Results FIM scores from 7569 patients with complete admission and discharge FIM data were used in this study. The characteristics of the study sample are shown in table 1.

3 Validity of the FIM instrument in burns 1523 Table 1 Demographic and medical characteristics of the burn population, UDSMR database 2002e2011 Characteristic Value Number of subjects 7569 Age (y) Male 5110 (67.5) Race White 5465 (72.2) African American 1186 (15.7) Latino/Hispanic 547 (7.2) Asian 132 (1.7) American Indian/Alaskan 44 (0.6) Hawaiian/Pacific Islander 34 (0.4) Multiracial 34 (0.4) Missing 128 (1.7) Onset days TBSA, median decile (%) 10e19 Admission FIM total Married 2857 (37.7) Living alone 1997 (26.4) Primary payer source Medicare 2601 (34.4) Medicaid 1085 (14.3) Workers Compensation 992 (13.1) Unreimbursed 61 (0.8) Commercial 899 (11.9) Other 1931 (25.5) Number of comorbidities Operating beds Burn volume by facility NOTE. Values are mean SD, n (%), or as otherwise indicated. Abbreviations: TBSA, Total Body Surface Area burned; UDSMR, Uniform Data System for Medical Rehabilitation. CFA of the 2 different factor modelsda 2 factor model and a 6 subfactor modeldshowed better model fit for the 6 subfactor model than for the 2 factor model. The comparative fit index for fit of the whole model was.862 for the 2 factor model and.941 for the 6 subfactor model. The results of the CFA can be seen for both the 2 factor model and the 6 subfactor model in table 2, which shows unstandardized factor loadings for each item and stan dardized regression weights for each item. MSA of the motor and cognitive domains showed unidimen sionality for each domain. The FIM motor had a whole test scalability coefficient of.681 (SEZ.05) and item level scalability coefficients ranging from.599 to.757, suggesting that this is a strongly unidimensional test. The FIM cognitive had a scal ability coefficient of.891 (SEZ.004) for the test as a whole and item level scalability coefficients ranging from.866 to.901, indicating stronger evidence of unidimensionality than the FIM motor. Scalability coefficients for both domains can be seen in table 3. The assumption of latent monotonicity was also tested for both domains. There were no significant violations of latent monotonicity in either the motor subtest or the cognitive subtest. Item means were used to determine difficulty level and are dis played in table 3. IIO was also tested for both the motor and the cognitive domains, and neither subtest was found to satisfy the requirement of IIO. Mean scores for each item to establish difficulty for use in IIO testing are shown in table 3. A further discussion of results from analyses of IIO is provided in supplemental material 1, available online only at pmr.org/. The examination of residual correlation matrices to test for local independence showed 1 violation between the bowel management and the bladder management items on the motor domain. The residual between these 2 items was.21, exceeding the threshold of 0.2 used for local independence. There were no violations of local independence in the cognitive domain. Measures of internal consistency showed high internal consistency for the FIM instrument as a whole, the FIM motor scale, and the FIM cognitive scale. The FIM motor scale had a Molenaar and Sijtsma s statistic of.958 and a Cronbach alpha of.956. The FIM cognitive scale had a Molenaar and Sijtsma s statistic of.974 and a Cronbach alpha of.968. Discussion To the best of our knowledge, this is the first study to examine the validity and reliability of the FIM instrument in the inpatient burn rehabilitation population. This study showed that the FIM instrument has evidence of construct validity in this population using CFA and nonparametric IRT, and evidence of reliability as measured by internal consistency. Although this is the first study, to the best of our knowledge, to formally examine the construct validity and reliability of the FIM instrument in the inpatient burn rehabilitation population, prior studies have already shown that the FIM instrument has criterion validity as a predictive tool for other objective outcome measures, including community discharge and readmission to an acute care facility. 7,25,26 This study examined 2 different factor structures in the CFAda 2 factor model and a 6 subfactor model. The 6 subfactor model appeared to have better fit to the CFA model based on comparing comparative fit index values for each model. The 2 factor model serves as a gross classification of functional deficits by body function/structure, while the 6 subfactor model refines this to a categorization scheme based on functional consequence. In addition to providing better model fit, the 6 subfactor model is also likely to provide enhanced clinical information because of the categorization by functional consequence. The improved fit of the 6 subfactor model over the 2 factor model is likely in part because of the increased granularity of this model and grouping of clinically similar functional items together. Importantly, this is not only a statistical finding, but also evidence of a formal conceptual model of functional independence in burn patients with clinically useful groupings of items. This 6 subfactor conceptualization can be used in both outcome measurements for future clinical studies or for the development of concrete therapeutic targets. In the case of burn injuries, this is especially important, because very focal areas of the body may be affected, which would be expected to result in very specific functional deficits. For example, significant injuries to the legs would be expected to affect the FIM motor as a whole, but enhanced care of lower extremity burn injuries might be expected to show improvement in locomotion more than self care, a differ ence that might not be captured in a study that looks only at the FIM motor factor as a whole. Factor loadings were generally quite strong for each item on its respective factor in both models. For the sake of completeness, both standardized and unstandardized information is provided, but standardized regression weights are likely easier to interpret and more informative in the case of CFA. The standardized regression weight indicates the proportion of the SD that a given item will

4 1524 P. Gerrard et al Table 2 Item Results of the confirmatory factor analyses and Mokken scale analysis Factor Loadings Regression Weights 2 Factor Model 6 Subfactor Model 2 Factor Model 6 Subfactor Model Motor Self care NA 1.000* NA 0.879* Eating Grooming Dressing lower body Dressing upper body Bathing Toileting Sphincter NA 1.154* NA 0.838* Bladder Bowel Transfers NA 1.294* NA 0.978* Bed, chair, wheelchair transfers Toilet transfers Tub transfers Mobility NA 1.218* NA 0.918* Locomotion Stairs Cognitive Communication NA 1.000* NA 0.955* Comprehension Expression Social cognition NA 0.995* NA 0.971* Interaction Problem solving Memory NOTE. The factor loadings of the 2 factor model and the 6 subfactor model are shown. The 2 factor model divides the FIM instrument into a motor and cognitive domain with the items shown in the table. The 6 subfactor model divides the 2 factor model into 6 subfactors, and these 6 subfactors are composed of the observable items from the FIM instrument. In the 2 factor model, items load directly onto the motor or cognitive factors. In the 6 subfactor model, items load onto a subfactor, and each subfactor loads onto the motor or the cognitive factor. Abbreviation: NA, not applicable. * Indicates subfactors. change for a unit change in its factor. The fact that the Stairs item has a substantially lower weight than the other items indicates that this item is one of the least significant drivers of a patient s FIM instrument score, which is likely due to the difficulty of this task. In addition to CFA, IRT was performed for the FIM instrument s motor scale and cognitive scale independently to examine the construct validity of these specific scales as outcome measures. IRT was not performed for the subfactors in the 6 subfactor model, because it would not be particularly meaningful in subfactors composed of 2 or 3 items. Both the FIM motor scale and the FIM cognitive scale have validity based on the MSA performed. These measures met the assumption of unidimensionality quite strongly, and they had no significant violations of latent monotonicity. This study used mean scores for each item to establish relative item difficulties. The results found for the motor subscale in the burn population resemble prior relative difficulty levels estab lished for the FIM instrument in mixed inpatient rehabilitation populations. 27,28 As in prior studies, eating is the easiest item, and stair climbing is the most difficult item in the burn population, but there are some differences in the ordering of the intermediate items. Because MSA makes no assumption about the probability function relating person abilities to item difficulties, interval level differences in item difficulty cannot be calculated as has been done with the FIM instrument using Rasch analysis in a mixed rehabilitation population. 27 Mean scores on the cognitive subscale established Expression as the easiest item and Problem Solving as the most difficult, though all 5 items had a mean score within less than half a point of each other. From the test for IIO, both the FIM motor and cognitive subscales meet criteria to be a monotone homogeneity model scale. Monotone homogeneity model scales represent adequately strong tests so long as the items are not used for item banking or adaptive testing. 17 In this study, nonparametric IRT was chosen, because it uses minimal assumptions about the item response functions. Because the added information provided by parametric IRT models, such as interval level item difficulty or item discrimination, was not of interest in this particular study, nonparametric IRT was used. Local independence, which is an assumption of a number of IRT models, 29 was tested using a residual correlation matrix method. A slight violation of local independence between 2 items on the motor domain was found between bowel management and bladder management. Local independence is frequently not explicitly tested in validity studies despite the fact that it is assumed in other IRT models, such as the Rasch model. 29,30 The violation of local independence between the bowel and bladder items is not surprising from a clinical standpoint. This small violation is unlikely to be of substantive importance in this

5 Validity of the FIM instrument in burns 1525 Table 3 Scalability coefficients and mean scores for each item Item Scalability Coefficient Score Motor Eating Bowel Bed transfers Grooming Toilet transfers Bladder Locomotion Toileting Dressing upper body Dressing lower body Bathing Tub transfers Stairs Cognitive Expression Comprehension Social interaction Memory Problem solving NOTE. Scalability coefficients >0.3 are considered adequate, and scalability coefficients >0.5 indicate strong unidimensionality. Items are sorted into motor and cognitive subscale and sorted from lowest to highest mean score (ie, easiest to most difficult) within the subscale. The item level scores are expressed as mean SD and taken from discharge scores. particular study because it uses methods with different assump tions than the application for which the threshold of 0.2 was designed. The threshold of 0.2 was developed for use in computerized adaptive testing and item banking based on a para metric IRT model. 21 Such applications of IRT warrant the use of parametric IRT models, 17 which specify a particular function (typically a logistic function) relating latent trait ability to performance on any given item. In such a model, local dependence may confound the analysis, because a parameter relating a subset of items exists for which the model does not account. However, this study used a nonparametric IRT method that makes only very general assumptions about the relation between the latent trait ability and performance on any given item. The 2 subdomains of the FIM instrument tested here had very high internal consistency when measured with either Cronbach alpha or the MS statistic. Study limitations This study had a number of limitations. As noted previously, local independence was violated for the relation between 2 items, which may slightly inflate internal consistency. However, with the very high internal consistencies measured for the FIM instrument, there is likely good reliability even in the presence of inflated values. In addition, MSA was used in this study rather than a parametric IRT, because the score distributions did not support logistic models. However, this has implications only in instances in which infor mation regarding difficulty levels measured in logit values is needed, which is not the case for basic test administration. Another important limitation is that the results of this study are applicable only to burn patients in an inpatient rehabilitation setting. Conclusions The FIM instrument has evidence of validity and reliability as an outcome measure for burn patients in the inpatient rehabilitation setting. It has practical use as a predictor of discharge outcome from rehabilitation for burn patients, 7,25,26 and it has evidence of construct validity as a measure of functional independence. CFA strongly supports a 6 subfactor model of the FIM instrument over a 2 factor model, but the validity of a motor scale and a cognitive scale as independent scales is supported by IRT. Internal consis tency is also quite high for both the motor and cognitive subscales. Suppliers a. The R Foundation for Statistical Computing, c/o Institute for Statistics and Mathematics, Wirtschaftsuniversität Wien, Augasse 2 6, 1090 Vienna, Austria. b. Yves Rosseel, Dept of Data Analysis, Ghent University, 9000 Ghent, Belgium. c. L. Andries van der Ark, Dept of Methodology and Statistics, Tilburg University, PO Box 90153, 5000 LE, Tilburg, The Netherlands. Keywords Burns; Factor analysis; Outcome measures; Psychometrics; Rehabilitation; Validation Corresponding author Paul Gerrard, MD, Department of Physical Medicine and Reha bilitation, Spaulding Rehabilitation Hospital, 125 Nashua St, Boston, MA E mail address: pbgerrard@partners.org. References 1. Brosseau L, Potvin L, Philippe P, Boulanger Y L, Dutil E. The construct validity of the Functional Independence Measure as applied to stroke patients. Physiother Theory Pract 1996;12: Brosseau L, Wolson C. The inter rater reliability and construct validity of the Functional Independence Measure of multiple sclerosis subjects. Clin Rehabil 1994;8: Kidd D, Stewart G, Baldry J, et al. The Functional Independence Measure: a comparative validity and reliability study. Disabil Rehabil 1995;17: Ottenbacher KJ, Hsu Y, Granger CV, Fiedler RC. The reliability of the Functional Independence Measure: a quantitative review. Arch Phys Med Rehabil 1996;77: Stineman MG, Shea JA, Jette A, et al. The Functional Independence Measure: tests of scaling assumptions, structure, and reliability across 20 diverse impairment categories. Arch Phys Med Rehabil 1996;77: Williams N, Stiller K, Greenwood J, Calvert P, Masters M, Kavanagh S. Physical and quality of life outcomes of patients with isolated hand burnsea prospective audit. J Burn Care Res 2012;33: Tan WH, Goldstein R, Gerrard P, et al. Outcomes and predictors in burn rehabilitation. J Burn Care Res 2012;33: The R Foundation for Statistical Computing. R: A language and environment for statistical computing. Version Vienna: R Foundation for Statistical Computing; 2012.

6 1526 P. Gerrard et al 9. lavaan: Latent Variable Analysis. Version Ghent: Department of Data Analysis, Ghent University. 10. van der Ark LA. New developments in Mokken scale analysis in R. J Stat Softw 2012;48: mokken: an R package for Mokken scale analysis. Version Tilburg: Department of Methodology and Statistics, Tilburg University. 12. Yuan K H, Bentler PM. Structural equation modeling. Handbk Stat 2007;26: Sijtsma K, Verweij AC. Mokken scale analysis: theoretical consider ations and an application to transitivity tasks. Appl Meas Educ 1992;5: Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and wellbeing questionnaire item responses: a nonparametric IRT method in empirical research for applied health researchers. BMC Med Res Methodol 2012;12: van der Ark LA. Mokken scale analysis in R. J Stat Softw 2007;20: van Schuur WH. Mokken scale analysis: between the Guttman scale and parametric item response theory. Pol Analysis 2003;11: Meijer RR, Sijtsma K. Theoretical and empirical comparison of the Mokken and the Rasch approach to IRT. Appl Psychol Meas 1990;14: Sijtsma K, Meijer RR. Nonparametric item response theory and special topics. Handbk Stat 2007;26: Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21st century. Med Care 2000;38:II Sijtsma K, Meijer RR. A method for investigating the intersection of item response functions in Mokken s nonparametric IRT model. Appl Psychol Meas 1992;16: Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health related quality of life item banks: plans for the Patient Reported Outcomes Measurement Information System (PROMIS). Med Care 2007;45:S Velozo CA, Seel RT, Magasi S, Heinemann AW, Romero S. Improving measurement methods in rehabilitation: core concepts and recom mendations for scale development. Arch Phys Med Rehabil 2012; 93(Suppl2):S Sijtsma K, Molenaar IW. Reliability of test scores in nonparametric item response theory. Psychometrika 1987;52: Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16: Gerrard P, Ryan CM, Niewczyk P, et al. Risk factors associated with transfer from inpatient rehabilitation to acute care in the burn pop ulation. PM&R 2011;3:S Gerrard P, Goldstein R, DiVita M, et al. A scoring system for risk stratification of acute transfers from inpatient rehabilitation in the burn population. J Burn Care Res 2012;33(2 Suppl1):S Granger CV, Hamilton BB, Linacre JM, Heinemann AW, Wright BD. Performance profiles of the Functional Independence Measure. Am J Phys Med Rehabil 1993;72: Stineman MG, Ross RN, Fiedler R, Granger CV, Maislin G. Functional independence staging: conceptual foundation, face val idity, and empirical derivation. Arch Phys Med Rehabil 2003;84: Reckase MD. The past and future of multidimensional item response theory. Appl Psychol Meas 1997;21: Mair P, Hatzinger R. Extended Rasch modeling: the erm package for the application of IRT models in R. J Stat Softw 2007;20:1 20.

7 Validity of the FIM instrument in burns 1526.e1 Supplemental Material 1: Additional Background Information In recent years, improvements in burn care have resulted in increased survival after thermal injury. 1 Consequently, an under standing of the evaluation and treatment of problems related to quality of life in these individuals is becoming increasingly important, and in accordance with this need there have been a number of studies in recent years that have addressed functional outcomes and community reintegration in burn survivors. 1-6 Functional outcomes are less likely to have clear differences than survival versus mortality, yet these outcomes are still likely to be regarded as important to quality of life by patients with burn injuries. The scientific study of interventions aimed at functional outcomes, therefore, requires methods of measurement that accurately reflect some meaningful notion of function. For burn patients who have been treated at an inpatient rehabilitation facility, the FIM instrument has been used as an objective func tional outcome measure that is predictive of community discharge, 4,6 yet it remains to be validated as a meaningful outcome in the burn population, and adequate outcome measures should be both reliable and valid to produce useful scientific data. 7 The underlying idea of validation is to demonstrate that an outcome measure is meaningful. In some instances, the outcome measure of interest is both directly measurable and intrinsically meaningful such as in mortality, cost, or time to hospitalization. In these cases, it is desirable to directly measure these outcomes. However, rehabilitation is frequently concerned with outcomes that are not themselves directly measurable, such as function and disability. Thus, if one wishes to measure the conceptual outcome of functional status, some directly observable surrogate measures must be used, such as clinical scales or indices of functional status. The goal of validation is to demonstrate that these metrics of observable items are meaningful representations of the conceptual outcome of interest. The FIM instrument is a widely used metric among inpatient rehabilitation hospitals whose staff members are experienced in administering it, and it has been used on thousands of burn patients in the inpatient rehabilitation facility setting in the past decade. Prior research has already shown criterion validity of the FIM instrument to predict discharge home 6 and risk of read mission to an acute care hospital. 8,9 Construct validity, reliability, and ability to detect a meaningful change remain unstudied in the burn population specifically, which this study seeks to do. To establish construct validity, this study uses a combination of CFA and IRT. These are commonly used psychometric methods to establish construct validity, and methods such as these have previously been used to establish construct validity of the FIM instrument in other inpatient rehabilitation populations Both of these methods are concerned with the measurement of a latent trait by a clinical metric (ie, scale or index). The items on a clin ical metric are observable and measurable manifestations of the latent trait the metric seeks to quantify, which itself cannot be directly measured. In the case of this study, latent traits of func tional independence related to motor and cognitive function are the latent traits of interest. Further Information on Data Analysis Methods The 13 motor items were as follows: (1) eating; (2) grooming; (3) bathing; (4) dressing upper body; (5) dressing lower body; (6) toileting; (7) bladder management; (8) bowel management; (9) chair, bed, and wheelchair transfers; (10) toilet transfers; (11) tub and shower transfers; (12) walking or wheelchair locomotion; and (13) stairs. The 5 cognitive items were (1) comprehension, (2) expression, (3) social interaction, (4) problem solving, and (5) memory. We also performed CFA on a modification to this factor structure, which further subdivides the motor and cognitive domains into a total of 6 subfactors. 16,17 In this scheme, the motor domain is composed of self care, sphincter control, transfers, and locomotion subfactors. The cognitive domain is composed of communication and social cognition subfactors. Additional Results of IIO Tests IIO was tested for both the motor and the cognitive domains, and neither subtest was found to satisfy the requirement of IIO. In the motor domain, there were a number of violations to latent monotonicity. With the presence of these violations, the FIM motor had a scalability coefficient of the transposed matrix equal to.285, which is below the threshold of 0.3, which must be exceeded to be considered to have IIO. 18 A mild double mono tonicity model measure could be created using an automated stepwise item elimination procedure eliminating the item with the greatest violations to IIO in each round. This procedure elimi nated the items bowel control, grooming, upper body dressing, bathing, and tub transfers; this resulted in a motor test composed of the 7 remaining items with a scalability coefficient of the transposed matrix of.370. The FIM cognitive domain had no violations to latent monotonicity, but the test as a whole had a scalability coefficient of the transposed matrix equal to.244, which indicates inadequate IIO. As such, no item elimination procedure was tried. Ability to detect a meaningful change The ability of the FIM instrument to detect a meaningful change was a planned but secondary question in this study. As such, this is not discussed in the body of the manuscript. Ability to detect meaningful change was measured as the proportion of patients who had a change that was greater than or equal to the minimal clinically important difference (MCID) in the FIM rating between admission and discharge. We calculated these proportions as the number of subjects who met the thresholds of an MCID divided by the total number of subjects in the study sample. An empirically derived value of the FIM instrument s MCID in the burn pop ulation specifically has not been established. However, prior research regarding MCID in general has found that the MCID is approximately equal to half of the SD of the sample of scores. 19 Therefore, we defined the MCID for the FIM instrument, the motor scale, and the cognitive scale as one half of the SD of samples for each. Ability to detect a meaningful change showed that 71%, 75%, and 27% of the patients showed a clinically important change in total FIM score, FIM motor score, and FIM cognitive score, respectively. These results are depicted in figure S1. These find ings represent a modest ability to detect change, especially in the cognitive domain. The nature of cognitive deficits, especially as they relate to function, in burn patients has not been well studied.

8 1526.e2 P. Gerrard et al The low ability to detect change is likely due to relatively high FIM cognitive scores at admission producing a ceiling effect of the FIM cognitive domain in many of these patients. Further study of cognition in burn rehabilitation patients is warranted to determine whether the high cognitive FIM scores reflect the nature of burn injuries or a more refined measure of cognition is warranted. Notably, the use of the half an SD method is a limitation of this analysis. Preferably, MCID for the FIM instrument should be established by comparison with an external indicator of clinically important change, but no such measure was available for comparison in this study. Fig S1 Percentage of patients achieving an MCID.

9 Validity of the FIM instrument in burns 1526.e3 Supplemental Material 2: Hypothesized FIM Instrument Factor Structure Two Major Domains Six Subdomains Observable Items Eating Grooming Dressing Lower Body Self-Care Dressing Upper Body Bathing Toileting Bladder Sphincter Bowel Bed, Chair, Wheelchair Motor Transfers Toilet Tub FIM Mobility Locomotion Stairs Comprehension Cognitive Communication Expression Social Cognition Interaction Problem Solving Memory

10 1526.e4 P. Gerrard et al References 1. Gomez M, Cartotto R, Knighton J, Smith K, Fish JS. Improved survival following thermal injury in adult patients treated at a regional burn center. J Burn Care Res 2008;29: Jarrett M, McMahon M, Stiller K. Physical outcomes of patients with burn injuriesea 12 month follow-up. J Burn Care Res 2008;29: Schneider JC, Bassi S, Ryan CM. Employment outcomes after burn injury: a comparison of those burned at work and those burned outside of work. J Burn Care Res 2011;32: Sliwa JA, Heinemann A, Semik P. Inpatient rehabilitation following burn injury: patient demographics and functional outcomes. Arch Phys Med Rehabil 2005;86: Williams N, Stiller K, Greenwood J, Calvert P, Masters M, Kavanagh S. Physical and quality of life outcomes of patients with isolated hand burnsea prospective audit. J Burn Care Res 2012;33: Tan WH, Goldstein R, Gerrard P, et al. Outcomes and predictors in burn rehabilitation. J Burn Care Res 2012;33: Jette AM. Measuring subjective clinical outcomes. Phys Ther 1989;69: Gerrard P, Goldstein R, DiVita M, et al. A scoring system for risk stratification of acute transfers from inpatient rehabilitation in the burn population. J Burn Care Res 2012;33(2Suppl1):S Gerrard P, Ryan CM, Niewczyk P, et al. Risk factors associated with transfer from inpatient rehabilitation to acute care in the burn population. PM&R 2011;3:S Yuan K-H, Bentler PM. Structural equation modeling. Handbk Stat 2007;26: Zumbo B. Validity: foundational issues in statistical methodology. Handbk Stat 2007;26: Bock RD, Moutaski I. Item response theory in a general framework. Handbk Stat 2007;26: Brosseau L, Potvin L, Philippe P, Boulanger Y-L, Dutil E. The construct validity of the Functional Independence Measure as applied to stroke patients. Physiother Theory Pract 1996;12: Granger CV, Hamilton BB, Linacre JM, Heinemann AW, Wright BD. Performance profiles of the Functional Independence Measure. Am J Phys Med Rehabil 1993;72: Kidd D, Stewart G, Baldry J, et al. The Functional Independence Measure: a comparative validity and reliability study. Disabil Rehabil 1995;17: Brosseau L, Wolson C. The inter-rater reliability and construct validity of the Functional Independence Measure of multiple sclerosis subjects. Clin Rehabil 1994;8: Ottenbacher KJ, Hsu Y, Granger CV, Fiedler RC. The reliability of the Functional Independence Measure: a quantitative review. Arch Phys Med Rehabil 1996;77: Ligtvoet R, van der Ark LA, te Marvelde JM, Sijtsma K. Investigating an invariant item ordering for polytomously scored items. Educ Psychol Meas 2010;70: Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003;41: