Nonrandomized Studies of Rehabilitation for Traumatic Brain Injury: Can They Determine Effectiveness?

Transcription

1 1235 Nonrandomized Studies of Rehabilitation for Traumatic Brain Injury: Can They Determine Effectiveness? Janet M. Powell, PhD, OT, Nancy R. Temkin, PhD, Joan E. Machamer, MA, Sureyya S. Dikmen, PhD ABSTRACT. Powell JM, Temkin NR, Machamer JE, TRAUMATIC BRAIN INJURY (TBI) IS a major cause of Dikmen SS. Nonrandomized studies of rehabilitation for long-term disability in the United States. The National traumatic brain injury: can they determine effectiveness? Arch Center for Injury Prevention and Control has estimated the Phys Med Rehabil 2002;83: annual combined incidence of hospitalization and death from TBI at 95 per 100,000 people between 1995 and 1996, with one of the highest incidence rates occurring in young adults. 1 Although more individuals are now surviving TBI, these survivors typically have residual physical, cognitive, emotional, and/or behavioral impairments. 2 Each year, 70,000 to 90,000 individuals in the United States sustain a TBI that results in long-term substantial loss of function. 2 Estimates in 1998 of the costs of providing care for a person with severe TBI over an average lifetime ranged from $600,000 to $1,875, These estimates do not include the additional costs stemming from lost wages of survivors or of family members who remain home to provide care. One of the major aims of rehabilitation efforts after TBI is to reduce disability and, consequently, to reduce the societal burden associated with providing long-term care for previously healthy individuals. Rehabilitation also strives to improve the quality of life of the survivor. Services typically consist of comprehensive, coordinated multidisciplinary programs within a medical model. 2-4 As might be expected, there are extensive costs related to the provision of rehabilitation services for TBI. Between 1991 and 1995, the average cost of inpatient rehabilitation for individuals with TBI served by Traumatic Brain Injury Model Systems centers ranged from $39,000 to $67,000 Objective: To examine the feasibility of investigating rehabilitation effectiveness for traumatic brain injury (TBI) with a nonrandomized design. Design: Observational cohort with confounder control by regression methodology. Setting: Level I trauma center. Participants: Consecutive series of 365 individuals with TBI discharged to inpatient rehabilitation or home (78% follow-up). Interventions: Not applicable. Main Outcome Measures: The Glasgow Outcome Scale (GOS), Sickness Impact Profile (SIP), Burden Inventory, and Perceived Quality of Life (PQOL). The predictors of interest: discharge to comprehensive inpatient rehabilitation or home and inpatient rehabilitation length of stay (LOS). Results: Discharge to rehabilitation was associated with poorer functioning on the GOS (P.03) and SIP (P.57), an increase on the Burden Inventory (P.14), and improved PQOL (P.20). Similar results were found for longer lengths of inpatient rehabilitation. Conclusions: The results appear to be because of a confounding effect rather than rehabilitation. The study design could not control for confounding that resulted from unmeasured or difficult to measure aspects of the clinical decisions for discharge placement and rehabilitation LOS. Furthermore, typical severity indices were inadequate to control for injury severity and recovery. Matching designs that investigate TBI rehabilitation are also at risk for inadequate confounder control. Key Words: Brain injuries; Rehabilitation; Research design; Treatment outcome by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation From the Departments of Rehabilitation Medicine (Powell, Machamer, Dikmen), of Neurological Surgery (Temkin, Dikmen), of Biostatistics (Temkin), and of Psychiatry & Behavioral Sciences (Dikmen), University of Washington School of Medicine, Seattle, WA. Supported by the National Institute on Disability and Rehabilitation Research (grant no. H133A980023), the National Institute of Neurological Disorders and Stroke (grant no. R01 NS19643), and the National Center for Medical Rehabilitation Research (grant no. R01 HD 33677). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated. Correspondence to Janet Powell, PhD, University of Washington School of Medicine, Dept of Rehabilitation Medicine, Box , Seattle, WA , jmpowell@u.washington.edu. Reprints are not available /02/ $35.00/0 doi: /apmr per person. 5 Between 1989 and 1991, the average cost of inpatient rehabilitation hospital services (excluding physician fees) for those patients with severe TBI in the Uniform Data System for Medical Rehabilitation was $111,000 per person. 6 At present, there is limited knowledge of the effectiveness of traditional models of TBI rehabilitation. This is coupled with increased use of alternative methods of service delivery resulting from recent pressures to reduce health care spending. 2,5,7 These alternative methods include increased provision of rehabilitation services in less expensive settings (eg, subacute facilities, skilled nursing facilities, home, outpatient centers) and shortened lengths of stay (LOSs) in inpatient rehabilitation facilities. 5,7 Knowledge of the effectiveness of these alternative methods of service delivery is also limited. As a result, clinicians must make decisions about the provision of rehabilitation services without adequate knowledge of the long-term consequences to the individual. Randomized controlled trials (RCTs), in which participants are randomly assigned to treatment and nontreatment groups, are considered by many to provide the best evidence of the effectiveness of treatment interventions The main advantage of this design stems from the expectation that participant differences will be randomly distributed between the groups. This reduces the possibility of a confounding effect from variables other than the intervention on trial results. Because the groups are expected to differ only in exposure to the intervention, the only remaining explanations for a difference in outcome between the groups are the intervention or chance. However, researchers face many difficulties investigating rehabilitation for TBI by using this study design. There are

2 1236 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell ethical concerns related to withholding standard treatment from a control group. 2,11,12 There are practical concerns, such as the need for large sample sizes to overcome variability resulting from the multiple factors influencing outcome (eg, preinjury risk factors, severity). 2,13 Great effort and commitment from study personnel are required to prevent sample bias resulting from high rates of individuals lost to follow-up in this typically unstable population. 12 The independent variables of interest (eg, rehabilitation treatments, procedures) are difficult to define and quantify operationally. 12 Only 11 RCTs were identified in a systematic review of TBI studies that focused on (1) timing of rehabilitation interventions, (2) intensity of rehabilitation, (3) effectiveness of cognitive rehabilitation, (4) supported employment, or (5) case management. 14 All of these RCTs were related to the effectiveness of specific cognitive interventions. A review in 1996 that was limited to studies of the effectiveness of services that occur subsequent to inpatient services, such as community integration programs, yielded only 1 study that used randomized controls. 3 After those reviews, an RCT compared inpatient cognitive rehabilitation with home-based rehabilitation. 15 The ethical and practical difficulties with using RCTs for investigating TBI rehabilitation effectiveness have led to increased interest in nonrandomized designs such as quasi-experiments and observational studies. In 1 common type of quasi-experimental design, the researcher identifies naturally occurring groups that appear similar, provides 1 group with the intervention, and compares the outcomes for both groups. In observational cohort studies, the researcher does not manipulate the intervention. Instead, a group of individuals that happened to receive the treatment of interest are identified, and their outcomes are compared with a group of individuals who did not receive the treatment. In studies of rehabilitation effectiveness using these study designs, the comparison (nontreatment) group is often from an underserved population with either no or limited access to the treatment of interest. In contrast to randomized controlled designs, neither quasiexperimental nor observational cohort study designs control inherently for confounding effects. Instead, the effects of confounders are controlled through multivariate data analysis techniques such as analysis of covariance (ANCOVA) or multiple regression. For example, a retrospective study investigating the effect of varying rehabilitation LOSs for individuals with TBI reported improved progress for a group of individuals with longer LOSs, after controlling for injury severity, extracranial injuries, and time between injury and rehabilitation admission using ANCOVA. 16 In other instances, study designs include matching to control for possible differences. The nontreatment group can be selected based on similarity with the treatment group in the distributions of potential confounding variables (ie, frequency matching), or each individual in the nontreatment group can be selected based on a match on important confounders to an individual in the treatment group (ie, individual matching). Frequency matching does not control for confounding on the basis of the study design alone and must be combined with other methods of control such as multivariate statistical analysis. 17 Individual matching controls for differences on those variables used for matching. One study 13 that combined frequency matching and multivariate regression analysis found that comprehensive rehabilitation for individuals with severe TBI was associated with significantly better long-term outcome on a study-specific measure. However, there are concerns about the high risk of compromised internal validity in nonrandomized study designs. 18,19 Without a randomized control group, improvement because of rehabilitation interventions cannot be distinguished conclusively from simultaneous improvement because of spontaneous recovery. All critical confounders may not be identified. 12 Others may not be included because of difficulties in operational definition, measurement, and/or data collection. Matching may be insufficient to control for important differences between treatment and comparison groups. 14 For example, matching on typical brain injury severity indices may not be sufficient to control for the greater severity of brain and other system injuries in those referred for inpatient rehabilitation services compared with those who are not. 13 On the other hand, some investigators 20 have advocated the use of nonrandomized study designs based on the similarity of findings from RCTs and the average results of multiple observational studies for certain clinical questions (eg, risk of stroke associated with antihypertensive treatment, effectiveness of a vaccine for active tuberculosis). The present study investigated the feasibility of using an observational cohort design to examine rehabilitation effectiveness for moderate to severe TBI when multiple potential confounders are controlled for by using regression methodology. The decision to conduct the study in this manner was based on the availability of a data set that appeared optimal for overcoming many of the concerns with this study design. All participants in the sample received postinjury care at the same trauma center. This avoided dissimilarities stemming from forming groups based on geographic location with the concomitant increased risk for differences in preinjury participant characteristics and postinjury medical care between the groups. Data were available on a wide variety of variables typically considered possible confounders in this population. There was a high follow-up rate, and, with the exception of a few cases (3%), all individuals had complete data as required for inclusion in a regression analysis. The primary research question was the following: For individuals with TBI who received acute care, does discharge to inpatient rehabilitation services, compared with discharge home, result in (1) better overall recovery, (2) better functional outcome, (3) reduced caregiver burden, and (4) better perceived qualify of life at 1 year postinjury, after adjusting for basic demographics, injury severity, and preinjury risk factors? A secondary objective was to investigate differences in the outcomes listed above for varying LOSs in inpatient rehabilitation services, after adjusting for basic demographics, injury severity, and preinjury risk factors. METHODS Participants This study included 365 participants with TBI who either received services in a comprehensive inpatient rehabilitation setting or were discharged home immediately after acute care and who were evaluated for outcome at 1 year. Based on availability and willingness to participate, a significant other (typically a close relative) was identified for each individual with brain injury. The sample represented a subgroup of participants from 2 prospective, longitudinal clinical trials of antiseizure medications that enrolled subjects between 1983 and 1987 and 1991 and ,22 For each research question, sample size differed for the outcome measures because of changes in study protocols, the ability of individuals to participate in assessments, and the availability of significant others. Both clinical trials enrolled individuals who were admitted to Harborview Medical Center, Seattle, WA (a level I trauma center), who were at least 16 years of age (14y old during the last year of the second study), and who had a qualifying TBI.

3 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell 1237 A qualifying injury was characterized by at least 1 of the following: (1) acute posttraumatic seizures, (2) depressed skull fracture, (3) penetrating brain injury, or (4) computed tomographic evidence of a cortical contusion or subdural, epidural, or intracerebral hematoma. The first trial also included individuals with Glasgow Coma Scale 23 (GCS) scores of 10 or less regardless of other injury characteristics. Individuals with preexisting conditions (eg, significant brain injury, preinjury seizures) were excluded. Participants were enrolled following procedures approved by the University of Washington Institutional Review Board. Response Variables (Dependent Measures) Response variables included the Glasgow Outcome Scale 24 (GOS), the Sickness Impact Profile 25 (SIP), the Burden Inventory, 26 and Perceived Quality of Life 27 (PQOL). The GOS and SIP were administered in both studies. The Burden Inventory and PQOL were added to the protocol for the second study. The GOS is a global measure of outcome consisting of 5 categories: death, persistent vegetative state, severe disability, moderate disability, and good recovery. The rating is based on the amount of an individual s dependence on others and the individual s ability to participate in a normal life. For this study, an examiner determined the GOS score after a full day of testing by using all available sources of information, including observation of the participant, information from the significant other, and the results of an extensive battery of neuropsychologic and psychosocial measures. There were no participants in the death category in this study because of the requirement that subjects be able to participate in the assessment. Because of the small number of participants in the vegetative state category, this category was combined with the severe category for the purposes of analysis. The SIP is a functional status measure that assesses alteration in everyday life as a result of health. A self-administered questionnaire assesses 12 areas: ambulation, mobility, body care and movement, alertness behavior, emotional behavior, communication, social interaction, sleep and rest, eating, work, recreation and pastimes, and home management. The total score was used for this analysis, with a higher score indicating higher endorsement of problems. If the individual with TBI was untestable on this measure, the score from a significant other s assessment of the participant was used. The Burden Inventory assesses the burden of the individual with TBI on a significant other, most frequently a close relative. Twenty-two items that describe negative aspects of caregiving are rated on a 5-point scale, from never feel that way to nearly always feel that way. The total score was used in the analysis, with a higher score indicating more negative feelings of burden related to caregiving. PQOL measures satisfaction with 12 different aspects of life: health of body, thinking and remembering, sexual activity, contact with family or friends, help from family or friends, contribution to the community, major activity, leisure activities, income, respect from others, meaning and purpose of life, and happiness. Ratings indicate 0% to 100% satisfaction with each area, with ratings given postinjury for both pre- and postinjury satisfaction. Each participant s score represents the mean change in satisfaction pre- to postinjury, with a higher score indicating more satisfaction with life postinjury. This change score was calculated by subtracting each item s preinjury rating from the postinjury rating and taking the mean of the differences for each participant. Predictor of Interest (Independent Variable) The predictor of interest for the primary research question was a 2-level categorical variable indicating whether the person was discharged to inpatient rehabilitation or home immediately after acute care. The predictor of interest for the secondary question was inpatient rehabilitation LOS. This variable was modeled with 5 levels: 1 to 13 days (level 1), 14 to 27 days (level 2), 28 to 55 days (level 3), 56 to 83 days (level 4), and 84 days (level 5). Potential Confounding Variables A confounding variable in regression analysis is defined as a variable independently associated with both the predictor of interest (ie, discharge disposition, LOS) and the response variable (ie, the 4 outcome measures). Because confounding variables can influence the relationship between the predictor of interest and outcome, a researcher must account for their effect before looking at the relationship between the predictor of interest and response variable. 28 We used the model-building strategy of a priori identification of potential confounding variables based on scientific knowledge to avoid basing the analysis on characteristics that might be specific to this data set. The potential confounders were added to the analysis as a group, followed by the predictor of interest. The following variables were identified as potential confounders: (1) basic demographics (age at time of injury, gender, race, education), (2) injury severity and circumstances (severity of brain injury, violent vs nonviolent nature of the brain injury, extent of other system injuries, time between injury and discharge from acute care), and (3) preinjury risk factors (preinjury work stability; history of psychiatric and/or central nervous system [CNS] condition; history of substance abuse, arrest, and/or juvenile court referral). A variable indicating study time period was included to capture possible changes in service delivery between the 2 studies. Basic demographics. The basic demographic variables were modeled as follows. Age at injury was modeled in 3 groups: less than 30 years, 30 to 49 years, and 50 years. The cutoff for the oldest group was based on evidence for poorer outcome from TBI after age ,30 The division between the 2 younger groups allowed comparison of young adults and older individuals and was similar to groupings used in prior studies. 29 Race was modeled as white or nonwhite. Education at time of injury was modeled with 3 levels: (1) less than high school graduation (including General Educational Development), (2) high school graduation or some college (including community college), and (3) college graduate. Individuals in high school at the time of injury were considered high school graduates. Injury severity and circumstances. The injury severity and circumstances of injury variables were defined and modeled as follows. Severity of brain injury was modeled with 6 levels. It was based on a composite of common severity indices including (1) postresuscitation GCS, a measure of depth of coma; (2) time from injury to consistently following simple commands (ie, GCS motor score, 6), a measure of length of coma 29,31 ; (3) nonreactive pupil(s); and (4) neurosurgery for specific intracranial lesions (ie, subdural and intracerebral hematomas). The mildest severity group (level 1) consisted of those participants with all of the following: (1) GCS scores from 13 to 15, (2) time to follow commands 24 hours, (3) reactive pupils, and (4) no neurosurgical interventions for subdural or intracerebral hematoma. The next most severe group (level 2) consisted of those with (1) either GCS scores from 9 to 12 or time to follow commands between 25 hours and 6 days, (2) reactive pupils,

4 1238 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell and (3) no surgery for subdural or intracerebral hematomas. Assignment to severity levels 3 through 6 was based on the following: (1) GCS scores of 3 to 8, (2) time to follow commands 7 days, (3) at least 1 nonreactive pupil, (4) surgery for subdural hematoma, and (5) surgery for intracerebral hematoma. Participants in level 3 had 1 of these criteria, level 4 had 2 of these criteria, and level 5 had 3 of these criteria, and so forth. Levels 6 and 7 were combined based on the small number of participants in the most severe level. For those participants missing GCS scores (primarily due to paralytic agents), the time to follow commands category was considered twice. The violent versus nonviolent nature of the brain injury was defined as a dichotomous variable. Violence-related brain injuries were those resulting from suicide attempts or assault (including fighting). Nonviolence-related injuries were those resulting from accidents (eg, motor vehicle or pedestrian accidents; falls). The extent of other system injuries was modeled as a dichotomous yes-no variable based on scores on the Abbreviated Injury Scale 32 (AIS). The AIS ranks severity of injuries for 9 body regions on a 6-point ordinal scale, ranging from minor injury to maximum injury (ie, so severe it is currently untreatable). Based on clinical experience, we defined an injury sufficiently severe to have potentially an impact on whether a person went to rehabilitation and on outcome as any score of 3 or greater on the upper extremity, lower extremity, or spine regions. The time between injury and discharge was defined as the time in days between the day of injury and the day the individual was discharged from acute care either to home or to inpatient rehabilitation. This variable was modeled with 6 levels: less than 8 days, 8 to 14 days, 15 to 21 days, 22 to 28 days, 29 to 63 days, and 64 to 213 days. Preinjury risk factors. Preinjury risk factors were defined and modeled as follows. Preinjury work stability was modeled with 3 levels: (1) nonworkers (eg, retired individuals, homemakers, students), (2) unstable workers (unemployed at the time of injury, working less than half-time, or at the same job 6mo), and (3) stable workers (working half-time or more at the time of injury and having been at the same job for at least 6mo). A history of significant psychiatric (eg, hospitalization and/or major diagnosis) and/or CNS condition (eg, learning disability) was modeled dichotomously. A history of substance abuse, arrest(s), and/or juvenile court referral(s) was modeled as a dichotomous yes-no variable based on a positive history for at least 1 of the following: (1) illicit drug abuse (eg, drugs interfered with life, got into trouble because of drugs, drug addiction), (2) treatment for alcohol abuse (eg, inpatient alcohol treatment, prior participation in Alcoholics Anonymous), (3) any prior arrest, and/or (4) any prior referral to juvenile court. Additional confounder. Study time period was modeled as either conducted between 1983 and 1987 or 1991 and Data Analysis The levels of each demographic, risk, and severity variable were analyzed for statistical differences between the group discharged to inpatient rehabilitation and the group discharged home. Chi-square tests were used for the dichotomous and unordered multilevel variables, and Mann-Whitney U tests were used for the ordered multilevel variables. 33 We visually inspected the relationship between each of the outcome measures and discharge to rehabilitation versus home, by severity level, before adjustment for confounders. A bar graph was used for analysis of the unadjusted GOS scores and box plot graphs for examination of the unadjusted SIP, Burden Inventory, and PQOL. In the box plots, the area within each box indicates values within the interquartile range (IQR), with the median denoted by the horizontal line. Separate multiple linear regression analyses were conducted for each of the 4 outcome variables. The significance level (2-tailed tests) was set in advance at.01 because of multiple comparisons. As noted earlier, in each instance, variables were entered into the model in 2 blocks, in the following order: (1) potential confounding variables and (2) the predictor of interest. Visual inspection of plots of Cook s D (a measure of how much the residuals of all the cases would change if a particular case were excluded from the calculation of the regression coefficients) and of centered leverage values (a measure of the influence of a specific point on the fit of the regression) was used to identify potential outliers and influential points. 28 Analyses were rerun without those points to determine the effect on the results. We evaluated adherence to the assumptions required for various levels of inference from linear regression analysis. The homoscedasticity assumption of equal variance 28 was evaluated by visual analysis of plots of the standardized residuals (ie, the distance of each point from the calculated regression line) against the standardized predicted values. The normal distribution assumption 28 was evaluated by visual analysis of cumulative proportion plots. Because these assumptions were not fully met by the initial linear regression models for the inpatient rehabilitation and home analysis, the outcome measures were subsequently transformed and reanalyzed. GOS values were dichotomized, with severe and moderate disability combined into 1 category, and analyzed by using logistic regression. The scores for the SIP, Burden Inventory, and PQOL were ranked and analyzed with linear regressions of the ranked values. We subsequently conducted additional analyses in response to questions raised by the results of the study. The relationship between Wechsler Adult Intelligence Scale Performance IQ (PIQ) scores at 1 month and severity categories for the discharge to inpatient rehabilitation or discharge home groups was examined by visual inspection of box plot graphs. A value of 1 less than the lowest observed value was substituted for those who were untestable at 1 month because of CNS impairment. Additional analyses for the rehabilitation and home question included logistic regression on the GOS, with the addition of 1 month PIQ as a potential confounder. RESULTS A total of 710 participants met the eligibility requirements for the original drug studies during the time period that the 1-year outcome was collected. Of those, 474 were discharged from acute care to inpatient rehabilitation or home (see Temkin et al 21,22 for additional follow-up information). Three of those discharged died, and 106 were lost to follow-up before the 1-year assessment. The remaining 365 participants with 1-year outcome on the GOS comprised the largest sample in this study. This yielded a follow-up rate of 77.5% (365/471). Of the 365 participants, 11 were subsequently excluded from the GOS regression analysis because they had 1 or more missing covariates. As seen in table 1, the inpatient rehabilitation and home groups were similar on all covariates with the exception of race, severity of brain injury levels, and time from injury to

5 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell 1239 Table 1: Percentage of Participants by Potential Confounders in the Home and Inpatient Rehabilitation Groups (GOS sample) D/C Home (n 171) D/C to Rehabil (n 183) Age (y) Gender.64 Male Race.02 White Education.48 Less than HS graduate HS graduate/some college College graduate Severity level.00 1 (mildest) (most severe) Violence-related injury.08 Yes Other system injuries.70 Yes Time from injury to acute care discharge (d) Preinjury work stability.59 Nonworker Unstable worker Stable worker History of psychiatric/cns condition(s).15 Yes History of substance abuse, arrest, juvenile court referral.38 Yes Study time period NOTE. Percentages do not sum to 100% because of rounding. Abbreviations: D/C, discharge; HS, high school. P acute care discharge. The group discharged home had a higher percentage of nonwhite participants, milder injuries, and were discharged from acute care more quickly. The relationship between each of the outcome measures and severity levels for the rehabilitation and home groups prior to adjustment for other covariates is presented in figures 1, 2, 3, and 4. As seen in figure 1, at the mildest severity level, the 2 groups had similar recovery as measured by the GOS. With increasing severity, the home group had better recovery compared with the rehabilitation group. As seen in figure 2, the median unadjusted problem endorsement on the SIP of participants in the home group was lower than the rehabilitation group at severity levels 2, 4, and 5; equal at level 3; and higher at level 1. There were no participants at the most severe level in the home group for comparison. As presented in figure 3, the median unadjusted negative burden reported by caregivers of participants in the home group was less than that of the rehabilitation group for all severity levels where a comparison was possible. As seen in figure 4, there was no consistent trend in the PQOL scores for the home and rehabilitation groups by severity level. Table 2 shows the associations between rehabilitation versus home discharge with the outcome measures after adjustment for the confounders. For individuals with the same values on all the other variables (eg, individuals with the same gender, same race group, same education level, similar severity score), discharge to rehabilitation in comparison with discharge home was associated in the initial linear regression models with (1) a trend toward a decrease in the GOS score (poorer recovery), (2) a nonsignificant increase in the SIP (more problems in everyday life), (3) a nonsignificant increase in Burden Inventory score (more negative burden), and (4) a nonsignificant increase

6 1240 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell Fig 1. Percentage in the GOS categories by severity level for home versus rehabilitation groups. in PQOL (better perceived quality of life). Secondary analyses without outliers and influential points yielded similar results. The results of the analyses of the transformed outcome scores are also included in table 2. Odds ratios from the logistic regression on the dichotomized GOS scores for the full sample and with the addition of the PIQ as a confounder are presented. In both analyses, for comparisons of individuals with the same values on all the other variables, discharge to inpatient rehabilitation was associated with worse odds of having good recovery on the GOS than a person discharged home (nonsignificant trend for analysis with the full sample). Rank regression was performed on the ranked scores for the SIP, Burden Inventory, and PQOL. These values indicate a nonsignificant increase in the SIP and Burden Inventory rank (worse outcome) associated with inpatient rehabilitation after adjustment for confounders. The PQOL rank regression found a trend toward increased rank (better outcome). Fig 3. Total negative Burden Inventory (caregiver burden) scores by severity level for home and rehabilitation groups (1 mildest level). Increases in Burden Inventory scores represent worse outcome. Circles indicate the outliers, ie, values greater than 1.5 but less than 3 IQRs from the end of a box. Asterisks indicate extremes, ie, values that are more than 3 IQRs from the end of a box. Separate analyses were conducted for the LOS predictor. There was a significant association for the combined LOS levels with the GOS (P.009), with worse outcome as LOS increased compared with the reference level (1 13d). (See table 3 for comparisons of each LOS group with the reference level.) There was a nonsignificant association of the combined LOS levels with the SIP (P.22), with similar outcomes for each LOS level compared with the shortest stays. On the Burden Inventory, there was a trend toward an association (P.047) with worse outcome as LOS increased. There was a nonsignificant association for the combined levels with PQOL Fig 2. The SIP scores by severity level for home and rehabilitation groups (1 mildest level). Increases in SIP scores represent poorer outcome. Circles indicate outliers, ie, values greater than 1.5 but less than 3 IQRs from the end of a box. Asterisks represent the extremes, ie, values that are more than 3 IQRs from the end of a box. Fig 4. Mean change in PQOL by severity level for home and rehabilitation groups (1 mildest level). Increases in PQOL scores indicate better outcome. Circles indicate outliers, ie, values greater than 1.5 but less than 3 IQRs from the end of a box. Asterisks indicate extremes, ie, values that are more than 3 IQRs from the end of a box.

7 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell 1241 Table 2: Associations Between Rehabilitation Versus Home Discharge and the GOS, SIP, Burden Inventory, and PQOL After Adjustment for Potential Confounders* Model n OR 99% CI P GOS Linear regression to Logistic regression to Logistic regression (plus PIQ) to SIP Linear regression to Rank regression to Burden Inventory Linear regression to Rank regression to PQOL Linear regression to Rank regression to Abbreviations: OR, odds ratio; CI, confidence interval. * As discussed in text, associations do not appear to be causal. Poorer outcome for rehabilitation group. Better outcome for rehabilitation group. PIQ at 1 month added as potential confounder. (P.08), with worse outcome for LOS levels 2 and 4 and better outcome for level 3 compared with the group with the shortest stays. There were no participants in the longest LOS level for the Burden Inventory and PQOL samples. DISCUSSION This study examined the feasibility of investigating the effectiveness of inpatient rehabilitation for TBI with an observational cohort design. The results indicated that discharge to inpatient rehabilitation from acute care compared with discharge home was associated with worse outcome on 3 measures (GOS, SIP, Burden Inventory) and with better outcome on 1 measure (PQOL). Longer inpatient rehabilitation LOS as compared with the shortest LOS was associated with worse outcome on all measures, with the exception of PQOL for 1 LOS level. Although only a few of the results were statistically significant, the poorer outcome on 3 measures for patients who were discharged to rehabilitation versus home and those who had longer rehabilitation LOSs was unexpected. The important question is whether these results represent the effects of rehabilitation or are because of methodologic artifacts. In observational cohort studies, an association between a predictor and an outcome does not necessarily mean that the predictor in question caused the outcome. Associations can also result from features of the study design or study implementation that reduce our confidence in inferences of causality. 17 In this study, the most likely explanation for the associations between rehabilitation or LOS and outcomes must be a confounding effect. With confounding, an apparent treatment effect results from the effect of 1 or more variables other than Table 3: Associations Between Rehabilitation LOS Levels and the GOS, SIP, Burden Inventory, and PQOL After Adjustment for Potential Confounders* Model n 99% CI P GOS 185 LOS level 2 (14 27d) to LOS level 3 (28 55d) to LOS level 4 (56 83d) to LOS level 5 (84 d) to SIP 172 LOS level 2 (14 27d) to LOS level 3 (28 55d) to LOS level 4 (56 83d) to LOS level 5 (84 d) to Burden Inventory 83 LOS level 2 (14 27d) to LOS level 3 (28 55d) to LOS level 4 (56 83d) to PQOL 94 LOS level 2 (14 27d) to LOS level 3 (28 55d) to LOS level 4 (56 83d) to * As discussed in text, associations do not appear to be causal. Poorer outcome for stated LOS level as compared with level 1 (1 13d). Better outcome for stated LOS level as compared with level 1 (1 13d).

8 1242 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell the predictor of interest. Although we used regression methodology to control for the effect of many of the variables thought to cause confounding in this population, in retrospect, we could not control for 1 confounder. It exists because the group assignments were based on clinical decisions. The confounder results from factors that are associated with poor outcome that mimic treatment effects. This may come about if the same factors associated with poor outcome serve as a basis for the decisions to refer individuals with TBI for inpatient rehabilitation and to provide inpatient rehabilitation for longer time periods. This type of confounding, termed confounding by indication 34 or exposure selection bias, 35 presents a serious concern because it can result in systematic biases substantial enough to cause even a reversal in the true effect. 36 Although it is impossible to know the exact nature of the criteria that the clinicians responsible for the care of participants in this study used to make decisions about discharge disposition or length of rehabilitation services, based on clinical experience, several factors could have had a confounding effect. For example, among individuals with similar severity of brain injury as measured in this study, those with worse personal support networks, more emotional and behavioral problems (eg, anger management, agitation), and/or worse brain injury related physical problems (eg, spasticity) may have been more likely to have been discharged to inpatient rehabilitation than home. These factors may also have contributed to the decision to provide rehabilitation services for longer periods. Longer stays for individuals who were otherwise similar to those with shorter stays could have been because of worse physical and emotional ability of the family (or other caregivers) to manage the person at home. These factors could affect the clinical decisions for rehabilitation services and could independently result in poorer outcome. In other words, the worse outcome for those receiving rehabilitation services in this study could have resulted from individuals receiving services having a higher likelihood of poor outcome on unmeasured indices. In addition to the factors identified earlier, there may be other, more subtle, factors related to these clinical decisions that may not be known or expressed. For ease of reference, we have termed all the factors that contribute to the decision for discharge to inpatient rehabilitation the rehab factor. At first glance, it appears that adding such a factor to the analysis (if it could be identified and measured) would allow comparison of individuals within the rehabilitation and home groups who needed inpatient rehabilitation. However, if clinicians applied the rehab factor reliably to make referrals for inpatient rehabilitation services, there would not be any participants in the home group with this factor. A comparison would only be possible to the extent that the clinical decisions were imperfect, ie, with people discharged home who, in fact, needed rehabilitation and vice versa. Similar limitations would apply to the use of a LOS factor for comparisons of those with differing LOSs. In addition to these factors for which this study design did not control, it appears that there was inadequate control for at least 2 of the included confounders, severity and rate of recovery. Objective evidence that traditional measures of neurologic severity may not adequately capture the severity of injury and/or recovery is found in the poorer performance of the rehabilitation group on a neuropsychologic measure (ie, the PIQ) at 1 month for all severity levels as seen in figure 5. We had hoped to control for the effect of severity by using a composite multilevel variable of several different types of injury severity indices, including both length and depth of coma. However, although such measures are the best clinical Fig 5. Relationship between 1-month Wechsler Adult Intelligence Scale PIQ scores and severity level (1 mildest level) for home and rehabilitation groups. Circles indicate outliers, ie, values greater than 1.5 but less than 3 IQRs from the end of a box. Asterisks indicate extremes, ie, values that are more than 3 IQRs from the end of a box. indices of severity available, they are not perfectly related to functional outcomes and may not be sufficient to capture fully the complexities of the nature and degree of injuries to brain structures. As would be expected with any observational study in which individuals with TBI have good access to rehabilitation services, there were only a small number of individuals in the home group at the higher severity levels. Although some might think that collapsing the higher severity levels (ie, levels 4 6) would moderate the results, this comparison actually results in a more negative effect for rehabilitation (eg, a GOS score of.16 worse for the rehabilitation group vs the home group with collapsed severity levels for the linear regression vs a GOS score of.15 worse with the original severity levels). We used the time from injury to discharge from acute care as a surrogate for brain injury recovery rate in the absence of a more direct measure. However, this modeling may have been inadequate to capture differences in the groups for the research question investigating inpatient rehabilitation versus home discharge. Longer acute LOS for some individuals discharged home may have primarily been because of medical complications (eg, respiratory compromise, internal injuries) that resolved by acute care discharge, did not require rehabilitation intervention, and did not have long-term consequences for recovery. On the other hand, longer acute LOS for individuals subsequently discharged to rehabilitation may have been because of slower recovery from brain injury (in addition to any medical complications) that could have contributed to referral for rehabilitation services and poorer outcome. This difference in the reason for longer stays would not have been captured by the variable indicating the extent of other system injuries that focused on limb and spine injuries. We had hoped that the addition of a measure such as PIQ to the analysis would have aided in capturing the effect. However, doing so did not substantially change the results. Unfortunately, even if one were able to identify, operationally define, and measure all the variables contributing to the group differences in this study and to add them as covariates to the regression analysis, selection biases related to the group

9 NONRANDOMIZED STUDIES OF TBI REHABILITATION, Powell 1243 assignment would remain and contribute to erroneous findings. Other researchers have discussed how systematic biases resulting from this type of confounding cannot be overcome by multivariate statistical techniques. As noted by Adams et al 37 in their discussion of the limitations of ANCOVA, covariate adjustment can control successfully for group differences that result randomly, but not for those that are systematic. Cook and Campbell 36 noted that although a treatment variable can be used successfully in regression for prediction purposes when it does not matter if a variable is a symptom or a cause, causal inference should not be based on such variables because they often lead to erroneous conclusions regarding the reason for the treatment effects. It is important to realize that such incorrect conclusions are not limited to findings of negative treatment effects. Studies finding positive effects are equally at risk for attributing cause to the treatment variable erroneously. The study by Aronow 13 that found a positive effect for rehabilitation may have avoided such biases by finding a population for the nontreatment group that only differed from the population for the rehabilitation group on access to rehabilitation services. On the other hand, it is possible that factors (eg, higher socioeconomic status or better acute medical care for those with access to rehabilitation) could have biased those results in the opposite direction, that is, toward a positive effect for rehabilitation. Similar concerns apply to the LOS study by Spivack et al. 16 An additional problem with this study is the possibility that the comparison group received similar treatment to the intervention group. In the inpatient rehabilitation and home sample, 30% of participants in the home group received outpatient rehabilitation services in the first year postinjury. Sixty percent of participants in the LOS sample also received outpatient rehabilitation services. These outpatient services may have been similar enough to the inpatient services to dilute an effect of inpatient rehabilitation. Although this study did not use matching strategies to control for group differences, the issues it raises are relevant to studies investigating rehabilitation effectiveness for TBI that do. Some of the variables that would be important to use in matching are difficult to identify and measure. Typical neurologic severity indices may not be sufficient to capture severity. When matching is used to select a nontreatment group from a population that does not have access to inpatient rehabilitation, it appears that including the rehab factor in the matching would improve the comparability of intervention and comparison groups. However, the elements making up this factor are difficult to identify, define operationally, and measure. In addition, the elements may vary by geographic location, facility, or clinician, thereby reducing comparability of study results. The issue of comparable groups appears difficult to overcome in nonrandomized designs. Although comparable groups are not guaranteed in randomized designs, they are more likely. Thus, 1 response to these limitations of nonrandomized designs is to revisit the possibility of randomized designs. Perhaps ethical concerns in the use of these designs could be addressed by randomizing participants to different interventions or to standard care versus standard care plus an intervention of interest. An additional possibility is to use underserved populations, such as individuals with Medicaid reimbursement, when randomization may be more acceptable given that they typically receive limited care. 38 With current shorter durations of rehabilitation, perhaps one should emphasize randomized trials that occur after inpatient rehabilitation discharge and that aim to improve long-term outcome. However, other practical difficulties with implementing randomized studies remain. CONCLUSION In examining the study design and methods used in this investigation of rehabilitation effectiveness for TBI, we found sufficient evidence to prevent us from concluding that the associations (whether resulting in better or worse outcome) are a result of inpatient rehabilitation. We found that sophisticated statistical methodology could not overcome basic design flaws. There can be problems with systematic biases between treatment and control groups. Furthermore, it is difficult to ensure that other interventions have not occurred simultaneously resulting in insufficient treatment differences between the groups. The issue of noncomparability of groups in studies of rehabilitation effectiveness for TBI is difficult to resolve but must be addressed to trust results indicating the effects of rehabilitation whether positive or negative. References 1. National Center for Injury Prevention and Control. Epidemiology of traumatic brain injury in the United States [online] Available at: Accessed Oct 31, NIH Consensus Development Panel on Rehabilitation of Persons with Traumatic Brain Injury. Rehabilitation of persons with traumatic brain injury. JAMA 1999;282: Malec JF, Basford JS. Postacute brain injury rehabilitation. Arch Phys Med Rehabil 1996;77: Mazaux JM, Richer E. Rehabilitation after traumatic brain injury in adults. Disabil Rehabil 1998;20: Traumatic Brain Injury Model Systems National Data Center. Traumatic brain injury facts and figures [serial online]. 1996;1(2). Available at: Accessed Oct 31, Whitlock JA, Hamilton BB. Functional outcome after rehabilitation for severe traumatic brain injury. Arch Phys Med Rehabil 1995;76: Hedrick WP, Pickelman HL, Walker W. Analysis of demographic and functional subacute (transitional) rehabilitation data. Brain Inj 1995;9: Abel U, Koch A. The role of randomization in clinical studies: myths and beliefs. J Clin Epidemiol 1999;52: Preventive Services Task Force. Guide to clinical preventive services: report of the U.S. Preventive Services Task Force. 2nd ed. Baltimore: Williams & Wilkins; Sackett DL, Rosenberg WM, Gray MJ, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn t. BMJ 1996;312: Evans RW, Ruff RM. Outcome and value: perspective on rehabilitation outcomes achieved in acquired brain injury. J Head Trauma Rehabil 1992;7(4): Hall KM, Cope DN. The benefit of rehabilitation in traumatic brain injury: a literature review. J Head Trauma Rehabil 1995; 10(1): Aronow HU. Rehabilitation effectiveness with severe brain injury: translating research into policy. J Head Trauma Rehabil 1987; 2(3): Chestnut RM, Carney N, Maynard H, Mann NC, Patterson P, Helfand M. Summary report: evidence for the effectiveness of rehabilitation for persons with traumatic brain injury. J Head Trauma Rehabil 1999;14(2): Salazar AM, Warden DL, Schwab K, et al. Cognitive rehabilitation for traumatic brain injury: a randomized trial. JAMA 2000; 283: Spivack G, Spettell CM, Ellis DW, Ross SE. Effects of intensity of treatment and length of stay on rehabilitation outcomes. Brain Inj 1992;6: Elwood JM. Critical appraisal of epidemiological studies and clinical trials. 2nd ed. Oxford: Oxford Univ Pr; Haffey WJ, Lewis FD. Rehabilitation outcomes following traumatic brain injury. Phys Med Rehabil State Art Rev 1989;3: