FOR BOTH RESEARCH PURPOSES and the assessment

Transcription

1 1989 The Mayo-Portland Participation Index: A Brief and Psychometrically Sound Measure of Brain Injury Outcome James F. Malec, PhD From the Department of Psychiatry and Psychology, Mayo Clinic College of Medicine, Rochester, MN. Supported in part by a Traumatic Brain Injury Model System grant from the National Institute of Disability and Rehabilitation Research, US Department of Education (grant no. 133A ). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors(s) or upon any organization with which the author(s) is/are associated. Reprint requests to James F. Malec, PhD, Generose ME TBI, Mayo Clinic, Rochester, MN 55905, malec.james@mayo.edu /04/ $30.00/0 doi: /j.apmr ABSTRACT. Malec JF. The Mayo-Portland Participation Index: a brief and psychometrically sound measure of brain injury outcome. Arch Phys Med Rehabil 2004;85: Objective: To evaluate the internal consistency, interrater agreement, concurrent validity, and floor and ceiling effects of the 8-item Participation Index (M2PI) of the Mayo-Portland Adaptability Inventory (MPAI). Design: M2PI data derived from MPAIs completed independently by the people with acquired brain injury undergoing evaluation, their significant others, and rehabilitation staff were submitted to Rasch Facets analysis to determine the internal consistency of each independent rater group and of composite measures that combined rater groups. Correlations with the full-scale MPAI were examined to assess concurrent validity, as was interrater agreement. Setting: Outpatient rehabilitation in academic physical medicine and rehabilitation department. Participants: People with acquired brain injury (N 134) consecutively seen for evaluation, significant others, and evaluating staff. Interventions: Not applicable. Main Outcome Measures: The MPAI and M2PI. Results: The M2PI showed satisfactory internal consistency, concurrent validity, interrater agreement, and minimal floor and ceiling effects, although evidence of rater bias was also apparent. Composite indices showed more desirable psychometric properties than ratings by individual rater groups. Conclusions: The M2PI, particularly in composite indices and with attention to rater biases, provides an outcome measure with satisfactory psychometric qualities and the potential to represent the varying perspectives of people with acquired brain injury, significant others, and rehabilitation staff. Key Words: Brain injuries; Rehabilitation; Treatment outcome by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation FOR BOTH RESEARCH PURPOSES and the assessment of rehabilitation and other clinical outcomes, a brief, easily completed, psychometrically sound measure of community integration after acquired brain injury would be of value. A brief measure that focuses on participation as represented by concrete indicators of community integration would be useful in a number of applications. Such applications include outcome evaluation of rehabilitation programs and other interventions in the longer term when follow-up is limited to a brief personal or telephone contact with the participant. Research studies of rehabilitation and medical interventions as well as of the natural history of recovery after various types of acquired brain injury would also benefit from the availability of a brief participation measure that could document real-world outcome through a relative brief, low-cost follow-up process. A short measure representing key aspects of community integration is desirable not only because of its brevity of completion time but more importantly because judging the level of independence and participation in community-based activities often does not require an extensive clinical evaluation. Furthermore, studies that include measures of ability, activity, and community reintegration suggest that success in community reintegration may be the most difficult to achieve (Johnson MV et al, unpublished data, 2004). 1,2 Indicators of community participation represent the acid test of outcome both metrically as well as philosophically. Currently, a generally acceptable measure of this type is not available. The most commonly used brief measures include the Glasgow Outcome Scale 3,4 (GOS), Disability Rating Scale 5 (DRS), and the Community Integration Questionnaire 6 (CIQ). The GOS in its original 5-level form shows significant ceiling effects in a postacute sample, as does the DRS. 7,8 The Glasgow Outcome Scale Extended 6 (GOS-E) provides a slightly greater range of possible scores (8 levels) than the original GOS. However, because the GOS-E requires a single rating on a strictly unidimensional scale, it may result in an inaccurate representation of community integration. For instance, a person who cannot shop or travel without assistance will obtain a relatively low score on the GOS-E even though he/she is participating in the community in other ways, such as through vocational, social, and recreational activities. The CIQ shows greater variability in an outpatient sample without an extremely high percentage of ceiling effects, 8 but has been criticized on a number of other counts, including limited breadth and consistency in measurement. 9 Hall et al 8 reviewed a number of outcome measures in a postacute brain injury sample and found that a large percentage ( 50%) of the sample had maximum (or minimum) scores for most measures. Hall reported that specific items from the CIQ and other more extended measures, such as the Craig Handicap Assessment and Reporting Technique and Neurobehavioral Functioning Inventory that focus on cognition and employment variables, showed a more appropriate distribution and appeared more resilient to ceiling effects. 8 However, extreme scores were present even for these measures in 25% to 35% of the cases. Hall 8 was unable to recommend a specific measure as a criterion standard for brain injury outcome. Hall 8 suggested developing new measures by using contemporary psychometric techniques, such as multivariate and Rasch analyses. New measures should include the best item content, particularly scales for employment and cognition, from existing measures.

2 1990 M2PI: A BRIEF MEASURE OF BRAIN INJURY OUTCOME, Malec Another issue for outcome measurement after acquired brain injury is the reporting source, that is, a person with acquired brain injury, family or significant other, or professional staff. Each of these rater groups shows characteristic perspectives and biases in assessing outcome after brain injury. 10,11 This is an unfortunately neglected area in the study of brain injury outcome measurement. There has been no direct attention to rater bias other than our own previous work. 10,11 All extant measures (both brief and extended) appear to be based on the assumption that the person completing the measure is reporting accurately. Extensive literature documenting impaired selfawareness after brain injury raises concern about accuracy of self-report by patients with acquired brain injury. Even these studies typically assume that the evaluations of medical or rehabilitation staff (to which patient self-report is compared) represent an accurate criterion standard for outcome assessment without providing any test or support for this assumption. The Mayo-Portland Adaptability Inventory (MPAI), now in its fourth revision (MPAI-4), was developed by using contemporary multivariate and Rasch psychometric techniques. The MPAI-4 is a 29-item inventory that includes items for rating key aspects of physical and cognitive ability; emotional, behavioral, and social adjustment; and community integration. The MPAI has demonstrated reliability and validity. 1,2,11,17-19 The instrument may be completed independently by people with acquired brain injury, significant others, and staff to communicate their specific perspectives on the assessment of disability and outcome provided by each of these rater groups. Recent analyses 1 of MPAI data obtained on 386 persons involved in residential, outpatient, or community-based rehabilitation in the midwest, southwest, west, and southeast regions of the United States found good internal consistency for the overall measure as well as for subscales for ability, adjustment, and participation. The participation index showed adequate internal consistency by Rasch indicators (person reliability.78; item reliability.98) as well as by the more traditional psychometric index (Cronbach.83). In a subsequent study of MPAI-4 data provided by rehabilitation staff, significant others, and people with acquired brain injury in an outpatient setting, the reliability of the subscale scores for each rater group was confirmed. 11 This study also revealed potential rater biases. Potential sources of bias on the part of the person with brain injury may include lack of self-awareness, a tendency toward positive self-evaluation, and aspirational bias (ie, rating one s aspirations rather than one s current status). Outcome ratings assessed by significant others may be biased by their motivation to be an advocate for the person they care for and their sensitivity to the impact and burden of the difficulties of said person. The enhanced objectivity of staff may be mitigated by more limited sensitivity to and experience with the person with acquired brain injury. This study concluded that, although rater bias presents an obstacle to accurate outcome measurement, examination of outcome ratings from multiple sources is clinically useful to understand the perspectives of each of these sources and results in the most comprehensive representation of outcome. 11 That the issue of rater bias has generally been ignored in the development of other brain injury outcome measures makes this issue no less pertinent to these other measures. In the current study, the potential of the 8-item MPAI participation index (M2PI) to serve as a brief outcome measure was examined with further attention to the issue of rater bias. As mentioned previously, there is no generally accepted criterion standard for the evaluation of brain injury outcome. However, the 29-item MPAI-4 full scale serves as a reasonably representative index of key aspects of the most common sequelae of acquired brain injury. Through Rasch techniques, full-scale ratings by each of the 3 groups (patients, significant others, staff) may be combined to form a composite index that potentially minimizes (by essentially summing over) opposing biases of each of these rater groups. In the absence of a generally accepted criterion standard for the evaluation of brain injury outcome, the MPAI-4 full scale composite index served in this study as the comparison measure for evaluating the briefer M2PI as completed by each group. The objectives of this study were to compare independent and composite ratings on the 8-item M2PI completed by people with acquired brain injury, significant others, and staff with the composite index obtained from the 29-item full scale MPAI-4; to examine interrelationships and agreement among Rasch M2PI measures for individual rater groups and composite indices; and to evaluate the range and distribution of M2PI scores for ceiling and floor effects. METHODS Participants Data for the MPAI-4 full scale and M2PI were obtained from a sample of 134 people with acquired brain injury who participated in consecutive evaluations for outpatient brain injury rehabilitation through the physical medicine and rehabilitation department at a major academic medical center in Minnesota. Data from this sample for the full inventory were reviewed in a previous study. 1 Under Minnesota law, patients in health care facilities must authorize that their health care data may be used in research. Nine individuals who would otherwise have been included in this series had declined to authorize use of their medical data for research and consequently were not included. Demographic and medical characteristics of the sample are shown in table 1. Half of the sample was seen within 1.8 years of injury or onset. Among the other half, many were evaluated a number of years after the initial event. Consequently, early injury-related information was not available in many of these remotely injured cases. The trauma cases were classified based on available data as follows. Cases were classified as severe if they met any of the following criteria: (1) Glasgow Coma Scale (GCS) score less than 9, (2) loss of consciousness (LOC) for more than 24 hours, or (3) posttraumatic amnesia (PTA) for more than 7 days. Those not classified as severe were classified as moderate if any of the following criteria were met: (1) GCS score between 9 and 12, (2) LOC from 1 to 24 hours, (3) PTA for 1 to 7 days, or (4) computed tomography (CT) scans or magnetic resonance images positive for trauma-related intracranial abnormality. Those not classified as moderate or severe by this system were classified as mild if data were available to indicate that any of the following applied: (1) GCS score between 13 and 15, (2) LOC for less than 1 hour, and (3) PTA for less than 24 hours. If insufficient data were available for classification using this system, severity was unknown. Because of the extended time between injury and evaluation and associated difficulty in obtaining injury-related information, data were unavailable for classification in 15% of the traumatic cases. Measures A copy of the full MPAI-4 and subscales including the M2PI is available electronically (at mpai) along with scoring procedures and an administration manual. In each of the 134 cases included in the sample, the

3 M2PI: A BRIEF MEASURE OF BRAIN INJURY OUTCOME, Malec 1991 Table 1: Sample Characteristics (N 134) Gender (%) Male 61 Female 39 Age (y) y; 38y; 17 77y Race (%) White 92 Nonwhite (African American, Native American, Hispanic, mixed) 8 Education (%) Less than HS, HS with special education, GED 13 HS degree, HS and some college 56 College degree, advanced degree 31 Type of injury (%) Traumatic (TBI) 65 Cerebrovascular accident, other vascular 15 Resected tumor 8 Encephalitis, infection 5 Other (including anoxia, toxic exposure, multiple sclerosis) 7 Severity of TBI (n 87) (%) Mild 29 Moderate 12 Severe 44 Unknown 15 Time since injury/onset y; 1.8y; 1mo 43.4y NOTE. Values are percentage or mean standard deviation, median, and range, unless otherwise noted. Abbreviations: GED, General Education Development diploma; HS, high school. TBI, traumatic brain injury. full scale was completed by consensus of a rehabilitation team who had evaluated the person with acquired brain injury. In almost all cases, the evaluating team consisted of a clinical neuropsychologist, occupational therapist, physiatrist, physical therapist, rehabilitation nurse, social worker, speech therapist, and vocational counselor. In 111 of the 134 cases, the subject being evaluated also independently completed the inventory with initial instruction from the rehabilitation nurse. The 23 people who did not complete the inventory did so either because they were unwilling or cognitively unable to do so. In 100 of the 134 cases, a significant other accompanied the patient to the evaluation and also independently completed the full scale as it pertained to said person. Staff did not review results of the inventories completed by people with acquired brain injury or significant others until they had completed their own consensus ratings. For the set of analyses reported here, full-scale and M2PI data were derived from these inventories and scored according to procedures described at the web site earlier. Through Rasch analyses performed with the Facets computer program, 20,21,a measures were obtained for the M2PI ratings made independently by people with acquired brain injury, significant others, and staff, as well as for composite measures derived by combining pairs of ratings and for a composite measure combining ratings by all 3 rater groups. In Rasch terminology, measure refers to the interval-equivalent scale resulting from logarithmic conversion of item data through Rasch analysis. 22,23 Throughout this article, the term measure will be reserved to refer to this type of scale. Additional analyses were performed by using Microsoft Excel b and SAS. c RESULTS Internal Consistency of Independent and Combined M2PI Ratings Rasch Facets analysis revealed strong internal consistency for a composite index of the 8-item M2PI that combined ratings of staff, people with acquired brain injury, and significant others (table 2). In Rasch analysis, person reliability and separation describe the degree to which items and the overall measure consistently distinguish among persons measured; item reliability and separation describe the degree to which these items form a consistent hierarchy for measurement. 22,23 We examined Rasch indicators of internal consistency for the M2PI as completed independently by people with acquired brain injury, significant others, and staff, as well as composite indices that combined pairs of raters. As can be seen in table 2, ratings by staff and significant others showed acceptable person reliability (.80) and item reliability (.90). Person reliability for independent ratings by people with brain injury was weak (.74) but improved when combined with ratings by significant others or by staff. Person and item reliabilities were acceptable in all cases for composite indices based on any 2 pairs of raters. Correlations of Independent and Composite M2PI Ratings With the Full Scale The first column of table 3 describes Pearson correlations between the Rasch measure for M2PI ratings and the Rasch measure for the 29-item full scale. The composite M2PI rating, which combines ratings by all 3 rater groups, correlated moderately to strongly with the full scale (r.77). M2PI measures that combine the ratings of people with brain injury and significant others (r.86) and those of persons with brain injury and staff (r.81) correlated with the full scale at about the same level or slightly higher. The M2PI measure that combines ratings by staff and significant others correlated less well with full scale (r.72). The full-scale measure correlated relatively well with independent M2PI ratings by people with brain injury (r.80), less well with significant others ratings (r.72), and even less well with staff ratings (r.61). Differences Among Rater Groups on M2PI Examination through Facets analysis of potential differences in rating level among the 3 rater groups revealed a highly Table 2: Rasch Indicators for M2PI Completed by Each of 3 Rater Groups and Composite Indices Participation Index Completed by: Person Reliability (Separation) Item Reliability (Separation) People with ABI, SO, and staff.89 (2.80).99 (9.80) (3-rater composite) People with ABI and staff.85 (2.43).99 (8.69) (2-rater composite) People with ABI and SO.84 (2.33).98 (7.26) (2-rater composite) Staff and SO (2-rater.89 (2.78).99 (8.96) composite) People with ABI.74 (1.70).97 (5.70) SO.82 (2.15).97 (5.50) Staff.85 (2.41).99 (8.17) Abbreviation: ABI, acquired brain injury; SO, significant others.

4 1992 M2PI: A BRIEF MEASURE OF BRAIN INJURY OUTCOME, Malec Table 3: Pearson Correlations Among Measures for MPAI Full Scale and M2PI Composite and Independent Ratings M2PI Completed by: Full Scale (29-item) 3-Rater Composite Index People With ABI, SO, and Staff People With ABI and Staff M2PI Completed by: People With ABI and SO SO and Staff People With ABI People with ABI, SO, and staff.77 (3-rater composite) People with ABI and staff (2-rater composite) People with ABI and SO (2-rater composite) Staff and SO (2-rater composite) People with ABI SO Staff SO reliable effect for rater group (rater reliability.98; rater separation 6.70). Fair mean item scores were 1.27 for people with acquired brain injury, 1.80 for significant others, and 2.0 for staff. Lower scores indicate greater community integration. Thus, people with brain injury tended to rate the extent of their involvement in independent and community activities as greater than did either significant others or staff. Correlations Among Independent and Composite M2PI Ratings As table 3 shows, composite M2PI ratings based on pairs of rater groups correlated very highly (r range,.90) with the 3-rater M2PI composite index. Two-rater composites also correlated highly with each other. Independent ratings by people with acquired brain injury, significant others, and staff correlated only moderately with each other (lower right-hand corner of table 3), with the lowest correlation between independent ratings by staff and people with acquired brain injury (r.50). Agreement Among Rater Groups on M2PI Table 4 displays agreement between pairs of raters and overall. As might be expected based on correlational data, agreement was higher between ratings made by staff and significant others and between significant others and people with acquired brain injury than between people with brain injury and staff. Overall agreement was higher for concrete indicators of functional tasks (self-care, transportation, employment) than for social and behavioral indicators (initiation, social contact). M2PI Distributions and Ceiling and Floor Effects Examination of cumulative distributions displayed in figure 1 reveals minimal floor or ceiling effects for the M2PI. (Raw scores for the 3-rater composite index were placed on the same scale as independent ratings by dividing the total score by 3; all scores were prorated for missing data.) As can be seen in figure 1, the cumulative distribution of raw scores for the 3-rater M2PI composite index approximates the S-curve characteristic of normal distributions. The curve for M2PI raw scores for ratings made by people with acquired brain injury climbs more rapidly than the 3-rater curve and reaches the median value (50th percentile) between scores of 10 and 11, showing the tendency of this group toward relatively favorable self-ratings. Despite this tendency, very low scores ( 3) were infrequent (9%) among self-ratings. The distribution for staff raw scores climbed less rapidly and found the median value at about 16, showing the tendency of this rater group toward higher, less positive ratings. The lower third of the distribution for significant others tracked closely to the distribution of persons with acquired brain injury then crossed over to track more closely with the distribution for staff. Both staff and significant other distributions had a low percentage ( 7%) of very low scores and very high scores ( 27) were rare for all groups ( 5%). Figure 2 displays distributions for composite indices derived from pairs of ratings compared with the 3-rater composite. To place figure 2 on the same metric as figure 1, total scores for paired ratings were divided in half. Effects of Injury Severity and Time Since Injury Possible relationships between injury severity and M2PI measures were examined for the 74 participants with traumatic brain injury for whom injury severity ratings were available. Analyses of variance (ANOVAs) were performed on each of the M2PI measures with the 3 severity categories (mild, moderate, severe) as the independent variable. Severity had a sig- Table 4: Percentage of Exact Agreements and Agreements Within 1 Point Between Rating Groups on Individual Items Items People With ABI and SO (%) SO and Staff (%) People With ABI and Staff (%) All Rater Pairs (%) Exact 1 Exact 1 Exact 1 Exact 1 Initiation Social contact Leisure Self-care Residence Transportation Employment Money management

5 M2PI: A BRIEF MEASURE OF BRAIN INJURY OUTCOME, Malec 1993 Fig 1. Cumulative distributions of M2PI total raw scores by rater group and 3-rater composite index. Abbreviations: ABI, acquired brain injury; SO, significant other. nificant effect for staff M2PI (F 4.52, P.01) and the staff/ significant other composite measure (F 5.38, P.01). Means across severity group for both these variables were in the expected direction: those with severe injuries were rated as lower in participation than those with moderate injuries and those with moderate injuries were rated as lower than those with mild injuries. Post hoc comparisons by using the Tukey honestly significant difference (HSD) test showed a significant difference only between the mild and severe groups for staff M2PI. For the staff/significant other composite measure, the Tukey HSD showed differences between the severe group compared with either the moderate or mild group but did not show a significant difference between the moderate and mild groups. ANOVAs of all other M2PI measures were not significant. We also examined a variable developed by subtracting the self M2PI measure from the staff M2PI measure. Differences among severity groups were highly significant (F 9.22, P.001). Those with mild injuries tended to rate themselves as more impaired than staff did, whereas those with moderate and severe injuries tended to rate themselves as less impaired than staff. Post hoc comparisons with the Tukey HSD revealed differences on this variable only between the mild and severe groups. Pearson correlations revealed consistently negligible correlations (r.07) between M2PI measures and time since injury. DISCUSSION Rasch analyses reported here are consistent with previous findings, 1 indicating that the M2PI generally shows satisfactory internal consistency in differentiating among people evaluated (person reliability, separation) and in describing a consistent hierarchical range of activities and behavior (item reliability, separation). The concurrent validity of the M2PI was supported by moderately strong correlations of M2PI measures with the measure representing the 29-item MPAI-4 full scale. Inspection of the raw score distributions of various M2PI measures reveals minimal floor and ceiling effects. The apparently satisfactory psychometric properties of the M2PI are likely caused by successive iterations and refinements of item content and scaling of the overall inventory that includes the M2PI subscale. 1,2 As suggested by Hall et al, 8 the M2PI includes items relevant to employment (employment, money management, transportation). The M2PI does not include items that reflect specific cognitive functions because this would have been contrary to the intention of this brief index to focus on participation and community integration and not impairment. However, the item for initiation is sensitive to executive cognitive impairment that may serve as an obstacle to some aspects of community integration for people with acquired brain injury who otherwise show a relatively high level of community involvement. The question of reporting source remains a salient one, that is, who provides the most accurate assessment of functional status for people with brain injury they themselves, their significant others, or professionals? This is not only a scientifically challenging but also a socially and politically sensitive question to address. Absent a criterion standard for outcome measurement after acquired brain injury, it is not possible to answer the question definitively. By the same token, development of such a criterion standard will require systematic research elaborating sources of rater bias and potential methods to address these biases. The current study provides some preliminary insights into this critical issue for outcome measurement after acquired brain injury. Although rater bias has not been systematically addressed in the brain injury outcome measurement literature, prior studies have reported differences between ratings made by people with acquired brain injury and their significant others. Similar to findings reported here, previous studies have documented generally moderate to strong correlations and concordance between ratings made by people with acquired brain injury and significant others. However, significant differences in ratings of cognitive and emotional symptoms 25 and aspects of community integration 24,26 were also noted with self-ratings, indicating less impairment or restriction than ratings made by significants. The literature on impaired self-awareness after brain injury 12,14,15 consistently shows that people with moderate to severe brain injury tend to underestimate impairment and limitations relative to staff estimates. This literature tends to consider such differences as evidence of impaired self-awareness, neglecting Fig 2. Cumulative distributions of total raw scores for 3- and 2-rater composite indices.

6 1994 M2PI: A BRIEF MEASURE OF BRAIN INJURY OUTCOME, Malec to consider the possibility that staff ratings may also be biased. The possibility that rater differences apparent in the present study were caused by imperfections of the M2PI rating scales cannot be entirely dismissed. However, that such differences have been reported across a variety of scales in the previous literature suggests that these differences represent rating source bias that merits further investigation. Clinically important differences in rater bias appear to be present in this and our previous work comparing MPAI measures completed independently by people with acquired brain injury, significant others, and rehabilitation staff. These prior studies 10,11 have suggested that possible sources of bias include self-awareness, values, perceived burden, aspirations, understanding of terms, and a tendency on the part of subjects with acquired brain injury toward positive self-assessments in contrast to a tendency on the part of rehabilitation staff to focus on impairments and obstacles. In the present study, contrasting cumulative raw score distributions (fig 1) clearly shows the relative biases of people with acquired brain injury to communicate greater community participation and of staff to rate these same people as more limited. Bias in making outcome ratings was explored in a study 11 that reviewed aspects of the full inventory. Impaired ability to make an accurate self-appraisal (ie, impaired self-awareness) may partially explain the tendency of people with acquired brain injury to rate themselves as more competent and participatory than staff members do. Other possible sources of bias for the person with brain injury include an aspirational bias (ie, rating oneself in terms of perceived potential rather than actual status) and a positive bias toward rating oneself at one s best. In contrast, staff and significant others, motivated by concern or desire to identify problems in order to be of help, may be biased to rate the person with acquired brain injury at their worst. Although potentially more objective, staff may also lack extensive experience with the patient with brain injury. Consequently, staff may make their ratings based on reports of problems that are concerning but occur rarely. Staff may fail to consider functioning in the context of the whole person, for instance, rating social and recreational activities as very impaired when in fact these activities are at a normal level for a person with disability who is managing a household, working, and raising a family. Staff may tend to set a higher standard for competency and safety than people with acquired brain injury set for themselves or their significant others set for them. In this regard, staff may be overly critical of the patient with brain injury. It is not unusual for people to confuse objectivity with hypercriticality. Significant others may also be overly critical of the person with acquired brain injury, based on a variety of motivations. Significant others may emphasize problems in their advocacy role, that is, emphasizing concerns to obtain help. Although staff may tend to be overly critical in an attempt to be objective, significant others may tend to emphasize problems because of their lack of objectivity and sensitivity to even small problems or concerns. Compassion and a lack of objectivity may lead some significant others to adopt the same perspective as their loved one with brain injury. Conversely, some significant others may deal with the stress of living with brain injury by attempting to adopt a more objective and critical view, like staff. The crossover of the distribution of ratings by significant others is intriguing and raises the possibility that many significant others tend to fall in 2 groups: 1 group that tends to view the situation of the person with brain injury similarly to that person, and another group that tends to be more conservative and critical in their assessments like staff. In the current study, Rasch analyses indicated that selfratings by people with acquired brain injury tend to be less reliable than ratings by significant others or staff. Comparisons within the trauma cohort by level of injury severity revealed no relationship to injury severity for M2PI ratings made by people with brain injury or significant others. In contrast, ratings made by staff independently or in combination with significant other ratings revealed an expected trend of lower participation ratings for those with more severe injuries. The lack of a relationship between injury severity ratings and self and significant other ratings raises a question about the validity of their M2PI ratings. Analysis of a variable constructed by subtracting the self M2PI from the staff M2PI revealed the tendency for those with mild injuries to rate themselves as more impaired and those with severe injuries to rate themselves as less impaired than staff do. This finding is consistent with clinical experience (ie, a portion of people with mild acquired brain injury presenting for rehabilitation services overemphasize their limitations, whereas a portion of those with severe acquired brain injury appear to lack self-awareness) and further challenges the validity of self-ratings in some cases. However, it is also possible that staff members were biased by knowledge of injury severity and that discrepancies between staff and self ratings reflect biases from both rater groups. The staff M2PI measure correlated less well with the 29-item 3-rater composite MPAI index than self or significant other measures. To some degree, this reflects discrepancies between self, significant others, and staff ratings, all of which contribute to the 3-rater composite index. However, without assuming that staff ratings provide the criterion standard, it is not possible to dismiss the combined assessment of self and significant others as inherently lacking in validity. Hence, the lower correlation between the staff M2PI measure and the combined MPAI measure invites consideration of a degree of bias or inaccuracy in staff ratings as well. Rater bias most likely also explains why correlations among measures derived from independent ratings by people with acquired brain injury, significant others, and staff were not consistently high (table 3). Nonetheless, overall levels of agreement (table 4) for specific items were generally acceptable and suggest that, despite concerns about rater bias in some cases, there is substantial agreement among raters in most cases. The method for assessing rater agreement may also influence results. Although overall agreement within 1 rating scale point is in the range of 66% to 82%, less frequent but extreme rating differences affect the Pearson correlation. For instance, the moderate Pearson correlation between the M2PI measure for staff and people with acquired brain injury probably represents the deviation of extreme scores (ie, people with brain injury who rated themselves very low but whom staff rated very high or vice versa) from the linear model. Because of the sensitivity of linear modeling to extreme scores, this approach has been criticized by advocates of Rasch modeling. 23 As previously suggested, 10,11 differences in ratings of acquired brain injury outcome among the minority of people with acquired brain injury, significant others, and staff do not necessarily represent unreliability of the scale; more likely they reveal varying perspectives and biases that are potentially of clinical and scientific significance. The 3-rater composite index of the M2PI provides a measure that combines ratings by people with acquired brain injury, significant others, and staff through Facets analysis and may provide a partial remedy to rater bias. The 3-rater M2PI composite index represents the perspectives of each of these 3 rating sources and, as such, may negate opposing biases. In-

7 M2PI: A BRIEF MEASURE OF BRAIN INJURY OUTCOME, Malec 1995 spection of Rasch indicators (table 2) and figure 1 suggests that the 3-rater composite measure has the most desirable psychometric properties among those examined here. For these reasons, the 3-rater M2PI composite appears to be a viable option for measuring outcome after acquired brain injury. Composite indices that include 2 raters correlate highly with the 3-rater composite. Figure 2 shows that the 2-rater composite that combines ratings by staff with those of patients with acquired brain injury most closely approximated the distribution of the 3-rater composite. For research or program evaluation, the 3- or 2-rater (staff/ patients with acquired brain injury) M2PI composites may serve as representative measures of outcome after acquired brain injury. In addition, researchers might further refine the measurement method by using a comparison variable (ie, self M2PI minus staff M2PI) to identify outliers who generate extremely discrepant ratings. Data for these outliers might either be eliminated from further analyses or submitted to arbitration methods. If M2PI data for a specific study can be submitted to Rasch analysis, examination of individual person fit provides another option for identifying discrepant or unusual ratings for specific individuals. Data reported here were based on inventories completed independently by each rater group. Evaluation of whether M2PI ratings that were done conjointly by 2 types of raters possess sound psychometric properties will be of interest. An example of such a rating situation is the common research procedure in which a staff person (in person or on the phone) questions and works with a person with acquired brain injury to complete the inventory. Evaluating the M2PI in such assessment situations will require future research. Data obtained here were for a specific sample in a specific evaluation situation. Thus, generalization of these findings to other groups and assessment environments would also require replication in other settings. Another question that remains for future research is whether M2PI measures show concurrent validity with other indicators. However, in the absence of any measure that approximates a criterion standard for outcome after acquired brain injury, it will be difficult to know whether a lack of coherence between the M2PI and another measure represents a weakness of the M2PI or a weakness in the other measure. As reviewed in the introduction, other brief measures commonly used in brain injury outcome research generally show substantial ceiling effects and other flaws in construction. CONCLUSIONS The M2PI provides a brief, psychometrically sound measure of outcome after acquired brain injury which may be completed by patients with acquired brain injury, significant others, and medical and rehabilitation staff. Indices that combine ratings by all 3 ratings sources or that combine self ratings and staff ratings result in measures with the most desirable psychometric properties. The concurrent validity of these composite indices was supported by their relatively high correlations with the MPAI-4 full-scale measure. These composite indices appear to provide measures of outcome after acquired brain injury that possess better psychometric qualities than other currently available brief measures. With attention to rater bias in specific cases, M2PI measures have the potential to represent the varying perspectives of people with brain injury, significant others, and staff in the assessment of community integration after brain injury. References 1. Malec JF, Kragness M, Evans RW, Finlay KL, Kent A, Lezak M. Further psychometric evaluation and revision of the Mayo-Portland Adaptability Inventory in a national sample. J Head Trauma Rehabil 2003;18: Malec JF, Moessner AM, Kragness M, Lezak MD. Refining a measure of brain injury sequelae to predict postacute rehabilitation outcome: rating scale analysis of the Mayo-Portland Adaptability Inventory. J Head Trauma Rehabil 2000;15: Jennett B, Snoek J, Bond MR, Brooks N. Disability after severe head injury: observations on the use of the Glasgow Outcome Scale. J Neurol Neurosurg Psychiatry 1981;44: Wilson JT, Pettigrew LE, Teasdale GM. Structured interviews for the Glasgow Outcome Scale and the Extended Glasgow Outcome Scale: guidelines for their use. J Neurotrauma 1998;15: Rappaport M, Hall KM, Hopkins K, Bellesa T. Disability rating scale for severe head trauma: coma to community. Arch Phys Med Rehabil 1982;63: Willer B, Rosenthal M, Kreutzer JS, Gordon WA, Rempel R. Assessment of community integration following rehabilitation for traumatic brain injury. J Head Trauma Rehabil 1993;8(2): Hall KM, Mann N, High WM, Wright J, Kreutzer JS, Wood D. Functional measures after traumatic brain injury: ceiling effects of FIM, FIM FAM, DRS, and CIQ. J Head Trauma Rehabil 1996; 11(2): Hall KM, Bushnik T, Lakisic-Kazazi B, Wright J, Cantagallo A. Assessing traumatic brain injury outcome measures for long-term follow-up of community-based individuals. Arch Phys Med Rehabil 2001;82: Dijkers M. Measuring the long-term outcomes of traumatic brain injury: a review of Community Integration Questionnaire studies. J Head Trauma Rehabil 1997;12(6): Malec JF, Machulda MM, Moessner AM. Differing problem perceptions of staff, survivors and significant others after brain injury. J Head Trauma Rehabil 1997;12(3): Malec JF. Comparability of Mayo-Portland Adaptability Inventory ratings by staff, significant others and people with acquired brain injury. Brain Inj 2004;18: Lam CS, McMahon BT, Priddy DA, Gehred-Schultz A. Deficit awareness and treatment performance among traumatic head injury adults. Brain Inj 1988;2: Prigatano GP, Klonoff P, O Brien KP, et al. Productivity after neuropsychological oriented milieu rehabilitation. J Head Trauma Rehabil 1994;9(1): Sherer M, Bergloff P, Levin E, High WM, Oden KE, Nick TG. Impaired awareness and employment outcome after traumatic brain injury. J Head Trauma Rehabil 1998;13(5): Sherer M, Boake C, Clement V, et al. Awareness of deficits after traumatic brain injury: Comparison of patient, family, and clinician ratings [abstract]. J Int Neuropsychol Soc 1996;2: Fordyce DJ, Roueche JR. Changes in perspectives of disability among patients, staff, and relatives during rehabilitation of brain injury. Rehabil Psychol 1986;31: Malec JF, Degiorgio L. Characteristics of successful and unsuccessful completers of three postacute brain injury rehabilitation pathways. Arch Phys Med Rehabil 2002;83: Malec JF, Buffington AL, Moessner AM, Degiorgio L. A medical/ vocational case coordination system for persons with brain injury: an evaluation of employment outcomes. Arch Phys Med Rehabil 2000;81: Malec JF. Impact of comprehensive day treatment on societal participation for persons with acquired brain injury. Arch Phys Med Rehabil 2001;82: Linacre JM. Facets for Windows. Chicago: Winsteps; Linacre JM. Many-facet Rasch measurement. Chicago: Mesa Pr; Wright BD, Masters GN. Rating scale analysis. Chicago: Mesa Pr; Bond TG, Fox CM. Applying the Rasch model: fundamental measurement in the human sciences. Mahwah: Lawrence Erlbaum; 2001.

8 1996 M2PI: A BRIEF MEASURE OF BRAIN INJURY OUTCOME, Malec 24. Sander AM, Seel RT, Kreutzer JS, Hall KM, High WM Jr, Rosenthal M. Agreement between persons with traumatic brain injury and their relatives regarding psychosocial outcome using the Community Integration Questionnaire. Arch Phys Med Rehabil 1997;8: Hart T, Whyte J, Polansky M, et al. Concordance of patient and family report of neurobehavioral symptoms at 1 year after traumatic brain injury. Arch Phys Med Rehabil 2003;84: Cusick CP, Gerhart KA, Mellick DC. Participant-proxy reliability in traumatic brain injury outcome research. J Head Trauma Rehabil 2000;15: Suppliers a. Winsteps, PO Box , Chicago IL b. Microsoft Corp, One Microsoft Way, Redmond, WA c. SAS Institute Inc, 100 SAS Campus Dr, Cary, NC