ATTENTION DEFICITS ARE nearly universal after traumatic

Transcription

1 966 SPECIAL SECTION: ORIGINAL ARTICLE The Moss Attention Rating Scale for Traumatic Brain Injury: Further Explorations of Reliability and Sensitivity to Change John Whyte, MD, PhD, Tessa Hart, PhD, Colin A. Ellis, ScB, Inna Chervoneva, PhD ABSTRACT. Whyte J, Hart T, Ellis CA, Chervoneva I. The Moss Attention Rating Scale for traumatic brain injury: further explorations of reliability and sensitivity to change. Arch Phys Med Rehabil 2008;89: Objective: To examine the interrater agreement and responsiveness to change of the Moss Attention Rating Scale (MARS), 22-item version, during acute inpatient rehabilitation after traumatic brain injury (TBI). Design: Observational study of clinician ratings (physical therapy [PT], occupational therapy [OT], speech-language pathology [SLP], nursing) of each patient s attentional function at 2 points in time, near the time of admission and near the time of discharge from inpatient rehabilitation. Setting: Dedicated acute inpatient brain injury rehabilitation program. Participants: Inpatients (N 149) with moderate to severe TBI (58% enrolled in the National Institute on Disability Rehabilitation Research funded Traumatic Brain Injury Model System); age 16 years or older; receiving OT, PT, SLP, and nursing care on the inpatient TBI rehabilitation unit; and having Rancho Los Amigos Levels of Cognitive Functioning Scale scores of IV (confused/agitated) or higher at enrollment. Patients were excluded if they had premorbid history of attentiondeficit hyperactivity disorder, major psychiatric disorder (eg, bipolar), or neurologic impairment (eg, stroke). Interventions: Not applicable. Main Outcome Measure: Scores on the MARS (22-item version) and its 3 factor scores. Results: Intraclass correlations among ratings from PT, OT, and SLP ranged from.69 to.78 at the initial assessment and.67 to.72 at the follow-up assessment. Agreement between nursing and the other disciplines was somewhat lower (at initial assessment,.59.68; at follow-up,.48.59), although still substantial. Agreement for 2 of the factor scores (restlessness and/or distractibility, initiation) was similar but agreement for the third factor (consistent and/or sustained attention) was lower (.25.27). The total MARS scores were highly significantly improved (P.001) at follow-up compared with initial assessment (mean, 27.6d between ratings; median, 21d; range, 4 125d) for each of the rating disciplines, with change scores ranging from 7.8 points (OT) to 13.1 points (nursing). Factor scores also improved significantly during the same interval. When different occupational therapists provided the initial and From the Moss Rehabilitation Research Institute, Philadelphia, PA (Whyte, Hart, Ellis); and Department of Rehabilitation Medicine, Jefferson Medical College (Whyte, Hart), Division of Biostatistics, Department of Pharmacology and Experimental Therapeutics, (Chervoneva), Thomas Jefferson University, Philadelphia, PA. Supported by the National Institute on Disability and Rehabilitation Research (grant no. H133A020505). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are associated. Reprint requests to John Whyte, MD, PhD, Moss Rehabilitation Research Institute, 60 E Township Line Rd, Elkins Park, PA 19027, jwhyte@einstein.edu /08/ $34.00/0 doi: /j.apmr follow-up OT ratings, these follow-up ratings were significantly lower, but this pattern was not seen among other rating disciplines. Conclusions: The 22-item MARS showed good interrater agreement among PT, OT, and SLP and lower but still acceptable agreement between nursing and the other disciplines. Two of the 3 factor scores also showed good agreement. The 22- item total score and all 3 factor scores were highly sensitive to change occurring during inpatient rehabilitation. These results show that the 22-item MARS is a reliable instrument for the observational rating of attentiveness in an acute TBI rehabilitation sample. Lower agreement between nursing and the other disciplines suggests that the less structured environment of the nursing unit compared with therapy sessions reduces interrater agreement. The utility of the factor scores, particularly the least reliable sustained and/or consistent attention factor, requires additional investigation. Further research on construct validity and impact of the use of the MARS on clinical practice are warranted. Key Words: Attention; Brain injuries; Rehabilitation by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation ATTENTION DEFICITS ARE nearly universal after traumatic brain injury (TBI), across the severity spectrum. 1 Neural networks subserving various attentional processes are thought to be widely distributed, with important nodes in the brainstem and frontal and parietal regions. 2 This, coupled with the characteristic occurrence of diffuse axonal injury and focal frontal damage, 1,3 may help account for the prevalence of these deficits. Consistent with control by several interactive neural networks, attention is thought to be a multifaceted cognitive process rather than a single dimension. 2,4 These attention deficits after TBI are a significant clinical problem, interfering with optimal participation of rehabilitation patients in learningbased tasks. Persistence of attention deficits after discharge may reduce work productivity and increase subjective distress, even in people with milder injuries. To effectively assess attention deficits after TBI and their response to pharmacologic or behavioral treatments, it is important that they be measured accurately. However, this is a challenging task, particularly for patients with the most severe problems. In many cases, these patients cannot complete standardized neuropsychologic tests intended to assess attention, and the tests available do not capture some of the real-world manifestations of inattentiveness that are troublesome to clinicians and caregivers. 5-7 For this reason, we developed the Moss Attention Rating Scale (MARS), an observational rating scale of attention-related behaviors, for use in assessing people with moderate to severe TBI, particularly those undergoing acute inpatient rehabilitation. The MARS, originally a 53-item scale (45 attention, 8 control items) that can be completed by clinicians working with patients based on their routine clinical interactions, was previously shown to have good agreement between occupa-

2 MOSS ATTENTION RATING SCALE FOR TRAUMATIC BRAIN INJURY, Whyte 967 tional therapists and physical therapists treating the same patient and to have good reliability and coverage of the severity spectrum characteristic of inpatient rehabilitation. 5 Further, Rasch and factor analyses successfully reduced the scale to 22 attention items and identified 3 correlated factors (factor 1, restlessness and/or distractibility; factor 2, initiation; factor 3, sustained and/or consistent attention) reflecting different aspects of attentiveness. 7 In this phase of the research we sought to extend our analysis of the psychometric properties of the MARS and examine the possibility of broader use of the instrument during inpatient brain injury rehabilitation. In particular, we wanted to examine the suitability of the MARS for use by speech-language pathologists and nurses, in addition to the originally studied occupational and physical therapists. Although the MARS may also be useful for neuropsychologists, we did not assess that in the current study because the observations of neuropsychology staff occur in more variable settings (eg, cotreatments with other therapists, one-on-one counseling sessions, family meetings). Specifically, we wanted to answer the following questions: (1) Is interrater agreement on the MARS among a broader set of disciplines (occupational therapy [OT], physical therapy [PT], nursing, speech-language pathology [SLP]) sufficient to justify its multidisciplinary use, despite the very different treatment environments in which observations are made? (2) Does agreement among disciplines differ between the initial and follow-up ratings (either because the scale performs differently at lower vs higher scores or because therapists are more familiar with patients near the time of discharge)? (3) How does agreement among disciplines on the overall MARS compare with agreement regarding individual factor scores? (4) Is the MARS sensitive to the degree of recovery occurring during inpatient rehabilitation? (5) Does sensitivity to change depend on whether the same or different clinicians rate at both time points? (This question was added after the study was begun and it became clear that it would not always be possible to have the same clinician provide both initial and follow-up ratings.) (6) Are individual attention factors of the MARS similarly sensitive to change? METHODS Participants Participants were inpatients (N 149) with moderate to severe TBI who were admitted for acute rehabilitation at Moss- Rehab Hospital. Many of these participants (86/149 [58%]) were enrolled in the Traumatic Brain Injury Model System (TBIMS) project supported by the National Institute on Disability and Rehabilitation Research. The remaining 63 participants were not enrolled in the TBIMS database because they were treated acutely in a non-tbims trauma center, but they were otherwise similar to the TBIMS sample and met all of the following inclusion criteria: participants had sustained moderate to severe TBI, including loss of consciousness, intracranial neuroimaging abnormalities, and/or abnormal neurologic examination consistent with head trauma; were age 16 years or older; were receiving OT, PT, SLP, and nursing care on the inpatient TBI rehabilitation unit; and had a score on the Rancho Los Amigos Levels of Cognitive Functioning Scale of IV (confused and/or agitated) or higher at enrollment. Patients were excluded if they had a premorbid history of attentiondeficit hyperactivity disorder, major psychiatric disorder (eg, bipolar), or neurologic impairment (eg, stroke). Patients who met all of these criteria and gave informed consent (or whose caregivers gave informed consent) were enrolled in the study. The study was reviewed and approved by the institutional review board of Albert Einstein Healthcare Network. Instrument The development of the MARS has been previously described. 5,7 Briefly, it was originally a 45-item Likert-type rating scale with response choices ranging from 1 (definitely false) to 5 (definitely true). Some of the items are worded such that a rating of 5 indicates attention impairment, whereas for other items that rating indicates attentiveness. Data collection for this project began before completion of the data analyses that justified reducing the MARS from 45 to 22 items and before determining the factor structure of the MARS. 7 Consequently, the long form of the MARS was used in this project. However, in view of the fact that we expect researchers and clinicians in the future to primarily use the 22-item form (appendix 1), we report below on analyses that used scores derived from those 22 items to address our aims. Thus, total MARS scores reported here could range from 22 to 110, with higher scores indicating better attention. We have conducted parallel analyses with the long form of the MARS, and results were consistent between the 2 scales. Procedure After enrollment, participants were rated on the MARS by their treating physical therapists, occupational therapists, speech therapists, and treating nurse over a 2-day period of normal treatment and interaction. This initial rating period occurred as soon after hospital admission as possible given several constraints. First, the study protocol required that any prescribed psychoactive medications be stable throughout the rating period. Second, the rating period was scheduled so that each rater treated the patient on both days; if necessary, the rating period was expanded to 3 days during which each rater treated the patient for at least 2 days. Third, patients who were admitted with Rancho scores less than IV were not enrolled until they recovered to meet this criterion. On average, initial ratings were made 11.5 days (median, 9d; range, 3 50d) after rehabilitation hospital admission. A total of 142 participants received initial ratings. Raters received minimal training in the use of the MARS, other than a reminder to complete all items even in the face of rating uncertainty, in view of the selfexplanatory nature of the scale items and our desire to assess its use in routine clinical environments without extensive training and supervision. A subset of 104 participants received a second MARS rating as close as possible to the time of discharge. When providing follow-up ratings, raters did not have access to their earlier ratings. This rating period was subject to the same constraints described above and took place at least 7 days after the initial rating with a few exceptions (1 subject had a between-rating interval of 4 days, 2 had an interval of 6 days). Participants whose initial ratings took place within 1 week of hospital discharge did not receive follow-up ratings. In most cases (60/149 [58%]), the follow-up rating was performed by the same 4 clinicians who performed the initial rating; however, in other cases a different clinician from one or more disciplines was treating the patient at the time of follow-up, and this change in rater is considered in the analyses reported below. Data Analysis All MARS rating scores (1000 observations on 149 subjects made by 53 raters) were modeled in a linear mixed-effects model 8 that incorporated random effects of rater and subject as well as the fixed effects of time (initial or follow-up rating),

3 968 MOSS ATTENTION RATING SCALE FOR TRAUMATIC BRAIN INJURY, Whyte Table 1: Characteristics of Participants (N 149) Characteristics Values Age (y) Mean SD Range Sex, n (%) Male 111 (74) Female 38 (26) Race, n (%) White 90 (60) Black 38 (26) Other minority 21 (14) Cause of injury, n (%) Transportation 55 (37) Fall 49 (33) Pedestrian 19 (13) Gunshot 6 (4) Assault 9 (6) Sports/other 11 (7) Education (y) Mean SD Range 1 20 FIM score at rehabilitation admission (n 141) Mean SD Range GCS score at ED admission (n 110) Mean SD Range 3 15 Abbreviations: ED, emergency department; GCS, Glasgow Coma Scale; SD, standard deviation. discipline (nursing, OT, PT, SLP), and their interaction. Separate models were fitted for the 22-item score and each of 3 factor subscores. Initial inspection of residuals and random effects estimates showed that 1 individual rater was an outlier according to standard boxplot outlier detection rule, 9 applied to the random effects estimates, and removal of ratings performed by this rater (76 observations) substantially improved the overall fit of the models. Thus this rater was excluded from all analyses. Reliability of the MARS was assessed within the main mixed-effects model using the intraclass correlation coefficients (ICCs), based on the estimates of the variance components. 10 Pairwise ICC values were calculated for each pair of disciplines to allow comparison of the reliable use of the scale across disciplines. Note that although each discipline contributed only 1 rating per rating period, a statistically defined within-discipline ICC value can be calculated based on agreement for different participants whom other disciplines rated as equivalent. The 95% confidence intervals (CIs) for ICC values were computed from the asymptotic covariance matrix of the covariance parameter estimates from the mixed-effects models using the standard delta method. Sensitivity of the MARS to change was assessed within the same model. Although this model contains some participants who were rated only once, these participants do not affect the results of the analyses of change between 2 rating periods. Mean change (initial to follow-up) for each discipline and pairwise differences among each discipline s mean ratings at each time period were estimated from the main mixed-effects model, and significance was assessed with model-based t tests. Finally, participants rated by the same rater at both the initial and follow-up intervals within each discipline were compared with those rated by 2 different raters. The results reported here are based on 149 patients, rated by 52 staff members, for a total of 924 ratings, after exclusion of the outlying clinician s ratings. Different staff members contributed different numbers of ratings, and in some cases a given discipline s ratings were missing at one or another time point because of schedule changes; however, the mixed-effects model methodology accounts properly for such imbalances. RESULTS The demographic and clinical characteristics of the study participants are shown in table 1. The study participants were ethnically diverse and represented a broad age range. As is typical of TBI samples, the male-to-female ratio was approximately 3 to 1. The vast majority sustained their injuries in motor vehicle collisions, as pedestrians, or in falls, with a smaller number injured through blunt or penetrating assaults. The average participant had a high school education, had a moderate injury as defined by the emergency department Glasgow Coma Scale score, and was admitted with very severe disability as measured by the FIM instrument. Interrater Agreement Interrater agreement was examined separately at the initial and follow-up periods. Intraclass correlations are shown among all disciplines at both time points in table 2. As mentioned, the statistical model used is able to derive an index of agreement within a discipline and between disciplines, even though a patient was never rated by 2 members of the same discipline at the same time. Implicitly the model assesses the similarity of scores provided by 2 members of the same discipline when the different patients they are rating are rated equivalently by members of the other disciplines. As can be seen from the Table 2: Interrater Reliability of the MARS-22 Ratings Nursing OT PT SLP Initial ratings (n 142) Nursing.59 (.49.71) OT.64 (.56.73).69 (.61.79) PT.68 (.60.76).73 (.67.80).78 (.71.85) SLP.66 (.57.77).72 (.63.82).76 (.68.86).75 (.61.92) Follow-up ratings (n 111) Nursing.48 (.36.65) OT.58 (.48.70).69 (.58.82) PT.57 (.47.69).68 (.59.78).67 (.57.78) SLP.59 (.48.72).70 (.61.82).69 (.60.80).72 (.58.89) NOTE. Values are ICC (95% CI).

4 MOSS ATTENTION RATING SCALE FOR TRAUMATIC BRAIN INJURY, Whyte 969 Table 3: Interrater Reliability of Factor Scores Factor Initial (n 142) Follow-Up (n 111) F1: restlessness/distractibility.60 (.51.66).62 (.58.66) F2: initiation.52 (.45.57).55 (.39.67) F3: consistent/sustained.25 (.21.33).27 (.20.36) NOTE. Values are mean ICC (range). Values in parentheses in table 3 refer to the range of ICC results across disciplines, unlike those in table 2, which refer to the 95% CIs. table, agreement among disciplines was generally substantial (as defined by values of.60.80), despite the fact that the rating environments included individual speech therapy in a private office, self-care tasks in a patient s bedroom, and gross motor activities in a busy gymnasium and ranged from relatively prolonged treatment sessions to briefer interactions. Nevertheless, it appears that agreement among PT, OT, and SLP was essentially identical, but all 3 agreed less well with nursing (although only the difference in level of agreement among PT ratings differed significantly from those of nursing). In addition, it appears that agreement among disciplines was slightly lower at follow-up than at the initial rating period. We also examined interrater agreement on the 3 factor scores at the initial and follow-up periods. As can be seen from table 3, agreement was slightly lower for restlessness and/or distractibility and initiation (moderate to substantial) than for the total scale, whereas it was only fair for sustained and/or consistent attention. For each factor, similar to the full scale, agreement between nursing and the other disciplines was somewhat lower than among the other 3 disciplines themselves. However, there was no clear pattern of higher or lower agreement on factor ratings between the initial and follow-up periods (data not shown). Sensitivity to Change Because patients are routinely admitted to acute inpatient rehabilitation in a highly confused and disorganized state and many are discharged in a far more independent state, it is important that the MARS produces significantly different scores at these 2 time points as part of its validation. MARS ratings were not performed precisely at admission or just before discharge because psychoactive medications had to be stable during the rating intervals, because patients admitted with Rancho scores less than IV were not rated until they reached that score, and for a variety of other logistic reasons. The interval between the initial and follow-up ratings averaged 26.7 days (median, 19d; range, 4 125d). There was only a weak relationship between the amount of change seen on the MARS between these 2 ratings and the length of the interval (Spearman range, depending on discipline; P range,.02.06), perhaps because of longer lengths of stay among patients with slower recovery. Table 4 shows the mean initial and follow-up ratings of each discipline. Figure 1 shows the change between the initial and follow-up periods by discipline and the impact of being rated by different members of a discipline at the initial and follow-up periods (discussed in the next paragraph). Participants scores increased, on average, by 9.9 points from the initial to follow-up ratings (95% CI, ; P.001). The initial scores did not differ significantly by discipline, but there was a significant interaction between rating interval and discipline (P.003). Nursing scores at follow-up were significantly higher than OT (mean difference, 6.2 points; 95% CI, ; P.006) or PT (mean difference, 5 points; 95% CI, ; P.013) but not SLP. For some patients, the member of a particular discipline providing the follow-up rating differed from the person who performed the initial rating. This occurred between 12.7% (SLP) and 22.2% (nursing) of the time. Although statistical power is somewhat limited by the size of these subsamples, the only significant difference was found when a different occupational therapist provided the follow-up rating. A new occupational therapist rated the patient approximately 5.7 points lower than an occupational therapist who had provided the initial rating (95% CI, ; P.014), after adjusting for patient severity using the initial rating (see fig 1). All 3 factors were sensitive to change between the initial and follow-up periods (all P.001 for the main effect of assessment time) (fig 2). There was no consistent pattern across the 3 factors with respect to which disciplines provided the highest versus lowest ratings on a given factor. DISCUSSION These results extend our previous research 5,7 by showing that the MARS can be used by more disciplines than the original OT and PT samples we studied, by showing that the scale is sensitive to improvement occurring during inpatient rehabilitation, and by strengthening our confidence that the MARS is relevant to specific domains of attention as manifested in observable behavior. With respect to agreement among disciplines, these data suggest that physical and occupational therapists and speechlanguage pathologists all show substantial agreement about a patient s attentional function at a given point in time, although there were some systematic differences in absolute scores given to patients and in the amount of change perceived by specific disciplines between the initial and follow-up periods. In addition, within 1 discipline (OT), the amount of a patient s change from the initial to follow-up ratings depended on whether he/she was being rated by the same or a different occupational therapist at the different intervals. The pattern of agreement with nursing is more difficult to interpret. We suspect that the lower agreement between nursing and the other disciplines relates to the less structured and perhaps more variable observation environment of the nursing ward. It should be noted that agreement between nurse and nurse (statistically defined) was also low. This suggests that scoring is more variable in the nursing environment, not systematically lower or higher than in other settings. In contrast, the instances Table 4: Sensitivity to Change Discipline Initial Follow-Up Change P Nursing 72.2 ( ) 85.3 ( ) 13.1 ( ).001 OT 71.2 ( ) 79.1 ( ) 7.8 ( ).001 PT 70.2 ( ) 80.3 ( ) 10.1 ( ).001 SLP 71.9 ( ) 80.3 ( ) 8.4 ( ).001 NOTE. Values are mean (95% CI).

5 970 MOSS ATTENTION RATING SCALE FOR TRAUMATIC BRAIN INJURY, Whyte Fig 1. Mean initial and follow-up ratings ( 95% CI) are shown for (A) nursing, (B) OT, (C) PT, and (D) SLP. In each case, the follow-up ratings are shown with open circles when completed by the same clinician who provided the initial rating and with filled circles when a new therapist provided the rating. where there were significant differences in static ratings or ratings of change between the other disciplines suggest more systematic environmental differences (or, conceivably, disciplinary biases) that make particular attention problems more or less visible to each discipline. Nevertheless, the (moderate) agreement level within nursing (.48.59) would be sufficient to justify the use of the scale for that discipline alone. In multidisciplinary use of the scale, however, one should be sensitive to the possible differences in disciplines perspectives and should be cautious in using average scores across disciplines, particularly if a particular staff member is variably missing from the data. In addition, although agreement was, in general, quite good, we eliminated the ratings from 1 clinician as an outlier. This person represented less than 2% of the staff involved in this study but provided 7.6% of the total ratings. Clearly, outlying ratings such as this could occur during routine clinical use and might not come to light without the structured comparison with other raters that was possible in this study. It is possible that more extensive training in the use of the scale, particularly with feedback about discrepancies with other raters, might have resulted in more homogeneous rating practices. This study showed that improvement in attention during approximately 3 weeks of inpatient rehabilitation, as measured by the MARS and by its 3 specific factors, was highly significant (an average of about 10 points increase [11% of the available score range] over the 20d between ratings). This suggests that the scale can be used to monitor progress during inpatient rehabilitation and, potentially, to measure response to treatment, although that was not directly tested in this study. Study Limitations One limitation of this analysis is the fact that the same clinician (in most instances) assessed each patient at the initial and follow-up periods and may have been biased in expecting Fig 2. Means are shown for each factor score at the initial and follow-up periods. Abbreviation: NS, nursing.

6 MOSS ATTENTION RATING SCALE FOR TRAUMATIC BRAIN INJURY, Whyte 971 a (presumably high) level of change. The MARS, by definition, must be completed by someone who has known and interacted with the patient for at least 2 days, making blinded assessment very difficult to implement in a study like this. Analysis of perceptions of change when the patient was rated by the same versus 2 different members of a discipline suggested that in most cases the amount of perceived change was similar. However, a familiar occupational therapist appeared to see greater improvement than a new occupational therapist. In any case, statistical power to detect small amounts of this bias was limited because this was not an intended experimental manipulation. In addition, to the extent that such a bias is introduced through interdisciplinary team discussion, even changing the rating clinician would not fully protect against that bias. The current study attempted ratings only of patients at Rancho level IV and above. This cutoff was chosen because many of the MARS items make reference to the quality of a patient s participation in tasks. Because meaningful interaction with the environment is absent or questionable at Rancho levels II and III, these people are more appropriately assessed with instruments tailored to their limited level of consciousness. This study was limited by a number of methodologic and practical constraints. The fact that ratings could not be obtained at both the initial and follow-up periods for all study patients with all 4 consistent staff members introduces additional variability into the data. However, we believe that the statistical approach we used, which models these differences, was able to minimize their effects. An additional limitation of this study was the use of raw MARS scores for both the reliability and sensitivity to change analyses. As noted, a Rasch analysis was previously performed on the long version of the MARS and could, in principle, allow transformation of raw scores to logit scores on the underlying attentional dimension. A 1-point change in raw score implies a different amount of change on the underlying dimension depending on whether that change occurred near the middle of the scale range (small change) or near 1 of the ends of the scale (a larger change). Thus, by conducting our analyses on raw scores, we have introduced noise into the calculations of change (by using absolute differences in scores between initial and follow-up ratings even though those differences have different meanings along the scale) and into the calculations of agreement (because a 1-point score discrepancy between raters, similarly, has different implications at different points in the scale). However, use of raw scores would not be expected to bias the results and had the benefit of greater ease of use by allowing front-line clinicians to quickly compute scores. Although this study also suggests adequate interrater agreement for at least 2 of the MARS factor scores (restlessness and/or distractibility and initiation factors) and that scores on all 3 factors show highly significant recovery, it is too early to say whether these factor scores provide useful information not conveyed by the total MARS score. Future studies will need to examine whether individual factor scores are differentially related to different real-world outcomes, or respond to different treatments, to validate their individual utility. The relatively low reliability for the sustained and/or consistent attention factor also deserves further study. This finding does not seem to be due to an inherent instability caused by a low number of items in the subscale (because both this factor and the initiation factor contain 3 items). Rather, it may be that sustained and/or consistent attention is inherently difficult for different disciplines to agree on because, by definition, there is more variation on this factor throughout a day or several day periods compared with initiation and distractibility. Both of these factors are aspects of attention that may be observed readily within a single session. The variable interval between the initial and follow-up ratings (occasionally as short as 4d) may have reduced our ability to measure change with the MARS. However, despite the variable and sometimes short recovery interval, all measures of improvement on the MARS were highly significant. For the purposes of this study, the goal was not to measure the precise amount of change likely to occur in a given interval but rather to show that the MARS was able to detect clinically meaningful change during the inpatient rehabilitation stay, which is itself highly variable. CONCLUSIONS Attention is a multifaceted construct that may be difficult to assess with conventional psychometric measures, particularly early after injury or in more impaired patients. The 22-item MARS has been proposed as a reliable and clinically useful observational measurement tool for attention deficits post-tbi. The current findings show that MARS ratings near rehabilitation admission and discharge show acceptable interdiscipline reliability and excellent sensitivity to change over time during the subacute recovery period. Slightly lower agreement between nursing staff and therapy staff in OT, PT, and SLP may relate to the different environments in which observations are based. A number of future research directions are warranted based on these findings. Although we have expanded the use of the MARS to rehabilitation nurses and therapists, we have not yet assessed its use by neuropsychologists. Because we required that raters in this study have at least 2 days of clinical interaction with the patient being rated, we do not know the minimum amount of interaction required to provide accurate ratings, nor the limits on the types of observational environments in which accurate ratings can be obtained. In addition, although we have shown that the MARS is sensitive in statistical terms to the magnitude of improvement in attentiveness typically seen during inpatient rehabilitation, we do not yet know how much change is considered clinically meaningful and how different MARS scores relate to more global impressions of attentional impairment. Related to this, a comparison of raw and logit score computations would be of interest to see how substantial a difference this makes to the interpretation of MARS scores. Research to date on the MARS has been conducted on an acute inpatient rehabilitation sample. Although we already know that the MARS is less suitable for use with people with more subtle impairments, we have not yet determined whether the MARS can be used in chronic TBI when substantial impairments persist. Further research is also needed on the individual factor scores reflecting restlessness and/or distractibility, initiation, and sustained and/or consistent attention to determine whether these scores provide additional specific information useful in planning treatment or predicting real world problems. Acknowledgments: We thank Sooja Cho, MD, and Jeanne Pelensky, MD, for their assistance in medication and eligibility review for this study. We also thank the many rehabilitation therapists and nurses who provided the MARS ratings; Caron Morita, BA; Kelly Bognar, BA; and Jason McLaughlin, BA, for assistance in patient recruitment and Gemma Baldon, BA, for database management.

7 972 MOSS ATTENTION RATING SCALE FOR TRAUMATIC BRAIN INJURY, Whyte APPENDIX 1: MOSS ATTENTION RATING SCALE The 22-item MARS and its scoring principles (appendix 2). Moss Attention Rating Scale A. Subject s name ID# B. Person doing rating C. OT / PT / Sp / Nursing (circle) D. Complete the ratings based on two of the following three days of observations:,,, *Note: If you have worked with the patient all three days, base your ratings on the second & third day. Write the 2 dates on which your ratings are based on here E. &. Did any of the observations you made over the 2 days involve co-treatment with another rater? F. Y / N (circle)... Using the number key below, please indicate to what degree each descriptor applies to the person you are rating. If any of your sessions during the 2 days were done in co-treatment with another rater, please note this above, and please make sure that you don t discuss the rating scale at all with your co-rater until both of you have filled it out independently. Please don t leave any items blank. If you are not sure how to answer, just make your best guess. 1 Definitely false 2 False, for the most part 3 Sometimes true, sometimes false 4 True, for the most part 5 Definitely true 1. Is restless or fidgety when unoccupied 2. Sustains conversation without interjecting irrelevant or off-topic comments 3. Persists at a task or conversation for several minutes without stopping or drifting off 4. Stops performing a task when given something else to do or to think about 5. Misses materials needed for tasks even though they are within sight and reach 6. Performance is best early in the day or after a rest 7. Initiates communication with others 8. Fails to return to a task after an interruption unless prompted to do so 9. Looks toward people approaching 10. Persists with an activity or response after being told to stop 11. Has no difficulty stopping one task or step in order to begin the next one 12. Attends to nearby conversations rather than the current task or conversation 13. Tends not to initiate tasks which are within his/her capabilities 14. Speed or accuracy deteriorates over several minutes on a task, but improves after a break 15. Performance of comparable activities is inconsistent from one day to the next 16. Fails to notice situations affecting current performance, eg, wheelchair hitting against table 17. Perseverates on previous topics of conversation or previous actions 18. Detects errors in his/her own performance 19. Initiates activity (whether appropriate or not) without cuing 20. Reacts to objects being directed toward him/her 21. Performs better on tasks when directions are given slowly 22. Begins to touch or manipulate nearby objects not related to task Abbreviations: OT, occupational therapist; PT, physical therapist; SP, speech-language pathologist.

8 MOSS ATTENTION RATING SCALE FOR TRAUMATIC BRAIN INJURY, Whyte 973 APPENDIX 2: SCORING INSTRUCTIONS FOR THE MARS The table shows the scoring procedure for the 22-item version of the MARS. The first column provides the item number from the original MARS instrument, and the second column provides the number corresponding to the current 22-item form. To score the MARS, first transform the scores for the items that are reverse worded (indicated by 6 X in the column labeled scoring direction ). Once these items are transformed, the item scores are summed to get a total MARS score. Specific items also contribute to factor scores as indicated by the next 3 columns. Because the number of contributing items varies by factor, means of these items (after score transformation) are used to calculate the factor scores. Original Item Number New Item Number Item Text Scoring Direction Factor 1: Restless/ Distraction Factor 2: Initiation Factor 3: Sustained/ Consistent 1 1 Is restless or fidgety when unoccupied 6-X X 2 2 Sustains conversation without interjecting irrelevant or offtopic X comments 4 3 Persists at a task or conversation for several minutes X without stopping or drifting off 6 4 Stops performing a task when given something else to do 6-X or to think about 7 5 Misses materials needed for tasks even though they are within sight and reach 6-X 8 6 Performance is best early in the day or after a rest 6-X X 11 7 Initiates communication with others X X 13 8 Fails to return to a task after an interruption unless 6-X prompted to do so 16 9 Looks toward people approaching X Persists with an activity or response after being told to stop 6-X X Has no difficulty stopping one task or step in order to begin X the next one Attends to nearby conversations rather than the current task or conversation 6-X X Tends not to initiate tasks which are within his/her capabilities 6-X X Speed or accuracy deteriorates over several minutes on a 6-X X task, but improves after a break Performance of comparable activities is inconsistent from 6-X X one day to the next Fails to notice situations affecting current performance, eg, wheelchair hitting against table 6-X Perseverates on previous topics of conversation or previous 6-X X actions Detects errors in his/her own performance X Initiates activity (whether appropriate or not) without cuing X X Reacts to objects being directed toward him/her X Performs better on tasks when directions are given slowly 6-X Begins to touch or manipulate nearby objects not related to task 6-X X Total Mean Mean Mean References 1. Whyte J, Hart T, Laborde A, Rosenthal M. Rehabilitation issues in traumatic brain injury. In: DeLisa JA, Gans BM, Bockenek WL, editors. Rehabilitation medicine: principles and practice. 3rd ed. Philadelphia: Lippincott; p Fernandez-Duque D, Posner MI. Brain imaging of attentional networks in normal and pathological states. J Clin Exp Neuropsychol 2001;23: Povlishock JT. Pathophysiology of neural injury: therapeutic opportunities and challenges. Clin Neurosurg 2000;46: Fan J, McCandliss BD, Sommer T, Raz A, Posner MI. Testing the efficiency and independence of attentional networks. J Cogn Neurosci 2002;14: Whyte J, Hart T, Bode R, Malec JF. The Moss Attention Rating Scale for traumatic brain injury: initial psychometric assessment. Arch Phys Med Rehabil 2003;84: Whyte J, Hart T, Vaccaro M, et al. The effects of methylphenidate on attention deficits after traumatic brain injury: a multidimensional randomized controlled trial. Am J Phys Med Rehabil 2004;83: Hart T, Whyte J, Millis S, et al. Dimensions of disordered attention in traumatic brain injury: further validation of the Moss Attention Rating Scale. Arch Phys Med Rehabil 2006; 87: Vonesh E, Chinchilli V. Linear and nonlinear models for the analysis of repeated measures. New York: Marcel Dekker; Hoaglin D, Iglewicz B, Tukey J. Performance of some resistant rules for outlier labeling. J Am Stat Assoc 1986;81: Rousson V, Gasser T, Seifert B. Assessing intrarater, interrater and test-retest reliability of continuous measurements. Stat Med 2002;21: