THE STROKE DRIVERS Screening Assessment 1 (SDSA)

Transcription

1 324 Concurrent Validity of the Stroke Drivers Screening Assessment Kathryn A. Radford, PhD, Nadina B. Lincoln, PhD ABSTRACT. Radford KA, Lincoln NB. Concurrent validity of the Stroke Drivers Screening Assessment. Arch Phys Med Rehabil 2004;85: Objective: To determine the concurrent validity of the Stroke Drivers Screening Assessment (SDSA). Design: Comparison of the SDSA with criterion standards. Setting: Subjects homes in the community in the United Kingdom. Participants: Ninety-three stroke patients (age range, 22 83y) who were referred for assessment of fitness to drive or who had been driving before their stroke. Participants were assessed a median of 5 months poststroke. Interventions: Not applicable. Main Outcome Measures: Patients were assessed on the SDSA and tests of visuospatial ability (Visual Object and Space Perception [VOSP] Battery), executive abilities (Stroop Neuropsychological Screening Test, Trail-Making Test [TMT], Cognitive Estimates Test [CET]), and visual memory (Recognition Memory Tests [RMT], Verbal Descriptions of Road Signs [VDRS]). Results: The SDSA subtests all correlated significantly with the Stroop test (P.001) and TMT (P.001), which suggests that they measure executive abilities and attention. The SDSA Dot Cancellation (DC) also correlated significantly with the VDRS (P.01). The SDSA Square Matrices (SM) test correlated significantly with the VOSP cube analysis (P.01) and the RMT faces subtest (P.001), which suggests that the SM test also measures visuospatial abilities and visual memory. The SDSA Road Sign Recognition (RSR) test also correlated significantly with the VOSP cube analysis (P.05), which suggests that the RSR test also measures visuospatial abilities. Factor analysis produced a 2-factor solution with DC time, SM compass, and RSR all loading on factor 1, together with the Stroop and TMT. This factor was interpreted as executive abilities and attention. The RMT faces subtest and CET loaded onto a second factor. Conclusions: The SDSA seems to measure predominantly attention and executive abilities, which have previously been shown to be important determinants of safety to drive. This may account for the high predictive validity of the SDSA. Key Words: Automobile driving; Cerebrovascular accident; Cognition disorders; Rehabilitation; Stroke by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation From the School of Psychology, University of Nottingham, Nottingham, UK No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors(s) or upon any organization with which the author(s) is/are associated. Reprint requests to Kathryn A. Radford, PhD, Sch of Psychology, University of Nottingham, University Park, Nottingham, NG7 2RD, UK, Nadina.Lincoln@nottingham.ac.uk /04/ $30.00/0 doi: /s (03) THE STROKE DRIVERS Screening Assessment 1 (SDSA) was designed to predict whether stroke patients are fit to resume driving. In the development, 79 stroke patients were assessed on a battery of 14 tests of cognitive abilities thought to be required for driving. Their fitness to drive a car was then tested on the public roads by a UK Department of Transport approved driving instructor. The cognitive tests, which best predicted on-the-road performance, were selected for use as a screening procedure and were compiled together as the SDSA. 2,3 The SDSA comprises 3 tests Dot Cancellation (DC), Square Matrices (SM), and Road Sign Recognition (RSR) from which 6 scores are derived (DC time, DC errors, DC false positives, SM directions, SM compass, RSR). To predict fitness to drive, 4 of these scores (DC time, DC false positives, SM compass, RSR) are entered into an equation derived from discriminant function analysis. Although the SM directions score is not included in the equation, its completion is believed to prepare subjects for the more complex SM compass task. From the equation, a prediction of pass or fail is generated, which is used to recommend whether stroke patients have the cognitive abilities needed for safe driving. It is a better predictor of driving ability than standard clinical practice. 4 The SDSA correctly predicted the road performance of 81% of stroke patients in comparison with standard procedures, which correctly predicted 56%. It has good reliability over time. 5 Although the SDSA has predictive validity for driving performance, 2,3 the concurrent validity has not been determined. DC is a shortened version of the Bourdon-Wiersma stipple test 6 and was developed to assess attention. However, it was possible that it also involved other cognitive abilities. SM was developed from a children s game (What s in the Square?), which had seemed useful clinically 7 and has face validity, because it included items related to driving. RSR was developed to assess knowledge of road signs and to give the test battery used in the initial studies 2,3 face validity for driving. It was therefore not known which cognitive skills were being assessed in these tasks. Because the SDSA was being widely used to screen stroke patients wanting to resume driving, it seemed important that the SDSA be investigated further. The purpose of our study was to examine the concurrent validity of the SDSA. METHODS Stroke patients were identified from 3 sources: (1) patients assessed by Derby Regional Mobility Centre (DRMC) who wanted to resume driving after stroke; (2) patients discharged from Derbyshire Royal Infirmary (DRI) Stroke Unit who were more than 6 months after onset, had a Barthel Index score of more than 10, and who were driving in the 3 months before the stroke; and (3) patients randomized to receive an occupational therapy intervention in 1 center of a multicenter Trial of Occupational Therapy and Leisure 8 (TOTAL) who wanted to resume driving after a stroke. Sixty-two stroke patients were recruited from patients assessed by the DRMC during a year, 12 from those admitted to the DRI Stroke Unit, and 19 from those referred by therapists conducting the TOTAL trial. A total of 93 patients (81 men, 12

2 STROKE DRIVERS SCREENING ASSESSMENT VALIDATION, Radford 325 allocated to each correctly matched road sign (maximum, 12). Although developed as a measure of road sign knowledge, the task required subjects to interpret information from line drawings, relate this information to road situations, and decide which road sign (from a maximum of 20) best matched the situation. Scores from the 3 tests of the SDSA were entered into equations, derived from discriminant analysis, to provide an overall prediction of fitness to drive. They were also assessed on the following established tests of cognitive function to assess concurrent validity. Recognition Memory Tests The Recognition Memory Tests (RMT) faces subtest 9 is a measure of visual memory. It was considered appropriate for stroke patients because it allows retention to be assessed in patients with impaired language and is not restricted by motor disorders. 10 The number-correct score was used in the analysis. Fig 1. SM directions. women) were recruited and assessed. Their ages ranged from 22 to 83 years (mean standard deviation, y), and they were assessed between 3 and 112 months poststroke (median, 5.2mo; interquartile range, mo). Forty-four had right hemiplegia, 46 had left hemiplegia, and 3 had no lateralized signs. The participants were contacted by letter and asked to take part in a study to improve an existing cognitive assessment. Consenting participants were randomly allocated to 2 groups, which determined the order of assessments. Participants were assessed at home on the SDSA. 1 In the DC task, participants were presented with a sheet containing 625 groups of 3, 4, and 5 dots. They were required to cancel out all groups of 4 dots within a time limit of 15 minutes. The time taken to complete the task, the number of errors (groups of 4 dots that were missed), and the number of false positives (groups of 3 and 5 dots cancelled in error) were recorded. In the SM directions task, participants were presented with a board depicting a 16-square matrix. A set of large arrows facing in different directions was placed along the left-hand side, and a set of small arrows was placed across the top. Subjects were given a set of 16 cards depicting trucks and cars traveling in different directions. They were instructed to position these cards so that each truck was traveling in the direction of a large arrow and each car was traveling in the direction of a small arrow (fig 1). In the SM compass task, a set of 8 compass cards was placed around the board. The black arm of the compass card indicates a direction of travel. Participants were presented with 28 cards depicting two 3-dimensional model cars traveling in different directions. Participants were required to position the cards so that both vehicles on the cards were traveling in directions indicated by the compass cards. There were more cards than available spaces. Both parts had a time limit of 5 minutes. A maximum of 32 points could be scored that is, 1 point for each correctly placed vehicle. In the RSR test, participants matched a set of road signs with 12 pictures of road situations in 3 minutes. One point was Stroop Neuropsychological Screening Test The Stroop Neuropsychological Screening Test 11 is a measure of selective attention and concentration. The stimuli are color names printed in conflicting color ink. Patients were required to name the color of the ink of a color name rather than giving the name. The color word task produces an interference effect in which the color name hinders the verbal report of the color of the ink. The score used was the number correct on the color word task (ie, 112 minus number of errors and corrected errors). Trail-Making Test The Trail-Making Test (TMT), from the Halstead-Reitan Neuropsychological Test Battery, 12 is a measure of divided attention and complex conceptual tracking, which may reflect mental sequencing and the subjects ability to cope with complex stimuli. The time taken to complete Part B was used as the overall score. Cognitive Estimation Test The Cognitive Estimation Test 13 (CET) requires people to guess about quantities and dimensions that they would not be expected to know but about which they could be expected to make a reasonable estimate. It was included as a measure of executive function, which was independent of attention and visuospatial abilities. The CET comprises 10 questions, each scored on a 3-point scale, giving a possible range of 0 to 30. Verbal Descriptions of Road Situations The Verbal Descriptions of Road Situations (VDRS) was developed to determine whether the SDSA RSR measures road sign knowledge or whether it involves more complex cognitive abilities. A set of 20 road signs (the same road signs used in the RSR) was displayed in front of the participant. After an example, 12 descriptions of road situations were read aloud, and participants were required to point to the road sign that best matched the description they had just heard (eg, On a road where a set of traffic lights is out of action, which road sign might you expect to see? ). The participant was expected to point to the sign that indicates traffic light out of order that is, the traffic light with a red line through it. It was intended that the VDRS would test knowledge for road signs, without requiring visual reasoning skills. One point was allocated for each correct answer (maximum, 12). Visual Object and Space Perception Battery The Visual Object and Space Perception (VOSP) Battery cube analysis 14 was chosen to measure spatial ability, namely,

3 326 STROKE DRIVERS SCREENING ASSESSMENT VALIDATION, Radford Table 1: Relationship Between SDSA and Tests of Cognitive Function DC Time DC Errors DC False Positives SM Directions SM Compass RSR Stroop P CET P TMT P VDRS P VOSP cube analysis.15.22* * P RMT faces subtest P *Significant at the.05 level (2-tailed). Significant at the.01 level (2-tailed). Significant at the.001 level (2-tailed). the subject s ability to interpret 2-dimensional representations of 3-dimensional objects and space. The 10 items are scored as correct or incorrect, giving a possible score range of 0 to 10. The order of assessments was determined by random allocation to 2 groups. Subjects in group 1 were tested on the SDSA first, followed by the additional cognitive tests in a fixed order (RMT faces subtest, Stroop, TMT, CET, VDRS, VOSP cube analysis). Group 2 was tested on the additional cognitive tests, followed by the SDSA. We did this to overcome any order effect and to determine whether performance on the RSR was influenced by the VDRS. RESULTS The distribution of scores on the tests was examined to determine the nature of analysis that could be performed. Normality was examined by using the Kolmogorov-Smirnov- Lilliefors test. SM directions, DC errors, DC false positives, VDRS, and VOSP cube analysis were significantly skewed. Therefore, initial analysis was conducted using nonparametric statistics. Mann-Whitney U tests were used to compare the RSR and VDRS between patients assessed on the SDSA first or last. There were no significant differences between the 2 groups on VDRS scores, according to order of SDSA administration. However, patients who completed the RSR first performed significantly worse than those who completed it after the VDRS (P.001). Relationships between the SDSA tests and the additional cognitive tests were explored by using the Spearman rank correlations. DC time, DC false positives, and CET are negatively scored tests, that is, poorer performance is indicated by higher test scores. Missing data occurred on the CET and TMT because of aphasia (1 patient) and on the Stroop test because of aphasia (1 patient) and color blindness (2 patients). Results are shown in table 1. DC time correlated significantly with the Stroop test (P.001), TMT (P.001), and VDRS (P.01). This supports the assumption that DC measures attention. DC errors correlated significantly with VOSP cube analysis (P.05), which suggests that this score also measures visuospatial abilities. DC false positive did not correlate significantly with other measures. The SM directions and SM compass did correlate significantly and positively with VOSP cube analysis (P.01), which suggests measurement of spatial abilities, and the RMT faces subtest (P.001), which suggests measurement of visual recognition memory. There were also significant correlations with the CET (SM compass only, P.01), Stroop test (P.001), and TMT (P.001), which suggests that these tests also measure executive abilities. The RSR correlated significantly with VOSP cube analysis (P.05), Stroop test (P.001), and TMT (P.001), which suggests that this test also measures visuospatial and executive abilities. The RSR did not correlate significantly with the RMT faces subtest (P.05), which suggests that it does not measure visual memory. Correlation between the RSR and VDRS was also significant (P.001), which suggests that both tests measures similar abilities. Because these 2 tasks used the same materials, performance on them was compared. There was a significant difference (z 6.89, P.001), with patients performing significantly better on the VDRS than on the RSR, irrespective of the order of test administration. To explore the relationships between the additional cognitive tests and the SDSA tests further, multiple linear regression analyses were performed, with the effects of the common variance partialed out. Before analysis, data were screened for fit with the assumptions of multivariate analysis. One case was found to be a multivariate outlier and was excluded from the analysis. This patient was severely aphasic and performed poorly on tests requiring verbal skills. DC errors, DC false positives, SM directions, VOSP cube analysis, and VDRS were excluded from this analysis because they were not normally distributed. DC time, SM compass, RSR, RMT faces subtest, Stroop test, TMT, and CET were entered into the analysis. For each SDSA subtest, 1 measure (DC time, SM compass, RSR) was entered as a dependent variable and the established measures as independent variables. For DC time, only the TMT was a significant predictor of performance, accounting for 13% of the variance. For SM compass, the significant predictors were the Stroop, TMT, and CET, accounting for 32% of the variance. The Stroop and TMT made the greatest contribution to the prediction. The TMT accounted for most of the prediction of performance on the RSR, accounting for 21% of the variance. A principal components factor analysis with orthogonal (varimax) rotation was performed to explore relationships suggested by the correlations and to evaluate the extent to which the SDSA tests tap the same underlying constructs. Factorability was tested by using the Bartlett test of sphericity (119.5,

4 STROKE DRIVERS SCREENING ASSESSMENT VALIDATION, Radford 327 Table 2: Factor Matrix Factor 1 Factor 2 DC time.56 SM compass RSR.75 Stroop TMT.82 CET.81 RMT faces subtest.63 P.001) and the Kaiser-Meyer-Olkin measure of sampling adequacy (.75). Both tests suggested that the data sample was suitable for factor analysis. Two factors, explaining 56% of the variance, were extracted from the analysis. The tests loading most heavily onto the first factor were the TMT, RSR, DC time, SM compass, and Stroop test. This factor explained 39% of the variance. Those tests loading most heavily on the second factor, explaining 17% of the variance, were the CET and RMT faces subtest. Two tests (SM compass, Stroop) cross-loaded onto both factors. The factor loading and pattern of crossloading suggest that 1 main factor underlies performance on most of the SDSA. DC time, SM compass, and RSR all loaded on factor 1, together with the TMT and Stroop test. This factor has been interpreted as executive abilities and attention. The orthogonally rotated factor matrix, showing factor loadings (above 0.3) for each cognitive test, is given in table 2. DISCUSSION The SDSA tests were correlated with a group of established cognitive assessments in our attempt to determine the concurrent validity of the SDSA. All SDSA subtests were associated with the Stroop and TMT, which suggests that the SDSA predominantly measures executive abilities and attention. The DC was also related to the VDRS. The SM compass was significantly related to the CET (a measure of practical judgment using everyday knowledge), which is thought to involve selecting and evaluating cognitive plans before responding. 13 Good performance on the SM compass may require the ability to develop a response strategy, which suggests that the SM compass, like cognitive estimation, involves reasoning abilities dependent on frontal lobe function. The SM compass and RSR were both significantly related to VOSP cube analysis, which suggests that they require spatial awareness. However, the choice of cognitive tests in this evaluation was constrained by practical considerations. The tests chosen were quick and easy to administer, required no formal training, and could be easily transported and administered in patients homes. VOSP cube analysis was selected as a measure of visuospatial abilities, but because most patients scored the maximum on this test, a more sensitive measure would have been preferable. In addition, none of the validation tests are in themselves pure measures of any cognitive domain. Therefore, the validity of the SDSA can be established only insofar as any cognitive ability can be accurately assessed. The correlation between the RSR and VDRS suggests that, although they share measurement of some abilities, they are not identical. There was a significant difference in performance on the 2 tests. Apart from the emphasis on verbal skills in the VDRS, the main difference between the 2 versions seems to be that the RSR patients are required to devise and adopt a strategy for matching and checking the road situations with the road signs within a restricted time period, whereas the VDRS patients simply point to cards laid out before them. The pattern of results indicates that the RSR assesses more than simple memory for road signs and requires other cognitive skills. Overall, the results suggest that the SDSA is primarily a measure of executive abilities and attention. Although the pattern of correlations was consistent, the correlations were not high. This suggests that the SDSA may be tapping skills other than those assessed in this study. Further validation studies are needed to explore the nature of abilities assessed by the SDSA. However, the finding that the SDSA is predominantly a measure of attentional capacity and executive abilities is consistent with research relating to driving in other diagnostic groups. Among older drivers, selective attention has been found to be a major contributor to crashes. 15 In healthy older drivers and those with mild Alzheimer s disease, computerized measures of selective attention have been found to be better able to differentiate safe from unsafe drivers than dementia severity or traditional psychometric tests. 16 As a measure of visual attention, the Useful Field of View has been found to be a significant predictor of collisions involving older adults. 17,18 Tests of divided attention and information-processing speed have been found to be predictive of road test failure among other drivers with brain damage. 19 Executive abilities are thought to be important for car positioning, maintaining safe distances, driving on roundabouts, journey planning, 20 estimating risk, and for adapting behavior, such as adjusting speed to traffic conditions. 21,22 Our findings are compatible with research relating executive abilities to on-road performance among drivers with brain injury and, more specifically, with stroke. 19,26 The results suggest that the SDSA also measures visuospatial abilities. These are thought to relate to the ability to judge space and distance when monitoring oncoming cars and when overtaking and to the ease with which drivers notice cues that lie ahead, such as traffic lights and road signs, and cues that lie in the periphery, such as cars pulling out, cyclists, and pedestrians. 27 Our findings are consistent with other studies 26,28 that have found significant relationships between on-road driving and tests of spatial abilities among drivers with brain damage. CONCLUSIONS Although it is difficult to distinguish clearly between cognitive abilities required for specific driving tasks, research to date has indicated combinations of cognitive skills that are important for safe driving among people with brain damage. In view of these findings, the association between the SDSA subtests and known measures of executive abilities, attention, visuospatial skills, and memory would explain the SDSA s predictive validity for driving. 2,3 It also indicates the nature of cognitive skills that might need to be treated, if any attempt was being made to improve cognitive abilities to enable a stroke patient to return to driving. Acknowledgments: We thank Sarah Melly, Kate Hawkins, and Steven Lilley for assistance with data collection and Dr Christian Murray-Leslie and the staff of DRMC for their support. References 1. Nouri FM, Lincoln NB. The Stroke Drivers Screening Assessment. Nottingham (UK): Nottingham Rehab; Nouri FM, Tinson DJ, Lincoln NB. Cognitive ability and driving after stroke. Int Disabil Stud 1987;9: Nouri FM, Lincoln NB. Validation of a cognitive assessment: predicting driving performance after stroke. Clin Rehabil 1992;6: Nouri FM, Lincoln NB. Predicting driving performance after stroke. BMJ 1993;307: Lincoln NB, Fanthome Y. Reliability of the Stroke Drivers Screening Assessment. Clin Rehabil 1994;8:

5 328 STROKE DRIVERS SCREENING ASSESSMENT VALIDATION, Radford 6. Grewell F. Le test de Bourdon-Wiersma. Folia Psychiatr Neurol Neurochir Neerl 1953;56: Smith T. Assessing safe driving ability of head injured drivers: a pilot study [MSc thesis]. Cranfield (UK): Cranfield Institute of Technology; Parker CJ, Gladman JR, Drummond AE, et al. A multicenter randomized controlled trial of leisure therapy and conventional occupational therapy after stroke. TOTAL Study Group. Trial of Occupational Therapy and Leisure. Clin Rehabil 2001;15: Warrington EK. Recognition Memory Test. Windsor (UK): NFER-Nelson; Lezak MD. Neuropsychological assessment. 3rd ed. Oxford: Oxford Univ Pr; Trenerry MR, Crosson B, DeBoe J, Leber WR. The Stroop Neuropsychological Screening Test. Odessa (FL): Psychological Assessment Resources; Reitan RM, Wolfson D. The Halstead-Reitan Neuropsychological Test Battery: theory and clinical interpretation. Tucson: Neuropsychology Pr; Shallice T, Evans ME. The involvement of the frontal lobes in cognitive estimation. Cortex 1978;14: Warrington EK, James M. The Visual Object and Space Perception Battery. Bury St. Edmunds (UK): Thames Valley Test; Parasuraman R, Nestor PG. Attention and driving skills in ageing and Alzheimer s disease. Hum Factors 1991;33: Duchek JM, Hunt L, Ball K, Buckles V, Morris JC. Attention and driving performance in Alzheimer s disease. J Gerontol B Psychol Sci Soc Sci 1998;53: Ball K, Owsley C. Identifying correlates of accident involvement for the older driver. Hum Factors 1991;33: Owsley C, Ball K, McGuin G, et al. Visual processing impairment and risk of motor vehicle crash among older adults. JAMA 1998; 279: Lundqvist A, Gerdle B, Ronnberg J. Neuropsychological aspects of driving after a stroke in the simulator and on the road. Appl Cogn Psychol 2000;135: Hakamies-Blomqvist L. Research on older drivers: a review. IATSS Res 1996;20: Brouwer WH, Withaar FK. Fitness to drive after traumatic brain injury. Neuropsychol Rehabil 1997;7: Engum ES, Cron S, Hulse CK, Pendergrass TM, Lambert W. Cognitive behavioral driver s inventory. J Cogn Rehabil 1988;6: Brooke MM, Questad KA, Patterson DR, Valois TA. Driving evaluation after traumatic brain injury. Am J Phys Med Rehabil 1992;71: Christie N, Savill T, Grayson G, Ellison B, Newby G, Tyerman A. The assessment of fitness to drive after brain injury or illness. Crowthome (UK): Department of the Environment, Transport and the Regions, Transport Research Laboratory; Report No Lundqvist A, Alinder J, Alm H, Gerdle B, Levander S, Ronnberg J. Neuropsychological aspects of driving after brain lesion: simulator study and on road driving. Appl Neuropsychol 1997;4: Mazer BL, Korner-Bitensky NA, Softer S. Predicting ability to drive after stroke. Arch Phys Med Rehabil 1998;79: Simms B. Perception and driving: theory and practice. Br J Occup Ther 1985;Dec: Akinwuntan AE, Feys H, DeWeerdt W, Pauwels J, Baton G, Strypstein E. Determinants of driving after stroke. Arch Phys Med Rehabil 2002;83: