PERFORMANCE, WORKLOAD, AND STRESS IN VIGILANCE: THE POWER OF CHOICE

Transcription

1 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 50th ANNUAL MEETING PERFORMANCE, WORKLOAD, AND STRESS IN VIGILANCE: THE POWER OF CHOICE J.L. Szalma and P.A. Hancock University of Central Florida Orlando, Florida Observers were provided the illusion of choice by permitting them to choose an easy or a hard version of a vigilance task. Participants were then assigned at random to either the condition they expressed preference for or their non-preferred condition. Participants in the comparison control condition were not offered the opportunity to choose. Task demand was manipulated using event rate variation, divided into high and low event rates. Results indicated that permitting participants a choice regarding difficulty level and subsequently assigning them to that level enhanced signal detections but did not reduce their perceived workload and stress. In contrast, offering a choice and then assigning observers to their non-preferred condition impaired performance relative to a condition in which no choice was provided. This pattern of effects was more pronounced at the low compared to the high event rate. These results confirm the importance of motivational effects in vigilance and the impact choice has on performance in tasks requiring sustained attention. INTRODUCTION Vigilance represents the capacity to maintain attention over prolonged periods of time. It is an essential component of monitoring tasks performed by operators of modern automated systems. One of the most reliable findings in vigilance is the decline in performance with time on watch or the vigilance decrement function (See, Warm, Howe, & Dember, 1995). As well as this systematic diminution in performance capacity over time, vigilance also imposes a considerable degree of workload and stress on operators (Hancock & Warm, 1989; Szalma, Hancock, Dember, & Warm, 2006; Szalma et al., 2004; Warm, Dember, & Hancock, 1996). However, the demands of modern automation and current security concerns render vigilance tasks vital to public safety and health (Hancock & Szalma, 2003). It is therefore crucial to understand the factors that affect vigilance and how potential decrements in such performance may be ameliorated. There is considerable evidence that the characteristics of the task itself are critical determinants of performance (e.g., see Davies & Parasuraman, 1982; Warm & Dember, 1998). One such characteristic is the limited autonomy of the individual operator engaged in these often tedious tasks with little or no control over how their own performance. Indeed, Scerbo (1998) and Hancock (1998) have argued that a considerable degree of the stress of vigilance may derive from this imposition of the task on the individual by an external authority that permits little autonomy for action. Evidence to date indicates that when observers are allowed some measure of perceived or real control over the task environment, performance is enhanced. Thus, Gunn et al. (2005) reported that when observers in a simulated UAV task were permitted to take action against targets they detected, their performance improved and stress symptoms decreased. Further, even the illusion of control can facilitate performance. Dember, Galinsky, and Warm (1992) asked observers whether they wished to experience an easy or hard version of a vigilance task, and then randomly assigned them to one of two signal salience conditions. They reported that the provision of choice attenuated the performance decrement relative to observers in a control condition who were not permitted such a choice. One component purpose of the present study then was to extend the results of Dember et al. (1992) by manipulating not only whether observers have a choice in which task they experience, but also whether they were actually assigned their preferred condition. In other words, among observers provided the illusion of choice, half experienced a condition matching their preference while the other half received the version of the task they did not prefer. Unlike Dember et al. (1992), who manipulated task difficulty by signal salience, in our experiment demand was manipulated using event rate. Event rate was chosen because it is a task characteristic that exerts substantial influence on performance, and as such tends to show substantial performance differences (Warm & Jerison, 1984). It is important to note that the participants were unaware of these different event rate conditions when they made their choice. Finally, the study reported by Dember and his colleagues (1992) examined the influence of choice on quality of performance. The current study included evaluation of the effect of choice on perceived workload and stress as well as objective performance outcome. We hypothesized that an attenuation of the vigilance decrement would be observed for individuals who received a choice, and that such performance benefits would be greater for those whose assignment matched their preference. In addition, these effects were expected to occur at both low and high event rates, but would be more pronounced in the latter condition because of the associated higher level of task demand. Specifically, we hypothesized that the benefit of choice would be greater in the context of the more demanding version of the task. It was also anticipated that the performance benefits in the choice conditions would result in reduced workload and stress relative to a control condition in which participants received no opportunity for choice, and that again this benefit would be greater at the higher event rate relative to the low event rate condition.

2 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 50th ANNUAL MEETING EXPERIMENTAL METHOD Seventy-two undergraduates at the University of Central Florida served as participants in this experiment. They were paid $7.50 per hour for such participation. All observers had normal or corrected-to-normal vision and were free of known hearing impairments. The study design was a 3 (choice: matched, mismatched, and no-choice) by 2 (event rate: high, low) by 4 (period) mixed design with repeated measures on the last factor. Twelve participants were assigned at random to one of the six experimental groups, with the restriction that in the choice conditions an equal number of individuals in each event rate condition were assigned to the choice-matched and choicemismatched conditions. Thus, of the 24 participants in the choice-matched group, 12 received the low event rate and an equal number the high-event rate. Participants experienced a forty minute vigil divided into four continuous 10-minute periods. They were not informed of the length of the vigil except that it would not exceed 1.5 hours in total. In each condition there were 10 critical signals per 10-min period on watch. In the high-event-rate condition stimuli appeared at an event rate of 59.2/minute. In the low event rate condition events occurred at a rate of 12.5 per minute. The task required observers to monitor the state of a synthetic system consisting of two inputs and an output, as shown in Figure 1. The stimuli consisted of three black dots and three rectangles. Each black dot was positioned above a rectangle. The input and output values were represented as the vertical distance of each.6 cm diameter black dot from its respective rectangle and observers monitored changes in these relative distances. Neutral events, requiring no overt response from the participant, were defined as cases in which each dot was a standard distance (.7 cm) from its rectangle. Critical signals for detection were cases in which any one of the dots was.5 cm closer than the standard distance to the rectangle. There were no cases in which more than one dot was closer than the standard distance. These stimuli have been used in a previous experiment and were associated with a demonstrated performance decrement (Szalma, 2002). Following instructions regarding the nature of the task and the stimuli, participants in the choice conditions (matched and unmatched) were informed that they could experience either an easy or hard version of the task. The individuals in the no-choice group were not provided this opportunity. A 5- minute practice session was provided in which the participants experienced the event rate condition to which they were assigned for the vigil. Perceived Workload and Stress Perceived mental workload was measured by a computerized version of the NASA Task Load Index (TLX; Hart & Staveland, 1988). The standard version of the TLX was used, in which participants first provided ratings on each of the six subscales and then engaged in a paired-comparison procedure to determine the relative contributions of the subscales to the overall workload score. Perceived stress was measured using the Dundee Stress State Questionnaire (DSSQ), a well-established multidimensional measure of stress that assesses transient states associated with mood, arousal, and fatigue (Matthews et al., 1999). Specifically, the DSSQ provides three scale scores regarding the affective (Distress), motivational/cognitive (Task Engagement), and cognitive (Worry) components of stress state (Matthews et al., 2002). The NASA-TLX was administered after the vigilance session. The DSSQ was administered in two parts: a pre-vigil questionnaire completed prior to the initiation of the vigil, and a post-vigil questionnaire completed after the vigil. The order in which these were administered was counterbalanced across participants. EXPERIMENTAL RESULTS Data for all dependent measures were analyzed via a 2 (event rate) by 3 (choice) by 4 (period) mixed ANOVA with repeated measures on the last factor. For all tests involving repeated measures, Box s epsilon was used to adjust the degrees of freedom for violations of sphericity (Maxwell & Delaney, 2004). For the analyses of performance, the hit and false alarm rates were converted via arcsine transform for the inferential tests. The descriptive statistics are reported as untransformed proportions. Correct Detections The ANOVA revealed statistically significant main effects for choice, F(2,66)=57.63, p<.001, event rate, F(1,66)=13.35, p<.01, and period on watch, F(2,115)=7.00, p<.01. The three-way interaction among condition, event rate, and period on watch was also statistically significant, F(3,115)=2.935, p<.05. To further explore this interaction separate choice condition by period ANOVAs were computed within each event rate condition. At the higher event rate, statistically significant effects were observed for choice, F(2,33)=20.93, p<.001, and period on watch, F(2,66)=4.63, p<.05. The interaction between these factors failed to reach statistical significance (p=.57). These results are shown in Figure 2. Tukey HSD tests indicated that observers who experienced the condition matching their preference achieved higher detection rates (M=0.77) than those who whose group assignment was mismatched (M=0.23), or the no-choice control condition (M=0.40). Although the latter two conditions did not differ significantly from one another (p=.13), it is noteworthy that providing choice but assigning the individual to a mismatched condition impairs performance more than providing no choice at all (pairwise effect size, d=.29). With respect to the effect of period on watch, correct detections declined over time (M s=0.50, 0.52, 0.45, and 0.41 for periods 1-4, respectively). At the low event rate, a statistically significant effect was observed for the choice condition, F(2,33)=59.44, p<.001, and a marginal effect for period (after correction for sphericity), F(2,51)=3.16, p=.06. These effects are shown in Figure 3. Tukey HSD tests for the choice conditions indicated that observers in the choice-matched condition (M=.85) achieved significantly higher detection scores than their cohorts in

3 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 50th ANNUAL MEETING either of the other two groups, and that those in the no-choice condition (M=.65) achieved higher detection scores than those in the choice-mismatched group (M=.34). The interaction between choice condition and period on watch was also statistically significant, F(2,51)=2.76, p=.05. Tests for the simple effects of period within each choice condition indicated that detections declined with time on watch in the choicemismatched condition, F(1,14)=4.52, p<.05, but not in the choice-matched or the no-choice conditions (p>.50 in each case). In sum, providing participants with choice and then mismatching them with respect to their preference induces a steeper performance decrement as well as lower overall performance relative to a condition in which preference and group assignment were matched. Note that this was true regardless of whether the individual experienced a high or low event rate (i.e., whether they preferred an easy or difficult task). False Alarms The ANVOA indicated a significant effect for event rate, F(1,66)=19.73, p<.001, and period on watch, F(3, 175)=11.73, p<.01. All other sources of variance did not reach conventional levels of statistical significance (p>.32 in each case). As expected, there were more false alarms committed at the high event rate (M=.11) than at the low event rate (M=.02), and for both conditions false alarms declined with time on watch (M s=.08,.07,.06, and.06 for periods 1-4, respectively). Perceived Workload A 2 (event rate) by 3 (choice condition) ANOVA of global workload revealed a significant effect for choice, F(2,66)=6.266, p<.05. The effects for event rate and the interaction between choice and event rate were not statistically significant. Tukey HSD tests indicated that observers in the choice-mismatched condition reported significantly lower workload (M=52.025) than those in either the choice-matched (M=67.053) or no-choice (M=69.150) groups. The perceived workload of the latter two groups did not differ significantly from one another. An ANOVA of the NASA TLX weighted subscales revealed a significant effect for scales, F(3, 218)=12.74, p<.001 and for choice, F(2,66)=6.35, p<.01. All other sources of variance were not statistically significant. However, across all conditions the profile of workload was consistent with prior research (Warm, Dember, & Hancock, 1996). Thus, Mental Demand (M=219.65), Temporal Demand, (M=223.47), and Frustration (M=210.90) contributed most to the workload of the task. Stress The pre-post change scores for the three secondary factors of the DSSQ were subjected to a 2 (event rate) by 3 (choice condition) ANOVA. Neither Distress nor Worry showed statistically significant pre-post differences across experimental conditions. For Task Engagement a significant effect was observed for choice, F(2,66)=3.02, p=.05. Although pre-post Task Engagement declined for participants in each of the three choice conditions, Tukey HSD tests indicated that the decline in Task Engagement in the choice-mismatched condition (M= -.80) was smaller than that associated with the no-choice group (M= -1.46). The decline in task engagement for observers in the choice-matched condition (M= -1.27) did not differ significantly from the scores of those in the other two groups. DISCUSSION Consistent with previous experiments, a higher event rate was associated with poorer performance (Warm & Jerison, 1984), although the concomitant effect on workload was not observed here as it had been in previous procedures (Warm et al., 1996). In addition, providing a choice attenuated the vigilance decrement, but this did depend on the particular event rate and whether the group assignment either matched or did not match the person s preference. At the higher event rate individuals who had a choice and experienced that condition still exhibited a decrement in detections, but their overall detection scores exceeded those in either of the other two groups. For the low event rate condition, allowing choice and matching individuals to that choice improved overall detection performance and attenuated the vigilance decrement. In contrast, providing the illusion of choice induced performance that was worse than those who had no choice at all. A striking result was that, at the low event rate, providing a choice and then assigning the individual to a condition incongruent with that choice resulted in performance that was worse that that of individuals in the control condition who did not have the opportunity of choice at all. A similar but nonsignificant trend was observed for the high event rate, although the effect size was in the small to medium range (d=.29). Thus, the choice-mismatched condition impaired performance regardless of event rate. That is, regardless of whether the individual chose easy or hard or whether that was actually a low or high event rate, those in the choice mismatch condition performed more poorly. Hence, providing choice must be a real and not an illusory effect, and at low event rates mismatching may actually make things worse than providing no choice at all! It is particularly interesting that this effect was stronger at low event rates, since a mismatch in that condition means that the observers preferred a more difficult task. We assume that such participants wanted the challenge, and when they did not receive it they performed more poorly than those who did. These results partially confirm our hypotheses, in that the facilitative effect of choice on performance was greater when the group assignment was matched to the individual s preference. However, the expectation that these effects would be more pronounced in the high event rate condition was not confirmed. Indeed, it was at the low event rate that the larger differences were observed (see Figures 2 and 3). Recall that the lower detection rates associated with the choice-mismatched condition were accompanied by lower

4 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 50th ANNUAL MEETING perceived workload relative to the no-choice control condition. This represents a performance-workload dissociation, in which the linkage between performance and workload is in the direction opposite to that expected (Hancock, 1996; Yeh & Wickens, 1988). Hancock (1996) observed that these patterns of dissociation can be diagnostic with respect to how observers deal with task demands. The pattern observed in the current study is consistent with a disengagement from the task among observers assigned to the choice-mismatched group. That is, participants who were mismatched gave up and diverted effort away from task performance (cf., Hockey, 1997). Again, it is noteworthy that this pattern occurred across event rate conditions, indicating that participants who were mismatched gave up regardless of the actual difficulty of the task. However, the disengagement hypothesis is not consistent with the findings in regard to the DSSQ, that the decline in Task Engagement was smaller for the choice-mismatched condition relative to the other two groups. One possibility is that the lower perceived workload may have derived from greater Task Engagement on the part of these participants, but that this commitment of effort was not sufficient to overcome the performance effects of mismatching their task experience with their preferences. The reason for the failure of higher levels of effort to compensate for mismatching preferences is a matter for future research. However, it may be that mismatching preferences diverts cognitive resources away from the task by directing attention to the violation of expectancy regarding task difficulty, thereby forcing the task into a data-limited region in which further allocation of effort does not improve performance (Norman & Bobrow, 1975). In sum, the results of the current study underscore the importance of motivational effects in vigilance in general and, in particular, the assertion by Dember and his colleagues (1992) that choice is a crucial element in vigilance. ACKNOWLEDGEMENTS This work was supported by the Department of Defense Multidisciplinary University Research Initiative (MURI) program administered by the Army Research Office under Grant DAAD P.A. Hancock, Principal Investigator. The views expressed in this work are those of the authors and do not necessarily reflect official Army policy. The authors wish to thank Dr. Sherry Tove, Dr. Elmar Schmeisser, and Dr. Mike Drillings for providing administrative and technical direction for the Grant. The authors also thank H.C.N. Ganey for assistance in data collection. REFERENCES Davies, D.R., & Parasuraman, R. (1982). The psychology of vigilance. London: Academic Press. Dember, W.N., Galinsky, T.L., & Warm, J.S. (1992). The role of choice in vigilance performance. Bulletin of the Psychonomic Society, 30, Gunn, D.V., Warm, J.S., Nelson, W.T., Bolia, R.S., Schumsky, D.A., & Corcoran, K.J. (2005). Target acquisition with UAVs: Displays and advanced cueing interfaces. Human Factors, 47, Hancock, P.A. (1996). Effects of control order, augmented feedback, input device, and practice on tracking performance and perceived workload. Ergonomics, 39, Hancock, P.A. (1998). The price of freedom. Proceedings of the Human Factors and Ergonomics Society, 42, Hancock, P.A., & Szalma, J.L. (2003). Vigilance and the price of freedom. HSIAC Gateway, 13, 5, 20. Hancock, P.A., & Warm, J.S. (1989). A dynamic model of stress and sustained attention. Human Factors, 31, Hart, S.G., & Staveland, L.E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: P.A. Hancock and N. Meshkati (Eds.), Human mental workload (pp ). Amsterdam: North-Holland. Hockey, G.R.J. (1997). Compensatory control in the regulation of human performance under stress and high workload: A cognitive energetical framework. Biological Psychology, 45, Matthews, G., Campbell, S.E., Falconer, S., Joyner, L.A., Huggins, J., Gilliland, K., Grier, R., & Warm, J.S. (2002). Fundamental dimensions of subjective state in performance settings: Task engagement, distress, and worry. Emotion, 2, Matthews, G., Joyner, L., Gilliland, K., Campbell, S., Falconer, S., & Huggins, J. (1999). Validation of a comprehensive stress state questionnaire: Towards a state big three? In: I. Mervielde, I.J. Deary, F. DeFruyt, and F. Ostendorf (Eds.), Personality psychology in Europe (vol 7) (pp ). Tilburg: Tilburg University Press. Maxwell, S.E., & Delaney, H.D. (2004). Designing experiments and analyzing data: A model comparison perspective, 2 nd ed. Mahwah, NJ: Erlbaum. Norman, D., & Bobrow, D. (1975). On data-limited and resourcelimited processing. Journal of Cognitive Psychology, 7, Scerbo, M.W. (1998). Fifty years of vigilance research: Where are we now? Proceedings of the Human Factors and Ergonomics Society, 42, See, J.E., Howe, S.R., Warm, J.S., & Dember, W.N. (1995). A metaanalysis of the sensitivity decrement in vigilance. Psychological Bulletin, 117, Szalma, J.L. (2002). Workload and stress of configural displays in vigilance tasks. Proceedings of the Human Factors and Ergonomics Society, 46, Szalma, J.L., Hancock, P.A., Dember, W.N., & Warm, J.S. (2006). Training for vigilance: The effect of KR format and dispositional optimism and pessimism on performance and stress. British Journal of Psychology, 97, Szalma, J.L., Warm, J.S., Matthews, G., Dember, W.N., Weiler, E.M., Meier, A., & Eggemeier, F.T. (2004) Effects of sensory modality and task duration on performance, workload, and stress in sustained attention. Human Factors, 46, Warm, J.S., & Dember, W.N. (1998). Tests of vigilance taxonomy. In: R.R. Hoffman, M.F. Sherrick, and J.S. Warm (Eds.). Viewing psychology as a whole: The integrative science of William N. Dember. (pp ). Washington, DC: American Psychological Association. Warm, J.S., Dember, W.N., & Hancock, P.A. (1996). Vigilance and workload in automated systems. In: R. Parasuraman & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp ). Hillsdale, NJ: Erlbaum. Warm, J.S., & Jerison, H.J. (1984). The psychophysics of vigilance. In: J.S. Warm (Ed.), Sustained attention in human performance. (pp ). Chichester, United Kingdom: Wiley.

5 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 50th ANNUAL MEETING Neutral Event Critical Signal Figure 1. Example stimuli employed for the present study. Correct Detections p(h) Choice Matched Choice Mismatched No choice Period (10-min) Figure 2. Proportion of correct detections as a function of period on watch at high event rate. Note. Error bars are standard errors. Correct Detections p(h) Choice Matched Choice Mismatched No choice Period (10-min) Figure 3. Proportion of correct detections as a function of period on watch at low event rate. Note. Error bars are standard errors.