The Influence of Visual Workload History on Visual Performance

Transcription

1 HUMAN FACTORS, 1986,28(6), The Influence of Visual Workload History on Visual Performance MICHAEL L. MATTHEWS! University of Guelph, Ontario, Canada Two experiments are reported that demonstrate that visual search for a signal from a number of potential signal sources in a sustained monitoring task is dependent upon previous visual-load history. It is shown that both temporal and spatial variations in load produce performance decrements, and occasionally increments, that cannot be predicted from static-load experiments. These data are not consistent with previous attempts to explain performance changes associated with workload history. An interpretation is offered in temzs of the persistence of information-processing strategies across changing task conditions. INTRODUCTION Although mental workload has been the subject of increased interest and research activity in recent years, one variable that may be of some significance, namely, workload history, has been consistently overlooked. Workload (or load) history refers to the level of workload under which an operator was performing prior to the slice in time when performance was being assessed. Traditionally, temporal effects that are most often reported in the vigilance literature are described in terms of workload levels that are stable over time, and interest has focused largely on the performance changes associated with time on task. Further, when workload has been varied it tends to be treated as a between-subjects variable, thereby producing cleaner experimental designs that permit load to be assessed inde- I Requests for reprints should be sent to Michael 1. Matthews, Department of Psychology, University of Guelph, Guelph. Ontario. Canada NIG 2WI. pendently of any confounding effects that may be due to previous load demands. As more emphasis is being placed on understanding the interaction between human and system performance in the context of functions that are shared, it should be recognized that operator workload may vary dramatically over time as systems attain a more dominant role in controlling the information rate to which the operator must respond. Consequently, the key question becomes: By how much will we be in error if we attempt to estimate load effects on performance without taking into account load history? We can identify two extreme cases of workload variations that can occur: (1) a sudden increase in the number of signals that must be processed following a period in which signals were infrequent, and (2) a rapid reduction in signal-processing demands following a period of sustained high signal load. Informal evidence points to the performance consequences of such patterns. For instance, the first case is a broad description of the The Human Factors Society, Inc. All rights reserved.

2 624-December 1986 HUMAN FACTORS events that have become known simply as Three Mile Island, about which little more needs to be said in the human factors literature at this time. An example of the second type-the consequences of workload reduction-comes from Allnutt (1982), who quotes anecdotal evidence of the driver of a railway train driving through a red signal in a period of low load following a sustained period of high load that had required intense concentration. The basic literature on workload shows that both load stress (increasing the number of signal sources) and speed stress (increasing the rate of signals) are known to produce decrements in visual and auditory information processing in tasks requiring monitoring and/or choice reaction time (Conrad, 1955, Goldstein and Dorfman, 1978; Mackworth and Mackworth, 1958; Raouf, Hatami, and Chaudhary. 1981). Such decrements are manifested as increases in error rates and/or response latencies. In one of the few studies to examine the effects of variable load conditions on monitoring performance, Cumming and Croft (1973) have examined choice reaction time as a function of varying the presentation rate of auditory signals. Signal rate was varied in a cyclical manner, and it was found that during the decreasing demand portion of the cycle, performance was significantly worse than in the corresponding period during the ascending phase. This performance decrement persisted beyond the high signal rates of its inception, continued through a period of reduced signal rate, and was subsequently ameliorated only after a disproportionate reduction in task demand. This result has been called the hysteresis effect. Cumming and Croft (1973) have offered the explanation that the effect occurs as a result of the subject failing to adequately match his or her expectancies with the changing task demands. However, this explanation was experimen- tally evaluated and rejected by Goldberg and Stewart (1980). They presented subjects with a series of visually presented characters in a choice reaction time paradigm, and in one of the conditions a spatial cue was available to indicate changes in task demand. They found that the hysteresis effect was still present (although somewhat reduced) under the cued condition, and argued that this was inconsistent with the hypothesis of Cumming and Croft. Instead, they proposed an account that explained the performance decrement associated with load reductions following load peaks in terms of a temporary, short-term memory overload. They argued that such an overload persists until task demands are sufficiently reduced to clear a short-term memory buffer. It should be noted that in both of these studies there were significant demands placed upon short-term memory because of the requirement to retain serial order information (i.e., the mapping of sequential choice responses to serially presented stimuli). The primary aims of the two experiments reported in the present paper are: (1) to determine generally whether load history (as opposed to rate history) influences performance in a visual monitoring situation, and (2) to examine whether the hysteresis effect can be demonstrated in a task with varying visual load that does not require the preservation of serial order information. These objectives will provide a critical test of the short-term memory overload theory by employing conditions under which memory capacity is not considered to be a significant factor. If a hysteresis effect can be demonstrated in these conditions, alternate explanations to those offered previously in the literature will be required. In the first experiment, subjects were required to monitor a display for the occurrence of target signals while the number of target locations was systematically varied

3 VISUAL WORKLOAD HISTORY December over time. In the second experiment, the number of target locations remained constant, but target rate was varied. In each case, in order to properly evaluate the effects of load history, performance of subjects with a varying load history was compared wi th that of controls, who maintained a static load level. METHOD A task was designed that would capture two of the basic elements of operator behavior that can normally be found in realworld tasks involving visual monitoring over time, namely, search and decision making. In the present case, the search component required an operator to visually scan all potentially relevant sources for information; that is, a potential signal would have to be detected from background noise or clutter. The subsequent decision stage required that information within the signal be interpreted and an appropriate motor response be initiated. In devising a suitable paradigm, we had to ensure that the task could be readily learned in the laboratory, as situationallimitations precluded opportunities for extensive training or testing periods. Experiment J Signals for the task were numeric strings of simple ari thmetic expressions that could easily be evaluated (e.g., < 27). Half of the signals represented expressions that were true and half false. Signals were presented in a context of noise strings that had a similar format but were not arithmetic expressions that could be immediately evaluated (e.g., 26 + #1 > 1$). Signal and noise strings were displayed on a monochrome video monitor driven by a microprocessor and were presented in a three column by four row matrix. There were four levels of visual load that utilized either one, two, three, or four rows of the display, thus yielding 3, 6, 9, or 12 target/noise strings, respectively. The probability of a signal string on any trial was set at 0.5. The subject's first task was to locate and indicate the position of a target string (if it was present) by pressing a key in a panel that was spatially compatible with the display configuration. The subject was then required to evaluate the target expression and press another key to indicate whether the expression was true or false. Ten seconds were available for the performance of the task before presentation of the next trial. If the subject completed both the search and the decision components within the 10- second limit, the screen was made blank for the balance of the time available for that trial. Instructions were given that stressed the need for both speed and accuracy. Subjects, who were all volunteers from an undergraduate subject pool, could also earn a performance bonus based upon fast and accurate responding. Dependent measures were speed and accuracy of both search and decision making. There were eight experimental conditions in all, with 10 subjects allocated randomly to each group. Each group received 240 trials, divided into 16 blocks of 15 trials each. Within every consecutive block of 30 trials, half contained a signal message. Four groups acted as controls and performed the entire monitoring session at a single load level (i.e., 3,6,9, or 12 signal-source locations per trial). The remaining four groups were exposed to systematic variation of the visual load during a session, as follows: Group VI-simple cyclic variation of load across successive trial blocks; Group V2-block randomization of load level; Group V3-a constant load level of 12 signal sources that suddenly reduced to three on four trial blocks during the session; and Group V4, in which the pattern for Group V3 was reversed. Samples of the types of load variation for each of these groups are

4 626-December 1986 HUMAN FACTORS Table 1 Details of the Serial Order of the Load Levels in the Variable Load Conditions* Trial Block Load V condition V V V For simplicity, only trial blocks for the first half of a session are indicated since the pattern was duplicated for the second half. given in Table 1. Within the variable load groups, load level always remained constant within a trial block. These load conditions were selected to provide a broad cross-sectional sample of variations that might typically occur in a variety of monitoring tasks. Subjects performed one practice session under the particular load condition to which they were assigned before actual testing began. Practice and testing sessions were each about 40 minutes long. RESULTS Before discussing the results in detail, we will outline the general strategy for analyzing the data. In order to compare performance of the variable-load groups with the constant-load controls at a macro level, each of the load levels was contrasted with ordinally equivalent trial blocks. For example, in order to compare the performance at the load level of three potential signal sources for the variable-load groups with the performance of controls, the mean trial blocks on which the three-load level occurred in these groups were contrasted with the mean of the trial blocks in the three-load control group at the same ordinal position. Data from the first four blocks of trials were not included in this analysis because at that point in the experiment subjects had been exposed to too few trials to establish a workload history. This procedure yielded three points of comparison for each of the load levels in each of the treatment conditions. For each of these points of comparison, each subject's mean score was used. As the number of data points contributing to the mean could be potentially quite small, initially separate analyses were conducted using both median and mean data. A similar pattern of results emerged no matter which of these measures was used: therefore, only the mean data are reported in the discussion that follows. Separate analyses of variance were then conducted for each of the load levels and treatment conditions in order to generate error terms for comparing means. Planned comparisons were conducted between the relevant means using this error term with D.F. = 18 and p < At a micro level, the effects of an immediately preceding load level in the variableload conditions could be determined in a similar manner by comparing equivalent blocks of trials from similar ordinal test positions of the appropriate constant-load control group. For example, in the VI variableload condition (cyclical variation), the fourth block of trials was performed at the load level of 12. The data for this trial block could therefore be contrasted with the 12-load control group at its fourth block of trials. In all cases, latency data are based on trials in which the subject responded correctly. The

5 VISUAL WORKLOAD HISTORY December results will be discussed separately for search performance and decision performance. Search Performance In spite of the practice session, there was considerable evidence of faster performance as the experiment progressed. Each control condition showed a reduction in search latency of the order of 15% from the first to the last trial block. Table 2 shows summarized data for all the experimental groups for each load level collapsed over trial blocks. Each of the means for the variable-load conditions has been contrasted with its appropriate control. using planned comparisons based on the mean square error indicated previously with D.F. = 18. To further clarify this table, it should be noted that each individual mean in the control conditions is based on data summarized from trial blocks in a serial position equivalent to the relevant variable-load level against which it is being compared. Thus, the first control mean under Load Level 3 represents the mean for those trial blocks that coincided in time with the load level of 3 that occurred in Condition VI (blocks 8, 9, and 16; see Table 1). The second control mean listed under Load Level 3 represents the relevant comparison for V2, and the third mean represents the control comparison for V3. Because load levels of 6 and 9 were not used in conditions V3 and V4, there were only two control means of relevance (i.e., for VI and V2) for these two load levels. In general, it can be seen that at the macro level the effects of load history are neither consistent in direction nor predicated by the specific nature of the workload variation. The variable-load conditions tend to produce decrements in performance (i.e., increases in search latency) at the low-load level, with the largest effect occurring when the load level suddenly drops to 3 potential signal sources after a sustained period of monitoring] 2 (V3). At the highest load level, there appears to be a trend showing a performance advantage for the variable-load condition. A more detailed analysis was conducted on the data from the two extreme load conditions in order to determine the specific nature of the local effects of previous load history. Specifically, for the group that received the block randomized sequence (V2), each of the blocks of the low-load condition (three Table 2 Mean Search Latency as a Function of Load and Treatment Condition* Load condition Control V1 V2 V3 Number * of * signal * sources V Means for the veriable load conditions with an asterisk ere significantly different (p < 0.05) from their appropriate controls. which appear to their left on the same line of the table.

6 628-December 1986 HUMAN FACTORS signal sources) following a higher load was contrasted with the equivalent block (in the same ordinal position) from the control group that maintained a constant, threesignal load level. These data showed that, on average, immediately following a block with a load of 12, 9, or 6 signals, subjects were slower by 0.432, 0.409, and s, respectively. A similar analysis for the high-load condition (12 signal sources) following blocks with a load of 9,6, or 3 revealed that subjects were faster by 0.157, 0.248, and s, respectively. In both cases a critical difference of s would be necessary for statistical significance, with D.F. = 18 and p < By examining the data within those trial blocks that showed some influence of the load level of the previous trial block, an estimate could be made of the length of the "disruption" in performance related to load history. This analysis was performed only for those high- and low-load-level trial blocks that showed, overall, a statistically significant deviation in performance from their appropriate control. In general, it was found that it took from five to six trials before search latencies returned to a point where they were statistically indistinguishable from equivalent controls. Thus, it would appear that the disruptive effects of the previous trial block were lasting about one minute. Error rates for detecting the target strings were consistently low «1%), and analysis of variance failed to reveal any differences in rate across treatment conditions or trials. Decision Performance As indicated previously, subjects were also required to make a true/false decision based on the target's contents. No significant effects of variable-load history were found for decision latency, and only one significant difference was found across the four control groups. (Load Level 12, mean = s, was significantly different from Load Level 3, mean = 2.42 s; D.F. = 36, p < 0.05.) However, there were, in general, clearer effects of load history on decision accuracy, and these are shown in Table 3, which shows that accuracy under Condition V2 (randomized load history) was significantly lower than the appropriate control for each of the four load levels. It would appear that there is some evidence of a speed/accuracy trade-off for the Table 3 Mean Decision Accuracy as a Function of Load and Treatment Condition* Load Condition Control Vl V2 V3 Number of signal sources V Means for the variable load conditions with an asterisk are significantly different (p < 0.05) from their appropriate controls.

7 VISUAL WORKLOAD HISTORY December high-load condition, which showed both the fastest decision latencies and the lowest accuracy. Subjects may have felt that they were approaching the total time available for the trial in this condition, although, on average, search time and decision time together were about 6.5 s, which was comfortably within the lo-s trial limit. These data clearly show that there are effects of visual load level under varying load history that would not have been predicted from experiments on static load levels. The data are consistent with the findings of Cumming and Croft (1973), who showed that performance seems to be more disrupted following load reductions than load increases. Further, this experiment can be seen to extend the generality of the hysteresis effect from the temporal domain to the spatial. The proposed explanation of the hysteresis effect by Cumming and Croft (1973), which was subsequently favored by Goldberg and Stewart (1980) in terms of short-term memory overload, is not consistent with the present findings. First, given the 100- to 250 ms limit of visual short-term memory, it is unlikely that the present experimental conditions produced a carryover from an overflowed visual buffer from one trial to the next. It should be remembered that as soon as the subject made a decision, the screen was made blank; this left, on average, 6 s at the 3-load level and 3.5 s at the 12-10ad level before the information for the next trial appeared. Second, the micro analysis revealed a persistence of the disruptive effects of a sudden load reduction for a considerable period beyond the limit of visual short-term storage. One possible explanation for the present findings (and also for similar results reported previously) that has not been considered lies in the area of subjects' strategies. If it is assumed that different strategies may be employed to scan the displays at different load levels, then it is possible that subjects fail to switch to the optimum strategy when load levels change. The data imply that subjects readily mobilize more efficient strategies as load levels increase but appear to retain such strategies long after the load level has subsequently fallen. One problem with this explanation that is not addressed by the present data is why a strategy that is effective at high load levels ceases to be as effective at lower load levels. As a test of this "strategic persistence" hypothesis, and in order to further evaluate the explanations offered by Cumming and Croft (1973) and Goldberg and Stewart (1980), a second experiment was conducted using a temporal variation in load while maintaining a constant number of signal sources. Experiment 2 Essentially, the design of the first experiment was repeated in Experiment 2 but was simplified so that subjects were required to perform only the search component. All 12 signal sources were presented on every trial, and rate of presentation was varied as a between-subjects variable as follows: one trial every 3 s, one every 6 s, and one every 9 s. It should be noted that because the mean time to find a signal at the 12-10ad level in the first experiment was 4.4 s, subjects would clearly be under some speed stress at the 3-s rate. Similarly, they would be quite underloaded at the 9-s rate. Because the rate of signal presentation varied across experimental conditions, the number of trials was manipulated in order to equate total time on the task for each of the rate conditions. In the 3-, 6-, and 9-s conditions there were 168, 84, and 56 trial blocks, respectively. Each trial block consisted of six presentations with a signal probability of 0.5 on any trial. The experimental design was similar to that of the first experiment. There was a con-

8 630-December 1986 HUMAN FACTORS trol group for each of the levels of presentation rate that received trials at a constant rate throughout the test session. There were two variable-rate conditions: cyclical, in which the signal rate varied in a predictable cyclical sequence from block to block (i.e., 3, 6, 9, 6, 3, 6, etc.), and random, in which the rate varied randomly from block to block with the constraint that all three rates were represented equally often throughout the test session. Subjects were again given one practice session, instructions that stressed both speed and accuracy, and a performance incentive. Ten subjects were randomly assigned to each of the five groups; none had participated in the first experiment. SEARCH LATENCY (SECONDS) 3 1 2~ '( " --+- CONSTANT ~ CYCLICAL 3 SEC;~:~::;A~ SECONDSI TRIAL SECONDSI TRIAL 4 RESULTS In order to provide an appropriate basis for comparing the variable-temporal-load and static-load groups, the data were collapsed into equivalent time blocks. The mean latency for correct target identifications for each group as a function of time block is shown in Figure 1. The data from each load level have been extracted from the two variable-load conditions for ease of comparison with the control-group data. Analysis of variance revealed that there was a significant interaction between load level and treatment condition, F(4,54) = 9.32, P < Planned comparisons were conducted on means collapsed over time blocks using the mean square error from the above Load Level x Treatment Condition interaction, (D.F. = 54, P < 0.05). These showed that there were no significant differences between experimental groups at the fastest rate of presentation. However, at both of the other rates of presentation the variable-load-history groups performed significantly faster than did their respective controls. There were no differences, however, be- Table 4 3 tween the random and cyclic variable-load conditions. Error rate data are indicated in Table 4. Analysis of variance of these data showed only one significant effect, which was the main effect for presentation rate, F(2,27) = Mean Error Rate Probabilities as a Function of Presentation Rate (Seconds per Trial) and Load History. Rate ~... '" m ~ M Constant TIME Load Condition Cyclic BLOCKS Figure 1. Mean search latency (seconds) as a (urlction o( rate o( presentation (seconds per trial) and workload history. Random

9 VISUAL WORKLOAD HISTORY December , p < This is clearly attributable to the large number of errors obtained with the fastest presentation condition. DISCUSSION Overall, these results show a somewhat different pattern from the data of the first experiment, in that subjects in the variableload conditions show a performance improvement compared with their static-load controls when the overall information rate can be handled comfortably. When the rate of information is so high that chance levels of performance are obtained, rate history appears to have little effect. Clearly, the data do not support the shortterm overload explanation for the performance decrement that is associated with return to lower information rates. Subjects in the variable-load condition failed to show the sort of decrements in accuracy that such an explanation requires, and performed consistently faster than did subjects with a constant rate history. At the same time, the failure to find a difference between the cyclical and random conditions also argues against Cumming and Croft's explanation that the performance decrement following high load levels is due to a failure to anticipate or recognize load changes. This explanation would predict higher performance levels for the cyclical condition (in which subjects could readily perceive the pattern of rate changes) compared with the random rate condition. The data do support, however, the notion of strategic persistence, since it could be argued that when subjects in the two variableload groups move from a set of trials at high rate to a block at a lower rate, their higher rate of responding tends to persist. Thus, they perform the task with greater urgency even when working at much lower load levels. This explanation is consistent with Poulton's (1982) claims that subjects exhibit strategic persistence from one experimental condition to another in within-subject designs. Poulton has argued that in the normal case, this has led to research results that are an artifact of the within-subject design. In the present case, it should be noted that variable-load situations are more representative of real-world monitoring tasks than are the static-load experiments that have previously been used to infer information-processing capabilities and limitations under load and speed stress. However, we do not argue that the temporal parameters selected and the patterns of load variation employed in the present experiments are representative of specific realworld situations. The data from the first experiment are somewhat neutral with respect to the strategic persistence hypothesis but clearly do not support Goldberg and Stewart's (1980) short-term memory overload explanation of the hysteresis effect, since that effect was found under conditions in which no overload could occur. In summary, the present experiments demonstrate that performance under conditions in which visual load levels may change over time-either as a result of a change in the number of signal sources to be monitored or as a result of a change in the rate of signal presentation-cannot be predicted from static-load laboratory tasks. Sometimes the persistence of old strategies will benefit information processing, and in other cases such persistence will be detrimental. The experiments show clearly that performance is more likely to be affected by a sudden reduction in the amount of information to be processed than by a sudden increase. Data from traditional experiments on single-load levels can therefore continue to be used to predict performance under variable-load conditions when the information load is high. Such data will be in error, however, when predicting performance at lower levels of information

10 632-December 1986 HUMAN FACTORS load under conditions of variable-load history. Definitive explanations of the hysteresis effect are not yet possible given the paucity of data from variable-load experiments. It seems probable that the effect should not be classified as a single unitary phenomenon since it is dependent upon specific task characteristics. In conditions in which the preservation of the serial order of information is important (the experiments of Cumming and Croft, 1973, and Goldberg and Stewart, 1980), temporary short-term memory overload can lead to performance decrements, which persist until the buffer is cleared. In conditions in which subjects have produced speeded responses in the past, there may be a beneficial carry-forward of a strategy to a condition that otherwise would not have elicited such efficient processing (Experiment 2 above). Finally, as the inconsistent direction of the results of Experiment 1 demonstrate, hysteresis may be manifested either as a performance decrement or an increment. These effects are found in the absence of memory overload and are also apparently unrelated to subjects persevering with inappropriate strategies. The effects may also be quite durable, lasting as much as one minute after the termination of the circumstances that gave rise to them. It would appear that a more useful course for future work in this area would be to document the limits under which workload history affects performance and to determine the temporal parameters of its consequences for the performance measure under study. ACKNOWLEDGMENTS This research was supported by a grant to the author from the Natural Sciences and Engineering Research Council of Canada. REFERENCES Allnutt. M. (1982). Human factors: Basic principles. In R. Hurst and L. Hurst (Eds.), Pilot Error (pp. 1-22). London: Granada. Conrad, R. (1955). Some effects of performance in perceptual load. Journal of Experime>ltal Psychology, 49, Cumming, R. W., and Croft, P. G. (1973). Human information processing under varying task demand. Ergo- >Iomics, 16, Goldberg, D. R., and Stewart, M. R. (1980). Memory overload or expectancy effect? 'Hysteresis' revisited. Ergonomics, 23, Goldstein, I. L., and Dorfman, O. W. (1978). Speed and load stress as determinants of performance in a time sharing task. Human Factors, 20, Mackworth, N. H., and Mackworth, J. H. (1958). Visual search for successive decisions. British Journal of Psychology, Poulton, E. C. (1982). Influential companions: Effects of one strategy on another in the within-subjects designs of cognitive psychology. Psychological Bulletin, 91, Raouf. A., Hatami, 5., and Chaudhary, K. (1981). CRT display terminal task pacing and its effect on performance. In G. Salvendy and M. J. Smith (Eds.), Machine pacing and occupational stress (pp ). London: Taylor and Francis.