Investigating Display-Related Cognitive Fatigue in Oil and Gas Operations Abstract Ranjana Mehta 1,2, Mark Riddell 4, Thomas Ferris 1,3, and S. Camille Peres 1,2 1: Texas A&M Ergonomics Center; 2: Dept of Environmental and Occupational Health; 3: Dept of Industrial and Systems Engineering Texas A&M University, College Station, Texas, 77843, USA 4: The Marrell Group Communication and Human Performance Network Houston, Texas, 77386, USA rmehta@tamu.edu, mriddell@tchpnet.com, tferris@tamu.edu, peres@tamu.edu In the oil and gas industry, rapid technology adoption from the well site to the refinery has led to the introduction of increasingly complex digital displays and operator interfaces. The introduction of these technologies poses new challenges to managing fatigue-related risks for human operators, who already face challenges associated with working extended and irregular shifts to monitor oil and gas operations. Strategies for fatigue risk management, such as shift scheduling and fatigue assessment surveys, primarily target sleep-related fatigue issues; however recent research has indicated that operators also experience significant performance deficits from cognitive fatigue, as well as other task-related mental factors which are conceptually distinct from sleepiness. All of these factors contribute to performance failures and the cost of preventable incidents. As part of an industry-academic collaboration, this paper highlights ongoing research efforts to isolate and document the extent of operator performance- and safety-related implications of cognitive fatigue, as well as to develop effective countermeasures in mitigating the negative effects of this particular type of fatigue. Examples of how aspects of common display and interface technologies in current upstream operations directly contribute to cognitive fatigue are discussed. Concrete examples of well-known problems in the oil and gas industry, such as alarm fatigue, are described within a developed theoretical paradigm that incorporates physical and cognitive fatigue, sleep-related fatigue, and other mental and physical factors such as motivation and the prevalence of environmental stressors. The paper also describes a planned in situ task analysis, and explores assessment methods for human factors display/interface design guidelines that will ultimately be applied to address the risk factors identified in those efforts. 764
Keywords: Fatigue Risk Management, cognitive fatigue, display design, operator performance, human factors Introduction In the oil and gas industry, rapid technology adoption from the well site to the refinery has led to the introduction of increasingly complex digital displays and operator interfaces in monitoring centers and control rooms, which allows the centralization of effort and expertise. The introduction of these technologies poses new challenges to managing fatigue-related risks for human operators who monitor oil and gas operations. These challenges compound existing fatigue issues associated with working extended and/or irregular shifts, which are somewhat common practices. In response to significant incidents involving operator performance failures, the U.S. Department of Transportation (DOT) and the American Petroleum Institute (API) have introduced comprehensive fatigue risk management system (FRMS) standards and requirements as key elements of safety management programs for pipeline monitoring centers and refinery control rooms. Strategies for fatigue risk management, such as shift scheduling and fatigue assessment surveys, primarily target sleep-related fatigue issues (Murray et al., 2013); however recent research has indicated that operators also experience significant performance deficits from cognitive fatigue, as well as other task-related mental factors which are conceptually distinct from sleepiness (Balkan and Wesensten, 2011). Cognitive fatigue can develop with extended time on task and/or heavy cognitive load associated with the task (Gunzelmann et al., 2011). All of these factors contribute to performance failures and costs associated with preventable incidents. Figure 1 summarizes some of the main factors influencing operator performance, and illustrates how fatigue should be handled as a separate but interacting factor with sleepiness. Comprehensive Fatigue Management Fig 1. Conceptual Model of Performance. Adapted from Balkin and Wesensten (2011). With the continued adoption of advanced technology, it is expected that the scope of FRMS will expand to include additional display monitoring settings (e.g., drilling and production monitoring) and new regulatory inspection criteria for a broader set of human factors 765
management procedures (DOT/PHMSA 49 CFR Parts 192 and 195). In order to inform this expanded set, there is a need to more precisely define factors contributing to the different types of fatigue, and how they interact, to ultimately understand the extent to which these factors affect operator performance. To understand how sleepiness and cognitive fatigue differentially affect operator performance, it is important to first distinguish the neurophysiological consequences of each (Lorist and Faber, 2011). Sleep-related performance deficits are a result of low stimulation when the task fails to counterbalance the onset of physiological sleepiness (Carskadon et al., 1986). In contrast, cognitive fatigue is associated with decrements in performance that arises from work-related depletion of cognitive resources over time. This could be attributed to a reduction in the efficiency with which mental resources are allocated, or a general reduction in available resources to perform tasks (Balkin and Wesensten, 2011). Many times subjective reports are used to provide an understanding of the individual and synergistic impacts of sleepiness and fatigue on performance deficits. While assessing fatigue in this way has the benefits of low cost and are easy to conduct, they can also be problematic. Subjective assessment must be conducted either by interrupting an ongoing task (and thus temporarily disengaging the operator from sources of fatigue), or retrospectively at the conclusion of a task activity, when operators may not have the same levels of fatigue as when the activity was being conducted. As an alternative approach, neurophysiological indicators, such as heart rate variability, functional brain activity, and galvanic skin response, can elucidate the nature and extent to which fatigue (either cognitive or sleepiness) impacts operator performance as they perform the tasks of interest. For example, Fig 2 illustrates operator performance, subjective report of mental workload, and functional brain activation during a 30-minute working memory task (from Mehta et al., 2014). Performance on the task and perception of mental workload for the task remained comparable across three stages of the task time (N1, N2, N3). However, a detailed analysis of the prefrontal cortex, i.e., the brain region responsible for working memory, revealed that greater mental resources were required as time on task increased, implying a greater neural cost to maintain task performance. Thus, it is recommended that any investigations of cognitive fatigue, that is separate from sleep-related decrements in performance, include more in-depth measures of workload to comprehensively understand the influence of both time-on-task and cognitive load on operator performance. Oxygenated Hemoglobin 0-2E-09-4E-09-6E-09-8E-09-1E-08-1.2E-08-1.4E-08 N1 N2 N3 Le hemisphere Right hemisphere 766
Fig 2. Performance on the n-back test (top left), perceived mental demand (top right), and functional brain activation (bottom) during a 30-minute working memory task. Display-Related Cognitive Fatigue In vigilance monitoring environments, cognitive fatigue may be influenced by display design elements and multi-display configurations; the intensity and duration of cognitive workload; operator experience and knowledge; and cognitive fatigue may additionally interact with other factors, including sleepiness and motivation, to affect performance (Balkin and Wesensten, 2011). Management of sleepiness, cognitive fatigue, and motivation are required to effectively mitigate operator performance risks. While relatively less is known about how to mitigate the specific performance effects of cognitive fatigue, mitigation approaches may include a combination of periodic rest, task switching, task sharing, training, and display design changes may reduce the negative performance effects. A comprehensive research effort that takes into account and controls for several of these factors may serve to more clearly document the effects of cognitive fatigue and identify the best approaches for mitigation. Existing research may offer some insight into display factors that most effectively mitigate cognitive fatigue, provided operator task demands and other relevant factors of the operational environment are known. For example, display elements that rely heavily on spatial cognitive processing, such as judging and comparing the levels of multiple gauges (representing, for example, fluid flow rates) may be transformed to relay the same information via nonspatial/symbolic processing channels. The transformation could represent flow rates numerically, and/or by using other symbolic encoding such as color coding (e.g., having display elements turn red to denote problematic flow rates for a given context). If the load on an operator s spatial processing resources is heavy, offloading to a relatively available nonspatial processing channel in this way may help reduce cognitive fatigue and improve performance (Wickens, 2002). It is a substantial effort to exhaustively define the cognitive functions that may be required of a monitoring operator while interpreting displays toward task goals. However there are heuristic methods that may be applied to determine the effectiveness of different displays in an operational context. For example, the cognitive efficiency of display media can be determined by measuring the informativeness of a display and employing various existing methods for assessing workload during the processing effort required to gain that information (Yang, Shukla, & Ferris, 2012; Yang & Ferris, in progress). Cognitive efficiency can be calculated for several display prototypes in a representative context, and displays that support processing more information per unit of required cognitive resource may prove to be the best options for minimizing cognitive fatigue. Early efforts in defining cognitive efficiency of display media have used simple calculations of information transmission according to Shannon s Communication Theory (quantified information that is present in a display and accurately interpreted by the operator) (Shannon, 1948) and workload assessments by using both neurophysiological correlates (e.g., heart rate variability, galvanic skin response, etc.) and subjective workload surveys (NASA-Task Load Index; Hart & Staveland, 1998). Initial findings with basic display signal dimensions generally concur with the literature, showing significantly more cognitive efficiency for auditory displays compared to visual displays when information content is very low (2 or 3 possible levels representing a max of 1 to 1.58 bits of information) 767
(left side of Fig 3) (Yang et al., 2012). Also, the choice of encoding dimension for the displays such as whether to encode according to the intensity of the signal or spectral dimensions such as visual hue or auditory pitch does not impact the cognitive efficiency of low-information content displays, but as the amount of displayed information increases, spectral encoding appears to be more efficient than intensity (right side of Fig 3) (Yang et al., 2012). Ongoing research will determine whether the cognitive efficiency heuristic method can scale up to the more complex displays (with considerably higher information content levels) to predict performance and provide insight into potential sources of display-related cognitive fatigue in vigilance monitoring environments. Fig 3. Select findings of initial studies of cognitive efficiency according to basic display signal dimensions (adapted from Yang et al., 2012). Higher values of cognitive efficiency represent more bits of information transmitted per unit of required cognitive resource. Error bars represent standard error. Left side: cognitive efficiency for auditory (A) and visual (V) displays, when display states include 2, 3, or 4 levels (maximum information transmission of 1, 1.58, or 2 bits). Right side: cognitive efficiency for intensity-encoded (I) and spectrum-encoded (S) displays for each number of display levels. Proposed Approach to Address Critical Gaps/Needs Our position is that cognitive fatigue may be a contributing factor to safety risk and performance in vigilance environments; and that display design may contribute to cognitive fatigue and performance deficits. As such, investigations that focus on mitigating Display-related Cognitive Fatigue (DCF) in order to engineer safety and performance improvements in Vigilance Monitoring Environments (VME) are needed to test these hypotheses. For example, empirical studies that determine contributing factors to cognitive fatigue-related performance deficits and inform the design of new assessment and mitigation tools and processes will provide the basis to improve current fatigue assessment methods. Table 1 lists detailed research questions, that when completed, may address and fill the critical gaps identified earlier, particularly in oil and gas operations. We propose a method for investigating these research questions in three phases, each with specific main objective: 768
1. Systematic observation - identify the possible contributors to cognitive fatigue in typical oil, gas, or petrochemical monitoring environments 2. Simulation - conduct empirically studies to confirm the contributors to cognitive fatigue 3. Design - develop a test a new monitoring environment that may mitigate the elements of cognitive fatigue 1. Determine whether, and to what extent, operators in digita monitoring centers and control rooms experience cognitive fatigue related to their work or task environment 2. Determine how various elements of the task environment contribute to overall levels of workload and cognitive fatigue 3. Identify any interactions between cognitive load and an operator s knowledge structure 4. Identify opportunities to improve digital interface designs and layouts that are associated with higher levels of cognitive fatigue 5. Propose new practical tools for assessing and mitigating cognitive fatigue 6. Create and test new interface designs that mitigate sources of cognitive fatigue Table 1. Potential research questions that address critical needs The systematic observation phase of the study should include a cognitive task analysis (CTA; e.g., Kirwan & Ainsworth, 1992) and should be done with real workers while they are doing their real jobs. An ethnographic observational method (including CTAs) in the workplace setting will provide a detailed description of the task domain. To assess levels of cognitive fatigue, the measures below should be used. Each measure contributes a different piece of information about the human-machine interface and interaction in monitoring and vigilance environments. 1. Cognitive assessments 2. Performance measures 3. Neurophysiological measures 4. Subjective self-reports This type of investigation will allow for the identification of cognitive fatigue risk factors associated with cognitive displays particularly for vigilance monitoring environments. The simulation phase should be conducted in a simulated monitoring environment to empirically confirm those contributions once clear contributors to cognitive fatigue in a monitoring environment are identified, a study. This study will involve not only observational, neurophysiological, and subjective data collection similar to that in the observation phase but also will involve representative task sets. This will allow the researchers to manipulate the work situations to: empirically confirm performance risk factors associated with cognitive fatigue; 769
investigate how cognitive fatigue impacts workers situation awareness; construct new risk assessment and management tools; and identify opportunities for improved interface designs. At the end of this phase of study, it will be possible to generate design recommendations for display designs that mitigate cognitive fatigue. Further, these results will identify effective operational tools for assessing and mitigating cognitive fatigue, including improved heuristics for assessing the risk of cognitive fatigue in individual situations. The design phase will incorporate methods for mitigating fatigue in vigilance monitoring environments into the development of display design prototypes based on the recommendations from the simulation study. The designs would leverage existing industry guidelines and standards for operator display design, combined with findings from this observational and simulation studies. These prototypes should then be tested on individuals that would use those displays to determine their effectiveness. Industry Benefits The oil and gas industry is already on a path to developing improved methods of managing safety and operator performance in increasingly complex digital environments, which are becoming ubiquitous from upstream to downstream. Fatigue is known to be an important factor in vigilance monitoring settings, although more comprehensive fatigue risk management methods are needed. In contrast to sleepiness, there is less known about the specific drivers for and consequences of cognitive fatigue, and motivational factors as they relate to performance risks in oil and gas monitoring settings, although recent research indicates that cognitive fatigue may have a significant role in operator performance. The DCF-VME study offers the opportunity to broaden and improve current industry practices and standards by gathering empirical data that can be used to inform the design of fatigue assessment techniques, as well as operator interfaces, task strategies, and training procedures to mitigate the effects of these factors. The findings from this study will also inform ongoing industry efforts to produce guidelines for interface design best practices (Buellmer and Reising, 2013). The drivers for these development efforts include finding improved ways to maintain real-time operator situational awareness, by producing interface designs for process monitoring systems that are intuitive and understandable. The DCF-VME study expands on this work by seeking to understand how designs influence operator cognitive fatigue over time in real-life vigilance monitoring situations, which will enable a more comprehensive understanding of the factors that may be influencing situational awareness and performance, and how to mitigate performance risks. The research should be conducted by an interdisciplinary team of academics and industry members with experience in fatigue research, human factors design, and oil & gas domain expertise; in collaboration with oil and gas companies exposed to performance risks of this type. Since process safety and risk management will remain top priorities in oil and gas, a greater understanding of cognitive fatigue, its mechanisms and consequences, and display-related contributing factors will enable us to better inform safety management practices, satisfy future regulatory criteria, and provide improved design guidelines and strategies for avoiding costs associated with preventable incidents related to operator fatigue. 770
References Ackerman, P. (2011). Cognitive fatigue: Multidisciplinary perspectives on current research and future applications. Washington, D.C.: American Psychological Association. Balkin, T., & Wesensten, N. (2011). Differentiation of sleepiness and mental fatigue effects. In Ackerman, P. (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future applications (pp. 47-66). Washington, D.C.: American Psychological Association. Buellmer, P., & Reising, D.V. (2013). ASM consortium guidelines: Effective Console Operator HMI Design. Houston, TX: Honeywell International, Inc. Carskadon, M. A., Dement, W. C., Mitler, M. M., Roth, T., Westbrook, P. R., & Keenan, S. (1986). Guidelines for the multiple sleep latency test (MSLT): a standard measure of sleepiness. Sleep, 9(4), 519-524. DOT/PHMSA 49 CFR Parts 192 and 195. U.S. Federal Register, Vol. 74, No. 231 (pp. 63310 63330). Gunzelmann, G., Moore, R., Gluck, K., Van Dongen, H.P.A., & Dinges, D. (2011). Fatigue in sustained attention: Generalizing mechanisms for time awake to time on task. In Ackerman, P. (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future applications (pp. 83-96). Washington, D.C.: American Psychological Association. Hart, S. and Staveland, L. (1998). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock, P. and Meshkati, N. (eds.), Human mental workload, North Holland B.V., Amsterdam, 139-183. Kirwan, B. & Ainsworth, L.K. (1992). A Guide to Task Analysis. London: Taylor & Francis. Lorist, M. and Faber, L. (2011). Consideration of the influence of mental fatigue on controlled and automatic cognitive processes and related neuromodulatory effects. In Ackerman, P. (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future applications (pp. 105-126). Washington, D.C.: American Psychological Association. Mehta, RK., Hutton, M., Shortz, A. (submitted). The neural cost of operator performance. Submitted to MKOCPSC International Conference. Oct 28-30, College Station, TX. Murray, S., Khalafi, N., & Thimgan, M. (2013). Countermeasures to improve workers performance and reduce errors due to inadequate sleep and fatigue. Proceedings of the 16 th Annual MKOPSC International Symposium, Oct 22-24, College Station, TX. Shannon,C.E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27, 379-423, 623-656. Wickens, C.D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2),159-177. Yang, S. & Ferris, T.K. (in progress). The relationship between cognitive efficiency and performance (working title). To be submitted to Human Factors. 771
Yang, S., Shukla, K., & Ferris, T.K. (2012). Cognitive efficiency in display media: A first investigation of basic signal dimensions. Proceedings of the Human Factors and Ergonomics Society 56 th Annual Meeting, 56, 1371-1375. 772