A case study of driver behaviour using eye-tracker

Transcription

1 BORYS Magdalena 1 A case study of driver behaviour using eye-tracker INTRODUCTION The earliest eye-trackers were built in the late 1800s. But at the beginning of the twentieth century the much less invasive technique for recording eye movements using the principle of photographing the reflection of an external light source from the fovea was introduced, and since then individual researchers have developed a number of different techniques to build eye-tracking systems. Nowadays the most popular are a video-based eye-tracker that has an infrared illumination and an eye video camera. Although the manufactures use the same video-based pupil-to-corneal reflection measurement technology, the diversity of customer groups has led them to produce very different kind of eye-trackers. There are three types of video-based eye-trackers: most common static eye-tracker, head-mounted eye-tracker and head-tracker [1]. Head-mounted systems have cameras and illuminations mounted on top of a helmet, cap, headband or a pair of glasses, thus allow the participants of eye-tracking experiment maximum mobility [1] and as a result the participants can take part in many different real life activities such as driving, riding a bicycle, flying a plane, shopping, playing sports, observing the art in non-laboratory environments. Moreover, head-mounted eye-trackers are usually equipped with additional scene camera. The set-up allows to combine the scene video with the gaze coordinates and results with a gaze-overlaid video that can be used for further analysis such as a scanpaths, attention maps or Areas of Interest (AOI). Eye-trackers have been increasingly used in research concerning driving, initially as an element of a driving simulator, but now more often as a part of instrumented vehicle in field studies. Eye-trackers enabled researchers to gain insights into driving behaviour and events often not remembered by drivers. Eye-tracking data provide important information for understanding the nature of the driving activities by showing significant gaze patterns. Moreover, eye-tracking data are important for developing driver training strategies and accident countermeasures [2]. The pilot study on driver behaviour using a non-instrumented vehicle and head-mounted eyetracker were proposed and conducted. The objective of the study was to examined whether the used equipment is appropriate for conducting driving experiment and do collected data, as well used software, are sufficient to provide the analysis of driver behaviour. In particular the distraction of drivers by roadside advertisements were investigated using the selected measures of visual and cognitive load. 1. BACKGROUND AND RELATED WORKS In recent years driving studies incorporating an eye-tracker has been focused on the following areas of investigation: detection of driver fatigue or drowsiness, driver awareness in different road situation, driver visual attention and cognitive load. The research connected with fatigue or drowsiness detection are aimed to develop monitoring and/or warning system dedicated mainly for professional drivers. Those systems are based generally on tracking data such as pupil movements, eyelid movements or face pose and sophisticated algorithms that process tracking data and initiate alarm in real-time. 1 Politechnika Lubelska, Wydział Elektrotechniki i Informatyki; Lublin, ul. Nadbystrzycka 38A. m.borys@pollub.pl Author is a participant of the project: "Qualifications for the labour market - employer friendly university", co-financed by European Union from European Social Fund. 699

2 Situation awareness is defined as the perception of elements in the driving environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future [3]. The importance of situation awareness in driving studies is emphasized by the fact that lacking of situation awareness or having inadequate situation awareness has been classified as one of the primary factors in accidents caused by human error. As situation awareness is a complex process, the research focus on measuring and understanding components of situation awareness, defining the model of multitasking and testing how training can improve with driving experience, and thus driving safety [4]. Cognitive load (or mental workload) is defined as the relationship between the cognitive demands placed on a user by a task and the user s cognitive resources [5]. Therefore, the load is relative to the user and the task being performed in a given environment. The higher the user s cognitive load is, the higher the chance is that the user will not complete a task without an error [6]. Cognitive load can be assessed using performance, physiological and subjective measures. While cognitive load is associated with cognitive, auditory or verbal tasks, visual attention (or visual load) is related to visually demanding tasks (distractions) performed by user. Visual and cognitive load affect driving performance in different ways. In all eye tracking experiments in driver behaviour area there are two main eye movements called a fixation and a saccade. A fixation is defined as the maintaining of the visual gaze on a single location, while a saccade is a fast movement of an eye Visual and cognitive load works The visual demand imposed by secondary tasks can be directly quantified by means of eye movement measures, for example glance frequency and mean duration [7, 8]. Selected comprehensive studies in this area are reviewed below. Since the safe driving depends on driver ability to rapidly search visually complex scene, the experiment using latency and eye movement as a measure to examine the effect of clutter and secondary task (conversation) on visual search for traffic signs in two age groups were proposed by McPhee et al. [9] based on the earlier research [10, 11]. The results showed that clutter as well as the secondary task have additive effects on visual search accuracy, speed and oculomotor involvement in both age groups. However, the older adults were less accurate, especially with high-clutter scenes what effects in higher average fixation number, and were slower in making decision based on visual search. The older adults, compared with the younger, exhibited longer fixations in the dividedattention condition. This may increase their risk of missing critical information in the environment and limit their ability to perform secondary tasks while driving. The effect of cognitive load on visual search, discrimination and decision making was investigated by Recarte and Nunes [12, 13]. In the latest work, the cognitive load was manipulated by requesting participants perform several mental tasks while driving. Cognitive tasks produced spatial gaze concentration and visual-detection impairment. This impairment, according to ocular behaviour analysis, was due to late detection and poor identification. Moreover secondary tasks such as production tasks, complex conversations, whether by phone or with a passenger, were more dangerous for road safety that verbal acquisition tasks. For visual search and cognitive tasks measures such as pupil diameter, effort ratings, spatial gaze variability and glances at the speedometer and mirrors are analysed. For detection, discrimination, and response selection measures such as response time, glances at the targets, eccentricity and detection were used. The impact of visual and cognitive demand on driving performance and driver state were systematically investigated within the EU project HASTE [14]. The study based on performing secondary tasks on in-vehicle system (also referred as In-vehicle Information System, IVIS) concurrently with the primary driving task by a driver in three different experiment settings: in a fixed base simulator, in a moving base simulator and in an instrumented vehicle driven in real traffic. The results reveal that visual load led to reduced speed and increased lane keeping variation, while cognitive load resulted in increased gaze concentration towards the road centre. 700

3 Within the same project the newer measures of the sensitivity of eye movement to the demands of visual and auditory in-vehicle tasks and driving tasks were discussed [15]. In their work, Victor et al. presented percent road centre and standard deviation of gaze as more sensitive, robust and reliable that glance-based measures such as total glance duration, glance frequency, glance duration, and total task duration, where a glance describes the transition to a given area, and one or more consecutive fixations on the area until the eyes are moved to a new location. Moreover, the results showed that drivers adapt their eye movement behaviour to the driving environment. They increase viewing time in the road centre area when primary driving task difficulty increases, as evidenced by the spatial gaze concentration being highest in the rural curves, followed by rural straight sections, the simulated motorway, and the field motorway. The research on estimation of driver s cognitive load based on pupil size measurement were presented in [6]. Authors proposed the experiment in which pairs of subjects are engaged in two spoken tasks and one of the subjects (the driver) also operates a simulated vehicle. During the experiment the physiological pupillometric date and driving performance measures were recorded and compared. The results showed correlation between two driving performance measures and the mean pupil diameter change under our experimental conditions. Therefore, authors indicated that the mean pupil diameter change and the mean pupil diameter change rate were finer measures of cognitive load in a driving simulator than variances of lane position and steering wheel angle. Many studies have reported degrading effects of cognitively loading tasks in terms of reduced event detection performance due to various distractions: cognitive, auditory, verbal, visual, motoric, somatosensory, smell, taste. However, the phenomenon of change blindness, presented in work of Rensink [16] as insensitivity of observers to change appearance, appears irrespectively. Change blindness in driving scenes resulting from brief flicker, eye blinks or saccadic eye movement were discussed advanced in [17]. 2. RESEARCH METHOD 2.1. Research objective and setup The research hypothesis was to investigate whether (1) it is possible to use head-mounted eyetracker (in this case SMI Eye Tracking Glasses 2.0) to collect eye movement data from an user driving a non-instrumented vehicle for studying his visual attention and cognitive load and (2) do the physiological attributes, such as a user silhouette construction or user height, affect the quality of obtained data from the study. Moreover, the build-in software for eye movement analysis, SMI BeGaze, delivered with eye-tracker was tested in the context of analysis of driver visual and cognitive load. The case study aimed to examined the influence of roadside advertisements like billboards, business signs or logos, on driver visual attention and therefor for driving safety. Since advertisements are located along main roads they distract the detection of traffic signs and possibly also other objects relevant to the driver s task. In the study there were 2 adults (one female, 40 years old and one male, 30 years old). The participants were volunteers from the university and were not paid for their effort. Both participants had held a license for at least 12 years. They drove their cars regularly and they had driven approx km the past 12 months. None of the participants was a professional driver. The case study was conducted on urban road in city Lublin, Poland. The route was 7 km long and had from one to three lanes in the different sections. The route was known by the participants. During the study there was moderated road traffic. The study was conducted between 11 am and 2 pm, so it was done in daylight Equipment and software The participants had driven the route in their own cars (manufactured by Honda and Opel) in order to eliminate distractions resulting from the use of an unknown vehicle. Eye movements were measured using head-mounted eye-tracker SMI Eye Tracking Glasses 2.0 with pocket-size recorder. The used eye-tracker relies on unmatched binocular eye tracking 701

4 performance of 60 Hz and determines gaze direction based on video signals from remote cameras. The HD scene camera of used eye-tracker runs at 30Hz. Gaze position accuracy is 0.5 degree over all distances with parallax compensation. The participant wearing the eye-tracker in a vehicle is presented in Figure 1. The eye movement data were analysed using SMI BeGaze software compatible with the eyetracker. Build-in signal-processing algorithms for calibration, data quality management, saccade, fixation and AOI analysis were used Procedure In the case study each participant drove once on the road; both in the same direction. The participants drove the route with comparable road traffic, weather and lighting conditions. The participants drove alone, no moderator/experimenter accompanied them during runs in order to minimized the additional distractions. The participants were requested to drive the route in their own car. Moreover they were instructed to behave as usual during driving. Before each runs the build-in 3-points calibration procedure for SMI Eye Tracking Glasses 2.0 were performed. Fig. 1. The participant wearing the eye-tracker in a vehicle 3. RESULTS During the study raw data in the form of video of driving scene (stimulus), left and right eyes positions and left and right eyes pupil diameters were recorded. The raw data were used for further analysis such as basic statistics (tab. 1), scanpaths, analysis of AOIs. Tab. 1. Statistics of eye movements recording for each participant Participant 1 Participant 2 Duration 00:27:39 0:25:41 Number of samples Number of fixations Number of saccades Number of blinks The scanpaths based on eye position (fig. 2) and fixation were calculated. The scanpaths allowed to conduct manual analysis of driver behaviour and to mark interesting events such as looking into side-mirrors or central mirror, looking at light signs, pedestrians and so on. The marked events might be used for statistics or to get eye and gaze properties at specific time. What is more, during the manual analysis it was concluded that quality of data obtained from participant 2 was better, what is proved by total number of scored fixation and saccades. 702

5 During the analysis the Areas of Interest (AOIs), such as left and right roadside advertisements, were marked on the stimulus. The marked advertisements were at least 1.5 m high and wide and marking process were conducted frame by frame of video. Having marked AOIs on stimulus enabled the calculation of the relative fixation time on each AOI (dwell time). However, it was not possible to get statistics such as average fixation time, total fixation number in area or average pupil diameter, or group the AOIs to calculate statistics in group. Fig. 2. A video of driving scene with marked driver s gaze point at a billboard [4] CONCULUSIONS The research aimed to investigate whether it is possible to use head-mounted eye-tracker to collect eye movement data from an user driving a non-instrumented vehicle for studying his visual attention and cognitive load and if the physiological attributes affect the quality of obtained data from the study. The conducted case study allowed to check the technical facility of used equipment and setup as well as to examine the recorded data and their usefulness for driver behaviour analysis. Moreover, thanks to the study the usefulness of software delivered by eye-tracker manufacturer in such studies were determined. There was observed the differences in data quality between users, but it is not possible to conclude if it is connected with participants physiological attributes such as silhouette or head construction or just improper calibration. It was concluded that lack of a instrumented vehicle or additional equipment for measuring the driving speed, acceleration, speed and brake pressures very limited the study design and analysis. The other disadvantage of study in the field compering to simulation was lack of whole driving scene recording, which disable possibility of registration of all on-road events. The participants also complained on covering the part of peripheral vision area by eye-tracking glasses, so they have to turn head to check the road situation on sides. Therefore, it can be concluded that used eye-tracker could have an influence on driver safety. During the analysis it was observed that it is not possible to distinguish the aim of fixation between advertisements and other objects such as cars, road signs or traffic light, because they overlapped in scene. This exposed the laboratory study is more liable than the field study in this context. Moreover, according to differences in driver cognitive load during various driving setup (urban or rural driving, lever of clutter or traffic) the experiment should be conducted in different driving setup. The results of case study are promising and could be led on significant user sample, but several improvement connecting to experiment design and used equipment must be provided. Furthermore, the additional software for data analysis should be proposed and used. 703

6 Abstract The paper presents the purpose of using the eye-tracker, as well as points the main area of its eye-tracking data application in driving experiments. The review of related works on driver visual and cognitive load is presented and discussed. The method and results from pilot study aimed to examine the possibility of usage of head-mounted eye-tracker (SMI Eye Tracking Glasses 2.0) to collect eye movement data from an user driving a non-instrumented vehicle in order to analyse his visual attention and cognitive load. The study also tries to verify if user s physiological attributes do affect the quality of data. The paper points out the limitations of used study design as well as used equipment. It is concluded that the experiment might be extended on significant user sample, but several improvement must be provided. Studium przypadku zachowania kierowcy przy użyciu eye-tracker Streszczenie Niniejszy artykuł prezentuje wykorzystania śledzenia ruchu gałek ocznych, jak również główne obszary zastosowania tej technologii w badaniach związanych z pojazdami. Zaprezentowany i opisany został przegląd istniejących prac nad nadmiernym obciążeniem wizualnym oraz poznawczym kierowców. Zaprezentowana metoda badawcza i wyniki badań pilotażowych miały na celu sprawdzenie możliwości wykorzystania mobilnego urządzenia do śledzenia ruchu gałek ocznych (SMI Eye Tracking Glasses 2.0). Przeprowadzone badania również próbują zweryfikować pytanie badawcze, czy cechy fizjologiczne mają wpływ na jakość otrzymanych danych. Artykuł wskazuje na ograniczenia zastosowanej metody badawczej, jak i użytego oprzyrządowania. Zaprezentowany eksperyment może zostać przeprowadzony na zwiększonej próbie badawczej, ale muszą zostać wprowadzone udoskonalenia wskazane przez autora. REFERENCES 1. Holmqvist K., Nystrom M., Andersson R., Dewhurst R., Jarodzka H., van de Weijer J., Eye Tracking: A comprehensive guide to methods and measures. Oxford University Press, USA, Duchowski A., Eye Tracking Methodology: Theory and Practice. Springer, Endsley M.R., Toward a theory of situation awareness in dynamic systems. Human Factors 37(1), 1995, pp Gugerty L., Situation awareness in driving. Handbook for driving simulation in engineering, medicine and psychology, pp Wickens C. D., Multiple Resources and Performance Prediction. Theoretical Issues in Ergonomics Science, 3(2), 2002, pp Palinko O., Kun A. L., Shyrokov A., Heeman P., Estimating cognitive load using remote eye tracking in a driving simulator. Proceedings of the 2010 Symposium on Eye-Tracking Research and Applications. ACM, ISO Road vehicles Measurement of driver visual behaviour with respect to transport information and control systems. Part 1: Definitions and parameters. 8. ISO Road vehicles Measurement of driver visual behaviour with respect to transport information and control systems. Part 2: Equipment and procedures. 9. McPhee L. C., Scialfa C. T., Dennis W. M., Ho G., Caird J. K., Age differences in visual search for traffic signs during a simulated conversation. Human Factors: The Journal of the Human Factors and Ergonomics Society 46(4), 2004, pp Ho G., Scialfa C. T., Caird J. K., Graw T., Visual Search for Traffic Signs: The Effects of Clutter, Luminance, and Aging. Human Factors, 43(3), 2001, pp Scialfa C. T., McPhee L., Ho G., The effects of a simulated cellular phone conversation on search for traffic signs in an elderly sample. Proceedings of the 2000 symposium on Eye tracking research & applications. ACM, Recarte M. A., Nunes L. M., Effects of verbal and spatial-imagery tasks on eye fixations while driving. Journal of Experimental Psychology: Applied 6(1), Recarte M. A., Nunes L. M., Mental workload while driving: effects on visual search, discrimination, and decision making. Journal of experimental psychology: Applied 9(2),

7 14. Engström J., Johansson E., Östlund J., Effects of visual and cognitive load in real and simulated motorway driving. Transportation Research Part F: Traffic Psychology and Behaviour 8(2), 2005, pp Victor T. W., Harbluk J. L., Engström J. A., Sensitivity of eye-movement measures to in-vehicle task difficulty. Transportation Research Part F: Traffic Psychology and Behaviour 8(2), 2005, pp Rensink R. A., Change detection. Annual review of psychology 53(1), 2002, pp Galpin A., Underwood G., Crundall D., Change blindness in driving scenes. Transportation research part F: traffic psychology and behaviour, 12(2), 2009, pp