ACOUSTIC TOUCH SCREEN FOR DOLPHINS FIRST APPLICATION OF ELVIS - AN ECHO-LOCATION VISUALIZATION AND INTERFACE SYSTEM J. Starkhammar Dept. of Electrical Measurements, LTH, Lund University, Lund, Sweden M. Amundin Kolmårdens Djurpark, Kolmården and Linköping University, Linköping, Sweden H. Olsén Linköping University, Linköping, Sweden M. Ahlmqvist Dept. of Electrical Measurements, LTH, Lund University, Lund, Sweden K. Lindström Dept. of Electrical Measurements, Lund University, Lund, Sweden H. W. Persson Dept. of Electrical Measurements, LTH, Lund University, Lund, Sweden 1 INTRODUCTION Dolphin sonar has been extensively studied over several decades, and much of its basic characteristics are well known (Au, 1993) [1]. Most of these studies have been based on an experimental setup where the dolphin has been trained to be voluntarily fixed, so its directional sonar beam could be recorded with fixed hydrophones. Although this allows for very exact measurements, it most likely has prevented the full dynamic potential of the dolphin s sonar to be revealed. Also the dolphin s response to scientific questions, e.g. in target detection threshold or discrimination trials, mostly has been a go/no go response or pressing a yes/no paddle. This traditional experimental methodology to measure the response makes rather coarse indications of choice. It is difficult to refine and will be impractical with a multi-choice paradigm. In cognitive studies with primates, e.g. the chimpanzee, a computerized symbol interface, based on a finger operated touch screen, has been successfully used (Rumbaugh et al. 1975) [3]. Even with birds, like chicken and doves, a similar approach has been used where the birds have used their beak to indicate their choices (Cheng & Spetch, 1995) [4]. So far, however, to our knowledge a conventional computer touch screen has not been attempted with dolphins, mainly because the electro-magnetic grid over the touch screen would be short-circuited by the salt water. Delfour [2] (2007) used a system that functionally was a touch screen. It was based on infrared light beams projected through an underwater viewing panel, and guided by mirrors to create a grid in front of the panel, and in front of the TV screen that was placed on the dry side of the panel. The dolphin indicated its choice by breaking the light beams by its rostrum. This system was used in a study testing the dolphin s ability for self-recognition. Although dolphins are known to use their rostrum to touch and manipulate things, we were interested in exploiting and studying their main sensory system, their sonar. Therefore a new tool to study dolphin sonar, psychophysics and cognitive skills was based on the dolphin s sonar beam. The new system was called the EchoLocation Visualization and Interface System (ELVIS) and was developed at Lund University in cooperation with Kolmården Wild Animal Park [5]. It has been further developed and is presently being tested in a dolphin food preference study at the Kolmården Dolphinarium. This system can function as an acoustic touch screen for the dolphins, i.e. the dolphin can indicate a choice by aiming its sonar beam axis at designated areas on the screen. Hence, for the first time the dolphins are given the opportunity to execute and run a computer program using their sonar beam like we use a mouse cursor. This may be a much more intuitive response mode for dolphins than the traditional go/no go or paddle press responses and equal to using their rostrum in the Delfour touch screen [2]. The system is highly adaptable to testing a variety of different scientific questions, since the core of the interface features is software based.
One primary reason to perform this particular study was to evaluate if the system will be possible to use in future food preference investigations at Kolmården Dolphinarium. The aim of this study was to develop a training procedure for introducing the acoustic touch screen concept to the dolphins, to investigate if the concept of the interactive features of the system is comprehensible for the dolphins and to test the software in order to develop it further and optimize it for this purpose. Figure 1. The basic configuration of ELVIS (Echolocation Visualization and Interface System). The screen in this particular mode plots the sound pressure level distribution in the dolphin sonar beam. 2 METHOD 2.1 System configuration ELVIS is based on a matrix of 16 hydrophones as seen in Figure 1. The distance between the hydrophones is 300mm. The hydrophones are attached to a semi-transparent screen lowered into the water of the pool, in front of an underwater acrylic panel. The hydrophones pick up the dolphins sonar signals aimed at the screen. The signals are transferred via cables to an amplifier and signal conditioning unit, and from there via the parallel port to a computer, where the signal analysis is performed by custom designed LabVIEW software. This software constitutes the core of the interactive features of the system. Among many features, it makes it possible to trace the sonar beam axis in real time by generating a round colour spot on the computer screen, corresponding to the maximum intensity in the sound beam. The sound intensity can be coded in colour and/or light intensity. In order to improve the rather coarse resolution given by the 16 hydrophones, the exact location of the maximum sound intensity point is derived through interpolation between the hydrophones in the matrix. The resulting computer screen image is continuously projected back onto the hydrophone matrix screen, using a standard PC projector, hence giving the dolphin an immediate visual feedback to its sonar output. In the food preference study presented here, the theoretical centre of the beam was traced by displaying a dark red spot on the screen. This colour was chosen to make the spot inconspicuous or even invisible to the dolphin (Madsen and Herman, 1980,) [6], but still make it possible for the experimenter to trace the scanning of the sonar beam over the screen.
The software used in this food preference study designates buttons or active areas on the screen, indicated by visual symbols. See Figure 2. These symbols represented different fish species. According to Yaman [7] these symbols should be easily discriminated visually by the dolphins. Which symbols to be displayed on the screen were selected by the systems operator prior to each trial. The symbols were randomly placed over the entire screen, but were always placed over a hydrophone. This was done to make sure that each symbol could be activated at any time, even if the dolphin was very close to the screen, and the whole sonar beam may fall completely in between the hydrophones. The size of these active areas of the screen as well as the trig level can easily be altered so that, as the dolphin s skill in handling the task improves, more accurate aiming and higher sound pressure levels in the dolphin s sound beam can be required. 2.2 Procedure When the dolphin aims its sonar beam axis at a symbol, and generates clicks above a set threshold level, it flashes to indicate a hit and a bridging stimulus (a 400 ms, 10 khz sinus tone) is played through two speakers placed close to the underwater acrylic panel. In this study, each of three such symbols represented a different fish species (mackerel, capelin and squid; Figure 2). When the dolphin clicked on one of them, it was rewarded by the fish represented by it. To help evaluating if the choices were deliberate or random, a fourth symbol was introduced. This was essentially a wrong button, and clicking on it resulted in the trial being aborted without any reward being given. Three female bottlenose dolphins (Tursiops Truncatus) were used in this study: Vicky, born in 1973, arrived at Kolmården in 1979. Ariel, born on October 6, 1996, at Kolmården. Mother: Vicky. Luna, born on January 10, 2001, at Kolmården. Mother: Vicky. Figure 2. Each symbol represents a fish species, except for the unfilled circle which represent false. Only symbols and no text are displayed on the screen. Standard operant conditioning procedures were used for training the dolphins to perform the task. Approximately the same amount of fish of each species (approx. 50g) was given for each correct choice.
3 RESULTS AND DISCUSSION 3.1 Training Procedure The training procedure and the level of understanding the task could be divided into six main steps. Each training session was video taped (the screen and the behaviour of the dolphin from above) and selected parameters were logged by the software in spreadsheet files. These files contain data such as the timestamp of a choice, the active dolphin, the projected symbols (their positions were monitored by filming the screen), the chosen symbol, the required time of echolocating at the symbols to induce a trig of the reward signal, the trig intensity level, the size of the active area around the symbols, and the trig level. 1. At the very first training session only one symbol was projected onto the screen. In this initial phase of the training only capelin was used. See Figure 2. To encourage the dolphins to echolocate towards a symbol, a metal object was held in front of it. The trig level was low and a relatively large area around the symbol was included in the active area, to make it easier for the dolphin to obtain a trig/correct choice. 2. The metal object was phased out and the dolphins were required to aim their sonar beam axis at the symbol, each time appearing in the same place on the screen. The active area around the spot was decreased and the trig level was increased. 3. The position of the symbol was randomly altered for each trial. The dolphin was required to click deliberately on the symbol, wherever it was positioned. 4. A second symbol, representing mackerel, was introduced and appeared at a random position together with symbol number one, forcing the dolphins to make a choice in order to receive the reward. To avoid random trigs during the approach, the dolphin was required to line up in front of the screen before the symbols were displayed. 5. Symbol number three, associated with squid, was introduced, requiring the dolphin to select between all three symbols at the same time. 6. Symbol number four, the error symbol, was introduced when the dolphin was performing the task with confidence and clicking at symbols without hesitation. Only this symbol and one of the food symbols were displayed at a time. Clicking on the error symbol resulted in the trial being aborted, i.e. the screen was black and no reward signal was played. The dolphin had to return to the trainer, received no fish and was given the cue to make a new choice. This symbol was used as a control, aimed at indicating if the dolphin was capable of making deliberate choices between the symbols. The training sessions lasted 5-15 minutes for each dolphin, including 5-40 fish rewards, and were carried out two to three times a week. They were all very motivated to perform the task. Therefore the number of repetitions during each training session was limited only by the amount of fish available for these training sessions. The total daily ratio had to be shared with other daily training tasks, e.g. for the public display programmes and husbandry training. 3.2 Individual differences Luna was the first to understand that the light symbol indicated the active area on the screen. She reached step 6 after 16 training sessions within a total of 223 trials. The development of the other two subjects stagnated at step 5, and their behaviour indicated that they had not even reached step 3. They apparently associated the rewards with aiming their sonar beam axis towards a transducer, but failed to connect this to the symbols. Hence, in some trials they systematically tested the individual transducers until they by chance echolocated at the one covered by a symbol. This may
indicate that they were locked into the acoustic domain, and that the echo characteristics of the transducers were prioritized in the process of associating the stimuli to the task, and not the visual symbols. It may also indicate that the dolphins have a blind spot along the sonar beam axis, making it impossible for them to see the symbol while hitting it with the sonar beam axis. Still it should be possible for them to select a symbol, since often, when the symbols were lit, they were displayed outside the long axis of their snout. Their systematic tesing of hydrophones appeared to occur more frequently when the symbols were displayed at the top (near the water surface) of the screen. This also indicated that the performance of these two dolphins was influenced by the their visual problems. One reason for Luna s better performance might be that she lined up 0.5-1m away from the screen, whereas the other two had their rostrum only 0.1-0.2m from the screen. This probably helped her in getting a better overview of the entire screen, hence making it easier for her to see the symbols at any position on the screen. Luna also had a different clicking technique than the other two. She did not use her sonar at all until she had lined up in front of the screen and located the symbol of her choice. Then she generated a precise click train directly at it. Vicky and Ariel both echolocated during approach and then almost constantly after lining up in front of the screen, even when no symbols were shown. When the symbols were displayed, they continued to echolocate while turning their sonar beam towards the hydrophone of their choice. The preliminar analysis indicates that the dolphins did not make any deliberate choice between the three fish species. This may be due to all fish being equally appreciated they were all a part of their normal fish diet, and were used indiscriminately used during normal training - or just reflect that the dolphins failed to discriminate between the symbols. Luna s behaviour during the trials with the error button indicates that the latter may be true. She clicked equally often on the error button and displayed frustration and confusion when the screen was blanked and no bridging tone was played. One conclusion that can be drawn from this is that the dolphins should have been taught the concept of discrimination before being offered the option to select. 3.3 Evaluation of software The software worked as intended and turned out to be very robust. Most of the interactive features were automated. However, since it turned out to be cruicial not to display the symbols until the dolphin were lined up in front of the screen, this had to be implemented by starting the software manually at precisely the correct moment. After a correct response, the programme had to be stopped manually. Thereafter the symbols had to be deleted from the screen by running a short sequence of the programme with the setting no symbols displayed. These non-automated features will be eliminated in future versions of the software. The bridging stimulus was played via the speakers placed close to the underwater acrylic panel and via the PA system. It was feared that this might have introduced another difficulty for the dolphins in understanding to whom the signal was intended. Normally they would expect the whistle to come from the trainer who gave the the task cue. However, this turned out not to be a problem. All three dolphins reacted immediately to the stimulus, and never hesitated to swim back to the trainer to collect the fish reward. Since the core of the interface system is software based, is it easy to alter i. e. the properties of the visual feedback to help optimize the performance and find a solution that makes the system as intuitive as possible to the dolphins.
4 CONCLUSIONS The interface function of the system worked as intended. The dolphins quickly understood the requirement of echolocating towards the screen. They were highly motivated to perform the task and did not have a problem with understanding that the bridging stimulus, played by the software, was associated with their performance at the screen. One major advantage with the system set-up is the possibility to adjust the visual feedback via the software. The initial training of the dolphins showed that the concept of an acoustic touch screen was comprehensible to all of them. However, individual differences in the learning to use the visual symbols indicate that improvements in the visual feedback are motivated. This was demonstrated by the behaviour of two of the dolphins, that apparently searched for the discriminative stimulus in the acoustic properties of the hydrophones, instead of associating it with the visual symbol. This problem needs to be understood and solved to reach the full potential of the system as an intuitive response tool. The dolphins did not show a preferrence for any of the three fish species. This may be a true nonpreference, but also indicate a failure of the dolphins to discriminate between the symbols. Whether this was due to a visual problem or a result of the training regime remains to be resolved. From all the results discussed, the major conclusion must be that this acoustic touch screen has the potential to be a valuable research tool for future psychophysical and/or cognitive studies of dolphins, e.g. for the more elaborate food preference investigations that will be carried out at the Kolmården Dolphinarium. REFERENCES 1. Au, W. L.,The Sonar of Dolphins. Springer-Verlag, New York, 277pp. (1993) 2. Delfour F. Marine mammals in front of the mirror body experiences to self-recognition: A cognitive ethological methodology combined with phenological questioning. Aquatic Mammals 32(4):517-527.(2007) 3. Rumbaugh D.M, Gill, T.V, von Glasersfeld, E, Warner, H, and Pisani, P., Conversations With a Chimpanzee in a Computer-Controlled Environment. Biol Psychiatry. 10(6):627-41. (1975) 4. Cheng, K., & Spetch, M. L., Stimulus control in the use of landmarks by pigeons in a touchscreen task. Journal of the Experimental Analysis of Behaviour, 63, 187-201. (1995) 5. Nilsson, M., Lindström, K., Amundin, M., Persson, H. W., Echolocation and Visualization Interface System, Clinical Physiology and Functional Imaging, 24, 3, 169-178, (2004) 6. Madsen, C.J. and Herman, L.M., Social and Ecological Correlates of Cetacean Vision and Visual Appearance. In: Cetacean Behavior- Mechanisms and Functions. L.M. Herman (ed.) John Wiley & Sons, NY. (1980) 7. Yaman, S. von Fersen, L., Dehnhardt G., Güntürkü O., Visual Lateralization in the Bottlenose Dolphin (Tursiops truncatus): Evidence for a Population Asymmetry. Behavioural Brain Research, 143, 109-114. (2003)