Little to be Expected from Auditory Training for Improving Visual Temporal Discrimination

Transcription

1 95 ARTICLE Little to be Expected from Auditory Training for Improving Visual Temporal Discrimination Simon Grondin, Nicolas Bisson, Caroline Gagnon Pierre-Luc Gamache and Andrée-Anne Matteau Abstract It is known that the processing of temporal information is much more efficient in the auditory than in the visual modality. This study analyzes the possibility of improving the discrimination of short temporal intervals marked by two brief visual signals. The present experiment involves the simultaneous presentation of both auditory and visual signals and extensive duration discrimination training. The results show that visual duration discrimination gains a slight but statistically significant benefit from the auditory context, but that this benefit is not permanent and may partly depend on the initial level of discrimination. Finally, in spite of extensive training, performance levels remained much lower when only visual signals were presented than when auditory signals were presented. Key Words: Duration discrimination, sensory modalities, cross-modal transfer, learning, training NeuroQuantology 2009; 1: Introduction 1 People have the cochlea for audition or the retina for vision, but apparently, they have no sensory receptor for time. Similarly, there is no specific cerebral cortex for time that is comparable to the auditory or the visual cortex. Moreover, it is difficult to associate a specific type of stimulation with time perception in the same way that electromagnetic stimulations can be associated with vision. Nevertheless, people are able to make judgments about time and, therefore, there must be some process that leads to such judgments. However, it is still difficult to determine if this process is based on a dedicated neural mechanism (Gibbon et al, 1984; Killeen and Weiss, 1987; for a review, see Grondin, 2001; or Penney and Vaitilingam, Corresponding author: Simon Grondin Address: École de psychologie, 2325 rue des Bibliothèques, Université Laval, Québec, Qc, Canada G1V 0A6 Phone: x simon.grondin@psy.ulaval.ca 2008), or is part of modality-specific processing (Buonomano, 2000; Burr et al, 2007). The important thing at this stage of research on psychological time is to determine the extent to which the learning of temporal information can be transferred from one modality to another (Ulrich et al, 2006). There is evidence, for instance, that a common mechanism serves interval discrimination and production (Ivry and Hazeltine, 1995). Indeed, it has been shown that temporal learning based on auditory interval discrimination is transferred to the production of intervals of the same range (Meegan et al, 2000). If the learning of temporal information can be transferred from a perceptual task to a production task, it is reasonable to posit that it is possible to transfer temporal learning from one perception task to another. If the transfer of learning from a perception to a production task depends on the quality of the representation of temporal information developed with the

2 96 perception task, then such improved representation should also be helpful for transferring information from a perception task in one modality to a similar task in another modality. There is evidence of cross-modal transfer of temporal information involving auditory and visual signals in the timing literature on animals (Roberts, 1982) and humans (Warm et al, 1975). However, these experiments involve very long intervals. The most convincing proof for the learning transfer of short temporal intervals (in the range of hundreds of ms) comes from studies involving auditory and tactile modalities. There are withinmodality transfers, for interval discrimination at specific durations, in the auditory mode (Karmarkar and Buonomano, 2003; Wright et al, 1997). For example, it has been argued that the learning of 100-ms intervals marked by 1-kHz auditory signals can be transferred to 100-ms intervals marked by 4-kHz tones but not to 50- ms, 200-ms or 400-ms intervals marked by 1-kHz tones (Wright et al., 1997). As well, it has been shown that training for the discrimination of intervals marked by visual signals processed by one cerebral hemisphere is transferable to the other hemisphere (Westheimer, 1999). It has also been demonstrated that learning related to interval discrimination involving tactile stimuli can be transferred across skin locations (Nagarajan et al 1998). Most importantly, these authors have also reported that learning of a temporal task in the tactile modality can be generalized to a similar task involving auditory signals. Whether the learning of short interval discrimination can be transferred from one sensory mode to the visual mode remains an open and difficult question (Théroux, 2003). Recent experiments with humans have tended to show that visual temporal processing benefits from exposure to auditory temporal processing (Grondin, 2005; Grondin et al, 2005). In these experiments, presenting both auditory marked and visually marked intervals within blocks led to slightly better visual performances than did presenting only visual intervals within blocks. On the other hand, in a recent experiment where 25% of intervals were marked by visual signals and 75% by auditory signals within blocks, the discrimination of visual marked intervals was significantly weaker (higher discrimination threshold) than the controlled performance where only visual intervals were presented (Grondin et al, 2008). In a pilot experiment where participants had to discriminate a short (238 ms) and a long (262 ms) interval, a fivesession training of 200 trials per session (100 short and 100 long intervals) in the visual modality did not lead to any improvement in performance, although similar training with auditory presented intervals (or with simultaneous presentations of auditory and visual signals) led to a slight (but not significant) improvement in the discrimination of visual intervals (Grondin et al, 2007). In addition, Lapid et al (in press) reported no cross-modal transfer from auditory temporal discrimination to the visual modality. The purpose of the present study is to determine if discrimination of short time intervals marked by brief visual signals can be improved in a context involving multiple presentations of auditory marked intervals. More specifically, conditions were created during the experiment to favour as much as possible the transfer of what could be learnt in the auditory mode to the visual mode. Instead of presenting intervals of various lengths as is done when trying to establish a threshold value, the number of intervals was limited to two. In other words, only two intervals had to be learned. In addition, the experiment involved quite extensive training (26 sessions, 300 trials per session). Moreover, the training in the auditory modality involved not only the presentation of auditory signals, but also the simultaneous presentation of visual signals. Because our preliminary data indicated that it is very difficult to improve visual temporal discrimination (Grondin et al, 2007; 2008), we tried to maximise the chances of obtaining an improvement. It was posited that such improvement might come not only from extensive interval discrimination training, but from some form of conditioning resulting from the repeated association of auditory and visual signals. Method Participants Nine volunteer students at Université Laval took part in the experiment, including 4 women and 5

3 97 men. They received $130 for participating in the 26 sessions of the experiment. The average age was 26.3 years (SD = 7.45), 24.5 years for the women (SD = 5.69) and 27.8 years for the men (SD = 8.98). Apparatus and stimuli The intervals to be discriminated were silent durations between 20-ms auditory or visual stimuli. The auditory stimuli were 1-kHz pure sinusoidal sounds generated by an IBM Pentium IV micro-computer running E-Prime software (version SP3). The computer was equipped with a Sound Blaster Audigy 2 sound card, and the sounds were presented binaurally through headphones (Sennheiser HD 477) at an intensity of about 70 db SPL. The visual stimuli were produced by a circular red-light-emitting diode (LED; Radio-Shack # ) placed about 1 m in front of the participant and subtending a visual angle of about.57 o. Each observer was seated in a dimly lit room and asked to respond whether the interval presented between the brief signals was short or long by pressing 1 or 3 on the computer keyboard, respectively. Procedure On each trial, the participant was presented with one of two intervals (single stimulus procedure) and had to indicate, by pressing the appropriate button, whether the interval was the short or the long one. Feedback informing the participant whether the interval was short or long appeared on the screen,.5 s after each response, for 1.5 s. The next trial began.5 s after the feedback disappeared. Each participant completed from one to three calibration sessions before beginning the experimental trials. The results from the session(s) were used to calibrate the experiment for each participant such that the participant provided between 63% and 72% correct detection and correct rejection, in the visual duration discrimination condition, at the beginning of the training period. This calibration was used to make sure that the discrimination in the visual mode would not be too difficult but would also leave room for improvement. Depending on the calibration results, each trial consisted of two 20 ms signals separated by an interval of either 233 ms (short) or 267 ms (long) for 4 participants, or of either 236 ms (short) or 264 ms (long) for 5 participants. After calibration, each participant completed 26 experimental sessions of 20 to 30 minutes. Each session was divided into five blocks of 60 trials, including 30 long and 30 short randomly presented intervals. The five blocks were separated by a 20 seconds rest period. A minimum of one hour was required between sessions. The first five experimental sessions consisted exclusively of visual signals (see Table 1). This provided an opportunity to see if there was any improvement with trials restricted to visual markers. In Sessions 6-10, auditory and visual signals were presented simultaneously. The eleventh session was identical to the first five sessions, i.e., entirely visual, and was included to verify whether any improvement could be observed at this stage of the experiment. Table 1. Modality of markers in each session Sessions Blocks Modalities Trials (%) Visual only Visual + Auditory Visual only Visual + Auditory Visual + Auditory Visual only Visual only 100 Visual + Auditory: simultaneous presentation Sessions were structured as follows: the first block combined auditory and visual signals, like Sessions 6-10; however, in Blocks 2-5, nine times out of ten the signals marking the intervals were presented simultaneously in the auditory and visual modalities, and one trial in ten, distributed randomly, was exclusively visual. In other words, there were only 3 short and 3 long intervals per block with only visual markers, and, therefore, 12 short and 12 long intervals per session. Consequently, to calculate performance with visual signals only, the data of Sessions and of Sessions were grouped together (results based on 60 short and 60 long intervals in each case). It was postulated that, if the participants were sufficiently conditioned to associate auditory and visual markers, the

4 98 withdrawal of auditory markers in a given trial would not impair performance. The experiment was completed with Sessions 22-26, which, like Sessions 1-5, were exclusively visual. It was anticipated that any learning that moved the discrimination level with only visual intervals to the level observed with auditory signals would be apparent even without the context involving auditory signals. If such improvement was due to the association between auditory and visual markers, the effect would fade rapidly without the presentation of auditory signals. If the improvement was due to the learning of temporal information, i.e., to a stable representation of the short and the long intervals, then the performance level would remain high or even further improve in Sessions Data analysis The statistical analysis of data is based on the parameters of the Signal Detection Theory (SDT). There is dependent variable of interest, sensitivity, estimated with d. It was computed on the basis of two assumptions: the distributions for the short and long intervals were normal and had equal variances. Indeed, the short and long distributions are respectively the noise and signal + noise distributions in the SDT. In the present experiment, a hit was responding long when the interval was long, and a false alarm was responding long when the interval was short. d was computed with the computer program reported by Macmillan and Creelman (1991, Appendix 6). Results Figure 1 shows the mean d in the visual condition for each of Sessions 1-5, 11 and and for Sessions and 17-21, considered as two groups. A comparison of the mean of these sessions (1.03) with the results of Session 11 (1.29), which followed five sessions of training (1 500 trials in total) where both auditory and visual signals were presented in each trial, reveals that performance was significantly better after training, t(8) = 2.63, p <.05. However, additional training did not provide much benefit as the mean d went from 1.29 in Session 11 to 1.14 and 1.12 in Sessions and 17-21, respectively. However, a comparison of the mean d of Sessions 1-5 with the d value (1.33) of Session 22 (the first full visual session after 10 sessions of additional training involving auditory signals) shows a significant difference, t(8) = 3.02, p <.05, while a comparison of the mean d value of Sessions 1-5 with the d of the last (26 th ) session (1.17) reveals no significant difference, t(8) = 1.34, p =.22. Figure 1. Mean d in the visual condition, for each of Sessions 1-5, 11 and 22-26, and for Sessions and grouped. Bars indicate standard errors (having less judgments per data points explains the large bars in Sessions and 17-21)

5 99 Figure 2. Mean d for each sub-group ( ms vs ms) for Sessions 11-16, and The mean d for each sub-group ( ms vs ms) was about the same for Sessions 1-5 (1.00 vs. 1.06) and Session 11 (1.29 for both)

6 100 Discussion There are essentially two main findings in the present study. On the one hand, it was shown that extensive training can improve visual temporal processing in an interval discrimination task. On the other hand, discrimination involving the presentation of visual signals alone remained much weaker than discrimination involving auditory signals. There was some improvement in the conditions involving only visual signals. However, it is difficult to determine definitively if the effect is due simply to the number of trials (i.e., if there would have been an effect with only the presentation of intervals marked with visual signals) or if the presence of auditory signals played an essential role. Some effect in early sessions of training has been reported for duration discrimination in visual (Westheimer, 1999), tactile (Nagarajan et al, 1998) and auditory conditions (Lapid et al, 2006; but see Rammsayer, 1994). In Grondin et al (2007), the level of discrimination between one short and one long interval marked with flashes was about the same in Sessions 1 and 7, which were separated by trial training sessions for visual interval discrimination. This suggests that the benefit observed in Session 11 in the present study is probably due to the presence of auditory signals. This gain seems fragile though, given that the levels of performance in visual conditions tended to diminish once the auditory context was withdrawn (Sessions 22-26). Indeed, the capacity to maintain benefits after auditory exposure seems to depend on task difficulty level, which was set according to the initial capabilities of observers (Figure 2). In addition to the observed improvement in the visual condition, a striking finding is the huge gap between the conditions with and without auditory signals that remained present throughout the experiment. If any benefit is derived for visual interval discrimination from auditory interval discrimination, it does not compare to the findings reported by Nagarajan et al (1998) where discrimination levels in the auditory mode were comparable to those obtained in tactile training conditions. Indeed, in that study, it is surprising that the performance in audition was not better, per se, than the one in the tactile condition, given the results usually reported in the interval discrimination literature where sensory modalities are compared (Grondin and Rousseau, 1991; see Observer BH in Westheimer, 1999; or, for a review, Grondin, 2003). Had there been a genuine transfer of the learning of temporal information in the present study from the auditory to the visual mode, the d value in the visual conditions would not have stayed below 1.3, but would have increased to around 3 as in the case of the auditory trials. The difference between the auditory and visual performances was just as great as that usually reported in the literature (Grondin, 1993; Grondin and Rousseau, 1991). In other words, the basis of the auditory mode's efficiency for temporal processing does not seem transferable. Not only did extensive training in the duration discrimination task not lead to much improvement, but the potential link that could have been created over 4200 trials between simultaneous signals from the auditory and visual modes did not allow the visual performances to come close to those obtained when auditory signals were presented. This failure to boost visual performances is not due to a lack of attention to visual signals on the part of the participants when signals from both modalities were presented since the visual performances for blocks where auditory signals were randomly omitted remained above 1.1 (Figure 1). The difficulty of transferring perceptual learning from one sensory mode to the visual mode might be related in part to the intrinsic properties of the modalities themselves. While multisensory studies often indicate that there is an attentional bias for vision (Spence et al, 2001), tactile and auditory signals share alerting properties (Posner, 1978). Perhaps there is some specific, potentially bidirectional, transfer of learning related to temporal information between audition and tactile modes (Nagarajan et al, 1998), or between the auditory mode and the motor system (Meegan et al, 2000). The status of vision for temporal processing appears to be special. For instance, it is known that visual signals can distort the perceived duration of auditory temporal perception, but the duration of visual events is not affected by the addition of auditory information (van Wassenhove et al, 2008; see

7 101 also Schutz and Lipscomb, 2007). In the case of visual duration discrimination, the presence of an auditory context only contributes slightly to improving performance; and although vision influences auditory temporal perception, it is difficult to imagine how vision can help to improve auditory interval discrimination given the large superiority of audition over vision for duration discrimination. This experiment does not make it possible to decide whether or not there is a dedicated neural device for time perception. However, it does show the huge gap between visual and auditory tempo

8 102 References Buonomano DV. Decoding temporal information: A model based on short-term synaptic plasticity. J Neurosci 2000; 20: Burr D, Tozzi A, Morrone MC. Neural mechanisms for timing visual events are spatially selective in real-world coordinates. Nat Neurosci 2007; 10 (4): Gibbon J, Church RM, Meck WH. Scalar timing in memory. Ann N Y Acad Sci 1984; 423: Grondin S. Duration discrimination of empty and filled intervals marked by auditory and visual signals. Percept Psychophys 1993; 54: Grondin S. From physical time to the first and second moments of psychological time. Psychol Bull 2001; 127: Grondin S. Sensory modalities and temporal processing. In: Helfrich H editor. Time and Mind 02. Goettingen: Hogrefe & Huber; p Grondin S. Overloading temporal memory. J Exp Psychol Hum Percept Perform 2005; 31: Grondin S. Methods for studying psychological time. In: Grondin S editor. Psychology of time. Bingley: Emerald Group Publishing; p Grondin S, Gamache PL, Roussel MÈ, Pouliot M, Plourde M. Mixing durations and sensory modes in temporal memory. In: Monahan JS, Sheffert SM, Townsend JT, editors. Fechner Day 2005: Proceedings of the 21 st Annual Meeting of the International Society for Psychophysics; 2005 Oct 19-22; Mt. Pleasant, MI. Mt. Pleasant, MI: The International Society for Psychophysics. Mt. Pleasant; Grondin S, Gamache PL, Tobin S, Bisson N, Hawke L. Categorization of brief temporal intervals: An auditory processing context may impair visual performances. Acoust Sci & Tech 2008; 29: Grondin S, Gamache PL, Tobin S, Gosselin A, Bisson N, Hawke L. About the difficulty to transfer temporal learning across sensory modality. In Mori S, Miyaoka T, Wong W editors. Fechner Day 2007: Proceedings of the 23 rd Annual Meeting of the International Society for Psychophysics; 2007 Oct 20-23; Tokyo, Japan. Tokyo: The International Society for Psychophysics; Grondin S, Rousseau R. Judging the relative duration of multimodal short empty time intervals. Percept Psychophys 1991; 49: Ivry RB, Hazeltine RE. The perception and production of temporal intervals across a range of durations: Evidence for a common timing mechanism. J Exp Psychol Hum Percept Perform 1995; 21: Karmarkar UR, Buonomano DV. Temporal specificity of perceptual learning in an auditory discrimination task. Learn Mem 2003; 10: Killeen PR, Weiss NA. Optimal timing and the Weber function. Psychol Rev 1987; 94: Lapid E, Ulrich R, Rammsayer T. Perceptual learning in auditory temporal discrimination: No evidence for a cross-modal transfer to the visual modality. Psychon Bull Rev (in press). Macmillan NA, Creelman, CD. Detection theory: A user s guide. New York: Cambridge Univ. Press; Mauk MD, Buonomano DV. The neural basis of temporal processing. Annu Rev Neurosci 2004; 27: Meck WH. Functional and Neural Mechanisms of Interval Timing. Boca Raton, Fl: CRC Press; Meegan DV, Aslin RN, Jacobs RA. Motor timing learned without motor training. Nat Neurosci 2000; 3: Nagarajan SS, Blake DT, Wright BA, Byl N, Merzenich MM. Practice-related improvements in somatosensory interval discrimination are temporally specific but generalize across skin location, hemisphere, and modality. J Neurosci 1998; 18: Penney T, Vaitilingam L. Imaging time. In: Grondin S editor. Psychology of time. Bingley: Emerald Group Publishing; p Rammsayer T. Effects of practice and signal energy on duration discrimination of brief auditory intervals. Percept Psychophys 1994; 55: Roberts S. Cross-modal use of an internal clock. J Exp Psychol Anim Behav Process 1982; 8: Schutz M, Lipscomb S. Hearing gestures, seeing music: Vision influences perceived tone duration. Perception 2007; 36: Théroux JF. Transfert intermodal (visuel/auditif - auditif/visuel) de l apprentissage de la discrimination de courts intervalles de temps [Unpublished Master Thesis]. Québec: Université Laval; Ulrich R, Nitschke J, Rammsayer T. Crossmodal temporal discrimination: Assessing the predictions of a general pacemaker-counter model. Percept Psychophys 2006; 68 (7): Van Wassenhove V, Buonomano DV, Shimojo S, Shams L. Distortion of subjective time perception within and across senses. PLoS ONE 2008; 3 (1): e1437. Warm JS, Stutz RM, Vassolo PA. Intermodal transfer in temporal discrimination. Percept Psychophys 1975; 18: Westheimer G. Discrimination of short time intervals by human observers. Exp Brain Res 1999; 129: Wright BA, Buonomano DV, Manhcke HW, Merzenich MM. Learning and generalization of auditory temporalinterval discrimination in humans. J Neurosci 1997; 17: