Experimental Brain Research 9 Springer-Verlag 1992

Size: px

Start display at page:

Download "Experimental Brain Research 9 Springer-Verlag 1992"

Coral Heath
5 years ago
Views:

Exp Brain Res (1992) 91:484-488 Experimental Brain Research 9 Springer-Verlag 1992 Integration of multiple sensory modalities in cat cortex Mark T. Wallace t, M. Alex Meredith 2, and Barry E.

1 Exp Brain Res (1992) 91: Experimental Brain Research 9 Springer-Verlag 1992 Integration of multiple sensory modalities in cat cortex Mark T. Wallace t, M. Alex Meredith 2, and Barry E. Stein 1 1 Department of Physiology and 2 Department of Anatomy, Medical College of Virginia, Virginia Commonwealth University, Richmond, VA , USA Received March 10, 1992 / Accepted June 11, 1992 Summary. The results of this study show that the different receptive fields of multisensory neurons in the cortex of the cat anterior ectosylvian sulcus (AES) were in spatial register, and it is this register that determined the manner in which these neurons integrated multiple sensory stimuli. The functional properties of multisensory neurons in AES cortex bore fundamental similarities to those in other cortical and subcortical structures. These constancies in the principles of multisensory integration are likely to provide a basis for spatial coherence in information processing throughout the nervous system. Key words: Anterior ectosylvian sulcus - Multisensory integration - Visual - Somatosensory - Auditory Introduction Higher organisms deal remarkably well with the task of integrating information from the different sensory modalities. Indeed, a great deal of information has been generated concerning the perceptual effects of combining different sensory stimuli (Welch and Warren 1986). For this integration to occur, there must be sites at which information from different sensory modalities converges. Many such sites exist in the vertebrate brain (e.g., see Gordon 1973; Drager and Hubel 1975; Hartline 1984; Stein 1984; Meredith and Stein 1986a). Presumably, however, such "higher order" functions as perception depend on integration in cortical neurons. Here, too, multisensory convergence is common (see Albe-Fessard and Gillett 1961 ; Dubner and Rutledge 1964; Jones and Powell 1970; Pandya and Seltzer 1982). Unfortunately, very little is known about how sensory inputs are integrated in cortical neurons, or how this multisensory integration might ultimately contribute to perception. Correspondence to: M. Wallace In the cat, the region of cortex surrounding the anterior ectosylvian sulcus (AES) has been described as a "polysensory" area, where inputs from several sensory modalities converge (Graybiel 1972; Clemo and Stein 1982; Roda and Reinoso-Suarez 1983; Reinoso-Suarez and Roda 1985; Olson and Graybiel 1987; Meredith and Clemo 1989). Recent studies have shown that the AES is comprised of three modality-specific regions: a visual area which lies on the ventral bank of the sulcus, the anterior ectosylvian visual area (AEV; Mucke et al. 1982; Olson and Graybiel 1987); a somatosensory area which lies toward the rostral pole of the dorsal bank of the AES, the fourth somatosensory cortex (SIV; Clemo and Stein 1982); and an auditory area in the caudal regions of the dorsal bank, designated Field AES (Clarey and Irvine 1986). Near the borders of these unimodal regions are many neurons that respond to more than one sensory modality (i.e., multisensory; Clemo et al. 1991). The purpose of the present study was to explore how AES neurons deal with combinations of sensory cues from different modalities. Preliminary results of this work have appeared in abstract form (Meredith et al. 1991). Materials and methods The procedures used here were similar to those described in detail by Meredith and Stein (1986a, b). Briefly, a recording well/headholding device was implanted over the AES in two deeply anesthetized cats (pentobarbital sodium 40 mg/kg i.p.), and animals recovered over 7-10 days. For recording, animals were anesthetized (halothane %) and paralyzed (pancuronium bromide 2 mg/kg per hour). The level of anesthesia was routinely monitored by assessing corneal reflexes during periods in which the animal was allowed to recover from paralysis. Once a neuron was isolated and the effective modalities determined, the receptive fields were mapped as follows: visual receptive fields were mapped using bars or spots of light from a pantoscope projected onto a translucent hemisphere; somatosensory receptive fields were mapped using a camel's hair brush or calibrated on Frey hairs; auditory receptive fields were mapped using broad-band noise bursts from a hoop-

2 485 mounted speaker. Quantitative sensory tests were conducted using computer-controlled stimuli, which included: deflections of the hair or skin using a modified Ling 102A shaker (somatosensory); broad-band noise bursts (auditory) ; and galvanometer-driven spots or bars of light (visual). Either these stimuli were delivered alone (e.g., visual only, auditory only) as a "single-modality test" or stimuli from two modalities were presented together (e.g., visual and auditory) in a "combined-modality test," using the temporal criteria of Meredith et al. (1987). Single- and combined-modality tests were conducted in an interleaved manner, with long (12-15 s) interstimulus intervals. The number of impulses evoked during single- and combined-modality tests were compared, and a multisensory interaction was defined as a significant difference (P<0.05, t-test; increase represents response enhancement, decrease response depression) in the number of impulses elicited by the combinedmodality test when compared with that evoked by the most effective single-modality stimulus. The magnitude of a multisensory interaction was determined by the following formula: (CM-SMm,x/SMm,x) x 100 = percentage interaction, where CM is the response to the combined-modality stimuli and SMmax is the response to the most effective single-modality stimulus. Following a recording session, animals were allowed to recover from paralysis and anesthesia was discontinued. After the restoration of normal respiration and locomotion, they were returned to their home cage. At the end of the final recording session, the animals were overdosed with barbiturate and perfused intracardially with 10% formalin. Standard reconstruction techniques were used to determine the positions of recorded neurons. Results Neurons were recorded throughout the cortex surrounding the AES. A substantial proportion (76:351, 22%) ~ m AES - Unimodal 9 Multisensory, \ ~ -- 9 :S l i p VA U Mu, l ' I i I i 5 mm +/53 Fig. 1. The distribution of unimodal (dashes) and multisensory (filled circles) neurons in the anterior ectosylvian sulcus (AES) cortex of one cat. Neurons are labeled by modality and modality combination (A, auditory, S, somatosensory, V, visual, U, unre- sponsive). The lateral view of the cortex (upper left) shows the AP level from which each section was taken, and the modality distribution of all neurons studied is shown in the pie chart at the lower right

486 A. Receptive field registry Auditory- Somotosensory o o Somofosensory-Auditory Auditory-Visual 90 270 o ~, 1 3 5 ~ ~ 18o* B. Multisensory.

3 486 A. Receptive field registry Auditory- Somotosensory o o Somofosensory-Auditory Auditory-Visual o ~, ~ ~ 18o* B. Multisensory. integration [7 [] Response Enhancement 9 I[ 9 R:sponse Depression -I00 5 *6 E Fig. 2A, B. Receptive field register and multisensory integration in three bimodal AES neurons. A The horizontal plane of visual and auditory space and the dorsal body surface (shading delimits receptive fields: diagonal, auditory; horizontal, somatosensory; vertical, visual). Left and middle, two neurons whose auditory receptive fields encompassed contralateral space and extended into the caudal half of ipsilateral space, closely paralleling their somatosensory fields. A similar receptive field register is seen in the auditoryvisual neuron at the right. B schematics in A are shown in perspective to admit a third dimension: the magnitude of a multisensory interaction (it is scaled against the axis at the far right). Left, a somatosensory stimulus was presented at the same location (S) in all trials, while in different trials the auditory stimulus was positioned at 45 ~ intervals (bars). When auditory and somatosensory stimuli were both within their receptive fields, response enhancement always resulted (white bars). However, when the auditory stimulus was outside its receptive field (at 315~ response depression resulted (downward black bar). Similar results are shown in a neuron (middle) in which the auditory stimulus was kept at one location (A) while the somatosensory stimulus varied (e.g., forelimb, hindlimb), and in an auditory-visual neuron (right; V, indicates location of visual stimulus). Statistically significant multisensory interactions (two-tailed t-test; P<0.05) are denoted by an asterisk -5O of these neurons responded to, or were influenced by, stimuli from two or more modalities (Fig. 1). These multisensory neurons were found primarily in the region of the sulcal fundus and in the caudal aspect of the AES cortex (Fig. 1), and they were most prevalent at the borders between unimodal regions. Generally, the modalities of multisensory neurons reflected the modality types found within the adjacent unimodal regions. Thus, visual auditory neurons were most often encountered along the border between AEV and Field AES; auditory - somatosensory neurons were generally found at the border between Field AES and SIV, etc. For the most part, the response properties of multisensory AES neurons were indistinguishable from those of their unimodal neighbors. Both types of neurons responded best to rapid, low-amplitude displacements of the skin or hair (somatosensory), brief ( ms) broadband noise bursts (auditory), and/or small (<4 ~ ) bars or spots of light moving at slow-intermediate velocities (< 100~ in a temporal to nasal direction (visual). The individual unimodal receptive fields of multisensory neurons exhibited a spatial register that was unexpected, given the lack of obvious spatiotopic organization in the unimodal visual (i.e., AEV) and auditory (i.e., Field AES) regions of AES cortex. For the auditory - somatosensory neurons depicted in Fig. 2A (left and center), the azimuthal extent of the auditory receptive fields closely matched the extent of the somatosensory receptive fields: both encompassed all or most of contralateral space and extended into the caudal aspects of ipsilateral space. This same register is shown in an auditory - visual neuron (Fig. 2A, right). In this neuron, the azimuthal extent of the auditory receptive field entirely overlapped that of the smaller visual receptive field. A similar register of receptive fields was seen in nearly every multisensory cell encountered. The possible functional impact of this receptive field overlap on cortical multisensory integration was examined by presenting pairs of stimuli from different modalities at various locations. In the examples illustrated in Fig. 2 B one stimulus (i.e., the "test" stimulus) remained at the same position within its receptive field while the second ("modulating") stimulus was presented at a number of different positions, both within and outside

487 of its receptive field. Neuronal responses to the test stimulus were significantly enhanced when the modulating stimulus was presented concurrently within its receptive field.

4 487 of its receptive field. Neuronal responses to the test stimulus were significantly enhanced when the modulating stimulus was presented concurrently within its receptive field. When the modulating stimulus was presented outside its receptive field, the enhancement it provided was lost or replaced by depression (Fig. 2B). This spatial effect was observed in each of the multisensory categories encountered: visual auditory (n = 6), visual - somatosensory (n = 2), and auditory- somatosensory (n = 10). Discussion The results illustrate a characteristic organizational feature of multisensory neurons in cat AES cortex: their individual receptive fields are in spatial register. This finding was surprising given the absence of global spatiotopic auditory and visual representations in the AES, but it is a finding quite consistent with the overlapping pattern of receptive fields found in multisensory superior colliculus neurons (Stein et al. 1976). Indeed, receptive field overlap in multisensory neurons appears to be a general feature of the vertebrate brain. Such overlap is found in the midbrain of both mammalian and nonmammalian species (Knudsen 1982; Hartline 1984; Stein 1984), as well as in various polysensory regions of primate cortex (Bruce et al. 1981; Duhamel et al. 1989; Watanabe and Iwai 1991 ; Stein et al. 1992). The functional importance of this spatial organization becomes apparent when examining the way in which a multisensory neuron responds to combinations of sensory stimuli. When two stimuli originate from the same point in space, they will fall within the individual receptive fields of a given multisensory neuron. In this case, the combination of stimuli elicits a neuronal response far greater than that of either of the stimuli independently. If, however, one of those stimuli is separated from the other so that it falls outside its receptive field, it no longer enhances the neuron's response. In fact, it may depress the neuron's response to the other stimulus. Consequently, multiple sensory cues initiated by the same event are likely to increase the salience of that event because both stimuli fall within their receptive fields. On the other hand, the myriad ongoing unrelated events derived from spatially disparate locations will have no such effect. They may, in fact, inhibit one another, thereby relegating their presence into a less-evident sensory "background." The present data are very similar to those obtained from superior colliculus neurons in the cat, where the spatial relationship among multisensory stimuli has been shown to be a critical factor in enhancing or depressing neuronal activity (Meredith and Stein 1986b), as well as influencing superiorcolliculus-mediated orientation behaviors (Stein et al. 1989). The present observations suggest that the same spatial organization characterizes multisensory receptive fields in different structures, thereby forming a common foundation for integrating multisensory information. That this organization extends to humans as well is indicated by the observation that spatially coincident visual and auditory cues pro- duce a marked enhancement of event-related potentials and a decrease in reaction time (Costin et al. 1991). Thus, it appears that the same spatial determinants of multisensory integration that subserve the generation of immediate orientation behaviors (such as those mediated by the superior colliculus) also underlie the modulation of "higher order" cortical processes (such as those leading to perception and cognition). Ultimately, the constancy of the spatial principles of multisensory integration among structures and functions is likely to provide the basis for coherence in the processing of multisensory information throughout the vertebrate nervous system. Acknowledgements. We would like to thank Nancy London for her technical assistance. This work was supported by NIH grants NS and NS References Albe-Fessard D, Gillett E (1961) Convergences d'afferences d'origines corticale et peripherique vers le centre median du chat anesthesie ou eveille. Electroencephalogr Clin Neurophysiol 13 : Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of monkey. J Neurophysiol 46: Clarey JC, Irvine DRF (i986) Auditory response properties of neurons in the anterior ectosylvian sulcus of the cat. Brain Res 386:12-19 Clemo HR, Stein BE (1982) Somatosensory cortex: a 'new' somatotopic representation. Brain Res 235: Clemo HR, Meredith MA, Wallace MT, Stein BE (1991) Is the cortex of cat anterior ectosylvian sulcus a polysensory area? Soc Neurosci Abstr 17 : 1585 Costin D, Neville HJ, Meredith MA, Stein BE (1991) Rules of multisensory integration and attention: ERP and behavioral evidence in man. Soc Neurosci Abstr 17:656 Drager UC, Hubel DH (1975) Responses to visual stimulation and relationship between visual, auditory and somatosensory inputs to the mouse superior colliculus. J Neurophysiol 38: Dubner R, Rutledge LT (1964) Recording and analysis of converging input upon neurons in cat association cortex. J Neurophysiol 27 : 62~634 Duhamel J-R, Colby CL, Goldberg ME (1989) Congruent visual and somatosensory response properties of neurons in the ventral intraparietal area (VIP) in the alert monkey. Soc Neurosci Abstr 15 : 162 Gordon BG (1973) Receptive fields in the deep layers of the cat superior colliculus. J Neurophysiol 36: Graybiel AM (1972) Some ascending connections of the pulvinar and nucleus lateralis posterior of the thalamus in the cat. Brain Res 44: Hartline PH (1984) The optic tectum of reptiles: neurophysiological studies. In: Vanegas H (ed) Comparative neurology of the optic tectum. Plenum Press, New York, pp Jones EG, Powell TPS (1970) An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 93: Knudsen EI (1982) Auditory and visual maps of space in the optic tectum of the owl. J Neurosci 2: Meredith MA, Clemo HR (1989) Auditory cortical projection from the anterior ectosylvian sulcus (field AES) to the superior colliculus in cat: an anatomical and electrophysiological study. J Comp Neurol 289: Meredith MA, Stein BE (1986a) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:

5 488 Meredith MA, Stein BE (1986b) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365: Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons: I. Temporal factors. J Neurosci 7: Meredith MA, Wallace MT, Stein BE (1991) Integrating the different senses in neurons from cat association cortex (anterior ectosylvian sulcus). Soc Neurosci Abstr 17 : 1585 Mucke L, Norita M, Benedek G, Creutzfeldt O (1982) Physiologic and anatomic investigation of a visual cortical area situated in the ventral bank of the anterior ectosylvian sulcus of the cat. Exp Brain Res 179:1-11 Olson CR, Graybiel AM (1987) Ectosylvian visual area of the cat: location, retinotopic organization, and connections. J Comp Neurol 261: Pandya DN, Seltzer B (1982) Association areas of cerebral cortex. Trends Neurosci 5 : Reinoso-Suarez F, Roda JM (1985) Topographic organization of the cortical afferent connections to the cortex of the anterior ectosylvian sulcus in the cat. Exp Brain Res 59: Roda JM, Reinoso-Suarez F (1983) Topographical organization of the thalamic projections to the cortex of the anterior extosylvian sulcus in the cat. Exp Brain Res 49: Stein BE (1984) Multimodal representation in the superior colliculus and optic rectum. In: Vanegas H (ed) Comparative neurology of the optic rectum. Plenum Press, New York, pp Stein BE, Magalhaes-Castro B, Kruger L (1976) Relationship between visual and tactile representations in cat superior colliculus. J Neurophysiol 39: Stein BE, Meredith MA, Huneycutt WS, McDade L (1989) Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J Cogn Neurosci 1 : Stein BE, Meredith MA, Wallace MT (1992) Nonvisual responses of visually-responsive neurons. Prog Brain Res (in press) Watanabe J, Iwai E (1991) Neuronal activity in visual, auditory and polysensory areas in the monkey temporal cortex during visual fixation task. Brain Res Bull 26: Welch RB, Warren DH (1986) Intersensory interactions. In: Boff KR, Kaufman L, Thomas JP (eds) Handbook of perception and human performance, vol I. Sensory processes in perception. Wiley, New York, pp 1-36

Comparisons of cross-modality integration in midbrain and cortex

M. Norita, T. Bando and B. Stein (Eds.) hgms in Bmin Rrsrarrh, Vol 112 8 1996 Elsevier Sciencc BV. All rights reserved. CHAPTER 20 Comparisons of cross-modality integration in midbrain and cortex Barry