A Neural Signature of Divisive Normalization at the Level of Multisensory Integration in Primate Cortex

Transcription

1 Article A Neural Signature of Divisive Normalization at the Level of Multisensory Integration in Primate Cortex Highlights d Many neurons in macaque area MSTd show a specific form of cross-modal suppression Authors Tomokazu Ohshiro, Dora E. Angelaki, Gregory C. DeAngelis d d d Cross-modal suppression is predicted by a divisive normalization mechanism Neurons in area MT, which project to MSTd, do not show cross-modal suppression Multisensory integration in area MSTd likely involves divisive normalization Correspondence gdeangelis@cvs.rochester.edu In Brief The mechanisms of multisensory integration by single neurons remain unclear. Previous work suggested that divisive normalization could explain many multisensory response properties. Ohshiro et al. demonstrate that multisensory responses in area MSTd follow a critical prediction of the normalization model. Ohshiro et al., 217, Neuron 95, July 19, 217 ª 217 Elsevier Inc.

2 Neuron Article A Neural Signature of Divisive Normalization at the Level of Multisensory Integration in Primate Cortex Tomokazu Ohshiro, 1,2 Dora E. Angelaki, 3 and Gregory C. DeAngelis 1,4, * 1 Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14611, USA 2 Department of Physiology, Tohoku University School of Medicine, Sendai , Japan 3 Department of Neuroscience, Baylor College of Medicine, Houston, TX 773, USA 4 Lead Contact *Correspondence: gdeangelis@cvs.rochester.edu SUMMARY Studies of multisensory integration by single neurons have traditionally emphasized empirical principles that describe nonlinear interactions between inputs from two sensory modalities. We previously proposed that many of these empirical principles could be explained by a divisive normalization mechanism operating in brain regions where multisensory integration occurs. This normalization model makes a critical diagnostic prediction: a non-preferred sensory input from one modality, which activates the neuron on its own, should suppress the response to a preferred input from another modality. We tested this prediction by recording from neurons in macaque area MSTd that integrate visual and vestibular cues regarding self-motion. We show that many MSTd neurons exhibit the diagnostic form of crossmodal suppression, whereas unisensory neurons in area MT do not. The normalization model also fits population responses better than a model based on subtractive inhibition. These findings provide strong support for a divisive normalization mechanism in multisensory integration. INTRODUCTION We are often faced with situations in which multiple sources of information need to be combined to make good decisions. Multi-sensory integration provides an attractive model system in which to explore the neural mechanisms by which multiple sensory signals are combined to make perceptual decisions (Alais et al., 21; Ernst and B ulthoff, 24; Raposo et al., 212; Spence, 21, 211). Despite many empirical studies (Driver and Noesselt, 28; Stein and Stanford, 28), a detailed understanding of the neural computations underlying multisensory integration remains elusive. A diverse array of empirical observations has been usefully characterized as a set of empirical principles of multisensory integration (Meredith and Stein, 1986; Stein and Stanford, 28). For example, multisensory neurons in cat superior colliculus often show super-additive integration of weak visual and auditory inputs, whereas they show additive or sub-additive summation for stronger inputs; this is known as the principle of inverse effectiveness (Meredith and Stein, 1986; Perrault et al., 23, 25; Stanford et al., 25; Wallace et al., 1996). As another example, the spatial/temporal principle holds that multisensory stimuli need to be congruent in space and time for cross-modal enhancement of neural responses to occur; otherwise, cross-modal suppression may occur, a phenomenon in which the multisensory response of a neuron is less than its response to the stronger unisensory input (Kadunce et al., 1997; Meredith et al., 1987; Meredith and Stein, 1986; Wallace et al., 1996). These empirical principles generally involve nonlinear interactions between inputs from different modalities, but experimental evidence that can distinguish among different possible types of nonlinearities has remained very limited. We previously proposed a divisive normalization model of multisensory integration that accounts naturally for many of the key empirical principles of multisensory integration (Ohshiro et al., 211). The model is based on a few well-established neural operations: linear summation of inputs (Ferster, 1994; Jagadeesh et al., 1993; Skaliora et al., 24), a power-low non-linearity that may characterize the transformation between neural inputs and outputs (Carandini and Ferster, 2; Carandini et al., 1997; Heeger, 1992a; Miller and Troyer, 22; Priebe and Ferster, 28), and divisive normalization among neurons within a brain region (Carandini and Heeger, 211; Heeger, 1992b). Divisive normalization has been suggested to account for nonlinear aspects of stimulus interactions in V1 (Busse et al., 29), motion integration in the middle temporal (MT) area (Britten and Heuer, 1999), value representations in the lateral intraparietal cortex (area LIP) (Louie et al., 211), attentional modulation in visual cortex (Lee and Maunsell, 29; Reynolds and Heeger, 29), and stimulus interactions in olfactory processing (Olsen et al., 21). Thus, normalization may be a canonical neural computation (Carandini and Heeger, 211). We showed previously that a divisive normalization model of multisensory integration makes a critical prediction that can distinguish it from other models: specifically, a non-optimal stimulus for one modality, which activates a neuron when presented Neuron 95, , July 19, 217 ª 217 Elsevier Inc. 399

3 A Figure 1. Multisensory Normalization Model and the Diagnostic Prediction of Crossmodal Suppression (A) Schematic illustration of the divisive normalization model for visual-vestibular cue integration. Model MSTd neurons (rectangles) perform a weighted sum of heading-tuned inputs from unisensory (vestibular, visual) neurons, with weights given by d vest, d vis. The weighted sum is raised to an exponent and gain-modulated multiplicatively by a function of the total activity of the population of neurons (see STAR Methods). (B) The diagnostic prediction of the normalization model. When visual and vestibular stimuli are presented at the heading preferences of the model neuron (D = ), the combined response (black) exceeds the single cue responses (cyan, red) for all stimulus amplitudes (left). When the heading of cue2 is offset from the cell s heading preference by D =6 B C (middle), cue2 is activating on its own but suppresses the combined response below that of cue1 (middle). For a larger offset (D = 12 ), cue2 becomes suppressive on its own and suppression of the combined response is trivial (right). Motion amplitude indicates the total displacement of the body within a trial. (C) Summary of predictions of the normalization model (circles) and a family of alternative models involving subtractive inhibition (triangles). Responses of model neurons were simulated for heading offsets of cue2 (D) ranging from to 18, in steps of 15. The response to cue2 (R cue2 ) and the combined response (R comb ) are divided by the response to cue1 (R cue1 ), and the ratios (R cue2 /R cue1, R comb /R cue1 ) are plotted against each other. R cue1, R cue2, and R comb denote simulated responses to the largest stimulus amplitude. alone, should be able to suppress the response to a near-optimal stimulus of the other modality when the cues are combined (Ohshiro et al., 211). Here, we provide a direct test of this prediction by recording from multisensory neurons in the dorsal medial superior temporal (MSTd) area that combine visual and vestibular signals regarding self-motion (Duffy, 1998; Gu et al., 26). Neurons in area MSTd have been previously linked to perceptual integration of visual and vestibular heading cues (Gu et al., 28) and perceptual weighting of these cues according to their reliability (Fetsch et al., 211). MSTd has also been causally linked to perception of heading based on visual and vestibular cues (Britten and Van Wezel, 22; Gu et al., 27, 212). Here, we show that responses of MSTd neurons fulfill the critical prediction of the normalization model with regard to cross-modal suppression. We also demonstrate that the normalization model, fit simultaneously to data from our entire population of MSTd neurons, accounts better for the data than an alternative model based on subtractive inhibition. If cross-modal suppression is a diagnostic feature of a multisensory stage of normalization, then it should not be seen in relevant unisensory areas. MSTd receives its primary visual input from the adjacent MT area (Born and Bradley, 25; Maunsell and van Essen, 1983; Ungerleider and Desimone, 1986), and MT neurons are not thought to carry vestibular signals (Chowdhury et al., 29; Smith et al., 212). Thus, we also probed for the existence of cross-modal suppression in area MT and did not find any evidence for it. Together, our findings provide strong experimental support for the proposal that divisive normalization, operating at the level of multisensory integration, can account for many of the empirical principles of multisensory integration exhibited by single neurons (Ohshiro et al., 211). RESULTS Cross-modal Suppression as Predicted by the Model The divisive normalization model of visual-vestibular cue integration (Figure 1A) assumes that MSTd neurons receive heading-selective vestibular and visual (optic flow) inputs, with heading preferences that may be matched or mismatched to varying degrees (Gu et al., 26). Each neuron performs a weighted linear sum of its visual and vestibular inputs, with weights d vest and d vis that are fixed for that neuron. The result is then passed through a powerlaw non-linearity, after which its output gain is multiplicatively modulated by a function of the pooled spiking activity of other neurons in the population ( feedback normalization ; Heeger, 1992b; see Discussion). Note that the semi-saturation constant in the model (a in Equations 5, 6, and 7) produces responses that rise gradually and saturate as a function of stimulus intensity. Note also that, while this type of normalization model can produce averaging behavior under some circumstances (e.g., Busse et al., 29), it more generally produces a range of possible interactions depending on the relative strengths of the stimuli and input weights. 4 Neuron 95, , July 19, 217

4 More specifically, simulated responses of a typical model neuron show cross-modal enhancement when both cues are presented together at the visual and vestibular heading preferences of the neuron (Figure 1B, left). However, the same neuron shows a paradoxical cross-modal suppression when one of the two cues (cue2) is presented at a non-preferred heading (Figure 1B, middle): the non-preferred cue2 activates the model neuron when presented alone, but suppresses responses when presented together with the preferred cue1. This diagnostic crossmodal suppression only occurs at large stimulus intensities and only within a narrow heading range for the non-preferred cue. If the heading of cue2 is further from the cell s heading preference, then this non-optimal input can become suppressive on its own (Figure 1B, right). In such a case, cross-modal suppression is trivially explained by summation of an activating input with a suppressive input. Thus, to test for operation of divisive normalization at the level of multisensory integration, we seek evidence of this narrow regime in which the non-optimal input drives a response on its own but induces cross-modal suppression. To summarize these relationships graphically, we plot the combined response (R comb ) to both cues against the response to the non-preferred cue (R cue2 ), with both values normalized by the response to the preferred cue (R cue1 ). In this representation (Figure 1C), values above and below unity on the ordinate indicate cross-modal enhancement and suppression, respectively, and values greater than zero on the abscissa indicate that the non-preferred input activates the unit above baseline response. Thus, the critical prediction of multisensory normalization is that data should fall within the lower-right quadrant for some offsets (D) of cue2 from the preferred heading for that modality. Indeed, predicted responses of the model neuron move from the top-right quadrant through the lower-right quadrant as D is increased (Figure 1C, circles). Eventually, as D is further increased, data points enter the lower-left quadrant, corresponding to the case of trivial cross-modal suppression. Ursino et al. (29) proposed an alternative model of multisensory integration that accounts for the principle of inverse effectiveness and other key properties of multisensory integration in the superior colliculus (Cuppini et al., 21). This alternative model, which is based on subtractive inhibition, makes a distinct prediction from the normalization model with regard to cross-modal suppression (Ohshiro et al., 211). For the alternative (subtractive) model (Figure 1C, triangles), cross-modal suppression only occurs when the non-preferred cue is suppressive on its own. As a result, the predictions of this class of model do not pass through the lower-right quadrant of Figure 1C. We further demonstrate mathematically that this class of alternative model, in which the multisensory input to each neuron is subtractively modulated by the pooled spiking activity of other neurons, does not predict non-trivial cross-modal suppression under a realistic set of assumptions (see STAR Methods). Therefore, cross-modal suppression by an activating non-preferred input is a diagnostic indicator of multisensory divisive normalization. Cross-modal Suppression in Multisensory MSTd Neurons We sought to demonstrate the diagnostic form of cross-modal suppression by recording from MSTd neurons in macaques that were trained to fixate a visual target while multisensory (visual/vestibular) stimuli were presented using a projector mounted on a motion platform (see STAR Methods for details). We recorded from a total of 165 MSTd neurons in two animals; among these, we obtained sufficient data (at least five repetitions of each stimulus) from a group of 11 neurons (42 from monkey A and 59 from monkey B), which forms the main database for this study. Of the remaining 64 neurons, isolation was lost for 55 neurons before the protocol was completed, and 9 neurons were discarded because they showed no heading selectivity for visual or vestibular cues. Among the 11 MSTd cells studied in detail, 68 were multisensory neurons (26 from monkey A and 42 from monkey B) that showed clear heading selectivity for both visual and vestibular cues, as well as firing rates that increased with stimulus amplitude. We first examine cross-modal suppression in these multisensory neurons. The remaining 33/11 cells (16 from monkey A and 17 from monkey B) were unisensory neurons that responded reliably only to optic flow. These neurons are analyzed separately in a later section. Once an action potential was isolated, we first characterized the 3D heading selectivity of the neuron (Gu et al., 26) and determined its heading preferences for each modality (Figure 2A, white crosses). For most multisensory neurons, we used a search protocol (Figure 2B) to estimate the range of headings over which a non-preferred input might elicit cross-modal suppression while activating the neuron on its own. The heading for one modality (visual in this example) was offset by 3 steps in elevation from its preferred heading (white dots in Figure 2A) while the heading of the other modality (vestibular) was fixed at the preferred value. As expected, visual responses of this neuron peaked at zero offset (D = ) and declined with the absolute value of the offset, jdj (Figure 2B, red). The combined response (black) was suppressed below the vestibular response (cyan) for jdj >3. This cross-modal suppression is of the trivial form for jdj >9 because visual responses are below baseline activity (gray) for this range of D, suggesting that the visual cue becomes suppressive. Critically, within a narrow range of D values around ± 6, this neuron shows cross-modal suppression along with significant visual activation, consistent with the diagnostic prediction of the divisive normalization model. To quantify these effects in detail, we tested each neuron with a range of stimulus intensities and a substantial number of stimulus repetitions (5 15; median, 8). Stimulus intensity was manipulated by varying the amplitude of self-motion, which followed a Gaussian velocity profile (see STAR Methods; Figure S1). Each neuron was tested with stimuli having D =, as well as at least one non-zero D value chosen to probe for cross-modal suppression (many cells were tested with multiple non-zero values of D; see STAR Methods). For the example neuron, vestibular and visual responses rose monotonically with self-motion amplitude for D =, and combined responses exceeded unisensory responses for all amplitudes (Figure 2C, left). Offsetting the visual heading by D = +6 (Figure 2C, middle) greatly reduced visual responses, but these responses were still significantly greater than baseline activity for each non-zero motion amplitude (p < for each amplitude, Wilcoxon rank-sum tests). Critically, at this offset, the combined response was significantly suppressed below the vestibular response for the two largest Neuron 95, , July 19,

5 A C B Figure 2. Data from an Example Multisensory MSTd Neuron (A) Three-dimensional heading tuning functions (left, vestibular; right, visual) are shown as color contour maps. This neuron prefers rightward self-motion ( 3 azimuth, elevation; white crosses) for both modalities. White dots indicate visual headings that were tested in the screening protocol. (B) Screening test. Combined (black), vestibular (cyan), and visual (red) responses are plotted as a function of the offset (D ) of the visual heading from the cell s heading preference. The heading tuning curves were fit with a modified sinusoid (Equation 3). Motion amplitude = 1. cm. Error bars denote SEM. (C) Responses are plotted as a function of stimulus amplitude. Data are shown for D = (left), +6 (middle), and 6 (right). Asterisks indicate significant suppression or activation (**p <.1; Wilcoxon rank-sum test). Smooth curves show the hyperbolic-ratio functions (Equation 1) that best fit the amplitude-response curves. Error bars indicate SEM. stimulus amplitudes (p < for amplitudes of 4.4 and 11.8 cm, Wilcoxon rank-sum tests). Very similar observations were made for D = 6 (Figure 2C, right). These results demonstrate clearly that a non-optimally presented cue, which activates the neuron when presented alone, can suppress the response to an optimal stimulus of the other modality, as predicted by the divisive normalization model. Data from a few additional example neurons are illustrated in Figure 3. These include another neuron for which a non-preferred visual stimulus suppressed the combined response below the level of the vestibular response (Figure 3A), as well as two additional neurons for which the vestibular stimulus was offset from the preferred heading and caused a suppression of the visual response (Figures 3B and 3C). Note that the example neurons in Figures 3A and 3C had mismatched visual and vestibular heading preferences ( opposite cells ; Gu et al., 26). For such neurons, a combined stimulus with the preferred heading for each modality has different directions of motion for visual and vestibular cues. These examples illustrate that non-preferred stimuli of either modality can elicit cross-modal suppression while still producing responses on their own. This symmetry of the interaction between vestibular and visual inputs is expected from linear summation of tuned unisensory inputs, combined with multisensory normalization. Population Summary All 68 multisensory neurons were tested with D = and with either the visual or the vestibular stimulus at a non-preferred heading (non-zero D), and eight neurons were tested with multiple non-preferred headings. For nine neurons, an additional block of trials was run in which the other sensory modality was set to a non-preferred heading. In total, 86 datasets were collected with preferred headings for both modalities, and 97 datasets were collected with a combination of preferred and nonpreferred headings. To summarize our findings and pool data across conditions, we defined cue1 to be the cue that was held fixed at the neuron s preferred heading and cue2 to be the cue for which heading was varied from preferred to nonpreferred directions. To quantify cross-modal suppression, we fit the amplituderesponse curves for each neuron with a sigmoidal function (see STAR Methods; Equation 1), and we quantified the strength of activity using the response gain parameter (G). These fits provided an excellent description of the data (e.g., Figures 4A and 4B), with a median R 2 value of.97 across the population. We adopt a notation in which G(cue1, cue2) indicates the response gain for a particular combination of cue1 and cue2, while p and n denote whether each cue was preferred or non-preferred. Thus, G(p,p) represents the response gain when both cue1 and cue2 have preferred directions, G(p,n) denotes gain for a preferred cue1 and a nonpreferred cue 2, G(p,) indicates gain for a unisensory preferred cue1 stimulus, and G(,p) or G(,n) represent gains for unisensory stimuli with preferred or non-preferred headings for cue 2, respectively. We then computed the ratio of the combined response gain (G(p,p)or G(p,n)) to that of the modality presented at the preferred heading (G(p,)). This ratio indicates whether combining the two cues produces cross-modal enhancement (ratio > 1) or cross-modal suppression (ratio < 1). Similarly, the ratio of the gain parameter for cue2 (G(,p) or G(,n)) to G(p,) indicates whether cue2 was activating (ratio > ) or suppressive (ratio < ). These gain ratios were compared in a scatterplot, analogous to Figure 1C. For the example neuron of Figure 4, data obtained with stimuli at the preferred heading values for both modalities map onto a point in the upper-right quadrant, indicating cross-modal enhancement (Figure 4C, filled purple star). In contrast, data obtained with one cue at a non-preferred heading (D =9 ) map to a point in the lower-right quadrant (filled green star), the critical 42 Neuron 95, , July 19, 217

6 A A B B C C Figure 3. Demonstration of Cross-modal Suppression for Three Additional Example Neurons Format as in Figure 2C. (A) A second example neuron showing cross-modal suppression by a nonpreferred visual heading stimulus. Data are shown for D = (left) and D =8 (right). Note that this neuron is an opposite cell; thus, the stimulus headings for D = have different directions. (B and C) Two example neurons showing cross-modal suppression by a nonpreferred vestibular stimulus. The non-zero vestibular offsets were D =6 (B, right) and D = 18 (C, right). Asterisks indicate significant suppression or activation (*p <.5; **p <.1; Wilcoxon rank-sum test). Smooth curves show the best-fitting hyperbolic-ratio functions. Error bars indicate SEM. region in which cross-modal suppression is produced by a nonpreferred stimulus that activates the neuron on its own. Overall, population data from multisensory MSTd neurons provide strong support for the predictions of the normalization model. For combined stimuli with preferred heading values for both modalities, most data points (72/86) reside in the upperright quadrant, corresponding to cross-modal enhancement (Figure 4C, purple). In stark contrast, many of the data points (55/97) corresponding to combinations of preferred and nonpreferred headings for the two modalities reside in the lowerright quadrant, thus showing cross-modal suppression combined with activation by the non-preferred stimulus. Among these 55 data points in the lower-right quadrant, 25 cases (filled green symbols) show both significant cross-modal suppression (G(p,n)/G(p,) < 1; p <.5, bootstrap) and significant activation in response to the non-preferred heading stimulus (G(,n)/G(p,) > ; p <.5). These neurons clearly exhibited the diagnostic form of cross-modal suppression that is predicted by divisive normalization acting at the level of multisensory integration. A polynomial fit to the data of Figure 4C (black line) has a y intercept of.85 (SE =.26), which is significantly less than Figure 4. Quantification and Visualization of Cross-modal Suppression Effects (A) Amplitude-response functions for an example MSTd neuron, along with best-fitting hyperbolic-ratio functions (Equation 1). Here, both visual (cue1) and vestibular (cue2) stimuli were presented at the preferred headings, and cross-modal enhancement occurs. G(p,p), both cues at respective preferred headings; G(p,), preferred cue1 only; G(,p), preferred cue2 only. Error bars indicate SEM. (B) Data and fits from the same neuron when the vestibular stimulus (cue2) was offset by 9 from the vestibular heading preference. G(p,n), preferred cue1 and non-preferred cue2; G(,n), non-preferred cue2 only. Error bars indicate SEM. (C) Combined response gains for the preferred-preferred (purple, G(p,p)) and preferred-non-preferred (green, G(p,n)) stimulus combinations are plotted against the corresponding response gains for cue2 (G(,p)or G(,n)). Values on both axes are normalized by the response gain for cue1, G(p,). Multiple data points are plotted for each neuron: one for the preferred-preferred combination (purple) and one or more for preferred-non-preferred heading combinations (green). Filled green symbols represent cases with significant crossmodal suppression and significant activation by the non-preferred cue2. Filled purple symbols indicate the preferred-preferred stimulus combinations that correspond to the filled green symbols. Purple and green stars correspond to the data shown in (A) and (B), respectively. Solid black curve, second-order polynomial fit. Data points for a few cases with outlier values are plotted at the maximum values on the x and y axes. 1. (p = ), indicating a significant shift toward the lower-right quadrant. Nearly identical results were obtained when firing rates were computed within narrower (2 or 4 ms) time windows centered around the peak population response (Figures S2A and S2B), thus indicating that the results are robust to variations in the time window over which responses are measured. We also carried out the same analysis separately for experiments in which the vestibular or the visual modality was the non-preferred stimulus. We find similar effects in both cases (Figures S2C and S2D), although the strength of cross-modal Neuron 95, , July 19,

7 A B firing rate measured at the largest stimulus amplitude in lieu of the gain parameter of the sigmoidal function. This produced nearly identical results (Figure S3A). We also examined the distribution of heading offsets (D) that produced the diagnostic form of cross-modal suppression (Figure S3B). As expected from the model, cells showing significant cross-modal suppression and significant activation by cue2 were generally those tested with D values in a narrow range (6 < jdj <9 ). Overall, data from multisensory MSTd neurons largely conform to predictions of the divisive normalization model. C Figure 5. Cross-modal Suppression Is Exhibited by Unisensory MSTd Neurons, but Not MT Neurons (A) Data from a unisensory (visual only) MSTd neuron, along with best-fitting hyperbolic ratio functions. Format as in Figure 3, except that data are superimposed for two values of D: 135 (solid curves) and 16 (dashed curves). Significant cross-modal suppression occurs for the largest stimulus amplitude (*p <.5; **p <.1; Wilcoxon rank-sum test). Error bars indicate SEM. (B) Data from a typical MT neuron tested with D values of and 9. Smooth curves show the best-fitting hyperbolic-ratio functions. Error bars indicate SEM. (C) The ratio of combined:visual response gains is plotted against the ratio of vestibular:visual response gains. Data are shown for unisensory MSTd neurons (orange symbols; 56 observations from 33 neurons) and MT neurons (green symbols; 127 observations from 43 neurons). Multiple data points may be plotted for each neuron, corresponding to the multiple vestibular headings tested. Filled symbols indicate cases for which G combined /G visual is significantly different from 1.. A histogram of the ratio, G combined /G visual, is shown on the right margin. suppression was greater when the visual cue was non-preferred, which was the case in the minority of our experiments. We have previously shown that multisensory neurons in area MSTd can have visual and vestibular heading preferences that are closely matched ( congruent cells) or grossly mismatched ( opposite cells) (Gu et al., 26, 28). We found that the strength and incidence of cross-modal suppression was similar for both types of neurons (Figures S2E and S2F), thus suggesting that divisive normalization is a common property of MSTd neurons and is not likely related to the functional roles that these different cell types may play in cue integration or cue weighting (Fetsch et al., 211; Gu et al., 28; Morgan et al., 28). To rule out possible biases associated with systematic errors in curve fitting, we also performed similar analyses using the Cross-modal Suppression in Unisensory MSTd Neurons The literature on multisensory integration describes cases in which neurons that appear to be unisensory are suppressed by adding a non-effective stimulus of one modality to an effective stimulus of another modality (Avillac et al., 27; Meredith and Stein, 1986; Sugihara et al., 26). The normalization model accounts for this phenomenon because unisensory neurons have their responses normalized by the activity of a pool of neurons that includes many multisensory cells (Ohshiro et al., 211). In area MSTd, a sub-population of neurons responds to only one modality, and almost all such neurons respond only to optic flow, with no vestibular response (Gu et al., 26). Thus, we tested whether the responses of unisensory MSTd neurons (n = 33) would exhibit cross-modal suppression. We used the same stimulation protocol as for multisensory MSTd neurons, except that the vestibular stimulus was presented at a somewhat arbitrary set of offsets (jdj) relative to the visual heading preference, most typically,9, and 18 (see STAR Methods for details). Unisensory MSTd neurons typically showed combined responses that were slightly suppressed relative to the visual response, consistent with predictions of the model (Figure 5A). Across the population tested, most unisensory MSTd neurons showed cross-modal suppression (Figure 5C, orange), with 4/56 cases having a gain ratio, G combined /G visual, that was significantly less than 1 (p <.5, bootstrap, filled symbols). The median gain ratio was.86, which was significantly less than 1 (p = , Wilcoxon signed-rank test). The average gain ratio did not depend significantly on the value of jdj (three ranges, 6,6 12, and ; p =.87, one-way ANOVA), consistent with the idea that the normalization pool contains neurons with a broad range of vestibular heading preferences. Previous studies in visual cortex have shown that divisive normalization predicts a lateral shift of intensity response functions (also known as input gain control) when a non-effective stimulus is superimposed on an effective stimulus (Carandini and Heeger, 211; Carandini et al., 1997; Heeger, 1992b). Thus, for a subset of unisensory MSTd neurons (n = 8), we also examined the effect of the amplitude of vestibular stimulation on cross-modal suppression. We found that cross-modal suppression became stronger with larger vestibular movement amplitudes, and that data from the majority of neurons were significantly better described by a rightward shift of the amplitude-response function than by a change in response gain (Figure S4). The finding is reminiscent of the lateral shift of stimulusresponse functions of sensory neurons when an ineffective mask stimulus is superimposed on an effective stimulus (Bonds, 1989; 44 Neuron 95, , July 19, 217

8 Carandini et al., 1997; Freeman et al., 22; Olsen et al., 21). These observations further support the notion that divisive normalization takes place following multisensory integration in MSTd. Lack of Cross-modal Suppression in MT Neurons If cross-modal suppression is produced by divisive normalization acting at the level of visual-vestibular integration in area MSTd, then we would not expect to see cross-modal suppression in the unisensory inputs to MSTd. Alternatively, cross-modal suppression might be present in the inputs to MSTd and might not reflect normalization at the level of multisensory integration. MSTd neurons receive their predominant visual input from area MT (Born and Bradley, 25; Maunsell and van Essen, 1983; Ungerleider and Desimone, 1986), which is not thought to exhibit vestibular responses (Chowdhury et al., 29; Smith et al., 212). We found that vestibular stimulation caused no appreciable cross-modal suppression of MT responses (e.g., Figure 5B). Across the population of MT neurons tested, the median gain ratio (G combined /G visual ) was.99 (Figure 5C), which was not significantly different from unity (p =.29, Wilcoxon signed-rank test). Moreover, the median gain ratio for MT was significantly different from that for the unisensory MSTd neurons (p = , Wilcoxon rank-sum test). Similar analyses using the firing rate measured at the largest stimulus amplitude in lieu of the gain parameter of the sigmoidal function produced nearly identical results (Figure S3C). These results suggest strongly that the crossmodal suppression exhibited by MSTd neurons is not inherited from area MT, consistent with the idea of a stage of divisive normalization acting at the level of multisensory integration (Ohshiro et al., 211). Note that vestibular stimulation caused a weak response in some MT neurons (Figure 5C). Specifically, the median ratio of vestibular and visual response gains, G vestibular /G visual, was.6, which is very small but significantly greater than zero (p = , Wilcoxon signed-rank test). We believe that this apparent vestibular activation is not a genuine vestibular response to otolithic input, but rather arises due to retinal slip that is caused by imperfect VOR suppression (Chowdhury et al., 29). This effect is likely exacerbated in the present study because the vestibular stimulus contained stationary background dots (see STAR Methods). Consistent with this interpretation, the apparent vestibular response of a subset of MT neurons was strongly reduced when the background dots were removed (Figure S5C). In addition, the apparent vestibular response was only seen for MT neurons with receptive fields near the fovea (Figure S5D), which have stronger responses to slow speeds of motion (Figure S5E). A true vestibular response would not depend on receptive field location or visual speed selectivity. Critically, MT neurons failed to show any cross-modal suppression regardless of whether they exhibited an apparent vestibular response (Figure 5C), indicating that cross-modal suppression arises downstream of MT. Normalization Fits MSTd Population Responses Better Than Alternative Model We showed in Figure 1 that the normalization model exhibits the diagnostic form of cross-modal suppression, whereas the alternative (subtractive) model does not. This suggests that the normalization model should better fit the responses of a population of MSTd neurons than the alternative model. To test this directly, we fit both models to the amplitude response curves for our population of MSTd neurons (see STAR Methods for details). Because the response of any one neuron in each model depends on the activity of all other neurons, it was necessary to fit the models simultaneously to data from all neurons and all stimulus conditions. This involved solving a large-scale optimization problem with 833 free parameters (for each model), and required over 25, hr of computing time. Fits of both models to data from an example multisensory MSTd neuron are shown in Figures 6A and 6B. The normalization model accounts quite well for the responses of this neuron across the various stimulus conditions (R 2 =.998; Figure 6A), as does the alternative (subtractive) model, which has the same number of parameters (R 2 =.995; Figure 6B). Note, however, that the normalization model predicts cross-modal suppression at large motion amplitudes, whereas the alternative model does not (Figures 6A and 6B). Specifically, for the alternative model, the predicted response in the combined condition with a non-preferred cue2 (vestibular) never falls below the response to the preferred cue1 (visual). Despite failing to capture the cross-modal suppression effect, the alternative (subtractive) model still fits the data quite well because the cross-modal suppression effect is rather subtle for most neurons and often only appears for the largest stimulus amplitudes. To better quantify these subtle differences in performance between the two models, partial correlation coefficients between the neural data and each model fit were computed (see STAR Methods). The Fisher z-transformation was applied to normalize the partial correlation coefficients such that they can be compared effectively (Figure 6C). The median z-transformed partial correlation coefficient for the normalization model (3.34) was significantly greater than that for the alternative (subtractive) model (1.74) (Figure 6C; p =.8, signed-rank test), indicating that the normalization model provides a better overall fit to the data. Importantly, this difference between models was more pronounced for the subset of neurons that showed significant cross-modal suppression (Figure 6C, filled symbols). For this subset, the median z-transformed partial correlation coefficient for the normalization model (4.94) was substantially greater than that for the alternative (subtractive) model (1.22, p =.1, signed-rank test). Among the 25 cases with significant cross-modal suppression, 18 were significantly better fit by the normalization model (filled symbols in the top-left region of Figure 6C), whereas only 4 neurons were significantly better fit by the alternative model (filled symbols in the bottom-right region). Data and fits for these four cases are shown in Figures S6C S6F. Critically, for these neurons that were better fit by the alternative model, it is not because the alternative model captured crossmodal suppression. Rather, neither model fits some features of the data very well in most of these cases, especially for the weaker stimulus modality. This may result from inaccurate estimates of the modality dominance weights for these neurons, which were not free parameters. Note that the data from unisensory MSTd neurons were excluded from this model comparison Neuron 95, , July 19,

9 A A B B C Figure 6. Summary of Model Fits to the Population of MSTd Neurons (A and B) Amplitude-response functions for an example MSTd neuron, along with best-fitting curves based on the normalization model (A), or the alternative (subtractive) model (B). The vestibular stimulus (Cue2) was presented at both preferred and non-preferred headings. Note that the diagnostic cross-modal suppression effect is captured well in the normalization model fit (green curve below black curve), but not in the alternative model fit. The partial correlation coefficient between data and model fit is.76 for the normalization model and.3 for the alternative model. Error bars indicate SEM. (C) Fisher z-transformed partial correlation coefficient for the normalization model (ordinate) is plotted against that for the alternative model (abscissa). Data from multisensory MSTd neurons are shown. Filled symbols correspond to cases that exhibit the diagnostic form of cross-modal suppression as defined in Figure 4. The red filled symbol corresponds to the data shown in (A) and (B). The plot is divided into three areas by the dashed lines; data points in the top-left region are significantly better fit by the normalization model, and those in the bottom-right region are significantly better fit by the alternative model. Figure 7. Summary of Cross-modal Suppression in Fits of the Normalization and Alternative Models Response gains were obtained from model fits to the data from each neuron and are plotted in the same format as Figure 4C. (A) Results for the normalization model. (B) Results for the alternative (subtractive) model. In each panel, combined response gains are plotted against the corresponding response gains for cue2. Values on both axes are normalized by the response gain for cue1. Filled green symbols represent cases with significant crossmodal suppression and significant activation by the non-preferred cue2, as identified by the analysis of actual firing rate data shown in Figure 4. Solid black curve, third-order polynomial fit to the data. because data from these neurons cannot differentiate the two models. Indeed, partial correlation coefficients from the unisensory MSTd neurons were not significantly different between the two models (p =.611, signed rank test). To further illustrate the differential model predictions with regard to the diagnostic form of cross-modal suppression, we computed response gain ratios from the fitted curves and plotted them as in Figure 4C. For the normalization model, many data points lie in the lower-right quadrant, illustrating the diagnostic cross-modal suppression effect (Figure 7A). A polynomial fit to the data has a y intercept of.91 ±.19 (SE), which is significantly less than 1 (p = ). In stark contrast, gain ratios from fits of the alternative (subtractive) model avoid the lowerright quadrant (Figure 7B), and the y intercept of the polynomial fit (.98 ±.15 SE) is not significantly different from 1 (p =.11). Although the overall scatter of data around the fit of the alternative model is reduced, the key feature is that this model does not predict observations in the lower-right quadrant, inconsistent with data from MSTd (Figure 4C). Thus, the alternative model fails to capture the diagnostic form of cross-modal suppression exhibited by many MSTd neurons. Together, these results demonstrate that the divisive normalization model better accounts for the multisensory responses of MSTd neurons than the alternative (subtractive) model. DISCUSSION We tested a key diagnostic prediction of the divisive normalization model of multisensory integration and found that the activity of many neurons in macaque area MSTd exhibits cross-modal suppression by a non-preferred stimulus that activates the neuron on its own. This finding is incompatible with a class of 46 Neuron 95, , July 19, 217

10 alternative models of multisensory integration that involve subtractive inhibition, as discussed further below. We also find that unisensory MSTd neurons exhibit cross-modal suppression, whereas unisensory MT neurons do not. Our findings, including the horizontal shift of amplitude response functions induced by cross-modal suppression, are consistent with the action of a divisive mechanism operating at the level of multisensory integration in MSTd. Together, our findings provide strong experimental support for the proposal that basic properties of multisensory integration by single neurons can be accounted for by simple models that include mechanisms functionally equivalent to divisive normalization (Ohshiro et al., 211). Normalization as Feedback The divisive normalization operation is often schematized in a form such that the activity of a neuron after linear combination of inputs is divisively normalized by the summed pre-normalized activity of all other neurons (Heeger, 1992b; Ohshiro et al., 211). However, it is biologically implausible that a real neuron has access to the pre-normalized activity of other neurons. Fortunately, the original formula for the normalization model (Equation 5) can be transformed into a mathematically equivalent form (Equation 6) in which each neuron s response depends on the spiking activity of other neurons (see STAR Methods). The more intuitive formulation of the model in Equation 6 can be implemented in the form of feedback normalization (Heeger, 1992b), and our multisensory normalization model (Figure 1) is also schematized in this form. In this scenario, the membrane potential of a neuron is transformed into spiking activity after a gain-modulation step involving a variable membrane conductance that depends on the pooled spiking activity of other neurons (Anderson et al., 2; Borg-Graham et al., 1998; Carandini et al., 1997). However, a membrane-conductance mechanism is just one possibility. A pre-synaptic modulation of excitatory input (Boehm and Betz, 1997) is also compatible with the feedback normalization scheme. Consistent with this idea, withdrawal of excitation was recently demonstrated as a potential mechanism of divisive normalization in mouse V1 (Sato et al., 216). Indeed, our findings are consistent with any mechanism that is operationally similar to divisive normalization. The feedback normalization formulation (Equation 6) allowed us to fit the model to data from our population of MSTd neurons (Figures 6 and 7). In previous work, a simplifying assumption that the net unnormalized activity of a population of V1 neurons could be approximated by the local average image contrast allowed a normalization model to be fit to responses of individual neurons (Busse et al., 29; Carandini and Heeger, 211; Carandini et al., 1997; Heeger, 1992b; Sato et al., 216). Since we were not able to make a comparable simplifying assumption for MSTd responses to self-motion, and given that the response of each neuron depends on that of all other neurons, it was necessary to fit the models to data from all neurons simultaneously. This required solving a very large optimization problem. Relationship to Previous Studies We are not the first to describe cross-modal suppression in singleunit activity. Indeed, a number of examples of cross-modal suppression by a non-optimal stimulus from one modality have been demonstrated in the literature (Kadunce et al., 1997; Meredith and Stein, 1996; Wallace et al., 1996). However, in these cases, the response to the non-preferred input is generally very weak or absent, and it is not clear whether the non-preferred input activates or suppresses the neuron on its own. If the nonpreferred input is suppressive by itself, then cross-modal suppression is rather trivial and does not distinguish different mechanisms. In contrast, cross-modal suppression by an activating input is a diagnostic feature of multisensory divisive normalization (Ohshiro et al., 211). We have demonstrated conclusively, for the first time, that many multisensory neurons exhibit the diagnostic form of cross-modal suppression predicted by divisive normalization. Note that the diagnostic form of cross-modal suppression is only expected in a narrow stimulus regime (Figure 1). In most cases, one has to hunt through the stimulus space carefully to find such a stimulus regime (e.g., Figure 2). For many cells, we probably missed the optimal stimulus conditions to elicit the effect; thus, the percentage of neurons that show the diagnostic form of cross-modal suppression (filled green symbols, Figure 4C) is almost certainly underestimated by a substantial amount. The literature also contains some examples of cross-modal suppression in response to stimuli that may be preferred for both modalities tested (Avillac et al., 27; Diehl and Romanski, 214; Sugihara et al., 26). These are generally cases in which the response of one modality is much weaker than the other, and we have previously shown that normalization can also produce cross-modal suppression in such cases (Ohshiro et al., 211). In our sample of MSTd neurons, a few cells showed cross-modal suppression when tested with preferred heading stimuli for both modalities (purple symbols in the lower-right quadrant of Figure 4C), and these were neurons with rather imbalanced visual and vestibular responses. Our findings are analogous to stimulus interactions reported within sensory modalities. For example, an analogous form of suppression by a weakly activating stimulus has been demonstrated in responses of cat V1 neurons (Cavanaugh et al., 22). Our results are also closely related to experimental observations of response averaging, a property widely observed in visual neurons that are tested with multiple stimuli (Alvarado et al., 27; Britten and Heuer, 1999; Busse et al., 29; Carandini et al., 1997; Recanzone et al., 1997; Zoccolan et al., 25). Since the response of these neurons to multiple visual stimuli is typically intermediate between responses to the individual stimuli (similar to averaging), the response to the more effective stimulus is suppressed by simultaneous presentation of the less effective stimulus (which may activate the neuron when presented alone). Divisive normalization has also been shown to successfully account for these types of averaging phenomena within a single sensory modality (Busse et al., 29; Carandini and Heeger, 211; Carandini et al., 1997). This suggests that there are multiple stages of divisive normalization, and our findings provide novel evidence for stages of normalization following multisensory combination of inputs from different sensory systems. Comparison to Other Models and Mechanisms Groh (21) presented a summation/saturation model that has a similar architecture to the normalization model: a layer of Neuron 95, , July 19,

11 pre-normalized units is followed by an inhibitory neuron that sums this pre-normalized activity and modulates neural responses in the subsequent target layer (Figure 1C from Groh, 21). Although this model was not intended as an account of multisensory integration, it does predict interactions between multiple stimuli that are qualitatively similar to our normalization model. Specifically, it predicts that adding a weak stimulus to a strong stimulus produces response enhancement for low intensities but response suppression for high intensities (Figure 3C; Groh, 21), which is analogous to our diagnostic cross-modal suppression. Other published models of multisensory integration have assumed a subtractive inhibition mechanism to account for cross-modal suppression (Cuppini et al., 21; Ursino et al., 29). We have shown previously (Ohshiro et al., 211) that a generic form of these alternative models based on subtractive inhibition does not exhibit the diagnostic form of cross-modal suppression that is predicted by divisive normalization. In the alternative models (Cuppini et al., 21; Ursino et al., 29), a measure of total population activity is subtracted from the multisensory input to a neuron, and the resulting difference is transformed into firing rate by a sigmoidal static non-linearity. Indeed, this basic architecture involving subtractive inhibition is commonly used in neural network models (Dayan and Abbott, 21). In contrast, the multisensory input is divided by a measure of total population activity in normalization models (Carandini et al., 1997; Ohshiro et al., 211). We provide a mathematical argument (see STAR Methods) that alternative models based on subtractive inhibition only exhibit cross-modal suppression due to an excitatory nonpreferred input if the sigmoidal static non-linearity, h(x), has a sharp exponential rising phase and the network is activated with weak stimuli. This prediction lies in marked contrast to that of the normalization model, which shows the diagnostic form of cross-modal suppression more robustly for strong stimuli than weak stimuli (see Figure 1B). The subtractive inhibition term in our alternative model directly scales with population activity (Equation 1). Applying a threshold to this term would produce stronger inhibition for the combined stimuli than for the component unisensory stimuli, potentially producing the diagnostic cross-modal suppression. However, in separate simulations (data not shown), we found that this variant of the alternative model only produced the diagnostic cross-modal suppression effect for intermediate stimulus intensities, and that it was accompanied by a negative slope in the intensity-response function for the non-preferred cue. Neither of these features are predicted by the normalization model or observable in our data. Thus, the thresholded variant of the alternative model is less likely to account for our findings. Another recent subtraction-based model shows response normalization for combinations of stimuli (Rubin et al., 215). This model consists of separate networks of excitatory and inhibitory neurons, and exhibits richer network dynamics than our normalization model. This includes simultaneous reductions in both excitation and inhibition during strong recurrent activation of the local network (Ozeki et al., 29), which has been observed experimentally in mouse V1 (Sato et al., 216). It remains to be examined if the model of Rubin et al. (215) can demonstrate cross-modal suppression for strong stimuli, as observed for real MSTd neurons. Schwabe et al. (26) proposed a subtraction-based recurrent network model of V1 neurons. This model accounts for response facilitation/suppression induced by stimuli presented in the surround of V1 receptive fields (Ichida et al., 27), which critically depends on the intensity of stimuli presented simultaneously to the receptive field center. These effects can resemble our diagnostic cross-modal suppression effect. However, no positive activating response to the surround stimulus alone was demonstrated in the study by Ichida et al. (27) and a related study (Polat et al., 1998), or in the recurrent model of Schwabe et al. (26). Thus, the link between these studies and our cross-modal suppression effect is unclear. Importantly, our findings do not strongly constrain possible neural mechanisms of normalization; any mechanism that is operationally equivalent to division might account for the cross-modal suppression effects we observe. Unisensory Inputs to MSTd MSTd receives its major visual input from the adjacent area MT (Maunsell and van Essen, 1983; Ungerleider and Desimone, 1986). Our normalization model (Figure 1A) assumes that MSTd neurons receive purely visual inputs from area MT, which is consistent with previous studies that did not find vestibular signals in MT (Chowdhury et al., 29; Smith et al., 212). In contrast to the well-characterized pathway by which visual signals propagate to area MSTd, the source of vestibular inputs to MSTd remains unclear, and it is likely that MSTd receives vestibular input through other cortical areas (Chen et al., 21, 211a, 211b, 211c, 213; Fetsch et al., 21). Although our normalization model assumes that MSTd neurons receive visual and vestibular inputs from separate sources (Figure 1A), we cannot rule out the possibility that vestibular inputs to MSTd have already been integrated with visual signals. Identification and detailed characterization of the vestibular input to MSTd awaits further study. In our formulation of the multisensory integration model (Equation 2), a unisensory neuron is simply a cell for which either of the modality dominance weights is set to zero. Consistent with this idea of a continuum between unisensory and multisensory neurons, the gain ratio data from both types of MSTd neurons appear to fall along a single trend line (Figure S3D). This observation suggests that unisensory MSTd neurons are not a distinct class of neurons; rather, their response properties are as expected from a normalization model in which one modality dominance weight (typically, the vestibular weight in MSTd) is close to zero. In conclusion, our findings provide strong evidence for a stage of divisive normalization operating at the level of multisensory integration. These findings suggest that a diverse array of experimental findings in multisensory integration may be accounted for by a small number of nonlinear mechanisms. Together with previous results demonstrating a role for divisive normalization in diverse neural phenomena including attention, decision-making, and motor control, our results support the idea that normalization is a canonical neural computation that is repeated at multiple stages in the brain. 48 Neuron 95, , July 19, 217

12 STAR+METHODS Detailed methods are provided in the online version of this paper and include the following: d KEY RESOURCES TABLE d CONTACT FOR REAGENT AND RESOURCE SHARING d EXPERIMENTAL MODEL AND SUBJECT DETAILS B Macaca mulatta d METHOD DETAILS B Surgery B Motion platform and visual stimuli B Electrophysiological recordings B Experimental protocol B Quantification of amplitude-response functions B Post hoc classification of unisensory neurons B Divisive normalization model of multisensory integration in MSTd B Alternative (subtractive inhibition) model B Cross-modal suppression in the alternative model B Population curve-fit analysis d QUANTIFICATION AND STATISTICAL ANALYSIS d DATA AND SOFTWARE AVAILABILITY SUPPLEMENTAL INFORMATION Supplemental Information includes six figures and one data file and can be found with this article online at AUTHOR CONTRIBUTIONS T.O. and G.C.D. designed the experiments; T.O. collected and analyzed the data and performed the simulations; and T.O., G.C.D., and D.E.A. wrote the manuscript and consulted on revisions. ACKNOWLEDGMENTS This work was supported by NIH grant EY16178 (to G.C.D.) and by a CORE grant (EY1319) from the National Eye Institute. D.E.A. was supported by EY T.O. acknowledges support by JSPS KAKENHI (16K6984) and by a Grant-in-Aid for Scientific Research on Innovative Areas (15H5879, Non-linear Neuro-oscillology ). We thank Dina Knoedl and Swati Shimpi for excellent animal care and technical support, and Johnny Wen for programming support. Received: March 1, 217 Revised: June 19, 217 Accepted: June 26, 217 Published: July 19, 217 REFERENCES Adesnik, H., Bruns, W., Taniguchi, H., Huang, Z.J., and Scanziani, M. (212). A neural circuit for spatial summation in visual cortex. Nature 49, Alais, D., Newell, F.N., and Mamassian, P. (21). Multisensory processing in review: from physiology to behaviour. Seeing Perceiving 23, Albrecht, D.G., and Hamilton, D.B. (1982). Striate cortex of monkey and cat: contrast response function. J. Neurophysiol. 48, Alvarado, J.C., Vaughan, J.W., Stanford, T.R., and Stein, B.E. (27). Multisensory versus unisensory integration: contrasting modes in the superior colliculus. J. Neurophysiol. 97, Anderson, J.S., Carandini, M., and Ferster, D. (2). Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex. J. Neurophysiol. 84, Angelaki, D.E., Shaikh, A.G., Green, A.M., and Dickman, J.D. (24). Neurons compute internal models of the physical laws of motion. Nature 43, Avillac, M., Ben Hamed, S., and Duhamel, J.-R. (27). Multisensory integration in the ventral intraparietal area of the macaque monkey. J. Neurosci. 27, Boehm, S., and Betz, H. (1997). Somatostatin inhibits excitatory transmission at rat hippocampal synapses via presynaptic receptors. J. Neurosci. 17, Bonds, A.B. (1989). Role of inhibition in the specification of orientation selectivity of cells in the cat striate cortex. Vis. Neurosci. 2, Borg-Graham, L.J., Monier, C., and Frégnac, Y. (1998). Visual input evokes transient and strong shunting inhibition in visual cortical neurons. Nature 393, Born, R.T., and Bradley, D.C. (25). Structure and function of visual area MT. Annu. Rev. Neurosci. 28, Britten, K.H., and Heuer, H.W. (1999). Spatial summation in the receptive fields of MT neurons. J. Neurosci. 19, Britten, K.H., and Van Wezel, R.J. (22). Area MST and heading perception in macaque monkeys. Cereb. Cortex 12, Busse, L., Wade, A.R., and Carandini, M. (29). Representation of concurrent stimuli by population activity in visual cortex. Neuron 64, Carandini, M., and Ferster, D. (2). Membrane potential and firing rate in cat primary visual cortex. J. Neurosci. 2, Carandini, M., and Heeger, D.J. (211). Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, Carandini, M., Heeger, D.J., and Movshon, J.A. (1997). Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, Cavanaugh, J.R., Bair, W., and Movshon, J.A. (22). Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J. Neurophysiol. 88, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (21). Macaque parieto-insular vestibular cortex: responses to self-motion and optic flow. J. Neurosci. 3, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (211a). A comparison of vestibular spatiotemporal tuning in macaque parietoinsular vestibular cortex, ventral intraparietal area, and medial superior temporal area. J. Neurosci. 31, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (211b). Convergence of vestibular and visual self-motion signals in an area of the posterior sylvian fissure. J. Neurosci. 31, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (211c). Representation of vestibular and visual cues to self-motion in ventral intraparietal cortex. J. Neurosci. 31, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (213). Functional specializations of the ventral intraparietal area for multisensory heading discrimination. J. Neurosci. 33, Chowdhury, S.A., Takahashi, K., DeAngelis, G.C., and Angelaki, D.E. (29). Does the middle temporal area carry vestibular signals related to self-motion? J. Neurosci. 29, Cuppini, C., Ursino, M., Magosso, E., Rowland, B.A., and Stein, B.E. (21). An emergent model of multisensory integration in superior colliculus neurons. Front. Integr. Nuerosci. 4, 6. Dayan, P., and Abbott, L.F. (21). Theoretical Neuroscience (MIT Press). Diehl, M.M., and Romanski, L.M. (214). Responses of prefrontal multisensory neurons to mismatching faces and vocalizations. J. Neurosci. 34, Neuron 95, , July 19,

13 Driver, J., and Noesselt, T. (28). Multisensory interplay reveals crossmodal influences on sensory-specific brain regions, neural responses, and judgments. Neuron 57, Duffy, C.J. (1998). MST neurons respond to optic flow and translational movement. J. Neurophysiol. 8, Efron, B., and Tibshirani, R.J. (1993). An Introduction to the Bootstrap (Chapman & Hall). Ernst, M.O., and B ulthoff, H.H. (24). Merging the senses into a robust percept. Trends Cogn. Sci. 8, Ferster, D. (1994). Linearity of synaptic interactions in the assembly of receptive fields in cat visual cortex. Curr. Opin. Neurobiol. 4, Fetsch, C.R., Rajguru, S.M., Karunaratne, A., Gu, Y., Angelaki, D.E., and DeAngelis, G.C. (21). Spatiotemporal properties of vestibular responses in area MSTd. J. Neurophysiol. 14, Fetsch, C.R., Pouget, A., DeAngelis, G.C., and Angelaki, D.E. (211). Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 15, Freeman, T.C., Durand, S., Kiper, D.C., and Carandini, M. (22). Suppression without inhibition in visual cortex. Neuron 35, Groh, J.M. (21). Converting neural signals from place codes to rate codes. Biol. Cybern. 85, Gu, Y., Watkins, P.V., Angelaki, D.E., and DeAngelis, G.C. (26). Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area. J. Neurosci. 26, Gu, Y., DeAngelis, G.C., and Angelaki, D.E. (27). A functional link between area MSTd and heading perception based on vestibular signals. Nat. Neurosci. 1, Gu, Y., Angelaki, D.E., and DeAngelis, G.C. (28). Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci. 11, Gu, Y., DeAngelis, G.C., and Angelaki, D.E. (212). Causal links between dorsal medial superior temporal area neurons and multisensory heading perception. J. Neurosci. 32, Heeger, D.J. (1992a). Half-squaring in responses of cat striate cells. Vis. Neurosci. 9, Heeger, D.J. (1992b). Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, Ichida, J.M., Schwabe, L., Bressloff, P.C., and Angelucci, A. (27). Response facilitation from the suppressive receptive field surround of macaque V1 neurons. J. Neurophysiol. 98, Jagadeesh, B., Wheat, H.S., and Ferster, D. (1993). Linearity of summation of synaptic potentials underlying direction selectivity in simple cells of the cat visual cortex. Science 262, Kadunce, D.C., Vaughan, J.W., Wallace, M.T., Benedek, G., and Stein, B.E. (1997). Mechanisms of within- and cross-modality suppression in the superior colliculus. J. Neurophysiol. 78, Kolster, H., Mandeville, J.B., Arsenault, J.T., Ekstrom, L.B., Wald, L.L., and Vanduffel, W. (29). Visual field map clusters in macaque extrastriate visual cortex. J. Neurosci. 29, Komatsu, H., and Wurtz, R.H. (1988). Relation of cortical areas MT and MST to pursuit eye movements. I. Localization and visual properties of neurons. J. Neurophysiol. 6, Lee, J., and Maunsell, J.H. (29). A normalization model of attentional modulation of single unit responses. PLoS ONE 4, e4651. Louie, K., Grattan, L.E., and Glimcher, P.W. (211). Reward value-based gain control: divisive normalization in parietal cortex. J. Neurosci. 31, Maunsell, J.H., and van Essen, D.C. (1983). The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. J. Neurosci. 3, Meredith, M.A., and Stein, B.E. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol. 56, Meredith, M.A., and Stein, B.E. (1996). Spatial determinants of multisensory integration in cat superior colliculus neurons. J. Neurophysiol. 75, Meredith, M.A., Nemitz, J.W., and Stein, B.E. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 7, Miller, K.D., and Troyer, T.W. (22). Neural noise can explain expansive, power-law nonlinearities in neural response functions. J. Neurophysiol. 87, Morgan, M.L., DeAngelis, G.C., and Angelaki, D.E. (28). Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 59, Nelissen, K., Vanduffel, W., and Orban, G.A. (26). Charting the lower superior temporal region, a new motion-sensitive region in monkey superior temporal sulcus. J. Neurosci. 26, Ohshiro, T., Angelaki, D.E., and DeAngelis, G.C. (211). A normalization model of multisensory integration. Nat. Neurosci. 14, Olsen, S.R., Bhandawat, V., and Wilson, R.I. (21). Divisive normalization in olfactory population codes. Neuron 66, Ozeki, H., Finn, I.M., Schaffer, E.S., Miller, K.D., and Ferster, D. (29). Inhibitory stabilization of the cortical network underlies visual surround suppression. Neuron 62, Perrault, T.J., Jr., Vaughan, J.W., Stein, B.E., and Wallace, M.T. (23). Neuron-specific response characteristics predict the magnitude of multisensory integration. J. Neurophysiol. 9, Perrault, T.J., Jr., Vaughan, J.W., Stein, B.E., and Wallace, M.T. (25). Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J. Neurophysiol. 93, Polat, U., Mizobe, K., Pettet, M.W., Kasamatsu, T., and Norcia, A.M. (1998). Collinear stimuli regulate visual responses depending on cell s contrast threshold. Nature 391, Priebe, N.J., and Ferster, D. (28). Inhibition, spike threshold, and stimulus selectivity in primary visual cortex. Neuron 57, Raposo, D., Sheppard, J.P., Schrater, P.R., and Churchland, A.K. (212). Multisensory decision-making in rats and humans. J. Neurosci. 32, Recanzone, G.H., Wurtz, R.H., and Schwarz, U. (1997). Responses of MT and MST neurons to one and two moving objects in the receptive field. J. Neurophysiol. 78, Reynolds, J.H., and Heeger, D.J. (29). The normalization model of attention. Neuron 61, Rubin, D.B., Van Hooser, S.D., and Miller, K.D. (215). The stabilized supralinear network: a unifying circuit motif underlying multi-input integration in sensory cortex. Neuron 85, Sato, T.K., Haider, B., H ausser, M., and Carandini, M. (216). An excitatory basis for divisive normalization in visual cortex. Nat. Neurosci. 19, Schwabe, L., Obermayer, K., Angelucci, A., and Bressloff, P.C. (26). The role of feedback in shaping the extra-classical receptive field of cortical neurons: a recurrent network model. J. Neurosci. 26, Skaliora, I., Doubell, T.P., Holmes, N.P., Nodal, F.R., and King, A.J. (24). Functional topography of converging visual and auditory inputs to neurons in the rat superior colliculus. J. Neurophysiol. 92, Smith, A.T., Wall, M.B., and Thilo, K.V. (212). Vestibular inputs to human motion-sensitive visual cortex. Cereb. Cortex 22, Spence, C. (21). Crossmodal spatial attention. Ann. N Y Acad. Sci. 1191, Spence, C. (211). Crossmodal correspondences: a tutorial review. Atten. Percept. Psychophys. 73, Stanford, T.R., Quessy, S., and Stein, B.E. (25). Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, Stein, B.E., and Stanford, T.R. (28). Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, Neuron 95, , July 19, 217

14 Sugihara, T., Diltz, M.D., Averbeck, B.B., and Romanski, L.M. (26). Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J. Neurosci. 26, Ungerleider, L.G., and Desimone, R. (1986). Cortical connections of visual area MT in the macaque. J. Comp. Neurol. 248, Ursino, M., Cuppini, C., Magosso, E., Serino, A., and di Pellegrino, G. (29). Multisensory integration in the superior colliculus: a neural network model. J. Comput. Neurosci. 26, Van Essen, D.C., Drury, H.A., Dickson, J., Harwell, J., Hanlon, D., and Anderson, C.H. (21). An integrated software suite for surface-based analyses of cerebral cortex. J. Am. Med. Inform. Assoc. 8, Wallace, M.T., Wilkinson, L.K., and Stein, B.E. (1996). Representation and integration of multiple sensory inputs in primate superior colliculus. J. Neurophysiol. 76, Zoccolan, D., Cox, D.D., and DiCarlo, J.J. (25). Multiple object response normalization in monkey inferotemporal cortex. J. Neurosci. 25, Neuron 95, , July 19,

15 STAR+METHODS KEY RESOURCES TABLE REAGENT or RESOURCE SOURCE IDENTIFIER Experimental Models: Organisms/Strains Macaca mulatta Primate Products N/A Software and Algorithms MATLAB 216a statistics and machine learning toolbox MathWorks MATLAB 216a optimization toolbox MathWorks MATLAB 216a parallel-computing toolbox MathWorks MATLAB model simulation code This paper N/A TEMPO Experiment control system Reflective computing Visual stimulus generation code, written in Visual C++ using the OpenGL graphics libraries This paper; Gu et al., 26 N/A Other Tungsten microelectrodes, Epoxylite insulated Frederic Haer Company MOOG 6DOF2E, 6-degree-of-freedom motion platform Moog Christie Mirage S + 3K, digital light projector Christie Digital CONTACT FOR REAGENT AND RESOURCE SHARING Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Gregory C. DeAngelis (gdeangelis@cvs.rochester.edu). EXPERIMENTAL MODEL AND SUBJECT DETAILS Macaca mulatta Physiological experiments were performed with two male rhesus monkeys (Macaca mulatta) weighing 13 and 16 kg. Both animals were ages 6-8 years during the course of the studies, and were pair-housed in a vivarium with a 12 hr light cycle (6AM to 6PM). All animal surgeries and experimental procedures were approved by the University Committee on Animal Resources at the University of Rochester and were in accordance with National Institutes of Health guidelines. METHOD DETAILS Surgery Under sterile conditions, monkeys were chronically implanted with a circular Delrin ring (diameter: 7cm) for head stabilization, as described previously (Gu et al., 26), as well as a scleral search coil for measuring eye movements. After recovery, animals were trained to fixate visual targets for fluid reward using standard operant conditioning techniques. For electrophysiological recording, a Delrin grid ( cm) containing rows of holes was stereotaxically secured to the skull inside the head-restraint ring and was positioned in the horizontal plane. The holes in the grid (.8 mm spacing) allowed vertical penetration of microelectrodes into the brain via transdural guide tubes that were inserted through a small burr hole in the skull. Burr holes were made surgically under aseptic conditions while the subjects were anesthetized. The recording grid extended bilaterally from the midline to regions overlying areas MST and MT in both hemispheres. Motion platform and visual stimuli During experiments, monkeys were seated comfortably in a primate chair with their head restrained. The chair was securely attached to a 6-degree-of-freedom motion platform (MOOG 6DOF2E; Moog, East Aurora, NY) and resided within a field-coil frame used to track eye movements. The motion platform allowed physical translation along any axis in 3D (Gu et al., 26). Platform motion and optic flow stimuli could be presented either together or separately, and the directions of self-motion indicated by optic flow and platform motion could be either congruent or disparate. Inertial motion ( vestibular ) stimuli were generated by physically translating the subject with the motion platform (whole body translation). Each movement followed a Gaussian velocity profile with duration of 2 s and a standard deviation of 1/3 s (Gu et al., e1 Neuron 95, e1 e8, July 19, 217

16 26) (Figure S1). The command signal to the motion platform had a peak velocity at the middle (+1 s) of the stimulus presentation period; however, the actual motion of the platform is delayed by 1ms from the command signal due to platform dynamics. The amplitude of translation varied from to 12 cm, total displacement. At the maximum translation amplitude, the peak velocity and peak acceleration of the movement were 28.7 cm/s and 14.5 cm/s 2 (.1 g), respectively. Optic flow stimuli that simulated self-motion were rear-projected onto a cm tangent screen that was positioned 3 cm in front of the monkey, thus subtending 9 x9 of visual angle. Visual stimuli were presented using a three-chip digital light projector (Christie Mirage S + 3K, Christie Digital, Cyrus, CA) mounted on top of the motion platform. Optic flow stimuli were generated in Visual C++ using the OpenGL graphics libraries, and were rendered by a Quadro FX 14 graphics board. Self-motion was simulated by moving the OpenGL cameras through a virtual environment (Gu et al., 26). The virtual environment consisted of a random 3D cloud of stars that was 1 cm wide, 1 cm tall, and 4 cm deep. The star field had a density of.1 stars per cm 3, with each star being a cm triangle. To provide stereoscopic depth cues, the stimulus was rendered as a red-green anaglyph and was viewed through custom red-green goggles (Kodak Wratten filters; red #29, green #61). To avoid extremely large (near) stars from appearing in the display, stimulus elements within 5 cm of the eyes were not rendered by virtue of the near clipping plane. Precise synchronization of optic flow and platform motion was achieved through a dynamical systems analysis of the motion platform and a delay parameter was adjusted to achieve synchronous movement within a resolution of a few ms (Gu et al., 26). For all stimulus conditions, the animal was required to fixate a central visual target (.2 in diameter) for 2 ms before stimulus onset (fixation window spanned 1.7 x 1.7 of visual angle). The animal was rewarded at the end of each trial for maintaining fixation throughout stimulus presentation. To aid the animal to distinguish the fixation target from the star field and to reduce visual tracking of the optic flow stimuli, the fixation target was surrounded by a textured circular patch containing an orthogonal grid pattern of fine lines (width: 1 pixel) spaced.5 apart. The diameter of this fixation patch was gradually reduced to 1 to minimize occlusion of the optic flow stimulus. The sides and top of the search coil frame were covered with a black enclosure to prevent the animal from experiencing any visual motion other than the optic flow presented on the display screen. Electrophysiological recordings We recorded extracellular activity from single neurons in four hemispheres of two monkeys. A tungsten microelectrode (Frederic Haer Company, Bowdoinham, ME; tip diameter 3 mm, impedance 1-2 MOhm at 1 khz) was advanced into the cortex through a transdural guide tube, using a hydraulic micromanipulator (Narishige) mounted on top of the head-restraint ring. Action potentials were amplified and isolated using a head-stage pre-amplifier, a band-pass eight-pole filter (Krohn-Hite, model 3384; 4-5 Hz), and a dual voltage-time window discriminator (Bak Electronics, model RP-1). The times of occurrence of action potentials and all behavioral events were recorded with 1 ms resolution by the data acquisition computer. Raw neural signals were also digitized at 25 khz and stored to disk for offline spike sorting and additional analyses. Experimental control and data acquisition was coordinated by scripts written with TEMPO software (Reflective Computing) Area MSTd was localized with aid from structural MRI scans and a standard macaque atlas (Van Essen et al., 21). MSTd was typically identified as a region centered 15 mm lateral to the midline and 3-6mm posterior to the interaural plane. Electrode penetrations were also guided by the pattern of background activity as the electrode traversed through gray and white matter, as well as the response properties of neurons to visual stimuli. MSTd was usually the first gray matter encountered, 6 to 1 mm below the cortical surface, that exhibited prominent response modulation to flashing random-dot stimuli and direction-selective responses to motion of the dots. Once action potentials were satisfactorily isolated from a single neuron, we first mapped the receptive field (RF) of the neuron manually by manipulating the size, location, and motion of a patch of drifting random dots using a custom graphical interface. MSTd neurons typically had large RFs that occupied most of a quadrant, if not more, in the contralateral hemi-field. The RF often extended into the ipsilateral visual field and included the fovea. Recordings were also made from the middle temporal (MT) area by further advancing the electrode into the posterior bank of the superior temporal sulcus (STS). MT neurons were identified by their much smaller receptive fields (diameter eccentricity) and strong direction selectivity in response to random-dot stimuli. We also took advantage of the known retinotopic organization of MT receptive fields to identify this area: receptive field centers shifted gradually from foveal to peripheral as the recording sites moved from antero-lateral to postero-medial within the posterior bank of the STS (Kolster et al., 29; Komatsu and Wurtz, 1988; Maunsell and van Essen, 1983). We were careful to distinguish area MT from the lateral subdivision of area MST (MSTl), which is located anterior to area MT in the STS (Kolster et al., 29; Komatsu and Wurtz, 1988; Nelissen et al., 26). We mainly targeted regions of area MT that were located postero-medially, such that we were confident to avoid MSTl. Most MT neurons had receptive fields centered near the horizontal meridian, with a median eccentricity of 13 (Figure S5). Occasionally, we encountered neurons with RFs that covered the fovea but were fairly large in size (diameter 1 ), or RFs that extended considerably into the ipsilateral visual field. As these might have been MSTl neurons (Komatsu and Wurtz, 1988), they were excluded from the sample out of caution. The direction tuning of MT neurons was examined using the same optic flow stimulus used in MSTd recordings (see below), except that headings were restricted to the fronto-parallel plane. MT neurons that did not respond to our optic-flow stimulus due to strong surround suppression (N = 16) were not recorded. Experimental protocol After a basic characterization of visual response properties was obtained, we next examined visual and vestibular heading selectivity in three-dimensional space while monkeys performed a fixation task. Self-motion stimuli, either vestibular or optic flow, were Neuron 95, e1 e8, July 19, 217 e2

17 presented along 26 headings corresponding to all combinations of azimuth and elevation angles in increments of 45 (Gu et al., 26). The visual and vestibular conditions were randomly interleaved within a block of trials. The amplitude of translation was 1 cm. In the vestibular condition of this preliminary test, the display was blank, except for a head-fixed fixation target and a small patch of texture around the fixation target. In the visual condition, the motion platform was stationary and optic flow simulated self-motion. Heading tuning profiles were displayed in an on-line graphical user interface, similar to Figure 2A. We transformed the data using the Lambert cylindrical equal-area projection (Gu et al., 26). The heading preference of a neuron was estimated for each stimulus modality by visually inspecting the heading tuning profile after two to five stimulus repetitions were complete. An initial classification of neuron type (multisensory or unisensory) was also made based on the heading tuning profiles: a neuron with clear heading selectivity for both modalities was labeled as multisensory. In a substantial minority of cases, the vestibular heading preference was not determined confidently due to weak responses and the appearance of multiple peaks in the heading tuning profile. In these cases, the stimulus that produced the largest response was chosen as the vestibular heading preference. Neurons that responded only to stimuli of one modality were provisionally classified as unisensory, and all unisensory MSTd neurons were visual (n = 32). Some neurons were later reclassified based on more extensive data from the main experimental protocol, as described below. The experimental protocol for unisensory MSTd neurons was slightly different from that for multisensory neurons. For multisensory MSTd neurons, we proceeded to perform a search protocol to identify the range of heading offsets for one modality that might produce cross-modal suppression. The heading for one modality (e.g., vestibular) was fixed at the neuron s preferred heading, while the heading for the other modality (e.g., visual) varied in 3 steps of elevation around the clock from the neuron s preference. Motion amplitude was fixed at 1 cm for both modalities. The combined response, the two single-modality responses, and baseline activity were plotted on-line, as in Figure 2B. We sought a range of headings of the variable modality that produced: (1) a response greater than baseline activity, and (2) a suppression of the combined response relative to the fixed unimodal stimulus. Note that these criteria for the diagnostic form of cross-modal suppression were not satisfied during the search protocol for all neurons. In such cases, we still chose multiple non-preferred headings that met the requirements partially, and proceeded to perform the main experimental protocol. Based on outcomes from the first animal, we generally did not perform the search protocol in monkey 2. Instead, one or two non-preferred headings, located along a flank of the heading tuning profile, were selected by visual inspection. We found that this time-saving step did not reduce the frequency with which we were able to identify neurons that showed the diagnostic form of cross-modal suppression. For some multisensory neurons, we explored heading offsets as large as 18 (Figure S3). These were typically cases in which some activating response was produced even at the anti-preferred heading. Regardless of the approach, the end result of these search procedures was a set of non-preferred headings that was likely to produce cross-modal suppression. Next, we ran the main experimental protocol to measure neural responses to various stimulus amplitudes. We presented vestibular and visual stimuli separately and together at the following motion amplitudes:.,.6, 1.6, 4.4 and 11.8 cm. For the combined cue condition, the amplitudes of both stimulus modalities were varied together (yoked). We fixed the heading of one modality to the neuron s preferred direction, whereas the heading of the other modality was specified as either the neuron s preferred heading or one of the non-preferred headings that were identified in the search process described above. We more frequently fixed the heading for the dominant modality (typically visual) of the neuron (81% of cells), but we also sometimes fixed the heading of the non-dominant modality (19%). All stimuli for the vestibular, visual, and combined conditions were tested in a block of randomly interleaved trials. When isolation remained satisfactory, we repeated the search protocol after switching the roles of the two modalities (fixed versus varying heading), and then ran the main experimental protocol again with the other modality fixed at the preferred heading (n = 9 neurons). For MSTd neurons that were initially classified as unisensory (n = 32, all visual cells), we skipped the search protocol and proceeded directly to the main experimental protocol. In these cases, the visual stimulus was fixed at the preferred heading and was paired with vestibular stimuli presented at,9, and/or 18 offsets from the visual heading preference. Motion amplitude was varied over the same range (.,.6, 1.6, 4.4 and 11.8 cm) for most neurons, although a subset of neurons (n = 12) was tested with a different range (.,.1, 1., 2., 4. and 8. cm). MT neurons were tested using largely the same experimental protocol, with a couple of exceptions. First, the initial characterization of heading selectivity for MT neurons was restricted to the fronto-parallel plane. As for unisensory MSTd neurons, we did not carry out a search procedure for MT neurons; rather, the heading offsets for the variable modality were generally chosen to be,9, and 18. Motion amplitude was varied over the same range used for most MSTd neurons:.,.6, 1.6, 4.4 and 11.8 cm. Neurons were included in our main sample if each distinct stimulus was successfully repeated at least five times in the main experiment (median repetition number: 8). Data from 55 MSTd neurons were excluded based on this criterion. Baseline activity of neurons was measured while the animal fixated on a central target, and a cloud of stationary stars was presented on the screen while the motion platform remained stationary. This baseline condition was included in the block of randomly interleaved trials with the other stimulus conditions. All evoked neural activity elicited by moving stimuli was referenced to this baseline. We chose this baseline such that any deviation from baseline response is attributable to motion energy contained in the stimulus. In the vestibular condition (i.e., optic flow motion amplitude = ; vestibular amplitude s ), the cloud of stationary stars was also presented while the body was translated. We also tested 24 neurons for cross-modal suppression using a vestibular condition in which there were no stationary dots present on the display, and results (in the format of Figure 4C) were similar to those of our standard protocol (y-intercept =.78; 95% CI = [.63,.92]). e3 Neuron 95, e1 e8, July 19, 217

18 Quantification of amplitude-response functions Amplitude-response functions (e.g., Figures 4A and 4B) were fit with a hyperbolic ratio function (Albrecht and Hamilton, 1982): c n r = Gðcue1; cue2þ c n b; (Equation 1) cn where r represents the mean firing rate of the neuron, c represents the stimulus motion amplitude, c 5 is a semi-saturation constant that determines the motion amplitude at which the function begins to saturate, and b represents baseline response. G(cue1,cue2) indicates the response gain parameter that was used to quantify response strength for a particular combination of cue1 and cue2. In this notation, G(p,p) represents the response gain when both cue1 and cue2 have preferred directions, G(p,n) denotes gain for a preferred cue1 and a non-preferred cue 2, G(p,) indicates gain for a unisensory preferred cue1 stimulus, and G(,p) or G(,n) represent gains for unisensory stimuli with preferred or non-preferred headings for cue 2, respectively. Curve fitting was performed simultaneously for all of the amplitude-response functions measured during a single block of trials (e.g., Figures 4A and 4B). c 5, n and b are free parameters that are common to all of the response functions that were fit simultaneously. In other words, all fitted curves for a neuron were constrained to have the same shape and baseline response. In a separate analysis, amplitude-response functions were also fit with a simple linear function including a threshold, to determine whether a saturating function is needed to fit the data well. The hyperbolic ratio model and the linear model were compared using the corrected form of Akaike s information criteria (AICc), and the hyperbolic ratio model was preferred for 111/119 neurons. Across the population, the median AICc value was significantly less for the hyperbolic ratio model than the linear model (p = , Wilcoxon signed-rank test). A pair of response gain ratios ððgð; pþ=gðp; ÞÞ; ðgðp; pþ=gðp; ÞÞÞ was computed from the fits to stimulus conditions in which both modalities were presented at the heading preferences of the neuron (e.g., Figure 4A). An analogous pair of response gain ratios ððgð; nþ=gðp; ÞÞ; ðgðp; nþ=gðp; Þ ÞÞwas computed from conditions in which one modality was presented at the preferred heading and the other modality at a non-preferred heading (e.g., Figure 4B). These two ratios were used to determine whether a neuron exhibited the diagnostic form of cross-modal suppression predicted by the normalization model (Figures 1C and 4C). For Figure 5C, G vestibular and G visual denote the gains for the unisensory vestibular cue and the preferred (fixed) visual cue, respectively. G combined indicates the response gain for the combined cues. Post hoc classification of unisensory neurons During preliminary testing, each MSTd neuron was classified as multisensory or unisensory, as described above. However, in some cases, this classification was tenuous based on the limited data in our screening test. Thus, for all MSTd neurons, we performed a post hoc analysis to test whether the response gain ratio G vestibular =G visual for the preferred vestibular heading was significantly greater than zero. From this analysis, 8 neurons that were provisionally classified as multisensory were not found to be significantly activated by vestibular stimuli at the preferred heading. These neurons were thus re-classified as unisensory neurons. On the other hand, 7 neurons that were provisionally classified as unisensory were found to be significantly activated at some of the headings tested. These neurons were re-classified as multisensory neurons. Thus, there were 33 unisensory MSTd neurons in our sample (Figure 5C), and the remaining 68 neurons were multisensory. This reclassification had no substantive effect on our main findings. Divisive normalization model of multisensory integration in MSTd We used a modified version of the MSTd model described previously (Ohshiro et al., 211). Briefly, the multisensory input to the j th model MSTd neuron is expressed as a weighted linear sum of the vestibular and visual inputs: L j ðc vest ; c vis Þ = d vest:j ðc vest Þ n1:j S vest:j + d vis:j ðc vis Þ n2:j S vis:j + base j : (Equation 2) In this equation, d vest.j and d vis.j represent the modality dominance weights of each neuron. The modality dominance weights, which take non-negative values, determine the relative strengths of the vestibular and visual responses of a model neuron. c vest and c vis denote the vestibular and visual stimulus amplitudes, whereas n 1.j and n 2.j are exponents for the power-law non-linearity relating these physical quantities with neural activity. base j is a positive constant, which was not included in the previously published version (Ohshiro et al., 211), to model the non-zero baseline activity of MSTd neurons. It is assumed to be constant over the experiment and unique to each neuron. S vest.j and S vis.j represent the tuning functions of vestibular and visual inputs to the multisensory neuron (Figure 1A). The heading tuning function of the vestibular input is modeled as S vest:j = n:j 1 + cosbvest:j ; (Equation 3) 2 where B vest:j represents the angle (in 3D) between the vestibular heading preference of the neuron and the vestibular stimulus heading. S vest.j is a bell-shaped function, symmetric around its peak at B vest:j =. B vest:j can be expressed in terms of azimuth ðb4 vest:j Þ and elevation ð b q vest:j Þ components of the heading preference, as well as azimuth ð4 vest Þ and elevation ðq vest Þ components of the stimulus: = arccos bhvest:j,h vest ; (Equation 4) B vest:j Neuron 95, e1 e8, July 19, 217 e4

19 where b Hvest:j = ½cos b q vest:j cosb4 vest:j ; cos b q vest:j sinb4 vest:j ; sin b q vest:j Š and H vest = ½cos q vest cos 4 vest ; cos q vest sin 4 vest ; sinq vest Š. The dot operator, denotes the inner product of the two vectors. S vis:j, b4 vis:j and b q vis:j were defined analogously. n.j controls the width of the tuning surface around its peak, without changing the height of the peak. The larger (smaller) the value is, the narrower (wider) the width. For the simulation results shown in Figure 1, we assumed a homogeneous population of neurons: base was fixed at.3, n 1 and n 2 were set to 1, and n was set to 2 for all neurons. Each dominance weight parameter could take one of five possible values [1.,.75,.5,.25 or.]. Other than these parameters, our model was designed to roughly mimic known neurobiological properties of MSTd neurons. We incorporated the experimental observations that vestibular and visual heading preferences are often mismatched, and that there are more neurons tuned to lateral self-motion than fore-aft motion (Gu et al., 26). Specifically, two random vector variables, ðb4 vest; b qvest Þ and ðb4 vis; b qvis Þ were generated to mimic the experimentally observed distributions of heading preferences, then these preference vectors were paired randomly (124 pairs). We subsequently added 265 pairs with congruent heading tuning and 256 pairs with opposite tuning to better mimic the experimentally observed joint distribution of heading preferences in MSTd neurons (Gu et al., 26). For each distinct combination of visual and visual heading preferences, there were = 25 possible combinations of modality dominance weights. Combining these factors, a population of units constituted the model of Figure 1 (1545 heading preference combinations x 25 dominance weight combinations). The output of each model neuron was a divisively normalized version of its activity (Heeger, 1992b; Ohshiro et al., 211), given by ðl j Þ n3:j r j = R max:j a j + e j P N N k = 1 ðl n3:k: (Equation 5) kþ In this equation, N ( = 38625) denotes the number of neurons in the population, R max.j is the maximum firing rate of the j th model neuron, n 3. j is the exponent for the power-law nonlinearity that relates membrane potential and firing rate, and a j is the semi-saturation constant. Note that a j causes responses to rise gradually and saturate as a function of stimulus intensity. This semi-saturation constant is a common feature of divisive normalization models going back to Heeger (1992b). Finally, e j is a parameter introduced here to make the formulation of the normalization model consistent with the alternative (subtractive) model described below; it determines how much normalizing effect the population activity has on each model neuron. After a mathematical transformation of Equation 5, we obtain the following alternative description of the normalization model: ( 1 P!) N r k 1 k R r j = R max:j a j + e j max:k a k 1 P 1 N r k ðl j Þ n3:j : (Equation 6) k R max:k e k Importantly, the output of the j th neuron is now expressed in terms of the final (normalized) output of other neurons (r k ), as opposed to being a function of the pre-normalized activity, L k, of other neurons as it was in Equation 5. This transformation allows us to model the observable responses of one neuron in terms of the observable responses of other neurons in the population. When R max.j, e j, ( = 1) and a j are assumed to be common to all neurons in the population, Equation 6 can be simplified into a form that provides further intuition: r j = 1 1 R a max X N r k ðl j Þ n3:j : (Equation 7) k From this form, one can easily recognize that the pre-normalized neural activity, (L j ) n3.j, is gain modulated by a term that decreases with the total population activity, P k r k (Carandini et al., 1997). Somatostatin-positive inhibitory interneurons may play a critical role in pooling local spiking activity and exerting suppression on nearby pyramidal neurons (Adesnik et al., 212). For the simulations in Figure 1B, R max.j is fixed somewhat arbitrarily at 1 spikes/s for j = 1 to N, n 3. j is fixed at 1., a j is fixed to.3, and e j is 1.. All of our conclusions are robust to substantial variations in these parameters. Data are shown for a model neuron with equal dominance weights for both modalities (d vest = d vis = 1.) and congruent heading preferences ðb4 vest = b4 vis = 9 ; b q vest = b q vis = Þ. The visual stimulus for the simulation was fixed at the neuron s preferred heading ð4 vis ; q vis Þ = ð9 ; Þ, while the vestibular stimulus heading was offset from the preferred heading by, 6, and 12 : ð4 vest ; q vest Þ = ð9 ; Þ, ð9 ; 6 Þ, and ð9 ; 12 Þ. As in the actual experiments, vestibular, visual and combined responses were simulated for the following stimulus amplitudes: c vest = c vis =.,.6, 1.6, 4.4 and For Figure 1C, the sampling interval of the vestibular stimulus heading was decreased to 15, and stimulus amplitude was fixed at the maximum value: c vest = c vis = Alternative (subtractive inhibition) model We contrasted predictions of the divisive normalization model with those made by a general family of alternative models that involve subtractive inhibitory interactions among neurons. The temporal dynamics of neural responses in the alternative model are given by (Ohshiro et al., 211) t dv j dt = V j + L j + ej X N N w j;k r k k = 1 r j = h j ðv j Þ: (Equation 8) e5 Neuron 95, e1 e8, July 19, 217

20 In this equation, V j represents the membrane potential of the j th neuron, and L j denotes the linear combination of sensory inputs defined in Equation 2. e j has a similar role to that included in the normalization model, and is necessary to couple two physically distinct quantities, the multisensory inputs and the population firing rate. h j (x)(r: N < x < N) is a static non-linearity which relates the membrane potential to the instantaneous firing rate of the neuron. For the static non-linearity, we used a sigmoidal function, h j ðxþ = R max:j ððxþ n3:j =ða j + ðxþ n3:j ÞÞ for xr, and h j ðxþ =, otherwise. a j and n 3. j in the denominator have similar roles to those in Equation 5. w j,k (=w k,j for any k and j) represents the synaptic weights for lateral interactions between the j th and k th multisensory neurons. Therefore, the term P k w j;k hðv k Þ represents the total input to the j th neuron from all other neurons via lateral connections. This term should be negative for the lateral interactions to be inhibitory. For simplicity, we simply assumed equal weights among neurons, irrespective of their heading selectivity, modality dominance, and congruency (i.e., w j,k = - 1 for all k and j; Ohshiro et al., 211). We note that equal weights are also assumed in the normalization pool, SL k (Equation 5), of the normalization model. Note that the number of the model parameters, R max.j, n 1.j, n 2.j, n 3.j, a j, base j, and e j is the same as in the normalization model (Equation 5), simplifying comparison of fitting performance between the two models, as described further below. Neural activity obtained at steady-state equilibrium (dv/dt =inequation 8) was used in all quantitative analyses of the alternative (subtractive) model: ej r j = hðv j:t/n Þ = h j L j + X N N w j;k r k : (Equation 9) k = 1 Then, we obtain the final simplified formula for the alternative model by substituting w j,k = 1: ej r j = h j L j X N N r k : (Equation 1) k = 1 Note here that a quantity proportional to the net population activity is subtracted from the multisensory input, L j, prior to transformation by the static non-linearity. This lies in stark contrast to the multiplicative effect of population activity in the normalization model (Equation 7). Equation 1 was solved using the MATLAB fsolve command. As in simulations of the normalization model, Figure 1C shows data for an alternative model neuron with modality dominance weights equal to 1. and congruent heading preferences. Stimulus amplitude was fixed at the maximum value: c vest = c vis = base. j was fixed at 5. e j, a j and n 3. j were set to 1, 8.7 and 1 (j = 1 to N), respectively. Cross-modal suppression in the alternative model Here, we examine the mathematical conditions necessary for the alternative model (Equation 9) to produce the diagnostic form of cross-modal suppression. For the treatment below, we assume (arbitrarily) that the vestibular stimulus is fixed at the preferred heading and that the visual stimulus is allowed to deviate from the cell s preference. We also assume that e j is constant across neurons (e j = e) and that the lateral connection weights, w j,k, are normalized by (e/n) for simplicity. To determine whether the visual input is activating or suppressive on its own, we compute the difference between the visual-only response (c vest =,c vis s inequation 2) and baseline activity (c vest =,c vis = ): r j ð; c vis Þ r j ð; Þ = h d vis:j,ðc vis Þ n2:j,s vis:j + base j + X w j;k r k ð; c vis Þ h base j + X w j;k r k ð; Þ : (Equation 11) k k Because h() is a monotonically increasing function, index1 defined below should be positive if the visual input is excitatory (i.e., Equation 11 > ): index1 = d vis:j,ðc vis Þ n2:j,s vis:j + X X w j;k r k ð; c vis Þ w j;k r k ð; Þ : (Equation 12) k k Similarly, index2 defined below should be negative if the combined cue response r j ðc vest ; c vis Þ is suppressed below the vestibularonly response (c vest s, c vis = ): index2 = d vis:j,ðc vis Þ n2:j,s vis:j + X X w j;k r k ðc vest ; c vis Þ w j;k r k ðc vest ; Þ : (Equation 13) k k Importantly, index2 is related to index1 according to index2 = index1 + X k w j;k add k ðc vest ; c vis Þ; (Equation 14) where the additivity term, add k, represents the difference between the combined response and the sum of unisensory responses (after removal of baseline activity): add k ðc vest ; c vis Þ = ðr k ðc vest ; c vis Þ r k ð; ÞÞ fðr k ðc vest ; Þ r k ð; ÞÞ + ðr k ð; c vis Þ r k ð; ÞÞg: (Equation 15) The term P k w j;k add k ðc vis ; c vest Þ in Equation 14 has a critical influence upon whether the alternative model exhibits the diagnostic form of cross-modal suppression. If P k w j;k add k ðc vis ; c vest Þ is positive, then the diagnostic type of cross-modal suppression Neuron 95, e1 e8, July 19, 217 e6

21 (index1 > and index2 < ) cannot occur. We show below that this term is very likely to be positive in the alternative model under the conditions in which MSTd neurons show the diagnostic form of cross-modal suppression. The additivity term, add k, determines whether the neuron integrates the multisensory cues super-additively ðadd k > Þ, additively ðadd k = Þ, or sub-additively ðadd k < Þ. To be consistent with the responses of real multisensory neurons and the principle of inverse effectiveness (Stein and Stanford, 28), the model needs to show subadditive integration for strong inputs; thus, it must be the case that add k ðc vest ; c vis Þ% for strong stimuli. Given that the alternative model involves subtractive inhibition, we expect that the weights, w j,k, will be largely negative. Indeed, these weights would have to be largely negative for model neurons to show cross-modal suppression at all. Combining this observation with the fact that responses should be subadditive for strong stimuli, we conclude that P k w j;k add k ðc vest ; c vis Þ must be positive for the alternative model when stimulus intensities are relatively strong. Thus, the alternative model should not be able to exhibit the diagnostic form of cross-modal suppression in this regime. The simplest possibility is that all weights are negative (w j;k %, for all j and k); however, in simulations we have found that this result is robust for a variety of weight structures, as long as weights are negative on balance. It should be noted that the above argument is true only for strong stimuli. The alternative model can show super-additive cue integration (add k > ) for weak stimuli, consistent with some multisensory neurons (Stein and Stanford, 28). Therefore, the diagnostic form of cross-modal suppression may occur in the alternative model for weak stimuli. However, this prediction lies in stark contrast to the prediction of the normalization model (Figure 1B) that cross-modal suppression should occur most clearly for strong stimuli. In addition, MSTd neurons generally show the diagnostic form of cross-modal suppression only for strong stimuli, which is incompatible with the prediction of the alternative model and in line with the divisive normalization model. Lastly, we note that whether the alternative model shows super-additive integration for weak stimuli depends on the form of the static non-linearity, h(x), in the model. The nonlinearity must be expansive for weak stimuli to produce super-additivity in that range of intensities. This mathematical argument demonstrates that the alternative model should not be able to account for cross-modal suppression for strong stimuli. This argument is borne out by the fits of both models to the neural population data (Figure 7), which show that the alternative model did not produce the diagnostic form of cross-modal suppression. Population curve-fit analysis To more directly assess whether data from MSTd are better explained by divisive than subtractive mechanisms, responses of MSTd neurons to congruent and incongruent multisensory cues (e.g., Figures 4A and 4B) were fit by both the normalization model (Equation 6) and the alternative (subtractive) model (Equation 1). The goal of this analysis was to find a set of model parameters that minimize the sum squared error across datasets from all of the recorded MSTd neurons: h bb1 ; b i 2 ;.:: b bj = min rj;m r b Xj;m 2 b j ; Const j ; Stim m : (Equation 16) Here, b j denotes model parameters for the j th neuron, r j;m denotes the measured response of the j th neuron to the stimulus condition indexed by m, and r ðb j ; Const j ; Stim m Þ is the corresponding model prediction for the same stimulus condition. Const j represents the neuron-specific constants including the preferred headings and dominance weights (see below). Stim m indicates the stimulus parameters for the m th stimulus condition, including the heading and amplitude of motion. For both the normalization and alternative models, there are nine model parameters and two tuning functions to be estimated for each MSTd neuron in the population: d vest.j, d vis.j, S vest.j, S vis.j,n 1.j, n 2.j, base j (Equation 2), R max.j, n 3.j, a j, and e j (Equations 6 and 1). Among these, the modality dominance weights, d vest.j, d vis.j and the two heading tuning functions, S vest.j and S vis.j were estimated directly for each neuron by fitting the visual and vestibular heading tuning profiles (e.g., Figure 2A) with Equation 2. Thus, these four quantities were treated as fixed constants that are unique to each neuron ðconst j Þ. The remaining seven parameters, R max.j, n 1.j, n 2.j, n 3.j, a j, base j, and e j, were free parameters to be optimized ðb j Þ by fitting the models to MSTd response curves obtained as a function of self-motion amplitude (e.g., Figures 4A and 4B). In total, 119 datasets from 11 neurons (68 multisensory neurons and 33 unisensory neurons) were fit with the models, as some cells contributed more than one dataset resulting from different combinations of visual and vestibular headings. As a result there were 7 (free-parameters) x 119 (datasets) = 833 total parameters that needed to be simultaneously optimized at each step in the fitting process. These 833 parameters were used to fit a total of 344 data points from all neurons in the population. The optimization was performed using the MATLAB lsqcurvefit command with the parameter boundary constraint option (all free parameters were bounded between zero and positive infinity). During each optimization step, r ðb j ; Const j ; Stim m Þ was computed by solving the system of non-linear equations (Equation 6 or 1) using the MATLAB fsolve command. We started the optimization from a case of all parameters initialized to 1.. The optimization was stopped when the lsqcurvefit optimizer returned the same sum squared error (SSE) for more than twenty optimization steps (tolerance:.1%). To avoid stalling in a local minimum, the optimization was then resumed after adding.1 to 1% random noise to all of the free parameters. This noise treatment typically resulted in.1% increase in SSE. Twenty such noise treatments were independently performed and the optimization proceeded to convergence for each noise treatment (same stopping criteria as above). If no improvement in SSE was seen in these 2 noise treatments (i.e., all resulted in larger SSEs than before the noise treatment), we decided that we had reached an optimal solution. Otherwise, we took the optimization with the lowest resulting SSE, we performed 2 new noise treatments, and we continued the optimization. e7 Neuron 95, e1 e8, July 19, 217

22 Ultimately, five cycles of noise treatments were necessary for the normalization model to converge. In the final cycle, the SSEs after each noise treatment barely decreased with iterations of the optimization, and were never below the lowest SSE obtained in previous cycles. Therefore, we adopted the result from the fourth cycle as the final fit (SSE = ). For the alternative (subtractive) model, nine cycles of noise treatments were needed to reach the final fit convergence (SSE = ). The optimization was performed using the MATLAB parallel-computing toolbox installed on three multi-core (8 or 4 cores) computers, running continuously for a total of 26,28 hr. Partial correlation coefficients between the fits of these two models were computed and z-scored using Fisher s r-to-z transform (Angelaki et al., 24). The resulting amplitude-response functions for each neuron in each model were fit to a hyperbolic ratio function (Equation 1), and the gain ratios were plotted for each model neuron in Figure 7. QUANTIFICATION AND STATISTICAL ANALYSIS Action potentials recorded in the interval from 5-2ms after stimulus onset were used to compute firing rates because most of the evoked neural activity occurred during this period (Figure S1)(Gu et al., 26). Neurons were included in our main sample if each distinct stimulus was successfully repeated at least five times in the main experiment (median repetition number: 8). All data analyses and statistical tests were done using custom scripts written in MATLAB. Analyses of population data were performed using appropriate parametric and non-parametric statistical tests (as described in the main text), including Wilcoxon rank sum tests, Wilcoxon signed rank tests, and ANOVA. Error bars in Figures 2, 3, 4, and 5 denote s.e.m. Exact p values and numbers of neurons for each statistical test are provided in the Results section. Bootstrap analysis was also used to test whether specific values (e.g., response gain ratios for particular neurons in Figures 4C and 5C) were significantly different from reference values. For assessing significance of gain ratios, bootstrap amplitude-response functions were generated by randomly resampling (with replacement) the error residues and adding them to the fitted curves (Efron and Tibshirani, 1993). The bootstrap amplitude-response functions were curve-fit and the gain parameters were estimated. This was repeated 3 times to produce bootstrap distributions of each gain ratio, from which the 5th and 95th percentiles were determined. A ratio was considered significantly greater (less) than a particular value (e.g., or 1), if its 5 th percentile (95 th percentile) was above (below) the value. A stepwise regression analysis of the population gain-ratio data (Figures 4C, 7A, 7B, S2, and S3) was performed using the MATLAB stepwiselm command, with a 3 rd order polynomial as a starting model. Fits of the gain-ratio data that are shown in Figures 4C, 7A, 7B, S2, and S3 correspond to the lowest-order polynomial that was justified by the stepwise regression (with entry and exit p values of.1 and.1, respectively). Partial correlation coefficients between model fits and the amplitude-response curves of individual neurons were computed and transformed to z-scores using Fisher s r-to-z transform (Angelaki et al., 24). This allowed us to define regions in Figure 6C for which one model fit is significantly superior to the other. No statistical methods were used to pre-determine appropriate sample sizes for the neural recordings, but our sample size is comparable to those generally employed in similar studies in the field. Experimenters were not blind to the purposes of the study, but all data collection was automated by computer. All stimulus conditions in the main experimental test were randomly interleaved. DATA AND SOFTWARE AVAILABILITY Program code (written in MATLAB) for reproducing the simulation results of Figures 1B and 1C is provided in Data S1. Other custom code for analyses will be provided upon request to the Lead Contact. Neuron 95, e1 e8, July 19, 217 e8

23 Neuron, Volume 95 Supplemental Information A Neural Signature of Divisive Normalization at the Level of Multisensory Integration in Primate Cortex Tomokazu Ohshiro, Dora E. Angelaki, and Gregory C. DeAngelis

24 SUPPLEMENTAL FIGURES AND LEGENDS Displacement (cm) a b c Velocity (cm/s) 2 1 Firing rate (spikes/s) Preferred Combined Non-preferred Difference Time (sec) Time (sec) Time (sec) Figure S1, related to Figure 2. Temporal dynamics of the motion stimulus and MSTd responses. (a) Displacement (i.e., position) of the motion platform as a function of time, illustrated for four different stimulus amplitudes (displacement = 11.8, 4.4, 1.6 and.6 cm). (b) Velocity of movement as a function of time for the displacements illustrated in panel a. (c) Average peri-stimulus time histograms of the multisensory responses of a sub-population of MSTd neurons that show the diagnostic form of cross-modal suppression (n = 25). Red: response to the preferred heading cue; cyan: response to the non-preferred heading cue; black: combined response to the preferred and nonpreferred heading stimuli; green: difference in response between the combined condition and the preferred heading cue. Error bands show +/- 1 s.e.m. Responses were analyzed over the period from 5-2 ms post stimulus onset (orange bar on the X axis). 1