Probing Sensory Readout via Combined Choice- Correlation Measures and Microstimulation Perturbation

Transcription

1 Article Probing Sensory Readout via Combined Choice- Correlation Measures and Microstimulation Perturbation Highlights d Sensory-choice congruent and opposite cells in the MST and MT contribute to motion perception d d d Causality of MST and MT cells can be inferred from sensory tuning, but not choice signal Readout weight of MST and MT cells is affected by sensorychoice congruency Congruent and opposite cells may be from noise correlation and unequal readout weight Authors Xuefei Yu, Yong Gu Correspondence guyong@ion.ac.cn In Brief Yu et al. measured the sensory component, choice component, and microstimulation perturbation effect on a site-to-site basis in multiple cortical areas and propose a scheme implementing reversed correlated noise and unequal readout weight to explain the heterogeneous sensory-choice relationships among cortices. Yu & Gu, 218, Neuron 1, November 7, 218 ª 218 Elsevier Inc.

2 Neuron Article Probing Sensory Readout via Combined Choice-Correlation Measures and Microstimulation Perturbation Xuefei Yu 1,2 and Yong Gu 1,3, * 1 Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China 2 University of Chinese Academy of Sciences, Beijing 149, China 3 Lead Contact *Correspondence: guyong@ion.ac.cn SUMMARY It is controversial whether covariation between neuronal activity and perceptual choice (i.e., choice correlation) reflects the functional readout of sensory signals. Here, we combined choice-correlation measures and electrical microstimulation on a site-to-site basis in the medial superior temporal area (MST), middle temporal area (MT), and ventral intraparietal area (VIP) when macaques discriminated between motion directions in both fine and coarse tasks. Microstimulation generated comparable effects between tasks but heterogeneous effects across and within brain regions. Within the MST and MT, microstimulation significantly biased an animal s choice toward the sensory preference instead of choicerelated signals of the stimulated units. This was particularly evident for sites with conflict preference of sensory and choice-related signals. In the VIP, microstimulation failed to produce significant effects in either task despite strong choice correlations presented in this area. Our results suggest that sensory readout may not be inferred from choice-related signals during perceptual decision-making tasks. INTRODUCTION Awareness of our world typically involves two critical steps: external stimuli represented by sensory neurons in the brain (i.e., encoding) and sensory information about the external stimuli subsequently extracted from the population activity of spiking neurons (i.e., decoding). Compared to encoding, the decoding process is much more poorly understood, although it has aroused much attention and debate in cognitive neuroscience for some time. One crucial way to address this issue is to look at choice-related signals during perceptual decision-making tasks designed in neurophysiological laboratories, such as two-alternative forced-choice (2-AFC) tasks (see reviews by Crapse and Basso, 215; Nienborg et al., 212; Nienborg and Cumming, 21; Parker and Newsome, 1998). Specifically, if perceptual judgments are based on spiking activity of sensory neurons, then fluctuations in the firing of those neurons under identical stimuli conditions are expected to account for the variation in the animal s behavioral choice. Such covariation between neural activity and perceptual choice on a trial-by-trial basis (i.e., choice probability) has been identified in many sensory areas in a variety of perceptual tasks (Britten et al., 1996; Dodd et al., 21; Gu et al., 27, 28; Hernández et al., 21; Jiang et al., 215; Liu and Newsome, 26; Liu et al., 213; Nienborg and Cumming, 26; Purushothaman and Bradley, 25). These choice correlations have been widely used to infer readout of sensory information by downstream brain areas and imply functional roles of certain sensory areas involved in perceptual tasks (Gu et al., 27, 28; Law and Gold, 28; Purushothaman and Bradley, 25; Sasaki and Uka, 29; Uka and DeAngelis, 24; Uka et al., 212). In addition to readout, choice correlations are also determined by the correlated variability of spiking activity among neurons (Crapse and Basso, 215; Cumming and Nienborg, 216; Pitkow et al., 215; Shadlen et al., 1996), the relations of which have been mathematically quantified in a recent computational study (Haefner et al., 213). However, all the rationales in these studies are within a bottom-up framework, assuming that the sensory signals contribute to the final perpetual decisions. On the contrary, other studies propose that choice-related signals in sensory areas may not reflect readout but instead may be top-down signals sent from higher levels, such as attention, learning, expectation, or other decision-related factors (Cohen and Newsome, 28; Dodd et al., 21; Goris et al., 217; Haefner et al., 216; Nienborg and Cumming, 29; Sasaki and Uka, 29). The debate about the source and implication of the choice correlation has been long lasting (see reviews by Crapse and Basso, 215; Cumming and Nienborg, 216), with much effort taken in theoretical studies (Haefner et al., 213, 216; Pitkow et al., 215; Shadlen et al., 1996; Wimmer et al., 215). In contrast, direct evidence from neurophysiological experiments is rare. In the current study, we approach this issue by combing two existing methods to examine the functional implications of choice correlations in sensory cortices. Specifically, we combined choice-correlation measurements and electrical microstimulation on a site-to-site basis in multiple sensory cortices, including Neuron 1, , November 7, 218 ª 218 Elsevier Inc. 715

3 A 1ms 1ms Fine task θ Left(-θ ) Coarse task Right(+θ ) could be explained and reproduced with two critical constraints: specific noise-signal correlation structure and readout weights for different types of neurons in different areas. Our results thus provide deeper insight into how sensory signals are possibly decoded by downstream neurons for perceptual decisionmaking. RESULTS B Proportion rightward choice 1..5 Fine task Monkey Y Monkey R 1..5 σ=1.8 σ= Motion direction ( ) Coherence% % Left (-Coherence%) C Monkey Y MST MT VIP Monkey R ips sts Right (+Coherence%) σ=3.2% σ=3.5% Coherence(%) the middle temporal area (MT), medial superior temporal area (MST), and ventral intraparietal area (VIP). We found that the relationship between choice-related signals and microstimulation effects is heterogeneous both across and within sensory cortices. Aided with simulations, these physiological results 1..5 sts Coarse task Monkey Y 1..5 Monkey R Electrode trace Figure 1. Illustration of the Behavioral Paradigm, Psychophysical Performance, and Recording Sites for the Two Animals (A) Events flow in behavioral tasks. Each trial started with a fixation target. After fixation, the monkey was presented with a visual motion stimulus for 1 s. Subsequently, the monkey made a saccade to one of the two targets to report his perceived motion (left versus right). In the fine task (top), the motion stimulus was expanding forward with different leftward or rightward components. In the coarse task (bottom), only planar motion (leftward or rightward) was presented, and the visual coherence was varied across trials. (B) Average psychometric functions for monkey Y and monkey R in the fine task and coarse task. The probability of rightward choice was plotted against the motion parameters (motion direction in the fine task; coherence in coarse task), and the data points were fitted with a cumulative Gaussian function. A zero on the x axis indicates the neutral condition, namely, forward motion in the fine task and coherence in the coarse task. Positive values represent rightward motion, and negative values represent leftward motion. Error bars represent SEM. (C) Magnetic resonance imaging (MRI) reconstruction of recording sites in the MST (blue dots), MT (red dots), and VIP (green dots) for monkey Y and monkey R. Notice that all recording sessions along the anterior-posterior axis were projected onto a single coronal plane using pyelectrode software (Daye et al., 213). ips Two monkeys were trained for a visual motion-direction-discrimination task in two different behavioral contexts. In the fine context, random-dots stimuli were expanded from the center of the screen at a fixed coherence level (1%). Task difficulty was controlled by motion direction varied in fine steps around the reference (forward motion toward the subject, i.e., )in the horizontal plane, leading to a small component of lateral motion (<8 ; Figure 1A, top panel). In contrast, in the coarse context, the motion direction was always fixed to be completely leftward or rightward, ±9 away from straight forward. The task difficulty was controlled by the motion coherence that varied between % and 16% (Figure 1A, bottom panel). In both tasks, monkeys reported motion directions of leftward or rightward by making an eye movement to one of the two targets appearing on both sides of the screen at the end of each trial. The correct choice led to a reward of liquids. Behavioral performance was quantified by psychometric functions in which the proportion of rightward choices was plotted as a function of the varied task parameters (fine task, motion direction; coarse task, coherence). After becoming well trained, the animals reached stable performance with negligible bias. The average threshold was 1.8 for both monkeys in the fine task (Figure 1B, left panels) and 3.2% for monkey Y and 3.5% for monkey R in the coarse task (Figure 1B, right panels). We then recorded and manipulated neural activity in multiple sensory cortical areas to examine how they covaried with the animals perceptual choice on a trial-by-trial basis in the two task contexts. We targeted three areas, the MST, MT, and VIP (Figure 1C), in which neurons are sensitive to visual motion cues, as reported previously in numerous studies (MST, Britten and Van Wezel, 22; Celebrini and Newsome, 1994; Duffy and Wurtz, 1995; Gu et al., 26, 27, 28, 21; Heuer and Britten, 24; MT, Born and Bradley, 25; Britten et al., 1993; Chowdhury et al., 29; Yu et al., 218; VIP, Bremmer, 211; Bremmer et al., 22; Chen et al., 211a, 211c, 213; Maciokas and Britten, 21; Zhang and Britten, 211; Zhang et al., 24). Measuring Choice Correlation and Microstimulation Effects We recorded a total of 751 units from all three areas (MST, n = 39; MT, n = 269; VIP, n = 173; Figure 1C) while the animals performed the two behavioral tasks (751 for the fine task and 195 for the coarse task). For each recording site, we first measured choice correlations and then applied electrical microstimulation. In order to fairly compare to the microstimulation effects that affect a local cluster of neurons, choice-correlation analysis of multi-unit activity (MUA) was conducted. However, we also analyzed single-unit activity (SUA) and confirmed that the choice correlations measured from both sources (MUA and SUA) were 716 Neuron 1, , November 7, 218

4 A Number of cases B Proportion rightward choices C Number of cases Fine task CP Coarse task Choice probability (CP) *** *** *** Motion direction ( ) null *** *** Microstimulation prefer Induced PSE shift ( ) *** *** MST MT VIP CP Control + Microstim Motion coherence(%) ** ** MST MT VIP null prefer Induced PSE shift (%coherence) Figure 2. Choice Correlation Quantified by Choice Probability and Microstimulation Effect across Cortices Regions and Behavioral Tasks Shown is MST (blue), MT (red), and VIP (green) in the fine (left column) and coarse task (right column). (A) CP distributions across areas and tasks. Black bars, p <.5; colored bars, p >.5 (permutation test). Arrows show mean values. Vertical dashed line indicates.5 (zero correlation). *p <.5; **p <.1; ***p <.1 (t test). (B) Example psychometric functions for control trials (without microstimulation; black symbols) and microstimulated trials (orange symbols). Insets show local tuning curves measured at the stimulated site before the microstimulation experiment. (C) Summary of the PSE shift induced by microstimulation across areas and tasks. Filled bar, p <.5; open bar, p >.5 (probit regression). Arrows represent mean value. *p <.5; **p <.1; ***p <.1 (t test). See also Figure S1. comparable (Figure S1C), which has also been indicated in previous studies (Carnevale et al., 213; Chen et al., 28; Shao et al., 218). Choice correlations were first quantified using the traditional metric of choice probability (CP; Britten et al., 1996) in both the fine and coarse tasks. A CP value significantly above.5 indicates the neuron fires more vigorously when the animal s upcoming choice is in the neuron s preferred direction. In contrast, a CP value significantly smaller than.5 indicates that the animal s choice is in the neuron s anti-preferred direction when the neuron fires more spikes, a phenomenon that is counterintuitive. As summarized in Figure 2A, mean CP tends to be larger than.5 in all areas in both tasks but with a few differences. First, across areas, the VIP contains much higher CP than the MST and MT in both the fine (MST,.514 ±.4; MT,.54 ±.3; VIP,.652 ±.9; mean ± SEM) and coarse tasks (MST,.528 ±.7; MT,.533 ±.9; VIP,.613 ±.18; mean ± SEM). Second, CPs measured during the two tasks are significantly correlated on a site-to-site basis (Figure S1A), with the overall magnitude being slightly larger in the coarse condition than in the fine condition. We next applied electrical stimulation with weak currents (amplitude, 2 ma, 2 Hz, biphasic; cathodal leading on 351 sites, MST, n = 138; MT, n = 138; VIP, n = 75) with choice correlations measured in advance. Figure 2B shows one example, in which microstimulation applied on a site (in the MST) with leftward motion preference drove the animal to make more left choices in both the fine (left panel) and coarse (right panel) tasks. Such an effect was quantified by the horizontal shift of the point of subjective equality (PSE) in the psychometric function. A positive sign implies that the choice bias is toward the stimulated neurons preferred motion direction (i.e., the expected direction), and vice versa for negative signs (i.e., the unexpected direction). In Figure 2B, for example, the induced PSE shifts have positive signs and are highly significant in both the fine (DPSE = 5.5, p = 1.6E 6, probit regression, left panel) and coarse tasks (DPSE = 7.6%, p = 3.8E 6, probit regression, right panel). Across populations, microstimulation significantly biased the animals choice toward the expected direction only in the MST and MT, but not in the VIP, in the fine (MST,.98 ±.14, p = 2.2E 7, n = 138; MT,.69 ±.11, p = 6.5E 9, n = 138; VIP,.1 ±.9, p =.89, n = 75; mean ± SEM, t test) and coarse tasks (MST, 3.83 ±.11, p =.1, n = 51; MT, 2.26 ±.74, p =.5, n = 27; VIP,.14 ±.26, p =.59, n = 29; mean ± SEM, t test; Figure 2C). In addition, the induced PSE shift is analogous between the two tasks (Figure S1B). These heterogeneous microstimulation effects observed across cortices could reflect divergent readout of sensory motion signals from these areas. Alternatively, this heterogeneity might be due to other factors that have limited the efficacy of the microstimulation effects. Two such important factors are the strength of sensory tuning and clustering. To test these possibilities, we first analyzed whether the microstimulation effects were dependent on the sensory motion signals as quantified by neural sensitivity (lgjd j; Figure 3A). We found that in the MST and MT, this dependency is highly significant in both tasks (fine task, MST, r =.25, p =.3; MT, r =.28, p = 8.2E 4; coarse task, MST, r =.55, p = 2.9E 5; MT, r =.38, p =.4; Neuron 1, , November 7,

5 A B Induced PSE shift, fine task ( ) MST r=.25, p=.3 r=.55, p=2.9e-5 r=-.6, p=.5 r=.1, p=.5 fine task coarse task Choice Probability (CP) Spearman s rank correlation), indicating that perturbing units with stronger sensory motion signals in these areas tends to affect the animals perceptual choice. In contrast, the microstimulation effect does not significantly depend on sensory tuning in the VIP (fine task, r =.12, p =.3; coarse task, r =.2, p =.9; Spearman s rank correlation). This result demonstrates that sensory motions signals are unlikely to explain the different microstimulation effects across cortices. Second, in terms of the clustering factor, all three areas show a fairly clustered structure of sensory motion signals (Figure S1D). Therefore, the dramatically different microstimulation effects among the cortices cannot be simply due to tuning and clustering factors. Rather, these experimental results suggest a divergent readout of the sensory motion signals in different areas (Figure 2C). In particular, sensory signals in the MST and MT are causally involved in the animals perceptual judgments, while those in the VIP contribute little to both the fine and coarse motion-direction-discrimination tasks. Notice that this pattern is in sharp contrast to the CP observed across areas (Figure 2A). We next examined the relationship between microstimulation effects and choice correlations on a site-by-site basis in each area. In contrast to the variable of neuronal sensitivity, CP failed to predict DPSE induced by microstimualtion in all areas (fine task, MST, r =.6, p =.5; MT, r =.3, p =.9; VIP, r =.4, p =.7; coarse task, MST, r =.1, p =.5; MT, r =.6, p =.7; VIP, r =.13,p=.5,ttest; Figure 3B). To examine the exact relationship among all three variables, including the microstimulation effect, neuronal sensitivity, and choice correlation, we ran a multiple linear regression analysis. In particular, we computed the partial correlation between DPSE and lgjd j by removing the CP effect, and the partial correlation between DPSE and CP by removing the lgjd j effect. This analysis showed that in MST and MT, microstimulation effect significantly depended on neural sensitivity (fine task, MST, r =.29; p = 6.6E 4; MT, r =.28; p = 7.7E 4; coarse, MST, r =.58; p = 6.6E 4; MT, r =.39; p =.4), but not on CP (fine task, MST, r =.18; p =.4; MT r=.28, p=8.2e-4 r=.38, p=.4 r=.3, p=.9 r=-.6, p= Neural sensitivity(lg( d )) r=.12, p=.3 r=.2, p=.9 r=-.4, p=.7 r=-.13, p=.5 VIP Induce d P SE shift, c oars e ts k ( %) Figure 3. Relationships among Neural Sensitivity, Choice Probability, and Microstimulation Effects on a Site-to-Site Basis (A) PSE shift induced from microstimulation as a function of neural sensitivity. (B) PSE shift induced from microstimulation as a function of choice-related activity quantified by CP. Solid lines, linear regression fit; black symbols, fine task; gray symbols, coarse task; filled symbols, p <.5; open symbols, p >.5; probit regression. MT, r =.14; p =.1; coarse task, MST, r =.2; p =.17; MT, r =.6; p =.75). The failure of using CP to predict microstimulation effects may have been due to a number of factors other than readout. The first concern is that choice correlations may not be as clustered as the sensory tuning functions. To address this issue, we measured both sensory tuning and choice correlations at neighboring MUA sites (1 mm apart in vertical distance) along each electrode penetration. Our results showed that similar to the tuning curves, choice correlations at neighboring sites were also fairly analogous within a range of 1 and 2 mm (Figure S1E), thus excluding the possibility of the clustering factor that may limit its relationship with the microstimulation effect. The second concern is due to the metric of CP itself, as pointed out in a recent study (Zaidel et al., 217). In particular, the sign of CP (i.e., above or below.5) could be problematic, as it is typically assigned based on the preferred direction of a particular neuron s tuning function. If this tuning function is measured while the animals perform perceptual decision-making tasks at the same time, it could be confounded by the choice-driven signals arising from a top-down source (Cumming and Nienborg, 216; Goris et al., 217). Thus, in the following, we recomputed choice correlation using a partial correlation analysis (Zaidel et al., 217) and reexamined its relationship with the microstimulation effect. Identifying the Choice-Driven Signal with Partial Correlation Analysis For any tuning curves measured during the discrimination tasks, the responses under each stimulus condition reflected confounding information from two components: (1) the external sensory stimuli of leftward or rightward motion and (2) the perceptual choice of leftward or rightward decision. We dissociated these two components using a method developed in a recent study (Zaidel et al., 217). Briefly, the responses were regressed as a linear combination of sensory and choice signals. A partial correlation analysis was then applied to reveal how much response variance can be accounted for by the sensory (r-sensory) component and the choice component (r-signal). The partial correlation coefficients of r-sensory and r-choice range from 1 to 1. Positive signs indicate rightward sensory motion preference 718 Neuron 1, , November 7, 218

6 A Firing rate (spks/s) B r-choice Sensory-choice congruent cell (SCCC) Motion direction ( ) Prefer R choice Prefer L choice 3 2 SCOC MST SCCC -1 Prefer L motion SU SU SCCC SCOC Prefer R motion MU MU fine task coarse task 6 4 MT r-sensory Sensory-choice opposite cell (SCOC) Choose left Choose right All choice Motion direction ( ) VIP Figure 4. Sensory and Choice Components of the Neural Activity Measured during the Motion Discrimination Tasks across Sensory Cortices (A) Choice and sensory signals could be either congruent (left; a single unit from the VIP) or opposite (right; a single unit from the MST). Black symbols represent average firing rates across all trials in each stimulus condition. Cyan symbols represent average firing rates under leftward choices. Magenta symbols represent average firing rates under rightward choices. Error bars denote SEM. (B) Scatterplots of sensory and choice partial correlation coefficients (r-sensory and r-choice, respectively). Colored symbols represent single units. MST, fine, n = 148; coarse, n = 36; MT, fine, n = 95; coarse, n = 6; VIP, fine, n = 79; coarse, n = 34. Black symbols represent multiple units. Circle, fine tasks; triangle, coarse tasks. (C and D) Comparison of the sensory component (C) and choice component (D) between the fine task and coarse task. Colored symbols represent single units. MST, n = 31; MT, n = 6; VIP, n = 32. Black symbols represent multiple units. MST, n = 77; MT, n = 37; VIP, n = 65. See also Figures S3 S5 and Tables S2 S4. C r-sensory, coarse task D r-choice, coarse task SU MU r-sensory, fine task r-choice, fine task -.5 or rightward perceptual choice, and negative signs indicate leftward sensory motion preference or leftward perceptual choice. Note that more complex models with nonlinear transformation of the stimulus variable (i.e., that linearizes the relationship between neural responses and the stimulus) or nonlinear interaction terms have also been applied, and these models produced results fairly analogous to those of the linear model (STAR Methods; Tables S2 S4). Figure 4A shows the average responses across all trials as a function of motion directions from two examples based on SUA during the fine task (black curves, Figure 4A), which appear to exhibit leftward motion preferences (left case, r =.93, p = 8.4E 4; right case, r =.79, p =.1; Pearson correlation). When replotting these responses according to the animal s perceptual choices, they become separated (Figure 4A, cyan and magenta curves for left and right choice, respectively). Specifically, for the first example, the sensory preference is indeed leftward, as evidenced by the activity in either the leftward or rightward choice group (cyan or magenta curve, respectively). The animal s choice in favor of the unit s motion preference (i.e., leftward) tends to enhance its activity (Figure 4A, left panel, cyan symbols), while choice in the null direction (i.e., rightward) tends to reduce its activity (Figure 4A, left panel, magenta symbols). Partial correlation analysis revealed significant sensory (r-sensory =.24, p = ) and choice component (r-choice =.56, p = ). Since the signs are the same for r-sensory and r-choice in this example, units like this case are defined as sensory-choice congruent cells (SCCCs). In contrast, in the second example (Figure 4A, right panel), the choice effect on the neural activity has an opposite trend compared to the first case, as the decision in the null direction (i.e., rightward) tends to enhance the unit s activity. The sign of partial correlation coefficients is different for r-sensory (.28, p = ) and r-choice (+.21, p =.6). This type of unit is thus defined as a sensory-choice opposite cell (SCOC) Neuron 1, , November 7,

7 Normalized PSE shift towards sensory preferer A MST B MT C MST+MT D VIP Figure 5. Comparison of Microstimulation * *** ** *** *** ***.5.5 ***.5.5 * ***. SCCC SCOC.. SCCC SCOC SCCC SCOC. SCCC SCOC Effect between Groups of SCCCs and SCOCs Microstimulation effect is quantified as the normalized PSE shift with the sign adjusted according to the sensory preference as indicated by the positive sign. (A) MST, (B) MT, (C) MST and MT, and (D) VIP. In (C), data from MST and MT were pooled together. The error bars denote SEM. *p <.5; ***p <.1; ***p <.1, t test. See also Figure S6 and Table S1. Figure 4B summarizes the population results of the sensory and choice signals based on SUA as well as MUA in all three areas in the fine and coarse tasks. In general, the results are similar for SUA and MUA, confirming clustered signals in these sensory cortices. Thus, in the following, the population analyses were mainly applied based on MUA in order to better compare them with microstimulation effects. For the sensory signals, the partial correlation coefficient of r-sensory is highly consistent with the metric of d (Figure S2B). For the choice signals, consistent with the results as quantified by CP (Figure 2A), r-choice is also more prevailing in the VIP than in the MST and MT (Figure 4B, vertical axis). More importantly, in addition to SCCCs (Figure 4B, 1 st and 3 rd quadrant), there are also many cases of SCOCs (Figure 4B, 2 nd and 4 th quadrant). Compared to the traditional CP metric, the two choice correlation measures turn out to be largely analogous with some outliers (Figure S2A). In particular, cases with CP <.5 (i.e., unexpected choice correlations) are predominantly SCOCs (3 rd quadrant), whereas cases with CP >.5 could be either SCCCs (a larger proportion, 1 st quadrant) or SCOCs (a smaller proportion, 2 nd quadrant). In other words, a modest population of cortical neurons with opposite sensory and choice modulations could have been grouped as positive choice correlations under the traditional measures (i.e., CP >.5) (Zaidel et al., 217). Numerous previous studies have reported that neurons that are more sensitive to stimulus variables tend to have higher CP values (e.g., Britten et al., 1996; Gu et al., 27, 28; Law and Gold, 28; Purushothaman and Bradley, 25; Uka and DeAngelis, 24). In these studies, neuronal sensitivity was typically quantified by neuronal threshold computed from ROC analysis (Britten et al., 1996). We also observed significant correlations between CP and neuronal threshold in all three cortical areas in both the fine (MST, r =.15, p =.1; MT, r =.14, p =.3; VIP, r =.71, p = ; Spearman rank correlation) and coarse tasks (MST, r =.36, p =.4; MT, r =.53, p =.1; VIP, r =.65, p = 6.E 7; Spearman rank correlation). However, such a trend was largely diminished for jr-sensoryj and r-choice (sign adjusted according to preferred motion direction), as calculated from the partial correlation analysis in the MST and MT (fine task, MST, r =.7, p =.18; MT, r =.9, p =.1; coarse task, MST, r =.12, p =.25; MT, r =.21, p =.18; Spearman rank correlation) but was preserved to some degree in the VIP (fine task, r =.31, p = 2.9E 5; coarse task, r =.3, p =.1; Spearman rank correlation). This result is qualitatively similar to that reported in the MST and VIP in a previous study (Zaidel et al., 217). Thus, there is a difference in the relationship between choice-related signals and neuronal sensitivity based on the two different metrics. One possibility is that the neuronal threshold calculated from the responses measured during the discrimination tasks may have been confounded by the choicerelated signals arising from a top-down source. A more detailed examination of the two classes of cells (SCCCs and SCOCs) from the partial correlation analysis indicated that they do not differ significantly in terms of the distribution of preferred motion directions, motion sensitivity, magnitude of r-sensory or r-choice (Figures S2C S2E), and clustering structure (Figure S3). Furthermore, the definitions of r-sensory and r-choice are robust and largely independent of a number of factors, including MUA/SUA (Figure S4), slow drifts in firing rates over time (Figure S5), psychophysical performance (Figures S2G and S2H), and behavioral tasks (Figure 4C). In fact, between the fine and coarse tasks, both r-sensory and r-choice were highly significantly correlated in all three areas (r-sensory, MST, r =.6, p = 8.2E 9; MT, r =.66, p = 7.4E 6; VIP, r =.65, p = 6.E 9; r-choice, MST, r =.71, p = 4.9E 13; MT, r =.63, p = 2.7E 5; VIP, r =.91, p = ; Pearson correlation coefficient). Thus, in the following analyses, data in the two tasks were pooled to acquire more statistical power. In particular, the microstimulation-induced PSE shift was divided by the behavioral threshold measured under the controlled trials in each session (Uka and DeAngelis, 26). Such a normalized quantity is unitless and allows for direct comparison between tasks. Relationship between r-sensory and r-choice and Microstimulation Effects We first compared the overall microstimulation effects in each group of cells (Figure 5). In the MST and MT, microstimualtion induced a significant PSE shift in both SCCCs (MST,.77 ±.15, p = 1.11E 6; MT,.62 ±.1, p = 3.87E 8; MST + MT,.7 ±.9, p = 1.52E 12; SEM, t test) and SCOCs (MST,.39 ±.11, p = 7.72E 4; MT,.22 ±.9, p =.1; MST + MT,.31 ±.7, p = 3.77E 5; SEM, t test; Figures 5A 5C; Table S1). Thus, the mean magnitude of the microstimulation effect from the SCCC group was approximately twice as large as that from the SCOC group, and this difference was statistically significant (MST, p =.3; MT, p =.5; MST + MT, p = 9.85E 4; t test). By contrast, microstimulation of neither group of VIP neurons induced a significant PSE shift (SCCCs,.7 ±.5; SCOCs,.6 ±.8; p >.15, t test; Figure 5D). 72 Neuron 1, , November 7, 218

8 A Normalized PSE shift B Normalized PSE shift R bias L bias R bias R bias L bias L bias R bias L bias MST Sensory-choice congruent cell (SCCC) Prefer L choice MT r-choice Prefer R choice This heterogeneous microstimulation effect across and within areas cannot be due to other factors, including sensory tuning and clustering structure (Figures S3 and 6). Thus, motion information from both the SCCC and SCOC groups is decoded by the brain for motion direction computation, yet SCOC contributes less than SCCC. We next examined relationships between microstimualtion effects and r-sensory/r-choice on a site-by-site basis (Figure 6). Because the signs of r-sensory and r-choice indicated leftward or rightward motion directions and perceptual choice, respectively, the sign of the induced PSE shift was also defined in the same format. Specifically, positive signs indicated more rightward choices induced by microstimulation, and negative signs indicated bias in the leftward choice. For the SCCC group in the MST and MT, both r-sensory and r-choice are significantly positively correlated with the microstimulation effect (i.e., DPSE; VIP r=.57, p=1.7e-9 r=.61, p=7.9e-1 r=-.1, p= r=.48, p=1.2e-6 r=.56, p=4.5e-8 r=-.4, p=.7 r=-.38, p=1.5e-6 Sensory-choice opposite cell (SCOC) 5 5 Prefer L motion Prefer L choice r-sensory Prefer L motion r-choice r-sensory Prefer R motion Prefer R choice Prefer R motion r=-.27, p= r=.49, p=3.7e-7 r=.38, p=3.7e-4 r=.24, p= r=-.28, p= Figure 6. Relationship between the Microstimulation Effect and Sensory and Choice Components on a Site-by-Site Basis for SCCCs and SCOCs (A) SCCCs. (B) SCOCs. Solid lines represent the linear regression fit. Filled symbols represent cases with a significant PSE shift (p <.5, probit regression). Open symbols represent p >.5. See also Figures S3 and 6. r-sensory, MST, r =.57, p = 1.7E 9; MT, r =.61, p = 7.9E 1; Figure 6A, top panels; r-choice, MST, r =.49, p = 3.7E 7; MT, r =.38, p = 3.7E 4; Figure 6A, bottom panels; Spearman rank correlation). For the SCOC group in the MST and MT, r-sensory is also significantly positively correlated with DPSE (MST, r =.48, p = 1.2E 6; MT, r =.56, p = 4.5E 8; Spearman rank correlation; Figure 6B, top panels), but r-choice shows an opposite trend and is significantly negatively correlated with DPSE (MST, r =.38, p = 1.5E 6; MT, r =.27, p =.1; Spearman rank correlation; Figure 6B, bottom panels). In contrast to the MST and MT, the correlation between microstimulation effects and r-sensory/r-choice is basically lacking in the VIP for either group of cells (SCCC, r-sensory, r =.1, p =.44; r-choice, r =.4, p =.7; SCOC, r-sensory, r =.24, p =.15; r-choice, r =.28, p =.9; Spearman rank correlation). These results demonstrate that the microstimulation effect in the MST and MT can be predicted from the sensory component in both groups of cells instead of from the choice-related signals. In fact, adding the choice component provides no additional information to help predict readout in either group (Figure S6). However, this does not mean that choice-related signals are not useful. In fact, they help identify the congruency of sensory and choice signals (SCCCs versus SCOCs), and this knowledge in turn predicts the magnitude of microstimulation effects for the neurons in each group (i.e., higher readout weight of SCCCs than SCOCs; Figure 5). Indeed, adding sensory-choice congruency (i.e., celltype information) instead of r-choice in addition to r-sensory significantly increases the explained variance of the readout (MST, p =.2; MT, p =.5; sequential F test; Figure S6). Possible Mechanisms Underlying Heterogeneous Choice Effects in the MST and MT Unexpected choice correlations have been found frequently in previous studies (Britten et al., 1996; Dodd et al., 21; Gu Neuron 1, , November 7,

9 A D r-choice Group 1 Group 2 +NC +NC 1 w=1 1 Decoder +NC +NC -knc +NC w =[:1] 2 +NC Group 1 Group 2 k=2 w 2/w 1=.5 +NC r-sensory +NC E Noise correlation B Prefer motion direction ( ) Preferred motion direction ( ) Model SCCC-SCCC.4 SCOC-SCOC SCOC-SCCC Signal correlation (Δ θ) F Noise correlation C Weight Model Group 1 Group 2 Experiment Figure 7. Hypothetical Mechanism that May Underlie Heterogeneous Choice Effects in the MST and MT (A) Illustration of the model structure. A decoder pools sensory information simultaneously from two classes of cells, which have identical tuning properties. The only difference lies in two aspects: (1) the two classes are interconnected, with a reversed relationship with respect to the regular signal- and noise-correlation structure; and (2) one class has a degraded readout weight. (B) Resultant noise covariance matrix. (C) In the model, the readout weight for one group is fixed at 1, while the weight for the other group is set as.5. (D) The model produces two clusters of cells on the 2-dimensional r-sensory and r-choice map. (E) Noise correlation is plotted as a function of the signal correlation (Dq) on a pair-by-pair basis. Pairwise units from different classes in (A) were labeled by color. Black, SCCC-SCCC; blue, SCOC-SCOC; red, SCCC-SCOC. (F) Signal- and noise-correlation structure based on pairwise units recorded simultaneously in the MST. See also Figure S7. et al., 27, 28; Jiang et al., 215; Liu et al., 213; Nienborg and Cumming, 26; Purushothaman and Bradley, 25; Xu et al., 214; Yu et al., 215; Zaidel et al., 217). Here, we propose a possible algorithm implemented with two key constraints that could reproduce the heterogeneous sensory-choice relationship observed in the MST and MT. Specifically, we hypothesized that (1) SCOCs are reversely correlated with SCCCs with respect to regular signal- and noise-correlation structure, and (2) the readout weight from SCOCs is relatively smaller compared to that of SCCCs. To test our hypothesis, we constructed a model in which the decoder pooled motion direction from a population of input neurons (Gu et al., 214). We randomly selected cells and divided them into two classes (group 1 and group 2, respectively; Figure 7A). The two classes of cells were identical in terms of tuning properties except that they were implemented with the two constraints mentioned above. First, within each class, noise correlation between each pair of units was typically assigned according to the difference in the sensory preference (Dq; Figure 7A, +NC ). However, this relationship was specifically reversed for units if they were in different classes (Figure 7A, knc ), producing some variability in the overall noise covariance matrix of the model units (Figure 7B). Thus, while group 1 s activity drives the decoder s choice in the direction favoring its preferred direction, group 2 s activity would be driven in the opposite direction through knc, leading to unexpected choice correlations for units in group 2. Note that in this case, the decoder would receive conflicting information from the two groups of cells regarding the voted motion direction. Thus, we further implemented a second constraint by scaling down the readout weight of group 2 s units relative to group 1 (w 2 and w 1, respectively; Figure 7A). We found that simultaneously varying parameters of knc and the w 2 /w 1 ratio generated qualitatively similar patterns that did not change the main conclusion (Figure S7). Thus, in the following, we fixed the two parameters to better demonstrate our simulation results. In particular, the weight ratio of w 2 /w 1 was set to.5 to approximate our physiological findings (Figure 7C versus Figures 5A 5C). As to the reversed noise correlation, k was arbitrarily set to 2. As shown in Figure 7D, the model generated two clusters of cells within the 2-dimensional sensory-choice space. The majority of group 1 s units exhibited congruent sensory-choice signs and were mainly within the 1 st and 3 rd quadrants. Meanwhile, the majority of group 2 s units exhibited opposite sensory-choice signs and were mainly within the 2 nd and 4 th quadrants (also see example tuning curves in Figures S7C and S7D). Other noise correlation structures (e.g., Bondy et al., 218) generated similar results and conclusions (Figures S7E S7J). Thus, by implementing the two critical constraints, our simulation is able to reproduce both SCCCs and SCOCs as observed in the physiological experiment. DISCUSSION By combining choice-correlation measures and electrical perturbation on a site-to-site basis, we evaluated how much the choice-related signals reflect readout in multiple cortical areas. We discovered that among areas, VIP neurons exhibited the highest choice correlations, yet microstimulation failed to produce significant effects on the animals behavioral choice in this area. In contrast, although MST and MT neurons exhibited modest choice correlations, microstimulation in these areas significantly biased the animals decisions in both the fine and coarse motion-direction-discrimination tasks. Notably, these perturbation effects can be predicted from the sensory signals instead of the choice-related signals, even for those sites with opposite sensory-choice preferences. Combined with 722 Neuron 1, , November 7, 218

10 simulations, our results suggest that choice correlations may arise from multiple sources, including top-down and specific noise correlation structures both across and within areas. Thus, one should be cautious when using choice-related signals to infer a functional readout of sensory signals, especially when exploring new brain regions. Potential Source of Choice-Related Activity Choice-related signals have been previously identified in multiple areas in a variety of tasks (Britten et al., 1996; Dodd et al., 21; Gu et al., 27, 28; Jiang et al., 215; Liu et al., 213; Nienborg and Cumming, 26; Purushothaman and Bradley, 25; Williams et al., 23). Generally, these signals are lacking in afferents (Yang et al., 216; Yu et al., 215) but begin to emerge and become stronger along the hierarchy of cortical stages, for example, from the primary sensory cortices to decision-related areas (de Lafuente and Romo, 26; Hernández et al., 21; Thiele et al., 1999; Thiele and Hoffmann, 1996; Williams et al., 23). The three areas we studied in the current work are mainly within the sensory domains; however, they are at a different hierarchical level. Among them, the MT is located closer to the sensory side and is considered to be at a mid-stage along the dorsal visual pathway that integrates motion information from earlier areas, such as the primary visual cortex (Albright, 1993; Andersen et al., 199; Born and Bradley, 25; Felleman and Van Essen, 1991; Mineault et al., 212; Orban et al., 1992). Responses from the MT could be further pooled by the MST for even higher-order motion processing, such as complex optic flow that simulates linear translation or rotation of the body as it navigates through the environment (Bremmer et al., 21; Britten, 28; Britten and Van Wezel, 22; Duffy and Wurtz, 1995; Gu et al., 26; Orban, 28; Yu et al., 218). In general, the VIP has neural properties very similar to those of the MST (Britten, 28; Maciokas and Britten, 21) but is sometimes considered to be closer to the decision centers (Bremmer et al., 22; Chen et al., 211c). Accordingly, the choice-related signals measured in our study are relatively weaker in the MST and MT and much stronger in the VIP. However, the hierarchy-dependent pattern of choice-related signals could be consistent with either a bottom-up sensory source or a top-down feedback mechanism. Plus, the strength of choice-related signals is also determined by the correlated noise among single neurons in a certain area in addition to their readout (Haefner et al., 213; Shadlen et al., 1996). Hence, the most direct way to identify the source of choice signals is to examine their causal roles. For example, chemical inactivation and lesion or electrical microstimulation in area MT or MST significantly affects the monkey s motion discriminability (Britten and Van Wezel, 1998, 22; Celebrini and Newsome, 1995; Chen et al., 216; Gu et al., 212; Nichols and Newsome, 22; Salzman et al., 199; Salzman et al., 1992; Yu et al., 218), suggesting that the previously observed choice-related signals observed in these areas may arise from a bottom-up sensory source. Another piece of evidence supporting this idea is from the results of inactivation in V2 and V3, the areas of which provide major bottom-up inputs about depth signals to the MT. After removing these inputs, the MT s choice-related signals during a depth-detection task are largely diminished (Smolyanskaya et al., 215). By contrast, recent inactivation of VIP activity failed to significantly influence the animal s perceptual judgments about motion direction, suggesting that the choice signals in this area may not be from a bottom-up source (Chen et al., 216). Although the previous studies may give hints about potential sources of choice-related activity in the sensory cortices, it is still unclear whether the signals measured at each recording site in each area are specifically related to readout. This is mainly because of two reasons. First, in majority of these studies, choice-related activity measures and causal operations have been conducted in different experiments and on different animals (e.g., Chen et al., 216), making it hard to directly compare the two effects under identical experimental conditions. Second, the method of chemical inactivation usually does not allow manipulation of neuronal activity within a local-enough range, making it impossible to associate the properties of the manipulated neurons with the choice effect. Thus, the methods in our current study have advantages in that they allow us to fairly compare the choice and readout effects not only across areas but also on a site-by-site basis within each area (see more discussion in the following section). SCCCs and SCOCs Using a new analytic procedure (Zaidel et al., 217), we identified two classes of cells that are frequently encountered in all three areas: SCCCs with enhanced neural activity when the animal s choice is in the neuron s preferred direction, and SCOCs with enhanced neural activity when the choice is opposite to the neuron s preferred direction. Previously, choice correlation was mostly quantified by CP via ROC analysis (Britten et al., 1996). CP ranges from to 1, and neurons with a CP above.5 roughly correspond to SCCCs, whereas neurons with a CP below.5 correspond to SCOCs (Figures 2 and S2A). Similar to the SCCC and SCOC pattern, a large proportion of cases with CP <.5 were found in previous studies. These cases are unexpected and puzzling, yet they are largely neglected, as researchers typically show the average values across all cases that are more or less larger than the chance level of.5 in many sensory cortices (see review by Smith et al., 212). Some recent studies have discussed the origin of these SCOCs, proposing that they might arise from correlations with other causal neurons (Chaisanguanthum et al., 217) or an anticipatory source (Goris et al., 217). However, none of these studies directly test whether these cells causally contribute to the decision process. In the current study, for the first time (to our knowledge), we have shown that similar to SCCCs, SCOCs (CP < approximately.5) in the MST and MT also causally contribute to the animal s behavioral performance, as revealed by microstimulation perturbation. Furthermore, in most cases, the microstimulation effect in SCOCs is consistent with the neuron s preferred direction, indicating that the neural activities are read out with respect to the sensory instead of choice signals. During r-choice and r-sensory measures (no-microstimulation condition), why is the spontaneous covariation between SCOC activity and the animal s choice opposite to the neuron s preferred direction? Using simulations, we show that SCOCs could be those units that contain reversed signal- and noise-correlation relationships with the other units (SCCCs) and are Neuron 1, , November 7,

11 decoded with degraded weights. The particular assumed noise correlation structure has not been measured practically and requires future verification. One interesting aspect of this assumption is that it would add more variability into the classical noisecorrelation structure (Figures 7E and 7F). Indeed, previous studies have found large variations in the signal- and noise-correlation relationship, which could depend on a number of factors, including signal correlations, physical distance of pairwise units (Rosenbaum et al., 217; Gawne et al., 1996; Zohary et al., 1994), or cortical states (Carnevale et al., 212; Cohen and Maunsell, 29; Cohen and Newsome, 28; Ecker et al., 214; Gu et al., 211). In addition, stimuli features such as disparity and motion parallax, as well as a different sensory modality like vestibular information, may also affect noise correlations (Gu et al., 26, 28; Nadler et al., 213; Sanada et al., 212; Yang et al., 211), and these factors are not always in agreement in terms of a neuron s stimulus preference. For example, in the dorsal portion of the MST (MSTd), there is a subpopulation of neurons with incongruent visual and vestibular direction preferences. Interestingly, these cells often exhibit reversed CP patterns (<.5) in the visual stimuli condition (Gu et al., 28), and their correlated noise with the other subpopulation of cells with congruent visual and vestibular direction preferences is also different from that within each subpopulation (Gu et al., 214). Similarly, readout from sensory signals may also be affected by the factors mentioned above. For example, in the MT during a visual motion direction discrimination task, the key factor is the direction signals rather than the disparity signals. However, readout of the direction signals for motion direction judgements could still be influenced by irrelevant disparity tuning of the neurons (DeAngelis and Newsome, 24). In our current work, microstimulation of the SCOC class induces a PSE shift approximately half of that produced by stimulating the SCCC class. However, the two classes of cells did not show any significant differences in terms of their basic tuning properties; thus, whether and how downstream neurons recognize different types of neurons is still unclear at this stage. Future experiments need to be conducted to verify and explore these hypotheses. In summary, by measuring choice correlation and applying microstimulation perturbation on a site-by-site basis in multiple sensory cortices, we discovered heterogeneous choice effects both across and within brain regions. Aided with computations, our method provides a better probe into the functional implications of sensory and choice signals in local circuits. Efficiency of the Microstimulation Technique We have measured choice-related activities from multiple units in each recording site and compared them with the microstimulation effect evoked from these sites. One major concern about this comparison is that microstimulation may impact neurons more distant from the stimulated sites. According to previous studies, electrical currents as small as 2 ma (as used in our experiment) can affect neural activity a few hundred microns (3 mm) around the electrode tip (Histed et al., 29; Murasugi et al., 1993; Stoney et al., 1968; Yu et al., 218). In macaque sensory cortices, visual motion signals are typically clustered within this range (Born and Bradley, 25; Britten and van Wezel, 1998; Chen et al., 28; Dubner and Zeki, 1971; Gu et al., 211; Shao et al., 218; Tanaka et al., 1986). Indeed, in our experiment, neuronal tunings are highly similar between neighboring sites 1 or 2 mm apart, and the microstimulation-induced PSE shift is largely predictable based on the tuning properties of the stimulated sites in the MST and MT. In addition, our experiment revealed that similar to the sensory signals, choice-related signals also tend to be clustered in each brain area. Thus, although the electrical current spread is inevitable, we believe that our microstimulation, when applied in these brain regions, can fairly indicate readout of motion signals from local clusters. Compared to the MST and MT, the microstimulation effect is basically lacking in the VIP. We have excluded factors that may have limited microstimulation efficiency, including the tuning strength or clustering factors. Therefore, these results seem to suggest that motion signals in the VIP are processed by the brain in a different way from those in the superior temporal sulcus. At the same time, a question remains: why is neuronal activity in the VIP highly correlated with the animals choice? Compared to other sensory areas, VIP neurons appear to convey a large variety of signals, including visual (Chen et al., 211b, 211c), vestibular (Chen et al., 211b, 211c, 213), auditory (Schlack et al., 25), somatosensory (Cooke et al., 23), and oculomotor signals (Bremmer, 211; Zhang and Britten, 211). Simply because these signals are correlated with choice (as they easily could be) does not mean that the neurons have a causal role in forming the perceptual decision. In summary, it may go a long way to understand the exact functional implications for the high choice correlations observed in the VIP. Future experiments with different task paradigms (e.g., accompanied smooth pursuit eye movements; Zhang and Britten, 211) need to be conducted to understand the exact functional roles of the VIP. STAR+METHODS Detailed methods are provided in the online version of this paper and include the following: d KEY RESOURCES TABLE d CONTACT FOR REAGENT AND RESOURCE SHARING d EXPERIMENTAL MODEL AND SUBJECT DETAILS d METHOD DETAILS B Behavioral task procedures B Apparatus B Visual Stimuli B Electrophysiological recordings B Microstimulation Procedures d QUANTIFICATION AND STATISTICAL ANALYSIS B Neural sensitivity B Direction Discrimination Index (DDI) B CP B Isolation of single unit and MUA B Comparison of microstimulation effects between fine task and coarse task B Multiple linear regression and partial correlation B Detrending procedure B Simulation d DATA AND SOFTWARE AVAILABILITY 724 Neuron 1, , November 7, 218

12 SUPPLEMENTAL INFORMATION Supplemental Information includes seven figures and four tables and can be found with this article online at ACKNOWLEDGMENTS We thank Ralf M. Haefner for constructive suggestions for modeling, Wenyao Chen for monkey care and training, and Ying Liu for software programming. We thank the students and staff in computation and cognitive neuroscience summer school 216 for suggestions regarding modeling and simulation. This work was supported by the National Natural Science Foundation of China Project (grants and ), the National Key Basic Research Project (216YFC13681), the Strategic Priority Research Program of CAS (XDBS1721), and the Shanghai Key Basic Research Project (16JC14221). AUTHOR CONTRIBUTIONS X.Y. and Y.G. conceived the project and designed the experiments. X.Y. implemented the experiments and performed data analyses. X.Y. and Y.G. wrote the manuscript. DECLARATION OF INTERESTS The authors declare no competing interests. Received: May 26, 217 Revised: January 19, 218 Accepted: August 22, 218 Published: September 2, 218 REFERENCES Albright, T.D. (1993). Cortical processing of visual motion. Rev. Oculomot. Res. 5, Andersen, R.A., Snowden, R.J., Treue, S., and Graziano, M. (199). Hierarchical processing of motion in the visual cortex of monkey. Cold Spring Harb. Symp. Quant. Biol. 55, Bondy, A.G., Haefner, R.M., and Cumming, B.G. (218). Feedback determines the structure of correlated variability in primary visual cortex. Nat. Neurosci. 21, Born, R.T., and Bradley, D.C. (25). Structure and function of visual area MT. Annu. Rev. Neurosci. 28, Bremmer, F. (211). Multisensory space: from eye-movements to self-motion. J. Physiol. 589, Bremmer, F., Duhamel, J.R., Ben Hamed, S., and Graf, W. (22). Heading encoding in the macaque ventral intraparietal area (VIP). Eur. J. Neurosci. 16, Bremmer, F., Kubischik, M., Pekel, M., Hoffmann, K.P., and Lappe, M. (21). Visual selectivity for heading in monkey area MST. Exp. Brain Res. 2, Britten, K.H. (28). Mechanisms of self-motion perception. Annu. Rev. Neurosci. 31, Britten, K.H., and van Wezel, R.J. (1998). Electrical microstimulation of cortical area MST biases heading perception in monkeys. Nat. Neurosci. 1, Britten, K.H., and Van Wezel, R.J. (22). Area MST and heading perception in macaque monkeys. Cereb. Cortex 12, Britten, K.H., Shadlen, M.N., Newsome, W.T., and Movshon, J.A. (1993). Responses of neurons in macaque MT to stochastic motion signals. Vis. Neurosci. 1, Britten, K.H., Newsome, W.T., Shadlen, M.N., Celebrini, S., and Movshon, J.A. (1996). A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci. 13, Carnevale, F., de Lafuente, V., Romo, R., and Parga, N. (212). Internal signal correlates neural populations and biases perceptual decision reports. Proc. Natl. Acad. Sci. USA 19, Carnevale, F., de Lafuente, V., Romo, R., and Parga, N. (213). An optimal decision population code that accounts for correlated variability unambiguously predicts a subject s choice. Neuron 8, Celebrini, S., and Newsome, W.T. (1994). Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. J. Neurosci. 14, Celebrini, S., and Newsome, W.T. (1995). Microstimulation of extrastriate area MST influences performance on a direction discrimination task. J. Neurophysiol. 73, Chaisanguanthum, K.S., Shen, H.H., and Sabes, P.N. (217). Neural representation and causal models in motor cortex. J. Neurosci. 37, Chen, A., Gu, Y., Takahashi, K., Angelaki, D.E., and Deangelis, G.C. (28). Clustering of self-motion selectivity and visual response properties in macaque area MSTd. J. Neurophysiol. 1, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (211a). A comparison of vestibular spatiotemporal tuning in macaque parietoinsular vestibular cortex, ventral intraparietal area, and medial superior temporal area. J. Neurosci. 31, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (211b). Convergence of vestibular and visual self-motion signals in an area of the posterior sylvian fissure. J. Neurosci. 31, Chen, A., DeAngelis, G.C., and Angelaki, D.E. (211c). Representation of vestibular and visual cues to self-motion in ventral intraparietal cortex. J. Neurosci. 31, Chen, A., Deangelis, G.C., and Angelaki, D.E. (213). Functional specializations of the ventral intraparietal area for multisensory heading discrimination. J. Neurosci. 33, Chen, A., Gu, Y., Liu, S., DeAngelis, G.C., and Angelaki, D.E. (216). Evidence for a causal contribution of macaque vestibular, but not intraparietal, cortex to heading perception. J. Neurosci. 36, Chowdhury, S.A., Takahashi, K., DeAngelis, G.C., and Angelaki, D.E. (29). Does the middle temporal area carry vestibular signals related to self-motion? J. Neurosci. 29, Cohen, M.R., and Maunsell, J.H. (29). Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, Cohen, M.R., and Newsome, W.T. (28). Context-dependent changes in functional circuitry in visual area MT. Neuron 6, Cooke, D.F., Taylor, C.S., Moore, T., and Graziano, M.S. (23). Complex movements evoked by microstimulation of the ventral intraparietal area. Proc. Natl. Acad. Sci. USA 1, Crapse, T.B., and Basso, M.A. (215). Insights into decision making using choice probability. J. Neurophysiol. 114, Cumming, B.G., and Nienborg, H. (216). Feedforward and feedback sources of choice probability in neural population responses. Curr. Opin. Neurobiol. 37, Daye, P.M., Monosov, I.E., Hikosaka, O., Leopold, D.A., and Optican, L.M. (213). pyelectrode: an open-source tool using structural MRI for electrode positioning and neuron mapping. J. Neurosci. Methods 213, de Lafuente, V., and Romo, R. (26). Neural correlate of subjective sensory experience gradually builds up across cortical areas. Proc. Natl. Acad. Sci. USA 13, DeAngelis, G.C., and Newsome, W.T. (24). Perceptual read-out of conjoined direction and disparity maps in extrastriate area MT. PLoS Biol. 2, E77. DeAngelis, G.C., and Uka, T. (23). Coding of horizontal disparity and velocity by MT neurons in the alert macaque. J. Neurophysiol. 89, Dodd, J.V., Krug, K., Cumming, B.G., and Parker, A.J. (21). Perceptually bistable three-dimensional figures evoke high choice probabilities in cortical area MT. J. Neurosci. 21, Neuron 1, , November 7,

13 Dubner, R., and Zeki, S.M. (1971). Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey. Brain Res. 35, Duffy, C.J., and Wurtz, R.H. (1995). Response of monkey MST neurons to optic flow stimuli with shifted centers of motion. J. Neurosci. 15, Ecker, A.S., Berens, P., Cotton, R.J., Subramaniyan, M., Denfield, G.H., Cadwell, C.R., Smirnakis, S.M., Bethge, M., and Tolias, A.S. (214). State dependence of noise correlations in macaque primary visual cortex. Neuron 82, Felleman, D.J., and Van Essen, D.C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, Gawne, T.J., Kjaer, T.W., Hertz, J.A., and Richmond, B.J. (1996). Adjacent visual cortical complex cells share about 2% of their stimulus-related information. Cereb. Cortex 6, Goris, R.L.T., Ziemba, C.M., Stine, G.M., Simoncelli, E.P., and Movshon, J.A. (217). Dissociation of choice formation and choice-correlated activity in macaque visual cortex. J. Neurosci. 37, Gu, Y., Watkins, P.V., Angelaki, D.E., and DeAngelis, G.C. (26). Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area. J. Neurosci. 26, Gu, Y., DeAngelis, G.C., and Angelaki, D.E. (27). A functional link between area MSTd and heading perception based on vestibular signals. Nat. Neurosci. 1, Gu, Y., Angelaki, D.E., and Deangelis, G.C. (28). Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci. 11, Gu, Y., Fetsch, C.R., Adeyemo, B., Deangelis, G.C., and Angelaki, D.E. (21). Decoding of MSTd population activity accounts for variations in the precision of heading perception. Neuron 66, Gu, Y., Liu, S., Fetsch, C.R., Yang, Y., Fok, S., Sunkara, A., DeAngelis, G.C., and Angelaki, D.E. (211). Perceptual learning reduces interneuronal correlations in macaque visual cortex. Neuron 71, Gu, Y., Deangelis, G.C., and Angelaki, D.E. (212). Causal links between dorsal medial superior temporal area neurons and multisensory heading perception. J. Neurosci. 32, Gu, Y., Angelaki, D.E., and DeAngelis, G.C. (214). Contribution of correlated noise and selective decoding to choice probability measurements in extrastriate visual cortex. elife 3, e267. Haefner, R.M., Gerwinn, S., Macke, J.H., and Bethge, M. (213). Inferring decoding strategies from choice probabilities in the presence of correlated variability. Nat. Neurosci. 16, Haefner, R.M., Berkes, P., and Fiser, J. (216). Perceptual decision-making as probabilistic inference by neural sampling. Neuron 9, Hernández, A., Nácher, V., Luna, R., Zainos, A., Lemus, L., Alvarez, M., Vázquez, Y., Camarillo, L., and Romo, R. (21). Decoding a perceptual decision process across cortex. Neuron 66, Heuer, H.W., and Britten, K.H. (24). Optic flow signals in extrastriate area MST: comparison of perceptual and neuronal sensitivity. J. Neurophysiol. 91, Histed, M.H., Bonin, V., and Reid, R.C. (29). Direct activation of sparse, distributed populations of cortical neurons by electrical microstimulation. Neuron 63, Jazayeri, M., and Movshon, J.A. (26). Optimal representation of sensory information by neural populations. Nat. Neurosci. 9, Jiang, Y., Yampolsky, D., Purushothaman, G., and Casagrande, V.A. (215). Perceptual decision related activity in the lateral geniculate nucleus. J. Neurophysiol. 114, Law, C.T., and Gold, J.I. (28). Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area. Nat. Neurosci. 11, Liu, J., and Newsome, W.T. (26). Local field potential in cortical area MT: stimulus tuning and behavioral correlations. J. Neurosci. 26, Liu, S., Gu, Y., DeAngelis, G.C., and Angelaki, D.E. (213). Choice-related activity and correlated noise in subcortical vestibular neurons. Nat. Neurosci. 16, Maciokas, J.B., and Britten, K.H. (21). Extrastriate area MST and parietal area VIP similarly represent forward headings. J. Neurophysiol. 14, Mineault, P.J., Khawaja, F.A., Butts, D.A., and Pack, C.C. (212). Hierarchical processing of complex motion along the primate dorsal visual pathway. Proc. Natl. Acad. Sci. USA 19, E972 E98. Murasugi, C.M., Salzman, C.D., and Newsome, W.T. (1993). Microstimulation in visual area MT: effects of varying pulse amplitude and frequency. J. Neurosci. 13, Murphey, D.K., and Maunsell, J.H. (27). Behavioral detection of electrical microstimulation in different cortical visual areas. Curr. Biol. 17, Nadler, J.W., Barbash, D., Kim, H.R., Shimpi, S., Angelaki, D.E., and DeAngelis, G.C. (213). Joint representation of depth from motion parallax and binocular disparity cues in macaque area MT. J. Neurosci. 33, , 1474a. Nichols, M.J., and Newsome, W.T. (22). Middle temporal visual area microstimulation influences veridical judgments of motion direction. J. Neurosci. 22, Nienborg, H., and Cumming, B.G. (26). Macaque V2 neurons, but not V1 neurons, show choice-related activity. J. Neurosci. 26, Nienborg, H., and Cumming, B.G. (29). Decision-related activity in sensory neurons reflects more than a neuron s causal effect. Nature 459, Nienborg, H., and Cumming, B. (21). Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron s causality? Curr. Opin. Neurobiol. 2, Nienborg, H., Cohen, M.R., and Cumming, B.G. (212). Decision-related activity in sensory neurons: correlations among neurons and with behavior. Annu. Rev. Neurosci. 35, Orban, G.A. (28). Higher order visual processing in macaque extrastriate cortex. Physiol. Rev. 88, Orban, G.A., Lagae, L., Verri, A., Raiguel, S., Xiao, D., Maes, H., and Torre, V. (1992). First-order analysis of optical flow in monkey brain. Proc. Natl. Acad. Sci. USA 89, Parker, A.J., and Newsome, W.T. (1998). Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, Pitkow, X., Liu, S., Angelaki, D.E., DeAngelis, G.C., and Pouget, A. (215). How can single sensory neurons predict behavior? Neuron 87, Prince, S.J., Pointon, A.D., Cumming, B.G., and Parker, A.J. (22). Quantitative analysis of the responses of V1 neurons to horizontal disparity in dynamic random-dot stereograms. J. Neurophysiol. 87, Purushothaman, G., and Bradley, D.C. (25). Neural population code for fine perceptual decisions in area MT. Nat. Neurosci. 8, Rosenbaum, R., Smith, M.A., Kohn, A., Rubin, J.E., and Doiron, B. (217). The spatial structure of correlated neuronal variability. Nat. Neurosci. 2, Salzman, C.D., Britten, K.H., and Newsome, W.T. (199). Cortical microstimulation influences perceptual judgements of motion direction. Nature 346, Salzman, C.D., Murasugi, C.M., Britten, K.H., and Newsome, W.T. (1992). Microstimulation in visual area MT: effects on direction discrimination performance. J. Neurosci. 12, Sanada, T.M., Nguyenkim, J.D., and Deangelis, G.C. (212). Representation of 3-D surface orientation by velocity and disparity gradient cues in area MT. J. Neurophysiol. 17, Sasaki, R., and Uka, T. (29). Dynamic readout of behaviorally relevant signals from area MT during task switching. Neuron 62, Schlack, A., Sterbing-D Angelo, S.J., Hartung, K., Hoffmann, K.P., and Bremmer, F. (25). Multisensory space representations in the macaque ventral intraparietal area. J. Neurosci. 25, Neuron 1, , November 7, 218

14 Shadlen, M.N., Britten, K.H., Newsome, W.T., and Movshon, J.A. (1996). A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, Shao, M., DeAngelis, G.C., Angelaki, D.E., and Chen, A. (218). Clustering of heading selectivity and perception-related activity in the ventral intraparietal area. J. Neurophysiol. 119, Smith, J.E.T., Masse, N.Y., Zhan, C.A., and Cook, E.P. (212). Linking neural activity to visual perception: separating sensory and attentional contributions. In Visual Cortex: Current Status and Perspectives, S. Molotchnikoff and J. Rouat, eds. (IntechOpen), pp Smolyanskaya, A., Haefner, R.M., Lomber, S.G., and Born, R.T. (215). A modality-specific feedforward component of choice-related activity in MT. Neuron 87, Stoney, S.D., Jr., Thompson, W.D., and Asanuma, H. (1968). Excitation of pyramidal tract cells by intracortical microstimulation: effective extent of stimulating current. J. Neurophysiol. 31, Takahashi, K., Gu, Y., May, P.J., Newlands, S.D., DeAngelis, G.C., and Angelaki, D.E. (27). Multimodal coding of three-dimensional rotation and translation in area MSTd: comparison of visual and vestibular selectivity. J. Neurosci. 27, Tanaka, K., Hikosaka, K., Saito, H., Yukie, M., Fukada, Y., and Iwai, E. (1986). Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey. J. Neurosci. 6, Thiele, A., and Hoffmann, K.P. (1996). Neuronal activity in MST and STPp, but not MT changes systematically with stimulus-independent decisions. Neuroreport 7, Thiele, A., Distler, C., and Hoffmann, K.P. (1999). Decision-related activity in the macaque dorsal visual pathway. Eur. J. Neurosci. 11, Uka, T., and DeAngelis, G.C. (24). Contribution of area MT to stereoscopic depth perception: choice-related response modulations reflect task strategy. Neuron 42, Uka, T., and DeAngelis, G.C. (26). Linking neural representation to function in stereoscopic depth perception: roles of the middle temporal area in coarse versus fine disparity discrimination. J. Neurosci. 26, Uka, T., Sasaki, R., and Kumano, H. (212). Change in choice-related response modulation in area MT during learning of a depth-discrimination task is consistent with task learning. J. Neurosci. 32, Williams, Z.M., Elfar, J.C., Eskandar, E.N., Toth, L.J., and Assad, J.A. (23). Parietal activity and the perceived direction of ambiguous apparent motion. Nat. Neurosci. 6, Wimmer, K., Compte, A., Roxin, A., Peixoto, D., Renart, A., and de la Rocha, J. (215). Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT. Nat. Commun. 6, Xu, H., Wallisch, P., and Bradley, D.C. (214). Spiral motion selective neurons in area MSTd contribute to judgments of heading. J. Neurophysiol. 111, Yang, Y., Liu, S., Chowdhury, S.A., DeAngelis, G.C., and Angelaki, D.E. (211). Binocular disparity tuning and visual-vestibular congruency of multisensory neurons in macaque parietal cortex. J. Neurosci. 31, Yang, H., Kwon, S.E., Severson, K.S., and O Connor, D.H. (216). Origins of choice-related activity in mouse somatosensory cortex. Nat. Neurosci. 19, Yu, X.J., Dickman, J.D., DeAngelis, G.C., and Angelaki, D.E. (215). Neuronal thresholds and choice-related activity of otolith afferent fibers during heading perception. Proc. Natl. Acad. Sci. USA 112, Yu, X., Hou, H., Spillmann, L., and Gu, Y. (218). Causal evidence of motion signals in macaque middle temporal area weighted-pooled for global heading perception. Cereb. Cortex 28, Zaidel, A., DeAngelis, G.C., and Angelaki, D.E. (217). Decoupled choicedriven and stimulus-related activity in parietal neurons may be misrepresented by choice probabilities. Nat. Commun. 8, 715. Zhang, T., and Britten, K.H. (211). Parietal area VIP causally influences heading perception during pursuit eye movements. J. Neurosci. 31, Zhang, T., Heuer, H.W., and Britten, K.H. (24). Parietal area VIP neuronal responses to heading stimuli are encoded in head-centered coordinates. Neuron 42, Zohary, E., Shadlen, M.N., and Newsome, W.T. (1994). Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 37, Neuron 1, , November 7,

15 STAR+METHODS KEY RESOURCES TABLE REAGENT or RESOURCE SOURCE IDENTIFIER Deposited Data All data This paper Experimental Models: Organisms/Strains Rhesus macaque (Macaca mulatta) Beijing Institute of Xieerxin Biology Resource, China Software and Algorithms MATLAB MathWorks CED Spike2 CED TEMPO Software Reflective Computing ReflectiveComputing/Tempo.html PyEletrode Daye et al., 213 ftp://lsrftp.nei.nih.gov/lsr/pyelectrode Other Electrodes FHC microelectrodes Riverbend Eye Tracking system Riverband Instruments Singlechannel Neural Recording System BAK Electronics Multichannel Neural Recording and Stimulation system (AlphaLab SNR) Alpha Omega CONTACT FOR REAGENT AND RESOURCE SHARING Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Yong Gu EXPERIMENTAL MODEL AND SUBJECT DETAILS Two health male rhesus monkeys (Macaca mulatta, monkey R and monkey Y, both 7 years old) participated in the experiment. Both monkeys have been used for a previous study (Yu et al., 218). During the experiment, monkeys were seated in a custom-built primate chair with their heads restrained by a chronically implanted circular molded light weight plastic ring (5 cm in diameter). The ring was anchored to the skull using titanium inverted T-bolts and dental acrylic (Gu et al., 26). Both monkeys were implanted with scleral coils for measuring eye movements in a magnetic field (Riverbend Instruments). All procedures were approved by the Animal Care Committee of Shanghai Institutes for Biological Science, Chinese Academy of Science (Shanghai, China). METHOD DETAILS Behavioral task procedures The monkeys were trained to perform a fine and a coarse motion direction discrimination task. In each trial, a fixation point ( ) first appeared on the center of the display screen and monkeys were required to maintain fixation within an electronic window of After successfully acquiring the fixation point, the monkeys were presented with visual motion stimulus, with some leftward or rightward motion components, lasting for 1 s. At the end of the trial, the fixation point disappeared and two choice targets appeared on both sides of the screen at 1 of eccentricity. The animals signaled their motion judgments by making a saccade eye movement to one of the targets within 5ms (Figure 1). Apparatus The visual stimuli were generated by an OpenGL accelerator board (nvidia Quadro 2) and were displayed on a large LCD monitor (HP LD421, , 6Hz). The pixel-on/off delay was measured and defined as the interval between baseline and 95% of evoked signals reaching a plateau from a photodiode attached on the screen. According to this measurement, the pixel-on delay e1 Neuron 1, e1 e5, November 7, 218

16 was 41 ms and the pixel-off delay was 24 ms. All stimuli were plotted with subpixel accuracy using hardware antialiasing. Monkey subjects viewed the visual stimuli from a distance of 3cm, subtending a visual angle of The screen was mounted on the front side of the magnetic field coil used to measure eye movements. The side and top of the field coil were enclosed with black matte material, making the place dark enough to exclude any external light disturbance. During the experiment, behavioral tasks and data acquisitions were monitored and controlled by Tempo software (Reflective Computing, Olympia, WA). Visual Stimuli Visual stimulus consists of a 3D cloud of stars distributed within a virtual space of 1 cm wide, 1 cm high and 4 cm deep (corresponding to roughly viewing angle). Each star was depicted as a.2 cm 3.2 cm triangle with a distribution density of.1/cm 3. Motion coherence was manipulated by randomizing the 3D location of a percentage of stars on each frame while the remaining stars moved coherently (Gu et al., 212; Yu et al., 218). Fine task A forward flow motion ( ) with a small leftward or rightward component was presented, and the monkey subjects were required to report whether their perceived motion direction was either leftward or rightward. Across trials, the motion direction was varied in fine steps around straight ahead (Figure 1, upper panel). The range of motion directions was first guided by a staircase procedure of training, and was then adjusted according to the performance under the constant stimuli procedure. The final motion directions used in the experiment were nine logarithmically spaces values for both monkey, including an ambiguous straight-forward direction (±8, ± 3.2, ± 1.28, ±.51, ). Notably, the coherence of the visual stimuli under the fine task was fixed at 1% so that the psychophysical thresholds of the two animals were roughly a few degrees (13 ; Figure 1B, left panel). Each stimulus condition was presented with a minimum repetition of 1. In the neural recording experiments, majority (93%) of the experimental sessions contained more than 2 repetitions. Coarse task The motion directions were always completely leftward ( 9 ) or rightward (+9 ) in the coarse motion direction discrimination task. The strength of the motion signal was manipulated by the percentage of coherently moving dots, namely, coherence (Figure 1A, lower panel). The remaining dots were moving in random directions. Thus 1% coherence represents all the dots in the display are moving in a consistent direction, giving strong motion signals, whereas % coherence represents complete noise in motion directions. The range of the coherence used in the current experiment was determined in the same way as in the fine task (Figure 2B, right panel, monkey Y: ± 16%, ± 8%, ± 4%, ± 2%, ± 1%, %; monkey R: ± 7.8%, ± 3.1%, ± 1.25%, ±.5%, %). Each stimulus condition was also repeated for at least 1 times, but majority of the experimental sessions (92%) contained more than 2 repetitions. Electrophysiological recordings Extracellular recordings were carried out in three cortical areas including MST, MT and VIP. Briefly, single Tungsten microelectrode (FHC; impedance, 5 ku) was inserted into the cortex via a transdural guide tube, and drove by a hydraulic Microdrive (FHC). The signal recorded from the electrode was amplified (Bak Electronics), filtered (4 Hz to 5 khz), digitized (25 khz), and stored to disk. Single units were isolated offline using Spike2 (Cambridge Electronic) via template matching as well as PCA methods. Areas of MST, MT and VIP were localized via a combination of magnetic resonance imaging (MRI) scans, stereotaxic coordinates, and the physiological response properties. Specifically, MST locates on the anterior and upper bank of superior temporal sulcus. Neurons in MST typically contain large receptive fields that occupy a quadrant or a hemifield on the screen. Compared to MST, MT locates at the lower bank of the same sulcus, and usually can be accessed by advancing the electrode further down by another 12 mm after reaching MST during vertical penetrations (Gu et al., 26). MT neurons were further identified with several evident physiological properties, including smaller receptive field strictly within the contralateral visual field, high sensitivity to visual motion signals, and retinotopic maps (Yu et al., 218). For VIP, we moved electrodes more medial from MST and MT to reach medial tip of the intraparietal sulcus (IPS). Neurons in VIP were identified based on a high percentage of direction selective cells and lack of memory activity in a memory saccade task (Chen et al., 213; Zhang et al., 24). Receptive fields of VIP neurons are centered in the contralateral visual field but can also extend into the ipsilateral field including the fovea (Figure 1C). Microstimulation Procedures Along each vertical electrode penetration, multi-units activity (MUA) was first measured at multiple sites apart by 1 mm. Global tuning curves were measured at each site in the horizontal plane (1 directions relative to straight ahead:, ± 22.5,±45,±9,± 135, and 18 ). If there are three consecutive sites showing similar tuning, the middle site was then chosen for electrical stimulation with weak current (2 ma, 2 Hz, biphasic, cathodal leading, pulse width = 2 ms, pulse interval = 1 ms, Alpha Omega SnR). This procedure was applied for each microstimulation experiment (see more detail in Figure S1D). Each microstimulation experiment block consisted of 18 trials including 1 repetitions of 9 motion directions (fine task) or 9 coherence levels (coarse task), plus two microstimulation conditions (i.e., stimulated trials and control trials without stimulation). All trials were interleaved randomly in one block. Fine and coarse tasks were run in different blocks. After the microstimulation experiment, global tuning was sometimes re-measured to confirm the stability of the constitution of the stimulated sites. In microstimulation experiment, monkeys were rewarded according to the actual visual motion stimulus, excluding the possibility of learning to detect the current (Murphey and Maunsell, 27). Neuron 1, e1 e5, November 7, 218 e2

17 QUANTIFICATION AND STATISTICAL ANALYSIS Neural sensitivity The neural sensitivity measures the strength of heading selectivity around the reference which was computed as follows (Gu et al., 212): d Rright R left = s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi (Equation 1) s 2 right + s2 left 2 Neural sensitivity = lgðjd j Þ; (Equation 2) where R left,r right, s left and s right represented the mean responses and standard deviations in response to stimuli that were 22.5 (fine task) or 9 (coarse task) to the left and right of straight forward respectively. Direction Discrimination Index (DDI) To quantify the overall strength of the spatial tuning on the horizontal plain, we computed a direction discrimination index (DDI) as defined previously (Chen et al., 211a; DeAngelis and Uka, 23; Prince et al., 22; Takahashi et al., 27): DDI = ðr max R min Þ q ffiffiffiffiffiffiffiffi; (Equation 3) R max R min where R max and R min are the mean firing rates of the neuron along the directions that elicited maximal and minimal responses, respectively. SSE is the sum-squared error around the mean responses, N is the total number of observations (trials), and M is the number of stimulus directions. This index quantifies the amount of response modulation (due to changes in stimulus direction) relative to the noise level. Neurons with large response modulations relative to the noise will have DDI values close to 1, whereas neurons with weak response modulations relative to noise level will have DDI values close to. See Figure S1. CP To quantify the relationship between neural responses and the monkey s perceptual choices, we computed CPs using ROC analysis (Britten et al., 1996). Briefly, for each motion direction, neuronal responses were first normalized (Z-scoring) and then sorted into two groups, based on the choice that the animal made at the end of each trial ( preferred choices versus null choices). The preferred choices were determined by the sign of the local tuning curve that were measured during the behavioral task. ROC values were calculated from these distributions, yielding a grand CP for each neuron. The statistical significance of CPs (whether they were significantly different from the chance level of.5) was determined using permutation tests (1, permutations). Isolation of single unit and MUA On-line raw neural signals were processed offline to obtain single unit (SU) and multi-unit (MU) activity using Spike2 (Cambridge Electronic). Specifically, SUs were well isolated events via template matching and confirmed by PCA methods. MUs were defined as events with analog voltage signal exceeding a certain threshold level. To standardize the measurements across recording sites, the event threshold was adjusted to retain a spontaneous activity level of 5 Hz higher than the spontaneous activity level of the SU (Chen et al., 28). Such MU activity included the SU component, thus we also computed another MU metric with SU subtracted. This was done by removing any MU events that located within ± 1ms of each SU spike (Chen et al., 28). The efficacy of this procedure was quantified and confirmed by cross-correlation between simultaneous SU and MU recordings (Figures S1C and S3). Comparison of microstimulation effects between fine task and coarse task To compare the effects of microstimulation between fine and coarse task, the induced PSE shift was divided by the psychophysical threshold of the non-stimulated psychometric function (Uka and DeAngelis, 26). Induced PSE shift Normed PSE shift = (Equation 4) Psycho theshold nonstim As a result, the Normed PSE shift is unitless and can be compared directly across tasks. Multiple linear regression and partial correlation The linear model used in the main task Multiple linear regression analysis was performed to tease apart the sensory and choice signals in the tuning curves measured during the motion direction discrimination task (Zaidel et al., 217). In general, neuronal responses were fitted as below: Firing ratesðfrþ = b sensory 3 Sensory Parameter + b choice 3 Choice + C; (Equation 5) SSE N M e3 Neuron 1, e1 e5, November 7, 218

18 where sensory parameter is motion directions with positive and negative signs indicating leftward and rightward direction, respectively. Choice is the decision made by the animals on each trial with positive and negative signs indicating leftward and rightward choice, respectively. b sensory and b choice are the corresponding coefficients, and C is a constant denoting spontaneous activity. The distinct impact of sensory and choice component on the firing rate is measured by the partial correlation analysis. Specifically, the partial correlation between the response and sensory parameter given the choice, i.e., r (FR, sensory j choice) defines the sensory component, i.e., r-sensory. The partial correlation between the response and choice given the sensory parameters, i.e., r (FR, choice j sensory) defines the choice component, i.e., r-choice. The implication of the signs of r-sensory and r-choice, are the same as for the sensory parameters and choice terms as described above. Other complicated models More complicated nonlinear models with square root transformation, interaction term, and polynomial fit or logistic fit were also applied (corresponding to Tables S2 S4). Model I Firing ratesðfrþ = b sensory 3 Sensory parameter + b choice 3 Choice + b sensory 2 3 b interact 3 Sensory parameter 3 Choice + C (Equation 6) Model II Model III Firing ratesðfrþ = logistic b sensory 3 Sensory parameter + b choice 3 Choice + C (Equation 7) pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Firing ratesðrfþ = b sensory 3 Sensory parameter + b choice 3 Choice + C (Equation 8) Here, extra model I is the full extension of linear model (full model) taking both higher order sensory term ðsensory 2 Þ and interaction term ðsensory3choiceþ into account. As a result, the partial correlation coefficients of r-sensory and r-choice computed from this model were still highly analogous to that from the original linear model (linear model), with significant correlations and similar amplitude (Table S2). In addition, there are only a small proportion of cases exhibiting significant interaction term (5.6%) or second-order term (1.2%). Moreover, their magnitudes are much smaller compared to that of r-sensory and r-choice (Table S2). These results suggest that the nonlinearity factors do not significantly affect our main conclusions. Model II assumes a nonlinear relationship (sigmoid) between sensory parameter and response (Logistic). In this model, partial correlation of r-sensory was estimated based on a nonparametric Spearman rank(r-sensory (spearman)) correlation due to its sigmoid relationship with the stimuli. As a result, r-sensory and r-choice computed from this model were still quite analogous to that from the original linear model (Table S3). The only deviation is that overall the magnitude of r-choice from the logistic model is slightly smaller than that in the linear mode, which does not affect the main conclusions. Model III transformed the response by taking the square root to dissociate the variance from their mean due to Poisson statistics (Nienborg and Cumming, 26). Indeed, after the transformation, the correlation between variance and the mean response were largely reduced (Table S4). Re-performing partial correlation analysis showed that r-sensory and r-choice are largely the same as that based on the original firing rates (Table S4). Thus the assumption of stable variance does not appear to cause serious problems in our dataset. Detrending procedure Sometimes the overall neural response changes gradually over time. This slow-drift response trend may affect the evaluation of r-sensory and r-choice. In order to examine this potential influence, we implemented a detrending procedure as following. (1) Linear regression ƒƒ! ½k; sponš = regress response before detrending ; N trial (Equation 9) Here k is the slope of the linear regression to quantify the slow-drift effect, and the spon is the spontaneous activity. (2) Subtraction of the slow-drift trend from the raw response ƒƒ! response after detrending = response befor detrending k 3 N trial (Equation 1) Simulation Two groups of MST and MT like neurons (n = 1) were simulated with a cosine tuning: Firing mean k ðqþ = spon + A 3 cos ðq P K Þ 3 p + 1 ; (Equation 11) 18 Neuron 1, e1 e5, November 7, 218 e4

19 where k is the particular neuron. A (set as 1) is the peak to trough modulation, and spon (set as 5) is the spontaneous response, making 25 and 5 spikes/s for the peak and minimum response across all neurons, respectively. To simulate the fine task, we used a small range of motion directions of q = [±8,±4,±2,±1, ±.5, ±.25, ±.1, ]. P k denotes the preferred direction of each neuron, which ranges within [ ]. The noise correlation structure is constructed by assigning correlated noise (r noise ) to each pair of neurons proportional to their signal correlation (r signal ), i.e., difference in preferred direction (Dq). The relationship between r noise and r signal is decided by two factors. First in general, neuron pairs within the same decision pool (both preferred directions within [-18, ] or [, 18], 1 st and 3 rd quadrant in Figure 7B) are assigned positive r noise, whereas neuron pairs between decision pools (one s within [-18 ], and the other within [,18]) are assigned with negative r noise. This general rule holds for units within either group1 (SCCC-like) or group2 (SCOC-like) class: 8 >< r noise = >: :1 :1 3 Dq Within pools 18 : :1 3 Dq between pools 18 (Equation 12) Second, the general relationship between r noise and r signal is reversed between the group1(sccc-like) or group2(scoc-like) classes: 8 >< :1 :1 3 Dq k 3 :1 within pools 18 r noise = >: : :1 ; (Equation 13) 3 Dq + k 3 :1 between pools 18 where k defines the degree of the noise correlation reversal. In this case, r noise and pool relationship is opposite to the trend as in Equation 12. The population activity r for each trial under each motion direction (q) is: p responseðqþ = < responseðqþ > + Q 3 r rand 3 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1:5 3 < responseðqþ > ; (Equation 14) where Q is the square root of the correlation matrix, r rand is a random vector of standard normal deviates with the same length as the number of neurons. < > indicates the mean value. 2 trials were simulated for each motion direction. The population responses were decoded by computing a likelihood function (Jazayeri and Movshon, 26). The likelihood over motion direction ðq + Þ given the observed population activity on a particular trial was given by: LogðLðqÞÞ = Xn i = 1 ðresponse i ðqþ 3 Logð<response i ðqþ > Þ < response i ðqþ > Þ (Equation 15) We assigned a separate weight to the likelihood contributed by group1(sccc-like) and group2 (SCOC-like) (see details in the text), thus Equation 15 need to be described more accurate as the following: LogðLðqÞÞ = w SCCC 3 logðl sccc ðqþþ + w scoc 3 logðl scoc ðqþþ; (Equation 16) where w SCCC is the weight of the likelihood contributed by group1 (SCCC-like) and the w SCOC is the weight of the likelihood contributed by group2 (SCOC-like). In addition, noise correlation structure can also be generated with the method proposed by Bondy et al. (218) (Corresponding to Figure S7). Assuming that the noise correlation structure is modulated by task-related feedback signals, we may expect that the feedback signal selectively target the most sensitive neurons because they are most relevant with the perceptual task. For the task in our experiments, the neurons with motion preference around ± 9 show maximum discriminability and potentially are most useful for the task (Gu et al., 21). Thus these neurons may be largely modulated by feedback signals. This feed-back dynamics are simulated via a sinusoid function with peak and trough at ± 9, and the noise correlation structure is given as the outer-product of the sinusoid functions. f = sinð½ p : N : pšþ (Equation 17) NC = :1 f T f; (Equation 18) where N indicated the total number of cells in the simulation and it was set as 1. DATA AND SOFTWARE AVAILABILITY All data have been deposited in the Mendeley Data ( e5 Neuron 1, e1 e5, November 7, 218

20 Neuron, Volume 1 Supplemental Information Probing Sensory Readout via Combined Choice-Correlation Measures and Microstimulation Perturbation Xuefei Yu and Yong Gu

21 Supplemental information for: Probing sensory readout via combined choice correlation measure and microstimulation perturbation Xuefei Yu 1,2, Yong Gu 1+ 1 Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and intelligence technology, Chinese Academy of Sciences, Shanghai, China 2 University of Chinese Academy of Sciences, Beijing 149, China. Supplemental Results 1

22 Supplementary figure 1. Related to Figure 2. Choice probability (A) and microstimulation effect (B) under fine and coarse tasks, and the clustering of sensory and choice signals (C-E). (A B) Scatter plot of CP (A) and the normalized induced PSE shift (B) under the fine task and the coarse task. To compare between fine and coarse task, the PSE shift was firstly reassigned according to whether the bias was towards the preferred direction of the stimulated site (>: consistent with the preferred direction; <: opposite to the preferred direction). Then they were 2

23 divided by the psychophysical threshold in the non-stimulated trials in each session, leading to unitless value. Inset: the average strength of CP (in A) and the PSE shift (in B) compared between fine task and coarse task. The strength of choice probability is slightly larger under coarse task than that fine task for both MST (blue), MT (red) and VIP (green) (MST: p=.3; MT: p=.2; VIP: p=.4; paired t-test). The microstimulation effect is almost equal under coarse task and fine task (MST: p=.68; MT: p=.11; VIP: p=.85, paired t-test). (C) Comparison of CP between SUA and MUA. Scatter plot of CP based on SUA and MUA with SUA removed for MST (blue) MT (red) and VIP (green) (see details in Supplemental methods). (D) The cluster of sensory tuning. Left panel: Examples of global tuning curves measured in the whole horizontal plane (~36 ) during one vertical electrode penetration, measured at 3 neighboring sites apart by ~1 μm. For this example, MUA showed a consistent leftward motion preference and the middle site was chosen for microstimulation. Right panel: Clustering of MUA in area MST (blue), MT (red) and VIP (green), quantified by the Pearson correlation coefficient across sites apart by 1 μm (upper panels) or 2 μm (lower panels). Filled bars: p<.5; Open bars: p>.5. Arrows: mean value. (E) The cluster of choice related signals and correlated sensory signals. Choice related signals were measured for three consecutive sites during one vertical penetration in additional experiments. Upper panel: Scatter plot of CP measured at the reference site (x-axis) and CP measured at another site apart by ~1 μm (left; Fine: N=138; Coarse: N=34) or ~2 μm (Right; Fine: N=69; Coarse: N=17). Lower panel: Along with those consecutive sites which had been measured choice related signals, sensory tuning were measured at the same time for several sites ( 1μm: N=72; 2μm: N=36). It turned out that the sites with clustered choice signals (upper panels) often exhibit clustered sensory signals at the same time. In addition, there are 29 cases in MST and MT (MST: N=4; MT: N=25) which were firstly measured for clustering of choice related signals and then applied microstimulation. The main conclusions based on these 29 cases hold but with less statistical power due to limited sampling size. For example, microstimulation effect is more predictable from the r-sensory (r=.55, p=.2; Pearson correlation coefficient), but not r-choice (r=.39, p=.4, Pearson correlation coefficient, note that the marginal significance here is due to that there happens to be more congruent sensory-choice cases, i.e. SCCC among these 29 sites). Moreover, the mean induced PSE shift on SCCC sites (1.±.18) is approximately twice as large as that from the SCOC site (.44±.13). 3

24 4

25 Supplementary figure 2. Related to Figure 4. Comparison of r-sensory and r-choice with the traditional metrics of d (A) and Choice Probability (B), and comparison of tuning strength, preferred direction, sensory component, choice component and psychophysical performance between sensory-choice congruent cells (SCCC) and opposite cells (SCOC) (C-H). (A) Comparison of r-choice and CP. In order to reconcile the sign of the two metrics, we reassign r-choice according to its consistency with r-sensory, so that the corrected r-choice> indicates SCCC and the corrected r-choice< indicates SCOC. For CP, value>.5 indicates that the neuron fires more vigorously when the animal s upcoming choice is in the neuron s preferred direction. In contrast, CP<.5 indicates that the neuron fires more when then choice is in the neuron s null direction. CP=.5 (dashed line) means chance level, or zero-correlation between the neural activity and the monkey s choice. Thus, deciding the preferred direction from the recorded neuronal tuning curves is critical. Unfortunately, these tuning curves measured during the behavioral task are usually confounded by both of the sensory and choice component, especially for SCOC (Zaidel et al, 217). From (A), it could be seen that majority of cases are along the unity line in the 1 st and 3 st quadrants, indicating that the two metrics are consistent with each other. However, there are also some cases along the negative unity line mainly in the 2 st quadrant. These cells are defined as positive choice correlations under CP measurement, whereas they actually exhibit opposite stimulus-driven (i.e. sensory) and choice-driven signs. Upper panels: marginal distributions of corrected r-choice for fine and coarse tasks in MST (blue), MT (red) and VIP (green). (B) Tuning strength is approximated by discriminability (d ) based on neural responses measured under the passive viewing condition (see Supplemental Methods). The signs of d are the same as for r-sensory. The two metrics are significantly correlated (MST: r=.71, p=; MT: r=.71, p=; VIP: r=.6, p=; Pearson correlation coefficient), suggesting that the partial correlation of r-sensory could largely reflect stimulus-driven response that is not twisted by choice-related signals This consistence holds for both fine and coarse tasks (Fine task: r=.68, p=; Coarse task: r=.73, p=; Pearson correlation coefficient). (C) A direction discrimination index (DDI) is used to quantify the global tuning strength of the recorded neurons to motion directions varied in the whole horizontal plane (~36 ). DDI closer to 1 indicates strong tuning, whereas DDI close to indicates poor tuning. Across all three areas under the two behavioral contexts (fine and coarse task), both SCCC and SCOC show high DDI values without significant difference (MT: p=.62; MST: p=.29; VIP: p=.45; t-test). (D) Distribution of the preferred direction of SCCC and SCOC. (E, F) Comparison of r-sensory and r-choice between SCCC and SCOC. In MST and MT, there is no significant difference in r-sensory and only a marginal difference in r-choice between the two classes of cells ( r-sensory : MST: fine: p=.94; coarse: p=.26; MT: fine: p=.95; coarse: p=.18; r-choice : MST: fine: p=.4; coarse: p=.23; MT: fine: p=.12; coarse: p=.62; t-test). In VIP, r-sensory and r-choice of SCCC is slightly higher than that of SCOC ( r-sensory : VIP: fine: p=1.e-4; coarse: p=.16; r-choice :VIP: fine: p=.5; coarse: p=.5, t-test). (G) The psychometric function for the example SCCC and SCOC in Figure 4A. The behavioral performance for the example SCCC and SCOC are similar (SCCC: μ=.8, σ=1.63 ; SCOC: μ=.5, σ=1.26 ). (H) Similar psychophysical thresholds measured for SCCC and SCOC sessions under both tasks, suggesting SCCC and SCOC cannot be due to the factor of the animals behavioral performance (Fine: MST: p=.72; MT: p=.79; VIP: p=.93; Coarse: MST: p=.22; MT: p=.28; VIP: p=.79; t-test). 5

26 Supplementary figure 3. Related to Figure 4, 6. The clustering of r-sensory and r-choice for sensory-choice congruent cells (SCCC, A, C) and sensory-choice opposite cells (SCOC, B, D). Scatter plot of r-sensory (upper panel) and r-choice (lower panel) measured at the reference site (x-axis) and r-sensory and r-choice measured at neighboring sites apart by ~1 μm (A, B) or ~2 μm (C,D) for SCCC (A,C) and SCOC (B,D), respectively. Both r-sensory and r-choice showed similar clustering for SCCC and SCOC. r: Pearson s correlation coefficient. Blue: MST; Red: MT; Green: VIP. 6

27 7

28 Supplementary figure 4. Related to Figure 4. Comparison of r-sensory and r-choice between SU and MU activity (A-E), and r-sensory and r-choice as a function of preferred motion directions (F-G). (A) An example of choice-conditioned tuning curve based on SUA (dash line) and MUA with SUA removed (solid line) recorded from the same site. Left panel is the raw SU and MU waveforms distributed on the PCA space. It is clear that both SUA and MUA show consistent sensory-driven and choice-related signals (right panel). In order to separate SUA and MUA for each recording session, we implement an automatic method separating SUA and MUA for population results (see Supplemental Methods). (B) Averaged cross-correlogram between SU and MU spikes, and the averaged SU auto-correlogram across all cases. (C, D, E) Scatter plot of r-sensory (C), r-choice (D) and r-choice with sign corrected towards sensory preference (E) based on SUA and MUA with SUA removed. Dash line: unity line. Solid line: linear regression from type I fitting. In order to measure the clustering of SCCC and SCOC, we redefined the sign of r-choice according to its consistency with r-sensory. For SCCC, r-choice shares the same sign with r-sensory so that the r-choice towards sensory preference is positive (colored symbols). For SCOC, r-choice has reversed sign with r-sensory, so that the r-choice towards sensory preference is negative (grey symbol). For both SCCC and SCOC, the r-choice based on SUA is correlated with the r-choice based on MUA. (F, G) r-sensory (F) and r-choice (G) as a function of preferred motion direction. For each neuron, preferred direction is quantified via vector sum based of the global tuning curve measured in the whole horizontal plane under the passive viewing condition. For both fine and coarse task, r-sensory shows a significant positive-correlation with preferred direction for all three areas (MST: fine: r=.44, p=1.93e-14; coarse: r=.62; p=1.39e-1; MT: fine: r=.51; p=; coarse: r=.53; p=7.77e-4; VIP: fine: r=.58; p=7.33e-4; coarse: r=.68; p=8.81e-9; Spearman correlation), whereas r-choice shows significant positive correlation with preferred direction only for VIP neurons, not for MST and MT neurons (MST: fine: r=-.12, p=.6; coarse: r=-.6; p=.6; MT: fine: r=.8, p=.18; coarse: r=.4; p=.98; Spearman correlation). 8

29 Supplementary figure 5. Related to Figure 4. Comparison of r-sensory and r-choice before and after removal of slow-drift response effect. (A) An example of serious slow-drift effect during recording. Left panel: For some cases, the overall neural responses increase/decrease over time, i.e. trending effect, just as the example shown in (A, black line, neural trend (slope) =.88). In order to exclude this effect, we subtracted the linear trend from the raw response (details in supplementary methods). After this detrending procedure, the overall neural response was almost flat (red line, neural trend (slope) =). Right panel: The motion direction and monkey s choice for each trial during this recording session. (B) The fitted slopes before and after the detrending procedure for MST (blue), MT (red) and VIP (green). Upper panels are the marginal distributions of the linear fitted slopes before detrending. In fact, neural response with severe slow-drift effect 9

30 like (A) is rare in all three areas under both fine and coarse tasks. The asterisk in (B) indicates the example in (A). (C, D) Comparison of r-sensory (C) and r-choice (D) before and after removing the slow-drift response effect. They are generally distributed along the diagonal line, indicating that the slow-drift in response cannot account for r-sensory and r-choice. 1

31 Figure 6. Related to Figure 5 6. Comparison between sensory-choice congruent cells 11

32 (SCCCs) and sensory-choice opposite cells (SCOCs). (A) The magnitude of r-sensory are similar between SCCC and SCOC in MST (p=.74) and MT (p=.72, t-test), but different in VIP (p=9.2e-4, t-test). (B) The cell type congruency is defined as the proportion of cells with sensory-choice congruency that is consistent with neighboring sites (~1 um). There is no significant difference between SCCC and SCOC in MST and MT (MST: p=.51; MT: p=.73; MST&MT: p=.48, chi-squared test). In VIP, the cell type congruency for SCCC is significantly larger than that of SCOC (p=.2, chi-squared test). (C) The magnitude of r-choice is similar between SCCC and SCOC in MST (p=.6) and MT (p=.83, t-test), but different in VIP (p=.2, t-test). (D-I) Analysis of covariance (ANCOVA, Matlab function of aoctool ) of the relationship between r-sensory and the induced PSE shift (D-F), and the relationship between r-choice and the induced PSE shift (G-I) for SCCC and SCOC. Intercepts described the difference in the overall microstimulation effect between the two classes of cells that is not due to the difference in r-sensory or r-choice. The slopes described how much the microstimulation effect co-varies with the strength of the sensory-related or choice-driven signals. 12

33 13