Decoding a Perceptual Decision Process across Cortex

Transcription

1 Article Decoding a Perceptual Decision Process across Cortex Adrián Hernández, 1 Verónica Nácher, 1 Rogelio Luna, 1 Antonio Zainos, 1 Luis Lemus, 1 Manuel Alvarez, 1 Yuriria Vázquez, 1 Liliana Camarillo, 1 and Ranulfo Romo 1, * 1 Instituto de Fisiología Celular-Neurociencias, Universidad Nacional Autónoma de México, México, D.F., Mexico *Correspondence: rromo@ifc.unam.mx DOI /j.neuron SUMMARY Perceptual decisions arise from the activity of neurons distributed across brain circuits. But, decoding the mechanisms behind this cognitive operation across brain circuits has long posed a difficult problem. We recorded the neuronal activity of diverse cortical areas, while monkeys performed a vibrotactile discrimination task. We find that the encoding of the stimuli during the stimulus periods, working memory, and comparison periods is widely distributed across cortical areas. Notably, during the comparison and postponed decision report periods the activity of frontal brain circuits encode both the result of the sensory evaluation that corresponds to the monkey s possible choices and past information on which the decision is based. These results suggest that frontal lobe circuits are more engaged in the readout of sensory information from working memory, when it is required to be compared with other sensory inputs, than simply engaged in motor responses during this task. INTRODUCTION In its simplest formulation, a perceptual decision results from the interaction between past and current sensory information. A major problem in this formulation involves understanding how brain circuits represent past and current sensory events and how these representations are linked to perceptual reports (Romo and Salinas, 1999). Previously, we addressed this problem using a vibrotactile discrimination task (Hernández et al., 1997). In this task, trained monkeys compare information of the first stimulus frequency (f1) temporarily stored in working memory to the current sensory information of the second stimulus frequency (f2) to form a decision, i.e., whether f2 > f1 or f2 < f1, and to immediately report their perceptual evaluation by pressing one of two push buttons. Because this sequence depends on discrimination of highly simplified stimuli, the neuronal activity of diverse cortical areas can be examined during the same behavior (Brody et al., 2003; Chow et al., 2009; Hernández et al., 2000, 2002; Jun et al., 2010; Luna et al., 2005; Machens et al., 2005; Romo et al., 1999, 2002, 2003, 2004; Romo and Salinas, 2003; Salinas et al., 2000). The task used in these studies simulates the behavioral condition in which the decision based on a sensory evaluation is immediately reported through a voluntary movement (Hernández et al., 1997). There are, however, behavioral conditions in which a perceptual decision can be postponed for later report. But, in theory, once the subject reaches a decision, this becomes categorical, no matter whether it must be reported immediately or reported later. If postponed, memory circuits may store the categorical decision for later report (de Lafuente and Romo, 2005; Shadlen and Newsome, 1996). However, an alternative could be that the memory circuits store not only the categorical decision, but also the information on which the decision is based (Lemus et al., 2007). This last possibility could be extremely advantageous since it gives flexibility for the decision-making process. In this case, it is possible that the decision is revised or updated as long as there is time for it to be reconsidered. In a variant of the vibrotactile discrimination task, in which monkeys were asked to postpone their decision report, we found that the activity of medial premotor cortex (MPC, presupplementary motor area, and supplementary motor cortex) neurons during this period encodes both the result of the sensory evaluation (which corresponds to the monkey s two possible choices) and past information on which the decision is based (Lemus et al., 2007). These responses could switch back and forth with remarkable flexibility across the postponed decision report period. Moreover, these responses covaried with the animal s decision report. Thus, the MPC circuits appear critically suited to integrate and reorganize all of the elements associated with decision making in this task. Furthermore, they reflect the flexibility that is needed when a perceptual decision must be either immediately reported (Hernández et al., 2002) or postponed for later report (Lemus et al., 2007). This result prompted us to further explore whether the neuronal responses recorded during the postponed decision period are a unique property of the MPC circuit (Lemus et al., 2007) or whether similar processes are also present in other cortical areas of the parietal and frontal lobes during this variant of the task. To further investigate this question, we recorded the neuronal activities of diverse cortical areas while trained monkeys reported a postponed decision based on previous sensory evaluation. In this task, monkeys must hold f1 in working memory and must compare it to the current sensory stimulus (f2) and must postpone the decision report until a cue triggers the 300 Neuron 66, , April 29, 2010 ª2010 Elsevier Inc.

2 Figure 1. Discrimination Task (A) Sequence of events during discrimination trials. The mechanical probe is lowered, indenting the glabrous skin of one digit of the restrained hand (pd); the monkey places its free hand on an immovable key (kd); the probe oscillates vertically, at the base stimulus frequency (f1); after a fixed delay (3 s), a second mechanical vibration is delivered at the comparison frequency (f2); after another fixed delay (3 s) between the end of f2 and probe up (pu), the monkey releases the key (ku) and presses either a lateral or a medial pushbutton (pb) to indicate whether the comparison frequency was higher or lower than the base, respectively. (B) Stimulus set used during recordings. Each box indicates a base/comparison frequency stimulus pair. The number inside the box indicates overall percentage of correct trials for that (f1, f2) stimulus pair, except when the stimulus pair was identical (22 Hz; we plotted the number of times that animal pressed the lateral push button). (C) Psychophysical performance when f1 was maintained fixed at 22 Hz and f2 was variable (red curve), and when f2 was fixed at 22 Hz and f1 was variable (green curve). D.L. is the discrimination threshold in Hz. (D) Top view of the monkey brain and the cortical areas recorded during perceptual discrimination (orange spots). Recordings were made in primary somatosensory cortex (S1) and secondary somatosensory cortex (S2) contralateral to the stimulated hand (left hemisphere) and in primary motor cortex (M1) contralateral to the responding hand/arm (right hemisphere). Recordings were made contralateral and ipsilateral to the stimulated fingertip in prefrontal cortex (PFC), ventral premotor cortex (VPC), medial premotor cortex (MPC), and dorsal premotor cortex (DPC). motor report, i.e., whether f2 > f1 or f2 < f1. Clearly, the neuronal processes associated with the postponed decision report and the task components that precede it can be analyzed across diverse cortical areas. Here we report the extent to which the stimulus identity is encoded across diverse cortical areas in this task. We found that the encoding of f1 and f2 through all task periods is widely distributed across cortical areas. We also found that the activity of frontal lobe circuits encodes both the result of the sensory evaluation and past information on which those choices are based. Notably, the activity of primary motor cortex (M1) showed processes similar to those observed in the premotor areas (ventral premotor cortex, VPC; dorsal premotor cortex, DPC; and MPC) and prefrontal cortex (PFC), both during the comparison and postponed decision report periods. These results suggest that frontal lobe neurons have the capacity to encode during the comparison and postponed decision report periods both the final result of the sensory evaluation and past information about it. Here we also document the nature of the neuronal responses during the stimuli and their interactions. In addition to the standard discrimination test, the neuronal activity of all cortical areas was studied when the stimuli were delivered but monkeys were not requested to perform the task. Under this condition, most neurons across the cortical areas no longer encode information about the stimuli and their interactions during these trials. The only areas that responded in this case were S1 and S2. This would suggest that those cortical areas central to S1 that encode information about the stimuli are more likely associated with the sensory evaluation, than engaged simply in encoding the sensory stimulus. We also tested each neuron in a simpler task, in which trials proceeded exactly as in the vibrotactile task, but the stimuli were not delivered to the skin and the movements were guided by visual cues. Neurons responded during movement execution but not during the periods preceding it. These control tests show that the neuronal responses from all the cortical areas studied, except for S1, reflect both the active comparisons between f1 and f2 and the execution of the motor choice that is specific to the context of the vibrotactile discrimination task. RESULTS Optimal Conditions for Studying Perceptual Discrimination Four monkeys (Macaca mulatta) were trained to discriminate the difference in frequency between two consecutive vibrotactile stimuli, f1 and f2 delivered to one fingertip (Figure 1A). Monkeys were asked to report discrimination after a fixed delay period of 3 s between the end of f2 and the cue that triggered the motor report (probe up, pu in Figure 1A). This delay period thus separates the comparison between the two stimuli from the motor response. In this task, monkeys must hold f1 in working memory, must compare the current sensory input f2 to the memory trace of f1, and must postpone the decision until the sensory cue triggers the motor report. Animals were trained to perform the task up to their psychophysical thresholds (Figures 1B and C). After training, we recorded the activity of single neurons from diverse cortical areas while the monkeys performed the task (Figure 1D). These recordings were made in primary somatosensory cortex (S1), secondary somatosensory cortex (S2), PFC, VPC, DPC, and MPC contralateral to the stimulated finger and in PFC, VPC, DPC, MPC, and M1 contralateral to the responding hand/arm. All neurons were recorded using the stimulus set of Figure 1B. In these trials, the comparison frequency (f2) can be judged higher or lower than f1. Thus, the neuronal responses across trials can be analyzed as functions of f1, f2, f2 f1, or as functions of the monkey s two possible motor choices. Neuron 66, , April 29, 2010 ª2010 Elsevier Inc. 301

3 Neuronal Responses across Cortical Areas during Perceptual Discrimination We recorded from 2509 neurons (Table 1) that had average firing rates that were significantly different from their firing rates during a pre-trial control period (500 ms immediately before probe down; pd in Figure 1A; p < 0.01, Wilcoxon rank-sum test [Siegel and Castellan, 1988]). Some of these neurons enhanced or reduced their firing rates (222/2509, 9%), but their firing rate did not depend significantly on the applied stimulus frequencies or the animal s decision report (Experimental Procedures). Here we focus exclusively on the 91% of neurons (n = 2287) that were stimulus dependent (Romo et al., 1999), although not necessarily during the stimulus presentation periods: stimulus-dependent modulations could occur also during the working memory, comparison and postponed decision report periods of the task (Table 1). This is illustrated for an example neuron recorded in the MPC contralateral to the stimulated finger (Figure 2A). The discharge rate of this neuron during the delay period between f1 and f2 varied as a monotonic function of f1. This neuron discharged more strongly for the lower frequency and decreased its firing rate steadily for increasing f1. We refer for this type of response as negative monotonic (Romo et al., 1999). Other neurons had discharge rates that varied in the opposite direction. We refer to those as positive monotonic (Romo et al., 1999). Thus, for this example neuron the analysis showed that f1 appeared to be encoded directly with various strengths and at various periods of time during the delay period between f1 and f2. Neurons with positive or negative modulations were also recorded in S2 contralateral to the stimulated hand and bilaterally in PFC, VPC, DPC, and MPC. Neurons from these areas could respond exclusively to the f1 presentation, or encode f1 during the delay period between f1 and f2 (Figure 2A) or during both periods. It is also worth mentioning that the large majority of S1 neurons encoded f1 almost exclusively in a positive monotonic fashion during the first stimulus period only, and that none of the M1 neurons showed any modulation of their firing rates as a function of f1 during the f1 presentation or during the delay period between f1 and f2 (Figure 3 and see Figure S1 available online). As the task progressed, trials for the example neuron of Figure 2A can be divided into two types: those on which f2 > f1 (black label) and those on which f2 < f1 (gray label). During the final 200 ms of f2 presentation and early component (1 s) of the postponed decision report period of our example neuron, the firing rates were modulated by both f1 and f2. The main determinant of the firing rate was not, however, the particular values that f1 or f2 took on any given trial. Instead, it was simply whether the trial belonged to the f2 > f1 group or f2 < f1 group. This corresponds to the monkey s two possible choices, represented during the comparison and postponed decision report periods. Not only did f1 modulate the response to f2 in this neuron, but even more notably, this happened such that by the end of f2 and during the early component of the postponed decision period, the responses became mostly correlated with the monkey s choice. This type of response that occurred during the comparison and postponed decision report periods was observed in area S2 contralateral to the stimulated hand and bilaterally in PFC, VPC, DPC, and in M1 contralateral to the responding arm (Figure 3), but not in S1. In brief, the stimulus identity of f1 and f2 could be encoded in a positive (S1, S2, PFC, VPC, DPC, and MPC) or negative (S2, PFC, VPC, DPC, and MPC) monotonic fashion during the stimulus presentations. The f2 > f1 and f2 < f1 responses during the f2 and postponed decision report periods recorded in S2, PFC, VPC, MPC, and M1 suggest that they result from an interaction between the two stimuli. This requires, however, understanding precisely how f1 and f2 are encoded and how these interactions are computed during the comparison and postponed decision report periods. Decoding Perceptual Discrimination across Cortical Areas To further estimate the representation of f1 and f2 and their interactions during the task components, we used multivariate regression analysis. For each neuron, we modeled the firing rate s dependency on f1 and f2. In principle, the response during f2 could be an arbitrary linear function of both f1 and f2: firing rate(t) = a1(t) f1 + a2(t) f2 + a3(t) (Draper and Smith, 1966; Hernández et al., 2002; Press et al., 1992; Romo et al., 2002, 2004). In this formulation, t represents time, and the coefficients a1 and a2 serve as direct measurements of firing rate dependence on f1 and f2, respectively. These measurements were calculated in sliding windows of 200 ms moving in steps of 20 ms. To illustrate this analysis, the resulting coefficients a1 and a2 and their interactions were plotted in panels C and D of Figure 2 for the example neuron as a function of time. We also plotted the values of a1 and a2 against each other to compare the responses at different points during the task (Figure 2C). Three lines are of particular relevance in these plots: points that fall on the a1 = 0 axis represent responses that depend on f2 only (red dots in Figure 4); points that fall on a2 = 0 axis represent responses that depend on f1 only (green dots in Figure 4), and points that fall near the a2 = a1 line represent responses that are a function of f2 f1 only (black and blue dots in Figure 4). This last consideration is of particular importance since the sign of the difference between f1 and f2 determines correct task performance. However, the result of the analysis is not restricted to these three conditions. For example, in those hypothetical cases when the modulation imposed by f1 and f2 results in f1 + f2, the point would fall close to a1 = a2 line. In this case, the memory of f1 is added to the f2 representation, but this result was rarely observed. Importantly, the larger area of the plane represents those responses where the strengths of a1 and a2 are significantly different from zero and significantly different from each other. This plot can also reveal whether one of the two stimulus frequencies is more strongly represented than the other (ja1j s ja2j;a1s 0; a2 s 0; black dots in panels C and D of Figure 2 and black dots of Figure 4). The population analysis of each cortical area shows that most of the neurons encode f1 in a positive or negative monotonic manner at various coefficients strengths. This result agrees with results obtained from a large data base recorded in monkeys reporting discrimination immediately after f2 (Romo et al., 2004). As before, most S1 neurons encode f1 during the stimulus presentation in a positive monotonic fashion whereas the rest of cortical areas except for M1 neurons which did 302 Neuron 66, , April 29, 2010 ª2010 Elsevier Inc.

4 Table 1. Database Task Component Area Responsive Tuned f1 delay f1 f2 f2 delay f2 pu mt S1 189 f (31.2%) 5 (2.6%) f (32.2%) 4 (2.1%) c d S2 426 f (11.5%) 28 + (6.5%) 6 + (1.4%) 8 + (1.8%) 4 + (0.9 %) 45 (10.5%) 21 (4.9%) 5 (1.1%) 6 (1.4%) 3 (0.7%) f (13.3%) 31 + (7.2%) 6 + (1.4%) 54 (12.6%) 25 (5.8 %) 5 (1.1%) c 34 (7.9%) 15 (3.5%) 12 (2.8%) d 21 (4.9%) 26 (6.1%) 23 (5.4%) VPC 375 f (12.8%) 63 + (16.8%) 76 + (20.2%) 31 + (8.2%) 14 + (3.7%) 36 (9.6%) 45 (12.0%) 57 (15.2%) 26 (6.9%) 12 (3.2%) f (12.5%) 27 + (7.2%) 15 + (4.0%) 30 (8.0%) 25 (6.6%) 7 (1.8%) c 62 (16.5%) 11 (2.9%) 5 (1.3%) d 108 (28.8%) 68 (18.1%) 16 (4.3%) PFC 358 f (6.9%) 49 + (13.7%) 25 + (6.9%) 14 + (3.9%) 5 + (1.4%) 26 (7.7%) 42 (11.7%) 21 (5.8%) 9 (2.5%) 8 (2.2%) f (9.7%) 27 + (7.5%) 11 + (3.0%) 21 (8.0%) 31 (8.7%) 13 (3.6%) c 53 (14.8%) 26 (7.3%) 3 (0.8%) d 82 (22.9%) 72 (20.1%) 16 (4.5%) MPC 494 f (5.9%) 47 + (9.5%) 33 + (6.7%) 39 + (7.9%) 19 + (3.8%) 24 (4.9%) 39 (7.9%) 28 (5.7%) 32 (6.5%) 14 (2.8%) f (12.8%) 67 + (13.6%) 16 + (3.2%) 41 (8.3%) 59 (11.9%) 17 (3.4%) c 53 (10.7%) 42 (8.5%) 13 (2.6%) d 83 (16.8%) 74 (15.0%) 45 (9.1%) DPC 164 f (6.0%) 12 + (7.3%) 3 + (1.8%) 9 + (5.5%) 4 + (2.4%) 12 (7.3%) 8 (4.9%) 6 (3.7%) 12 (7.3%) 5 (3.0%) f (9.1%) 11 + (6.7%) 9 + (5.5%) 16 (9.8%) 13 (7.9%) 10 (6.1%) c 9 (5.5%) 3 (1.8%) 3 (1.8%) d 4 (2.4%) 9 (5.5%) 16 (9.8%) M1 281 f1 2 + (0.7%) 3 + (1.0%) 3 + (1.0%) 17 + (6.0%) 4 + (1.4%) 3 (1.0%) 4 (1.4%) 7 (2.4%) 24 (8.5%) 6 (2.1%) f (8.8%) 49 + (17.4%) 14 + (4.9%) 27 (9.6%) 52 (18.5%) 17 (6.0%) c 15 (5.3%) 27 (9.6%) 15 (5.3%) d 17 (6.0%) 35 (12.5%) 29 (10.3%) Recorded, n = Responsive, n = 2287 (91%). f1, first stimulus; f2 second stimulus; pu, probe up; mt, movement time; S1, primary somatosensory cortex; S2, second somatosensory cortex; VPC, ventral premotor cortex; PFC, prefrontal cortex; MPC, medial premotor cortex; DPC, dorsal premotor cortex; M1, primary motor cortex. Tuned to stimulus frequency with positive (+) or negative ( ) slopes; c, tuned to stimuli with differential activity for f2 > f1 or f2 < f1; d, differential activity to f2 > f1 or f2 < f1. Neuron 66, , April 29, 2010 ª2010 Elsevier Inc. 303

5 Figure 2. Responses of a MPC Neuron during the Discrimination Task and Control Tests (A) Raster plots of responses during the discrimination task. This neuron responded with an f1 negative monotonic fashion to the increasing stimulus frequency f1 during the delay period between f1 and f2 and during the early delay period between the end of f2 and the beginning of the decision motor report (pu). Each row of ticks is a trial, and each tick is an action potential. Trials were delivered in random order (10 trials per stimulus pair). Labels at left indicate f1:f2 stimulus pairs. Black indicates f2 > f1; gray indicates f2 < f1. (B) Firing rate modulation (mean ± SEM) as a function of f1 or f2. (C) Resulting coefficient values for f1 (a1, green) and f2 (a2, red) for panels in (B). (D) Coefficients values as functions of time. Green and red traces correspond to a1 and a2, respectively. Filled circles indicate significant values. Black circles indicate points at which a1 and a2 were significant and of different magnitudes, but had opposite signs; these are partially differential (c) responses. Blue circles indicate points at which a1 and a2 were significant and of similar magnitude but had opposite signs; these are fully differential (d) or categorical responses. (E) Responses of the same neuron when the same set of stimuli (A) was delivered to the fingertip, but discrimination was restricted, just by removing the key and the interrupt target switches. Thus, in this condition the animal remained alert by rewarding with drops of liquid at different times but was no longer using the stimuli to indicate discrimination with the free hand/arm. Under this test condition, the neuron does not encode information about the stimuli. (F) Choice probability indices as function of time during the discrimination task. Filled circles are significant values that deviated from 0.5 (green for f1 values; black for c values; and blue for d values of D). (G) Choice probability index for the same neuron tested in the light instruction task. Under this condition, the choice probability indices were calculated by comparing the response distributions for lateral versus medial push button presses. Arm movements in this situation were identical to those in the vibrotactile discrimination task but were cued by visual stimuli. not encode f1 at all do so in dual form: positive or negative monotonic encoding (green dots in Figure 4). These encodings are also observed during the delay period between f1 and f2, except for S1 and M1 neurons, which do not show any encoding during this period (green dots in Figure 4). During the comparison period, most S1 neurons encoded f2 in a positive monotonic fashion (red dots in Figure 4), whereas most of the cortical areas encoded the current stimulus f2, but also f1 (Figure 4). Again, the encoding of f1 and f2 could be positive or negative monotonic. But, in addition, the information of f1 and f2 is combined to generate a differential response (blue dots in Figure 4). An interesting observation is that some neurons during the comparison period switched from an f1 encoding to a combination of f2 and f1 (black dots in Figure 4), and then to a differential response (blue dots in Figure 4). During the postponed decision report, many neurons from PFC, VPC, MPC, DPC, and M1 encoded information on which the decision is based and the resulting operation, f2 > f1 or f2 < f1 (Figure 4). Again, S1 did not show any sign of participation in the postponed decision report (Figure 4). Dynamics of the Perceptual Discrimination Process across Cortical Areas Figure 4 shows the number of neurons with significant a1 and a2 coefficients during the relevant components of the task. However, it does not tell much about the dynamics of the population response of each cortical area across the task components. An analysis of the coefficients as functions of time shows that all cortical areas studied here encoded f1 during the f1 presentation, except M1 (green traces in Figure 5A). The earliest response began in S1 compared to S2, PFC, VPC, MPC, and DPC, then S2 responded earlier than PFC, VPC, MPC, and DPC, and finally PFC and VPC responded earlier than MPC and DPC (Figure 6; p < 0.01 between response distributions; 304 Neuron 66, , April 29, 2010 ª2010 Elsevier Inc.

6 Figure 3. Responses of a M1 Neuron during the Discrimination Task and Control Tests (A) Raster plots of responses during the discrimination task. This neuron responded with a negative monotonic fashion to the increasing f2 stimulus frequency during the delay period between the end of f2 and the beginning of the decision motor report (pu). Each row of ticks is a trial, and each tick is an action potential. Trials were delivered in random order (10 trials per stimulus pair). Labels at left indicate f1:f2 stimulus pairs. Black indicates f2 > f1; gray indicates f2 < f1. (B) Firing rate modulation (mean ± SEM) as a function of f1 or f2. (C) Resulting coefficient values for f1 (a1, green) and f2 (a2, red) for panels in (B). (D) Coefficients values as functions of time. Green and red traces correspond to a1 and a2, respectively. Red filled circles indicate significant f2 values. (E) Responses of the neuron when the same set of stimuli (panel A) was delivered to the fingertip, but discrimination was restricted, just by removing the key and the interrupt target switches. Thus, in this condition the animal remained alert by rewarding with drops of liquid at different times but was no longer using the stimuli to indicate discrimination with the free hand/arm. Under this test condition, the neuron does not encode information about f2, as shown in (D). (F) Choice probability indices as function of time. Filled circles are significant values that deviated from 0.5 for responses of (D). (G) Choice probability index for the same neuron tested in the light instruction task. Because in this test condition animals did not show incorrect responses, the choice probability index was calculated by comparing the response distributions for lateral versus medial push button presses. Arm movements in this situation were identical to those in the vibrotactile discrimination task, but were cued by visual stimuli. See also Figure S1. Wilcoxon rank-sum test; Siegel and Castellan, 1988). Therefore, the f1 encoding seems to proceed in a serial fashion, although there is broad overlap in the response latencies of these cortical areas (Figure 6). This would suggest that all these cortical areas are engaged in f1 processing, and it is natural to ask whether they also hold f1 information during the delay period between f1 and f2. The results show that S2, PFC, VPC, DPC, and MPC encode f1 at various coefficient strengths and at various times during the delay period of this task (green traces in Figure 5A). It is quite interesting to note that more neurons in PFC, VPC, and MPC than in S2 and DPC are engaged in encoding f1 during the working memory component just immediately before the f2 presentation, when the comparison takes place. All cortical areas studied here encoded information about f2 (red traces in Figure 5A). The earliest response began in S1, continued in S2, then PFC and VPC, and finally DPC, MPC, and M1 (Figure 6; p < 0.01). Except for S1, we also observed f1 signals in all cortical areas during presentation of f2 (green traces in Figure 5A). The presence of f1 information is essential for the comparison process in this task. We observed that some neurons reflected the comparison, the difference between f2 and f1 (blue traces in Figure 5A) and some others that switched from an f1 encoding to a combination with f2 (black traces in Figure 5A). Except for S1, these comparison signals were observed in all cortical areas studied here. These differential responses were significantly (p < 0.01) delayed in comparison to f1 and f2 signals (Figure 6). Also, all frontal lobe areas, including M1, showed information about f2 (red traces in Figure 5A), f1 (green traces in Figure 5) during the comparison period, and during the delay period between the end of f2 and the cue that triggered the motor report (black and blue traces in Figure 5A). Thus, Neuron 66, , April 29, 2010 ª2010 Elsevier Inc. 305

7 Figure 4. Population Coefficient Values across Cortical Areas during the Different Components of the Discrimination Task Each point represents one neuron with at least one coefficient significantly different from zero. We analyzed five periods: f1 (500 ms), delay between f1 and f2 (3000 ms), f2 (500 ms), delay between the end of f2 and pu (3000 ms), and during a period posterior to pu (1000 ms). For each neuron, we identified a 200 ms bin with the highest modulation during each period. n = number of neurons. Green and red circles correspond respectively to neurons with significant a1 coefficients only or a2 coefficients only. Black circles correspond to neurons with both significant a1 and a2 coefficients of opposite signs but of significantly different magnitudes; these are partially differential responses (c). Blue circles correspond to neurons with both significant a1 and a2 coefficients, but of opposite signs and statistically equal magnitude; these are fully differential or categorical responses encoding f2 f1 (d). S1, primary somatosensory cortex; S2, secondary somatosensory cortex; VPC, ventral premotor cortex; PFC, prefrontal cortex; MPC, medial premotor cortex; DPC, dorsal premotor cortex; M1, primary motor cortex. during the postponed decision report period these cortical circuits maintain in working memory all the elements associated with this cognitive operation. However, we noticed that there were more neurons in MPC, DPC, and M1 than in PFC and VPC that carried information about f2 during the postponed decision period (red traces in Figure 5A), indicating that the most recent information sensory information (f2) is more likely to be kept in working memory than immediately-preceding sensory information (f1). State-Dependent Responses across Cortical Areas Cortical population dynamics illustrated in Figure 5A shows that neurons of diverse cortical areas encode the different task components. But, to what extent are these neuronal events associated with the task components and animal s sensory evaluation? Do these events occur only during the task execution or are irrespective of the animal s state? To answer these questions, in addition to the standard test, many of the neurons that encoded information about the stimuli and motor choice were also tested in a variant of the task (Experimental Procedures). In this test, the neuronal activity of these cortical areas was studied when the stimuli were delivered but monkeys were not requested to perform the task. Under this condition, most neurons across the cortical areas no longer encoded information about the stimuli and their interactions during the task components (Figure 5B). The only areas that responded in this case were S1 and S2. This would suggest that those cortical areas central to S1 that encoded information about the stimuli are more likely associated with the sensory evaluation, than engaged in encoding the stimuli. Choice Signals across Cortical Areas Responses during correct trials alone did not allow us to determine to what extent (f2 f1)-dependent responses were correlated with the sensory stimuli, or with the monkey s action choice. For each neuron of each cortical area we sorted the responses into correct and errors trials and calculated a choice probability index as a function of time (Britten et al., 1996; Green and Swets, 1966; Romo et al., 2002). This quantified for each stimulus (f1, f2) pair whether neuronal responses during error trials were different from responses during correct trials (panel F in Figures 2 and 3). If the responses are exclusively stimulus dependent, they should show no differences between correct and errors trials, except when the differences expected at chance level result of the intrinsic variability of the neural activity. 306 Neuron 66, , April 29, 2010 ª2010 Elsevier Inc.

8 Figure 5. Cortical Population Dynamics during the Discrimination Task (A) Percentage of neurons with significant coefficients as a function of time. Green and red traces correspond to a1 and a2 coefficients, respectively. Black traces indicate percentage of neurons with a1 and a2 coefficients of opposite sign but of different magnitudes. These neurons combine differential response with a sensory component. Blue traces indicate percentage of neurons with coefficients a1 and a2 of opposite sign but similar magnitude; these produce a differential signal. (B) Percentage of neurons that responded during passive stimulation. All these neurons are part of the populations studied in (A). S1, primary somatosensory cortex; S2, secondary somatosensory cortex; VPC, ventral premotor cortex; PFC, prefrontal cortex; MPC, medial premotor cortex; DPC, dorsal premotor cortex; M1, primary motor cortex. n = number of neurons. In contrast, if the responses were dependent on the animal s choice, then they should vary according to which button the monkey chose to press (panel F in Figures 2 and 3). For all cortical areas, we computed a choice probability index separately for the neurons that responded as a function of f1 (green traces in Figure 7A), f2 (red traces in Figure 7A) or that depended specifically on f2 f1 (black and blue traces in Figure 7A). The result of this analysis shows that very few neurons from S1 reflect significant differences between correct and error trials. In contrast, many neurons from S2, PFC, VPC, DPC, MPC, and M1 predicted in their activity the animal s choice. These responses began during the f1 stimulus presentation for areas S2, PFC, VPC, DPC, and MPC, and were also present during the delay period between f1 and f2 for areas PFC, VPC, MPC, DPC, and M1, just before f2 presentation. During the f2 period, all these areas except S1 had significant choice probability indices (Figures 7A and S2). These signals were maintained during the postponed decision report, during the delay between the end of f2 and the cue that triggered the motor response (Figures 7A and S2). These results suggest that neuronal activity that correlates with error responses are widely distributed across cortical areas, and that an error could be due to a failure to encode the sensory inputs and their interactions during the task components. In addition to analyzing error trials, we tested each neuron in a variant of the task, in which trials proceeded exactly as in the vibrotactile task, but the stimuli were not delivered to the skin and the movements were guided by visual cues (Experimental Procedures). Thus, movements were triggered by visual cues. Neurons responded exactly as in Figure 7A during movement execution but not during the period preceding it (Figure 7B). These control tests show that some of the neuronal responses from all the cortical areas studied, except for S1, reflect the active comparison between f1 and f2 whereas some other neurons encode the motor choice that is specific to the context of the vibrotactile discrimination task. Neuronal Correlates of Bias Behavior across Cortex In the task used here, the decision must result of the difference between f1 and f2 (choice = f(f2 f1)). However, the discrimination thresholds plotted in Figure 1C suggest that subjects could base their decisions by paying more attention to the weight of f2 than to the weight of f1 (choice = f(w2*f2 w1*f1)). To further estimate the weights w2 and w1, we used multivariate regression analysis and plotted the subject s performance as the percentage of f2 > f1 correct responses for each stimulus pair we used only stimulus pairs labeled red and green in Figure 1C. Thus, the resulting coefficients a1 and a2 of the multivariate regression analysis correspond to w1 and w2, respectively. If during the decision subjects gave more weight to f2 than to the weight of f1, f2 must be reported higher than f1. This is what the analysis shows on the data of Figure 1C (a1 = 4.3, a2 = 5.6; r = a1/a2 = 0.76). If this analysis is carried out for each experiment of the 312 reported here, we obtain a geometric mean of 0.79 (Figure 8A). On the other hand, we know that those neurons with sensory differential activity (labeled in black in Figures 4 and 5) show coefficients a1 and a2 significant different from zero and between each other. Therefore, it is natural to ask whether the activity of these neurons is correlated with the behavioral bias illustrated in Figure 8A. For example, if we use the a1 and a2 values of the MPC neurons for those periods of times that show the condition described above and calculate the ratio a1/a2, we obtain the histogram of Figure 8B. This plot shows that the larger percentage of neural responses (77%; Neuron 66, , April 29, 2010 ª2010 Elsevier Inc. 307

9 data distribution to the left relative to 1 in Figure 8B) corresponds to a1 < a2 responses. Such proportion of bias can not be obtained by chance (p < 0.001; binomial test [100, 55, and 0.05]). Interestingly, there is a strong relationship between the direction and magnitude of the bias behavior. To further establish in which cortical area this bias is generated, we estimated the neuronal bias (a1/a2) for each neuron of each cortical area using a sliding window of 200 ms duration moving in steps of 20 ms (S2, m = 0.79; VPC, m = 0.82; PFC, m = 0.81; MPC, m = 0.81; M1, m = 0.86) beginning at f2 onset and ending at probe up that triggered the decision motor report and compared this value against the behavioral bias (a1/a2) obtained simultaneously in the same experiment. The results are shown in Figure 8C. Except for S1, we found that a large fraction of these neurons correlated with the behavioral bias. This neural bias was more evident in the PFC, MPC, DPC, and M1 than in S2 (Figure 8C). Thus, when one of the two stimulus frequencies is more strongly represented than the other, it could bias the psychophysical performance in this task. This interpretation is consistent with the fact that f2 is more strongly represented during the comparison and postponed decision periods than f1 (Figure 5). DISCUSSION To understand perceptual discrimination, we need to know where in the brain are the physical relevant variables encoded and what are their relative contributions to the final percept. Our study focuses on this problem using highly simplified stimuli, in which the neuronal responses from diverse cortical areas can be examined while trained monkeys executed the same task. Although not sufficiently exhaustive, this study shows how cortical circuits are associated with perceptual discrimination. For example, our results show that S1 is essentially sensory and M1 is not necessarily primarily associated with motor outputs only. Also, those cortical areas that receive the S1 inputs combine the sensory representations of S1 with sensory signals stored in working memory. Notably, these cortical areas encode at various strengths and times the stimulus parameters of both past and current sensory information on which the perceptual decision report is based. Moreover, the sensory, memory and comparison signals are gradually conveyed to the frontal lobe Figure 6. Box Plots Illustrate Response Latency Distributions for f1 (Green), f2 (Red), Comparison (c, Black), and Differential (d, Blue) across Cortical Areas These boxes have lines at the lower quartile, median, and upper quartile values. The whiskers are lines extending from each end of the boxes to show the extent of the rest of the data. A comparative analysis (Wilcoxon rank-sum test; Siegel and Castellan, 1988) of the response latencies between the cortical areas showed that the f1 and f2 began earlier in S1 (p < 0.01) than in S2, PFC, VPC, MPC, DPC, and M1 (f1 was not present in M1). The response latencies for f1 and f2 in S2 (p < 0.01) began earlier than PFC, VPC, MPC, DPC, and M1. The response latencies for f1 and f2 began earlier in PFC and VPC (p < 0.01) than in MPC, DPC and M1. We found no differences in the response latencies for f1 and f2 between MPC, DPC, and M1 (p > 0.01). All f1 and f2 response latencies in all these cortical areas began earlier (p < 0.01) than the comparison (c) and differential responses (d). We found no statistical differences (p > 0.01) between the comparison and differential responses across the cortical areas. L, left hemisphere (contralateral to the stimulated hand); R, right hemisphere (ipsilateral to the stimulated hand). Recordings in primary somatosensory cortex (S1) and secondary somatosensory cortex (S2) were made contralateral to the stimulated hand (left hemisphere) and in primary motor cortex (M1) contralateral to the responding hand/arm (right hemisphere). Recordings were made bilaterally in prefrontal cortex (PFC), ventral premotor cortex (VPC), medial premotor cortex (MPC), and dorsal premotor cortex (DPC). 308 Neuron 66, , April 29, 2010 ª2010 Elsevier Inc.

10 Figure 7. Correlation between Neuronal Responses of Diverse Cortical Areas and Behavioral Choice (A) Percentage of neurons that had significant choice probability indices as a function of time. Green trace: neurons that encoded information about f1; red trace: neurons that carried information about f2; black trace: partially differential neurons that carried information about f1 and f2 (c); blue trace: fully differential neurons that carried information specifically about f2 f1 only (d). See also Figure S2. (B) Percentage of the neurons in (A) that showed significant choice probability indices during the visual control task. In this test, animals had to follow a visual cue to produce the motor choice response. S1, primary somatosensory cortex; S2, secondary somatosensory cortex; PFC, prefrontal cortex; VPC, ventral premotor cortex; MPC, medial premotor cortex; DPC, dorsal premotor cortex; M1, primary motor cortex. n = number of neurons. circuits that in turn drive the motor circuits for a movement execution. Although this suggests a feedforward processing beginning in S1 and ending in M1, this seems unlikely given feedback/recurrent communications between cortical and subcortical areas (Lamme and Roelfsema, 2000). This problem is currently addressed by recording the simultaneous activity of neurons distributed across cortical circuits engaged in the task used here (Hernández et al., 2008). This study shows how distinct cortical areas contribute to the entire sequence of the processing steps that link sensation and decision making. One could argue that the neuronal events recorded in frontal lobe circuits during this task reflect other processes, such as preparation for a future action, particularly during the postponed decision report. This seems unlikely, however, because (1) delay responses between f1 and f2 depended on f1 regardless of subsequent movements; (2) responses during the postponed decision period often reflected f1 or f2 information; (3) choice probability indices indicated that there were significant differences between correct versus error trials except for S1, variability in the responses of those S2 and frontal neurons associated with encoding the stimuli correlates with the behavioral choice, although less stronger than for the differential responses; (4) when the same movements were guided by visual cues the differential activity disappeared, except for some neurons that maintained their differential activity during movement execution (Figure 7B); and (5) except for S1, all these processes are dependent on active stimulus comparisons, because they disappeared when subjects were not engaged in solving the task (Figure 5B). We found it surprising that during the comparison and postponed decision period some M1 neurons encoded information on which the decision is based. This result could suggest that M1 is engaged in the readout of sensory information from working memory, when it is required to be compared with other sensory inputs, than engaged simply in a motor response in this task. However, considering the activity observed in other cortical areas notably in PFC, VPC, MPC, and DPC during the same task, it would seem that this process involved conjoined activity of these cortical areas, not only during the postponed decision report, but also during the task components preceding it. Thus, a comparison of the strengths (Figure 4), dynamics (Figure 5A), and latencies (Figure 6) of the f1 and f2 responses and their interactions across cortical areas is instructive. Our results show that the strength of the f1 responses during the stimulus period is stronger in S1 and gradually decreasing in S2, PFC, VPC, MPC, and DPC (green dots in Figure 4). Also, more neurons with f1 responses were recorded in S1, S2, PFC and VPC than in MPC and DPC (green traces in Figure 5). This suggests that f1 is preferentially encoded in some of the cortical areas studied during this task. Accurate performance of the task can be consistent only with a sensory percept elicited during the f1 period. The lifetime of the percept directly induced by f1 could not be measured, if it were not kept in working memory. It therefore remained possible that the lifetime of a quantitative, induced percept was confined to the period of stimulation. Our results show that the induced percept can be quantitatively memorized as illustrated in Figures 4 and 5. Although the strength of this signal varies across areas (green dots in Figure 4), all of them except S1 and M1 store the value of f1 at different strengths and times during the working memory component of the task (green dots in Figure 4 and green traces in Figure 5A). These results are in accord with the proposal that there is a large cortical network that dynamically stores sensory information during working memory (Fuster, 1997; Romo et al., 2004). During the comparison period, f2 is processed similarly by the same cortical areas and also in M1 (red dots in Figure 4 and red Neuron 66, , April 29, 2010 ª2010 Elsevier Inc. 309

11 Figure 8. Neuronal Correlates of Bias Behavior (A) Distribution of coefficients a1/a2 ratios (312 experiments in four animals), obtained from linear regression analysis to the behavioral data. The histogram shows that coefficient a2 has a stronger weight than coefficient a1. (B) Bin distribution ratios for coefficients a1/a2 for neurons from medial premotor cortex (MPC) that showed coefficients a1 and a2 significantly different from zero and from each other. For each MPC neuron, we estimated the ratio between weights a1/a2 in a sliding window of 200 ms moving in steps of 20 ms, beginning during the onset of the comparison period and ending during the probe up that triggers the decision report. All these neurons showed the properties described in B and illustrated in Figures 4 and 5 (black dots and traces, respectively). This panel shows that coefficient a2 was more often higher than coefficient a1, and consequently there are more bins to the left relative to 1. (C) Distribution of bin ratios for behavioral bias/neuron bias. For each neuron of each cortical area, the resulting value a1/a2 of each cortical neuron was compared against the behavioral value a1/a2 obtained simultaneously in the same experiment. Data from primary somatosensory cortex (S1) are not shown, since there are no neurons that show the properties described in (B). DPC, dorsal premotor cortex; PFC, prefrontal cortex; M1, primary motor traces in Figure 5A). Again, accurate performance of the task can be consistent only with a sensory percept elicited during the f2 period. But, it is during the f2 period that the comparison between stored (f1) and ongoing sensory information (f2) takes place. During this period, f2 must not only be present, but f1 too, as shown in Figure 4 (red and green dots) and Figure 5 (red and green traces). The comparison between f1 and f2 is observed in S2, VPC, PFC, MPC, DPC, and M1, again at various strengths across these cortical areas (black dots in Figure 4 and black traces in Figure 5). Some of these comparison signals evolve into a signal that is consistent with the animal s motor choice (blue dots in Figure 4 and blue traces in Figure 5). During the postponed report period, the activity of all cortical areas except S1 encodes both the result of the comparison and past information on which the decision is based (Figures 4 and 5A) and covaried with the animal s decision report (Figure 7A). During the comparison and postponed delay periods, more neurons in MPC, DPC, and M1 encoded f2 (red traces in Figure 5A) than information about f1 (green traces in Figure 5A) and comparison signals (black and blue traces in Figure 5A). This would suggest that these frontal circuits are more likely to store recent sensory information (f2) than immediately preceding sensory information (f1) during this task. Consistent with this observation is the fact that the lifetime of the percept kept in working memory seems to impact the decision report, as observed in Figures 2B and 2C. These results suggest that frontal lobe circuits do not simply wait for the result of a sensory evaluation to be communicated but that actively participate in this process. Although highly speculative, we suggest that maintaining in working memory the original stimulus information on which the decision is based could serve to continuously update the postponed decision report in this task, and that very likely depends on the conjoined activity of these cortical areas. Assuming that neurons from distinct cortical circuits coordinate their activities to solve this perceptual discrimination task, we wonder how these events evolve in time. The comparative analysis of the response latencies of f1, f2, and comparison signals could shed some light on this problem. For instance, compare S1 and S2: their response latencies were significantly different (p < 0.01), with the f1 and f2 signals beginning earlier in S1 than in S2 (Figure 6). This type of comparative analysis also shows that the response latencies of S2 began significantly earlier (p < 0.01) than in VPC, PFC, MPC, and DPC. This would suggest that S2 could send information about the stimuli to these frontal lobe circuits because their response types are quite similar to S2 (Figures 4 and 5A). The question is whether frontal lobe circuits receive at the same time S2 inputs or at different times. An analysis of the response latencies for f1 and f2 showed that the PFC and VPC respond significantly earlier (p < 0.01) than DPC, MPC and M1 (f1 was not present in M1), with no significant differences between PFC and VPC (p > 0.01). This would suggest that the PFC and VPC receive the S2 inputs and that very likely cortex; S2, secondary somatosensory cortex; VPC, ventral premotor cortex; m, geometric mean (vertical line in each histogram). Gray bars in the histograms show percentage of bins close to 1 (arbitrary range, 1 ± 0.22). A value of 1 means close correspondence between neuronal activity and behavioral report. 310 Neuron 66, , April 29, 2010 ª2010 Elsevier Inc.