A Corticostriatal Path Targeting Striosomes Controls Decision Making under Conflict

Transcription

1 Cell Supplemental Information A Corticostriatal Path Targeting Striosomes Controls Decision Making under Conflict Alexander Friedman, Daigo Homma, Leif G. Gibb, Ken-ichi Amemori, Samuel J. Rubin, Adam S. Hood, Michael H. Riad, and Ann M. Graybiel

2 SUPPLEMENTAL EXPERIMENTAL PROCEDURES Animals Male Long-Evans rats ( g) were individually caged and were maintained under conditions of constant temperature (25 C) and humidity (50%), in a 12:12 h light/dark reverse cycle, with free access to regulated food and water quantities. All animal procedures were approved by the Committee on Animal Care at the Massachusetts Institute of Technology and were carried out in accordance with the U.S. National Research Council Guide for the Care and Use of Laboratory Animals. Seventeen groups of rats were included in this study, as listed below: 1) for development and validation of behavioral tasks (n = 9) 2) for localization of cortical regions that project to striosomes in the dorsomedial striatum (n = 15) 3) for localization of cortical regions that project to matrix in the dorsomedial striatum (n = 17) 4) for optogenetic inhibition of the PFC-PL projection to striosomes (n = 10) 5) for optogenetic inhibition of the PFC-ACC projection to matrix (n = 8) 6) for optogenetic activation of the PFC-PL projection to striosomes (n = 3) 7) for optogenetic inhibition of the PFC-PL projection to striosomes on the contralateral side (n = 2) 8) for optogenetic inhibition of the PFC-ACC projection to matrix on the contralateral side (n = 1) 9) for optogenetic inhibition of the PFC-PL projection to ventral tegmental area (n = 3) 10) for optogenetic inhibition of the PFC-PL projection to basolateral amygdala (n = 3) 11) for optogenetic inhibition of neurons in the PFC-PL (n = 3) 12) for electrophysiological recordings during behavioral task and neuronal targeting (n = 9)

3 13) for electrophysiological recordings with switch from benefit-benefit task to cost-benefit conflict task (n = 2) 14) for electrophysiological recordings during behavioral task combined with intrastriatal optogenetic inhibition of PFC-PL terminals (n = 2) 15) for PFC-PL electrical stimulation and validation of electrophysiological striosomal template (n = 5) 16) for simultaneous electrical stimulation of PFC-PL and intrastriatal optogenetic inhibition of PFC-PL terminals (n = 10) 17) for determining effective volume of optogenetic manipulation (n = 3) Behavioral Apparatus A T-maze, composed of an initial running track (stem) and two end-arms, was used for all behavioral experiments. Each end-arm of the maze had removable white or black walls that were changed on each experimental day. Two maze configurations were used for this study. Both had a 33 cm long stem and 33 cm long end-arms with 61 cm high surrounding walls. The first configuration included a 23 cm elevated track with a distance of 12.5 cm from the surrounding walls, whereas the second maze setup included a non-elevated track with surrounding walls at a 45 angle. Both maze configurations limited bumping of the rats headstages against the walls. Two light devices were connected to the walls above the endarms so as to be focused on the reward-delivery feeders. A click sound was used to indicate the beginning of each trial. This click triggered the laser onset during behavioral optogenetic studies using custom-designed hardware. An overhead CCD camera tracked LEDs on the headstage preamplifiers (30 Hz sampling rate) to allow detection of the position of the rats on the maze including the timing of turning events. Licking of the chocolate milk reward was detected by a photoelectric sensor (Panasonic Sunx EX31A). All behavioral and stimulation events (baseline period, click time, licking time, laser and electrical stimulation) were recorded simultaneously by

4 a Cheetah Data Acquisition System (Neuralynx, Bozeman, MT; Digital Lynx 16 SX, 32 khz sample rate). Behavioral Tasks and Reinforcement Scale Adjustment Throughout the first week, rats were habituated to the experimenter and were given free access to chocolate milk in their home cages (Figure 1B). Rats were then habituated in the T-maze for 2 weeks, during which time we blocked the running track of the maze so that the rats were required to fluctuate in forced-choice runs between the two end-arms in order to receive pure chocolate milk rewards. As the preferences for chocolate milk and aversion to light varied among the rats, we adjusted the scales of reinforcement according to the psychometric functions of each individual rat, as follows. After the initial phases of habituation described above, each rat was trained over a 3-4 week period to perform the benefit-benefit task (Figures 1A and 1B). In this benefit-benefit task, the two goals consisted of pure chocolate milk at one end-arm and chocolate milk diluted with whole milk (5-75% concentration) at the other. Two blocks of trials were run. First, in a learning-reward block, the rats performed 20 forced-choice runs to allow the rats to associate the pure chocolate milk and diluted chocolate milk rewards with their respective locations. The second block included 40 to 80 trials in which the rats could make a free choice to go to one of the end-arms. In each decision-making trial, the rats needed to wait for a variable s period on the starting platform until a start click was presented, after which the gate was opened by the experimenter (Figure 1D). The locations of pure and diluted chocolate milk were fixed across the blocks on a given day, and they were randomly varied day by day. The concentration of diluted chocolate milk was gradually decreased by random fluctuations throughout this period in order to familiarize the rats to low concentrations. The rats did not invariably choose pure chocolate even at the lowest concentrations of the diluted chocolate. After 3-4 sessions for each concentration, we obtained stable performance with consistent probabilities of choosing the

5 diluted chocolate milk over pure chocolate milk (p). Then, based on the psychometric curves of p vs. concentration of diluted chocolate milk for all rats (n = 55, Figure S1C), we choose four concentrations of diluted chocolate milk corresponding to Very low, Low, High and Very high which satisfied, respectively, p 12.5%, 12.5% < p 25%, 25% < p 37.5%, and 37.5% < p for each rat (Figures 1E, 1F, S1A and S1B; see also Methodological Considerations). Over the subsequent 4-5 weeks, the rats were trained to perform the cost-benefit conflict and cost-cost tasks (Figures 1A and 1B). In the cost-benefit conflict task, diluted chocolate milk was paired with dim light (7 Lux) to produce a combined offer consisting of low benefit and low cost. Pure chocolate milk was paired with brighter light ( klux) to produce a combined offer of high benefit and high cost. In the cost-cost task, pure chocolate milk was given at both end-arms with dim light at one end-arm and brighter light at the other. These tasks consisted of three blocks: (1) the learning-reward block, during which the rats performed 10 forced-choice runs with the lights off; (2) the learning-combination block, in which the rats performed an additional 10 forced-choice runs with the lights on; and (3) the decision-making block with 40 trials (Figure 1C), in which the lights were on. In the first week of training, pure chocolate milk or Very high concentrations of diluted chocolate milk and 0.4 klux light were used, and thereafter lower concentrations of chocolate milk and higher intensities of light were introduced (Figure S1D). Here again, after 3-5 sessions for each light intensity, we could obtain stable p levels for the cost-benefit conflict task and probability of choosing the brighter side (p ) for the cost-cost task. Then, we chose three light intensities, "Low", Medium, and High which satisfied p > 33.3%, 33.3% p > 16.6%, and 16.6% p of trials, respectively (Figures 1E and 1F). Finally in the last 2-3 weeks of training, by combining the parameters for concentration of chocolate milk and for light intensity, the rats were taught the non-conflict cost-benefit task, in which pure chocolate milk was consistently paired with dim light to produce a high-benefit, lowcost offer, and diluted chocolate milk was paired with high intensity light to produce a low-

6 benefit, high-cost offer (Figures 1A and 1B), with other task features the same as for the costbenefit conflict task. This task provided a direct control for the level of conflict in the cost-benefit conflict decision-making task (Figure 1E, see Behavioral Modeling). The exclusion criteria for rats in this study were (1) inability to run 20 forced-choice runs after an initial two-week period, (2) inability to run 40 benefit-benefit trials after three weeks of training to perform the task, and (3) variability of greater than 10% choice within the same task determined after performing 3-5 sessions. We successfully trained 55 rats, and 65 rats were excluded. Surgical Implantation Before surgery, rats were injected with meloxicam (1 mg/kg). Rats were then anesthetized with ketamine (100 mg/kg, IP) and xylazine (10 mg/kg, IP), or with isoflurane ( %) at an input flow rate of 1 l/min. Meloxicam (2 mg/kg) was administered subcutaneously once daily for a period of 2 post-surgical days, including the day of surgery. Viral Injection. For optogenetic experiments, virus containing halorhodopsin (AAV5-CaMKIIaeNpHR3.0-EYFP) or C1V1 (AAV-CaMKIIa-C1V1(E122T/E162T)-TS-EYFP), or control virus (AAV5-CaMKIIa-EYFP) was injected into the prelimbic cortex, here named PFC-PL, at AP: +3.1 mm; ML: ±1 mm; DV: 4 mm (3.2 mm from dura mater). The same viruses as listed above were also injected into the anterior cingulate cortex (here called PFC-ACC) at AP: +0.9 mm; ML: ±0.1 mm; DV: 2.6 mm, aimed with a 20 angle. Prior to each injection, the injection needle was advanced 0.1 mm beyond the target and then returned to the chosen target coordinates. A volume of 0.2 µl was injected through a 35 gauge needle attached to a NanoFil 10 µl syringe (World Precision Instruments, Inc., Sarasota, FL), at a rate of 0.02 µl/min. Following injection, the injection needle remained in place for 10 min to allow for the dispersion of the virus.

7 Headstage Implantation. For electrophysiology, headstages bearing 24 recording tetrodes and 4 bipolar stimulation electrodes were implanted. These tetrodes were aimed to reach the PFC-PL (AP: mm; ML: ± mm; DV: mm) and the dorsomedial striatum (AP: mm; ML: ± mm; DV: mm) (Figure S3D). For combining recording with optogenetics, 2-4 optical fibers (Doric Lenses, Quebec, Canada) were implanted ~ mm above the tetrodes in the dorsomedial striatum and the PFC-PL. Recording and stimulation tetrodes were constructed from tungsten wire (California Fine Wire, Grover Beach, CA) with a diameter of 0.02 mm and an impedance of kω. Viral injection and headstage implantation were separated by a minimum of one week. For manipulation of PFC-PL terminals in the ventral tegmental area and the basolateral amygdala, fibers were implanted bilaterally in the ventral tegmental area (AP: 5.3 mm; ML: ±0.7 mm; DV: 7.2 mm) or in the basolateral amygdala (AP: 3.3 mm; ML: ±4.7 mm; DV: 8.2 mm). For all coordinates, we used the Paxinos- Watson atlas (Paxinos and Watson, 1997). Behavioral Optogenetics To compare the effect of optogenetic silencing of the corticostriatal pathway among the different reinforcement contexts, we chose five maze configurations for the following studies (Figure 1A): the cost-benefit conflict, benefit-benefit (similar reward), benefit-benefit (dissimilar reward), nonconflict cost-benefit and cost-cost tasks. The goals of the cost-benefit conflict task consisted of a combination of Very high concentrations (37.5% < p of choice in the benefit-benefit task) of diluted chocolate milk and dim light at one end-arm and a combination of pure chocolate milk and High intensity (16.6% p of choice in the cost-cost task) of aversive light at the other. The goals of the benefit-benefit (similar reward) task were baited with pure chocolate milk on one end-arm and a Very high concentration of diluted chocolate milk on the other. In the benefit-benefit (dissimilar reward) task, the alternative to pure chocolate was a High concentration of diluted chocolate milk. In the cost-cost task, the goal offers consisted of a

8 combination of High intensity light and pure chocolate milk at one end and a combination of dim light and pure chocolate milk at the other. In the non-conflict cost-benefit task, the goals had a combination of pure chocolate milk and dim light at one end-arm, and a combination of Very high concentrations of diluted chocolate milk with High intensity light at the other. In sessions including optogenetic inhibition, the 40-trial block was divided into a 20-trial block with the halorhodopsin-activating laser off followed by a 20-trial block with the laser on. A 590 nm wavelength light was delivered bilaterally to the dorsomedial striatum through the implanted fibers by means of a laser source (OEM Laser Systems), fiber patch cords, a rotary joint, and a fused splitter (Thorlabs, Newton, NJ; Doric Lenses; OZ Optics, Ottawa, Ontario, Canada). Light delivery ( mw per hemisphere) was gated by a Master-9 pulse generator (A.M.P.I., Jerusalem, Israel) and a shutter (LS2 Uniblitz, Vincent Associates, Rochester, NY). Light was delivered simultaneously with the trial starting click sound, and was on for 3 s so as to overlap the period of trial initiation through goal arrival (Figures 1D and S1F). In sessions of optogenetic stimulation via C1V1, a 532 nm wavelength laser was used. Stimulation was delivered over 3 s starting with click (as above). Stimulation consisted of 46 pulses of 30 ms length, with an interval of 36 ms between pulses. Trial-by-trial analysis for behavioral optogenetics For both the laser-off and the laser-on block, we averaged the percent choice of pure chocolate milk in each trial over 24 cost-benefit conflict sessions and 5 rats (Figure S5A). Measurement of Effective Volume of Optogenetic Manipulation The effective volume of optogenetic manipulation in our experimental conditions was estimated by testing whether the effect of electrical stimulation at PFC-PL on striatal activity could be affected by laser illumination of the PFC-PL terminals within the dorsomedial striatum. The following two-block protocol was used for stimulation: (1) electrical stimulation of PL-PFC alone

9 (0.5 ms square pulse, 15 μa, 2 s interval, 100 cycles), and then (2) simultaneous PFC-PL stimulation paired with intrastriatal laser delivery (100 ms for each cycle, 590 nm, mw, 0.3 ms shutter open time). After the two blocks of stimulation, the recording tetrodes were gradually moved ventrally to find further unit activity in the dorsomedial striatum. The two-block stimulation protocol was repeated after a minimum 2-hr interval after tetrode lowering, to allow unit activity recorded by the advanced tetrodes to be stabilized. The two blocks of stimulation were repeated until all tetrodes reached a DV coordinate of 5 mm. The effect of the laser stimulation was tested by paired t-tests comparing the firing rates between the two stimulation blocks and baseline. To examine the significance of electrical stimulation vs. baseline, we compared firing rates in the interval from 3 ms to 15 ms after PFC-PL stimulation to baseline firing rates in the interval between 0 ms and 1000 ms before stimulation. Second, we compared firing rates between the first and second stimulation blocks in the interval from 0 ms to 15 ms after PFC-PL stimulation. Tetrodes were considered as responding to laser intrastriatal inhibition when differences between the first block vs. baseline and the second vs. first block firing rates were significant. Behavioral Modeling To infer the intrinsic variables of the rats choices, we implemented a logistic model (Amemori and Graybiel, 2012). The subjective value of the diluted chocolate milk was derived by the psychometric curves charactering the animals preference for each concentration (Figures S1A and S1C). We graded the subjective values (x) for the selected 5 levels of diluted chocolate milk ( Very low, Low, High, Very high and pure ) by the logit of the probability of choosing the diluted chocolate milk over the pure chocolate milk (Figures 7A, 7B and 7D-7F) (McFadden, 1974; Train, 2003). Similarly, the subjective aversions (y) for the three selected levels of light intensities ( Low, Medium and High ) were graded by the logit of the probability of choosing the variable light over the dim light in the cost-cost task (Figure 7C). With these subjective

10 values of diluted chocolate milk and different light intensities, the choice behavior in the costbenefit conflict task was modeled as p(x) = 1/(1+exp( f(x)), where p is the choice probability of choosing diluted over pure chocolate milk and x is the subjective value of diluted chocolate milk (Figures 7A and 7D). The function f(x) was formulated by the difference in utilities of two goals as f(x) = (Bx+Cy d ) (Bx p +Cy H ), where B is the sensitivity to reward, C indicates the sensitivity to aversive light, x and x p represent the subjective values of diluted and pure chocolate milk, respectively, and y d and y H represent, respectively, the subjective values of dim and bright lights. The coefficient parameters, B and C, were derived by the logistic regression from the rats choice behavior in the cost-benefit conflict task. We compared this first-order model with the models that include an interaction term of cost and of benefit, and confirmed that the first-order approximation was valid in terms of the Bayesian information criterion. Similarly, in the non-conflict cost-benefit task, the function was formulated as f(x) = (Bx+Cy H ) (Bx p +Cy d ), and the coefficient parameters, B and C, were derived independently from the dataset for this task (Figures 7B and 7E). We assume that the motivation to approach reward could be quantitatively characterized by the subjective value of the positive component of an approached goal, and that the motivation to avoid cost could be quantified by the subjective value of the negative component of an avoided goal. We thus adopted the costbenefit ratio (CBR) to approximate the degree of interaction (or conflict) between these two opponent motivations, because this ratio provides an estimate of the subjective value of the cost that the rats had to accept to gain a unit amount of benefit (Figures 1E, 7A, 7B, 7D and 7E). The CBR becomes zero in the case that the rats did not receive any cost to reach the goal, and becomes one when the rats receive, on average, similarly strong cost and benefit at the goal. For the cost-benefit conflict task, the cost-benefit ratio was thus formulated as r(x) = C(y d p(x)+y H (1 p(x)))/ (B(xp(x)+x p (1 p(x)))). For the non-conflict cost-benefit task, the costbenefit ratio was formulated as r(x) = C(y H p(x)+y d (1 p(x)))/ (B(xp(x)+x p (1 p(x)))). In the benefit-benefit task, the difference in utility was formulated as f(x) = B(x x p ), and B was derived

11 from the dataset for this task (Figure 7F). In the cost-cost task, the choice behavior was modeled as p (y) = 1/(1+exp( C(y y d ))), where p is the choice probably of choosing bright over dim light and C was derived from the dataset for this task (Figure 7C). To characterize how the optogenetic inhibition (halorhodopsin, solid pink lines) and excitation (C1V1, dotted pink lines) of the PFC-PL terminals (putative striosomal pathway) changed the choice behavior, we performed regression analysis in the cost-benefit conflict, nonconflict cost-benefit and cost-cost tasks (Figures 7A-7C). For each task, we derived the optimal C value that best fits the behavior induced by optogenetic manipulations after fixing other parameters. The effects of halorhodopsin and C1V1 manipulations could be explained by decrease and increase in the sensitivity to cost, respectively. Although the optogenetic effect appeared only when the outcome of the decision included an aversive component, we do not rule out the possibility that the optogenetic effect could reflect an increase in sensitivity to reward (B) that exclusively happens under the approach-avoidance context of the cost-benefit conflict task. To examine how the optogenetic manipulation of the PFC-ACC terminals (putative matrix pathway) affects the choice behavior, we also performed regression analysis based on the data obtained in the cost-benefit conflict, non-conflict cost-benefit and benefit-benefit tasks. We observed an increase in B in all task conditions (cyan lines in Figures 7D-7F), suggesting that the optogenetic manipulation increased the sensitivity to reward (B) by increasing the approach motivation to obtain reward irrespective of the task condition. However, we do note that there was a marginal effect of optogenetic inhibition observed in the cost-cost task (Figure 2D, t-test, p < 0.1), suggesting that the effect could not be strictly attributed to the exclusive change in B but could include modulation of aversive sensitivity to the light. Thus we do not claim that the PFC-ACC pathway is exclusively involved in controlling the sensitivity to benefits, but suggest that it plays a major role in controlling sensitivity to benefit irrespective of the task condition.

12 Behavioral Electrophysiology, Spike Sorting and Cluster Quality Extracellular spike activity was recorded as the rats performed the five different types of task that we used for the behavioral optogenetic study, using tetrodes implanted in the PFC-PL and dorsomedial striatum. A tetrode channel served as reference, and a screw implanted in the skull as an animal ground. The signals were band-pass filtered for multiunit recording ( Hz) and were acquired through a Cheetah Data Acquisition System (Neuralynx Digital Lynx 16 SX, 32 khz sample rate). Single units were identified as isolated waveform clusters by the use of Offline Sorter (Plexon, Inc., Dallas, TX) and were confirmed using an in-house custom program (Friedman et al., 2015). Neuronal Targeting After each daily behavioral session, each rat was placed into an empty plastic cage, and orthodromic and antidromic stimulation procedures, described below, were performed to allow determination of response properties of neurons at sites monitored or manipulated during the behavioral session. Antidromic stimulation. A tungsten bipolar stimulation electrode was constructed using two wires of a tetrode as the anode and two wires of the tetrode as the cathode. Electrical stimulation (0.5 ms square pulse, 15 μa, 2 s interval between pulses; 100 pulses) was delivered to the dorsomedial striatum using ISO-Flex and Master-9 systems (A.M.P.I.) (Figures 3A and 3B). For each PFC-PL neuron, we counted the total number of spikes for all 100 stimulation pulses in a window from 1 to 10 ms, aligned to striatum stimulation (see above for session schedule). A PFC-PL neuron was defined as a candidate striosome-projecting neuron if it generated at least 10 spikes in this window over 100 stimulation pulses (Figures 3A and 3B), a criterion adopted in order to exclude neurons with very low spontaneous activity.

13 The final classification of each PFC-PL neuron as a putative striosome-projecting neuron was based on the following statistical test: for each PFC-PL neuron, the mean firing rate between 1 and 10 ms after all stimulation pulses was calculated. Additionally, the mean and SD of the firing rate in the 1 s baseline window immediately preceding the stimulation pulses were calculated. A PFC-PL neuron was classified putatively as a striosome-projecting (PFC-PLs) neuron if its mean firing rate in the post-pulse window was at least 4 SDs greater than that in the baseline window. Orthodromic stimulation. Similarly, a tungsten bipolar stimulation electrode was implanted in the PFC-PL, and electrical stimulation (0.5 ms square pulse, 15 μa, 2 s interval; 100 cycles) was delivered to detect orthodromic responses in PFC-PL (Figure 3E and 3F). The response of striatal neurons was used for the classification of putative striosomal and non-striosomal (putative matrix) neurons (see below). Classification of Putative Striosomal Neurons and Putative Matrix (Non-Striosomal) Neurons Taking advantage of the highly predominant projection of PFC-PL neuron to striosomes (Figures 2A and S2A), we sought an electrophysiological method that could reliably predict in which compartment the recording tetrode was placed in vivo. As a preliminary study, we electrically stimulated at PFC-PL in parallel with the tetrode recording, and histologically examined the tip position (see Triple Immunofluorescence Staining and Image Analysis). The PFC-PL stimulation evoked either biphasic (short-latency excitation followed by inhibition) or triphasic (including rebound after the excitation-inhibition) responses in spike activity when tetrodes were estimated to be in a putative striosome, as histologically estimated post-mortem (Figures 3M and 3N), whereas with tetrodes placed in putative matrix regions, as estimated by histology, such patterns of spike activity rarely occurred (Figure 3O). Based on

14 these experiments, we set the three grades for the classification of striatal medium spiny neuron (SPN) as candidate striosomal or candidate matrix neurons. The three grades were based on the first two phases of response, excitation and inhibition: Grade 3: both excitation and inhibition (candidate striosomal neuron) Grade 2: either excitation or inhibition (some response to stimulation) Grade 1: neither excitation nor inhibition (candidate matrix, i.e., non-striosomal, neuron). However, many neurons had different levels of excitation and/or inhibition. Therefore, we developed an algorithm to verify the significance of the neuronal response to the PFC-PL stimulation. Our algorithm had three steps: (1) For each unit, we determined the start and end times of short-latency excitation and inhibition by optimizing a metric that we named "peak-power" within a predefined time window (3-15 ms for short-latency activation and up to 250 ms after stimulation for inhibition). Peakpower was defined as the difference between the number of spikes in an interval and the number of spikes expected in the same interval based on the baseline firing rate (i.e., baseline firing rate x width of time interval). This was maximized for short-latency activation (Figure 3I) and minimized for the inhibition that followed. (2) We then used bootstrapping to check the statistical probability of short-latency excitation and inhibition: we computed 1000 bootstrapped baselines using the 1 s time interval before the PFC-PL stimulation. If we had previously been optimizing a peak window, then we searched for activation as above in each of the 1000 baselines. Similarly, if we had previously been optimizing an inhibition window, we did the same in each baseline to create a distribution. Means and SDs were calculated for each bootstrapped distribution. (3) We then combined the response probabilities of excitation and inhibition. The mean of the two phases was combined as: μ fffff = μ eee + μ iih

15 where μ eee and μ iih, are the means for changes from baseline during the excitation and the inhibition phases obtained from the bootstrapped distributions. The SD of the combination of changes from baseline for the two bootstrap distributions was defined as: σ u = σ2 2 eee + σ iih where σ eee aaa σ iih are, respectively, the SDs for the excitation and inhibition phases. Responses to PFC-PL stimulation that were two SDs above the mean were determined as significant. Classification of SPN and HFN Neuronal Types Recorded units were classified as putative SPNs or HFNs on the basis of their spike widths, firing rates, and interspike intervals (ISIs). Our starting point was the large body of previous work that classified SPNs, HFNs, and tonically active neurons (TANs) based on spike width, ISI and firing rates (Adler et al., 2013; Atallah et al., 2014; Berke, 2008; Gage et al., 2010; Jin et al., 2014; Thorn and Graybiel, 2014). To this approach, we added the further dimension of the natural logarithm of the median ISI divided by the mean ISI, which aids in the classification of neurons that have variable spiking patterns, such as SPN bursty neurons having many short-isi bursts separated by longer ISIs that lower the mean. To aid in the classification, we represented neurons as points in a space in which the three dimensions are mean spike width, firing rate, and the natural logarithm of the median/mean ISI (Figure S3E). Three clusters were apparent by eye in striatal data plotted in this manner. The SPN and HFN clusters had a small amount of overlap in the region of short mean/median ISIs and low firing rate; we omitted analysis of the neurons in this region. A Gaussian mixtures method (the gmdistribution function in the MATLAB Statistics toolbox) was used to cluster the neuronal data in this space. We identified four clusters: SPNs, TANs (not considered here), HFNs, and unidentified overlapping units.

16 Binning and Alignment of Neuronal Activity We excluded the 17% of trials with a click-to-lick period greater than 5 s, because either these trials were too long to average into our trial activity, or they had a hardware malfunction. The click-to-lick period of the remaining trials was 2.65 ± 0.69 s (Figure S1F). Within each peri-trial period, we calculated the firing rate in each of 600 time bins (number of spikes / bin size). For each trial, we adjusted the bin size such that the starting click occurred between bins 300 and 301 and the first lick occurred between bins 360 and 361. Analysis and Plotting of Task-Related Neuronal Activity Each peri-trial period began 20 s before the click or at the beginning of the session, and ended 20 s after the click. To illustrate firing rates across the spatial layout of the maze, and to capture firing rates during not only the maze runs but also the periods before the click and after the first lick, we used a dual plotting system. We illustrate the maze itself in solid outline, with the plots oriented so that the start position (where the rat is at the time of the start-click) is down and the end-arms are up (with their terminations representing where the first lick occurs). We represent with the inner solid lines the time periods before the click and after the first lick. Thus in the inner maze plots, time and space are represented uniformly, and in the extensions, time is represented uniformly. Firing rates were calculated for 200 bins (bins 231 to 430, see Binning and Aligning of Neuron Activity), approximately the time window 3 s before click to 3 s after click, and these were converted to z-scores (normalized firing rate = (firing rate mean(baseline firing rate))/sd(baseline firing rate)), using the firing rate and SD in the baseline period from bin 60 to bin 240, approximately 11 s to 3 s before the click (e.g., Figures 3C top and 3G top) For line-plotting SPN activity, we used the means and SEM of the firing rate of each time bin (e.g., Figures 4D and 4E). For HFNs, we used a min-max normalization of intra-burst firing rate, as well as the overall rates of firing of these neurons, based on our finding that the firing

17 rates were high and not apparently differentiated across behavioral epochs in the different tasks (data not shown). Average Baseline Activity For laser-on and laser-off blocks, we averaged the baseline firing rate for striosomal SPNs and non-striosomal SPNs (all SPNs recorded in the dorsomedial striatum that were not identified as striosomal) (Figure S5B). Neuronal Burst Extraction We employed a burst extraction analysis to identify phasic neuronal activity patterns (Figures S6A-S6E). In this method, the algorithm first calculated the mean and SD of the firing rate of a neuron over the entire time of the recordings, including both trial periods and inter-trial intervals. These were defined as the baseline mean and SD of the firing rate. Potential bursts were defined as sets of consecutive spikes with instantaneous firing rates higher than the baseline firing rate (Figure S6A). The firing rate within potential bursts was then computed (Figures S6C and S6E). If this intra-burst firing rate was greater than 3 SDs above the baseline mean, the potential burst was categorized as a burst. To illustrate the bursts detected by our analysis, we plotted them in a color-scale histogram (e.g., Figure 3C bottom and 3G bottom). Bursts for each neuron were drawn as rectangles from the start of the burst to the end of the burst, with a color scale ranging from yellow (low firing) to red (high firing). Each neuron s bursts are plotted on a separate row. For all plots illustrating activity without optogenetic manipulation, we plotted a 1.5 s time interval centered exactly at turn; for sessions with optogenetic manipulation, we plotted the full 3 s of laser duration. For heat-map plotting relative to maze location of HFN intra-burst spike activity (Figure 6A), the firing rate outside of bursts was set to zero, and the resulting intra-burst firing was used

18 in subsequent analyses (see Analysis and Plotting of Task-Related Neuronal Activity). The bursty firing detected by the detection algorithm showed sharp task-related differences (Figures 6A and S6F). Averaged Z-Scores in Click-to-Turn Period For the average z-score bar graphs for PFC-PLs neurons, putative striosomal SPNs, putative matrix SPNs, and HFNs (Figures 3D, 3H, 3L and 6B), the mean and SD of the z-scores were calculated for a small time-window (0 s to 0.66 s after click, bins 301 to 315). Analysis of Striosomal Responses in Behavioral Switch from Benefit-Benefit to Cost- Benefit Conflict Tasks To investigate how the putative striosomal neurons respond to the different reinforcement contexts, we ran a two-block consecutive recording task, following single neurons across each task-period. The first block included 19 trials of the benefit-benefit task. The second block included 21 trials of the cost-benefit conflict task. Both the benefit-benefit and cost-benefit conflict blocks were preceded by 20 learning-reward and learning-combination trials for the corresponding tasks. We identified 10 putative striosomal neurons successfully recorded across the full experimental time (Figure 4). The average firing rate of each trial, except the reminder trials before the cost-benefit block, from click to lick was calculated in order to illustrate changes in firing rate after the task switch (Figure 4C). Analysis of Putative Striosomal Neuron Responses to Optogenetic Disconnection of PFC-PL and Striatum To investigate how putative striosomal SPNs, identified by their responses to PFC-PL stimulation, respond to the PFC-PL disconnection, we compared their activity during laser-off and laser-on blocks (see Behavioral Optogenetics). First, neurons in which the baseline firing

19 rates differed between the two blocks by more than 50% were removed (167 of 343 initial neurons filtered out due to inconsistent baseline). Second, putative striosomal neurons with an average firing rate change of at least 15% in the click-to-turn period between the laser-off and laser-on periods were selected (49 of the 176 remaining neurons selected). Of those 49 striosomal SPNs responsive to optogenetic inhibition, 46 showed an increased firing rate in the click-to-turn period (Figure 5). The effect of laser manipulation was determined to be significant both relative to the baseline firing rate of the same neuron and relative to firing in the initial laser-off block, by paired t-tests. To show the immediate change in firing rate after the laser is turned on, the average firing rate of each trial was calculated and plotted (Figure 5C). Timing of PFC-PLs and HFN Activation and SPN Inhibition in Cost-Benefit Conflict Task To compare the pattern of activity between SPNs and HFNs in the cost-benefit conflict task, pairs with one each of these two types of neurons recorded from a single tetrode were selected from the dataset (Figures 6C and S6G). Similarly, pairs of simultaneously recorded PFC- PLs/HFN neurons were selected if both neurons showed task-related activity. Their neuronal activities were binned as described above. Peak firing rates of the HFNs in the pre-turn period were identified, and the firing rates of the SPNs were aligned to the peaks of the paired HFNs and averaged across the pairs (Figures 6C and S6G). To compare the time-course of the response pattern of paired neurons, the time of peak neuronal activity of PFC-PLs neurons and HFNs and the time of valleys in SPN activity were plotted by pair (Figure 6E). Firing rate peaks or valleys were determined as firing rate maximums or minimums within the time window starting from 3 s before the click to 3 s after the first lick. Comparison of SPN Activity in Response to Varying Frequency of Phasic (Burst) and Tonic (Non-Burst) HFN Activity

20 In order to test the influence of phasic (burst) firing of HFNs on the activity of SPNs, we calculated the average activity of SPNs during periods of both tonic and phasic HFN activity within different frequency ranges. First, the firing rates of individual HFNs were calculated for successive 240 ms recording bins within each peri-trial period. Then, using neuronal burst extraction as described above, each bin was categorized as having either phasic (burst) HFN activity if any HFN had one or more bursts during the bin, or tonic (non-burst) HFN activity if not. The firing rate of each HFN within a bin was rounded to the nearest 5 Hz; this was done separately for burst and non-burst bins. For all burst or non-burst bins having a given rounded HFN firing rate, the average firing rate of SPNs recorded during the same bins was calculated; only SPNs recorded on the same tetrodes as the HFNs were included. To combine data from different HFN-SPN pairs, their firing rates were min-max normalized. Correlations between firing rate of SPNs and the tonic or phasic activities of HFNs were tested by chi-square test (Figures 6D and S6H). Comparison of Neuronal Activity between SPNs and HFNs in Response to PFC-PL Stimulation To compare SPN and HFN activity following PFC-PL stimulation, we used electrical stimulation of PFC-PL combined with intrastriatal laser illumination (see Measurement of Effective Volume of Optogenetic Manipulation). For this experiment, tetrodes were aimed at coordinates for the dorsomedial striatum (AP: mm; ML: ± mm; DV: mm). Two optical fibers were implanted in the dorsomedial striatum at the following coordinates: AP: 1.5 mm; ML: ±1.6 mm; DV: 3.6 mm. Electrical stimulation was made between the anode (AP: 3.7 mm; ML: ±0.7 mm; DV: 3.6 mm) and cathode (AP: 2.7 mm; ML: ±1.0 mm; DV: 3.6 mm) of the stimulating tetrode implanted in the PFC-PL. Sessions in this experiment consisted of two blocks: one for electrical stimulation, and a second for a combination of stimulation with intrastriatal optogenetic silencing, as described above (Figure 6G).

21 First the spike activities of all SPNs and all HFNs recorded during the laser-off block were aligned to the time of the PFC-PL stimulation. The response to PFC-PL stimulation was estimated by comparing the firing rates between the period 0 to 1 s before stimulation and the period 3-20 ms after stimulation using paired t-tests. SPNs and HFNs with significant responses to stimulation were selected, and the latencies of the responses of the SPNs and HFNs were compared by Wilcoxon rank sum tests and two-sample Kolmogorov-Smirnov tests (for cumulative sum of the distributions, MATLAB statistics toolbox; Figure 6F). The firing rates of the selected neurons recorded in the second block were aligned to the time of PFC-PL stimulation (Figure 6H). The effect of the laser manipulation was tested by comparing firing rates in the laser-on and laser-off blocks by paired t-tests. Tetrodes that exhibited any response to electrical stimulation were kept in place through perfusion, and histological evaluation of tetrode tip positions was performed (see Triple Immunfluorescence Staining and Image Analysis). Triple Immunofluorescence Staining In the group of rats that was used for identification of striosome and matrix recording sites, measurement of distances between tetrode tips and striosome boundaries, and analysis of the responses of SPNs and HFNs to PFC-PL stimulation and optogenetic inhibition (Figures 3M-3O and 6G-6H), electrolytic lesions of various sizes were made at some tetrode tips by passing current between two pairs of tetrode wires (3-5 μa for 2-5 s) approximately 36 hr before perfusion. Tetrodes were kept in place through perfusion, and tetrode tip positions were identified histologically (Figures 3M and 3N). Rats were perfused with 0.9% saline followed by 4% paraformaldehyde in 0.1M phosphate buffer (PB). Brains were blocked, frozen in dry ice and cut in the coronal plane on a sliding microtome. Sections were stored in sodium azide in 0.1M PB.

22 For immunohistochemistry of behavioral optogenetics brains, sections were rinsed 3x2 min in 0.01M phosphate buffered saline (PBS) containing 0.2% Triton X-100 (Tx) and then were pre-treated with 3% H202 in PBS-Tx for 10 min. Then the sections were rinsed 3x2 min in PBS- Tx, and blocked in tyramide signal amplification (TSA) blocking reagent (PerkinElmer) for 20 min. The sections were then incubated with primary antibody solution containing goat anti-gfp (ab5450; Abcam), rabbit anti-mu opioid receptor (MOR1; 24216; Immunostar), and mouse anti- CD11b/c (ab1211; Abcam) in TSA blocking reagent in PBS-Tx for 48 hr at 4 C with gentle shaking. Following primary incubation, sections were rinsed 3x2 minutes in PBS-Tx, and then were incubated in goat SuperPicture polymer HRP (Life Technologies) for 10 min. Sections were rinsed 4x3 min in PBS-Tx, and incubated in TSA Plus Fluorescein (PerkinElmer) for 10 min. Following TSA amplification, sections were rinsed for 4x3 min in PBS-Tx, and then were incubated for 2 hr in the secondary antibody solution containing goat anti-rabbit Alexa Fluor 647 [1:300] (Life Technologies) and goat anti-mouse Alexa Fluor 546 [1:300] (Life Technologies) in TSA blocking reagent in PBS-Tx. Sections were then rinsed 3x2 minutes in 0.1M PB, mounted onto subbed glass slides, and were coverslipped using ProLong Antifade Reagent (Life Technologies). The brains examined for identification of striosomal and matrix recording sites, measurement of distances between tetrode tips and striosome boundaries, and analysis of the responses of SPNs and HFNs to PFC-PL stimulation and optogenetic inhibition (Figures 3M-3O and 6G-6H) were processed similarly, but a different anti-gfp primary was used, and MOR1 rather than EYFP was amplified. Some sections were not stained for EYFP. We used TSA blocking reagent in PBS-Tx containing chicken anti-gfp (gfp-1020, Aves Lab), rabbit anti- MOR1, and mouse anti-cd11b/c as the primary antibody solution. Following primary incubation and rinses, sections were incubated in biotinylated goat anti-rabbit [1:500] for 1 hr, rinsed for 3x2 min, incubated in streptavidin-hrp for 30 min, rinsed for 3x2 min, incubated in TSA Biotin

23 for 15 min, again rinsed for 3x2 min, and incubated for 2 hr with a solution containing streptavidin Alexa Fluor 647 [1:1000] (S-21374) (Life Technologies), goat anti-chicken Alexa Fluor 488 ([1:300]) (Life Technologies), and goat anti-mouse Alexa Fluor 546 [1:300] in TSA blocking reagent in PBS-Tx. Sections were rinsed and mounted as above. Image Analysis Measurement of Distances between Tetrode Tips and Striosome Boundaries. Serial sections (30 μm intervals) were scanned with a Zeiss Axio Imager.Z2 epi-fluorescence microscope with TissueFAXS Whole Slide Scanning System Software (Figures 2A, 2B, and 3M), and 3Dreconstructed with the aid of the MultiStackReg plugin (Brad Busse) for the Fiji image processing package. CD11-positive tracks of tetrodes and optical fibers were traced to their tips, and more finely aligned 3D reconstructions were generated around the tips with the aid of custom MATLAB software and MultiStackReg, based primarily on the pattern of local white matter bundles (internal capsule) in processed images obtained from MOR1 images. The images used for these finely aligned 3D reconstructions were cropped to 0.6 x 0.6 mm before image registration. We manually checked and corrected all finely aligned image stacks. On a case-by-case basis, we evaluated the certainty of tip placement as being either in a striosome or in the matrix. Striosome tips evaluated as certain were surrounded by substantial MOR1 staining. Certain matrix tips lacked substantial MOR1 staining, including MOR1 staining in a filamentous pattern, in their vicinity, both in the tip-containing section and in adjacent sections. For both certain striosome and certain matrix tips, generally if the lesion produced any damage to the MOR1 staining at the tip (visible as a dark spot), the staining in sections adjacent to the lesion needed to be undamaged in order for the placement to be classified as certain. We evaluated 29% of tetrode tips as being certain. For those tips outside striosomes, we measured the distance between the tip and the nearest striosome boundary in 3D, assisted by a custom MATLAB program.

24 Densitometric Analysis. The selectivity of optogenetic manipulation for the targeted compartment for PFC-PL and PFC-ACC cases was evaluated as follows. First, we chose typically five sections with μm intervals around the tip. EYFP, MOR1 and CD11 signals as well as autofluorescence of the tissue through the DAPI filter in a ca. 2.5 x 2.5 mm area around the tip were scanned with a Zeiss LSM 510 confocal microscope with ZEN software, and these images were 3D-reconstructed. Using MOR1 and the autofluorescence channel, we prepared image filters to categorize each pixel as belonging to striosome, matrix, internal capsule, corpus callosum or neocortex with or without septum (Figure S2D). Regions with lesions, damage made through the immunostaining process or obscured by debris on the slide were masked. Then the EYFP signal from each brain region was scored by three different strategies. Binary Model. In this strategy, we evaluated the expression levels of EYFP around the tips of optical fibers by conventional densitometric methods. We defined the region of interest (ROI) as the prolate spheroidal region of 0.5 mm equatorial radius and 1 mm polar radius below the tip coordinates (Figure S2D), and the EYFP intensities of pixels within the ROI were summed for each brain region of each section. Distance-Normalized Model. Because of the propagation and scattering of laser illumination in brain tissue, the EYFP-positive axons close to an optical fiber tip should be more effectively manipulated by the laser (Chow et al., 2010; Petreanu et al., 2007; Zhang et al., 2007). To incorporate this idea into the analysis, the EYFP intensity for each pixel was weighted by the laser transmittance at coordinates relative to the tip of the optical fiber (Figure S2D), and the weighted intensities were summed together for each brain region of each section. The laser transmittance was estimated according to the model described by Foutz et al. (2012). The

25 absorbance coefficient (0.027 mm 1 ), scattering coefficient (9 mm 1 ), and refractive index (1.36) of brain tissue for yellow light were taken from previous reports (Bernstein et al., 2008; Tuchin, 2007; Yaroslavsky et al., 2002), and the numerical aperture of the optical fiber we used was 0.37 (Doric Lenses). The ROI was defined as the set of pixels with an irradiance of at least 0.01 mw/mm 2. Measured Response. The laser irradiance strongly decreases with the distance from the tip, whereas the effect of the laser on the EYFP-positive axons should saturate if they are close enough to the tip of the optical fiber (Petreanu et al., 2007). Therefore, the laser irradiancebased weighting could overvalue the proximal EFYP signals for our purpose of estimating the compartmental selectivity of the optogenetic manipulations. To avoid this potential error, the EYFP intensity for each pixel was weighted, in this strategy, by the probability of the laser to cancel the PFC-PL stimulation-induced postsynaptic response measured with the procedures described in Measurement of Effective Volume of Optogenetic Manipulation (Figure S2C), and these weighted intensities were summed for each brain region in each section. First, the linear regression line for the percentage of responsive tetrodes vs. relative depth from the optical fiber was calculated, and the EFYP intensity of each pixel within the column of 1 mm diameter and 2.11 mm height below the tip of the optical fiber was weighted according to its relative depth (Figure S2D). The obtained EYFP scores were normalized to the total of the five brain regions analyzed, and these values were averaged across the brains analyzed. We also analyzed the EYFP score per areal dimension by dividing the EYFP score by the number of pixels belonging to each brain region in the ROI. The resulting scores were normalized to the average of the five brain regions, and averaged across the brains. The EYFP scores of each category were calculated by custom MATLAB software.

26 Histological Staining and Evaluation of Electrophysiology Brains For rats in which tetrode recordings were made during the behavioral tasks (and which were not used for behavioral optogenetics or measurement of distances between tetrode tips and striosomal boundaries), electrolytic lesions were made at tetrode tips (25 μa for 20 s) approximately 24 hr before perfusion. Tetrodes were kept in place through perfusion, and lesion positions were identified histologically in cresyl violet stained sections (Figure S3D). Statistics In order to distinguish the significance level of optogenetic manipulations (Figures 2D, 2F, 2H- 2J, S2F and S2G), two-tailed t-tests were used. The variances of data points in all tasks were examined by F-tests and were determined to be similar (Excel statistics toolbox). The statistical differences of counts in histograms (Figure 3O) were determined by the chi-square test (MATLAB statistics toolbox). The significant differences of firing rates during the click-to-turn period across the five tasks (Figures 3D, 3H and 6B) were determined by two-tailed t-tests (MATLAB statistics toolbox). To test for significant differences among patterns of activity in all 5 tasks between PFC-PLs neurons vs. PFC-PL neurons excluding PFC-PLs neurons, or putative striosomal vs. putative matrix neurons, we used MANOVA tests and two-way ANOVA tests (MATLAB statistics toolbox). Comparison of EYFP scores among labeled regions was made by one-way ANOVA followed by a Tukey's post hoc test. The significance of differences in means of in-run activity between benefit-benefit blocks vs. cost-benefit blocks (Fig 4C) and between laser-off blocks vs. laser-on blocks (Fig 5C) was determined by paired t-tests. Methodological Considerations Predominant Projection of PFC-PL to Striosomes, and of PFC-ACC to Matrix. Our attempt to target functionally the preferential projections from the PFC to the dorsomedial striatum