Emergence of Context-Dependent Variability across a Basal Ganglia Network

Transcription

1 Article Emergence of Context-Dependent Variability across a Basal Ganglia Network Sarah C. Woolley, 1,2, * Raghav Rajan, 1,3,4 Mati Joshua, 3,5 and Allison J. Doupe 1,3 1 Department of Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA 2 Department of Biology, McGill University, Montreal, QC H3A 1B1, Canada 3 Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA 4 Indian Institute of Science Education and Research, Pashan Road, Pune , Maharashra, India 5 Department of Neurobiology, Duke University, Durham, NC 27710, USA *Correspondence: sarah.woolley@mcgill.ca SUMMARY Context dependence is a key feature of corticalbasal ganglia circuit activity, and in songbirds the cortical outflow of a basal ganglia circuit specialized for song, LMAN, shows striking increases in trial-bytrial variability and bursting when birds sing alone rather than to females. To reveal where this variability and its social regulation emerge, we recorded stepwise from corticostriatal (HVC) neurons and their target spiny and pallidal neurons in Area X. We find that corticostriatal and spiny neurons both show precise singing-related firing across both social settings. Pallidal neurons, in contrast, exhibit markedly increased trial-by-trial variation when birds sing alone, created by highly variable pauses in firing. This variability persists even when recurrent inputs from LMAN are ablated. These data indicate that variability and its context sensitivity emerge within the basal ganglia network, suggest a network mechanism for this emergence, and highlight variability generation and regulation as basal ganglia functions. INTRODUCTION The basal ganglia are critical for the learning, planning, and execution of movement (Graybiel, 2008; Grillner et al., 2005). Moreover, the activity of cells within the basal ganglia appears to integrate motor commands with signals reflecting motivation. For example, basal ganglia activity is modulated by behavioral context such that the neural response associated with performing a movement when a reward is expected can be entirely different, or even absent, under conditions when no reward is forthcoming (Gdowski et al., 2001; Handel and Glimcher, 2000; Kawagoe et al., 1998). Such an integration of motor control and reward signals may be critical for on-line modulation of motor behavior as well as for motor and reinforcement learning. A recent addition to our view of basal ganglia motor signals and their modulation by behavioral context has come from studies of a songbird cortical-basal ganglia circuit, the anterior forebrain pathway (AFP), dedicated to a single behavioral output, the bird s learned song. This circuit, which is essential for both juvenile and adult song plasticity (Figure 1A; Andalman and Fee, 2009; Bottjer et al., 1984; Kao et al., 2005; Ölveczky et al., 2005; Scharff and Nottebohm, 1991; Warren et al., 2011), is strikingly sensitive to social context. Both immediate early gene (IEG) induction and neural activity change markedly when adult birds switch from singing alone ( undirected singing ) to singing a courtship song to a female ( directed singing ) (Hessler and Doupe, 1999; Jarvis et al., 1998; Kao et al., 2008). In particular, during undirected singing, activity in the cortical output nucleus of the AFP, the lateral magnocellular nucleus of the anterior nidopallium (LMAN), is dominated by bursts and is highly variable in the timing and reliability of firing across song renditions (Figures 1B and 1C). During directed singing, however, LMAN neurons shift rapidly to single-spike activity that is precisely timed and reliably locked to song across trials (Figure 1C; Hessler and Doupe, 1999; Kao et al., 2008). With the same switch from solo to social contexts, the song itself changes, in a dopaminedependent manner, from song that is variable to stereotyped performance song that is less plastic, is preferred by females, and seems to represent the bird s best current version of song (Kao and Brainard, 2006; Kojima and Doupe 2011; Leblois et al., 2010; Murugan et al., 2013; Sakata and Brainard, 2009; Sasaki et al., 2006; Woolley and Doupe, 2008). Lesions or inactivations of LMAN eliminate the song variability seen in adult undirected song, as well as in juveniles that are still learning to sing, indicating that the behavioral variability is not a default state but actively requires cortical-basal ganglia circuitry (Goldberg and Fee, 2011; Kao and Brainard, 2006; Ölveczky et al., 2005, 2011; Scharff and Nottebohm, 1991). Together, these data suggest that one crucial function of cortical-basal ganglia circuits is to generate behavioral variability. In adult birds, such variability has been shown to generate motor exploration that can enable reinforcement learning (Andalman and Fee, 2009; Tumer and Brainard, 2007; Warren et al., 2011), and the structure of the variance in song crucially determines what is learned (Charlesworth et al., 2011). Because most neurophysiological studies of contextdependent variability have focused on the cortical nucleus LMAN, it remains unclear whether this neural variability and its social sensitivity emerge in the cortical-basal ganglia circuit or elsewhere in the brain. To investigate these questions, we 208 Neuron 82, , April 2, 2014 ª2014 Elsevier Inc.

2 Figure 1. Social Context Influences the Activity of Single Neurons in LMAN (A) In the motor pathway (white boxes), HVC (proper name) projects to the robust nucleus of the arcopallium (RA), which innervates motor neurons used for singing ( vocal output ). The anterior forebrain pathway (AFP, gray boxes) also receives input from HVC. Specifically, HVC neurons project to the basal ganglia nucleus Area X, which in turn projects to the lateral magnocellular nucleus of the anterior nidopallium (LMAN) via the medial nucleus of the dorsolateral thalamus (DLM). LMAN provides the output of the AFP to the motor pathway and sends a recurrent projection back to Area X. The connectivity of putative cell types from which we recorded (filled in black) is indicated by circles and lines; glutamatergic HVC X neurons project to GABAergic spiny neurons (SNs) in Area X that inhibit pallidal output neurons (P). Pallidal neurons send strong inhibitory projections to the thalamus. The other classes of neurons in Area X (interneurons and GPe neurons) are omitted for simplicity. (B and C) As described in Kao et al. (2008), social context influences the firing rate and variability of neurons in LMAN. (B) Spectrogram of a representative song motif. (C) Raster plots of firing of a single LMAN neuron during 25 motif renditions of undirected (top) and directed (middle) song and during ten nonsinging epochs (bottom). course of a circuit, and point to the striatopallidal network as a key locus in the generation of variability and its context sensitivity. RESULTS recorded neural activity during singing at multiple earlier stages in the circuit, including (1) the inputs to basal ganglia from cortical premotor neurons in HVC, and both (2) putative spiny neurons (SNs) and (3) internal globus pallidus-like (GPi) neurons in the avian basal ganglia nucleus Area X (Figure 1A). We found that, in marked contrast to LMAN, the activity of striatal-projecting neurons in HVC was highly stereotyped and equally precise in both behavioral conditions. Similarly, the firing of Area X SNs was equally precisely timed in both social conditions, although the firing rate of SNs showed some social context dependence. In contrast, the putative GPi output cells in Area X displayed dramatic social modulation of both firing rate and spike timing variability, and this variability did not depend on the recurrent cortical input from LMAN neurons. These data provide a striking illustration of how neural variability emerges and acquires context sensitivity over the To investigate where the marked trial-bytrial neural variability of LMAN and its social context dependence arise (Figures 1B and 1C), we recorded and analyzed the singing-related activity of 28 neurons in the cortical input nucleus to the AFP and HVC, and 70 neurons within the basal ganglia nucleus Area X (Figure 1A) in male zebra finches within and across renditions of the bird s motif (Figure 1B). AFP-Projecting HVC Neurons We first examined the activity of HVC neurons projecting to Area X (HVC X, n = 28 single units in 6 birds, including 10 antidromically identified from Area X; Figures 2A 2C; Experimental Procedures). HVC X neurons exhibited low but measurable spontaneous activity (Table 1; Figure 2E, bottom panel). During singing directed at a female, HVC X neurons increased their activity and displayed sparse firing events (1 5/motif) that consisted of single spikes or bursts that were tightly locked to song (Figure 2E; e.g., Kozhevnikov and Fee, 2007). For example, the cell in Figure 2 shows a firing event at the start of syllable b. We recorded the activity of the same HVC X neurons during interleaved Neuron 82, , April 2, 2014 ª2014 Elsevier Inc. 209

3 Figure 2. Social Context Has No Effect on the Firing of Individual HVC X Neurons (A) Recordings from HVC X neurons (gray circle). (B) Amplitude oscillogram of a song (top) and raw trace of the associated neural activity. (C) Trace of the stimulus artifact (black arrow) and evoked spikes (gray arrow) resulting from antidromic stimulation of HVC from Area X. (D) Spectrogram of a representative song motif ( abcdefg ). (E) Raster plots of a single HVC X neuron during ten motif renditions of directed (DS; top) and undirected (US; middle) song, and during six nonsinging epochs (SP; bottom). (F) Mean firing pattern during DS (gray line) and US (black line) singing and during spontaneous activity (SP; dashed line). (G I) There were no significant differences in firing rate (G), percentage of spikes in bursts (H), or CC (I) between DS and US for identified HVC X neurons (gray diamonds) or putative HVC X single units (black diamonds). In this and all subsequent figures, open diamonds indicate unpaired data and * indicates p < trials of undirected singing and found that neither the firing rate (p = 0.69; Figures 2E 2G; Table 1) nor the percentage of spikes in bursts (p = 0.62; Figure 2H; Table 1; see Experimental Procedures) differed between the two contexts. The lack of social modulation of HVC X neurons, which is also apparent in IEG expression (Jarvis et al., 1998), stands in striking contrast to LMAN. Along with their stable firing rates, the HVC X neurons had extremely precise and reliable firing in both social conditions (Figures 2E and 2F). To quantify the reproducibility of the pattern of spikes across trials, we calculated the correlation coefficient (CC) between the instantaneous firing rates across the entire motif for all possible pairs of trials in each social condition (see Experimental Procedures; Kao et al., 2008). This measure was close to the maximum of 1.00 during both directed and undirected singing (Figure 2I; Table 1). Moreover, when we focused specifically on the burst events (see Experimental Procedures), we found that for both directed and undirected singing, spikes occurred on every trial with almost 100% reliability (see Experimental Procedures; Table 1), and the number of spikes per event was highly consistent from trial to trial (cross-trial spike count variance; Table 1). Altogether, these data demonstrate a marked stereotypy of HVC X neurons regardless of context. Thus, striatalprojecting HVC neurons do not appear to contribute to either the variability in LMAN neuron firing or its modulation by social context. Area X Neurons Given the equally precise and reliable firing of HVC neurons in both social contexts, we turned to the striatopallidal nucleus 210 Neuron 82, , April 2, 2014 ª2014 Elsevier Inc.

4 Table 1. Activity of HVC X and Spiny Neurons during Socially Modulated Singing Firing Rate CC Reliability Spikes in Bursts Cross-Trial Spike Count CV HVCX Directed 12.2 ± 1.2 Hz 0.97 ± % ± 0.62% 77.2% ± 3.1% 0.19 ± 0.02 Undirected 11.8 ± 1.1 Hz 0.95 ± ± 0.41% 77.7% ± 2.7% 0.22% ± 0.02% Spontaneous 2.67 ± 0.9 Hz SN Directed 7.3 ± 0.8 Hz a 0.79 ± % ± 4.41% 47.0% ± 7.6% a 0.70 ± 0.03 Undirected 11.3 ± 2.0 Hz 0.74 ± % ± 3.24% 67.4% ± 3.8% 0.58 ± 0.05 Spontaneous 0.19 ± 0.2 Hz Values for the firing rate, CC, reliability, percent of spikes in a burst, and spike count variance for HVC neurons and Area X spiny neurons. Only the firing rate and number of spikes per burst of SNs were significantly modulated by social context. a Significant difference between activity during directed and undirected singing at p < 0.05 Area X to determine whether the increased variability and context dependence of LMAN activity emerge in this basal ganglia nucleus. We focused in particular on the input and output cells (Figure 1A), the putative SNs, which are a major target of input from HVC, and the GPi-like pallidal neurons, which receive input from HVC, SNs, and other Area X neurons, and project strongly to the thalamus (Farries et al., 2005; Goldberg et al., 2010; Leblois et al., 2009; Luo and Perkel, 1999). In this study, we recorded from multiple cell types within Area X. Consistent with prior descriptions (Goldberg and Fee, 2010), we could clearly distinguish between pallidal and striatal neurons based on their spontaneous firing rates (see Figure S1 available online and Experimental Procedures). In addition, using a measure of spike count variance, the Fano factor (F), during undirected singing, we could further subdivide the striatal and pallidal neurons into additional classes, including the two that we focus on here, which are comparable to SNs and to GPi neurons (Goldberg et al., 2010; Goldberg and Fee, 2010; see Experimental Procedures; Supplemental Experimental Procedures; Figure S1). Spiny Neurons in Area X Like SNs in mammals and birds (Farries and Perkel, 2002; Goldberg and Fee, 2010; Hikosaka et al., 1989; Kimura, 1990; Schmitzer-Torbert and Redish, 2008), our putative SNs all had extremely low spontaneous rates (Table 1; n = 8 in 3 birds) and sparse firing events during singing. These firing events, like those of HVC X neurons, consisted of single spikes or short bursts that were tightly locked to song and were precise from trial to trial (average 1.7 events/motif, range 1 4; Figure 3). For example, the SN in Figure 3 had one firing event during syllable g, and rarely fired at other times during song. When we compared the variability of SN spike timing between directed and undirected singing, we found that, like HVC X neurons, SN firing was equally well timed during both conditions. Specifically, neither the average trial-by-trial CC (p = 0.41; Figure 3F; Table 1) nor the temporal jitter of the first spike within a spike event (1.69 ± 0.27 ms versus 1.75 ± 0.26 ms, p = 0.59; see Experimental Procedures) was significantly different between directed and undirected song. Finally, there were no significant differences between directed and undirected song in the reliability of firing across trials (p = 0.14; Table 1). These data indicate that the temporal precision of singing-related SN firing was not significantly affected by the social context in which song was produced. Although the spike timing variability and the reliability of SNs were not modulated by social context, they differed from those of the HVCx inputs. First, the average trial-by-trial CC of SNs was significantly lower than that of HVC X neurons (p < ; Table 1). Second, the number of spikes per event was significantly lower (p < ) and more variable from trial to trial (p < ; Figure 3G) in SNs than in HVC X neurons. Third, firing was less reliable in SNs than in HVC X neurons (p < ; Table 1). Thus, although the song-locked firing of SNs is precisely timed in both social conditions, these neurons spike less and show a slight but significant decrease in their cross-trial consistency and reliability compared to their inputs from HVC. A similar cross-trial variance in spike count was previously observed in SNs of young birds (Goldberg and Fee, 2010). Finally, in contrast to the HVC X neurons, we found that the average firing rates of SNs in Area X were modulated by social context. For the SN depicted in Figure 3, the firing rate during undirected song was double the firing rate during directed song (17.6 Hz versus 8.7 Hz, respectively). Such differences were consistent across SNs, and, on average, the firing rate during undirected song was 62% higher than during directed song (p = 0.05; Figure 3H; Table 1). The increase in the undirected SN firing rate was due primarily to more spikes/burst occurring within the same song-locked events as those during directed song, resulting in a significantly greater percentage of spikes occurring in bursts during undirected than directed singing (p = 0.018; Figure 3I; Table 1). This difference in the average firing rates of SNs across conditions provides the first neurophysiological evidence within the AFP of social-context modulation. However, individual SNs appear to faithfully transmit the timing of the precise song-locked HVC X events, and SN cross-trial firing reliability, although lower than that of HVC X neurons, is similar between social conditions. Thus, the considerable social modulation of spike timing variability in LMAN neurons is not present in the individual striatal neurons that receive a major input from the cortical nucleus HVC. GPi Output Neurons in Area X Because the dramatic trial-by-trial spike timing variability and marked sensitivity to social context of LMAN neurons were not Neuron 82, , April 2, 2014 ª2014 Elsevier Inc. 211

5 Figure 3. Social Context Modulates the Firing Rate, but Not the Precision, of SNs (A) Recordings from SNs (gray circles) in Area X. (B) Amplitude oscillogram of a song (top) and a raw trace of the associated single unit activity. (C) Spectrogram of a representative song motif ( abcdefg ). (D) Raster plots of firing of a single SN during 20 motif renditions of directed (DS; top) and undirected (US; middle) song, and during ten nonsinging epochs (bottom). (E) Mean firing pattern during DS (gray line), US (black line), and spontaneous activity (SP; dashed line). (F) Similar to what was observed for HVC X neurons, SN firing was highly precise and equally stereotyped for both behavioral conditions, as indicated by the high pairwise CCs that were not significantly different between social contexts. (G) HVC X and Area X SNs differed in the variability of their cross-trial firing rates; in particular, the variance (CV) of the number of spikes per trial was significantly higher for SNs than for HVC X neurons regardless of social context. (H) The average SN firing rate was slightly but significantly higher during US than during DS. (I) A larger percentage of spikes were produced in bursts than as single spikes during US compared to DS. See also Figure S1. evident in SNs, we investigated whether these phenomena were present in the putative GPi output neurons in Area X. We recorded the activity of 23 such GPi neurons (in ten birds; Figure 4; Experimental Procedures; Figure S1). In general, adult GPi neuron activity was strongly modulated during singing and showed a consistent pattern associated with the motif. Notably, the song-related activity consisted of a tonic increase in the neurons already high firing rates, punctuated by phasic, song-locked slowing of this firing. For example, the cell illustrated in Figure 4 exhibits consistent decreases in activity during syllables c and g. Moreover, the pattern of activity, when averaged across all trials, was highly similar regardless of the social context in which song was produced (e.g., Figure 4D; Hessler and Doupe, 1999). To quantify the degree of similarity in the firing patterns of individual GPi neurons between contexts, we computed the CC between the average activity pattern during directed and undirected singing. For the neuron depicted in Figure 4, this CC was 0.68, and across all cells the average CC 212 Neuron 82, , April 2, 2014 ª2014 Elsevier Inc.

6 Figure 4. Social Context Modulates Firing of GPi Neurons in Area X (A) Recordings from pallidal projection neurons in Area X (P; gray circle). (B) Spectrogram of a representative song motif ( abcdefgh ). (C) Raster plots of singing-related activity of a GPi neuron during 25 motif renditions each of directed song (DS; top) and undirected song (US; middle), and during spontaneous activity (SP; bottom). (D) Mean firing rate over time during US (black line) and DS (gray line) singing as well as during SP (dashed line). The patterns of activity during DS and US are highly similar despite the difference in firing rate. (E) Average firing rate for DS (gray line), US (black line), and SP (dashed line) prior to the start of the motif (vertical dashed line). Asterisks indicate the first introductory note for DS (gray) and US (black). The firing rate increases earlier prior to the motif and the introductory notes (Figure S2) during US than during DS. (F) Group data showing the highly consistent firing rate increases between SP, DS, and US (all groups are significantly different from each other). (G) Group data showing the time relative to the onset of the motif (time 0) when activity increased to 25%, 50%, and 75% of the peak firing rate. The US firing rate (black) reached each of these benchmarks significantly earlier than the DS firing rate (gray). See also Figure S2. was 0.70 ± Consequently, as is the case for neurons in LMAN, on average GPi neurons in Area X produce a common signal during directed and undirected singing. Despite the similar average firing patterns between the two social contexts, there were striking differences in activity between directed and undirected singing. First, the firing rate was Neuron 82, , April 2, 2014 ª2014 Elsevier Inc. 213

7 Table 2. Activity of GPi Neurons during Socially Modulated Singing Firing Rate CC Reliability GPi spikes (intact) Directed ± 11.0 Hz a 0.49 ± 0.03 a Undirected ± 18.3 Hz 0.25 ± 0.02 Spontaneous ± 7.4 Hz GPi spikes (LMAN lesion) Directed ± 12.2 Hz a 0.53 ± 0.05 a Undirected ± 17.3 Hz 0.28 ± 0.03 Spontaneous ± 7.7 Hz GPi pauses (intact) Directed 0.24 ± 0.03 a 40.1% ± 4.0% a Undirected 0.11 ± % ± 3.3% GPi pauses (LMAN lesion) Directed 0.21 ± 0.03 a 42.3% ± 5.4% a Undirected 0.11 ± % ± 4.1% The firing rate of GPi neurons and the CC and reliability of GPi spikes and pauses were significantly modulated by social context in both intact males and males with lesions of LMAN. a Significant difference between activity during directed and undirected singing at p < significantly affected by social context. For example, the activity of the GPi neuron in Figure 4 increased from a baseline rate of 205 Hz to 292 Hz during directed song and 437 Hz during undirected song. Across all GPi neurons, the firing rates increased significantly during directed singing relative to spontaneous activity but showed even greater increases during undirected singing (Table 2; Figure 4F). The context-dependent difference in firing rate was apparent even in the activity leading up to singing. Although both directed and undirected GPi neuron activity increased from the same spontaneous baseline activity level, those increases in activity occurred significantly earlier prior to undirected singing than prior to directed singing (Figures 4E and 4G; Figure S2; Supplemental Experimental Procedures). In addition to the changes in firing rate, there was a marked increase in the trial-by-trial spike timing variability of GPi neurons over neurons earlier in the circuit, and this variability was socially modulated, with undirected singing showing a more variable pattern of activity across trials. In particular, across all cells, the average CC of spiking between all pairs of trials was significantly lower during undirected singing (0.25) than during directed singing (0.49; Table 2, p < ; Figure 5B). The trial-by-trial spike timing variability and its change with context were particularly apparent in the pattern of breaks or pauses in firing, which differed markedly between directed and undirected singing. To analyze context-related differences in pauses, we defined a pause as any interspike interval (ISI) > 2 SDs above the mean directed ISI, and a pause onset as the start time of each ISI above the threshold (see Experimental Procedures, Supplemental Experimental Procedures, and Figure S3 for discussion of pause definition). As was the case for spikes, the pattern of pauses was less consistent from trial to trial during undirected singing. To quantify the reproducibility of the pattern of pauses across the entire motif, we calculated the average CC specifically for the pause onsets as defined above (see Experimental Procedures; Figure S3). For the cell in Figure 4, the CC of pause onsets was low during directed singing (0.24), but decreased even further during undirected singing (0.05). Across all GPi neurons, the average CC of pause onsets was significantly lower during undirected than directed singing (Table 2; p = ; Figure 5C). The difference in the trial-by-trial CC of pauses between directed and undirected song resulted not only from differences in pause timing but also from context-dependent differences in the reliability of pauses across trials. In general, the consistent pattern of decreases in activity associated with the song (e.g., Figure 4D) was evident as increases or peaks in the average pattern of pause onsets (Figure S3D). We called such increases in the average pause rate pause events, and investigated the reliability of pause onsets across trials within these singingrelated pause events (see Experimental Procedures). We found that pause events were significantly less reliable from trial to trial during undirected singing (Table 2; p < ; Figure 5D). Thus, salient song-related pause events were not only variable in timing but also occurred less reliably across trials during undirected than directed singing. Previous work has demonstrated that pauses in GPi neuron activity allow DLM neurons to fire single spikes or bursts, and that the probability of spiking in DLM is influenced by changes in the firing rate of GPi input neurons, with greater decreases in GPi neuron activity leading to more DLM spikes (Goldberg and Fee, 2012; Goldberg et al., 2012; Kojima and Doupe, 2009; Person and Perkel, 2005). To examine context-dependent differences in the firing rate preceding pauses, we used a pausetriggered average (Figure 5E), and found that the mean firing rate of GPi neurons immediately preceding a pause was significantly higher during undirected than directed singing (379 ± 7 versus 248 ± 5 spikes/s; p < ; Figure 5F). Spiking in DLM is also influenced by the duration of decreases in firing rate ( decelerations ), with longer decelerations leading to more DLM spikes (Goldberg and Fee, 2012; Goldberg et al., 2012; Kojima and Doupe, 2009; Person and Perkel, 2005). Thus, we also looked at whether there was a context-dependent difference in how long the firing rate remained low following the onset of deceleration. To investigate this, we smoothed each trial and measured the duration of epochs when the firing rate dropped at least 2 SD below the mean rate (see Experimental Procedures; Figure 5G). We found that these periods of decreased firing were significantly longer during undirected song (14.9 ± 0.9 ms) than during directed song (9.5 ± 1.0 ms; Figure 5H; p < ). Together, these data showing both greater and longer slowing of GPi neuron firing rates during undirected singing, suggest that undirected GPi activity could allow more thalamic spikes, including bursts, than during directed singing. Moreover, in combination with the marked spike timing variability that we observed during undirected singing, the contextual differences in pallidal firing could explain context-dependent differences in both the frequency and variability of bursts in LMAN. 214 Neuron 82, , April 2, 2014 ª2014 Elsevier Inc.

8 Figure 5. Social Context Modulates Pauses in GPi Neuron Firing (A and B) For the population of GPi cells (A), the variability of the spike trains was consistently higher during undirected singing (US) than during directed singing (DS), as illustrated by the significantly lower CC for US (B). (C) Similarly, pause onset times across the motif were also more variable during US, as evidenced by the lower CC. (D) Within pause events, the reliability of pauses across trials was significantly lower during US. (E and F) The firing rate immediately preceding a pause was significantly higher during US than DS. (E) Depicted is the mean spike rate preceding a pause for a single neuron during US (black) and DS (gray). (F) For all but one cell, the activity immediately preceding a pause was significantly greater during US. (G) To measure the duration of decelerations in the firing rate we smoothed each spike train (black curve; see Experimental Procedures) and then thresholded the smoothed spike trains at 2 SD below the mean (gray horizontal line). The durations of these decelerations were then calculated as the difference between onsets and offsets determined based on threshold crossings of the smoothed curve (gray vertical lines). (H) Group data showing that the durations of decelerations were significantly longer during US than during DS. See also Figure S3. Sources of Variability Activity of GPi Neurons following LMAN Lesions The considerable increase in spike and pause timing variability of GPi neurons specifically during undirected singing, compared to SNs and HVC neurons earlier in the circuit, suggests that such variability and its context dependence arise within Area X. Alternatively, because GPi neurons and SNs receive inputs not only from HVC but also recurrently from LMAN, they could arise first in LMAN and be fed back to Area X. We therefore lesioned LMAN and assessed whether such lesions abolished the variability or context dependence of Area X GPi neurons activity (Figure 6; n = 15 GPi neurons from 3 birds with lesions of over 75% of LMAN; see Experimental Procedures). Firing-rate modulations of GPi neurons in LMAN-lesioned birds were not different from those of control birds. These cells had high spontaneous firing rates and significantly increased their firing rates during both directed and undirected singing (Table 2; p = ). As was the case in unlesioned birds, the firing rate was significantly higher during undirected than directed singing (p = ; Figure 6E). Moreover, there were no significant differences in the spontaneous, directed, or undirected firing rates between lesioned and unlesioned birds (p = 0.44; Figure 6E; Table 2). In addition to showing normal social modulation of the firing rate, the GPi neurons in LMAN-lesioned males continued to exhibit significant spike timing variability, and social modulation of this variability. There were no significant effects of the LMAN lesions on any of our measures of variability during undirected singing (for CC of spikes, p = 0.27 between lesions and controls; for CC of pauses, p = 0.93; for reliability of pauses, p = 0.66). Moreover, activity was significantly more variable during undirected than directed singing, as evidenced by a lower trial-by-trial CC of spikes during undirected than directed singing (p = ; Figure 6F; Table 2), and pauses were more variable (p = ; Figure 6G; Table 2) and less reliable across trials (p = ; Table 2) during undirected than directed singing. These data indicate that LMAN inputs into Area X are not necessary for the variability of adult GPi neuron activity or its social context dependence, and suggest that neural variability and its social modulation emerge within Area X. Neuron 82, , April 2, 2014 ª2014 Elsevier Inc. 215

9 Figure 6. LMAN Lesions Do Not Eliminate Context-Dependent Differences in Variability (A) Recordings from GPi neurons (P; gray circle) in LMAN lesioned males. (B) Spectrogram of a representative song motif. (C) Raster plots of firing of a single GPi neuron during 25 motif renditions of directed (DS; top) and undirected song (US; middle), and during ten nonsinging epochs (bottom). (D) Mean firing pattern during singing (DS in gray, US in black) and spontaneous activity (SP; dashed line). (E G) As was the case for GPi neurons in unlesioned birds (gray dots and line in E and F are the means for unlesioned birds), the firing rate of GPi neurons in birds with LMAN lesions was significantly higher during US than during DS. In addition, the variability was significantly higher during US than during DS, indicated by the lower cross-correlation of the spike trains (F) and the pauses (G). Context-Dependent SN Correlations as a Source of GPi Neuron Variability: Hypothesis and Computational Model Our data indicate that the spiking of SNs is precisely timed in both social contexts, with only a social modulation of the average firing rate. However, the output cells of this circuit, the GPi neurons, exhibit a marked social modulation not only of firing rate but also of spike timing variability, particularly in the pattern of pauses presumed to reflect SN inputs. How might such a transformation occur? Our results, along with existing data on the striatum in both mammals and songbirds, suggest one simple hypothesis for how the spike timing variability of GPi neurons and its social regulation might arise within the Area X network. 216 Neuron 82, , April 2, 2014 ª2014 Elsevier Inc.

10 Figure 7. Singing-Related Activity of a Population of SNs and GPi Neurons (A H) Sample plots of song-aligned activity for a randomly selected subset of trials during directed (DS; A D) and undirected (US; E H) singing, for four SNs and two GPi neurons recorded in a single male. (A and E) Representative spectrograms of DS (A) and US (E). (B and F) Raster plots of four SNs ordered based on the timing of their activity relative to the motif. Different neurons are indicated by different colors and separated by black lines. (C and G) Raster plots of two GPi neurons indicated by different colors and separated by black lines. (D and H) Average pause rate for the second (purple) GPi neuron, generated from 19 DS and 21 US trials. Note the larger number of singing-related events, distributed across the song, compared with SNs. This hypothesis builds on several observations. For one, GPi neurons in mammals are a site of major convergence of SN inputs (Bergman et al., 1998; Percheron et al., 1984). Our neurophysiological recordings are consistent with this: the SNs we recorded had sparse firing events during each motif (range 1 4 events; e.g., Figures 7B and 7F), whereas, in contrast, GPi neurons exhibited multiple pauses that were often distributed across the entire motif (range 1 14, average 4.85 pauses per motif; Figures 7C, 7D, 7G, and 7H). Thus, if SNs provide a major source of the inhibition generating pauses in GPi neurons, there must be a convergence of multiple SNs onto a single GPi neuron (Figure 7 illustrates this possibility with recordings of SNs and GPi cells from the same bird). Second, our own observations, as well as those made in juvenile birds (Goldberg and Fee, 2010; Figure 3G), suggest that there is a greater cross-trial variance of spike count in adult SNs than in their inputs from HVC, and that this does not differ between social contexts. Finally, existing data suggest that dopamine levels in Area X are lower during undirected than directed singing (Sasaki et al., 2006). In mammalian basal ganglia, low-dopamine states are associated with increased cross-neuronal correlation and highly synchronized bursting in ensembles of striatal neurons (Bergman et al., 1998; Costa et al., 2006; Goldberg et al., 2004). This raises the possibility that such an increase in correlation in lower-dopamine states might occur in songbird basal ganglia as well. These observations suggest a simple model in which (1) the degree of correlation in firing rate across SNs can greatly influence the trial-by-trial variability of their downstream target neurons, and (2) this SN rate correlation varies between social contexts. This model is illustrated intuitively in Figure 8A. This figure shows three SNs firing during one event locked to song, and the resulting inhibition of the GPi neuron onto which they Neuron 82, , April 2, 2014 ª2014 Elsevier Inc. 217

11 Figure 8. Increasing the Rate Correlation in SNs Increases Pause Variability in GPi Neurons (A) Illustration of how altering the degree of rate correlation among SNs could influence the output of GPi neurons. The top panel depicts the responses of three SNs (SNs 1 3) and one GPi neuron to three different motif renditions (trials 1 3) when the rates of the SNs are uncorrelated. The SN responses are depicted as raster plots and the GPi neuron response is the inhibitory potential created by the SN inputs, with greater deflection from zero indicating a greater response. The bottom panel depicts the responses of these same neurons when the firing rates are correlated. Note that the GPi responses are predicted to be of similar size in the uncorrelated case, whereas the GPi responses in the correlated condition vary depending on the quantity of spikes in the SN burst. (B) Example of the output of the model. Top panel is a generic spectrogram. The activity of a hypothetical GPi neuron in response to uncorrelated (middle panel) and correlated (bottom panel) SN inputs is depicted as raster plots. (C and D) Like the data from GPi neurons, the group data for 50 runs of the model show that the CC of spikes (C) and pauses (D) was higher for the uncorrelated, directed-like repetitions than for the correlated, undirected-like repetitions. See also Figure S4. converge, in two rate correlation conditions. In the upper panel of Figure 8A, individual SNs are uncorrelated in the number of spikes they fire on each trial, so that the downstream neuron will effectively receive the average of all the input neurons variable firing rates on each trial, resulting in equal inhibition of intermediate intensity across trials. However, if changes in the SN network occur so that SNs covary in the number of spikes they produce on each trial (Figure 8A, lower panel), the summed effect of these inputs on the downstream neuron s firing will differ across trials. Thus, covariation in the SN rate will propagate the individual cross-trial spike count variance forward and result in downstream pauses that are highly variable in magnitude across trials. According to this model, undirected singing and the associated decrease in dopamine levels should result in greater cross-sn rate correlations, just as suggested in mammalian basal ganglia (Bergman et al., 1998; Costa et al., 2006), and this will create greater trial-by-trial pause variability in GPi neurons. To investigate more quantitatively whether altering the degree of rate correlation in a population of SNs like ours could result in differences in the variability and reliability of pauses in GPi neurons, we constructed a simple computational model (see Experimental Procedures; Figure 8) incorporating (1) the observed cross-trial variance in SN firing rate, (2) the observed directed-undirected differences in the number of spikes per burst of SNs, and (3) postulated differences in correlations between neurons, assuming higher correlations among SNs during the lower-dopamine, undirected state. We found that just by manipulating the degree of rate correlation between SNs, it was possible to generate differences in the reliability of pauses across trials of a high-firing neuron, thereby affecting the neuron s overall variability in a manner similar to what we observed in GPi neurons in Area X (Figure 8B). For example, across 50 simulations, each involving 50 trials and 1,000 SNs synapsing onto a single GPi neuron, the average CCs of firing rate and pause onsets were both significantly higher and less variable when SN rates were uncorrelated than when they were correlated (Figures 8C and 8D). Although the simplicity of our model creates spike trains with greater regularity than we observed, it nonetheless indicates that substantial differences in GPi neuron pause variability can arise from the very small differences in trial-by-trial firing rates seen in SNs, via the plausible mechanism of changing the SN covariation in firing rate in a state-dependent manner. 218 Neuron 82, , April 2, 2014 ª2014 Elsevier Inc.

12 DISCUSSION Neural variability and its context sensitivity have been well described in LMAN, the output of the songbird cortical-basal ganglia circuit (Hessler and Doupe, 1999; Kao et al., 2008; Murugan et al., 2013). Here, by systematically recording neurons at earlier stages in the circuit, we traced the emergence of trialby-trial variability and its social modulation. We found that individual HVC neurons, the corticostriatal inputs to the circuit, were highly stereotyped in all respects in both social conditions, underscoring the generation of variability downstream of HVC. SNs, a major target of HVC X inputs in the striatopallidum, also were not differentially affected in their spike timing and crosstrial reliability by social context. The first neurophysiological sign within the circuit of social modulation appeared in the average firing rates of SNs, which were lower during directed singing than during undirected singing. However, by the output stage of the striatopallidal circuit, in the GPi neurons, a marked trial-by-trial variability in spike timing and its dependence on social context were both apparent. Variability in the timing of pauses of GPi neurons and their modulation by social context were maintained even after lesions of LMAN, indicating that these properties were not simply fed back into the basal ganglia from LMAN. Taken together, our findings show that variable singing-related firing patterns and their social modulation first emerge over multiple steps within the songbird basal ganglia, and highlight several critical loci and potential mechanisms in the generation and regulation of variability. More recent data reveal that lesions of Area X reduce song plasticity (Ali et al., 2013; Kojima et al., 2013) but only temporarily reduce behaviorally measured song variability (Goldberg and Fee, 2011; Ali et al., 2013). This would seem to support a model proposing LMAN as the initial source of neural and behavioral variability (Fee and Goldberg, 2011). However, our data here, which have the advantage of measuring pallidal neural activity directly rather than only behavior, reveal the striking persistence of neural rate and timing variability in GPi neurons after lesions of LMAN, and thus indicate that basal ganglia variability is not strictly dependent on the presence of LMAN. One of several possible explanations for the residual song variability after Area X lesions is that such lesions do not silence LMAN but create abnormal firing. Specifically, after Area X lesions, LMAN activity still increases during singing, but there is a marked loss of patterned neuronal bursting (Kojima et al., 2013) and associated egr-1 expression (Kubikova et al., 2007). Although such activity appears insufficient for allowing plastic changes to song (Ali et al., 2013; Kojima et al., 2013), the presence of elevated and unpatterned activity in LMAN could provide an alternate source of variability that could lead to the residual variability in song. In birds and mammals, SNs, and more specifically cortical-sn synapses, are well-described loci of neuromodulatory control (Ding and Perkel, 2002; Surmeier et al., 2007). Overall, such synapses are thought to be weak (Wilson, 1995), which could explain the greater trial-by-trial spike count variance in SNs compared to HVC neurons. Moreover, although HVC X neurons had similar firing rates and bursting in both social settings, SNs in Area X showed less bursting during directed than undirected singing. Such a diminished occurrence of SN bursting during directed singing is consistent with the dopaminergic gating of striatal excitability seen both in mammalian striatum (Nicola et al., 2000; Surmeier et al., 2007) and in songbirds (Ding and Perkel, 2002; Leblois et al., 2010). In songbirds, more dopamine is released in Area X during directed than undirected singing (Sasaki et al., 2006), and the decreased bursting of SNs during directed song could result from a dopamine-dependent damping of the influence of HVC X inputs on SNs. Additionally, such a reduction in bursting is also consistent with studies of IEG expression, which have raised the possibility that norepinephrine also participates in the social modulation of gene expression (Castelino and Ball, 2005; Hara et al., 2007; Jarvis et al., 1998). Despite the social modulation of SN rate, the precision and reliability of SN firing were unchanged between directed and undirected singing. SN inputs are thought to give rise to pauses in the firing of GPi neurons downstream (Chevalier and Deniau, 1990; Farries et al., 2005; Luo and Perkel, 1999). However, the highly variable timing and reliability of pauses in GPi neurons during undirected singing indicates that their context-dependent variability is not simply inherited from individual SNs. One possible mechanism suggested by these results is that GPi variability arises from context-dependent differences in the network properties of Area X. In particular, in mammals, decreases in dopaminergic tone are associated with increases in synchronized bursting and firing-rate covariation in the cortex and basal ganglia (Goldberg et al., 2004; Costa et al., 2006). Using a simple computational model incorporating our observations about individual SNs, we compared the effects of altering the degree of rate correlation among multiple SN inputs to the same GPi neuron. We found that increasing the degree of SN rate correlation (as might occur during a low-dopamine state) resulted in significant decreases in the reliability of pauses across trials in this downstream neuron, in a manner similar to what we observed in actual GPi neurons in Area X during undirected singing (Figure 8). These data suggest that if there are contextdependent differences in network correlations, even the small cross-trial changes in firing rates that we observed in SNs could lead to large context-dependent differences in the trial-by-trial variability of individual downstream pallidal neurons. Although our model is not the only possible explanation for how changes in variability could be generated within Area X (for example, the role of other interneurons in this circuit remains unclear; Gittis et al., 2011), it highlights the need for future studies to investigate whether social context and dopamine release modulate the degree of rate correlation among SNs, and how such modulations influence the precision of song. In addition to modulating the trial-by-trial variability of GPi neuron pauses, social context also influenced the average singing-related firing rate of GPi neurons. There are several possible mechanisms for the larger increase in GPi firing rate during undirected versus directed song. Other than SNs, known inputs to GPi neurons include the glutamatergic HVC X neurons (Farries et al., 2005; Figure 1A), which are thought to provide the functional equivalent of a hyperdirect pathway. Such excitatory HVC inputs have been shown to increase pallidal neuron firing in vitro and in anesthetized birds, and this increase is damped by dopamine (Leblois et al., 2009, 2010). Thus, increased dopamine during directed singing could decrease Neuron 82, , April 2, 2014 ª2014 Elsevier Inc. 219

13 the effectiveness of synaptic inputs to pallidal neurons from HVC neurons. Dopamine can also directly decrease pallidal neuron firing rates (Leblois et al., 2010) and could have additional effects on SN synapses and other inputs to GPi neurons. Thus, the contextual differences in Area X GPi neuron firing rate may have multiple, independent sources. Regardless of the sources of Area X pallidal neuron firing and its modulation by context, differences in the magnitude and duration of pallidal decelerations between directed and undirected singing are particularly well suited to drive strong context-dependent differences at pallidothalamic synapses. Neurons in the thalamic nucleus DLM are tonically inhibited by GPi projections through GABAergic calyceal synapses (Luo and Perkel, 1999), and fire single spikes or bursts during pauses or decelerations in GPi activity. Moreover, greater and longer decelerations of GPi neuron activity lead to more DLM spikes (Person and Perkel, 2005; Kojima and Doupe, 2009; Goldberg and Fee, 2012). Our findings of social context-dependent differences in GPi firing rates preceding and during pauses predict that DLM neurons should exhibit marked social differences in firing. Indeed, the socially modulated switch from isolated spike to bursting modes of firing in LMAN could first emerge in the thalamus, which is well known to switch between bursting and nonbursting states across other contexts, such as sleep and wake (Fanselow et al., 2001; McCormick and Bal, 1997). Reinforcement learning is critically dependent on the basal ganglia and behavioral variability (Bar-Gad and Bergman, 2001; Doya, 2002; Doya and Sejnowski, 1998; Graybiel et al., 1994; Sutton and Barto, 1998). Studies in songbirds have suggested that basal ganglia circuits contribute to reinforcement learning by providing a source of behavioral variability for trialand-error learning (Andalman and Fee, 2009; Charlesworth et al., 2011; Hampton et al., 2009; Kao et al., 2005, 2008; Kao and Brainard, 2006; Ölveczky et al., 2005, 2011; Tumer and Brainard, 2007; Warren et al., 2011). By systematically recording at multiple stages in the basal ganglia circuit, we demonstrate that neural variability is introduced and amplified as activity progresses through the striatopallidal network, and acquires its context-dependence there as well. Our findings indicate that the striatopallidal circuit is integral for generating and modulating variable firing patterns downstream that normally promote behavioral variation and plasticity. EXPERIMENTAL PROCEDURES Animals Adult (>100 days old, n = 22) male zebra finches (Taeniopygia guttata) from our colony were used. A few days prior to surgery, males were isolated and housed in a small cage inside a sound-attenuation chamber (soundbox; Acoustic Systems) where they received seed, water, and grit ad libitum. The procedures adhered to UCSF IACUC-approved protocols and NIH guidelines for the care and use of animals. Surgery Surgery for the implantation of a lightweight microdrive carrying electrodes was performed as previously described in Kao et al. (2008). Two or three tungsten electrodes with impedances of 3 5 MOhm (Microprobe) were stereotaxically targeted to either Area X or HVC. For HVC recordings, two birds had custom-made bipolar stimulating electrodes implanted into Area X that were used for antidromic identification of HVC neurons projecting to Area X. For LMAN lesions, a unilateral electrolytic lesion of LMAN was made using a low-impedance electrode (Kao and Brainard, 2006; Hampton et al., 2009). A microdrive carrying recording electrodes was then implanted ipsilateral to the lesion into Area X during the same surgery (n = 2). For a third bird, a bilateral lesion of LMAN was made, and electrodes were implanted into Area X following recovery. The effectiveness of all lesions was histologically verified (see below). Physiological Recordings Sound, spontaneous activity (both alone and in the presence of a female) and singing-related activity were recorded using custom-written LabVIEW software (A. Leonardo [California Institute of Technology] and C. Roddey [University of California, San Francisco]). Neural signals were digitized, amplified (31,000), filtered ( khz), and recorded. Sounds were digitized, bandpass filtered ( khz; Krohn-Hite), and recorded. Behavior was monitored through a video camera inside the sound box. Activity during directed singing was recorded when males performed courtship song for females (see Kao et al., 2008; Woolley and Doupe, 2008). Upon completion of individual experiments, small electrolytic lesions were made at different depths along the recording track. All recorded sites were verified to lie within Area X or HVC based on their position relative to marker lesions on Nissl-stained sections. For birds with LMAN lesions, we also performed immunohistochemistry for calcitonin gene-related peptide (CGRP), which reliably labels the boundaries of LMAN, and quantified the volume of LMAN using ImageJ (W.S. Rasband, ImageJ, NIH; nih.gov/ij/). The average lesion size across all birds was 84% (range 76% 88%; Supplemental Experimental Procedures). Analysis Spike Sorting and Cell-Type Classification Neural activity was analyzed offline using software written in MATLAB (MathWorks). Units in HVC (ten antidromically identified and 18 putatively identified single HVC X units; Table 1) were sorted offline using custom-written MATLAB programs and Klustakwik (Ken Harris). For HVC units, we assumed a premotor latency of 40 ms. To identify putative HVC X neurons, we used the spontaneous firing rate as well as the number of song-locked firing events. Putative HVCx neurons had sparse song-locked firing events (one to five per motif), distinguishing them from interneurons that fire throughout song (Kozhevnikov and Fee, 2007; Prather et al., 2009; Sakata and Brainard, 2008), and spontaneous firing rates that were higher than the extremely low rates of HVC neurons projecting to RA ([HVC RA ]; Table 1; Hahnloser et al., 2002; Prather et al., 2009). For Area X recordings, large single units were discriminated using a Bayesian spike-sorting program (Kao et al., 2008). Isolation of single units was based on waveform shape and the presence of a refractory period in the ISI histogram; examples of typical waveforms and ISI distributions are shown in Figure S2. For all neurons, <1% of the ISIs were shorter than 1.0 ms. Using a Ward s hierarchical cluster analysis of spontaneous firing rates (JMP v.8.0.1; SAS Institute), we found that the Area X cells we recorded fell into three discrete groups: one with almost no spontaneous activity (mean Hz, range Hz; putative SNs), one with medium spontaneous firing rates (mean 11.3 Hz, range 5 18 Hz; putative interneurons), and one with high spontaneous firing rates (mean Hz, range Hz; putative pallidal neurons). The F value during undirected activity further distinguished these groups (Supplemental Experimental Procedures; Figure S1). For example, cells with medium firing rates (interneurons) had highly variable firing during undirected singing, indicated by a higher F value, further distinguishing them from the regular and precise singing-related firing of cells with very low spontaneous firing (SNs). The F value also subdivided the neurons with high firing rates into a more variable (higher F) population with long, irregular singing-related bursts similar to avian and mammalian GPe neurons (Elias et al., 2007; Goldberg et al., 2010), and a second, less variable population that displayed greater increases firing during singing and very brief singingrelated pauses similar to avian and mammalian GPi neurons (Goldberg et al., 2010; Goldberg and Fee, 2012; see Supplemental Experimental Procedures and Figure S1 for more details). 220 Neuron 82, , April 2, 2014 ª2014 Elsevier Inc.