Lesions of the Basolateral Amygdala Disrupt Selective Aspects of Reinforcer Representation in Rats

Transcription

1 The Journal of Neuroscience, November 15, 2001, 21(22): Lesions of the Basolateral Amygdala Disrupt Selective Aspects of Reinforcer Representation in Rats Pam Blundell, 1 Geoffrey Hall, 1 and Simon Killcross 2 1 Department of Psychology, University of York, Heslington, York, YO10 5DD United Kingdom, and 2 School of Psychology, Cardiff University, Cardiff CF10 3YG, United Kingdom The amygdala is known to play a role in learning about motivationally significant events. We investigated this role further by examining the effects of excitotoxic lesions of the basolateral amygdala on the ability of rats to use instrumental outcomes to direct responding (the differential outcomes effect) and on the ability of Pavlovian cues to modulate instrumental performance based on shared outcomes (reinforcer selective Pavlovian-toinstrumental transfer). We found that basolateral amygdala (BLA) lesions did not affect the ability of rats to learn a basic instrumental conditional discrimination, but did disrupt the ability of differential outcomes to facilitate acquisition. In Pavlovianto-instrumental transfer, BLA lesions did not disrupt the basic enhancement of instrumental performance but did abolish the reinforcer specificity of that enhancement. These results suggest that the BLA is involved in the representation of the sensory aspects of motivationally significant events. Key words: appetitive conditioning; basolateral amygdala; reward; reinforcement; Pavlovian; instrumental Many studies have demonstrated that lesions of the basolateral amygdala (BLA) impair a range of forms of learning about motivationally significant events. In fear conditioning, BLA lesions produce deficits in freezing to an aversive context (Phillips and LeDoux, 1992), freezing to a specific cue that has been paired with a shock (LeDoux, 2000), conditioned punishment (Killcross et al., 1997), and fear-potentiated startle (Davis, 1992, 2000). However, despite the well defined role of the amygdala in aversive learning, its role in appetitive tasks is much less clear. BLA lesions have no effect on many simple appetitive Pavlovian conditioning tasks, including autoshaping (Willoughby and Killcross, 1998; Parkinson et al., 2000), conditioned orienting (Holland, 1997), and conditioned magazine approach (Hatfield et al., 1996; Willoughby and Killcross, 2000). Similarly BLA lesions do not influence simple instrumental conditioning (B.W. Balleine, S. Killcross, and A. Dickinson, unpublished observations) or the nonspecific modulatory influence of Pavlovian stimuli on instrumental performance (Everitt et al., 2000). However, in some circumstances BLA lesions do influence aspects of appetitive learning. Specifically, Hatfield et al. (1996) report a deficit in appetitive Pavlovian second-order conditioning. Although firstorder conditioning proceeded normally, the first-order conditional stimulus (CS 1 ) failed to act as a reinforcer when it is subsequently paired with a second stimulus (CS 2 ). Parallel deficits have been reported in conditioned reinforcement (Cador et al., 1989; Burns et al., 1993). Here, BLA-lesioned and shamlesioned rats received first-order appetitive Pavlovian conditioning and then were allowed to make instrumental responses that were reinforced by presentation of the first-order CS. Sham-, but not BLA-lesioned rats came to respond more on a lever that resulted in the presentation of this CS than on a control lever. Received June 29, 2001; revised Aug. 28, 2001; accepted Aug. 30, This work was supported by a United Kingdom Medical Research Council (MRC) Career Establishment Grant to S.K. and by an MRC studentship to P.B. Correspondence should be addressed to Dr. Simon Killcross, School of Psychology, P.O. Box 901, Park Place, Cardiff University, Cardiff, CF10 3YG UK. KillcrossAS@cardiff.ac.uk. Copyright 2001 Society for Neuroscience /01/ $15.00/0 Although BLA lesions do not affect acquisition of appetitive first-order Pavlovian and instrumental conditioned responses, when BLA-lesioned rats are required to use a CS reward association in higher-order learning tasks such as second-order conditioning and conditioned reinforcement, this association is less effective in modifying behavior. Hatfield et al. (1996) provided further insight into this deficit. They showed that rats with BLA lesions do not respond in the same way as do normal rats after post-training devaluation of the reward by pairing with LiCl. Control animals showed both reduced magazine approach during the CS in an extinction test and reduced consumption of pellets. Although BLA-lesioned animals showed a reduced intake of pellets, demonstrating that they had acquired the food illness association, they continued responding to the CS during the test. One obvious interpretation of these effects is that responding in BLA-lesioned animals is not sensitive to the current value of reward representations. However, it is clear by their unimpaired performance in first-order conditioning that BLA-lesioned rats are sensitive to some aspects of reward presentation. The following experiments examine appetitive conditioning after BLA lesions, using the differential outcomes effect and reward-specific Pavlovian-to-instrumental transfer to assess the nature of reward representations in lesioned animals. Here, we assess animals ability to represent the precise sensory nature of rewards by examining their ability to respond differentially in the presence of cues for different reward types. MATERIALS AND METHODS Experiment 1: BLA lesions and the differential outcomes effect Subjects We used 32, naive, male hooded Lister rats. At the beginning of the experiment their mean ad libitum weight was 372 gm (range, gm). The rats were housed in pairs in a climate-controlled vivarium (lights on 8:00 A.M. to 8:00 P.M.). Subjects were tested during the light portion of the cycle. Before the start of training the rats were reduced to 80% of their ad libitum postsurgical recovery weights. After completion

2 Blundell et al. BLA Lesions and Reinforcer Representations J. Neurosci., November 15, 2001, 21(22): of behavioral testing, the rats were killed, and their brains were removed for histological analysis. Surgery Under anesthesia 16 rats received bilateral lesions of the BLA produced by injection of quinolinic acid (0.09 M); the remainder received a control surgical procedure. Anesthesia was induced with 4% halothane, delivered in O 2 and N 2 O gas ( 1.0 l/min of each) in an induction chamber (IMS Ltd., Congleton, UK). When a rat was fully anesthetized, it was placed in a stereotaxic frame (Stoelting Inc., Kiel, WI) with its nose in a face mask (IMS Ltd.). Anesthesia was maintained with delivery of 1.5% halothane in O 2 and N 2 O gas ( 0.7 l/min of each), through the face mask. The depth of anesthesia was monitored by assessing the pedal withdrawal reflex and responsivity to a mild tail pinch. An incision was made along the skull, and then skin and fascia were cleared to reveal bregma. A drill mounted on the stereotaxic frame was used to make burr holes above the injection sites. Injections (two on each side) were made with a 30 gauge needle attached by polythene tubing to a 1 l syringe, which was controlled by an infusion pump (Harvard Apparatus, Holliston, MA). We made 0.25 l injections at the following coordinates: lateral 4.6 mm, anteroposterior 2.3 mm, 3 mm, and ventral (from dura) 7.3 mm. Each injection was made over 2.5 min, and the injection needle was left in place for an additional 2.5 min to allow the neurotoxin to diffuse. The skin was then closed with suture. Animals were then given an injection of saline if they were dehydrated. Animals were observed during recovery from anesthesia, and they were returned to the vivarium after starting to eat and drink. Animals were allowed to recover for at least 1 week with ad libitum access to food and water. The control surgical procedure was identical to that which was used to produce lesions, the only difference being that no quinolinic acid was injected. Histology After the completion of behavioral testing, animals were anesthetized with a lethal overdose of sodium pentobarbitol (Sagital; 2 ml/animal, i.p.) and perfused via the ascending aorta with 0.1 M PBS, ph 7.4, followed by 4% paraformaldehyde. The brains were then removed and post-fixed in 4% paraformaldehyde solution before being transferred into 20% sucrose solution. After 24 hr the brains were frozen on a freezing microtome fast-freeze plate, before coronal sections (40 m) were cut on a freezing microtome throughout the full extent of the lesioned area. Every fourth section was taken and mounted on a gelatin-coated glass slide and then stained for analysis using cresyl violet. Slides were coverslipped, dried, and then examined under microscope to assess the extent and nature of excitotoxin-induced neuronal damage. Areas of neuronal loss were mapped onto standardized sections of the rat brain (Swanson, 1998). Apparatus Behavioral procedures were performed in eight, identical, standard operant chambers (MedAssociates, St. Albans, VT), housed in soundattenuating boxes. The dimensions of the chambers were 30.5-cm-wide 24.1-cm-high 21.0-cm-deep. The ceiling and the two longest sides where made from clear polycarbonate, and the front and back of the chambers were of stainless steel. The floor was a standard grid floor, with 19 stainless steel rods, each with a diameter of 4.8 mm, arranged with centers 1.6 cm apart. The chambers were fitted with two retractable levers, at identical heights at the left-hand and right-hand side of the front wall. Each lever could be inserted 19 mm into the chamber. The maximum travel of the levers was 6 mm. Above each lever was a 100 ma stimulus light, 25 mm across. Between the two levers there was a troughtype dual pellet/dipper receptacle, into which either pellets or liquid could be delivered. Nose pokes into this receptacle were measured by an infrared detector. A heavy-duty relay was fitted to the outside of the chamber, on the wall opposite the levers. This could be switched on and off to create a 10 Hz train of clicks. A speaker was also mounted on this wall and was connected to a tone generator was that could deliver tones of various frequencies. A 28 V, 100 ma house light was mounted at the top-center of this wall, with an open hood that pointed to the ceiling of the box. For all experiments, the sound level of the tone was measured and matched to that of the click-train, at 80 db. All stimuli and responses were controlled and measured by Med-PC software (MedAssociates) running on an IBM-compatible computer. Table 1. Contingencies in experiment 1 examining the differential outcomes effect S1 S2 Consistent R1 3 O1 R1 3 A R2 3 A R2 3 O2 Inconsistent R1 3 O1/O2 R1 3 A R2 3 A R2 3 O1/O2 R1 and R2 are left and right lever presses, O1 and O2 are sucrose and food pellet reinforcers, and S1 and S2 are tone and click-train. A indicates no reward (see Materials and Methods; experiment 1). Procedure Training. After recovery from surgery, each animal was assigned to one of the eight operant chambers, and thereafter was always trained in that chamber. At the start of each session, the house light came on and remained on throughout the session. The house light went out at the end of each session. The reinforcers used were 45 gm Noyes food pellets and 0.5 ml 20% w/v sucrose solution. Pilot studies indicated that in normal rats these two rewards produce very similar levels of performance of conditioned behaviors such as magazine approach and lever pressing and are well matched for their motivational value. Training consisted of three stages: magazine training, lever press training, and finally training on a continuous performance conditional discrimination task. There were then two test sessions, one in extinction and one in which the tone and clicker were not presented. One session was run on each day. Magazine training. All of the rats were trained to collect food rewards during two, 30 min magazine training sessions. Half the animals (equal numbers of sham- and BLA-lesioned animals) were trained to collect food pellets in the first training session, and half were trained to collect sucrose solution. The next day rats were trained to collect the alternative reward. The rewards were delivered on a random time (RT) 60 sec schedule. Lever training. The rats were initially trained to respond on the levers during two sessions with a continuous schedule of reinforcement (i.e., every lever press was rewarded). One lever was inserted into the chamber at the beginning of the session and retracted at the end of the session, and both sucrose solution and food pellets were used with equal probability as rewards. The rats were given identical training on each lever in successive sessions. Each session continued until the rat had earned 50 reinforcers or 1 hr had passed. The order in which animals were trained on each lever was counterbalanced such that half the lesioned and half the shamlesioned rats were trained on the left lever first, the remainder on the right lever first. If animals did not complete 50 lever presses in an hour, they underwent an additional training session on that lever, before proceeding to the next stage of training. In the next two sessions of training rewards were delivered according to a random interval (RI) 30 sec schedule. Sessions lasted 40 min. In the first of these sessions, the rats were trained on the same lever as they had been trained with on in the first session of lever press training. The other lever was present during the second session. Pellets and sucrose solution were used as rewards with equal probability. Discrimination training. In the instrumental discrimination phase, which lasted 10 d, each session was 40 min long, and consisted of eight, alternating 5 min presentations of an auditory stimulus: either a3khz tone or a 10 Hz click-train. Both levers were present for the duration of the session, but during each stimulus only one lever was reinforced. For example, during tone presentation pressing on the left lever would be reinforced, and during click-train presentation pressing on the right lever would be reinforced. Reinforcement was delivered on a RI 30 sec schedule. Rats were randomly allocated to two groups, consistent and inconsistent, with eight sham- and eight BLA-lesioned animals in each group. The contingencies for each of these groups are shown in Table 1. Animals in the consistent group received a specific reinforcer sucrose solution or food pellets after an appropriate lever press in the presence of one of the auditory discriminative stimuli. Animals in the inconsistent group received either sucrose solution or food pellet reinforcement with equal probability after appropriate lever presses. All conditions were fully counterbalanced across lesion and sham groups, such that during the tone, the left lever was as likely to be the correct response as the right lever, and in group consistent, a response during the tone was as likely to be reinforced by a food pellet as by delivery of sucrose solution. The first

3 9020 J. Neurosci., November 15, 2001, 21(22): Blundell et al. BLA Lesions and Reinforcer Representations discriminative stimulus presented in any training session was determined randomly by the computer. Responding on the reinforced and the nonreinforced lever was recorded separately for each 5 min bin. Extinction tests. After training, rats received a test session conducted in extinction. The procedure was identical to that used for the training sessions, but no reinforcers were delivered. The rats then received a single reacquisition session, identical to the training sessions, and were then tested again, this time in the presence of rewards but in the absence of tone or clicker stimuli. This test assessed the possibility that the reinforcers themselves might have come to act as discriminative stimuli. That is, receiving a particular type of reward could indicate which response an animal should make in the consistent group, but not in the inconsistent group. If animals were using this cue to solve the original discrimination we would expect performance to decline to chance levels in the extinction test but differential responding to be maintained in this outcomes only test. If discriminative performance is based on the auditory cues, the reverse pattern should be obtained. Experiment 2: reinforcer-specific Pavlovian-to-instrumental transfer Subjects The subjects were 16, naive, male, hooded Lister rats, with a mean weight of 305 gm (range, gm) at the time of surgery. The rats were maintained as described for experiment 1. Surgery and histology Eight of the rats underwent surgery to destroy the cell bodies of the BLA, and the remainder received sham control surgery. All surgical details and histological procedures were identical to those for experiment 1. Apparatus Training was performed in the operant chambers described for experiment 1. Procedure Magazine training. Magazine training proceeded as described for experiment 1. All rats then received Pavlovian and instrumental training sessions. Pavlovian and instrumental training. On days 3, 5, 7, 9, and 11, all rats received instrumental training. Each instrumental conditioning session lasted 20 min, and there were two sessions per day. At the start of each session, one of two levers was inserted into the Skinner box. Responding on this lever was reinforced on an RI 30 sec schedule. The rats were trained in separate sessions (one session in the morning, and one in the afternoon) to press each of the two levers. Two reinforcers (sucrose solution and food pellets) were used, and each was earned by responding on a particular lever. Which lever earned which reinforcer was counterbalanced across groups, such that, for half of the rats in each group, responding on the left lever was reinforced by sucrose solution and responding on the right lever was reinforced by food pellets, and for the other half, these pairings were reversed. On days 4, 6, 8, 10, and 12, the rats received Pavlovian conditioning sessions. Each session lasted 50 min, and there was one session per day. No levers were present during these sessions. Each session comprised 10, alternating 5 min presentations of auditory stimuli either a tone or a click-train (the first stimulus presented during each session was randomly determined by the computer). During each stimulus, one of two possible reinforcer types (sucrose solution or food pellets) was delivered on an RT 30 sec schedule. The reinforcer that was delivered was different during each of the stimuli, and this was counterbalanced across groups. For half of the rats in each group, sucrose solution was delivered during tone presentations, and food pellets were delivered during the click-train. For the other half of the rats in each group, these pairings were reversed. Pavlovian to instrumental transfer tests. After the training, there was a test session that lasted 45 min and consisted of nine, 5 min trials. There were three trial types (each of which occurred three times) baseline, click, and tone. During baseline trials, no auditory stimuli were presented, during click trials, the click-train was presented for 5 min, and during tone trials, the tone was presented for 5 min. The trials were ordered such that each three-trial block consisted of one of each of the trial types, but within that block the order was determined pseudorandomly by computer. For the entirety of the test session both levers were present in the chamber, but no reinforcers were presented. After this test session, the rats underwent a further 10 d of training: 5dofinstrumental training and 5dofPavlovian training, with the procedures outlined above, such that in total there were 10 sessions of Pavlovian conditioning and 10 sessions of instrumental training on each response lever. Another test session, identical to the first, was then administered. RESULTS Experiment 1: BLA lesions and the differential outcomes effect Trapold (1970) first reported the differential outcomes procedure. He compared the performance of two groups of animals in learning a discrete-trial instrumental conditional discrimination. On each trial the rats were presented with two levers, along with either a tone or click stimulus. In the control group, correct responses in the presence of a given stimulus were always rewarded by delivery of the same reward (a food pellet for some animals, sucrose solution for the others). In the experimental group, correct responses in the presence of one stimulus were consistently reinforced by one reward; correct responses in the presence of the other stimulus were consistently reinforced by the other reward. The animals in the experimental group learned the discrimination faster than those in the control group. There are a variety of possible explanations for the differential outcomes effect, but they all assume that the more rapid acquisition of the conditional discrimination in the experimental group is attributable to associations forming between the discriminative stimuli and the different sensory properties of the reinforcers. If BLAlesioned animals cannot form or cannot use associations involving the specific sensory properties of motivationally significant stimuli, we would predict that BLA-lesioned rats would not show the differential outcomes effect. The probability level for rejection of the null hypothesis was p Histology Histological analysis revealed that two rats had damage to the central nucleus of the amygdala. These were excluded from all further analysis. A further rat was excluded because it had damage to the amygdalohippocampal area. All other rats were retained in the analysis, leaving 13 lesioned and 16 sham-lesioned rats. Figure 1 shows the extent of the smallest and largest lesions. Although there was moderate variability in the size of the lesions, all lesions included damage to the basolateral and lateral nuclei of the amygdala. Some animals also had damage to the basomedial nucleus. Of the 13 lesioned rats, seven were in the consistent group, and six were in the inconsistent group. Acquisition of conditional discrimination The mean number of correct and incorrect responses in each session was calculated for each group of rats. The left-hand panel of Figure 2 shows acquisition of the conditional discrimination in the sham-lesioned groups. Rats in both inconsistent and consistent groups came to respond more on the correct lever than on the incorrect lever as training proceeded. However, acquisition of the discrimination was retarded in the inconsistent group, who had responded at a higher rate on the incorrect lever and a lower rate on the correct lever than the consistent group; that is, the sham-lesioned rats showed the differential outcomes effect. The right-hand panel of Figure 2 shows acquisition of the conditional discrimination in the BLA-lesioned groups. Again, across training sessions, rats in both inconsistent and consistent groups came to respond more on the correct lever than on the incorrect lever. Hence, BLA lesions do not prevent acquisition of the conditional discrimination. There was, however, no difference between the performance of

4 Blundell et al. BLA Lesions and Reinforcer Representations J. Neurosci., November 15, 2001, 21(22): Figure 3. Group mean difference scores (total correct incorrect responses) during the extinction test session of experiment 1. Bars represent 2 SED. Figure 1. Schematic representation of excitotoxic lesions to the basolateral amygdala from experiment 1. Shaded areas represent the smallest (black) and largest ( gray) extent of neuronal damage. Coronal sections are 1.8 to 3.8 mm relative to bregma (Swanson, 1998). Figure 2. Acquisition of a conditional instrumental discrimination in sham-lesioned (left) and BLA-lesioned (right) rats. Closed symbols indicate consistent groups; open symbols indicate inconsistent groups; circles indicate correct responses; squares indicate incorrect responses. Bars represent 2 SEs of the differences between means (SED) derived from the appropriate ANOVA term and may be used as a guide to variability. the inconsistent and the consistent groups; that is, BLA lesions eliminated the differential outcomes effect. It may be added that this experiment also confirmed that the BLA-lesioned rats show no deficit in basic instrumental or Pavlovian conditioning. For the purposes of analysis, the data presented in Figure 2 were converted into difference scores: the number of correct lever presses in a session minus the number of incorrect lever presses. A three-way ANOVA was conducted out on the difference scores, with variables of session (1 10), lesion (BLA, sham), and group (consistent, inconsistent). This revealed significant main effects of session (F (9,225) 71.86; p 0.001) and group (F (1,25) 9.21; p 0.01), but not lesion (F 1). Of the two-way interactions, only the session group interaction was significant (F (9,225) 2.70; p 0.01). Simple main effects analysis of this interaction revealed a significant effect of group on sessions 4 10 (smallest F (1,62) 4.75; p 0.05) and an effect of session in both groups (sham, F (9,225) 52.80, p 0.001; BLA, F (9,225) 23.73, p 0.001). Neither the session lesion interaction (F 1), nor the lesion group interaction (F (1,25) 2.92) was significant. Critically, however, the three-way interaction of session lesion group was significant (F (9,225) 1.93; p 0.05). To examine this three-way interaction, separate analyses were conducted on the data from sham-lesioned and BLA-lesioned rats. Analysis of the data from the sham-lesioned rats revealed significant main effects of session (F (9,126) 47.07; p 0.001) and of group (F (1,14) 15.00; p 0.005). There was a significant session group interaction (F (9,126) 5.46; p 0.05). Analysis of the simple main effects revealed a significant effect of group from session 4 onward (smallest F (1,33) 5.07). The analysis of the data from BLA-lesioned rats also revealed a significant main effect of session (F (9,99) 28.00; p 0.001). However, there was no main effect of group (F 1) and no session group interaction (F 1). Extinction test Figure 3 shows group mean difference scores for the extinction test session. All groups maintained the discrimination (all had positive difference scores) confirming that the performance shown during acquisition was not dependent on cues supplied by the delivery of rewards. The overall pattern of group differences shown during acquisition was maintained during this session. That is, for sham-lesioned animals, the consistent group showed much better performance than the inconsistent group. The two BLA-lesioned groups did not differ and their performance fell midway between that shown by the two sham-lesioned groups. Statistical analysis largely confirmed these impressions. A two-way ANOVA, with variables of lesion (BLA, sham) and group (consistent, inconsistent), revealed a significant main effect of group (F (1,25) 5.31), no significant main effect of lesion (F 1), and a lesion group interaction that fell just short of the conventional level of significance (F (1,25) 3.54; p 0.07).

5 9022 J. Neurosci., November 15, 2001, 21(22): Blundell et al. BLA Lesions and Reinforcer Representations Figure 4. Group mean difference scores of responses during the outcomes only session of experiment 1, in which no discriminative stimuli were presented. Bars represent 2 SED. Simple main effects analysis showed an effect of group in shamlesioned animals (F (1,25) 9.80; p 0.005), but no effect of group in BLA-lesioned animals (F 1), or any effect of lesion across consistent (F (1,25) 1.46) or inconsistent (F (1,25) 2.12) groups. Outcome-only test Figure 4 shows the number of correct responses, minus incorrect responses, during the outcomes-only session. Again, all groups showed positive difference scores, indicating that the presentation of the reinforcer could serve as a cue controlling performance: presumably once they had received a reward for a response on a given lever, animals tended to persist in responding on that lever. There is some suggestion that this effect was more powerful in sham-lesioned than in BLA-lesioned animals and also that animals in the consistent groups produced more correct responses that those in the inconsistent groups. However, variability within groups was large (and potentially caused by lesion variability), and these differences proved to be statistically unreliable. A two-way ANOVA conducted on the data summarized in Figure 4, with variables of lesion (BLA, sham) and group (consistent, inconsistent), revealed no significant main effects (group, F (1,25) 1.38; lesion, F (1,25) 2.55) and no significant interaction (F 1). This suggests that differential reinforcement provides little or no advantage unless it is signaled by the tone or clicker. Experiment 2: reinforcer-specific Pavlovian-instrumental transfer Experiment 1 demonstrated that rats with BLA lesions can learn a conditional discrimination but do not show the differential outcomes effect. This result may be taken to suggest that rats with BLA lesions do not form, or cannot use, representations of the sensory aspects of rewards that enter into associations with other stimuli. Their solution of the conditional discrimination might be achieved in one of two ways: it may be that BLA-lesioned rats are simply stimulus response (S R) animals, incapable of forming associations between CSs and any aspect of the US, be it sensory or motivational, but capable of associating a given response with the prevailing stimulus conditions. As pressing on one lever was reinforced during the tone, and on the other lever during the click-train, S R associations would permit solution of the discrimination and would fail to show any effects of response reinforcer pairings. An alternative possibility is that rats with BLA lesions may be able to form associations between CSs and the motivational aspects of a US, but not associations involving the sensory aspects of a US. To determine which of these accounts is correct we examined the effect of BLA lesions on Pavlovian-toinstrumental transfer of control. Pavlovian CSs can strongly influence instrumental behavior (Rescorla and Solomon, 1967; Kruse et al., 1983). For example, presentation of a CS that has been paired with reward will, in some circumstances, potentiate instrumental responding that has been reinforced by that reward. This potentiation of responding is reinforcer-specific: if an animal has a choice of two responses, when a CS is presented it will increase responding more on a lever that resulted in the delivery of the reward that has followed the CS than on a lever that previously led to a different reward. For example, a CS that has been paired with food pellets will cause an increase in responding more on a manipuladum that has been reinforced by the same food pellets than one that has been reinforced by a different appetitive reinforcer (Kruse et al., 1983). We used this phenomenon to study the associations that are formed during Pavlovian and instrumental conditioning in rats after BLA lesions. There are two aspects to the potentiation of responding caused by presentation of a Pavlovian CS, a general enhancement in instrumental responding by virtue of the association of the CS with arousing, motivational aspects of the US, and a reinforcer-specific potentiation by virtue of the association of the CS with the specific sensory aspects of the reward. If BLAlesioned rats are simply S R animals, incapable of forming associations between a CS and any aspect of the reinforcer, then we would not expect them to show any potentiation of responding caused by superimposing a CS on instrumental responding. In fact, we might expect a reduction in responding, because the response normally elicited by presentation of a CS (foodmagazine approach) is likely to interfere with lever pressing. However, if BLA-lesioned animals form associations between CSs and the motivational, but not sensory, aspects of a US, then we would expect instrumental responding to increase during presentation of CSs, but not to increase specifically on the response that had previously earned the same reward as that predicted by the CS. We predicted that sham-lesioned rats would show an increase in responding on both levers during the tone and the click-train (by virtue of their conditioned arousing effects) but that the increase would be greater on the lever that had previously resulted in delivery of the same reward as was predicted by the stimulus. The question of interest was whether BLA-lesioned rats would show a general increase in responding during presentation of the CSs, and if they did, if that increase would be reinforcer-specific. Histology Figure 5 shows the extent of the smallest and largest lesions. On the whole the lesions proved to be smaller than in experiment 1. Shrinkage had occurred in all of the accepted cases, as had gliosis (though often this was not complete). All animals had bilateral damage to the basolateral nucleus of the amygdala and at least unilateral damage to the lateral nucleus of the amygdala. In one case there was limited damage to the cortical nucleus of the amygdala. All animals had an intact central nucleus.

6 Blundell et al. BLA Lesions and Reinforcer Representations J. Neurosci., November 15, 2001, 21(22): Figure 6. Group mean response rates during presentation of a CS and during the baseline (no CS) period from extinction test sessions of experiment 2. Bars represent 2 SED. produce modulate instrumental performance by Pavlovian-toinstrumental transfer. Figure 5. Schematic representation of excitotoxic lesions to the basolateral amygdala from experiment 2. Shaded areas represent the smallest (black) and largest ( gray) extent of neuronal damage. Coronal sections are 1.8 to 3.8 mm relative to bregma (Swanson, 1998). Pavlovian conditioning Both groups of rats made more magazine entries during the trials in which pellets were delivered than during the trials in which sucrose solution was delivered (average magazine entries per minute during CS: BLA sucrose 9.1; BLA pellet 13.4; sham sucrose 8.7; sham-pellet 13.6). A three-way ANOVA, with the variables of lesion (BLA, sham), session (1 10), and reinforcer (pellets, sucrose), revealed a significant main effect of reinforcer (F (1,126) 17.52; p 0.001), a significant main effect of session (F (9,126) 4.86; p 0.05), but no significant main effect of lesion (F 1). None of the interactions were significant (all F values 1). The difference in performance for pellets and sucrose is not likely to reflect differences in the motivational value of the rewards. Because reward was delivered throughout the duration of the CS on a RT schedule, assessment of magazine approach reflects both conditioned and unconditioned aspects of performance. In our apparatus actual presentation of pellets evokes more frequent magazine entries than does sucrose presentation, although there is little difference in the duration of head entry into food magazines. As for experiment 1, pilot studies indicate that for these two rewards levels of conditioned approach and lever pressing for sucrose and pellets occur at equal rates (see instrumental training below) and the rewards are well matched for their motivational properties. Because performance reflected both conditioned and unconditioned magazine approach, there was no indication of acquisition across sessions, as the animals entered this phase of training (which followed magazine training) with a pre-established high rate of magazine entry. However, a similar procedure (Balleine, 1994; Corbit et al., 2001) has been shown to endow the stimuli with the ability to Instrumental training There was no difference in the acquisition of instrumental performance by BLA-lesioned and sham-lesioned rats (average lever presses per minute: BLA sucrose 7.1; BLA pellet 6.8; sham sucrose 7.0; sham pellet 5.6). Both groups of responded at a higher rate in later sessions than in earlier sessions, and neither group showed a bias toward responding more on one lever than the other, supporting pilot studies indicating that the rewards were matched for motivational salience. A three-way ANOVA, with the variables of session (1 10), lever (left, right), and lesion (BLA, sham), revealed a significant main effect of session (F (9,126) 25.57; p 0.001), no significant effect of lever (F 1), and no significant main effect of lesion (F 1). None of the interactions were significant (maximum F (9,126) 2.19). Pavlovian to instrumental transfer test There were no main effects or interactions involving test (all F values 1), so the results from tests 1 and 2 were collapsed for presentation and analysis. Figure 6 shows the overall rate of lever pressing during presentation of the auditory CSs and during the baseline period when no CS was presented, for both BLAlesioned and sham-lesioned rats. Lever pressing is pooled over both levers and over both CSs. Both BLA- and sham-lesioned rats responded at a higher rate during presentation of a CS than during the baseline period and did so to primarily the same extent. A two-way ANOVA with the variables of CS (CS, baseline) and lesion (BLA, sham) revealed a significant main effect of CS (F (1,14) 16.59; p 0.005), but no significant main effect of lesion (F 1) or CS lesion interaction (F (1,14) 1.03). Figure 7 shows the selective changes in lever pressing produced by presentation of specific auditory stimuli. A same response is defined as one that had previously been reinforced by the foodtype associated with the CS being presented. A different response is pressing on the other lever. The score given is an elevation ratio, calculated as the rate of responding on the lever during presentation of the CS, divided by the rate of responding on that lever during the baseline period. If there was no increase in the rate of responding during presentation of the CS, the

7 9024 J. Neurosci., November 15, 2001, 21(22): Blundell et al. BLA Lesions and Reinforcer Representations Figure 7. Group mean elevation ratios for same and different responses in experiment 2 (see Results for details). Bars represent 2 SED. elevation ratio would be 1. Results are again collapsed across tests 1 and 2 for presentation. Although levels of transfer were generally lower in test 2 than test 1 (average elevation ratios: BLA test 1, 2.5; BLA test 2, 2.0; sham test 1, 2.7; sham test 2, 2.2), there was no difference in the pattern of responding. Sham-lesioned rats showed a greater increase in responding on the same lever than on the different lever during both tests 1 and 2, demonstrating reinforcer-specific Pavlovian-to-instrumental transfer. In contrast, BLA-lesioned rats showed an equivalent increase in the rate of lever pressing on the same lever as on the different lever. Thus, although BLA-lesioned rats do show an increase in responding during the CS, they are not sensitive to the relationship between the CS and the type of reinforcer produced by the instrumental response. A three-way ANOVA, with the variables of response (same, different), lesion (BLA, sham), and test session (test 1, test 2) revealed no significant main effect of response (F (1,14) 2.04), a marginal effect of test session (F (1,14) 4.58; p 0.05), and no significant main effect of lesion (F 1). There was, however, a significant response lesion interaction (F (1,14) 4.69; p 0.05). Simple main effects analysis revealed a significant effect of response in sham-lesioned animals (F (1,14) 6.46; p 0.05) but not in BLA-lesioned animals (F 1). No other interactions were significant (F values 1). Hence, sham-lesioned animals show a reinforcer selectivity of transfer, whereas BLA-lesioned animals show a general transfer effect, but no reinforcer specificity. DISCUSSION Experiment 1 used the differential-outcomes effect to examine whether BLA-lesioned rats could use differences in the sensory properties of reinforcers to aid discrimination learning. Shamlesioned rats trained on a discrimination in which different rewards were correlated with alternative responses learned the discrimination faster than rats for which reward-type was not correlated with response. BLA-lesioned rats also acquired the discrimination but did not show this differential-outcomes effect. BLA-lesioned rats cannot make use of the differences between rewards to help them learn the discrimination. This is consistent with the possibility that BLA-lesioned rats cannot form associations between neutral events and the sensory properties of motivationally significant stimuli or that BLA-lesioned rats were learning an S R association. Experiment 2 examined the effect of superimposing a CS on instrumental responding, in BLA- and sham-lesioned rats. Both groups increased instrumental responding when a CS was presented, suggesting that they formed an association between the CS and the arousing properties of the US, a general expectation of reward, but not an expectation that supplies information about which specific reward will occur. The fact that BLA-lesioned rats show increased responding during presentation of a Pavlovian CS is inconsistent with the suggestion that these are purely S R animals. By this analysis any conditioned responding established during Pavlovian training must be S R magazine approach, a response that would be expected to compete with, rather than enhance, lever pressing. When we examined the reinforcer specificity of this Pavlovianto-instrumental transfer, we found that sham-lesioned animals increased their rate of responding more on the lever on which they had previously earned the same reward as that predicted by the Pavlovian CS than on the lever on which they had previously earned a different reward. In contrast, BLA-lesioned rats showed an equal increase in responding on both levers, suggesting that BLA-lesioned rats were insensitive to the differences between the rewards, i.e., to their unique sensory properties. In normal animals, when a CS is paired with a US, several different associations may form (Konorski, 1967; Dickinson and Dearing, 1978). CS presentation evokes an internal CS representation (initially sensory in nature, see Wagner and Brandon, 1989). US presentation activates both sensory (US s ) and motivational (US m ) representations, which elicit their own characteristic responses (consummatory and preparatory, respectively). The CS representation may form associations with both US s and US m, and/or with the responses these produce. Although BLA lesions do not interfere with simple Pavlovian conditioning (experiment 2), BLA-lesioned rats are not sensitive to post-conditioning changes in reward value (Hatfield et al., 1996). Because reward devaluation by LiCl injection is likely to act via US s (Garcia, 1989; Holland, 1990; Berridge, 1991; Balleine, 2000) (see also Balleine and Dickinson, 1998a), reducing palatability and hedonic evaluation of the reward, we suggest that CS US S associations are not formed in BLA-lesioned animals. Rather, conditioned responding depends on the formation of associations between CS and response or between CS and US m. We should be explicit about what we mean by US s in contrast to US m.byus s we refer to those aspects of the reinforcer that are specific to the individual outcome. Substantial evidence (Balleine, 2000) suggests that reinforcers are represented by a number of sensory features to which incentive value is ascribed. This value is manifest by changes in hedonic evaluation, palatability, and subsequent desirability, of the reinforcer (Berridge, 1996; Balleine, 2000). Animals with BLA lesions are incapable of forming, or using, associative US s representations, and they cannot respond in accordance with associatively activated changes in hedonic evaluation or palatability. Note that BLA lesions do not disrupt direct perception of changes in palatability; rats ability to show rejection of devalued foods in consumption tests is maintained (Dunn and Everitt, 1988). This contrasts with US m, by which we mean the arousing aspects of motivation that lead to alterations in preparatory activity and to changes in the strength, but not direction, of Pavlovian modulation of instrumental performance (Dickinson and Dearing, 1978). The differential outcomes effect occurs because the discriminative stimulus forms an association with the specific properties of the food that accompanies it (Trapold, 1970; Trapold and Overmier, 1972; Rescorla, 1994). According to the analysis pre-

8 Blundell et al. BLA Lesions and Reinforcer Representations J. Neurosci., November 15, 2001, 21(22): sented above, BLA-lesioned rats do not show this effect because the discriminative stimulus cannot form associations with a representation that includes the sensory properties of the reward. Accordingly, BLA-lesioned rats do not show any advantage of differential reinforcement in discrimination learning. A similar logic explains the results of experiment 2. During Pavlovian conditioning, associations form between CS and reward, but in BLA-lesioned animals, this does not include US s. When the CS is presented during an extinction test in which animals are making instrumental responses, the CS creates the expectation of reward (US m ) but does not activate a representation of the specific sensory aspects of that reward (US s ). Hence, there is a general increase in responding, but no selective effect based on specific response reward relationships. This analysis can also account for the results of previous research. Consider second-order conditioning. Hatfield et al. (1996) paired CS 1 with reward in a first phase, and in a second phase CS 2 was followed by CS 1. There are three proposed mechanisms for the acquisition of responding in second-order conditioning. First, animals may form associations between CS 2 and CS 1, which, by virtue of an associative chain and existing CS 1 reward associations, will produce responding during CS 2 (S S learning, Rashotte et al., 1977; Rescorla, 1979, 1980, 1982). Second, animals may form S R associations between CS 2 and any response evoked by CS 1, increasing responding during CS 2 (S-R learning, Rizley and Rescorla, 1972; Holland and Rescorla, 1975). Finally, animals may learn direct associations between CS 2 and the representation of reward evoked by presentation of CS 1 (Ross, 1986). There is substantial evidence in favor of S S and S R interpretations, but rather less favoring CS 2 reward associations [indeed, theories of learning (Wagner, 1976) suggest that associations between CSs and evoked representations are difficult to form (but see Holland, 1981)]. Whether S R ors S associations form depends primarily on the parameters of the experiment, the similarity between CS 1 and CS 2 (Rescorla, 1982) and the capacity of CS 1 to evoke a response during CS 2 CS 1 pairings (Rescorla, 1973). Hence, the failure of BLA-lesioned animals to show second-order conditioning can be accounted for in two ways: first, if BLA-lesioned animals cannot learn about the sensory properties of motivationally significant events, then during phase 2 they will have difficulty forming the association between CS 2 and CS 1 (note that CS 1 will have acquired motivational significance caused by first-order associations with US M.) Second-order conditioned responding based on an associative chain would not be possible. Second, if BLA-lesioned animals cannot show S R learning, they will be unable to form associations between CS 2 and the response evoked by CS 1.Thefirst of these explanations fits our interpretation in terms of the failure of BLA animals to represent the sensory properties of motivationally significant events. However, a failure of S R learning in BLA-lesioned animals has problems accounting for deficits after reinforcer devaluation (Hatfield et al., 1996) and the reinforcer-specificity effects presented in experiments 1 and 2. If appetitive conditioning proceeds normally, but is insensitive to reward devaluation, animals must either be learning by S R mechanisms or with an impoverished representation of the US. If the former explanation is invoked to account for normal appetitive conditioning, then a disruption of S R learning cannot then be used to explain deficits in second-order conditioning. For these reasons we favor the parsimonious explanation that BLA-lesioned animals have a deficit in representing the sensory properties of motivationally significant events. A similar explanation applies to the finding that BLA lesions disrupt the ability of Pavlovian CSs to act as reinforcers for instrumental conditioning. Because CSs are purely sensory stimuli with acquired motivational significance, BLA-lesioned rats will have difficulty learning lever press CS associations, and acquisition of lever pressing rewarded by CSs will be disrupted. However, even if this associative chain mechanism is not functioning, acquisition could occur if animals still show S R learning, formation of a stimulus lever press association reinforced by the acquired motivational properties of the CS. There is evidence that BLA-lesioned monkeys can show conditioned reinforcement (Malkova et al., 1997) and that BLA-lesioned rats show only partial (Cador et al., 1989; Everitt et al., 1989; Burns et al., 1993; Alderson et al., 2000) or transient (Burns et al., 1999) deficits in conditioned reinforcement. In BLA-lesioned rats, the ability of a CS to evoke representations of the motivationally arousing aspects of a US does not appear to be impaired. Rather, BLA-lesioned rats seem incapable of forming associations between events and the sensory-specific aspects of motivationally significant stimuli. These sensory aspects are important in the hedonic evaluation of rewards and may depend on connections between the BLA, insular cortex (Balleine and Dickinson, 1998b), and nucleus accumbens core (Corbit et al., 2001). In contrast, other data suggest that the central nucleus of the amygdala is involved in arousing aspects of the reinforcer (Galaverna et al., 1993; Robledo et al., 1996), mediated via the nucleus accumbens shell (Corbit et al., 2001). REFERENCES Alderson HL, Robbins TW, Everitt BJ (2000) The effects of excitotoxic lesions of the basolateral amygdala on the acquisition of heroin-seeking behavior in rats. Psychopharmacology 153: Balleine BW (1994) Asymmetrical interactions between thirst and hunger in Pavlovian-instrumental transfer. Q J Exp Psychol 47B: Balleine BW (2000) Incentive processes in instrumental conditioning. In: Handbook of contemporary learning theories (Mowrer RR, Klein SB, eds), pp Mahwah, NJ: Lawrence Erlbaum. Balleine BW, Dickinson A (1998a) The role of incentive learning in instrumental outcome revaluation by specific satiety. Anim Learn Behav 26: Balleine BW, Dickinson A (1998b) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropsychopharmacology 37: Berridge KC (1991) Modulation of taste affect by hunger, caloric satiety, and sensory-specific satiety in the rat. Appetite 16: Berridge KC (1996) Food reward: brain substrates of wanting and liking. Neurosci Biobehav Rev 20:1 25. Burns LH, Robbins TW, Everitt BJ (1993) Differential effects of excitotoxic lesions of the basolateral amygdala, ventral subiculum and medial prefrontal cortex on responding with conditioned reinforcement and locomotor activity potentiated by intra-accumbens infusions of d-amphetamine. Behav Brain Res 55: Burns LH, Everitt BJ, Robbins TW (1999) Effects of excitotoxic lesions of the basolateral amygdala on conditional discrimination learning with primary and conditioned reinforcement. Behav Brain Res 100: Cador M, Robbins TW, Everitt BJ (1989) Involvement of the amygdala in stimulus-reward associations: interaction with the ventral striatum. Neuroscience 30: Corbit LH, Muir JL, Balleine BW (2001) The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell. J Neurosci 21: Davis M (1992) The role of the amygdala in conditioned fear. In: The amygdala: neurobiological aspects of emotion memory and mental dysfunction (Aggleton JP, ed), pp New York: Wiley-Liss. Davis M (2000) The role of the amygdala in unconditioned and conditioned fear and anxiety. In: The amygdala: a functional analysis (Aggleton JP, ed), pp New York: Oxford UP. Dickinson A, Dearing MF (1978) Appetitive-aversive interactions and inhibitory processes. In: Mechanisms of learning and motivation: a memorial volume to Jerzy Konorski (Dickinson A, Boakes RA, eds), pp Hillsdale, NJ: Erlbaum.