Parallel incentive processing: an integrated view of amygdala function

Transcription

1 ARTICLE IN PRESS Parallel incentive processing: an integrated view of amygdala function Bernard W. Balleine 1 and Simon Killcross 2 1 Department of Psychology and the Brain Research Institute, University of California, Los Angeles, Box , CA , USA 2 School of Psychology, Cardiff University, Tower Building, Park Place, Cardiff CF10 3AT, UK The amygdala is a heterogeneous structure that has been implicated in a wide variety of functions, most notably in fear conditioning. From this research, an influential serial model of amygdala processes has emerged in which aversive learning is mediated by the amygdala basolateral nucleus whereas performance, in this case of various defensive reflexes, is mediated by the central nucleus. By contrast, recent evidence from appetitive conditioning studies suggests that the basolateral and central nuclei operate in parallel to mediate distinct incentive processes: the basolateral nucleus encodes emotional events with reference to their particular sensory-specific features, whereas the central nucleus encodes their more general motivational or affective significance. Given that there is little if any direct behavioral evidence for the serial model, we suggest that more attention should be given to the claims of the parallel view. Introduction Although it has long been known that the amygdala has a central role in emotional learning and memory, the exact nature of its involvement is still disputed. Several general theories of amygdala function have been advanced [1 3], with perhaps the most influential being the model developed from studies of aversive learning, such as Pavlovian fear conditioning [4,5]. This view proposes that the formation of associations between sensory perceptual cues and biologically significant events takes place in the lateral amygdala, where pathways carrying these sources of information converge, and that these associations then mediate the performance of various defensive responses via output through the central amygdala to diverse hypothalamic, midbrain and medullary nuclei. By this model, therefore, processing of emotional information within the amygdala proceeds in a serial manner between the lateral and central nuclei. This is in general agreement with recent anatomical studies suggesting that the amygdala is not a unitary structure but, rather, is composed of anatomically distinct nuclei Corresponding author: Balleine, B.W. (balleine@psych.ucla.edu). broadly divisible into two groups: one whose internal circuitry is more cortical, including the lateral, basolateral and basomedial nuclei (BLA), and another that is more striatal, involving the medial and central nuclei (CeN) [6 8]. Nevertheless, recent evidence from several sources challenges this serial view. Studies investigating the role of the amygdala in motivational processes other than fear, including studies assessing appetitive Pavlovian conditioning and reward-related learning in instrumental conditioning, suggest that the BLA and CeN function not serially but independently and in parallel to mediate distinct aspects of emotional processing [9]. When added to recent evidence questioning key assumptions of the serial model in fear conditioning, including the necessity of plasticity in the lateral amygdala for fear learning and the interaction of lateral and central nuclei in performance [5], these findings suggest that it is perhaps time to consider alternative models of amygdala functioning. This paper advances a parallel model of amygdala function in emotional learning, using three lines of argument. First, evidence from appetitive Pavlovian conditioning studies is considered, suggesting that the BLA and CeN function independently and in parallel to control distinct aspects of emotional learning. The BLA mediates associations between predictive stimuli and the sensory properties of biologically significant events, and the CeN mediates the association of predictive stimuli with the affective or emotional properties of those events. Second, recent evidence from Pavlovian fear conditioning is re-evaluated, suggesting that, whereas it is at least partially inconsistent with the serial model, it is wholly consistent with predictions from the parallel model. Finally, we consider evidence from studies of rewardrelated learning in instrumental conditioning that demonstrate that, whereas the BLA mediates the emotional significance of specific rewards, the CeN seems to establish the general affective response that underlies the nonspecific reinforcing effect of those events. Together, this evidence suggests that the amygdala subserves incentive learning, the process through which sensory-perceptual features of appetitive (or aversive), and rewarding (or punishing) events acquire affective significance and, hence, the ability to motivate or incite responses and actions. The amygdala as a whole controls /$ - see front matter Q 2006 Elsevier Ltd. All rights reserved. doi: /j.tins

2 2 ARTICLE IN PRESS both the associative significance (through Pavlovian incentive learning) and the reward value (through instrumental incentive learning) [10 13] of sensory perceptual events. In this respect, the different nuclei of the amygdala have a coordinating function: they establish the incentive value of motivationally significant events within specific functional domains. Establishing associative significance: Pavlovian incentive learning Numerous authors have suggested that Pavlovian conditional stimuli (CSs) that are initially motivationally neutral can acquire incentive properties through association with representations of biologically significant unconditional stimuli (USs) [10 13]. However, it has long been recognized that USs are represented in terms of multiple features or components and that these can enter into independent associations with CSs. Konorski [14] was the first to articulate this idea to explain his distinction between consummatory and preparatory conditioning. He argued that independent associations are formed between the representation of the CS and both the sensory features and the motivational properties of the US. In this model, the sensory associations mediate USspecific consummatory responses, such as chewing, licking, blinking and flinching, and the motivational associations mediate more general preparatory behavior, such as changes in heart rate, blood pressure, approach and withdrawal. Current evidence suggests that the amygdala is heavily involved in both of these aspects of Pavlovian incentive learning: that the BLA mediates the essential associative processes that support consummatory conditioning, whereas the CeN mediates processes that support preparatory conditioning (Figure 1). Appetitive incentives In the appetitive situation, lesions of the BLA produce a deficit in the association of CSs with sensory-specific features of US representations without affecting the production of preparatory conditioned responses (CRs), most notably conditioned approach responses motivated by the arousing nature of the US [15]. For example, whereas lesions of the BLA do not affect acquisition of conditioned approach towards food during a tone or light CS, they do abolish the sensitivity of approach responses elicited by that CS to post-training devaluation of its associated US [16]. In the past few years, it has become clear that BLA lesions are in fact affecting highly specific consummatory aspects of CS US associations. For example, in addition to other consummatory reflexes, CSs paired with food can elicit eating in non-deprived animals, an effect that depends on the integrity of the BLA and its connections with ventromedial prefrontal cortex and the lateral hypothalamus [17,18]. Pavlovian CSs can also exert specific effects on food seeking. Evidence from experiments employing Pavlovian instrumental transfer procedures (PIT; Box 1) suggests that Pavlovian CSs can signal highly specific features of, for example, a food US, and hence prime or facilitate performance of actions that have been trained with that food as an outcome relative to performance of other actions [12]. Several studies suggest (a) CS US (b) Sensory thalamus and cortices Visceral brainstem and hypothalamic afferents Affective CeN system CS US BLA Preparatory CRs (general CRs based on US valence) Consummatory CRs (US-specific CRs) Brainstem, midbrain and reticular nuclei Bed nucleus, hypothalamus, prefrontal cortex and n.accumbens TRENDS in Neurosciences Figure 1. Role of the amygdala in Pavlovian incentive learning. (a) Current model of the motivational and affective associations acquired by the conditioned stimulus (CS) during both appetitive and aversive Pavlovian conditioning (derived from [14]). Differential associations with the sensory-specific and general affective properties of the unconditioned stimulus (US) produce various conditioned responses (CRs) grouped generally into discrete consummatory reflexes (e.g. chewing, salivating, blinking, flinching or retraction) and more diffuse preparatory reflexes (e.g. heart rate, blood pressure, approach or withdrawal). Solid arrows reflect excitatory connections and the broken arrow represents the facilitatory effect of conditioned affective activation on the performance of consummatory CRs. (b) Plasticity involving segregated inputs to the lateral, basolateral and basomedial nuclei (BLA) and the medial and central nuclei (CeN) from sensory and visceral brainstem and hypothalamic afferents mediates the formation of stimulus-specific CS US associations in the BLA, and general CS affect associations in the CeN. Differential outputs to brainstem autonomic and hypothalamic nuclei mediate preparatory and consummatory CRs, whereas CeN control of afferents to the ventral striatum and direct BLA projections to the nucleus accumbens mediate the influence of Pavlovian CSs on instrumental actions in general [i.e. conditioned emotional responses (CERs)] and specific transfer paradigms. Overlapping inputs between CeN and BLA in the anterolateral bed nucleus of the stria terminalis are hypothesized to mediate the facilitatory effects of preparatory CSs on the performance of consummatory CRs, whereas CeN inputs to midbrain nuclei seem to control CS-specific CRs. These relationships are argued to be common to both appetitive and aversive conditioning paradigms.

3 ARTICLE IN PRESS 3 Box 1. Pavlovian instrumental transfer Addictive behavior readily illustrates the principle that rewardrelated cues exert a powerful excitatory influence on the performance of actions in animals and humans [59]. In fact, the first formal demonstration of this influence was reported in an early study by Estes [60]. He found that a tone paired with food increased the likelihood that rats would press a lever that had been previously reinforced with that food reward, even though the rats had not been trained to press the lever in the presence of the tone. As Estes s study makes clear, there are three components to experiments that assess effects of this kind: a Pavlovian phase, in which the signals for reward are established; an instrumental phase, in which the association between the reward and actions leading to it is learned; and a test phase, in which the impact of the signals for reward on the performance of instrumental actions is assessed. For this reason, the protocol for assessing the influence of reward-related cues on instrumental performance is often referred to as Pavlovian instrumental transfer (or PIT). Evidence suggests that PIT can be mediated by the representation of the specific sensory features of the Pavlovian reinforcer. For example in hungry rats, a CS paired with food pellets increases the performance of actions that the animals have learned to associate with the pellets, and a CS paired with sucrose increases the performance of actions the animals have learned to associate with sucrose even though both CSs predict a nutritive commodity [12]. PIT effects can also be produced by a general motivational influence of a CS rather than the specific sensory features of the reinforcer. A stimulus paired with food will increase the performance of lever pressing in hungry rats even if they were trained to press the lever for water when they were thirsty [12]. However, this motivational modulation seems to be specific to CSs that elicit appetitive arousal; CSs paired with food tend generally to increase food-related actions whereas CSs paired with aversive events result in the suppression of those actions [12]. that US-specific PIT is mediated by the BLA [19,20] and, indeed, recent evidence suggests that this function is dissociable from that of the CeN. Thus, whereas CeN lesions had no effect on US-specific PIT, BLA lesions completely abolished it [20]. Previous experiments examining the role of the amygdala in PIT had reported that lesions of the CeN, rather than of the BLA, abolished the general excitatory effect of CSs associated with food USs [21 23]. One crucial difference between these and the more recent studies is their design: whereas BLA lesions were shown to affect performance using a US-specific PIT protocol, the effects of CeN lesions were demonstrated using a protocol in which performance could be influenced by the general affective and motivational effects of food-related cues on instrumental performance. Therefore, these differences in the roles of the CeN and BLA in PIT could be reconciled if the US-specific effect were mediated by the BLA and the more general motivational effect by the CeN. To assess this claim directly, a procedure was developed in which both the US-specific and the general forms of PIT could be generated in the same animal [20]. In this case, three CSs were paired with three different food rewards: two with food rewards that could also be earned by performing distinct lever-press actions, and one with a food reward that was not available as an instrumental outcome. The CSs paired with foods also earned by the actions generated a US-specific PIT effect: each elevated performance specifically of the action that had earned the food paired with that CS and not the other action. By contrast, the third CS generated a general effect, elevating performance on both actions. Using this procedure, the effects of BLA and CeN lesions on the general and specific forms of PIT were compared, and it was shown that lesions of the BLA abolished US-specific but not general PIT, whereas lesions of the CeN abolished general but not US-specific PIT [20]. This finding confirmed the explanation proposed for the different effects of BLA and CeN lesions on PIT, and also provided evidence of a structural basis for the distinct consummatory and preparatory conditioning processes proposed by Konorski. In essence, the results of this study suggest that the BLA is involved in the formation of Pavlovian incentives involving the association of a CS with the specific sensory features of the US. By contrast, the CeN is involved in preparatory conditioning that is, in the association of a CS with the general affective properties of the US. Finally, this double dissociation provides evidence that the BLA and CeN can function independently in appetitive conditioning, in line with other reported dissociations between these structures [21 23]. Aversive incentives Perhaps the most thoroughly researched function of the amygdala is its role in fear conditioning, particularly as evaluated by freezing or the facilitation of unconditioned reflexes such as startle [3,5,24,25]. This research suggests that these defensive responses are produced by the association of CS-related inputs from primary sensory relays with US-related nociceptive inputs from the insular cortex and thalamic relays that project to the lateral nucleus of the BLA. Aspects of other aversive conditioning paradigms, notably those that are amenable to training of US-specific CRs (e.g. CS eyeshock associations and conditioned punishment), are also mediated by the BLA [26 28]. Similarly, because conditioned freezing is likely to be controlled by an association between the CS and the nociceptive sensory properties of the aversive US, it might be best characterized as a US-specific defensive reflex paralleling US-specific consummatory reflexes such as chewing, licking and swallowing. (Indeed, Bolles [29] described the marked differences in the responses produced by constant-current and constant-voltage shock sources, emphasizing the dependence of these distinct fear responses on the specific properties of the US.) However, in marked contrast to the dissociations reported in appetitive conditioning, contemporary students of fear conditioning have argued that, beyond the lateral nucleus, the BLA and CeN work serially in aversive situations to determine the aversive CR. According to the standard view, the BLA mediates the CS US association whereas the CeN mediates consequent performance of the defensive CR [3,5,24,25]. Recent evidence poses substantial problems for this serial theory. First, anatomical evidence suggests that the crucial site of plasticity in the lateral nucleus is not connected directly with the area of the CeN that controls important aspects of the CR [30]. Second, several studies have shown that BLA and CeN can mediate distinct fear responses. For example, BLA damage can abolish conditioned fear responses to a discrete cue but leave intact

4 4 ARTICLE IN PRESS aversive learning about contextual cues, as assessed by a place-preference measure [31]; it can also impair conditioned freezing without markedly impairing avoidance of the shocked side of a Y-maze [32]. Similarly, although lesions of the BLA reduce freezing, they are without substantial influence on conditioned suppression of lever pressing. This measure has much in common with general PIT effects, but it evaluates the suppression of ongoing lever pressing for food produced by a CS paired with shock, rather than facilitated responding produced by an appetitive CS [27]. Despite suggestions to the contrary [4,33], detailed analysis has shown that this dissociation is not due to the number or duration of suppression-training trials [13]. Nevertheless, although conditioned suppression does not necessarily reflect freezing (lesions of the ventral periaqueductal gray that abolish freezing have no effect on conditioned suppression [34]), in some procedures the observed suppression might be due in part to freezing. Indeed, evidence suggests that, when very few conditioning trials are used, BLA lesions can simultaneously affect correlated measures of freezing and conditioned suppression [33]. Here suppression is likely to reflect freezing rather than aversive PIT. When conditioned suppression reflects aversive PIT rather than freezing, BLA lesions affect freezing without affecting suppression [27]. Interestingly, and paralleling effects on appetitive general PIT, lesions of the CeN do abolish conditioned suppression, contributing to a double dissociation between BLA and CeN function in mediating conditioned punishment and suppression, respectively [27]. Furthermore, freezing can be generated in rats that have BLA lesions after overtraining of the CS shock association, and this non-bla-mediated fear learning has been argued to depend on the CeN [35]. Indeed, recent data suggest that the CeN is not simply a source of efferents descending to the brainstem but is itself a target both of sensory afferents involving the CS [36] and of nociceptive inputs from the parabrachial nucleus [37]. Clearly, such data question the necessity of the serial model of BLA and CeN involvement in aversive conditioning and have led some researchers to entertain the possibility that these systems contribute to fear learning in parallel, much as was argued earlier in this article for appetitive conditioning [5,30]. In fact, what evidence there is points to substantial similarities in function between the BLA and CeN in appetitive and aversive situations: in both cases, the BLA appears to mediate US-specific (in this case nociceptive) associations of the CS, whereas the CeN mediates associations with the more general affective (in this case aversive) aspects of the US. Consummatory preparatory interactions A further area of similarity in appetitive and aversive Pavlovian conditioning is the influence that general affective states have on the performance of US-specific consummatory CRs. Konorski [14] proposed that conditioned emotional states facilitate the performance of US-specific CRs and more recent evidence has largely confirmed this suggestion [38]. It has also been established that the amygdala, and in particular the CeN, has a central role in this effect both in appetitive conditioning, where it mediates attentional processing of the CS [39 41], and in aversive conditioning. The role in aversive conditioning is most obvious in the potentiation of startle responses by CSs associated with shock [42,43], but it is also found in eyeblink conditioning. In the latter case, although lesions of the interpositus nucleus in the cerebellum abolish acquisition of the eyeblink CR, they do not affect emotional responses to the CS such as changes in heart rate and ultrasonic vocalization (USV) [44,45]. Conversely, lesions of the amygdala abolish USVs and significantly delay acquisition and attenuate performance of the eyeblink CR even though the CR is ultimately acquired [44]. Therefore, in addition to functioning in parallel, consummatory and preparatory processes subserved by the BLA and CeN must also interact at a site of major convergence, and one area that has been implicated in this regard is the anterolateral region of the bed nucleus of the stria terminalis [46,47]. To the extent that US-specific CRs are potentiated by CeN-mediated emotional states, CeN lesions should produce a deficit in CRs that are otherwise controlled by the BLA. In support of this prediction, it has been widely reported that lesions of the CeN produce deficits in conditioned freezing [48,49], an effect that recovers with continued training [33]. Although the observation that both BLA and CeN lesions disrupt freezing has often been interpreted as evidence for the serial model [49], it is interesting to note that it is no less consistent with the parallel incentive processing approach (Figure 1). Reward value: instrumental incentive learning The influence of Pavlovian incentives on the performance of goal-directed actions in various PIT and conditionedsuppression studies has been dissociated, both behaviorally and neurally, from the incentive processes that establish the reward value of a goal or outcome earned as a consequence of instrumental performance [10,11]. For example, after hungry rats have been trained to press a lever to obtain food, post-training shifts, such as from hunger to satiety, directly reduce the potency of Pavlovian incentives. By contrast, in instrumental conditioning, such shifts in motivational state often have little direct effect on the reward value of a food outcome until the effect of the shift on the incentive value of that food event has been made explicit through consummatory experience (i.e. through instrumental incentive learning) [50,51] (Figure 2). With regard to instrumental responding for food, states such as hunger and satiety thus affect performance not because they affect energetic conditions such as drive, but because they affect the incentive value of nutritive outcomes (Box 2). Primary reward Considerable evidence suggests that the representation of the reward value of food is mediated by chemosensory processing involving the gustatory cortex [52]. However, the gustatory cortex is not involved in encoding changes in incentive value. This requires the memory of the sensory properties of food, which involves the gustatory cortex, to be integrated with an affective signal mediated by a further component of the incentive system. Thus, changes

5 ARTICLE IN PRESS 5 (a) (c) Sensory features Motiv. system Drive Stimulus Response (Habit based) Emotional feedback Incent. learning Action outcome association, goal-directed actions General affective signal Reinforcement signal (b) Peripheral metabolic and humoral visceral signals (d) Sensory cortices Hypothalamic and medial thalamic nuclei Sensorimotor cortex Dorsolateral striatum, ventromedial prefrontal cortices Globus pallidus, motor cortex Insular and prefrontal cortices BLA Medial prefrontal cortex, dorsomedial striatum and accumbens core CeN SNr Box 2. Instrumental incentive learning The motivational state of animals is a major determinant of their instrumental performance; not surprisingly, hungry rats work more vigorously for a food reward than do sated ones. However, current evidence suggests that this increase in vigor occurs because food deprivation induces rats to assign a higher incentive value to nutritive outcomes when they are contacted in that state, and that this value then causes the more vigorous performance [61]. In fact, the role of incentive learning in instrumental performance, and the incentive value that it derives, is common across all motivational systems so far investigated. For example, incentive learning has been found to influence performance after: (i) sensory-specific satiety; (ii) shifts in food deprivation; (iii) shifts in water deprivation; (iv) changes in outcome value mediated by drug states; (iv) changes in the value of thermoregulatory and sexual rewards; and (v) outcome devaluation induced by taste aversion learning (reviewed in [12,50,61]). In all of these cases, it is clear that animals have to learn about changes in the incentive value of an instrumental outcome through consummatory contact with that outcome before this change affects performance of their goal-directed, instrumental actions [50]. Instrumental incentive learning involves two interacting associative processes (Figure 2). The first derives from evaluative conditioning and involves association of the representation of salient sensory features of the instrumental outcome with motivational systems that are sensitive to the detection of signals induced by factors such as nutrients, fluids or toxins. This connection opens a feedback loop to provide the basis for the second associative connection, between the outcome representation and emotional feedback (this is produced by the affective system through its activation by motivational inputs, and determines the value of the instrumental outcome). It is the formation of this second associative connection that constitutes instrumental incentive learning [10,12,14]. TRENDS in Neurosciences Figure 2. Role of the amygdala in instrumental incentive learning. (a) Current model of reward processes in instrumental conditioning based on two processes: one involving association between the sensory features of the instrumental outcome and specific motivational processes, and the other involving association between those sensory features and emotional feedback produced by motivational activity [10,50]. The strength of the incentive learning connection is determined by the degree of emotional feedback modulated by drive processes such as hunger and thirst. Instrumental incentive learning can control the direction and intensity of instrumental performance both directly and through integration with processes that mediate action outcome learning. Arrows represent excitatory connections; dots represent modulatory connections. (b) The amygdala specifically the BLA is in a position to establish the reward value of the specific goals or outcomes of goaldirected instrumental actions, through the association of cortical sensory inputs with motivational and emotional inputs to the BLA from the hypothalamus, thalamus, prefrontal cortex and insular cortex. BLA efferents to the medial prefrontal cortex and dorsomedial striatal circuitry, which are implicated in instrumental learning, can influence the acquisition of goal-directed actions and the BLA can influence instrumental performance directly through connections with the nucleus accumbens core [9]. (c) A general model of stimulus response learning. In this model, the motivational basis for the reinforcement signal that mediates the acquisition of habitual, as opposed to goal-directed, actions comes from the affective system. This signal acts as a catalyst to strengthen the connection between sensory and motor representations [11]. (d) A possible systems-level implementation of the stimulus response model. The affective signal mediated by the CeN controls dopaminergic afferents from the substantia nigra pars reticulata to the dorsolateral striatum. The CeN also makes direct projections to ventromedial prefrontal cortices (particularly the medial orbitofrontal and infralimbic areas), which have been implicated in the motivational control of habitual actions [62,63]. in the value of the sensory features of nutritive outcomes seem to be a function of emotional feedback that is, of the emotional response experienced contiguously with detection of the taste [10]. If the emotional response is pleasant, the value of the outcome is correspondingly increased, whereas if it is unpleasant the value is reduced (Figure 2). The gustatory cortex has strong reciprocal connections with the BLA, and two recent series of experiments have provided clear evidence for the involvement of the BLA in incentive learning. In one series, lesions of the BLA rendered the instrumental performance of rats insensitive to devaluation of the instrumental outcome by sensoryspecific satiety; they appeared no longer able to associate the sensory features of the instrumental outcome with its incentive value [53]. Similarly, although the acquisition of concurrent instrumental discriminations benefits from the use of different outcomes in each case, BLA lesions abolish this advantage [19]. Furthermore, BLA-lesioned animals appear unable to use the specific sensory features of the instrumental outcome as a discriminative cue to control choice performance [53]. Because animals with BLA lesions are not impaired in normal discrimination performance or sensory preconditioning [54], these findings suggest that they have a specific deficit representing the sensory features of motivationally significant events. More recently, the role of the BLA in instrumental incentive learning was confirmed using post-training infusions of the protein-synthesis inhibitor anisomycin [55]. It is well documented that the consolidation of stimulus affect associations that underlie conditioned freezing and the reconsolidation of these associations after retrieval both depend on the synthesis of new proteins in the BLA [56]. To assess whether the same is true of instrumental incentive learning, hungry rats were trained to press two levers, one earning food pellets and the other a sucrose solution. After this training, the rats were sated and given the opportunity for incentive learning that is, they were allowed to consume either the food pellets or the sucrose solution in the sated state.

6 6 ARTICLE IN PRESS Box 3. Outstanding questions (i) Can the functional disconnection of the BLA and CeN by asymmetrical lesions of these structures provide evidence for serial processes in either appetitive or aversive Pavlovian conditioning? (ii) Do BLA and CeN lesions affect CS-elicited eating by affecting sensory-specific or general affective influences on feeding, in parallel with the effects of these lesions on PIT? Does either structure mediate the effects on eating produced by autonomic arousal, such as that elicited by tail-pinch? (iii) What can disconnection studies involving the BLA and CeN reveal about their important output projections to the nucleus accumbens core and shell in PIT? (iv) What is the role of CeN midbrain connections in general affective PIT? (v) Might CeN lesions influence the acquisition and/or performance of habitual instrumental actions? (vi) What is the role in instrumental incentive learning of the connectivity of the BLA, including afferent sensory cortical and efferent prefrontal and striatal connections? Do these latter connections mediate action goal associations or estimates of goal value? Immediately after this consumption phase, half of the rats were given an infusion of anisomycin and the remainder received an infusion of vehicle into the BLA. A subsequent two-lever choice extinction test conducted when the rats were sated found that the rats in the vehicle group performed fewer responses on the lever that, in training, delivered the outcome to which they were exposed when sated prior to the test. Infusion of anisomycin into the BLA completely blocked this shift in preference. A similar logic revealed that incentive learning was also subject to reconsolidation involving the BLA, again revealing strong parallels between appetitive and aversive functions of the BLA [55]. Secondary reward and punishment Studies of conditioned reinforcement (and its aversive counterpart, conditioned punishment) enable assessment of the role of the amygdala in instrumental performance for secondary rather than primary reward. Such studies demonstrate further parallels between appetitive and aversive domains, in addition to parallel functions of the BLA and CeN. Lesions of the BLA attenuate the acquisition of a new instrumental response in conditioned reinforcement procedures [57]. By contrast, lesions of the CeN have no effect on conditioned reinforcement of lever presses, but do attenuate the response-enhancing effect of intra-accumbens infusions of amphetamine. Thus, once again, evidence suggests that the CeN modulates the arousing and energizing properties of Pavlovian cues, albeit in this instance in a response-contingent manner [58]. In direct parallel, lesions of the BLA abolish conditioned punishment, but CeN lesions are without effect in this particular aversively motivated task [27]. It is likely that the role of the BLA and CeN in these higherorder tasks depends on multiple associative processes. Undoubtedly, the reward-related stimulus in conditioned reinforcement studies is to some extent an instrumental incentive for responding. At the same time, the affective response elicited by this stimulus might also provide a reinforcement signal to strengthen stimulus response associations that control instrumental performance [9]. We suggest that the function of the reward-related stimulus as an instrumental incentive is mediated by the BLA, whereas its function as a reinforcement signal is mediated by the CeN. Certainly BLA lesions leave outcome-independent instrumental performance intact [53], which is consistent with the proposition that the BLA and CeN operate in parallel to mediate dissociable aspects of Pavlovian and instrumental incentive learning. Thus, the BLA establishes the incentive value of specific instrumental outcomes whereas the CeN mediates the general affective properties of instrumental reinforcers (Figure 2). Concluding remarks Although many details are still to be determined (Box 3), the evidence reviewed here suggests that two fundamental conclusions can be drawn. First, in line with the division first proposed in Konorski s incentive theory [14], it seems that the BLA is involved in encoding the incentive properties of Pavlovian USs and instrumental outcomes on the basis of their specific sensory features, and so has a direct role in consummatory conditioning; by contrast, the CeN mediates the influence of the general affective and reinforcing properties of these events and so seems to be more directly involved in preparatory conditioning. Second, and despite a long tradition of theorizing to the contrary, it does not seem to be necessary, nor is there any direct evidence to suggest, that the BLA and CeN must function serially. Rather, it seems likely that they function simultaneously and in parallel to encode associations of conditioned stimuli and instrumental actions with the sensory-specific and general affective incentive properties of both Pavlovian USs and instrumental outcomes. Although interactions between these parallel processes can occur, they do not necessitate a serial processing view. Therefore, we conclude that the evidence reviewed here is consistent with the suggestion that the BLA and CeN have a common function in both the appetitive and aversive domain. It has commonly been thought that the BLA and CeN function serially in aversive conditioning but in parallel in appetitive conditioning; however, we believe that the proposal that these structures function independently and in parallel in both appetitive and aversive situations is not only more parsimonious, but also more consistent with the available evidence. Acknowledgements Preparation of this manuscript was supported by the National Institute of Health, grant #56446 to B.W.B. and a UK MRC Career Establishment Grant to S.K. (G ). References 1 McGaugh, J.L. (2004) The amygdala modulates the consolidation of memories of emotionally arousing experiences. Annu. Rev. Neurosci. 27, Cardinal, R.N. et al. (2002) Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci. Biobehav. Rev. 26, Fanselow, M.S. and Gale, G.D. (2003) The amygdala, fear, and memory. Ann. N. Y. Acad. Sci. 985, LeDoux, J.E. (2000) The amygdala and emotion: A view through fear. In The Amygdala: A functional analysis (2nd edn) (Aggleton, J.P., ed.), pp , Oxford University Press

7 ARTICLE IN PRESS 7 5 Maren, S. (2005) Building and burying fear memories in the brain. Neuroscientist 11, Swanson, L.W. (2003) The amygdala and its place in the cerebral hemisphere. Ann. N. Y. Acad. Sci. 985, Swanson, L.W. and Petrovich, G.D. (1998) What is the amygdala? Trends Neurosci. 21, Swanson, L.W. (2000) Cerebral hemisphere regulation of motivated behavior. Brain Res. 886, Balleine, B.W. (2005) Neural; bases of food seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol. Behav. 86, Balleine, B.W. (2004) Incentive behavior. In The Behavior of the Laboratory Rat: a Handbook with Tests (Whishaw, I.Q. and Kolb, B., eds), pp , Oxford University Press 11 Dayan, P. and Balleine, B.W. (2002) Reward, motivation, and reinforcement learning. Neuron 36, Dickinson, A. and Balleine, B.W. (2002) The role of learning in the operation of motivational systems. In Learning, Motivation & Emotion, Volume 3 of Steven s Handbook of Experimental Psychology (3rd edn) (Gallistel, C.R., ed.), pp , John Wiley & Sons 13 Killcross, S. and Blundell, P. (2002) Associative representations of emotionally significant outcomes. In Emotional Cognition: From Brain to Behaviour. Advances in Consciousness Research (Moore, S. and Oaksford, M., eds), pp , Benjamins Publishing Company 14 Konorski, J. (1967) Integrative Activity of the Brain, University of Chicago Press 15 Holland, P.C. and Gallagher, M. (1999) Amygdala circuitry in attentional and representational processes. Trends Cogn. Sci. 3, Hatfield, T. et al. (1996) Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci. 16, Holland, P.C. et al. (2002) The effects of amygdala lesions on conditioned stimulus-potentiated eating in rats. Physiol. Behav. 76, Petrovich, G.D. et al. (2005) Amygdalar and prefrontal pathways to the lateral hypothalamus are activated by a learned cue that stimulates eating. J. Neurosci. 25, Blundell, P. et al. (2001) Lesions of the basolateral amygdala disrupt selective aspects of reinforcer representation in rats. J. Neurosci. 21, Corbit, L.H. and Balleine, B.W. (2005) Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian instrumental transfer. J. Neurosci. 25, Hall, J. et al. (2001) Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. Eur. J. Neurosci. 13, Holland, P.C. and Gallagher, M. (2003) Double dissociation of the effects of lesions of basolateral and central amygdala on conditioned stimulus-potentiated feeding and Pavlovian instrumental transfer. Eur. J. Neurosci. 17, Everitt, B.J. et al. (2003) Appetitive behavior: impact of amygdaladependent mechanisms of emotional learning. Ann. N. Y. Acad. Sci. 985, LeDoux, J.E. (2000) Emotion circuits in the brain. Annu. Rev. Neurosci. 23, Davis, M. (1998) Anatomic and physiologic substrates of emotion in an animal model. J. Clin. Neurophysiol. 15, Blair, H.T. et al. (2005) The lateral amygdala processes the value of conditioned and unconditioned aversive stimuli. Neuroscience 133, Killcross, S. et al. (1997) Different types of fear-conditioned behaviour mediated by separate nuclei within amygdala. Nature 388, Blair, H.T. et al. (2005) Unilateral storage of fear memories by the amygdala. J. Neurosci. 25, Bolles, R.C. (1967) Theory of Motivation, Harper & Row 30 Pare, D. et al. (2004) New vistas on amygdala networks in conditioned fear. J. Neurophysiol. 92, Selden, N.R.W. et al. (1991) Complementary roles for the amygdala and hippocampus in aversive conditioning to explicit and contextual cues. Neuroscience 42, Vazdarjanova, A. and McGaugh, J.L. (1998) Basolateral amygdala is not critical for cognitive memory of contextual fear conditioning. Proc. Natl. Acad. Sci. U. S. A. 95, Lee, J.L.C. et al. (2005) Conditioned suppression and freezing as measures of aversive Pavlovian conditioning: effects of discrete amygdala lesions and overtraining. Behav. Brain Res. 159, Amorapanth, P.et al. (1999) Lesions of periaqueductal gray dissociateconditioned freezing from conditioned suppression behavior in rats. Learn. Mem. 6, Maren, S. (1999) Neurotoxic basolateral amygdala lesions impair learning and memory but not the performance of conditional fear in rats. J. Neurosci. 19, Linke, R. et al. (2000) Differential projection of the posterior paralaminar thalamic nuclei to the amygdaloid complex in the rat. Exp. Brain Res. 134, Shi, C. and Davis, M. (1999) Pain pathways involved in fear conditioning measured with fear-potentiated startle: lesion studies. J. Neurosci. 19, Bombace, J.C. et al. (1991) Modulation of a conditioned eyeblink response by a putative emotive stimulus conditioned with hindleg shock. J. Exp. Psychol. Anim. Behav. Process. 17, Han, J.S. et al. (1999) Disconnection of the amygdala central nucleus and substantia innominata/nucleus basalis disrupts increments in conditioned stimulus processing in rats. Behav. Neurosci. 113, Gallagher, M. and Holland, P.C. (1994) The amygdala complex: multiple roles in associative learning and attention. Proc. Natl. Acad. Sci. U. S. A. 91, Lee, H.J. et al. (2005) Role of amygdalo nigral circuitry in conditioning of a visual stimulus paired with food. J. Neurosci. 25, Kim, M. and Davis, M. (1993) Electrolytic lesions of the amygdala block acquisition and expression of fear-potentiated startle even with extensive training but do not prevent reacquisition. Behav. Neurosci. 107, Campeau, S. and Davis, M. (1995) Involvement of the central nucleus and basolateral complex of the amygdala in fear conditioning measured with fear-potentiated startle in rats trained concurrently with auditory and visual conditioned stimuli. J. Neurosci. 15, Lee, T. and Kim, J.J. (2004) Differential effects of cerebellar, amygdalar, and hippocampal lesions on classical eyeblink conditioning in rats. J. Neurosci. 24, Lavond, D.G. et al. (1984) Effect of bilateral lesions of the dentate and interpositus cerebellar nuclei on conditioning of heart-rate and nictitating membrane/eyelid responses in the rabbit. Brain Res. 305, Dong, H.W. et al. (2001) Topography of projections from amygdala to bed nuclei of the stria terminalis. Brain Res. Rev. 38, Davis, M. and Shi, C. (1999) The extended amygdala: are the central nucleus of the amygdala and the bed nucleus of the stria terminalis differentially involved in fear versus anxiety? Ann. N. Y. Acad. Sci. 877, Amorapanth, P. et al. (2000) Different lateral amygdala outputs mediate reactions and actions elicited by a fear-arousing stimulus. Nat. Neurosci. 3, Goosens, K.A. and Maren, S. (2001) Contextual and auditory fear conditioning are mediated by the lateral, basal, and central amygdaloid nuclei in rats. Learn. Mem. 8, Balleine, B.W. (2001) Incentive processes in instrumental conditioning. In Handbook of Contemporary Learning Theories (Klein, R.M.S., ed.), pp , LEA 51 Corbit, L.H. and Balleine, B.W. (2003) Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain. J. Exp. Psychol. Anim. Behav. Process. 29, Balleine, B.W. and Dickinson, A. (2000) The effect of lesions of the insular cortex on instrumental conditioning: evidence for a role in incentive memory. J. Neurosci. 20, Balleine, B.W. et al. (2003) The effect of lesions of the basolateral amygdala on instrumental conditioning. J. Neurosci. 23, Blundell, P. et al. (2003) Preserved sensitivity to outcome value after lesions of the basolateral amygdala. J. Neurosci. 23,

8 8 ARTICLE IN PRESS 55 Wang, S.H. et al. (2005) Consolidation and reconsolidation of incentive learning in the amygdala. J. Neurosci. 25, Nader, K. et al. (2000) The labile nature of consolidation theory. Nat. Rev. Neurosci. 1, Burns, L. et al. (1993) Differential effects of excitotoxic lesions of the basolateral amygdala, ventral subiculum and medial prefrontal cortex on responding with conditioned reinforcement and locomotor activity potentiated by intraaccumbens infusions of D-amphetamine. Behav. Brain Res. 55, Robledo, P. et al. (1996) Effects of excitotoxic lesions of the central amygdaloid nucleus on the potentiation of rewardrelated stimuli by intra-accumbens amphetamine. Behav. Neurosci. 110, Everitt, B.J. et al. (1999) Associative processes in addiction and reward. The role of amygdala ventral striatal subsystems. Ann. N. Y. Acad. Sci. 877, Estes, W.K. (1948) Discriminative conditioning II: Effects of a Pavlovian conditioned stimulus upon a subsequently established operant response. J. Exp. Psych 38, Dickinson, A.B. and Balleine, B.W. (1994) Motivational control of goaldirected action. Anim. Learn. Behav. 22, Yin, H.H. et al. (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, Killcross, S. and Coutureau, E. (2003) Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb. Cortex 13,