Toward a Unified Model of Attention in Associative Learning

Transcription

1 Journal of Mathematcal Psychology 45, (2001) do: jmps , avalable onlne at on Toward a Unfed Model of Attenton n Assocatve Learnng John K. Kruschke Indana Unversty Two connectonst models of attenton n assocatve learnng, prevously used to model human category learnng, are shown to have specal cases that are essentally equvalent to N. J. Mackntosh's (1975, Psychologcal Revew, 82, ) classc model of attenton n anmal learnng. The models unfy formulas for assocatve weght change wth formulas for attentonal change, under a common goal of error reducton. Error-drven attentonal shftng accelerates learnng of new assocatons but also protects prevously learned assocatons from retroactve nterference. The models are ft to data from a recent experment n human assocatve learnng (J. K. Kruschke 6 N. J. Blar, 2000, Psychonomc Bulletn 6 Revew, 7, ), whch shows that blockng of learnng nvolves learned nattenton. The approach also provdes a novel and unfyng theory of latent nhbton (the preexposure effect) n terms of blockng. The dscusson summarzes how the approach accounts for a varety of other ``rratonal'' phenomena n assocatve learnng, ncludng base rate effects, perseveraton of attenton through relevance shfts, overshadowng, and the extrapolaton of rules near exceptons Elsever Scence The central role of attenton n learnng has been emphaszed repeatedly n accounts of both human and anmal learnng. At least as early as Lawrence (1949, 1950), theorsts of anmal learnng have argued that anmals learn whch cues should be attended to. One type of phenomenon addressed by such attentonal learnng theores s transfer of learnng: When subsequent learnng nvolves cues prevously learned to be relevant, the speed of learnng s faster than when the Ths research was supported n part by NIMH FIRST Award 1-R29-MH For comments on prevous versons of ths artcle, I thank Nathanel Blar, Mchael Erckson, Mchael Fragass, Mark Johansen, Peter Klleen, Ncholas Mackntosh, John Pearce, Roger Ratclff, Teresa Treat, and Peter Wood. Parts of ths research were presented at the Eghth Australasan Mathematcal Psychology Conference, Perth Australa, 29 November 1997; the 31st Annual Conference of the Socety for Mathematcal Psychology, Vanderblt Unversty, Nashvlle TN, 8 August 1998; the Twenteth Annual Conference of the Cogntve Scence Socety, Unversty of Wsconsn at Madson, 3 August 1998; and the Economc and Socal Research Councl Semnar on Knowledge, Concepts, and Categores, Unversty College London, UK, 12 August Address correspondence and reprnt requests to John K. Kruschke, Department of Psychology, 1101 E. 10th St., Indana Unversty, Bloomngton, IN E-mal: kruschkendana.edu. URL: http: Elsever Scence All rghts reserved. 812

2 ATTENTION IN LEARNING 813 new learnng nvolves cues prevously learned to be rrelevant. Typcally n these theores, the attenton to a cue modulates the cue's nfluence on the anmal's mmedate response. Importantly, moreover, attenton also modulates the cue's utlzaton n assocatve learnng, and therefore the attenton expresses the cue's assocablty. The well-known Rescorla and Wagner (1972) model of assocatve learnng formalzed the dea (e.g., Kamn, 1969) that assocatons are learned between cues and surprsng outcomes. The model acknowledged that dfferent cues mght be attended to dfferently, and therefore each cue was allowed a dfferent learnng rate to express the cue's ndvdual assocablty. Crucally, however, Rescorla and Wagner (1972) provded no theory or formula descrbng how cue-specfc learnng rates should be adjusted by experence. In a classc paper, Mackntosh (1975) proposed specfc formulas expressng the dea that attenton to cues that have been learned to be relevant should ncrease, but attenton to cues that have been learned to be rrelevant should decrease. ``In Mackntosh's model, however, surprse does not act va a comparable dscrepancy [as n the RescorlaWagner model] and ts role wthn [the formula for attenton change] s not readly nterpretable n terms of processng mechansms... [The formula for attenton change] can probably receve a psychologcal nterpretaton wthn a dfferent framework'' (Dcknson, 1980, p. 153). One goal of the present artcle s to provde a framework wheren Mackntosh's (1975) formulas for attenton learnng and for assocaton learnng derve from the same motvaton, gradent descent on error. In the human learnng lterature, attenton was for many years at the core of theores of concept learnng. Accordng to many theores of concept learnng, people learn what stmulus dmenson to attend to, and then people learn what the correspondence s from the features of that dmenson to the concept label. Trabasso and Bower (1968), for example, n ther book enttled Attenton n Learnng, provde a hstory and overvew of models and emprcal work. More recently, the dea of attenton has played an mportant role n theores of category learnng by humans. Buldng on work by Shepard, Hovland, and Jenkns (1961) and by Medn and Schaffer (1978), Nosofsky's (1986) generalzed context model (GCM) suggested that people dstrbute attenton to dmensons such that ntercategory dfferences and ntracategory smlartes are maxmzed. The GCM had no mechansm by whch those dmensonal attenton values were learned, however. Such a mechansm was provded by Kruschke's (1992) ALCOVE model, a connectonst mplementaton of the GCM n whch dmensonal attenton strengths are gradually learned across trals by gradent descent on error. Kruschke (1996b) showed that a varant of ALCOVE can address transfer of learnng across relevance shfts. Erckson and Kruschke (1998, see also Kruschke 6 Erckson, 1994) descrbed an expanded connectonst archtecture, called ATRIUM, that combnes the exemplar representaton n ALCOVE wth rule-lke representatons, together wth a mechansm that gradually learns to attend to one representaton or the other, dependng on the stmulus. Kruschke (1996a) ntroduced rapd attenton shfts n a connectonst model called ADIT to account for perplexng base rate effects. These rapd attenton shfts were used n an extenson of the ALCOVE model, called

3 814 JOHN K. KRUSCHKE RASHNL, by Kruschke and Johansen (1999) to account for a wde spectrum of results n probablstc category learnng. A second goal of the present artcle s to show that these connectonst archtectures of attenton learnng n humans (Erckson 6 Kruschke, 1998; Kruschke, 1996a; Kruschke 6 Johansen, 1999) have specal cases that are essentally equvalent to the model of attenton learnng n anmals proposed by Mackntosh (1975). The equvalence between the connectonst models of human learnng and Mackntosh's model of anmal learnng comes at the cost of abandonng Mackntosh's explanaton of latent nhbton, also known as the preexposure effect, whch s a phenomenon that occurs when an anmal s preexposed to a cue wth no novel outcome, but subsequently the cue s pared wth a consequence. Learnng to assocate the cue wth the consequence s retarded, compared wth anmals that were not preexposed to the cue (Lubow 6 Moore, 1959). Whereas several theores, lke Mackntosh's, have explaned the preexposure effect n terms of learned attenton (e.g., Lubow, 1989; Pearce 6 Hall, 1980; Schmajuk, Lam, 6 Gray, 1996), none of these theores has formulas for attenton change motvated n the same way as formulas for assocaton change. By way of contrast, a thrd goal of the present artcle s to explan the preexposure effect as error reducton, just as assocatve learnng s error reducton. In ths approach, the preexposure effect s treated essentally as a specal case of blockng of assocatve learnng (Kamn, 1969). Recent data (Kruschke 6 Blar, 2000) are summarzed whch demonstrate that the assocablty of a redundant relevant cue s weakened n blockng, just as the assocablty of a preexposed cue s weakened n the preexposure effect. The data are well ft by the models. It must be stated at the outset, however, that ths treatment of the preexposure effect s prelmnary and motvated at ths pont prmarly by theoretcal symmetry. The man emphass of ths artcle s the connectonst models of human attentonal learnng and ther relaton to Mackntosh's (1975) model; the treatment of the preexposure effect s put forward tentatvely. Asde from the three aforementoned goals of ths artcle, another theme s that shfts of attenton durng learnng accomplsh two complmentary effects: New learnng s accelerated and prevous learnng s protected. These dual benefts are accomplshed because attentonal shftng reduces nterference between old and new learnng, and ths reducton of nterference s a natural consequence of error reducton. Outlne of Artcle The next secton descrbes a new connectonst model of assocatve learnng, called EXIT, whch combnes rapdly shftng attenton (Kruschke, 1996a; Kruschke 6 Johansen, 1999) wth exemplar-specfc attentonal learnng. The model s then ft to recent data whch show that blockng of assocatve learnng nvolves learned attenton (Kruschke 6 Blar, 2000). A second connectonst model s then descrbed, based on a mxture of experts archtecture (Erckson 6 Kruschke, 1998; Jacobs, Jordan, Nowlan, 6 Hnton, 1991; Kruschke 6 Erckson, 1994). Ths model also combnes rapdly shftng attenton wth exemplar-specfc attentonal learnng, but treats each cue as a separate

4 ATTENTION IN LEARNING 815 ``expert'' that ndvdually attempts to predct the outcomes, wthout summng predctons from the other cues. Ths model s also ft to the blockng data. Specal cases of the two models are then shown to correspond very closely to the formulas presented by Mackntosh (1975) to account for anmal learnng. Unfortunately, the connectonst models cannot address the preexposure effect n the same way that Mackntosh (1975) suggested, so a new nterpretaton of the preexposure effect s suggested wthn the framework of the connectonst models. The fnal dscusson summarzes the ablty of these types of models to address a number of other phenomena based on attentonal shfts and learnng. These phenomena hghlght varous consequences of attentonal shfts: accelerated learnng when few dmensons are relevant, perseveraton of learned attenton, protecton of prevous learnng n base rate effects, and shfts of attenton between rule-lke and exemplar representatons. EXIT: THE EXTENDED ADIT MODEL An essental aspect of the ADIT model (Kruschke, 1996a) s that each cue s multplcatvely gated by an ndvdual attentonal strength. A cue's attentonal strength modulates (a) the cue's nfluence on mmedate respondng and (b) the cue's assocablty for mmnent learnng. On any gven tral of learnng, the attenton strengths are shfted rapdly n response to error before the assocatve weghts are adjusted. Attenton s shfted away from cues that cause error and toward cues that reduce error. After ths shft, the assocatve weghts from the attended-to cues are adjusted (proportonally to the attenton on each cue). Thus, the assocatve weghts are affected only by cues that are attended to, and the model attends predomnantly to those cues that reduce nterference wth prevously learned knowledge. The attenton shft thereby protects prevously learned assocatons whle acceleratng the learnng of new assocatons. The orgnal ADIT model shfted attenton on each tral n response to error, but the shft was not retaned n subsequent trals. Instead, attenton was reset to default values at the begnnng of each tral. Ths lack of learnng about attenton was not a theoretcal commtment, but was only a convenence to reduce the number of free parameters, because attentonal learnng was not needed to address the emprcal phenomena accompanyng ADIT's orgnal artcle (Kruschke, 1996a). On the contrary, the gradual learnng of attenton has always been an underlyng commtment for the approach (e.g., Erckson 6 Kruschke, 1998; Kruschke, 1992, 1996b; Kruschke 6 Johansen, 1999), and ths commtment wll be honored n the present extenson of ADIT. The extended verson s called EXIT. The name ``EXIT'' s mnemonc n three ways. Frst, t s short for Extended ADIT. Second, t s short for Exemplar-based attenton to dstnctve nput. Thrd, an ADIT s an entrance, and an EXIT s encountered later. Actvaton Propagaton to the Category Nodes Fgure 1 shows the archtecture of EXIT. Each component cue n the stmulus s represented by a correspondng nput node n a connectonst network, and each

5 816 JOHN K. KRUSCHKE FIG. 1. Archtecture of the EXIT model. Ths dagram llustrates a case wth two cues, one exemplar, and two outcomes. (A) The basc connectons from cues to outcomes are shown: Thck arrows denote learnable assocatve weghts, denoted by w k n Eq. (1). (B) The network mechansm for allocatng default, normalzed attenton to cues s shown. The actvaton of the gan nodes s expressed by Eq. (4). The crss-crossng lnes from gan nodes to attenton nodes represent the normalzaton of attenton expressed by Eq. (5). The X's n boxes drectly above the nput cues represent the multplcatve applcaton of the attenton on the cues, as expressed n Eq. (1). (C) The network mechansm for learnng new attentonal dstrbutons s shown. The actvaton of an exemplar node s specfed n Eq. (3). The thck arrows from the exemplar to gan nodes represent learned assocatve weghts, denoted w x n Eq. (4). (D) The complete archtecture, wth the components from panels (A), (B), and (C) supermposed, s shown. possble outcome s represented by a correspondng output node. If cue s present n a stmulus, then node s actvated, wth actvaton value a n =1. When cue s absent, a n =0. When outcome k s present, then the correspondng output node receves a ``teacher'' sgnal t k =1, whch ndcates that the node should be actvated. When the outcome s absent, then t k =0. Input node s connected to output node k va a lnk wth an assocatve strength, or weght, denoted w k. When a stmulus s presented, the correspondng nput nodes are actvated, and actvaton spreads to the output nodes va the weghted connectons. The attentonal strengths also modulate the nfluence of the nput actvatons, such that the output actvaton s determned by a weghted sum across the attentonally gated nput actvatons. Formally, the actvaton of the output node k s determned by a out =: k w k : a n, (1) where : s the attenton strength on the nput node. The source of these attenton values wll be descrbed below. The nput-to-category assocaton weghts are ntalzed at zero, but change wth learnng, as descrbed later. The path of actvaton from cues through assocatve weghts to outcomes s shown n Panel (A) of Fg. 1. Category node actvatons are mapped to response probabltes usng a verson of the Luce (1959) choce rule, also known as the ``softmax'' rule (Brdle, 1990;

6 ATTENTION IN LEARNING 817 Rumelhart, Durbn, Golden, 6 Chauvn, 1995). Specfcally, the probablty of choosng category c s gven by p(c)=exp(,a out c ) <: k exp(,a out k ), (2) where, s a scalng constant. In other words, the probablty of classfyng the gven stmulus nto category c s determned by the magntude of category c's actvaton relatve to the sum of all category actvatons. The constant,,, determnes the decsveness of the network: A large value of, expresses a hghly decsve choce, n that t causes just a small actvaton advantage for category c to be translated nto a large choce preference for category c. A small value of, expresses an ndecsve or unconfdent network, n that the small, causes large actvaton dfferences to be translated nto ambvalent choces. Ths rule for mappng output actvatons to choce probabltes has many precedents n the psychologcal lterature (e.g. Estes, 1988, 1994; Gluck 6 Bower, 1988a; Kruschke, 1992) and n the engneerng lterature (e.g., Brdle, 1990; Rumelhart et al., 1995). An added computatonal beneft beyond the psychologcal plausblty s that exponentaton of the output actvatons monotoncally transforms possbly negatve actvatons nto postve values, whch s essental f the transformed values are nterpreted as probabltes. Appendx 1 dscusses an alternatve mappng from actvatons to choce probabltes, n whch the actvatons are rased to a power nstead of exponentated. Base Rates. The orgnal ADIT model (Kruschke, 1996a, p. 15, Eq. (11)) used a separate formula for mxng category base rates wth the choce probabltes generated from the assocatve network. Appendx 2 shows that ths separate formula s essentally equvalent to handlng base rates as learned assocatons from a bas cue. The bas cue s fully actvated on every tral. In effect, the bas cue encodes the response prompt that appears n every tral durng an experment. It s possble, however, for the bas cue to have a dfferent salence than the other cues (cf. Kruschke 6 Johansen, 1999). It s also possble that the attenton should not be as mutable on the bas cue as on the other cues. Appendx 2 dscusses some ramfcatons of learnng wth a bas cue. In any case, what was presented n the orgnal ADIT model as a separate prncple for mxng base rates wth other choce probabltes s actually equvalent to the sngular attentonal and assocatve learnng system appled to a bas cue. In the experment ftted later n the artcle, the base rates of the categores were all equal (partcularly n the fnal phase of tranng), and so the bas cue s omtted from the model for smplcty. Actvaton Propagaton n the Attentonal System Panels (B) and (C) of Fg. 1 llustrate the attentonal system. Ths attentonal system maps nput actvatons to attenton strengths. An assumpton of the model s that total attentonal capacty s lmted, so that ncreasng attenton to one cue entals reducng attenton to other cues. Ths capacty lmt s mplemented n the

7 818 JOHN K. KRUSCHKE model by assumng that each cue has an underlyng attentonal gan, and these gans are normalzed (n a way to be specfed below) to generate attenton strengths. The normalzaton s ndcated n Panel (B) of Fg. 1 by the crss-crossng connectons from gan nodes to attenton nodes. The archtecture of the attentonal system s desgned to accomplsh two dstnct goals. The frst goal s to mplement the default assumpton that any presented cue should get some attenton. Ths goal s acheved by provdng each gan node wth a ``hard-wred'' connecton from each correspondng cue. These hard-wred one-toone connectons are shown n Panel (B) of Fg. 1 as sold lnes from the cues to the gan nodes. The second goal of the attentonal module s to learn how attenton should be dstrbuted over the stmulus cues as a functon of the partcular combnaton of cues. In prncple, the mappng from stmulus cues to attenton values could be hghly nonlnear, and so the module should be gven adequate computatonal capacty to learn nonlnear mappngs. Perhaps the most straghtforward archtecture for accommodatng nonlnear mappngs nvolves exemplar medaton of the mappng from nput to output. Therefore, the model recruts exemplar nodes whenever a novel stmulus confguraton s encountered, 1 and the connecton weghts from the exemplar nodes to the gan nodes learn to predct the shfted attentonal gans. These adaptve weghts are shown as thck arrows n Panel (C) of Fg. 1. Exemplar-medated mappngs are reasonably motvated psychologcally, as well as computatonally. In everyday lfe, f we learn to gnore a cue n one stuaton, we don't necessarly gnore t n all stuatons. In experments on human category learnng, t has been shown that dmensons gnored for some stmul are not gnored for other stmul (e.g., Aha 6 Goldstone, 1990). Because some nput cues can be context cues, the exemplar nodes n the attentonal module can also encode contextual cues and thereby mplement context specfcty. The actvaton of an exemplar node corresponds to the psychologcal smlarty of the current stmulus to the exemplar represented by the node. Smlarty drops off exponentally wth dstance n psychologcal space, as suggested by Shepard (1987), and dstance s computed usng a cty-block metrc for psychologcally separable dmensons (Garner, 1974; Shepard, 1964). Each exemplar node s sgnfcantly actvated by only a relatvely localzed regon of nput space;.e., t has a small ``receptve feld.'' Formally, the actvaton value of exemplar x s gven by a ex x =exp \ &c : x &a n +, (3) where the superscrpt ``ex'' ndcates that ths s an exemplar node; c s a constant called the specfcty that determnes the overall narrowness of the receptve feld; and x represents the presence or absence of cue n exemplar x, such that x =1 f cue s present n the exemplar and x =0 f cue s absent. Ths s the same 1 Alternatvely, the exemplar nodes could form a random coverng map of the nput space, as n the orgnal ALCOVE model (Kruschke, 1992). Perhaps the best opton would be recrut exemplars n response to error. These optons are not explored here.

8 ATTENTION IN LEARNING 819 exemplar-smlarty functon used n the ALCOVE model (Kruschke, 1992) and n the generalzed context model (Nosofsky, 1986). Wthn the attenton module actvaton propagates from the nput nodes to the gan nodes va two paths: along the prevously descrbed one-to-one connectons from nput nodes to gan nodes shown n Panel (B) of Fg. 1 and va exemplar nodes to gan nodes, shown n Panel (C) of Fg. 1. The actvaton of gan node s gven by g =a n exp \: x w x a ex x +, (4) where w x s the assocatve weght from exemplar node x to gan node. The weghts n Eq. (4) are ntalzed at zero but change to new values wth learnng, as descrbed below. Equaton (4) gves zero gan to nput cues wth zero actvaton, and a gan of 1 to nput cues about whch nothng has been learned yet. Note also that the gans on all cues are nonnegatve. From the gan nodes, actvaton propagates to the attenton nodes. The capacty constrant s formalzed by requrng the length of the attenton vector to be equal to 1, wth length measured by a Mnkowsk power metrc. Formally, ths s denoted as the constrant that : P =1, where P>0 s the value of the power n the Mnkowsk metrc. Then the attenton to the th cue s just the normalzed gan of the th cue, : = g <\ : j 1P g P j. (5) + The denomnator s certan to be greater than zero because the gans computed from Eq. (4) are nonnegatve, and at least one gan s nonzero by desgn. Increased attentonal capacty s reflected by larger values of P. When P=1, the attenton strengths must sum to unty, and the attenton to any one cue s just the proporton of ts gan relatve to the total of the other gans,.e., : = g j g j. In ths case, any ncrease of attenton to one cue comes at the cost of the same amount of decrease n attenton to other cues. When the capacty P approaches nfnty, the attenton to each cue approaches the proporton of ts gan relatve to the maxmal gan of any cue,.e., : = g max j [g j ]. The cue wth maxmal gan gets an attentonal strength of nearly 1, and other cues get attenton proportonal to the maxmal gan. If several cues are ted for maxmal gan, they all get attenton of nearly 1. When 0<P<1, any ncrease n attenton to a cue causes more than that amount of decrease to other cues; n ths case there s severe competton for attenton among cues, and there s relatvely lttle attenton to any cue unless all cues but one have attenton strengths close to zero. Attenton Shftng After actvaton s propagated to the category nodes and the categorzaton probabltes are determned, correctve feedback s suppled, just as n human

9 820 JOHN K. KRUSCHKE learnng experments. The frst response to ths correctve feedback s a rapd shft of attenton to reduce error. Error s measured as the sum squared devaton between the teacher values and the generated actvaton values, across the output nodes;.e., E=.5 : k (t k &a out k )2. (6) The coeffcent.5 appears n Eq. (6) only for convenence n subsequent dervatons. Ths defnton of error s typcal for models that learn by gradent descent on error (Gluck 6 Bower, 1988b; Kruschke, 1992; Rumelhart, Hnton, 6 Wllams, 1986), but other defntons of error are possble (Rumelhart et al., 1995). Attenton s adjusted by gradent descent on error wth respect to the underlyng gans. As a prelmnary step n dervng the gradent, the dervatve of attenton wth respect to gan wll be computed now. In ths and all subsequent formulas, a lower case subscrpt denotes an ndex that can vary, whereas an upper case subscrpt denotes an ndex that has a fxed value. From Eq. (5), we fnd that the dervatve of attenton to some cue wth respect to the gan of specfc cue I s : g I = _\: j 1P 1 g P j } + I & g P \ : j =(} I &: : P&1 I ) <\: j (1P)&1 g P j + \ : j Pg P&1 j } ji+&<_ : j 2P g P j & 1P g P j, (7) + where } I =1 f =I and } I =0 otherwse, whch s sometmes referred to as the Kronecker delta functon of and I. (The tradtonal notaton for the Kronecker delta functon, $ I, s avoded to prevent possble confuson wth the delta rule n connectonst learnng.) Then, applyng the chan rule to Eq. (6), we fnd that gradent descent on error wth respect to gans yelds 2g I =&* g E g I =* g : k =* g : k =* g : k (t k &a out ) : k (t k &a out) : k w k a n (t k &a out )(w k ki an : g I w k a n (} I &: : P&1 I I &:P&1 I a out ) k <\ : j 1P g P j + 1P g P j, (8) + ) <\: j where * g s a postve constant of proportonalty called the shft rate for attenton. Psychologcally, attenton s hypotheszed to shft a large extent on a sngle tral. Ths large shft cannot be acheved formally wth a sngle large step n the drecton of the gradent because attenton s a hghly nonlnear functon of gan; that s, the gradent changes as the attenton changes. Therefore, the change specfed by the equaton for gan change s terated 10 tmes (an arbtrary number) on each tral,

10 ATTENTION IN LEARNING 821 so that the nonlnearty of the functon can be approxmated wth 10 relatvely small steps. After each small attenton change the actvaton s repropagated to the category nodes to generate a new error, and attenton s changed a small amount agan, for 10 teratons. (On any one of these teratons, f a gan value s drven to a negatve value, t s smply reset to 0 before the attenton values are computed.) The result of these 10 small steps consttutes the sngle large shft. The same method was appled n the RASHNL model (Kruschke 6 Johansen, 1999). Learnng of Assocatons After the attenton s shfted the assocaton weghts are adjusted, also by gradent descent on error, 2w KI =&* w E w KI =* w (t K &a out ) : K I an I (9) where * w s a constant of proportonalty called the learnng rate for output weghts. The assocatve weghts for the gan nodes are also adjusted va gradent descent on error, where error s defned as the sum of squared dfferences between the shfted value and the ntal preshft value. That s, the shfted values act as the teachers for the gan node actvatons. Formally, ths yelds 2w g IX =* x (gshft I & g nt I ) g nt I a ex, (10) X where * x s the learnng rate for the assocatve weghts from the exemplar to gan nodes. For nfntesmal shfts, ths change s equvalent to gradent descent on the output error, E. Lst of Free Parameters n EXIT. the followng: The free parameters of the EXIT model are 1. the response probablty scalng constant,, used for convertng output actvaton to response probablty, n Eq. (2); 2. the specfcty c of the exemplar nodes n the attenton module, n Eq. (3). 3. the attenton normalzaton power P,.e., the attentonal capacty, n Eq. (5), 4. the attenton shft rate * g n Eq. (8); 5. the assocatve weght learnng rate * w for categorzaton module, n Eq. (9); and 6. the learnng rate * x for the assocatve weghts from exemplar nodes to gan nodes, n Eq. (10). EXPERIMENT: BLOCKING INVOLVES LEARNED INATTENTION One demonstraton of the mportance of attenton n EXIT s ts ablty to account for attenuated learnng about a prevously blocked cue. Blockng of

11 822 JOHN K. KRUSCHKE assocatve learnng was frst reported by Kamn (1968) and can be descrbed as follows: Consder two cues, A and B, that are always followed by outcome 1. Ths correspondence s denoted AB 1. On average, the two cues each gan some postve assocatve strength wth the outcome. In contrast, when AB 1 s preceded by an earler phase of tranng wthout B (.e., A 1), then the subsequent tranng wth A and B together seems to generate lttle learnng about B. The prevous tranng wth A alone has apparently prevented, or blocked, subsequent learnng about the redundant relevant cue, B. Blockng s a hstorcally crucal fndng because t dsconfrms all models of learnng n whch assocatve strength s ncremented by the mere contguty of cue and outcome. Ths s because Cue B and outcome 1 co-occurred many tmes yet there was apparently lttle assocatve strength bult up. ``No emprcal fndng n the study of anmal learnng has been of greater theoretcal mportance than the phenomenon of blockng'' (Wllams, 1999, p. 618). The domnant explanaton of blockng was suggested by Kamn (1968) and formalzed n the classc model of Rescorla and Wagner (1972). The dea s that assocatve strength changes only to the extent that the outcome s unexpected. The RescorlaWagner model s essentally a specal case of EXIT when the attentonal system s excsed and what remans s the cue-to-outcome assocatons. Output actvaton s defned as n Eq. (1) and weght changes are defned as n Eq. (9). The dscrepancy between teacher value and generated value, t k &a out k n Eq. (9), expresses the degree to whch the outcome s unexpected. The RescorlaWagner model mples that lttle s learned about the redundant relevant cue because the occurrence of the outcome s fully predcted by the frst cue. The RescorlaWagner model acknowledged that dfferent nput cues could have dfferent assocabltes, but provded no mechansm whereby these assocabltes change due to tranng. The RescorlaWagner model has had a monumental nfluence on research n assocatve learnng (Mller, Barnet, 6 Grahame, 1995; Segel 6 Allan, 1996), and t remans the standard explanaton of blockng (e.g., Domjan, 1998, pp ). An alternatve explanaton of blockng was propounded by Mackntosh and colleagues (Mackntosh, 1975; Mackntosh 6 Turner, 1971; Sutherland 6 Mackntosh, 1971). Ther attentonal theory suggested that somethng s learned about the redundant relevant cue, namely that t should be gnored. Mackntosh (1975) proposed specfc formulas to govern changes n attenton, and t wll be shown later n ths artcle that a specal case of EXIT yelds attentonal changes essentally the same as those proposed by Mackntosh (1975). Ths attentonal explanaton of blockng also mples that a blocked cue should subsequently suffer attenuated learnng, because the suppresson of attenton must be unlearned. Mackntosh and Turner (1971) reported just such attenuaton after blockng n rats. Recently, experments wth human partcpants also showed that learnng about a blocked cue s attenuated relatve to a control cue (Kruschke 6 Blar, 2000). One of these experments (Kruschke 6 Blar, 2000, Experment 1) s summarzed here, and then EXIT s ft to the data. The ft s reasonably good, but the ft s sgnfcantly worse and qualtatvely lackng when the attentonal shfts are fxed at zero and the model only mplements error-drven learnng of assocatve weghts. That s, attentonal shfts are needed for a full account of blockng.

12 ATTENTION IN LEARNING 823 Desgn of the Experment Partcpants learned to dagnose lsts of symptoms as varous dseases. On a gven learnng tral, the partcpant would see a lst of symptoms (e.g., ``ear ache'' and ``skn rash'') on hs or her computer screen and have to dagnose ths hypothetcal patent as havng one of sx dseases, D, F, G, H, J, or K, by pressng the correspondng key on the computer keyboard. After makng hs or her response, the partcpant saw correctve feedback. Intally, the partcpant would merely guess, but after several trals would learn the correct dagnoses. See Kruschke and Blar (2000) for complete procedural detals. The experment conssted of three phases of tranng, shown n Table 1. In the frst phase, symptom A always resulted n dsease 1, denoted A 1. (Please note that the abstract specfcaton uses letters to denote symptoms and numerals to denote dseases, unlke the actual stmul presented to partcpants.) In the second phase of tranng, the redundant symptom B was added to symptom A, always leadng to the same dsease as that whch prevously occurred wth symptom A (.e., AB 1), so that learnng about symptom B would, presumably, be blocked. The second phase also ncluded HI 6, whch acted as a comparson for the blocked symptom B. In the test phase for blockng, symptoms were presented wthout correctve feedback. Of several cases tested, one was a combnaton of symptoms B and I. If symptom B was blocked, then people should prefer the dsease pared wth the control symptom I over the dsease pared wth the blocked symptom B. In the subsequent, thrd phase of tranng, new symptoms and dseases were ntroduced, such that ABC 2 and DEF 4. The central motvatons for ths structure are the hypotheses that (1) learners wll shft attenton away from cues that already have been learned as ndcatve of dfferent dseases (Kruschke, 1996a; Kruschke 6 Johansen, 1999) and (2) learners wll tend not to shft attenton toward a cue that they have prevously learned to gnore. For the case DEF 4, attenton wll shft away from symptom D, because t s already known to ndcate dsease 3, leavng attenton on the dstnctve symptoms E and F. For the case ABC 2, TABLE 1 Desgn of Experment Showng Attenuaton after Blockng (Kruschke Y Blar, 2000, Experment 1) Control Control to assess to assess Phase Blockng attenuaton blockng Tranng I A 1 D 3 Tranng II AB 1 D 3 HI 6 Test for blockng e.g., BH, BI Tranng III A 1 D 3 G 5 ABC 2 DEF 4 GHI 6 Test for attenuaton e.g., BE, BF Note. Letters AI denote symptoms, and numerals 16 denote dseases.

13 824 JOHN K. KRUSCHKE attenton wll shft away from symptom A, because t s already known to ndcate dsease 1. If, as a consequence of blockng, people have learned to gnore symptom B, then attenton should also not be drected to Symptom B, leavng only symptom C to be sgnfcantly attended to. Then there wll be only a relatvely weak assocaton made between symptom B and Dsease 2. The strength of the assocaton s assessed n the fnal test phase, when symptoms B and E are presented together. It was predcted for ths case that people would prefer the dsease pared wth the nonblocked symptom better than the dsease pared wth the blocked symptom. On the other hand, f durng the second phase, when symptom B s blocked, there s no learned nattenton to symptom B, and nstead there s merely a relatve lack of assocatve learnng about symptom B, then subsequent learnng about t should be largely unaffected. That s, accordng to the RescorlaWagner (1972) model, assocatve learnng from B to the novel dsease 2 should be unaffected by any prevous (lack of) learnng from B to dsease 1. Therefore, n the thrd tranng phase, symptom B should be as strongly assocated wth the new dsease 2 as the control symptom E s assocated wth the new dsease 4. A varety of other symptom combnatons was presented n the fnal testng phase to further constran the theores. Table 1 shows that the thrd tranng phase also ncluded cases of symptom G pared wth outcome 5 and symptoms GHI pared wth outcome 6. These cases were ncluded merely to match the thrd phase of tranng n the other experments reported by Kruschke and Blar (2000), n order to facltate comparson of results across experments. Results of the Experment Table 2 shows the choce proportons for each of the test cases. The most crtcal elements of the results are set n boldface font n the table and graphed n Fg. 2. Robust blockng s exhbted for the test case BHBI: People preferred dsease 6, assocated wth the control symptoms H and I, over dsease 1, assocated wth the blocked symptom B, to 15.00, / 2 (df=1, N=59)2=10.4, p<.005. The / 2 value has been dvded by 2 as the most conservatve precauton aganst a possble lack of ndependence between the two repettons of the case seen by each partcpant (Wckens, 1989, p. 28). An nterestng aspect of the data from the frst test phase (the test for blockng) s that people often selected dseases they had not yet seen any cases of. For example, the test case AD elcts a total of responses for Dseases 2, 4, or 5, none of whch had yet occurred n tranng. A possble explanaton s that people were usng what I have prevously referred to (Kruschke 6 Bradley, 1995; Kruschke 6 Erckson, 1995) as strategc guessng, whereby people mght reason that ``ths s a case I haven't seen before, therefore t must be a dsease I haven't seen before.'' The models have no mechansm for strategc guessng. Ths strategc guessng cannot provde an alternatve explanaton of the effect attrbuted to blockng, however, because the preferred responses nvolved the already learned dseases.

14 ATTENTION IN LEARNING 825 TABLE 2 Human Choce Percentage n Test Trals of Experment 1 of Kruschke and Blar (2000) Dsease Symptoms Test for Blockng BHBI AB D HI BD AD AHAI DHDI Test for Attenuaton BEBF A ABC D DEF G GHI CECF BHBI CHCI AB AC DEDF GHGI Note. Letters AI denote symptoms, and numerals 16 denote dseases. A slash denotes structurally equvalent cases collapsed nto a sngle row; e.g., BEBF ndcates results for cases BE and BF combned. Robust attenuaton of learnng about the blocked cue s shown by the test case BEBF. People preferred Dsease 4, assocated wth the control symptoms E and F, over Dsease 2, assocated wth the blocked symptom B, to 22.50, / 2 (df=1, N=129)4=6.30, p<.05. (Agan, the / 2 value was dvded by the number of repettons seen by each partcpant, as a very conservatve precauton aganst a possble lack of ndependence.) Varous other statstcal analyses were presented n Kruschke and Blar (2000). These results are nconsstent wth the hypothess that blockng s caused entrely by lack of learnng about the blocked cue. Instead, the results are consstent wth the hypothess that blockng s caused, at least n part, by learned nattenton. The model fts presented below wll support the attentonal hypothess, n that models wth attentonal shftng ft the data well, but constraned models wthout attentonal shftng fal to ft the data.

15 826 JOHN K. KRUSCHKE FIG. 2. Essental data (shown n boldface font n Tables 2, 3, and 4) from the experment examnng learnng after blockng, wth predcted values from the models, are shown. The left-hand bars, labeled ``Blockng,'' plot data from the frst test-phase tem BHBI. In ths case the ``blocked cue'' s B, wth the correspondng choce of dsease 1, and the ``control cue'' s H or I, wth the correspondng choce of dsease 6. The rght-hand bars, labeled ``Attenuaton,'' plot the data from the second test-phase tem BEBF. In ths case the ``blocked cue'' s B, wth the correspondng choce of dsease 2, and the ``control cue'' s E or F, wth the correspondng choce of dsease 4. The experment desgn appears n Table 1.

16 ATTENTION IN LEARNING 827 Ft of EXIT to Blockng Data EXIT was traned on the same 40 sequences that were experenced by the 40 human partcpants, and the model's mean choce probabltes were ft to the 132 percentages shown n Table 2. The rows of the table are constraned to sum to 1.0, so there are 110 degrees of freedom n the data. The dscrepancy between predcted and emprcal proportons was assessed usng the log-lkelhood measure, G 2 =2N f log( f m ), where N s the number of subjects, f s the human choce proporton, m s the model's predcted proporton, and the ndex runs over all 132 data ponts. (The values n the formula for G 2 are proportons, whch are multpled by 100 to get the percentages dsplayed n the tables.) The parameter values were searched usng a hll-clmbng method, whch was started from several wdely dfferent ponts so that the fts reported below are lkely to be globally optmal. When N s large and when the predcted proportons are not too extreme, then G 2 s dstrbuted approxmately as / 2, and therefore goodness of ft can be tested. Many of the predcted proportons n the current fts do approach extreme values, however, and so the dstrbuton of G 2 s not necessarly accurately approxmated by / 2 and we cannot assess the goodness of ft from the crtcal values of / 2. Nevertheless, G 2 remans useful as a descrptve measure of dscrepancy between data and predctons, and G 2 weghs dscrepances from extreme proportons more heavly than dscrepances from moderate proportons. EXIT fts the data farly well, wth G 2 (104)=73.13, for the parameter values c=0.348, P=1.07,,=4.43, * g =1.27, * w =0.316, and * x = For these parameter values, the RMSD s , but ths was not mnmzed. The second column of Table 3 shows the value of G 2 for each test case ndvdually. These values sum to the total G 2 that was mnmzed by the parameter search. As each row has about 5 degrees of freedom, a row G 2 of about 10 or hgher ndcates an tem that s poorly ft by the model. Ths crteron for G 2 s only of heurstc value, because G 2 mght not be well approxmated by / 2 n these cases. The only tem for whch the model shows a poor ft s the case AD, presumably because the model has no mechansm for strategc guessng. The problem s that n the frst test phase the model gves strong choce preferences only to dseases that t has actually been traned on pror to the test phase;.e., dseases 1, 3, and 6. People, to the contrary, select other, theretofore unseen dseases (2, 4, and 5) notably often. Ths was descrbed earler as the result of strategc guessng, whereby people mght reason, ``ths s a case I haven't seen before, therefore t must be a dsease I haven't seen before'' (Kruschke 6 Bradley, 1995; Kruschke 6 Erckson, 1995). Strategc guessng mght be especally strong for the case AD, because both ndvdual symptoms have been learned thoroughly n pror tranng, and so t s obvous to people that ths combnaton has not been seen before. The model has no mechansm for strategc guessng, and therefore cannot account for people's stronger selecton of dseases 2, 4, and 5. Even for ths case AD, however, the model ncely shows the human preference for dsease 1 over dsease 3. EXIT robustly shows blockng and attenuaton of learnng after blockng. The crtcal test cases are shown n boldface font n Table 3 and are graphed n Fg. 2.

17 828 JOHN K. KRUSCHKE TABLE 3 Best Ft by EXIT to Choce Percentage n Table 2 Dsease Symptoms G Test for Blockng BHBI AB D HI BD AHAI AD DHDI Test for Attenuaton BEBF A ABC D DEF G GHI CECF BHBI CHCI AB AC DEDF GHGI Note. Notaton s the same as n Table 2. In the test for blockng,.e., the case BHBI, EXIT strongly prefers the controlsymptom dsease over the blocked-symptom dsease. In the test for attenuaton after blockng, EXIT agan strongly prefers the control-symptom dsease over the blocked-symptom dsease. EXIT shows attenuaton of learnng about the blocked cue because of attentonal shftng and learnng. In the frst phase of tranng (see Table 1), a strong assocaton between cue A and outcome 1 s learned. In the ntal trals of the second phase, when presented wth cases of AB 1, attenton s spread over both cues, whch mples that cue A s not gettng as much attenton as t dd n the frst phase (when cue A was presented by tself). Ths lower attenton to cue A causes the predcted actvaton of outcome 1 to be lower than t would be f cue A were presented alone. The frst response to ths error s to shft attenton toward cue A, away from cue B. Ths shft s then learned, so that n subsequent presentatons of cues A or B attenton s drected more to cue A than to cue B. In partcular, when the case

18 ATTENTION IN LEARNING 829 ABC 2 s presented n the thrd phase of tranng, attenton s ntally (before correctve feedback appears) drected away from cue B, wth cue C gettng more attenton than cue B. When the correctve feedback s presented, attenton shfts away from cue A, but the ntal dsadvantage of cue B perssts, leavng learnng about cue B attenuated. Ft by EXIT wth No Attenton Shftng. When the attentonal shftng n EXIT s turned off (.e., when * g =0, * : =0, and c s rrelevant), the model becomes a form of the RescorlaWagner model, wth two extra qualtes: EXIT retans (1) the attentonal capacty parameter and (2) the Lucesoftmax rule for mappng output actvatons to choce probabltes. Of course, when the attentonal capacty power P s very large (and when all nputs have the same salence), t s tantamount to no capacty lmtaton at all, as n the RescorlaWagner model. Therefore, t was antcpated that ths verson of the model, lke the RescorlaWagner model, should show blockng, but no attenuaton of learnng after blockng. TABLE 4 Best Ft by EXIT wth No Attenton Shftng to Choce Percentages n Table 2 Dsease Symptoms G Test for Blockng BHBI AB D HI BD AHAI AD DHDI Test for Attenuaton BEBF A ABC D DEF G GHI CECF BHBI CHCI AB AC DEDF GHGI Note. Notaton s the same as n Table 2.

19 830 JOHN K. KRUSCHKE The best ft of EXIT wth no attenton-shftng produced G 2 (107)=112.78, wth parameter values of P=1.16,,=4.35, and * w =0.186 (and a correspondng RMSD of ). The ncrease n G 2 of 39.65, for three degrees of freedom, s hghly sgnfcant, so full EXIT fts much better than EXIT wthout attenton. The predctons of the constraned model are shown n Table 4. As antcpated, the model ncely shows blockng for the test case BHBI n the frst test phase, but does not show any attenuaton after blockng, for case BEBF, n the fnal test phase. Specfcally, for case BEBF, the model wthout attenton shows equal preference (37.7 0) for dseases 2 and 4, unlke people. Interm Summary and Prevew To ths pont n the artcle, the EXIT model has been descrbed and successfully ft to data that demonstrate attenuated learnng about a blocked cue. EXIT s able to ft the data because of ts attentonal shftng and learnng. Assocatve weght learnng by tself, as n the RescorlaWagner model, cannot show attenuaton after blockng. Several other experments n my lab have shown robust attenuaton of learnng about a blocked cue. Some of these other experments are reported n Kruschke and Blar (2000) and Kruschke (n preparaton). Later n the artcle t wll be shown how a specal case of EXIT s very smlar to the model of attentonal changes n anmal learnng proposed by Mackntosh (1975). Other applcatons of EXIT and related models to human data wll also be summarzed later. In the next secton, a dfferent connectonst mplementaton of attentonal shftng s descrbed and ft to the blockng data. Ths mplementaton, n a mxture of experts archtecture, wll also be shown to have a specal case that s very smlar to the model proposed by Mackntosh (1975). MIXTURE OF EXPERTS MODEL In the recent connectonst modelng lterature, a recurrng queston s how to automatcally decompose complex learnng problems nto subproblems that are ndvdually more easly solved than the overall problem. One approach to ths ssue s the mxture of experts framework ntroduced by Jacobs et al. (1991). The underlyng noton s that a connectonst network can have several subnetworks, also called modules, each of whch can dscover and learn a subproblem. Each of these modules becomes an ``expert'' for ts partcular subproblem. The output and learnng of these expert modules are controlled by another module called the ``gatng'' network, whch learns how to allocate the experts n specfc stuatons. Thus, the scheme as a whole s a ``mxture of experts.'' The mxture of experts framework has been used extensvely n engneerng applcatons and s ganng popularty as a framework for models n psychology (for a recent revew see Jacobs, 1997). The gatng network learns to allocate attenton to whchever module s dong the best job of predcton. Ths allocaton s stmulus specfc, so that attenton can be

20 ATTENTION IN LEARNING 831 drected to one expert module for one stmulus, but to a dfferent expert module for a dfferent stmulus. Ths ablty to allocate attenton s the prmary motvaton for consderng the mxture of experts approach n ths artcle. Attenton affects both the output of the model and ts learnng. Thus, the expert module that s gettng the most attenton has the most nfluence on the overall output of the model. The module wth the most attenton also gets a much stronger error sgnal than the other modules, and so t learns more about the current stmulus than the other modules do. In ths way the dfferent modules can learn dfferent aspects of the problem. A mxture of experts approach was prevously used to model the learnng of rules and exceptons n classfcaton (Erckson 6 Kruschke, 1998; Kruschke 6 Erckson, 1994). One expert module conssted of exemplars, whereas other expert modules nstantated rules. The model, called ATRIUM, exhbted trends n both learnng and generalzaton much lke people. Unlke the EXIT model, however, the shfts of attenton n ATRIUM were relatvely gradual and executed at the same tme as the changes n assocatve weghts. The new mxture of experts model descrbed n the present artcle extends ths prevous work by allowng the attenton shft between modules to be relatvely rapd and to occur before assocatve weght changes, just as was done n EXIT (and n RASHNL, see Kruschke 6 Johansen, 1999). In the present applcaton, each ndvdual cue acts as a dstnct expert, tryng to predct the correct output. The gatng module learns whch ndvdual cues are effcacous for partcular stmul. Ths arrangement can account for blockng, at least n prncple, because the gatng module learns to gnore the expert module that contans the blocked cue. In ths secton, a varant of Jacobs et al.'s (1991) approach s descrbed and ft to the data from the attenuaton-after-blockng experment summarzed earler n the artcle. The orgnal mxture of experts approach s brefly descrbed later, and both the orgnal verson and ts varant are shown to mply attentonal shfts and assocatve weght changes vrtually dentcal wth those proposed by Mackntosh (1975) n hs model of anmal learnng. Actvaton Propagaton n the Expert Modules As mentoned above, each component cue comprses an ``expert'' that attempts to ndvdually predct the outcome. As shown n Fg. 3, the th expert module conssts of a sngle cue node that represents the presence or absence of the th cue, connected to a full set of output nodes. For the module, the actvaton of the kth output node s gven by a out k =w k a n. (11) Note that the output node actvatons wthn the th module are affected only by the th cue. Note also that attenton has no role wthn each expert module; attenton does not gate the nput actvatons.