Memory Retrieval Based on Cortical Interaction

Transcription

1 Memory Retrieval Based on Cortical Interaction O. Hoshino Oita Univ. 700, Dannoharu, Oita, , Japan K. Kuroiwa Oita Univ. 700, Dannoharu, Oita, , Japan Abstract We propose a cortical neural network model and investigate the neuronal relevance of cortical interactions to recalling long-term memory. The model consists of the left and right hemispheres, each of which has IT (inferotemporal cortex) and (prefrontal cortex) networks. The information about visual features and their categories were encoded into point attractors of the IT and networks, respectively. In a category-association task, the IT network of the right hemisphere was stimulated with a cue feature. After a delay period, the IT network of the left hemisphere was simultaneously stimulated with the choice feature and an irrelevant feature. The cue and choice features belong to the same category, while the irrelevant feature belongs to other category. We demonstrate that the top-down pathway (-to-it) triggers the retrieval of long-term memory of the choice feature from the IT, and the bottom-up pathway (IT-to-) contributes to the maintenance of the retrieved memory during the delay period. 1 Introduction It is an interesting problem how the brain retrieves relevant information from long-term memory. A series of neurophysiological experiments, conduced by Miyashita and his colleagues [1, 2], have made a great contribution to solving that problem. The inferotemporal cortex (IT), which is considered one of the sites for visual long-term memory, has been investigated in terms of its neuronal relevance to the retrieval of long-term memory. They trained a monkey to learn category-association [2], in which twenty visual pictures were divided into five categories. Each category includes four cue pictures and one choice picture. In a category-association task, the monkey is presented with a cue picture for a brief period of time ( 500 ms), followed by a delay period ( 1500 ms) during which no visual stimulus is presented to the monkey. After the delay period, the monkey chooses the choice picture from the two candidates (the choice picture and an irrelevant picture) that are simultaneously presented to the monkey. The cue and choice pictures belong to the same category, while the irrelevant picture belongs to other category. They found category-selective neurons in the IT. When these neurons were activated by stimulation with a cue picture, they keep firing during the following delay period. The delayed activity of these neurons implies that information about the cue is maintained, or kept in mind, until the choice stimulus is presented. The choice stimulus, which belongs to the same category as the cue, also induced strong responses of these IT neurons. These neurons showed specific responses to the stimuli from the same category but not to those from other categories and thus are category-selective. By using a partially split brain between the two hemispheres, they investigated how the category-selective neurons were activated when simulated with the cue feature. They concluded that the category-selective neurons were activated by topdown signals from to IT, which conveyed information about the category of the applied cue stimulus through rich feedback projections [3]. The neuronal relevance of the top-down process (-to-it) is a crucial first step for understanding how the cortical interactions contribute to the retrieval of long-term memory. However, it seems that the mutual interaction between the and IT may also contribute to the process of memory retrieval, because the IT sends its output signal back to the as soon as it receives the signal from the. Concerning the interaction between separate cortical regions, it is also well known that the right and left hemispheres interact to communicate with each other to select the choice picture with a cue stimulus in a pair association task [4]. Therefore, it seems quite important to investigate the neuronal mechanism and its significance of the interaction not only within but also between the two hemi-

2 spheres in recalling long-term memory. To understand the essential neuronal mechanisms of the retrieval of long-term memory, it is necessary to know how category information is represented in the through association learning and contributes to the retrieval of long-term memory. In the present study, we propose a neural network model that consists of the left and right hemispheres, each of which has IT and networks. The IT stores visual features as long-term memory while the stores category information of these features. By stimulating the IT network with a cue, we investigate how mutual interaction between the and IT networks contributes to the retrieval of the choice information that belongs to the same category as the cue. We also investigate roles of the interhemispheric interaction in the retrieval of the choice information because it is obvious that even a single hemisphere can process category-association tasks correctly [1, 2]. 2 Neural network model The schematic drawing of the present neural network model is shown in Fig. 1a. We made excitatory synaptic connections with one-to-one correspondence between the left and right networks. The IT network is divided into two networks, ITcue and ITcho (Fig. 1b), whose neurons send signals to neurons in a divergent/convergent manner and receive feedback signals from the. Both hemispheres have the same structure of neural network. Information about individual cue and choice features are processed in the ITcue and ITcho networks, respectively. Information about categories of these features is processed in the. Although the cortical and interhemispheric projections of the present model is intended only as an abstract representation of interacting cortical regions and not based on the precise structure of the brain, simulation results could provide fundamental insights into how the cortical and interhemispheric interactions mediate the retrieval of long-term memory from the IT network as will be shown in later sections. Each network is constructed based on the so-called associative neural network. Specific firing patterns of neural networks, which represent features and their categories, are embedded as stable point attractors according to the Hebbian learning rule [5, 6]. To construct the present network model, we used simple neural networks. Each network consists of projection neurons and interneurons such as pyramidal cells and basket cells, respectively. The projection neurons are connected to each other via either excitatory or inhibitory synapse. Connections be- Figure 1: The neural network model. a Interhemispheric connections are made between the left and right networks. The and IT interact with each other via reciprocal connections between them. b The IT network is divided into two networks (ITcue, ITcho) whose neurons send signals to neurons in a divergent/convergent manner. The IT neurons receive feedback signals from the in a parallel manner (not shown for the clarityof the divergent/convergent feedforward projection). A group of neurons indicated byxcn (X=I, II, III, IV; cn = c1, c3, c2) and Xcho with an open arrow are feature-sensitive neurons that respond to the features, Xcn and Xcho, respectively. A group of neurons indicated byxcat (X=I, II, III, IV) with an open arrow are category-selective neurons that respond to the features (Xc1, Xc2, Xc3, Xcho) belonging to the same categoryx. tween projection neurons are made directly from axoncollaterals to other neurons. Although the inhibitory connections between projection neurons should be indirectly, possibly mediated by some kind of interneurons, the connections here are made directly for simplicity. Each projection neuron has one inhibitory interneuron by which the activity of the projection neuron is suppressed locally. Dynamic evolutions of membrane potentials of projection neurons and interneurons of ITcue and ITcho networks are defined by τ IT p du IT p,i (t) NIT u IT p,i (t)+ N +wpr IT Vr,i IT (t)+ wpp, IT IT (t) Vp,j (t) L IT,P C Vp,j (t t)

3 τ IT r +Ip,i IT (t)+iit Osc (t), (1) du IT r,i (t) = u IT r,i (t)+wrp IT Vp,i IT (t), (IT = IT cue, IT cho) (2) where u IT p,i (t) and uit r,i (t) are the membrane potentials of ith projection neuron and ith interneuron at time t, respectively. τp IT and τr IT are the decay times of these membrane potentials. N IT and N are the numbers of the projection neurons of IT network and network, respectively. wpp, IT (t) is the strength of synaptic connection from projection neuron j to projection neuron i. wpr IT (wit rp ) is the strength of synaptic connections from the interneuron (projection neuron) to the projection neuron (interneuron). Vp,i IT (t) and Vr,i IT (t) are axonal outputs of ith projection neuron and ith interneuron, respectively. Vp,j (t t) is the axonal output of jth projection neuron of network, which arrives at the IT network with a transmission delay of t. L IT,P C describes the strength of synaptic connection from projection neuron j of network to projection neuron i of IT network. Ip,i IT (t) is an external stimulus to projection neuron i. IOsc IT (t) isa global oscillation of membrane potential and defined as IOsc IT (t) =I Oscsin2πνt (I Osc =0.3 and ν = 10Hz). The application of the global oscillation is exclusively implemented for binding cues (Xc1, Xc2, Xc3) and the choice (Xcho) to make neuronal representation of the information about their category (Xcat) in the network. Dynamic evolutions of membrane potentials of the projection neurons and interneurons of the network are similarly defined by the following equations. τp du p,i (t) N u p,i (t)+ w pp,(t) Vp,j (t) +w pr N IT Vr,i (t)+ L P C,IT Vp,j IT (t t) +Vp,i HM (t t), (3) τr du r,i (t) = u r,i rp V p,i (t), (4) where Vp,i HM (t t) denotes the from neuron j of of the contralateral hemisphere. Value 1 or 0 was chosen for the parameter sets, L IT,P C and L P C,IT, so that the divergent/convergent feedforward projections (Fig. 1b) and parallel feedback projections are properly organized, respectively. By such projections, stimulation of the ITcue network with a cue feature (Xcn) tends to induce firing patters (, or point attractors) corresponding to Xcat in the network and Xcho in the ITcho network. The probability of firing of neuron i is determined by the sigmoid function of the membrane potential [6]. Synaptic connections between projection neurons change temporally, that is, decay continuously but are modified according to the Hebbian rule whenever any stimulus is presented to the IT network [6]. The neurons act as a coincidence detector. The coincidence detection neuron is used here to bind three cues (Xcn; cn=c1,c2,c3) and one choice (Xcho) firing patterns and integrate them into a single firing pattern (Xcat) that represents the information about their category. 3 Formation of cognitive maps We describe here how stable point attractors, which encode cues, choices and their categories, are created in the dynamics of the IT and networks. This process corresponds to the creation of a cognitive map. 3.1 Cognitive maps for IT The ITcue and ITcho networks were trained with twelve cue features and four choice features, respectively. Each feature is represented by a corresponding on-off pixel pattern {ξ i ()} that is set to be sparse. For the ITcue (ITcho) network, n = c1,c2,c3 (n = cho) and i =1 N IT cue (i =1 N IT cho ). When a cue (a choice) feature is presented to the ITcue (ITcho) network, the amount of stimulation received by the ITcue (ITcho) projection neurons is described by IT cue I p,i (t) =γξ i (X n ), IT cho I X n = { X = I,II,III,IV n = c1,c2,c3 p,i (t) =γξ i (X n ), { X = I,II,III,IV X n = n = cho (5) (6) where γ denotes the intensity of feature stimulus. ξ i () takes on value 1 for on or 0 for off pixel. Equations (5)-(6) mean that the stimulus is not presented to the retina but to the IT cortex as neural coming through afferent fibers. The value of γ is 10 for the memorization process and 0.1 for a categoryassociation task. Such high intensity (γ = 10) is necessary for the creation of dynamical maps and corresponds to the sensitivity increase that may arise from some kinds of behaviorally attentional processes [5, 6]. The Hebbian learning process was carried out during the application period (Fig. 2a) [6]. Because

4 B.C.of O(;t) I I I I IVcat IIIcat IIcat Icat point attractor corresponding to, the firing pattern of the network fluctuates randomly around the pattern {ξ i ()}. Dynamic properties of these point attractors have been investigated in detail [5, 6]. 3.2 Cognitive maps for The network of each hemisphere was trained to learn categories of cues and choices (Fig. 2b). Pairs of a cue and a choice that belong to the same category were simultaneously applied to the ITcue and ITcho, respectively, together with a global oscillatory IT cue Osc IT cho Osc (I (t) =I (t) in Eq. 1). During the application period, the Hebbian learning was carried out for the network. After the training, the state of the network begins to itinerate randomly among the four point attractors, corresponding to the four categories (Xcat; X=I, II, III, IV) (Fig. 2b-bottom). This means that the information about these categories is encoded into category-specific point attractors of the network. To create these point attractors, we used here an additional external signal, the global oscillation of (t)). We show in Fig. 3 how the global oscillation contributes to encoding categories into the relevant point attractors. membrane potentials (I IT Osc Figure 2: Creation of dynamical maps. a Dynamical maps for the ITcue network (top) and ITcho network (bottom). One vertical bar is drawn on row when measure overlap O(; t) 0.8 [6], which means that the current firing pattern of the network is almost the same as the onoff pixel pattern corresponding to. b The dynamical map for network (bottom). Horizontal bars indicate duration of application period of the stimuli ( and Xcho) to the ITcue and ITcho networks, respectively. IT I Osc(t) ITcue ITcho the strength is strong enough (γ = 10), the activity of the network is almost the same as the pattern {ξ i ()}. By the learning process, these features are embedded into the ITcue and ITcho networks as point attractors, and the activity states of the networks begin to itinerate randomly among these point attractors. This means that individual features are encoded into point attractors and mapped onto the dynamics of the neural networks. We used here the term point attractor for descriptive convenience, because, in a broad sense one firing pattern {ξ i ()} appears almost consecutively in the basin of the point attractor corresponding to feature. However, the real dynamic properties of the present point attractor differ from the ideal one and should not be taken literally [5, 6]. That is, when the network state is within the basin of the ITcue ITcho Figure 3: Neuronal firings in the process of dynamical map formation for network. a Map formation with a global oscillation (IOsc(t)). IT ITcue and ITcho neurons fire in synchrony. The synchronized neuronal firings are detected by neurons. b Map formation without the global oscillation. ITcue and ITcho neurons fire out of synchrony and thus neurons cannot fire.

5 When the oscillation is applied to the network (Fig. 3a), the ITcue neurons and ITcho neurons, which respectively are responding to the cue () and the choice () stimulation, fire in synchrony. The synchronized firings of neurons effectively activate the neurons that receive s from these ITcue neurons and ITcho neurons. This results in firing of neurons. Synaptic modulation based on the Hebbian rule between the neurons that fire in synchrony develops stable point attractors in the network, corresponding to the categories to which the cues and choices belong. As addressed before, we assumed here that the neurons act as a coincidence detector so that the synchronized firings of ITcue and ITcho neurons can be detected by the neurons. In the case where the global oscillation is not present, neuronal firings of ITcue and ITcho networks do not coincide as shown in Fig. 3b. Without such synchronization, the neurons cannot fire, resulting in failure to develop point attractors in the dynamics of the network. Note that in both cases (Fig. 3a-b) the frequency of firings is almost the same for the two networks (ITcue and ITcho). This means that the states of the ITcue and ITcho networks are being in the same basin of attractor but their operating areas are completely different. 4 Dynamic process in memory retrieval In this section, we show fundamental neuronal mechanisms for the retrieval of long-term memory from the IT network. We let the present neural network model carry out a category-association task [2], in which the model is required to retrieve long-term memory of the choice feature when stimulated with a cue feature. The cue and choice features belong to the same category. The detail of the task process is as follows. i) A cue stimulus (Xcn) is briefly presented to the IT network of the right hemisphere. ii) The cue stimulus is switched off and any stimulus is not present during the following delay period. iii) After the delay period, the choice stimulus (Xcho) and an irrelevant feature stimulus (X cho; X X) are simultaneously presented to the ITcho network of the left hemisphere. Time courses of pattern overlap O(X; t) along processes (i)-(ii) are shown in Fig. 4a-f. When the ITcue of the right hemisphere is stimulated with the cue (Fig. 4d), the state of the ITcue network changes from the randomly itinerant state to the point attractor corresponding to the feature, which then induces the point attractor corresponding to IIcat in the net- B.C.of O(;t) I I IVcat IIIcat IIcat Icat I I IVcat IIIcat IIcat Icat (Left hemisphere) (Right hemisphere) Figure 4: Behaviors of neural networks during a category-association task. When the ITcue of the right hemisphere is stimulated with a cue (), relevant point attractors are induced in a cascade manner (d f e,c a,b) as indicated bya sequence of open arrows. The choice feature is recalled in both ITcho networks (b,e). work (Fig. 4f). The feedback signal from the to ITcho induces the point attractor corresponding to the choice feature () in the ITcho network (Fig. 4e). The point attractor (IIcat) (Fig. 4f) affects the left hemisphere by interhemispheric signal transmission, inducing the point attractor corresponding to IIcat in the network of the left hemisphere (Fig. 4c), which then induces the three point attractors corresponding to, and IT in the ITcue network (Fig. 4a) and the point attractor corresponding to the in the ITcho network (Fig. 4b). The emergence the point attractors () in the ITcho networks of both hemispheres (Fig. 4b,e) is interpreted as a neuronal representation of the retrieval of long-term memory of the choice feature () with respect to the cue feature (). As for process (iii), when the choice () and an irrelevant feature (e.g., ) were simultaneously presented to the ITcho network of the left hemisphere, the overall dynamic states of the networks were almost the same as those in the delay period. That is, the same point attractors continue to appear, though the ITcho neurons corresponding to show further

6 elevated activity. Similar enhancement in neuronal activity during the choice process has been reported [2]. This may be a neuronal representation for the correct selection of the choice feature from the two candidate features. As shown in Fig. 4a,d, the dynamic behaviors of the two ITcue networks are completely different. According to the neuronal architecture of the present model, features, and belong to the same category II. In general, via the top-down (-to-it) pathway the point attractor IIcat induces the three point attractors (,, ) in the ITcue network, as is the case in Fig. 4a. However, if the ITcue receives the additional external (Fig. 4d), the point attractor corresponding to the stimulus could preferably be singled out of the three point attractors. As addressed by Tomita et al. [2], the neuronal activation of the left (Fig. 4b) and right (Fig. 4e) ITcho networks could reflect the top-down and bottom-up neuronal processing, respectively, for retrieving longterm memory of the choice feature. They stated that the time period required for retrieving the longterm memory information is different between the two hemispheres, and the top-down process takes more time than the bottom-up process does. We obtained a similar result to those observed experimentally [2], in which the response latency of the ITcho of the left hemisphere is longer about 100 msec than that recorded in the right hemisphere. The signal delay due to the interhemispheric pathway could be the main cause for the longer response latency (Fig. 4b) in the left hemisphere. However, the order of a hundred of msec of response latency difference seems to be quite long because the signal delay between the two hemispheres is set to be only 10 msec ( t = 10 msec) in the present model. We consider that such a great difference in response latency arises from the difference in time period that is required to put the state of the ITcho network into the basin of the relevant point attractor (). That is, the cascade process (Fig. 6d f c b) takes much more period of time than the cascade process (Fig. 6d f e) does. 5 Concluding remarks We have shown that the retrieval of long-term memory of a choice feature is processed as a sequential induction of relevant point attractors across cortical neural networks. Such a sequential process causes a difference in response latency ( 100msec) of IT neurons between the two hemispheres. Such a large difference in response latency cannot be explained by the difference in anatomical length of their pathways because the velocity of axonal pulses is fast (tens of meters per second) [7]. Tomita et al. [2] suggested that the longer response latency could partly be ascribed to multisynaptic conduction delay or accumulation of s needed to reach a threshold of neuronal responses. We suggest here that one of the plausible neuronal mechanisms for the longer response latency could be ascribed to the time period required for the complete induction of relevant point attractors across cortical networks. When the IT-to- pathways of the left hemisphere were cut, the retrieved memory of the choice disappeared as soon as the cue stimulus, which was applied to the right hemisphere, was switched off. That is, the retrieved memory cannot be maintained in the following delay period. This implies that the top-down pathway (-to-it) triggers the retrieval of long-term memory from the IT, and the bottom-up pathway (IT-to-) contributes to the maintenance of the retrieved memory. Details of this result will be shown in ICONIP2001. References [1] I. Hasegawa, T. Fukushima, T. Ihara, and Y. Miyashita, Callosal window between prefrontal cortices: cognitive interaction to retrieve long-term memory, Science, Vol. 281, pp , [2] H. Tomita, M. Ohbayashi, K. Nakahara, I. Hasegawa, Y. Miyashita, Top-down signal from prefrontal cortex in executive control of memoryretrieval, Nature, Vol. 401, pp , [3] D.J. Felleman, D.C. Van Essen, Distributed hierarchical processing in the primate cerebral cortex, Cereb Cortex, Vol. 1, pp. 1-47, [4] R.W. Doty, J.L. Ring, and J.D. Lewine, Forebrain commissures and visual memory: a new approach, Behav Brain Res, Vol. 29, pp , [5] O. Hoshino, N. Usuba, Y. Kashimori, and T. Kambara, Role of itinerancyamong attractors as dynamical map in distributed coding scheme, Neural Networks, Vol. 10, pp , [6] O. Hoshino, Y. Kashimori, and T. Kambara, An olfactoryrecognition model based on spatio-temporal encoding of odor qualityin olfactorybulb, Biol. Cybern., Vol. 79, pp , [7] W.A.H. Rushton, A theoryof the effects of fibre size in medullated nerve, Journal of Physiology-London, Vol. 115, pp , 1951.