The following article was presented as an oral presentation at the Conference ICANN 98: Wiemer J, Spengler F, Joublin F, Stagge P, Wacquant S: A Model of Cortical Plasticity: Integration and Segregation based on Temporal Input Patterns. In Niklasson L, Boden M, Ziemke T (Eds): ICANN 98, Proceedings of the 8th International Conference on Artificial Neural Networks, Skövde, Schweden, 2-4 September 1998, (I):367-372. Springer-Verlag London (1998)
A Model of Cortical Plasticity: Integration and Segregation based on Temporal Input Patterns J. Wiemer, F. Spengler, F. Joublin, P. Stagge, S. Wacquant Institut für Neuroinformatik, Ruhr-Universität Bochum, Germany e-mail: jan.wiemer@neuroinformatik.ruhr-uni-bochum.de Abstract Early cortical areas reveal plastic topographic structures. The formation and alteration of these so-called cortical maps by self-organizing principles is often explained by Kohonen-like algorithms [7]. However, recent experiments concerning learning related reorganization of Area 3b of somatosensory cortex [1, 2] demonstrate task-specific cortical changes which cannot be explained by Kohonen s model. We therefore propose a model of stimulus induced cortical plasticity that takes the temporal structure of afferent inputs into account, i.e. temporal stimulus distances are transformed into spatial distances of their cortical representations. The simulations agree with the above cited experimental results and predict the alteration of sensory cortices for different sequences of input patterns. 1 Integration/Segregation Task Early sensory cortices represent sensory inputs from the external world in a topological way. They are to a large extent subject to stimulus induced plasticity [1, 2, 3] leading to cortical representations in which some stimuli are integrated and others are segregated. In order to make our point clear we define: representational distance as the distance between cortical activites induced by two stimuli, measured in cortical coordinates or in parameter space (e.g. as defined in population coding), integration as the fusion of different stimuli into one representation, or the reduction of their representational distance, and segregation as the process of further differentiating between stimuli by increasing their representational distance. Efficient stimulus representations require to define which of the incoming stimuli should be integrated and which should be segregated. We call this the integration/segregation task. Integration and segregation processes can be controlled by attentional topdown processes, and/or self-organizing processes using properties of the incoming stimuli. We consider self-organizing processes and put forward the following This work was supported by DFG, SFB 509. 2
Fig a) Paradigm bar A D5d b) Control c) Experiment bar B D5p D5m D4p D4m D4d D4d D5d D5m D4m D5p Bar B Bar A Experimental Stimuli Background Stimuli ISI =300 ms 80 ms reward Target Stimuli false positive hit miss light off Temporal Course of Training time Palm Palm D3p D1p D3m D2p D2m D1p 1mm C Receptive Field Characteristics: M D1d dorsum multi segment RF * non-cutaneous RF D2d D2d M D12d D1d D2d 2mp D2m C 3m 3d D2mp D2m D1p Bar A specific multiple segment/finger RF Bar B specific multiple segment RF D: digit, p: proximal, m: medial, d: distal D2p D2p D4p 3p Palm Figure 1: Neurobiological Experiment; Sketch of experimental paradigm used for training (a), cortical hand representation in primary sensory cortex of control (b) and experimental hemisphere (c) [1]. theses: The average temporal distance between two stimuli is a good measure of their similarity. Early sensory cortex is structured by a self-organizing process that transforms average temporal stimulus distances into spatial distances of their cortical representations. 2 Neurobiological Experiments Recent experiments on tactile object discrimination reveal that synchronous stimuli are integrated and asynchronous stimuli (200-300 ms interstimulus interval) are segregated in the cortex [1, 2]. Spengler et al. trained owl monkeys in a tactile two-bar recognition task for 5-7 months (cf fig. 1a). As background stimuli two vibrating bars A and B were applied alternately with an interstimulus interval (ISI) of 300ms. As target stimuli the monkeys had to detect two consecutive stimuli of either bar A or bar B. The resulting plastic reorganization is shown in figure 1c: receptive fields (RFs) show an integration effect, extending over synchronously stimulated finger segments (see existence of multiple segment RFs), while representations of the two bars A and B are segregated into different regions of the hand representation. The sensory neighbourhood between the two stimuli is not transformed into a cortical neighbourhood of their representations. Wang et al. used a similar experimental paradigm consisting of two parallel tactile bars and leading to similar results. In addition, they analyzed ventro-posterior thalamus response maps revealing no equivalent reorganization. Therefore the representational plasticity appears to be of cortical origin.
a) Cortical shift S B b) Algorithm S A d AB d w D t+isi ISI. V W t t+isi Initialization For each learning step Choose spatio-temporal stimulus Compute: 1. pre-cortical activity 2. cortical feedforward activation 3. propagated wave 4. resulting cortical activity Normalized Hebbian learning Homosynaptic depression Figure 2: Integration/Segregation Model; on-going cortical activity due to former stimulation S A influences the location of the cortical response to an in-coming stimulus S B (a). Sketch of the Integration/Segregation Algorithm (b). 3 The Model Hebbian learning can be seen as the underlying mechanism of integration of synchronous stimuli [4]. But cortical segregation of asynchronous stimuli requires an additional mechanism. As above postulated we assume the segregation distance (i.e. representational distance) to be ISI-dependent. As a consequence, interstimulus intervals are expressed in spatial distances in the stimulus representation. This time-to-space conversion can be generated by the propagation of neural activity into the neighbourhood of a cortical response. Such a wave produces in combination with the new stimulus a time-dependent shift of the resulting cortical activity. It allows a Hebbian learning mechanism to occur at an ISI-influenced cortical location. Our model of cortical plasticity is a three-layer model: A sensory array on which arbitrary shaped stimuli can be applied. In the following we choose Gauß shaped stimuli. A pre-cortical layer which integrates the activity of the sensor array using a fixed weight feedforward connectivity. The dynamics of this layer creates decaying responses. A cortical layer with plastic connectivity coming from the pre-cortical layer. Gauß shaped cortical activities with one fixed width are assumed. The cortical response is computed as follows (cf fig. 2a): After the application of a first stimulus at time t, the resulting cortical activity wave spreads on the cortex at a speed V W 1. When a second stimulus comes at time t + ISI, its cortical activation is shifted by an amount D in the direction to the wave front. This shift depends linearly on the distance d W to the wave front when d W is small (compared to the width of cortical activity) and decreases exponentially for larger d W. We use two learning rules: 1 Biological data suggests V W to be in the range of several mm/s.
Hebbian feedforward learning (in Kohonen-type normalization): W(x, y, t) = α c(x, t)(s(y, t) W(x, y, t)) (t: time, x: cortical coordinate, y: pre-cortical coordinate, W: synaptic weight, c: cortical activity, s: pre-cortical activity, α: learning rate), activity-dependent Homosynaptic Depression [8], i.e. presynaptic activity in the absence of postsynaptic activity causes synaptic depression: W(x, y, t) = ǫ s(y, t)w(x, y, t) when s(y, t),w(x, y, t) are above and c(x, t) below fixed thresholds (ǫ: unlearning rate). A sketch of the algorithm is given in figure 2b. 4 Simulation Results We use the above described model to simulate the experiment of Spengler et al. (cf fig. 3a,b). In addition, we vary the interstimulus interval leading to a prediction about experimental conditions that have not been examined so far (cf fig. 3d). The initial weights reflect topographically ordered synaptic weights. They are learned with the integration/segregation algorithm by applying natural stimuli (see below). In analogy to the neurobiological experiments we apply Experimental Phases, i.e. presentation of two alternating experimental stimuli A,B with fixed ISI, and Natural Phases, i.e. presentation of stimuli with natural spatio-temporal correlations (spatial distance proportional to temporal distance). These stimuli on their own lead to the initial state (cf fig. 3a). Figure 4 shows the temporal development of the reorganizing representation. After reaching a steady representation (besides fluctuations) we apply only natural stimuli and thereby demonstrate the reversibility of the process. The simulation results depend on the value of ISI relative to the ratio d AB (0)/V W, reflecting how far the cortical activity propagates during the ISI (d AB (0): initial representational distance between experimental stimuli). 5 Psychophysics Spatially separated stimuli that excite the skin lead to systematical mislocalization in perception when their time difference lies in the range of 15 to 200 or even more milliseconds [5]. This phenomenon called Saltation illustrates the continuous transformation of temporal distances into perceived spatial distances
Cort. activation RF centers a) Initial state b) Segregation c) Reversibility d) Integration n=0 n=16000 n=26000 n=16000 Figure 3: Learning Results; top row: receptive field centers in stimulus space, bottom row: cortical activation induced by simultanous presentation of both experimental stimuli; initial state (a), segregation caused by large experimental ISI, i.e. ISI> d AB(0)/V W (b), reversibility of b (c), integration caused by small experimental ISI, i.e. ISI< d AB(0)/V W (d); learning step (cf fig. 2b) denoted by n. and is described for different modalities (analogs in vision and audition). We regard the ISI-dependence of perceived spatial locations as a hint to dynamics of the kind assumed in our model. 6 Conclusion Our model of cortical plasticity reveals ISI-dependent representational distances. This constitutes an interpolation of neurobiological findings. In other words, we assume that the ISI-dependence of representational distances is not confined to a mere differentiation of synchronous and asynchronous stimuli, but that this dependence extends to the whole range from 0 to several hundreds of milliseconds. This interpolation can functionally be interpreted as an efficient and quite general way to solve the integration/segregation task. We therefore regard the transfer of the integration/segregation model to other modalities (vision, audition) to be promising. The assumed cortical wave dynamics that guides the self-organizing process is supported by psychophysical findings and can be interpreted as an expectation about incoming stimuli. The influence of such a wave dynamics may be permanently present during ontogenesis leading to ISI-dependent cortical maps, or it may depend on attention reflecting a kind of segregation mode of the system. Temporal aspects of stimuli may constitute a key to understand the structure of cortical maps. E.g. hand representations in cortical area 3b of owl monkeys do not reflect the proportions of the sensory dimensions. Given the topography of the cortical hand representation (cf fig. 1b) we notice a compression in rostro-caudal direction relative to an expansion in medio-lateral direc-
Cortical position Representational distance a) b) 20 bar A 20 10 bar B 10 0 0 10000 20000 Learning steps (n) segregation integration 0 0 10000 20000 Learning steps (n) Figure 4: Temporal Development of Segregation (diamonds) and Integration (squares), sampling every 2000 steps; after 16000 learning steps according to fig. 2b only natural stimuli are applied; centers of stimulus representation (vertical coordinate in fig. 3) (a), representational distance (b). tion. According to our model this distortion could reflect that the average time differences between stimuli on segments of the same finger are smaller than the ones of different fingers. In contrast to examples of cortical plasticity related to learning and improvements in discrimination ability, time dependent integration and segregation processes may play an important role in harmful side effects of input dependent cortical plasticity, e.g. focal dystonia, phantom-limb pain, and dyslexia [9]. References [1] Spengler F, Hilger T, Wang X, Merzenich MM. Learning Induced Formation of Cortical Populations Involved in Tactile Object Recognition. Soc Neurosc Abstracts 1996; 22(I):105 [2] Wang X, Merzenich MM, Sameshima K, Jenkins WM. Remodelling of hand representation in adult cortex determined by timing of tactile stimulation. Nature 1995; 378:71-75 [3] Joublin F, Spengler F, Wacquant S, Dinse HR. A columnar model of somatosensory reorganizational plasticity based on Hebbian and non-hebbian learning rules. Biol Cybern 1996; 74:275-286 [4] Grajski KA, Merzenich MM. Hebb-type dynamics is sufficient to account for the inverse magnification rule in cortical somatotopy. Neural Comput 1990; 2:71-84 [5] Geldard FA. Saltation in Somethesis. Psychological Bulletin 1982; 92(I):136-175 [6] Tanifuji M, Sugiyama T, Murase K. Horizontal Propagation of Excitation in Rat Visual Cortical Slices Revealed by Optical Imaging. Science 1994; 266:1057-9 [7] Kohonen T: Self-Organizing Maps. Springer, Berlin, 1995 [8] Brown TH, Kairiss EW, Keenan CL. Hebbian Synapses: Biophysical Mechanisms and Algorithms. Annu Rev Neuroscience 1990; 13:475-511 [9] Merzenich MM, Schreiner C, Jenkins W, Wang X. Neural Mechanisms Underlying Temporal Integration, Segmentation [...]. Ann NY Acad Sci 1993; 628:1-22 [10] von der Malsburg C. Self-organization of orientation sensitive cells in the striata cortex. Kybernetik 1973; 14:85-100