STDP Provides the Substrate for Igniting Synfire Chains by Spatiotemporal Input Patterns

Transcription

1 LETTER Communicated by Markus Diesmann STDP Provides the Substrate for Igniting Synfire Chains by Spatiotemporal Input Patterns Ryosuke Hosaka Graduate School of Science and Engineering, Saitama University, Sakura-ku, Saitama , Japan Osamu Araki Department of Applied Physics, Tokyo University of Science, Shinjuku-ku, Tokyo , Japan Tohru Ikeguchi Graduate School of Science and Engineering, Saitama University, Sakura-ku, Saitama , Japan Spike-timing-dependent synaptic plasticity (STDP), which depends on the temporal difference between pre- and postsynaptic action potentials, is observed in the cortices and hippocampus. Although several theoretical and experimental studies have revealed its fundamental aspects, its functional role remains unclear. To examine how an input spatiotemporal spike pattern is altered by STDP, we observed the output spike patterns of a spiking neural network model with an asymmetrical STDP rule when the input spatiotemporal pattern is repeatedly applied. The spiking neural network comprises excitatory and inhibitory neurons that exhibit local interactions. Numerical experiments show that the spiking neural network generates a single global synchrony whose relative timing depends on the input spatiotemporal pattern and the neural network structure. This result implies that the spiking neural network learns the transformation from spatiotemporal to temporal information. In the literature, the origin of the synfire chain has not been sufficiently focused on. Our results indicate that spiking neural networks with STDP can ignite synfire chains in the cortices. 1 Introduction Recent experimental and theoretical studies indicate a possible information processing mechanism based on the precise spike timings of the neurons in the brain, which include synchronous firings or the temporal correlation Neural Computation 20, (2008) C 2008 Massachusetts Institute of Technology

2 416 R. Hosaka, O. Araki, and T. Ikeguchi of spikes (Gray, König, Engel, & Singer, 1989; Riehle, Grün, Diesmann, & Aertsen, 1997; Aertsen, Gerstein, Habib, & Palm, 1989; Aertsen, Erb, & Palm, 1994; Vaadia et al., 1995; de Oliveira, Thiele, & Hoffmann, 1997; Fujii, Ito, Aihara, Ichinose, & Tsukada, 1996; Gerstner, Kempter, von Hemmen, & Wagner, 1996; Masuda & Aihara, 2004). With regard to synaptic plasticity, it has been reported that the modification of synaptic strength depends on the temporal difference between the pre- and postsynaptic action potentials in the cortices and the hippocampus; this is termed spike-timingdependent synaptic plasticity (STDP) (Abbott & Nelson, 2000; Markram, Lübke, Frotscher, & Sakmann, 1997; Bell, Han, Sugawara, & Grant, 1997; Bi & Poo, 1998; Zhang, Tao, Holt, Harris, & Poo, 1998; Debanne, Gahwiler, & Thompson, 1998; Nishiyama, Hong, Mikoshiba, Poo, & Kato, 2000; Froemke & Dan, 2002). Based on the STDP learning rule, it is believed that the precise timings of the spikes of the pre- and postsynaptic neurons are encoded into the patterns of the synaptic weights (Gerstner et al., 1996; Song, Miller, & Abbott, 2000; van Rossum, Turrigiano, & Nelson, 2000; Rubin, Lee, & Sompolinsky, 2001; Gütig, Aharonov, Rotter, & Sompolinsky, 2003). Several studies have reported the functional roles of the STDP in learning, memory (Matsumoto & Okada, 2002; Levy, Horn, Meilijson, & Ruppin, 2001; Rao & Sejnowski, 2001; Song & Abbott, 2001), and the transformation of spike patterns. Generally the transformation of spike patterns is classified into three levels: single-neuron, two-neuron, and neural network levels: Single-neuron level: In the case that a postsynaptic neuron receives random inputs from a considerable number of presynaptic neurons through the STDP synapses, although the interspike interval (ISI) of this neuron is highly irregular, the frequency of the output spikes is more stable than the change in the input spike frequency (Song et al., 2000; Rubin et al., 2001). Two-neuron level: Pulse-coupled Hodgkin-Huxley neurons have a tendency to synchronize through the STDP synapses (Zhigulin, Rabinovich, Huerta, & Abarbanel, 2003). Neural network level: The STDP learning and realistic axonal conduction delays make neurons self-organize into groups and repeatedly generate spike patterns with a millisecond spike-timing precision (Izhikevich, Gally, & Edelman, 2004). A neural network with an inhibitory neuron that suppresses other neurons transforms the applied periodic spatiotemporal patterns into periodic firing patterns (Gerstner, Ritz, & von Hemmen, 1993; Yoshioka, 2001). A recurrent neural network with STDP synapses activated by random external stimuli generates distributed synchronies (Levy et al., 2001). The distributed synchrony is a cyclic firing pattern expressed by some neural clusters. The neurons in each cluster simultaneously fire, which induces the next cluster to fire. A global synchrony is a unison of the

3 STDP Ignites Synfire Chains 417 firings of most of the neurons included in a neural network. In contrast to distributed synchrony, global synchrony does not evoke other synchronies. Masuda and Aihara (2004) reported an interesting finding for distributed synchrony: the higher the shared connectivity in the network, the more synchronous clusters that tend to occur. These studies showed that the STDP controls the output spike rates and generates synchronous spikes or spike patterns depending on the precise spike timings of the pre- and postsynaptic neurons. The spike timings of the neurons in a neural network are determined by the interactions of the neurons. In addition, external stimuli from other networks, for example, sensory organs, the thalamus, or other cortical networks, strongly affect the spike timings. In previous studies, most of the external stimuli were assumed to be random, with no specific structure (Levy et al., 2001; Masuda & Aihara, 2004; Izhikevich et al., 2004). However, it can be reasonably expected that the external stimuli have some structures, for example, spatial, temporal, or both. Thus, it is important to analyze the output spike patterns of the neural network with STDP synapses activating it with spatiotemporal inputs. Although some studies considered the case of an external stimulus with a spatiotemporal structure, they assumed only one inhibitory neuron that globally suppresses all of the neurons under consideration (Gerstner et al., 1993; Yoshioka, 2001). We conducted the simulations with a spiking neural network model comprising partially connected excitatory and inhibitory neurons; the inhibitory neurons had local interactions. As a result, the neural network generates a global synchrony. In other words, the input spatiotemporal pattern is transformed into a global synchrony by the neural network and the STDP synapses. If a synchrony occurs in the brain, it stably propagates through feedforward-layered neural networks (the synfire chain) (Diesmann, Gewaltig, & Aertsen, 1999; Reyes, 2003; Abeles, 1991). However, the source of the synfire chain has been assumed to be the stochastic spikes that occur almost simultaneously in divergent and convergent networks. The occurrence of the synfire chain in such a model is probabilistic. Our result suggests that a neural network with STDP could be a deterministic source of synfire chains. With an assumption that the spiking neural network with STDP learns a pattern whose spatiotemporal structure represents some information, our results suggest that the global synchrony ignites the synfire chain and announces that some information has been applied. 2 Methods 2.1 The Neural Network Model. We used a neural network model comprising leaky integrate-and-fire (LIF) neurons. The membrane potential

4 418 R. Hosaka, O. Araki, and T. Ikeguchi of an LIF neuron is described by C dv j dt = g(v rest V j ) + I j (t), (2.1) where C = 1 µf/cm 2 is the membrane capacitance; V j, membrane potential of the jth neuron; g = 0.05 ms/cm 2, membrane resistance; V rest, resting potential ( 70 mv); and I j (t), the sum of the recurrent and external synaptic current inputs. With these parameters, the time constant of the membrane potential is calculated as τ m = C/g = 20 ms. When the membrane potential exceeds a threshold of 54 mv, the neuron generates an action potential, and the membrane potential is reset to the reset potential, 90 mv. The neural network was composed of 800 excitatory and 200 inhibitory neurons, and the neurons were connected through synapses. Without the inhibitory neurons, the excitatory neurons would infinitely excite each other, and the network would burst. The probability of forming at least one synapse between two pyramidal cells in the cortex is generally estimated as being between 0.1 and 0.3 (Liley & Wright, 1994; Braitenberg & Schüz, 1991). Thus, from this anatomical viewpoint, we used a partially connected neural network in this study. The percentage of connections was 20%, excluding the self-feedback connections. That is, each neuron had 200 synapses. We modeled the excitatory postsynaptic currents and inhibitory postsynaptic currents of the postsynaptic neurons by using the Dirac delta function δ(t). The input to the neuron can be described as follows, I j (t) = N Q i w i j δ ( t ti l ) d i j + IEXT (t), (2.2) i=1 l where N is the number of neurons; w i j, the strength of the synaptic connection from the ith neuron to the jth neuron; ti l, the timing of the lth spike of the ith neuron; and d i j, the delay time of the spike transmission from the ith neuron to the jth neuron. Further, we assumed that Q i = Q EXC = na/cm 2 for the excitatory synapses and Q i = Q INH = 13.5 na/cm 2 for the inhibitory synapses. Each neuron received a suprathreshold stimulus as the external input, I EXT (t). If the external input exists, I EXT (t) = 0.2 µa/cm 2 ; otherwise, I EXT (t) = 0. The strength of the synapse connections, w i j, from the excitatory neurons was initially set to 0.45, while those from the inhibitory neurons were set to 1. Based on the result of Bi and Poo (1998), we applied the STDP rule (described below) only to the excitatory-to-excitatory connections; the other connections were fixed. Although we also attempted simulations that include the STDP learning of excitatory-to-inhibitory connections, the result was found to be qualitatively the same as that in the case without this

5 STDP Ignites Synfire Chains 419 learning. We used the Euler algorithm with a time step of 0.1 ms for the numerical simulations. In this study, synaptic propagation delays were entirely axonal. The period of the synaptic delays was randomly selected to be between 0.1 and 3 ms. 2.2 The STDP Learning Rule. Several studies have reported on the window functions of the STDP learning rule (Song et al., 2000; van Rossum et al., 2000; Rubin et al., 2001; Gütig et al., 2003). Song et al. (2000) modeled a synaptic modification rate that is independent of the present synaptic weight. This modeling is called an additive STDP rule. With this learning rule and random inputs, the distribution of the synaptic weights becomes bimodal. If the synaptic modification is assumed to depend on the present synaptic weight, a unimodal distribution appears (van Rossum et al., 2000; Rubin et al., 2001; Gütig et al., 2003). This modeling is called a multiplicative STDP rule. To cover both rules, we used the window function proposed by Gütig et al., (2003). With this window function, the extent of synaptic weight modification ( w) decreases exponentially with the temporal difference ( t) between the arrival of the presynaptic action potential and the occurrence of the postsynaptic action potential, t = t pre + d pre,post t post, (2.3) where t pre is the spike time of the presynaptic neuron; d pre,post, the delay time of the spike transmission from the presynaptic neuron to the postsynaptic neuron; and t post, the spike time of the postsynaptic neuron. The synaptic weight modification w is then described by the following equations: w( t) = { λf (w) exp( t /τ p ) ( t 0), λf + (w) exp( t /τ d ) ( t < 0), (2.4) where λ = 0.05 is the maximum modification rate. We used the same time constant for potentiation and depression: τ p = τ d = 20 ms (Bi & Poo, 1998). We assumed that the synaptic efficacy is limited to the range [0, 1]. According to Gütig et al. (2003), for the weight updating functions f + (w) and f (w) 0, we used nonlinear functions in which the weight dependence has a power law form with a nonnegative exponent µ, f + (w) = (1 w) µ and f (w) = αw µ, (2.5) where α > 0 denotes the asymmetry between the scales of the potentiation and depression. We set α to 1.05 unless otherwise stated. For µ = 0, the updating functions are independent of the present synaptic weight, which corresponds to the additive STDP rule. The case µ = 1 corresponds to the multiplicative STDP rule. Then the present synaptic weight linearly affects

6 420 R. Hosaka, O. Araki, and T. Ikeguchi Figure 1: Examples of input spatiotemporal patterns. (Left column) The periods are (a) 50 ms, (b) 100 ms, and (c) 200 ms, and the mean ISI is 100 ms. (Right column) The period is 100 ms and the mean ISIs are (d) 50 ms, (e) 100 ms, and (f) 200 ms. the updating functions. For 0 < µ < 1, the updating functions depend on the present synaptic weight in a nonlinear form. (See Gu tig et al., 2003, for further details on this learning rule.) The STDP learning can be implemented in two ways: all possible pairs of pre- and postsynaptic spikes (Song et al., 2000; Froemke & Dan, 2002) or only nearest-neighbor spike pairs (Izhikevich et al., 2004). In this study, we assumed the latter scheme to calculate the weight update. 2.3 External Inputs. We assumed that some important information is represented by the spatiotemporal spike pattern applied to the spiking neural network. For simplicity, independent Poisson-process spike trains were assumed to be the external input spatiotemporal patterns for the neurons. The amplitude of the external input was set to 0.2 µa/cm2 (i.e., IEXT (t) = 0.2 µa/cm2 in equation 2.2). If a neuron at the resting potential receives the external input at this amplitude, the neuron immediately emits a spike. Because we wish to examine the effects of the input information on the output spike patterns through the changes in the synaptic weights, the external input should be more significant than the other random external inputs. Thus, in this study, the input spatiotemporal pattern was applied repeatedly and frequently with period T ms, and the neural network could learn this pattern. Examples of the input stimuli are shown in Figure 1. The

7 STDP Ignites Synfire Chains 421 assumption that the inputs are repeatedly applied is not arbitrary because it is often observed that some spike patterns repetitively exist in long spike trains in the brain (Nadasdy, Hirase, Czurko, Csicsvari, & Buzsaki, 1999; Ikegaya et al., 2004). 3 Simulation Results We observed the output spike sequences of the neurons from the spiking neural network to reveal how the output spike pattern of the spiking neural network is altered by the STDP when the spiking neural network repeatedly receives a specific spatiotemporal pattern as an external input. Unless otherwise stated, the parameter values were as follows. The period and the mean ISI of the input spatiotemporal pattern were 100 ms and 100 ms, respectively, and α and µ in the STDP learning were set to 1.05 and 1, respectively. 3.1 STDP Transformation: A Synchrony in Response to a Spatiotemporal Pattern. Figure 2 shows a typical example of the simulation results. Figure 2a shows the input stimulus. The bar indicates the period (100 ms) of one input pattern. Without the STDP learning, the neural network generated spikes following the input stimulus. Thus, the output was an asynchronous pattern in response to the input (see Figure 2b). In contrast, if the neural network develops with the STDP learning, global synchronies gradually occur (see Figure 2c). Figure 2d shows three enlargements of the raster plot from t = 0 to t = 100 ms (left), t = 1900 to t = 2000 ms (center), and t = 19,900 to t = 20,000 ms (right). The synchrony sharpens as the learning proceeds. Only one synchrony occurs in each period of the input spatiotemporal pattern in this example. We evaluated the number of generated synchronies in the case that the spiking neural network received the input pattern 100 times; the percentages of one synchrony, more than one synchrony, and no synchronies were 99.2%, 0.4%, and 0.4%, respectively, over 1000 simulations. To evaluate the degree of synchrony, we introduced a synchrony coefficient S(t 0 ), S(t 0 ) = max C(t 0 t ; t 0 ), (3.1) t =0,1,...,T 1 C(t; t 0 ) = c Nτ s N i=1 l t+τ s/2 t =t τ s/2 δ ( t ti l ) / 1 NT N i=1 l t 0 t =t 0 T δ ( t t l i ), (3.2) where c is a scaling parameter (c = 0.5 in this study); N = 1000, the number of neurons; T, the period of the input spatiotemporal pattern; ti l, the timing of the lth spike of the ith neuron; and τ s, a temporal window epoch for synchrony evaluation. Even if the observed synchrony is not sufficiently

8 422 R. Hosaka, O. Araki, and T. Ikeguchi Figure 2: A typical example of the output spikes. (a) Input stimulus, which comprises repetition of the input spatiotemporal pattern. The period and the mean ISI of the input spatiotemporal pattern are 100 ms and 100 ms, respectively. The bar indicates the input pattern period T. A raster plot of the output spikes (b) without STDP learning and (c) with STDP learning. (d) Enlargements of c, from t = 0 to t = 100 ms (left), t = 1900 to t = 2000 ms (center), and t = 19,900 to t = 20,000 ms (right). (e) The synchrony coefficients of the original raster plot (filled circles) and the largest synchrony coefficient of the surrogate raster plots (open squares). The synchrony coefficients at t =100 ms are calculated using the raster plot from t = 0 to t = 100 ms, and those at other moments are calculated in the same manner. (f) Same as e, but up to t = 20,000 ms. Temporal changes of (g) mean synaptic weight of all the connections and (h) the phase of the synchrony. (i) Histograms of the synaptic weights of excitatory-to-excitatory connections at t =0, 2500, 5000, 7500, 10,000, 20,000, and 500,000 ms. sharp, the synchrony coefficient defined by equations 3.1 and 3.2 can detect the synchronies because the synchrony coefficient counts the number of spikes by shifting the windows temporally from τs /2 to τs /2. This temporal shift τs should be less than the refractory period of the neuron in order to avoid double-counting of the neuron spikes. In this study, the neuron is reset to 90 mv after a firing and takes approximately 10 ms to reach the

9 STDP Ignites Synfire Chains 423 Figure 3: Monte Carlo significance test. The filled circles indicate the synchrony coefficient of the original raster plot; the open circles, the synchrony coefficients of the surrogate raster plots; and the open square, the largest synchrony coefficient of the surrogate raster plots. In Figures 2e, 2f, 5, 6, and 7, only the largest synchrony coefficients of the surrogate raster plots are displayed. resting potential, 70 mv. Therefore, we set τ s to 5 ms. We evaluated the synchrony coefficient in every input period, that is, t = t 0 = T, 2T, 3T,... (see Figures 2c and 2e). To evaluate the statistical significance of the synchrony coefficient, we adopted the simplified Monte Carlo significance test (Hope, 1968) for the synchrony coefficient because we could not assume the distribution of the coefficients to be normal. Here, we summarize the method of performing the simplified Monte Carlo significance test (see Figure 3). 1. For every input period, we obtained a raster plot. We call this raster plot an original raster plot. 2. We produced a surrogate raster plot by randomly shuffling the temporal indices of the original raster plot. Although the temporal structure of the surrogate raster plot is different from that of the original raster plot, the firing rate of each neuron was maintained. We produced a set of 39 different surrogate raster plots for each original raster plot. 3. We evaluated the synchrony coefficients of the original raster plot and its surrogate raster plots by using equations 3.1 and We obtained = 40 synchrony coefficients by the procedures described above. If the synchrony coefficient of the original raster plot is larger than those of the surrogate raster plots, the synchrony coefficient of the original is considered to be statistically significant, which implies that synchronies exist in the original raster plot. Otherwise we cannot conclude that synchronies are present in the original raster plot. In this study, the synchrony coefficient of the original raster plot

10 424 R. Hosaka, O. Araki, and T. Ikeguchi was not below the distribution of the synchrony coefficients of the surrogate raster plots. Therefore, in the following simulations, we show only the largest synchrony coefficient of the surrogate raster plots. Figure 2e shows the synchrony coefficients of the original raster plot (filled circles) and the largest synchrony coefficients of the surrogate raster plots (open squares) of Figure 2c. In Figure 2c, no synchronies are observed during the first six periods of the input. The corresponding synchrony coefficients of the original raster plots are as low as those of the surrogate raster plots. When the seventh input is applied, a synchrony begins to occur at around t = 645 ms, and the synchrony coefficient begins to increase. This demonstrates that the synchrony coefficient reflects the synchrony. The synchrony coefficient becomes stable in longer simulations (see Figure 2f). In this study, the synchrony coefficients of the input pattern and the output of the neural network without STDP are smaller than the largest synchrony coefficient of the surrogate raster plots. These are not shown in the figures. Figure 2g shows a transition of the mean synaptic weight that includes not only excitatory-to-excitatory connections but also other connections. As the learning proceeds, the mean synaptic weight decreases and converges to As shown in Figure 2i, most of the synaptic weights are divided into two groups at the upper and lower limits. However, some weights remain between these limits. The shape of the synaptic weight distribution depends on µ of the STDP. This is discussed in section 3.4. We define the phase of the synchrony as the relative timing of the synchrony to the input pattern period. The phase is defined by (nt t ) mod T of max C(nT t =0,1,...,T 1 t ; t 0 ), where T is the period of the input pattern, C(t; t 0 ) is defined by equation 3.2 and n = 1, 2,.... The transition of the phase is shown in Figure 2h. The phase of this example is 25.1 ms at t = 20,000 ms. The phase of the synchrony depends on the input spatiotemporal pattern and the neural network structure. The neural network structure includes the pattern of the synaptic weights, the delays, and the type of neurons, that is, excitatory or inhibitory. If the same neural network learns a different input pattern, the synchrony occurs at a different phase. These results imply that the spiking neural network learns the spatiotemporal structure of the input pattern and encodes it as the phase of the synchrony. 3.2 Mechanism of Synchrony Generation. Here, we discuss how synchronies are generated. In this study, the synaptic weights from the excitatory neurons were initially set to With this value for the weights, the excitatory neurons rarely evoke postsynaptic neural firings; thus, the network had a low firing rate at first. If a connection has a causal relationship with firing, it is potentiated by the STDP learning. Due to the periodicity of the input, such potentiated connections are repeatedly potentiated. As a

11 STDP Ignites Synfire Chains 425 result, the neurons come to fire only due to the contributions of the other neurons through the potentiated connections, and the activity of the network increases. The excitatory neurons that receive potentiated synaptic inputs excite not only the excitatory neurons but also the inhibitory neurons. A sharp synchrony occurs if the strong firing of the excitatory neurons excites a sufficient number of inhibitory neurons to temporarily shut down the network activity. The mechanism of synchrony generation is similar to the gamma oscillation generation known as the pyramidal-interneurons network gamma (Whittington, Traub, Kopell, Ermentrout, & Buhl, 2000; Izhikevich, 2006). Although the above mechanism explains synchrony generation, it does not explain how the phase of the synchrony is decided and why multiple synchronies do not occur in every input period. For the former problem, we expected that the input pattern and the network structure would decide the phase. To confirm this hypothesis, we first observed the dependency of the phase of the synchrony on the initial synaptic weights by assigning random numbers between 0 and 1 to the excitatory-to-excitatory connections. If the input pattern and the network structure decide the phase, the initial values of the synaptic weights will not affect the phase. As expected, the emerged phases of the synchrony were the same as those in the case where the initial values were fixed at a constant value (0.45). Second, we show that the pattern of the initial w (see equation 2.4) strongly affects the phase of the synchrony. On repetitive application of the input pattern with a period of 500 ms, the spiking neural network generated the synchrony shown in Figure 4a; here, the initial synaptic weights of the excitatory-to-excitatory connections were equal to If the input pattern is applied once (see Figure 4b), the synaptic weights of the initial condition (see Figure 4h, initial) are slightly changed by the STDP, as shown in Figure 4h ( w). We then modified the synaptic weights; the slight changes in the synaptic weights (see Figure 4h ( w)) are multiplied by 11, as shown in Figure 4h (multiplied by w), which is an operation to adjust the smallest synaptic weight to be equal to zero. If the pattern of the initial w (see Figure 4h ( w)) strongly affects the phase of the synchrony, the neural network with the modified weights (see Figure 4h, multiplied w) would generate a synchrony in phase with that shown in Figure 4a. When the input pattern was reapplied, the modified spiking neural network generated four synchronies per input period (see Figure 4c). As the input pattern was repeatedly applied, three of the initial four synchronies gradually disappeared, and a single synchrony remained (see Figures 4c 4f). The phase of the remaining synchrony corresponded to the phase of the synchrony without the artificial modification (see Figures 4f and 4a), indicating that the pattern of the initial w strongly affected the phase of the synchrony. The pattern of the initial w was decided by the input pattern and the network structure. Thus, the firings resulting from the recurrent neural interactions did not affect the phase of the synchrony. These results indicate that the

12 426 R. Hosaka, O. Araki, and T. Ikeguchi Figure 4: A mechanism for synchrony generation. The period of the input pattern is 500 ms. (a) The raster plot after 100 applications of the input pattern. (b) The raster plot with the initial synaptic weights. (c f) The raster plots with a set of modified synaptic weights from 500 to 1000 ms, 1500 to 2000 ms, 3500 to 4000 ms, and 50,500 to 51,000 ms, respectively. (g) The raster plot after 100 applications of the different input pattern from that in a f without the weight modification. (h) A part of the synaptic weights (50 of the 128,000 excitatoryto-excitatory connections): (initial) initial condition (t = 0 ms); (#w) after onetime application of the input pattern; and (multiplied #w) that obtained by multiplying the difference between (#w) and (initial) with 11. phase of the synchrony is uniquely decided by the input pattern and the neural network structure. In fact, when we applied a different input pattern to the same spiking neural network shown in Figures 4a to 4f, with the same initial synaptic weights, delays, and types of neurons, the network generated a synchrony with a different phase (see Figure 4g).

13 STDP Ignites Synfire Chains 427 a b Figure 5: Each graph shows the synchrony coefficients when (a) the period and (b) the mean ISI of the input patterns are altered. After 100 applications of the input pattern, the input spatiotemporal patterns were reapplied, and the synchrony coefficients were then calculated. For both the original raster plot (filled circles) and the surrogate raster plots (open squares), the points and error bars indicate the mean and standard deviation of the synchrony coefficient calculated over 100 simulations, respectively. Finally, we considered why multiple synchronies do not occur in every input period. In Figure 4c, there were four synchronies at the beginning. However, three of them disappeared, and only a single synchrony remained. This is a result of competition between the synchronies. The synchronies shared the 800 to 900 neurons. However, in most cases, the order of the firings of the neurons was different for different synchronies. Therefore, connections that were potentiated in one synchrony were often depressed in another. If the depression was greater than the potentiation, the connection became smaller and finally reached zero. As a result, only a single synchrony remained. 3.3 Dependency on Input Pattern. In this section, we examine how the synchrony coefficients are altered by the properties of the input pattern. Figure 5a shows the synchrony coefficients when the period of the input spatiotemporal pattern is changed. When the period of the input pattern is less than 30 ms, the synchrony coefficient is below that of the surrogate raster plots; this indicates that the spiking neural network cannot generate synchronies with such an extremely short input pattern. If the period of the input pattern is longer, the synchrony coefficients increase. When the period of the input pattern is close to 1000 ms, the synchrony coefficient peaks. For periods longer than 1000 ms, the synchrony coefficients decrease. Figure 5b shows the synchrony coefficients when the mean ISI of the input pattern is changed. The synchrony coefficient peaks when the mean ISI is close to 100 ms. When the mean ISI is extremely low, every output spike tends to burst, and the synchrony coefficients decrease. And when the mean ISI is

14 428 R. Hosaka, O. Araki, and T. Ikeguchi extremely high, few spikes are observed, and the synchrony coefficients also assume low values. To quantitatively estimate the sensitivity of the response of the network to a specific spatiotemporal pattern, we examined the robustness of the synchrony generation to noise. We considered the following two cases for adding noise: noise is added (1) during the learning (learning period) or (2) after the learning (recalling period). The amount of noise added X% to an input pattern is defined as follows: 1. X% of the spikes are randomly selected from an input pattern. 2. The spike timings of the selected spikes are randomly changed by avoiding overlap with another spike timing. 3. For each input pattern, the above procedures are independently repeated. In the following two experiments, we referred the phase of the synchrony to evaluate whether the network correctly identifies the input pattern. If the phase of the synchrony differed from that of the noiseless input pattern by 10 ms (half value of the time constant τ m ), the synchrony coefficient was set to 0. Figure 6a shows the synchrony coefficients for the noiseless input patterns obtained after the learning with noise. When the noise was less than 40%, the synchrony coefficients were larger than those of the surrogate raster plots. Figure 6b shows the synchrony coefficients for the noisy input patterns obtained after the learning without noise. In contrast to Figure 6a, even if the noise level is 10%, the synchrony coefficients are below that of the surrogate raster plots, that is, an incomplete input pattern cannot be a cue for the synchrony. These results indicate that even if the input pattern is disturbed by low-level noise, the spiking neural network can learn the hidden spatiotemporal structure of the input pattern. In addition, once the spiking neural network has learned the input pattern, it can correctly identify incoming inputs by generating the synchrony in response to the learned input pattern. In addition, we investigated the case that noise is added during both the learning and recalling periods. The results are shown in Figure 6c. Interestingly, these results indicate that when noise is added during recall, the synchrony is recalled better when some noise has also been added during learning than that for noiseless learning. In addition, with appropriate levels of noise (within 5% 20%) during learning and recall, the synchrony generation exhibits relative robustness to increases in noise. It may be that the model implicitly learns to deal with noise during the training phase. Finally, we examined the effects of the repetition of input patterns: Can the spiking neural network learn the input pattern if random patterns are inserted between the input patterns? We found that the application of random patterns between two specific input spatiotemporal patterns

15 STDP Ignites Synfire Chains 429 a b c Figure 6: The same as Figure 5, but the learned spatiotemporal pattern is reapplied after the STDP learning. Noise is added (a) during the learning, (b) during the recalling (after the learning), and (c) during both the learning and recalling. does not significantly affect the results. Even when 200-ms-long random patterns (Poisson process trains) are inserted between the input patterns of T = 100 ms, the synchrony coefficients of the original raster plots exceed those of the surrogate raster plots for 82 out of 100 simulations; that is, the neural network can learn synchrony generation with rates greater than 80%. 3.4 Dependency on the Form of the STDP Function. Figure 7 shows the effects of the STDP rule on the synchrony coefficient. Figure 7a shows the synchrony coefficients when α of the STDP learning is altered. This parameter decides the asymmetry between the scales of the synaptic potentiation and depression. When α is 1.15, the synchrony coefficient peaks. As α is increased, the synchrony coefficient decreases. When α is 2.2, the error bar includes the synchrony coefficient of the surrogate raster plots. Therefore, α should be smaller than 2. On the other hand, α must be larger than 0.83, for otherwise the network bursts.

16 430 R. Hosaka, O. Araki, and T. Ikeguchi a b Figure 7: The same as Figure 5, but (a) α and (b) µ are altered. Several reports have indicated that the two candidates for the STDP learning rule the additive and the multiplicative STDP rules lead to different results (van Rossum et al., 2000; Rubin et al., 2001; Gütig et al., 2003). We investigated the effects of the rules on the synchrony coefficients (see Figure 7b). In Figure 7b, µ = 0 corresponds to the additive STDP rule, and µ = 1 corresponds to the multiplicative STDP rule. As shown in Figure 7b, as µ increases, the synchrony coefficients increase slightly, and the error bars shrink. This implies that the multiplicative STDP rule is more suitable for synchrony generation. However, the fact that the synchrony coefficient assumes a large value regardless of the value of µ indicates that the dependence on the rule is not essential for synchrony generation. It is well known that the synaptic weights with the multiplicative STDP rule driven by random Poisson inputs result in a unimodal distribution of the synaptic weights (van Rossum et al., 2000; Rubin et al., 2001; Gütig et al., 2003). However, in our study, even when the multiplicative STDP rule is adopted, the synaptic weight distribution does not become unimodal but bimodal, even after a significantly long learning time of t = 500,000 ms (see Figure 2i). This is attributed to the repetition of the specific inputs. In the case of the random inputs, the synaptic weights are equally altered by the potentiation and the depression of the STDP. In contrast, in the case of the repetitive inputs, the synaptic weights, which are strengthened by the one-time application of the input pattern, are also strengthened by the successively applied input patterns. As a result, even in the case of the multiplicative STDP rule, the synaptic weights have a bimodal distribution. However, the precise distribution forms are different for the case of the additive and the multiplicative STDP rules. Figure 8 shows the temporal evolution of the histograms of the synaptic weights (see Figure 8a) and outputs (see Figure 8b) of the spiking neural network with the additive STDP rule, where all the other conditions are the same those in Figure 2. If we compare Figures 2i and 8a, we can see that the synaptic weights

17 STDP Ignites Synfire Chains 431 a b Figure 8: (a) Temporal evolution of the distribution of the synaptic weights of excitatory-to-excitatory connections at t = 0, 1000, 1500, 2000, 2500, and 20,000 ms; and (b) the outputs of the spiking neural network with the additive STDP rule (µ = 0). The same parameters and conditions were used as those of the plots in Figure 2. Table 1: Shape of the Distribution of Synaptic Weights Resulting from the STDP Rule and Input Statistics. STDP Rule Inputs Additive Multiplicative Random Complete bimodal (Song et al., Unimodal (van Rossum et al., 2000; 2000) Rubin et al., 2001) Periodic Complete bimodal (in this letter) Incomplete bimodal (in this letter) with the multiplicative STDP rule do not completely split into lower and higher limits; some weights remain between these limits (see Figure 2i); however, the weights with the additive STDP rule do (see Figure 8a). In addition, the synaptic weights modified by the additive STDP rule are split into boundaries faster than in the case of the multiplicative STDP rule. However, the shape of the synchrony and its generation speed in the case of the additive STDP rule are almost the same as those in the case of the multiplicative STDP rule (see Figures 2c and 8b). We summarize the shapes of the histograms of the synaptic weights in Table 1. 4 Discussion From the results of the simulations, we observe that the spiking neural network transforms the input spatiotemporal pattern into a synchrony whose

18 432 R. Hosaka, O. Araki, and T. Ikeguchi timing depends on the input spatiotemporal pattern and the neural network structure. This implies that the neural network can self-organize its transformation function from spatiotemporal to temporal information. The transformation from a spatiotemporal pattern to a synchrony possibly exists in actual neural networks. Synaptic plasticity is usually observed in the cortices and the hippocampus. Thus, this transformation should play an important role in information representation in the cortices and the hippocampus. Although we used a simple spiking neural network in this study, the results are suggestive. In particular, the results raise an important issue: Does a similar mechanism truly exist in our brains? To construct more realistic neural network models, alpha-function-shaped postsynaptic currents should be considered instead of the delta-shaped ones and Hodgkin-Huxley dynamics instead of the leaky integrate-and-fire neuron model. 4.1 Relation to the Synfire Chain. In a synfire chain, synchronous spikes stably propagate through feedforward layered networks (Abeles, Bergman, Margalit, & Vaadia, 1993; Diesmann et al., 1999). The stability of the synfire chain has been ascertained using theoretical and experimental methods (Diesmann et al., 1999; Reyes, 2003). However, little attention has been paid to the mechanism by which the first synchrony triggers the synfire chain. In previous studies, dispersed synchronous spikes have been assumed a priori and have been gradually sharpened through the propagation process in the layered neural network (Reyes, 2003). Our result suggests the presence of a neural network located near the sensory cortex, wherein a synchrony can be generated from a specific input spatiotemporal pattern. In the neural network, the synapses with STDP have already learned the transformation from input sensory information to the synchrony. The neural network generates the synchrony if it detects a previously learned pattern. The propagation speed of a synfire chain is considerably higher than that of the other information representations using mean firing rates because the synfire chain does not need to calculate the temporal average. In other words, information representation by synchronies is indispensable to realize efficient information transmission. Thus, synchronies and synfire chains should be used in regions where fast processing is required. From this viewpoint, a candidate for the source of synfire chains is the connections between the thalamus, cortex, amygdala, and hippocampus. 4.2 Relation to the Phase Coding. Considering the synchrony, the phase, and its property of robustness to noise, we can correlate the functions of the STDP with the phase coding in the hippocampus. In a rodent hippocampus, the phases in the synchronous theta rhythm are thought to represent important information (O Keefe & Recce, 1993). For example, the place cells in CA1 encode the spatiotemporal information of the

19 STDP Ignites Synfire Chains 433 environment using the phases in the theta rhythm. Our results raise the issue that the neural networks in the hippocampus may encode the information applied to the hippocampus into phases through the STDP learning rule. 4.3 Relation to the Distributed Synchrony. We consider that different types of dynamics can operate in a neural network, depending on the properties of the external inputs and the network connections. For example, we can assume two dynamics. One dynamics is strongly affected by a spatiotemporal spike pattern of other networks. Thus, the dynamics is driven by the external inputs. The second dynamics is not affected by a spatiotemporal spike pattern of other networks but is driven by internal interactions (Araki & Aihara, 2001). We consider that the distributed synchrony of Levy et al. (2001) and our global synchrony are the results of the different dynamics. The distributed synchrony can be assumed to be the result of the second dynamics. Hence, the random external input is assumed to be background activity. Our study assumes the first dynamics and shows the occurrence of global synchrony. Our results indicate that the different dynamics in a neural network result in a different pattern transformation. Acknowledgments We thank the anonymous referees for their constructive comments and suggestions. We also thank Kazuyuki Aihara (Institute of Industrial Science, University of Tokyo) for encouragement. This research was partially supported by Grant-in-Aid for Scientific Research on Priority Areas (Advanced Brain Science Project) (No ) from MEXT to O.A. and Grants-in-Aid for Scientific Research (C) (No ) and (B) (No ) from JSPS to T.I. References Abbott, L. F., & Nelson, S. B. (2000). Synaptic plasticity: Taming the beast. Nature Neuroscience, 3, Abeles, M. (1991). Corticonics Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Abeles, M., Bergman, H., Margalit, E., & Vaadia, E. (1993). Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. Journal of Neurophysiology, 70, Aertsen, A., Erb, M., & Palm, G. (1994). Dynamics of functional coupling in the cerebral cortex: An attempt at a model-based interpretation. Physica D, 75, Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal firing correlation: Modulation of effective connectivity. Journal of Neurophysiology, 61,

20 434 R. Hosaka, O. Araki, and T. Ikeguchi Araki, O., & Aihara, K. (2001). Dual information representation with stable firing rates and chaotic spatiotemporal spike patterns in a neural network model. Neural Computation, 13, Bell, C. C., Han, V. Z., Sugawara, Y., & Grant, K. (1997). Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature, 15, Bi, G., & Poo, M. (1998). Synaptic modification in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18, Braitenberg, V., & Schüz, S. (1991). Anatomy of the cortex. Berlin: Springer-Verlag. Debanne, D., Gahwiler, B. H., & Thompson, S. M. (1998). Long-term synaptic plasticity between pairs of individual CA3 cell in rat hippocampal slice cultures. Journal of Physiology, 15, de Oliveira, S. C., Thiele, A., & Hoffmann, K. P. (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. Journal of Neuroscience, 17, Diesmann, M., Gewaltig, M. O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, Froemke, R. C., & Dan, Y. (2002). Spike-timing-dependent synaptic modification induced by natural spike trains. Nature, 416, Fujii, H., Ito, H., Aihara, K., Ichinose, N., & Tsukada, M. (1996). Dynamical cell assembly hypothesis theoretical possibility of spatio-temporal coding in the cortex. Neural Networks, 9, Gerstner, W., Kempter, R., von Hemmen, J., & Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383, Gerstner, W., Ritz, R., & von Hemmen, J. L. (1993). Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biological Cybernetics, 69, Gray, C. M., König, P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature, 338, Gütig, R., Aharonov, R., Rotter, S., & Sompolinsky, H. (2003). Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. Journal of Neuroscience, 23, Hope, A. C. A. (1968). A simplified Monte Carlo significance test procedure. J. R. Statist. Soc. B, 30, Ikegaya, Y., Aaron, G., Cossart, R., Aronov, D., Lampl, I., & Yuste, R. (2004). Synfire chains and cortical songs: Temporal modules of cortical activity. Science, 34, Izhikevich, E. M. (2006). Polychronization: Computation with spikes. Neural Computation, 18, Izhikevich, E. M., Gally, J. A., & Edelman, G. M. (2004). Spike-timing dynamics of neuronal groups. Cerebral Cortex, 14, Levy, N., Horn, D., Meilijson, I., & Ruppin, E. (2001). Distributed synchrony in a cell assembly of spiking neurons. Neural Networks, 14, Liley, D. T. J., & Wright, J. J. (1994). Intracortical connectivity of pyramidal and stellate cells: Estimates of synaptic densities and coupling symmetry. Computation in Neural Systems, 5,

21 STDP Ignites Synfire Chains 435 Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275, Masuda, N., & Aihara, K. (2004). Self-organizing dual coding based on spike-timedependent plasticity. Neural Computation, 16, Matsumoto, N., & Okada, M. (2002). Self-regulation mechanism of temporally asymmetric Hebbian plasticity. Neural Computation, 14, Nadasdy, Z., Hirase, H., Czurko, A., Csicsvari, J., & Buzsaki, G. (1999). Replay and time compression of recurring spike sequences in the hippocampus. Journal of Neuroscience, 1, Nishiyama, M., Hong, K., Mikoshiba, K., Poo, M. P., & Kato, K. (2000). Calcium stores regulate the polarity and input specificity of synaptic modification. Nature, 408, O Keefe, J., & Recce, M. (1993). Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus, 3, Rao, R. P. N., & Sejnowski, T. J. (2001). Spike-timing-dependent Hebbian plasticity as temporal difference learning. Neural Computation, 13, Reyes, A. D. (2003). Synchrony-dependent propagation of firing rate in iteratively constructed networks in vitro. Nature Neuroscience, 6, Riehle, A., Grün, S., Diesmann, M., & Aertsen, A. (1997). Spike synchronization and rate modulation differentially involved in motor cortical function. Science, 278, Rubin, J., Lee, D. D., & Sompolinsky, H. (2001). Equilibrium properties of temporally asymmetric Hebbian plasticity. Physical Review Letters, 86, Song, S., & Abbott, L. F. (2001). Cortical remapping through spike timing-dependent plasticity. Neuron, 32, Song, S., Miller, K. D., & Abbott, L. F. (2000). Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3, Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, van Rossum, M. C. W., Turrigiano, G. G., & Nelson, S. B. (2000). Stable Hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience, 22, Whittington, M. A., Traub, R. D., Kopell, N., Ermentrout, B., & Buhl, E. H. (2000). Inhibition-based rhythms: Experimental and mathematical observations on network dynamics. International Journal of Psychophysiology, 38, Yoshioka, M. (2001). Spike-timing-dependent learning rule to encode spatiotemporal patterns in a network of spiking neurons. Physical Review E, 65, Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., & Poo, M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. Nature, 395(6697), Zhigulin, V. P., Rabinovich, M. I., Huerta, R., & Abarbanel, H. D. I. (2003). Robustness and enhancement of neural synchronization by activity-dependent coupling. Physical Review E, 67, Received September 23, 2005; accepted April 12, 2007.