1 Empirical Evidence about Temporal Structure in Multi-unit Recordings

Transcription

1 Villa A.E.P., Empirical evidence about temporal structure in multi-unit recordings, pp. 1-61, in:robert Miller (Ed.),, 432 p. d Harwood Academic Publishers ( ) 1 Empirical Evidence about Temporal Structure in Multi-unit Recordings Alessandro E.P.Villa The brain is a highly interconnected network of neurones, in which the activity in any neurone is necessarily related to the combined activity in the neurones that are afferent to it. Due to the widespread presence of reciprocal connections between brain areas, reentrant activity through chains of neurones is likely to occur. Certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements as a consequence of developmental and/or learning processes. In cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring, ordered, and precise (in the order of few ms) interspike interval relationships are referred to as spatiotemporal patterns of discharges. This term encompasses both their precision in time, and the fact that they can occur across different neurones, even recorded from separate electrodes. This chapter introduces the fundamental assumptions and algorithms that lead to the detection of complex patterns of neural discharges and introduces a way of interpreting this activity within the framework of non-linear dynamics. Empirical results of experimental and simulation studies are provided in different sections of the chapter. KEYWORDS: Spatiotemporal firing patterns; Neural dynamics; Brain theory; Time code; Frequency code; Multi-unit recordings; Non-linear dynamics; Sensorimotor association 1. BRIEF HISTORICAL INTRODUCTION In the physiologist Albrecht von Haller published in Göttingen an historical essay, the Dissertation on the Irritable and Sensitive Parts of Animals (original title: ). This work was based on numerous experiments of vivisection, and on stimulation of organs using the new knowledge

2 Time and the brain 2 offered to physiology by physics, chemistry and natural history. With a rudimentary technique of stimulation, Von Haller classified the parts as irritable, sensible or elastic and noted that the reactions varied between different parts of the brain. The historical importance of the work by Von Haller is not so much related to the results obtained, but rather in systematically developing a transdisciplinary approach to brain research using the new technologies of his time. In 1791 the Italian physician Luigi Galvani started the publication of a remarkable series of studies that demonstrated muscle twitch in a frog by touching its nerves with electrostatically charged metal, and later using two dissimilar uncharged metals. These observations led Galvani to postulate that the circulation of a particular body fluid, that exists naturally in the nerves in a state of disequilibrium, provided the stimuli for the muscle fibres to contract. In addition, normal muscular contraction without a source of electrostatic electricity was, in Galvani s view, evidence for the existence of an additional and natural form of electricity, that he called animal electricity. This latter statement set the scene for the famous Galvani-Volta controversy. Alessandro Volta, a friend of Galvani and an Italian physicist who, in 1775, invented the electrophorus a device to generate static electricity gave an opposed and experimentally valid explanation of Galvani s experiment. The electricity did not come from the animal tissue but was generated by the contact of different metals, brass and iron, in a moist environment. The interest in this controversy resides in the confrontation of two basic, irreducible interpretations of the same observation, derived from each scientist s different background: Galvani saw the frog phenomenon as the work of biological organs, Volta as that of a physical apparatus. The outcome of the controversy was exceptional. On the one hand the challenge of Volta s opinion led Galvani to perform a new series of experiments that demonstrated muscular contraction by touching the exposed muscle of one frog with the nerve of another frog, thus showing for the first time the existence of bioelectric forces. On the other hand, Volta focused his research efforts upon the study of electric fluids between dissimilar metals, and in 1800 he presented the first electric battery, providing future researchers with a stable source of electricity not dependent on electrostatic forces. With the introduction of currents of Galvanic fluids into the brain, a way was opened to making new discoveries in physiology at the beginning of the XIXth century. Electricity was used not just as an experimental intervention, which formed the basis of electrophysiology (the study of the connection between living organisms and electricity), but in addition, the proper characteristics of propagation and generation of this type of energy became the basis of fertile hypotheses. In the last quarter of the XIXth century, Eduard Hitzig and Gustav Fritsch discovered the localization of cortical motor areas in the dog, using electrical stimulation, and Richard Caton was the first to record electrical activity from the brain. Electrophysiology started to develop rapidly, and Edgar D.Adrian published a seminal study suggesting the all-or-none principle in nerve (1912). In the late 1920s Hans Berger in Germany demonstrated the first human electroencephalogram and opened the way to clinical applications of electrophysiology. Nevertheless, the English School led investigations in electrophysiology in the first part of our century, and for his specific research on the function of neurones, Adrian shared the 1932 Nobel Prize for Medicine with Sir Charles Sherrington. Although most remembered for his scientific contributions to neurophysiology, Sherrington s research focused on spinal reflexes as

3 Empirical evidence about temporal structure 3 well as on the physiology of perception, reaction and behaviour. His approach was transdisciplinary, in the sense that disciplines were used not only one next to the other, but were really intermingled in his protocols, an extraordinary example of a nonreductionist view of neurophysiology. At present, and still based upon the work performed by Adrian (1934), most neurophysiologists make their deductions by observing mean frequencies of nervous discharges (spikes), i.e. whether there are a lot or only a few spikes over a relatively prolonged time interval. Important concepts and findings have been clarified by using the overall mean rate as a measure of neuronal activity, at both peripheral and central levels of the nervous system. In particular it is important to remind ourselves of the influence of the findings of Mountcastle (1963) indicating that the relationship between frequency of firing of selected neurones in the ventrobasal thalamus of deeply anaesthetized monkeys and the angle of extension of the contralateral knee was a perfect power function. Hence, the relationship between and is virtually a perfect linear function. Although this point will not be developed further in the present chapter, it is important to note that the effect, and the type of anaesthesia determine dramatically the neuronal discharge pattern, especially at thalamic and cortical levels (e.g., see Mukhametov 1970; Zurita 1994). This effect has not always been adequately taken into account by investigators eager to develop theoretical models of neuromimetic circuits. The slow integration time of nerve cells, operating in the range of a few milliseconds, roughly a million times slower than presently available supercomputers, and the huge number of connections established by a single neurone (for review, see Braitenberg and Schüz, 1991) has suggested that information in the nervous system might be transmitted by simultaneous discharge of a large set of neurones. Multiple dimensions of stimuli relevant to sensory function and behaviour are processed by thousands of neurones distributed over many areas of the brain. Indeed, the hypothesis that neurones (both individually and jointly) process information over time, and following precise time relationships, has pervaded the neurosciences ever since the nervous system was conceptualized as a set of dynamic networks of interacting neurones (McCulloch and Pitts, 1943; for review, see MacGregor, 1987). That neurones convey a temporal code has been well-known since electrophysiological studies in the 1960s led to the recognition that spike trains the time series formed by the sequences of time intervals between spikes were related to meaningful physiological variables (Bullock, 1961; Segundo 1963; Perkel and Bullock, 1965; Segundo 1966; Nafe, 1968). Subsequent work made an important contribution, by recognizing temporal coding in different experimental circumstances and animal species, and by proposing worthwhile quantification procedures (Perkel 1967a,b; Segundo and Perkel, 1969; Legéndy, 1975; Eckhorn 1976; Brillinger 1976; Klemm and Sherry, 1981; Abeles, 1982; Tsukada 1982; Rosenberg 1989; Bialek 1991; Rapp 1994; Friston, 1995; Martignon 1995; Rieke 1995; de Ruyter van Steveninck 1997). This provided new insights into how the association between simultaneously recorded spike trains, in the time and frequency domains, may reflect the degree of coordinated activity within a cell assembly. The integrative capabilities of neurones (for review, see Segundo, 1986) on the one hand, and parallel processing and redundancy

4 Time and the brain 4 inherent to neural pathways on the other hand, have become more obvious during the past two decades, thanks also to the advances in neuroanatomical methods using retrograde and/or anterograde tracers (e.g. horseradish peroxidase, fluorescent dyes, biocytin, etc.). However, for most functional systems the available electrophysiological data demonstrate merely the presence of temporally organized neural activity, so that during recent years investigations have mainly focused on establishing causal relations between the occurrence of precise temporal relationships and cognitive or motor processes (Fetz, 1997). Therefore, two hundred years after the Galvani-Volta controversy, two main classes of theories explaining information processing in the brain have been proposed: In one, neurones convey a precise temporal code (Abeles, 1982; Abeles, 1991), while the other is based on noisy rate coding (Shadlen and Newsome, 1994). According to the rationale of a the question has been raised whether the variability of spike intervals carries information. This variability may depend on the role of the decay of the post-synaptic potential (PSP) in determining the prevailing operating mode of network processing (König 1996). Softky and Koch (1993) have shown that, in order to find the same degree of variability seen coincidence detection with fast integration of small excitatory PSPs is required in models. Alternatively, Shadlen and Newsome (1994) have proposed that the balance between inhibition and excitation plays a critical role in the variability of network behaviour with membrane time constants of 8 20 ms. If an irregular interspike interval results from integration of excitatory and inhibitory postsynaptic potentials, then the timing of postsynaptic spikes is random and can no longer reflect the timing of presynaptic events. Precise patterns of spikes their intervals and coincidences would fail to propagate (Shadlen and Newsome, 1994). König (1996) have provided a summary of these two viewpoints and proposed that the interplay of excitation and inhibition (E-I) effectively reduces the time constant for synaptic integration, thus providing coincidence detection in the cortex, although the actual synaptic decay is relatively long. Indeed, a balance condition of excitatory and inhibitory post-synaptic potentials plays an important role in the activity of many network models (Wilson and Cowan, 1974; Douglas and Martin, 1991; Tsodyks and Sejnowski, 1995; Usher and Stemmler, 1995; Xing and Gerstein, 1996) and has many complex spatial and temporal influences (Thomson and Deuchars, 1994). Changes in cellular excitability modify the time to firing, thus altering the E-I dynamics. Some specific examples of time-varying factors influencing the balance of E-I levels are the cell threshold potential, PSP kinetics, receptor activity such as NMDA, and calcium currents (Edmonds 1995). The inability to preserve information about the time of a spike suggests that the discharge rate cannot be modulated in a time-locked fashion to specific inputs. In a random walk model (Gerstein and Mandelbrot, 1964) this assumption may be wrong because the discharge rate can follow the activity of inputs, but, given a base rate, the time to the next spike would appear as random. Then, the average instantaneous discharge rate through an ensemble of neurones belonging to a functional assembly would be capable of transmitting changes in spike rate with a precision of ms (Shadlen and Newsome, 1998). Recordings performed in several cortical areas of behaving monkeys support the hypothesis of a connection between fluctuations in neural discharge rate and behaviour (Georgopoulos 1993; Celebrini and Newsome, 1994). However, the existence of the rate code

5 Empirical evidence about temporal structure 5 mechanism does not imply that temporal codes do not also exist in the brain. The cerebral cortex is a highly interconnected network of neurones, in which the activity of each cell is necessarily related to the combined activity in the neurones that are afferent to it. Due to the presence of reciprocal connections between cortical areas, reentrant activity occurs, through chains of neurones. Furthermore, certain pathways through the network may be favoured by inhomogeneity in the number or efficacy of synaptic interactions between the neural elements, as a consequence of developmental and/or learning processes. According to the rationale of a in cell assemblies interconnected in this way, some ordered sequences of intervals within spike trains of individual neurones, and across spike trains recorded from different neurones, will recur. Such recurring and ordered interspike intervals, with a precision of the order of a few milliseconds, are referred to as spatiotemporal patterns of discharges. For true demonstration, such temporal firing patterns must occur to a statistically significant level (see Figure 1). If functional correlates of spatiotemporal neural coding exist, one would expect that whenever the same information is presented, the same temporal pattern of firing would be observed. Several lines of evidence exist showing Spatiotemporal patterns (Bair and Koch, 1996) and (Mainen and Sejnowski, 1995). Recent studies on active propagation of action potentials in dendrites have provided additional results supporting the existence of precise neuronal timing (Stuart and Sakmann, 1994). A synaptic response increases if the presynaptic spike precedes the postsynaptic spike, but if the order is reversed, the synaptic response decreases. The window for synaptic the order of 150 ms (Thorpe et al., 1996). Taking into account the firing rates of neurones in the visual pathway, only few spikes at most can be generated by each neurone in performing this highly demanding task, thus providing strong arguments for the importance of precise temporal coding. Indeed, electrophysiological experiments in primates have established that correlated firing between single neurones recorded simultaneously in the frontal cortex may evolve within tens of milliseconds, in systematic relation to behavioural events, without modulation of the firing rates (Vaadia et al., 1995). Furthermore, in multiple electrode recordings performed in the primary motor cortex of monkeys trained in a delay-pointing task, spike synchronization occurred in relation to purely internal events (stimulus expectancy), during which modulations of firing rate were distinctly absent (Riehle et al., 1997). These findings provide an important support for the general view that a stronger synaptic influence is exerted by multiple converging neurones firing in coincidence, thus making synchrony of firing ideally suited for highlighting responses and for expressing relations among neurones with high temporal precision.

6 Time and the brain 6 Figure 1. (a) (b) (c)

7 Empirical evidence about temporal structure 7 2. DETECTION OF SPATIOTEMPORAL FIRING PATTERNS An influential and remarkable model based on the assumptions of high temporal precision in brain processing is the synfire chain hypothesis. This model suggests how precise timing can be sustained in the central nervous system by means of feed-forward chains of convergent/divergent links and re-entrant loops between interacting neurones forming an assembly (Abeles, 1982). Structures like synfire chains may exhibit attractors in which a group of neurones excite themselves, maintaining elevated firing rates for long periods and allowing the same neurone to participate in many different synfire chains. A fundamental prediction of such a model is that simultaneous recording of activity of cells belonging to the same assembly and involved repeatedly in the same process should be able to reveal repeated occurrences of such spatiotemporal firing patterns. Note that the term firing pattern encompasses both their precision in time, and the fact that they can occur across different neurones, even when recorded from separate electrodes. The following example (Figure 2) illustrates schematically the recurrence of patterns in a network connected according to the synfire model. A chain of 56 model neurones is formed by 8 sets of 7 neurones connected by diverging/ converging links. This means one neurone in a set (in black in Figure 2a) receives 7 inputs from the previous set and projects to all 7 neurones of the next set. In the example shown in Figure 2a the links are subdivided into 5 excitatory (open circle) and 2 inhibitory connections (gray circles). This network is embedded into a larger network (200 neurones) with random connections. Note that the last link of the chain is connected to the first link. The model neurones follow simple integrate-and-fire dynamics (Villa and Tetko, 1995; Hill and Villa, 1997). Each neurone integrates all post-synaptic potentials. If the integrated depolarization passes the threshold, the neurone fires, becoming refractory, whereas if firing does not take place, the membrane potential tends to return to the resting level. Synaptic strengths and time constants have been selected in order to allow spontaneous activity to propagate but also to prevent epileptic spikes to self-generate in a simulation run. Figure 2b shows 28 successive frames of the neuronal chain activity, in arbitrary time units. Let us assume that three selected neurones, i.e. c10, c36, and c46, are recorded by virtual electrodes. The corresponding spike trains are shown in Figure 2c, and a firing pattern formed by four spikes was detected in these spike trains. More generally, let the list of cell labels that appear in a pattern of c spikes be noted as S c =(i 1,...,i j,...i c ). In this, any label of the recorded neurones can be assigned to i j. In particular, for a 4-tuple pattern of spikes (quadruplet) obtained from a simultaneous recording of ten spike trains, let us assume that we find S 4 =(c36, c36, c10, c46), meaning all patterns of complexity c equal to 4, formed by a spike recorded from cell c36, followed by another event of cell c36, then a spike recorded from cell c10, which in turn is followed by a spike recorded from cell c46. One specific example of such patterns with delays of 6 time units between the first two spikes from c36, 2 time units between c36 and c10 and 5 time units between c10 and c46 is notated as <c36, c36, c10, c46; 6, 8, 11> (see Figure 2). In addition, this notation assumes that the same time jitter was found for all events forming the pattern.

8 Time and the brain 8 Figure 2. Example of spatiotemporal pattern detected in a simulated chain of diverging/converging links. (a) Basic structure of one set of the chain, formed by 7 neurones. One neurone in a set (black circle) receives 5 excitatory (white circles and solid lines) and 2 inhibitory (gray circles and dashed lines) inputs from the previous chain. The output projections of one neurone on the cells of the next set follow the same pattern. (b) The results of the simulation of the activity in a network formed by eight successive sets from time 61 to 88 in arbitrary time steps. The large dots indicate that a neurone is firing. The events belonging to the pattern detected in panel (c) are enclosed by squares, (c) A significant pattern formed by four cells is detected. The pattern was: cell 36; 6 time units (t.u.) later cell 36 again; 8 t.u. later cell 10 and 11 t.u. later cell 46.

9 Empirical evidence about temporal structure 9 An apparent weak point of the synfire theory is the requirement for neural mechanisms able to support a precise timing of spike patterns even after large time delays, and in the presence of various neuromodulators. For instance, a prominent effect of cholinergic modulation is to reduce adaptation of spike frequency. This results in an increase of neuronal excitability and a shortening of inter-spike intervals, so that the overall effect of such modulation can be a modification of spike timing. Differential actions of acetylcholine on the excitability of two subtypes of inhibitory cortical interneurones exist, so that cortical cholinergic activation may change the direction o information flow within cortical circuits (Xiang 1998). Despite this, patch-clamp experiments in rat neocortical slices have shown that timing of spikes produced in response to fluctuating current injections may be preserved during cholinergic modulation (Tang 1997). Transmission across neocortical synapses depends on the frequency of presynaptic activity and studies of synaptic depression have demonstrated that different aspects of the firing patterns of their afferents are transmitted depending on the average presynaptic frequency (Abbott 1997; Tsodyks and Markram, 1997). When a single sensory stimulus drives many neurones to fire at elevated rates, the spikes of these neurones may become tightly synchronized, which could be involved in binding together individual firing-rate feature representations into a unified object ercept. However, the response to elevated rates of firing may be weak, if any, and opulation coding based on relative spike timing can systematically signal stimulus features following the stimulus time course even where mean firing rate does not change (decharms and Merzenich, 1996). These properties determine the fact that little information about steady-state frequency is transferred across synapses. This result is in contradiction with the rate code hypothesis, assuming that a higher input rate corresponds to a higher output rate. In this respect, information theory applied to the coding problem in the frog and house fly (Bialek and Rieke 1992; Rieke 1995) also suggests tha spike timing is important for reliable information transfer from receptors to brain, and may explain the subsequent behavioural events. In summary, several lines of evidence are in favour of precise temporal activity in the brain, but the development of analytical methods to detect reliably the transient temporal relations in sequences of spike intervals therefore represents a critical step to establish their function as a temporal code. One proposed technique aimed at detecting favoured patterns i.e. patterns that occur more often than can reasonably be expected at random consists of identifying template patterns even if the detailed timing of the patterns varies slightly (Dayhoff and Gerstein, 1983a). The advantages of this method, referred to as Favoured Pattern Detection (FPD), is that it can be used to detect other favoured patterns whose occurrences may have extra or missing spikes, and the firing patterns can be tested for significance by Monte Carlo algorithms. Unfortunately, this technique can be applied only for analysis of temporal patterns of activity generated by a single neurone, and for fixed jitter i.e. the fluctuation in temporal precision allowed for each spike. Moreover, the choice for settings, e.g. the threshold level parameter, may have discouraged its use among electrophysiologists, and limited its application to experimental data (Dayhoff and Gerstein, 1983b).

10 Time and the brain 10 The use of multiple microelectrode recordings, and of large artificial neural networks of spiking neurones has challenged investigators to search for methods allowing the analysis of higher complexity firing patterns, formed by several spike trains and involving several events within a pattern. An approach to the exact calculation of the robability of randomly obtaining each individual recurring pattern was proposed fo three and more distinct spike trains (Frostig 1984, 1990). In this method the statistical evaluation is based on the use of a 2 2 contingency tables and the application of Fisher s exact test. At the first step, the detection stage, all possible intervals between three spike trains are tested for significance, and at a second step, the expansion stage, all significant patterns found previously are checked for higher-order associations among spike trains. The main limitations of this method consist of its inability to detect recurring atterns involving less than three different spike trains and patterns of more than three spikes, if not all subpatterns formed by a triplet of spikes are statistically significant. Several investigators have pointed out that in most experimental preparations presently available the chance of observing patterns of discharges of higher-order complexity is very low, and specific analytical methods aimed at analyzing patterns formed by three spikes (triplets) were developed. The principles of three-fold correlation among spikes are illustrated in Figure 3. Suppose that the firing times of three neurones A, B, and C are recorded simultaneously (Figure 3a). One could ask whether the firing of cells B and C depends on the time elapsed from a spike of cell A. Let us consider the time delays t AB of the discharges of cell B after neurone A fired and similarly the time delays t AC. The joint distribution histogram may be constructed as shown in Figure 3b. At each discharge of cell A the spike train of neurone B is plotted in the X-axis and the spike train of neurone C is plotted in the Y-axis of the graph. The crossings of columns and rows, falling within a defined bin size, that correspond to the spike trains of neurones B and C recorded simultaneously are counted and a Z-axis, perpendicular to the plan of drawing, is plotted. The value of the compound rate plotted on this Z-axis may be coded in shades of gray (or on a colour scale). A graphic possibility for plotting all compound joint distributions between three neurones is based on a triangular coordinate system because the three variables t AB, t AC, t CB, are not independent (since t AB = t AC + t CB ). The X-Y graph can be skewed by 60 (Figure 3b, right panel) so that all three joint distributions can be plotted together (Figure 3c). A practical application of this method in practice is shown in Figure 3d. In this example the data were recorded from the thal amus o f an ana es thet ized c a t during spontaneous activity (Villa, 1988, 1990). Cell 1 (discharging at a rate of 2.8 spikes/s) was recorded in the posterior group of the thalamus, cell 3 (1.7 spikes/s) in the dorsal nucleus of the medial geniculate body and cell 7 (5.7 spikes/s) in the brachium of the inferior colliculus. A black triangular bin indicates a significant event corresponding to a spike of cell 3, then 8 ms later a spike from cell 7 and a spike of cell 1 occurring 116 ms after the onset of the pattern. The time accuracy of this estimation was limited to 15 ms so that spikes of cells 7 and 1 might be considered synchronous.

11 Empirical evidence about temporal structure 11 Detection of triplets was first achieved using the snowflake triple-spike renewal histogram (Perkel 1975; Abeles, 1983). However, triple-spike renewal histograms do not provide a reliable estimation of the significance of the detected patterns, and to solve this problem the method was developed further into the Joint PeriStimulus Time Histogram (JPSTH) (Aertsen 1989). A number of applications of this technique to Figure 3. Outline of the general procedure followed to study three-fold correlation among spikes, (a) Three spike trains, labelled A, B and C, are recorded simultaneously. The time of occurrence of spikes A are successively taken as onset time for calculation of delays of simultaneously recorded spikes of cell B (t AB ) and C (t AC ). Three

12 Time and the brain 12 markers indicate three successive spikes of cell A. (b) A plot of all possible patterns formed by B and C spikes as a function of the A spikes marked on the top panel is presented in form of a X-Y plot where the abscissa represents the delays of B spikes after A firing and the ordinate the delays of C spikes after A firing. The left panel shows the same plot with the Y-axis skewed by 60. (c) A three-cell correlogram is constructed by joining three graphs in order to represent all possible time relations among the firing of three neurones. The three intervals (t AB, t AC, t CB ) associated with any point on the graph can be read by projecting the point onto the three external axes, (d) The activity of three neurones, labelled 1, 3 and 7, were recorded from three different electrodes in the cat thalamus. A significant pattern is detected as shown by the black triangular bin. The pattern was: cell 3; 8 ms later cell 7 and 108 ms later cell 1. The shades of gray correspond to the number of counts per bin on a relative scale. the study of simultaneously recorded spike trains in behaving monkeys have been reported (Abeles 1993b,c). Since then, the algorithm has been improved fo statistical evaluation, in order to take into account the existence of significant crosscorrelation between cells. Analyses of multiple spike trains recorded in behaving monkeys using this most recent algorithm, referred to as Joint Triplet Histogram (JTH), have been published recently (Prut 1998). A careful analysis of the compound activity of three-cells may suggest interesting hypotheses about functional connections between distinct nuclei. Let us consider the result illustrated in Figure 3d. One could argue that cells in the brachium of the inferior colliculus and cells in the dorsal nucleus of the medial geniculate body receive common afferent activation from the ascending auditory pathway, and synchronous activation of these nuclei is easy to accept. However, the cross-correlation between the neurones recorded in these nuclei was flat, as indicated y the absence of dark diagonal bands centered on time zero in the snowflake plot, and the quasi-synchronous activation of cells 3 and 7 was only detected when a spike in cell 1 occurred more than 100 ms later. This delay cannot be due to direct connections through the thalamus (according to the present knowledge of thalamic organization). The observation of a complex pattern of firing in different thalamic nuclei suggests that these anatomical regions may form distributed cell assemblies that are functionally connected by some kind of reverberating activity, probably involving reentrant activity from the cerebral cortex. In chains of diverging/converging links there is no need for each of the synaptic connections to be particularly strong, since they become effective through coactivation with others. Then, several synapses can depolarize the postsynaptic cell to reach its firing threshold, by spatial and temporal summation. In the case of pure excitatory feed-forward chains of neurones (Abeles, 1982, 1991) it has been demonstrated that a strong stimulation (e.g. input fibres firing at a high asynchronous rate) applied to all the cells of a set in the chain would elicit a spike in all of them synchronously, so that the next set of neurones down the chain would also be activated synchronously. This synchronous activity

13 Empirical evidence about temporal structure 13 would then propagate along the whole chain, with a limited time jitter. Thus, interconversions from firing rate code to synchronous activity, and back, can be performed by a chain of diverging/converging links. This observation is related to the origin of the synfire chain model (Abeles, 1982). If these chains include excitatory and inhibitory links, the analysis of the propagating activity is not so simple. In addition when one considers that distinct sets of the chain can be activated independently, then the picture of a propagating wave becomes less clear. However, these chains will tend to produce unitary events, i.e. precise spike synchronization between neurones, which occur significantly more often than expected by chance. The temporal spacing between the spikes should not be regarded as corresponding to the relative location of the parent cells along the chain. Such characteristics of the model are illustrated by Figure 4. In this example the very same simulation results as in Figure 2 are considered, but different cells were sampled by the virtual electrodes. It is important to note that cells involved in unitary events would not necessarily show signs of synchronization in cross-correlograms, because they do not receive either common excitatory inputs or common inhibitory inputs. In Figure 4a it can be seen that, due to the presence of other inputs in some occurrences of the temporal pattern, a spike belonging to the pattern may be missing because a cell is in a refractory state. However, due to the peculiar properties of diverging/converging links the pattern is sustained by the other neurones and may persist in the network. An increasing number of studies are now investigating unitary events and preliminary experimental evidence has been reported (Riehle et al., 1997; Prut et al., 1998). It is therefore possible that unitary events in the primary motor cortex are related to pattern generation, for instance as starting points for delayed synchronous patterned activity related to the precise activation of sequences of muscles involved in motor output. 3. EMPIRICAL EXAMPLES The Pattern Detection Algorithm (PDA) developed by Abeles and Gerstein (1988) allows one to perform a comprehensive search for firing patterns and to test if there is a statistical excess of patterns. According to the complexity of the patterns to be analyzed,

14 Time and the brain 14 Figure 4. Example of a unitary event detected in a simulated chain of diverging/converging links, (a) Two events corresponding to the synchronous activation of cells 1, 17, 23 and 33 have been detected, plus an event corresponding to the synchronous activation of cells 1, 17 and 33. (b) Same simulation results of Figure 2b. The large dots indicate that a neurone is firing. The events belonging to the unitary events are enclosed by squares.

15 Empirical evidence about temporal structure 15 over the expected number is estimated by { }= where represents the incomplete gamma function (Press 1994). This estimation is simplified to {1 } = 1- if only one firing pattern is detected. Let us fix the level of significance to be = For example, if the number of patterns detected by PDA is Z=6, and only one pattern (i.e., =1) is expected by chance, then a significant excess of detected patterns is observed because {6, 1} <. However, this estimation cannot indicate which ones out of the six detected patterns are significant. In some cases PDA allows one to detect an excess of patterns when all sequences of spike intervals are significant. Consider an example where the expected number of patterns is as low as X= If one pattern is detected, its significance is equal to {1, 0.001} < thus indicating that the pattern is highly significant. If in the above example we detect six patterns instead of one, then any of these six can be considered as highly significant. (In such a case each of the six pattern can be considered significant by itself.) Routine studies using the classical PDA have demonstrated that this optimal case is exceptional. Most often, patterns that could be significant by themselves under appropriate values of jitter are detected by PDA as distinct patterns if the search is performed with maximum accuracy, i.e. with time jitter equal to 1 ms. The application of PDA to several experimental models, and to recordings from various brain areas, has revealed that complex Spatiotemporal firing patterns do indeed occur (Vaadia 1989; Villa and Abeles, 1990; Abeles 1993a; Villa and Fuster, 1992). In these studies the algorithm was usually set to find any repeating pattern of three or more spikes, provided that the entire pattern lasted not more than 900 ms at most, and was repeated with accuracy of 1 ms. Since high frequency bursts (200 spikes/s or more) may produce patterns which relate to intracellular processes, burst filtering should be performed. A filtering procedure was originally proposed by Abeles and Gerstein (1988). A good filtering algorithm should provide a minimal loss of information, for an acceptable rate of errors in the estimation method. This would require fine tuning of this procedure that includes several combinations of three independent parameters: the time window for burst detection, the time window and the number of counts in high-frequency filtering. An alternative filtering method based only on one parameter, the filtering frequency, has been recently proposed (Tetko and Villa, 1997a). This method, with filtering frequency ranging between 200 and 400 spikes/s, provides better estimates of the number of patterns, for a comparable loss of information. A record was considered as having an excess of repeating firing patterns only when the probability of finding as much (or more) repetitions by chance was less than the significance level Significant patterns that repeated exactly, more than four times, may be used as templates for searching approximate patterns with jitter of ±5 ms. Figure 5 shows an example of the activity of six neurones recorded in the cat thalamus during spontaneous activity (Villa, 1990; Villa and Abeles, 1990). It illustrates a pattern of 3 spikes, repeating 25 times, starting with spike 2, then after 89 ms spike 4, and then after 305 ms spike 5. On the left panel (Figure 5a) the rasters are aligned on pattern start and on the right panel (Figure 5b) the corresponding histograms (we may call peri-pattern histograms) are computed. In addition to the pattern itself, lasting hundreds of ms and formed by spikes recorded from different electrodes, this example is important because it shows an additional feature. The histogram of Figure 5b shows that two bursty neurones, cells 1 and 6, do not

16 Time and the brain 16 Figure 5. (a) Raster display of the activities of six neurones during activity recorded in the auditory thalamus of the cat. The rasters are aligned by displaying the first spike in the pattern at time 5. The units were recorded by four microelectrodes. The first, in the discharge in correspondence to the last spike of the triplet (cell 5 at 394 ms), but both cells show a significant increased tendency to discharge, in bursts, near 410 ms after the pattern onset. Therefore we could say that Figure 5 illustrates in fact a very complex pattern that should be noted. This is true, but no algorithms are yet available to detect such a pattern because the firing of the bursty cells

17 Empirical evidence about temporal structure 17 is only loosely, yet significantly, synchronized with the pattern onset. Furthermore, the selective inhibition of cells 1 and 6 at the same time as the third event of the triplet represents a val ua ble pie ce of i nformation, related to the highly precise temporal processing occurring in the thalamus. Unfortunately, complex temporal firing patterns mixing spike occurrences and missing spikes are not detected by the usual algorithms. This is partly due to the enormous increase in computations required by taking recursively the negative of one spike train with all other spike trains recorded s imult aneous ly, and partly by the fact that it is not correct to simply take the negative. However, there is no doubt that selective absence of spikes at specific timings would contribute to a better understanding the temporal processing in the brain, and it is reasonable to forecast that in the next few years methods will appear to deal with this problem. 4. PATTERN RECURRENCE IN RELATION TO STIMULI AND RECORDING CONDITION The next two examples illustrate some relations of firing patterns elicited by sensory stimuli, in the sense that they would not be observed during spontaneous activity (Villa 1988; Villa 1991). Recordings were made in the auditory thalamus of the cat. Figure 6 shows a triplet, <2, 3, 1; 49, 83> occurring selectively during the presentation of a stimulus which was a white noise burst lasting 200 ms, delivered to both ears. The firing pattern always started after the stimulus onset, but it always ended before the stimulus offset. However, the pattern was not time locked with the stimulus onset, as clearly shown by the irregular alignment of the stimulus onset and the start of the pattern. Similarly, a pattern could be elicited by a stimulus, but occurring after the stimulus offset and without a tight locking to it. Such an example is illustrated by the triplet in Figure 7, repeating 12 times, starting by spike 3, then after 285 ms spike 1 and then after 189 ms spike 2. Note that spikes 1 and 2 were recorded from the same electrode, thus indicating that these cells lay within a distance of few tens of microns of each other. Nevertheless the delay between these correlated discharges was near to 200 ms, and suggested long reentrant loops from the cortex to the thalamic reticular nucleus or, alternatively, from the substantia nigra and/or from modulatory mesencephalic nuclei, that are known to project to this thalamic region. In order to establish the significance of firing patterns as signposts of distributed network activity related to specific brain processes, it is crucial to demonstrate that whenever the network is working in the same state, the same temporal pattern of firing is observed. Several above-mentioned examples have suggested that cortical activity may affect the recurrent activity in the thalamus, as shown by several corticofugal studies, either experimental (Villa 1991; Payne 1996; Villa 1999b) or theoretical (Tetko and Villa, 1997b). The next examples report data recorded in the auditory thalamus of the rat during reversible cortical deactivation by cooling (Villa 1999b). This type of study illustrates an additional problem that may arise in studies of temporal coding. The number of spikes necessary for a fair statistical evaluation of the

18 Time and the brain 18 Figure 6. Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.0 mm. The lowermost raster shows the onset of the stimulus, as a thick tick, and the gray bars represent the time of stimulus duration. The rasters are aligned by displaying the first spike in the pattern <2,3,1; 49,83> at time 0. The pattern is repeated 19 times with a jitter of ±3 ms. The abscissa full scale is 600 ms. significance of the patterns may not be met in all data-sets because the experimental protocol, in this case the cortical cooling, may produce a massive decrease in firing rates of selected single units, or alternatively a massive increase of discharges, and a major change in the pattern of discharges (i.e. from regular to bursty). In case of a decrease in firing rate a partial methodological compensation might be obtained by increasing the recording time. Figure 8 shows an example of a pattern formed by three spikes belonging to two single units recorded simultaneously from two different electrodes in the rat auditory thalamus. The pattern is formed by cell 7 firing, then after 350 ms by a discharge of cell 11 and then after 21 ms cell 11 firing again. This exact pattern was used as template for searching approximate matching patterns with a jitter of ±2ms. Note that several spikes may occur in between the spikes belonging to the pattern without affecting the precise intervals separating the occurrences of the cell discharges. In this example, the significant pattern was observed 9 times in 400 s of spontaneous activity recorded during the control condition. No patterns were observed in 300 s of recording during cooling, but the very

19 Empirical evidence about temporal structure 19 Figure 7. Raster display of the activities of three thalamic neurones: cells 1 and 2 recorded from the same electrode, in the auditory part of the reticular thalamic nucleus, and cell 3 from a different electrode, in the ventral division of the medial geniculate body. The distance between the two electrode tips was 4.4 mm. The rasters are aligned by displaying the first spike in the pattern <3,1,2; 285,474> at time 0. The pattern is repeated 12 times with a jitter of ±2 ms. The abscissa full scale is 1000 ms. same precise pattern was observed 5 times in 300 s during the recovery period. Note that between the first and the last occurrences of the pattern more than 90 minutes had passed. Another example (Figure 9) illustrates that significant patterns observed during cortical cooling might be completely absent prior to and after cortical deactivation. This case shows a pattern of 3 spikes repeating 9 times (with a jitter of ±2ms) starting by a spike of cell 8, followed by a spike of cell ms later, then after 280 ms by a spike of cell 6. Note that only cells 5 and 6 were recorded from the same electrode. This second example also illustrates the tendency observed in firing patterns during cortical cooling to exhibit a higher complexity than patterns observed during control and recovery conditions (Villa and Abeles, 1990). 5. PATTERNS IN RELATION TO BEHAVIOURAL STATES Although the previous examples have provided solid evidence for the existence of precise temporal structure in neural activity, this does not yet establish its function as a temporal code. What is needed is some demonstration that spatiotemporal firing patterns occur

20 Time and the brain 20 Figure 8. Spatiotemporal pattern of spikes detected in the rat auditory thalamus 9 times prior to and 5 times after recovery from cooling of the auditory cortex, but absent during cortical deactivation. The firing pattern was formed by three spikes, starting by spike 7, then after 359 ms spike 11 and then after 21 ms spike 11 again. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated by thick ticks in the raster displays (right panels) aligned by displaying the first spike in the pattern at time 0. Note that spikes 7 and 11 were recorded from different electrodes. The analyzed records corresponded to 400 s of spontaneous activity during control condition, 300 s during cortical cooling and 300 s in the recovery condition. Each block of 100 s is represented by a thick mark along the time axis of the experimental protocol, and the dots indicate the start event of the pattern. reliably under particular behavioural conditions, and are related to cognitive activity. The following example shows experimental results recorded in the primate inferotemporal cortex, which is considered one of the last in an ascending hierarchy of cortical processing stages that begins in the striate cortex. Studies with behaving monkeys have led to the conclusion that the inferotemporal cortex is not only a higher order stimulusanalyzer, but is also involved in retention of visual information, and is modulated by

21 Empirical evidence about temporal structure 21 Figure 9. Triplet of spikes detected 9 times in the rat auditory thalamus, exclusively during cortical deactivation, starting by spike 8, then after 180 ms spike 5 and then after 280 ms spike 6. The pattern is repeated with a jitter of ±2 ms. The spike occurrences corresponding to the pattern are indicated as in Figure 8. Note that only spikes 5 and 6 were recorded from the same electrode. The analyzed records corresponded to 300 s of spontaneous activity during control condition, 500 s during cortical cooling and 300 s in the recovery condition. attention (Fuster, 1990). In the data presented here, the monkey performed a visual delayed matching-to-sample task (Villa and Fuster, 1992). A trial in one such task consisted of the following: (1) sample stimulus (a colour or a geometric figure on a luminescent button with a diameter of 25 mm); (2) seconds of delay (retention period); and (3) choice of one stimulus (among 2 or 4) that matched the sample by a given feature (colour or symbol). Correct responses were reinforced with a squirt of juice. About 40% of all single units recorded in the inferotemporal cortex showed a sustained elevation of firing during the delay after the sample stimulus (which was retained in short-term memory for performance of the task). This sample was further subdivided into selective units (14% of all units) showing an activation during the delay which was selectively and significantly higher after one particular tested stimulus, and non-selective units (26%) which were unspecifically activated by all tested stimuli.

22 Time and the brain 22 Figure 10. Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of two non-selective units recorded in the primate inferotemporal cortex during intertrial intervals (left panels) and during the delay (right panels). The measuring time was equal in both conditions. Cell 14 shows a pattern of 3 spikes repeating 6 and 19 times during the intertrial and delay periods, respectively: The second spike of the triplet occurred 165 ms after the pattern start and the third 195 ms after the second spike. Cell 47 shows a triplet repeating 7 and 19 times during the intertrial and delay periods, respectively. The first spike of cell 47 occurred at time 0 (aligning the rasters), the second 42 ms later, and the third 48 ms later. In these rasters the pattern is allowed an accuracy of ±3 ms.

23 Empirical evidence about temporal structure 23 The distribution of firing rates for both groups in spontaneous discharge was skewed and broad, but the median firing rate for non-selective units (4.4 spikes/s) was higher than that for selective units (2.5 spikes/s). In this study, only one spike train at a time was recorded and thus the temporal firing patterns were formed by multiple spikes of one neurone. Figure 10 illustrates two patterns formed by three spikes observed in two different neurones. The triplet of cell 14 repeated 6 times during the intertrial periods, which did not reach statistical significance, whereas its repetitions were significant (19 times out of 68 trials) during the delay period. The accuracy of the pattern was ±3 ms. The triplet of cell 47 repeated 7 and 19 times during the intertrial and delay periods, respectively, out of 75 trials. One pattern involved long intervals, of the order of hundreds of ms, whereas the other pattern involved relatively short intervals. However, the striking similarity between these two patterns is that both patterns appeared to a significant extent during the retention period of the sample stimulus, and that only one occurrence of the pattern (seldom two) was observed per trial (although in only 20 25% of all trials). In spike trains recorded in the inferotemporal cortex it was difficult to observe firing patterns composed of 4 6 spikes recurring more than twice within a record. Figure 11 shows an example of such a complex pattern formed by spikes of a selective unit during intertrial periods, and during the delay after the preferred stimulus (the only periods characterized by an increase in firing rate). The pattern was composed of 4 spikes: a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. This quadruplet repeated 6 times during the intertrial periods. The four sub-patterns corresponding to the triplets included in the original quadruplet are also displayed in Figure 11. The incidence of the highly organized pattern decreased when the cell was activated by its preferred stimulus, irrespective of an increase in the rate of discharges during the delay. However, it is important to note that one specific subpattern, i.e. <61, 61, 61; 380, 683>, was detected at a statisticallysignificant level during the intertrial and delay periods. This triplet occurred 5 and 9 times during the intertrial and delay periods, respectively. The example of Figure 11 is important, because it illustrates several additional characteristics of precisely timestructured activity in the cortex. Firstly, it illustrates the contribution of subpatterns to a higher order pattern, thus suggesting the robustness of a complex pattern even if at some occurrences a spike is missing. Secondly, it illustrates that subpatterns may express information processing different from that involving the superpattern (as shown by the disappearance of the event at a delay of 95 ms in Figure 11, and at the same time a relative increase in occurrences of the triplet <61, 61, 61; 380, 683>). On a more general level, the examples of Figures raise an interesting problem. Notably, the incidence of firing patterns tended to be inversely related to the selectivity of the units for the stimulus that the animal held in short-term memory (Villa and Fuster, 1992). During the delay periods, and in comparison with intertrial periods, selective units showed a decrease of firing patterns, whereas non-selective units showed an increase. On the one hand, the more patterned activity of non-selective units may reflect their involvement in wide networks representing multiple and general attributes of that particular stimulus held in memory. On the other hand the term selective unit shows the long-held belief and prejudicial influence of the rate code on our assumption of what is selective.

24 Time and the brain 24 Figure 11. Raster display (aligned by displaying the first spike in the pattern at time 0) of the activity of a selective unit (cell 61) recorded in the primate inferotemporal cortex during intertrial intervals (25 lines) and during the delay (17 lines). The pattern was composed of 4 spikes, a spike at time 0, the second 95 ms later, the third 285 ms later, and the fourth and last spike 303 ms later. The occurrences of the exact subpatterns formed by 3 out of 4 spikes of the original pattern are also displayed on this raster. The quadruplet is allowed an accuracy of ±3 ms, whereas the subpatterns ±1 ms. Note that during the delay the average firing rate of the cell increases, while the quadruplet pattern disappears, although one particular subpattern <61, 61, 61; 380; 693> remains visible. In fact, selective units were on the basis of an increased firing rate during the delay period that followed a particular stimulus of the tested set. It is not possible to rule out the possibility that in another paradigm or experimental condition the selective units would become non-selective and. Thus, one should bear in mind that the term selective should be restricted to a particular experimental situation. However, it is interesting to note that both populations of cells increased their rate of discharges during the retention period, but only the non-selective units increased the patterned activity. An attractive interpretation of these data could be the following: to retain all stimulus characteristics, some transfer of information occurs between the selective units, which encode the specific stimulus feature being retained, and the non-selective units, which are part of a wider network encoding contextual attributes of the stimulus that may be used for binding with other features. This model supports the hypothesis of conversion from asynchronous activity to synchronous activity (and back) through diverging/converging

25 Empirical evidence about temporal structure 25 connections, as proposed by Abeles (1982, 1991). One important limitation of the PDA method is its application to patterns as explained above. An additional limitation is the use of a fixed jitter. A new method, called Pattern Grouping Algorithm (PGA), has recently been developed, aiming to identify and evaluate the statistical significance of temporal patterns of spikes formed by three or more different events (Villa and Tetko, 1999; Tetko and Villa, 1999). In particular, this new algorithm is able to recognize patterns of spikes with slight differences in spike timing, and to cluster them as one single group of patterns. Firstly, PGA considers those patterns detected by the classic pattern detection algorithm as templates for search of similar patterns in the spike train data. There are three adjustable parameters in PGA: (i) the maximal duration of the pattern, measured as a delay between the first and the last spike in the sequence of spikes (i.e. the window duration); (ii) the level of significance to be used for detection of significant groups; and (iii) the maximum allowed jitter for timing accuracy of time delays in a pattern group. Based on the idea developed in the template search method (Dayhoff and Gerstein, 1983a) and on selection of optimal jitter, PGA is able to minimize the jitters allowed for each spike in the template independently, according to the actual distribution of spikes. Thus, the probability of detection of significant patterns of spikes is optimized. PGA is not restricted to the analysis of patterns formed by only one cell, and may be applied to the identification of precise temporal patterns regardless of their complexity and number of different cells in a pattern. Therefore, PGA can estimate the significance of each spatiotemporal pattern of firing. The method (described in detail in Tetko and Villa, 1999) consists of an independent estimation by the previously developed PDA, FPD and JTH methods, that have been updated in order to consider the variable jitters. Only those patterns that are significant by all methods are considered for the final step of PGA. Eventually, the final output of PGA consists of a central template pattern with variable jitters corresponding to previously detected patterns grouped into one class. Several tests and validation analyses have demonstrated that PGA can detect true patterns and avoid the detection of spurious patterns, even in the presence of several non-stationarities in the spike trains. These non-stationarities are likely to occur in recordings from behaving and freely-moving animals. Even so, it must be kept in mind that for most of the functional systems the available data demonstrate the presence of precisely timed neural activity whose relationship with behaviour is not always evident. To prove that these temporal patterns do indeed serve for response selection, data are required that establish causal relations between the occurrence of precise temporal relationships and cognitive or motor processes. The next examples will provide solid evidence about these processes. Spike trains from up to 15 single neurones were recorded simultaneously in the temporal cortex of freely moving rats, while animals waited for acoustic cues in a Go/ NoGo task (Villa et al., 1999c). The auditory stimulus contained two types of information: pitch ( high or low ), and position ( left or right ). During the first phase of training, the location was kept constant and the rats had to discriminate between tones of high and low pitch, with one signalling Go and the other NoGo. Reinforcement was given only after correct Go trials. In the second phase, pairs of tones were delivered (one tone from each location), with four possible tone-position combinations. We have shown that, despite the lack of reward for correct NoGo

26 Time and the brain 26 performance, and the lack of painful punishment for incorrect trials, all rats learn the task to a satisfactory level (performance in the range 70 90%). The paradigm allows analysis of activity during two functional epochs: the wait period prior to the stimulus, during which the animal must remain at the back of the cage, and restrain responding despite an increasing probability of imminent stimulus delivery; and the processing time, from stimulus delivery until beginning of movement, during which the information content of the signal is processed and motor output organized. Only the wait period is considered here for analysis. Significant patterns were detected using PGA from a data-set of 13 hours of recording, involving over one million spikes (Villa 1998a, 1999a). Go responses resulted both from correct movements to the feeder in response to low pitch sound at the right speaker and from incorrect movements when a high pitch was delivered to this speaker. Conversely, NoGo responses could be correct (failure to move in response to the high pitch from the right speaker) or incorrect (failure to move in response to low pitch from right speaker). Of part icula r interes t were the pat terns t hat were significantly associated with the type of the animal made later, independent of whether the response was that prompted by the cue. Figure 12 illustrates one such pattern, a triplet formed by the spikes of two cells recorded from distinct electrodes placed in the temporal cortex within the same hemisphere. Note that there were more than twice as many trials (n=27) with patterns that were followed by Go responses as there were trials (n=12) with patterns followed by NoGo responses, despite the fact that the total number of Go and NoGo responses was nearly the same (n=283 and 269, respectively). An alternative display of the spikes belonging to the triplets <5,11,5 ; 320±3, 662±3> is shown in Figure 12b. In this illustration, the rasters are aligned to an external event (the stimulus onset) instead of being aligned to the start of the pattern. Of these behaviour-predicting patterns, half were associated with an enhanced tendency to Go in response to the stimulus, and for about 20% of these patterns, trials including the pattern were associated with a faster or slower reaction time than those lacking the pattern (Villa 1998a, 1999a). Figure 13 illustrates one such pattern, and the corresponding reaction times. The triplet shown here is formed by three different neurones, two of which were recorded from the same electrode. It is important to notice that the patterns were associated in almost half (22/37) of the Go responses and that pattern occurrence during the wait period led to an accelerated reaction time, by 355 ms on average. Although the patterns could start at any time during the waiting period, we observed that in several cases the time course of pattern appearance was related to the reaction time. Figure 14 illustrates this finding on a cumulative display of reaction time vs. timing of pattern occurrence. The reaction times of the subjects were normalized to 800 ms on average, in order to cumulate eleven examples on the same curves. Notice that the individual timing of each pattern was by itself significantly correlated with the reaction time on a quadratic regression analysis (R>0.7 or R<-0.7, <0.05). The left panel of Figure 14 shows that the earlier the pattern occurred in the waiting period, the slower the reaction time. A pattern occurring just before the stimulus presentation elicited a fast reaction time, thus suggesting a kind of matching operation. The right panel of Figure 14 illustrates the opposite case, where the occurrence of the pattern just before the stimulus provoked a slower reaction.

27 Empirical evidence about temporal structure 27 Figure 12. (a) Raster display of the activities of two neurones recorded from two different electrodes in the temporal cortex of a freely-moving behaving rat. The rasters are aligned by displaying the first spike in the pattern at time 0. The pattern was: cell 5; 320±3 ms later cell 11, then 342±3 ms later cell 5 again. The data shown include trials recorded on two consecutive days and in which both Go (n=27) and NoGo (n=12) responses occurred, (b) Raster display of the data corresponding only to the Go responses shown in (a) but on a compressed time scale, and aligned to the time of stimulus onset instead of to time of the first spike of the pattern. Spikes involved in generating instances of the pattern during the waiting period (time to the left of the stimulus onset) are displayed as bars instead of ticks. Three spikes constituting one instance of the pattern are picked out by empty circles. Note the stability of this pattern recorded over two consecutive days.

28 Time and the brain 28 Figure 13. Raster display of the activities of three rat cortical neurones participating in a spatiotemporal firing pattern, aligned by the occurrence of the first spike in the pattern. The time of stimulus presentation is marked by horizontal ticks in the lower window. The pattern was formed by spike #12, 867±2 ms later cell #11, and then after 29±1 ms cell #5. The timing of the three spikes forming the pattern is indicated by the arrows below the stimulus raster. Note that this pattern repeated 22 times (p<0.001) the onset (i.e. during the waiting period) of a stimulus which triggered a Go response (either correct or incorrect). On average, the subject responded significantly faster after stimulus onset if a pattern was detected in the waiting period.