Processing of the incomplete representation of the visual world Andrzej W. Przybyszewski, and Tomasz M. Rutkowski Dept Psychology McGill University Montreal, Canada Schepens Eye Res Institute, Harvard Medical School Boston, US E-mail: przy@vdn.ca Multimedia Research Lab, Media Center, Kyoto University Kyoto, Japan E-mail: tomek@mm.media.kyoto-u.ac.jp Abstract Our behavior is mostly determined by what do we see. However depend on their position objects may have different meanings. We have analyzed the influence of the eye position on the information processing in the primary visual cortex of an awake monkey. We have compared brain responses to light stimuli in different scales: from large cell populations like EEG, from the local field potentials (LFP) and from single cells. In the power spectra of the EEG and LFP we were looking for changes in the dominant frequency and in the local maxima (ridges). Ridges give us information if and how the system trajectory changes, by bifurcation (B), resonance transition (RT), etc. In most cases action potentials (spikes) generation were associated with the change of the state like B or RT. However, eye position mostly influenced maximum spike frequency light responses. Therefore, eye position has effect on information processing already in the level of the primary visual area. It could be related to the transformation of the coordinates between eye-head-body (frame theory) or gaze modulated expectations. 1 Introduction Our eyes are constantly moving and scanning environment, telling what could happened in the next moment to us, where to move, what to reach. Whenever eyes move the image of the world sweeps across the retina, but our perception remains stable. Moreover, our actions in the environment must be independent on eye position. Thus, information obtained from the retina in the eye- 1
coordinate system must be transformed to the head-coordinate and later to the body-coordinate system. Helmholtz proposed that the copy of the motor commands (efferent copy) have to be sent to the visual areas correcting appropriately information from retinas. More recent experiments in which neurons from different cortical areas were recorded in awake monkeys supported this hypothesis. Andersen and his team [1] proposed that modulation of the posterior parietal neurons by the eye position (EP) means coordinate transformation to non-retinocentric frame of references. Also researchers from other labs observed influence of the eye position on neuronal activity in the lower visual areas from V4, V2, and in the primary visual cortex (V1). Weyland and Malpeli [2] have tested the effect of EP on activity of neurons in V1 in cats. Effects were in 4% cells mostly asymmetric. Trotter and Celebrini [3] found the EP influenced 72% cells studied for disparity and 85% cells studied for orientation in V1 of monkey. Rosenbluth and Altman [4] changed EP in 3D which affected 65% of cells in V1, V2 and V4. They found that EP can change the mean firing rate by a factor of 1. However, the EP affects neuronal activity in the lower cortical areas in the more unclear and complex way than in the parietal cortex. The purpose of this paper is to find what possible mechanisms could be involved in the influence of the EP on the neuronal activity in the primary visual cortex of the awake monkey. Seeing stable world in the right coordinates is essential, but what we see in a single eye fixation gives us very limited information about environment. In most cases in order to perceive and recognize objects one need to make many saccades (fast eye movements) and many eye fixations each with duration between 2 3ms. The questions is: How do we decide where to look next? One hypothesis is that the brain constructs an internal representation of the scene and stimulus location and uses it to move the eyes. Rensink [6] proposed that at first we construct a dynamic virtual representation of a scene which provides coherent structure of objects but without details. Objects are formed as needed by nonattentional seeing and scrutinizing items with visual attention. In order to approach this property of our central nervous system we compared information processing in the brain at different levels of integration: in the EEG, in the local field potentials (LFP) and in the single cell. An important question was which level of integration can tell us about decision where to look next? 2 Methods In experiments conducted to verify our hypothesis of possible influence of eye position on processed information in primary visual area, experimental monkeys were trained to fixate their eyes on direction of a light point (visible light emitting diode - LED) attached to the computer screen that was placed in different horizontal positions in front of monkeys eye: straight or deg ( position), 1deg or 2deg to the right (1R, 2R) or left (1L, 2L). The position of a dominant eye was monitored by a magnetic field search 2
coil. Stimuli were displayed on the computer monitor. There were moving bars optimized for orientation, length, velocity, and color. Bars were changed in contrast and they were brighter or darker than the background of 5cd/m 2. The size and location of a receptive field was estimated with narrow increment and decrement bars swept back and forth across the screen. We have measured contrast responses r(c) in each eye position and fitted them with the Naka-Rushton equation: r(c) = R max (c n /(c n + c n 5)), (1) where R max is the maximum response, c - contrast, c 5 -contrast at the half of R max, n-nonlinearity. We have analyzed only those responses with a sufficiently good fit (estimated by a root mean square - RMS). In our previous paper [5] we found that changing the eye positions significantly influence the maximum amplitude of the response in most of recorded cells. Also, in some cells, responses to lower contrasts changed significantly for different eye positions. Our preliminary data also suggest that the eye position could differently influence the size of the increment and decrement zones in the classical receptive field of V1 cells. In this work we have analyzed the same data as previously but this time we were looking into relationships between single cell responses and population responses like LFP and EEG. 2.1 Signals Analysis The three recorded previously data sets were analyzed separately in the timefrequency domain. The LFP and EEG signals were transformed into spectral domain (spectrograms) with the use of a short time Fourier transformation with sliding analysis window of 5ms. Highly overlapping windows helped to improve the resolution in time and frequency domains. It was a necessary step toward search of the local/sharp maxima of frequency distributions as a function of the time. In order to enhance the spectrograms the reassignment method was proposed [7]. For every value of computed spectrogram x(t, f) a new value x(t, f ) is calculated which is the center of gravity of surrounding signal energy distribution as: Ŝ x (t, f ) = δ + + ( f ˆf(x; t, f) S x (t, f)δ ( t ˆt(x; t, f) ) (2) ) dtdf, where the terms responsible for phase information preservation in STFT are as follows: ˆt(x; t, f) = dφ x(l, f) df (3) ˆf(x; t, f) = f + dφ x(t, f). (4) dt 3
STFT enhanced in such a way, has stronger maxima (peaks) for every time instant. These maxima represent energy distribution localized together with preserved phase information. Enhanced spectrograms are shown in Figures 1. to 4. in second and fourth traces from the top for EEG and LFP respectively. Points of observed bifurcations are labeled with a letter B. In the next step of the time-frequency analysis an identification of ridges was conducted. Ridges are defined as local maxima with respect to the energy density in time-scale plane [9]. The ridges are obtained by searching for the maximum values in enhanced spectrograms from equation 2 for every time slot in analyzed data. The observed resonance transitions are labeled with a letters RT in Figures 1. to 4. Responses of single cells recorded as a train of action potentials (spikes) were analyzed with two methods. Spike histograms were calculated using 1ms bin widths as is shown in fifth panels of Figures 1. to 4. marked as histograms of spike trains. In the second method as proposed by [8] the spike trains were convoluted with so called mexican-hat wavelet functions: Ψ(x) = 2 3 π 1/4 (1 x 2 )e x2 /2, (5) were the widths of wavelet functions were changing in a range of.6 384ms. Obtained in such a way time-frequency distributions give information about local oscillations in the spike trains (sixth trace from the top in Figures 1. to 4. marked as T/F of spike trains). 3 Results As mentioned in the previous sections, the eye position influences mean rate activity of neurons in the primary visual cortex. Figures 1 to 4 illustrate EEG, local field potential (LFP) and single cell activity in three different eye positions (see the bottom traces in every figure): Figure 1. deg (straight) in the horizontal plane; Figure 2. 1deg to the right in the horizontal plane; Figure 4. 2deg to the right in the horizontal plane. It is evident that the eye position has influence on a single cell spike activity. In this case turning eye more to the right causes stronger decrease in spikes activity, but it is not true for all V1 cells [5]. In all figures traces three and four from the top represent the local field potentials and their time-frequency changes - ridges (see Section 2. Methods). Notice that the changes in the dominating frequency (RT - resonance transition) or in the frequency spectrum (B-bifurcation) of the LFP correlate, in many cases, with spikes or bursts in the recorded cell. Also in many cases transitions in the spectrum of EEG correlate with transitions in LFP. Ridges of the LFP change significantly with the eye position (compare different figures). Analysis of the trace four in Figures 1.- 4 shows that when eye moves more to the right, the population activity increases its complexity. More careful analysis shows the complex behavior of the system. Let look in Figure 2 how the ridges of the LFP and spike rate change during period 22 to 3ms from 4
S EEG S LFP EEG LFP [sp/s] wf step [marc] 2 2 x 1 4 1 B 5 x 1 4 1 1 1 5 5 4 3 2 1 2 15 1 5 5 3b2 9trial #16 ridge plot of EEG RT RT RT B RT RT B RT RT RT RT B B RT B ridge plot of LFP RT RT RT RT RT B RT RT B RT RT RT B RT RT RT B B RT RT B RT histograms of spike trains T/F of spike trains eye position 5 1 15 5 1 15 2 3 35 4 45 5 time [ms] Figure 1: deg eye position example. From the top are: EEG, ridges plot of EEG, LFP, ridges plot of LFP, histogram of spike trains, T/F representation of spike trains as in equation (5) and eye position plot at the bottom. 5
S EEG EEG 2 2 1 5 x 1 4 RT 3b2 17trial #16 B RT B RT B ridge plot of EEG B B RT B RT RT RT B LFP S LFP spikes/s 5 5 1 RT RT RT RT B RT B 5 histograms of spike trains 5 4 3 2 1 T/F of spike trains 2 15 1 5 eye position 6 4 2 5 1 15 2 3 35 4 45 5 time [ms] wf step [marc] ridge plot of LFP Figure 2: 1deg-right eye position example. From the top are: EEG, ridges plot of EEG, LFP, ridges plot of LFP, histogram of spike trains, T/F representation of spike trains as in equation (5) and eye position plot at the bottom. 6
1 ridge plot of EEG S EEG 5 1 ridge plot of LFP S LFP 5 22 24 26 28 3 time [ms] Figure 3: 1deg-right eye position example. Magnified ridge plots as in Figure 2. for time range 27 3ms. 7
EEG S EEG LFP S LFP 5 5 spikes/s 1 5 5 5 wf step [marc] 1 5 5 4 3 2 1 2 15 1 5 1 5 B 3b2 27trial #11 ridge plot of EEG RT RT RT B RT RT B B RT RT B RT B RT RT B RT B RT RT RT B B RT RT ridge plot of LFP RT RT RT B RT RT B RT RT B histograms of spike trains T/F of spike trains eye position 5 1 15 2 3 35 4 45 5 time [ms] Figure 4: 2deg-right eye position example. From the top are: EEG, ridges plot of EEG, LFP, ridges plot of LFP, histogram of spike trains, T/F representation of spike trains as in equation (5) and eye position plot at the bottom. B 8
the trial beginning (for m oredetailsseefigure reffigfigure2mgn.thefirst spike burst is correlates with the transition in the main frequency and bifurcation in the LFP. After short period, around the time = ms, ridges show several main frequency transitions in the range between 1 and 3Hz. Normally one would assume that there are sharp frequency transitions: low frequency disappears and the high frequency suddenly appears. On the basis of the ridges analysis we can say that both low and high frequencies exist before and after the resonance transition (RT). Low and high frequencies only changed their amplitudes causing jumps of the main frequency. This new characterization of resonance transition has not been considered in the EEG analysis. Also notice, that resonance transitions were correlated with bursts of spikes. After another 2ms again RTs from high to low and back frequency appear, during this time spikes show bursts. The second RT follows shortly by bifurcations. Using standard frequency analysis with long window probably no significant changes in the main frequency would be observed during this period. Also it is worth to notice apparent correlations between RT and B in the LFP and in the EEG plots. 4 Conclusions Changing the eye position influences evoked neuronal activity in the primary visual cortex of awake monkey. As illustrated above, eye position introduced division of the neuronal activity to many basins of attractors. This effect could be related to the attention. It also suggests that the dimension of the population activity increases and the system is more flexible to process a new or a more complex stimuli. This effect can be also interpreted that the signal from V1 to higher areas is very variable and small when monkey looks more to the right. In other words there is no significant stimuli over there. Therefore the next eye movement will be in different direction: straight or to the left. If looking in certain direction would give high spike rate and synchronous LFP (we have also recorded such cells) than normally monkey would turn her head in this direction in order to check objects more carefully. This mechanism is sub-attentional, and it seems to be similar to the effect of attention causing increase of the single cell activity and synchronization between cells. References [1] Andersen R.A., and Buneo C. A.: Sensorimotor integration in posterior parietal cortex, In: Adv. Neurol. 23;93:159-77 [2] Weyand T.G., and Malpeli J.G.: Responses of neurons in primary visual cortex are modulated by eye position, In: Journal of Neurophysiology 1993 Jun;69(6):28-6. 9
[3] Trotter Y., and Celebrini S.: Gaze direction controls response gain in primary visual-cortex neurons. In: Nature 1999;398(6724):239-42. [4] Rosenbluth D., and Allman J.M.: The effect of gaze angle and fixation distance on the responses of neurons in V1, V2, and V4. In: Neuron. 22 33(1):143-9. [5] Przybyszewski A.W., Kagan I., and Snodderly M.: Eye position influences contrast responses in V1 of alert monkey. Journal of Vision,3(9): 398 [6] Rensink R.A.: Seeing, sensing, and scrutinizing. In: Vision Research 2;4:1469-87. [7] Auger F. and Flandrin P.: The why and how of time-frequency reassignment, In: IEEE International Symposium on Time-Frequency and Time- Scale Analysis, Philadelphia, 197 2. [8] Przybyszewski A.W.: An analysis of the oscilatory patterns in the central nervous system with wavelet method, In: Journal of Neuroscience Methods, 28:(1991):247-7 [9] Chandre C., Wiggins S., and Uzer T.: Time-frequency analysis of chaotic systems, In: Physica D, 181:(23):171-196 1