Dynamics of letter string perception in the human occipitotemporal cortex

Transcription

1 Brain (1999), 122, Dynamics of letter string perception in the human occipitotemporal cortex A. Tarkiainen, 1 P. Helenius, 1 P. C. Hansen, 2 P. L. Cornelissen 3 and R. Salmelin 1 1 Brain Research Unit, Low Temperature Laboratory, Correspondence to: Antti Tarkiainen, Brain Research Unit, Helsinki University of Technology, Finland, 2 Physiology Low Temperature Laboratory, Helsinki University of Department, Oxford University and 3 Psychology Technology, PO Box 2200, Fin HUT, Finland Department, Newcastle University, Newcastle upon Tyne, UK att@neuro.hut.fi Summary The inferior occipitotemporal brain areas, especially in ventral visual stream, extending laterally as far as V4v. the left hemisphere, have been shown to be involved in This response was systematically modulated by noise the processing of written words and letter strings. This but was insensitive to the stimulus content, suggesting processing probably occurs within 200 ms after involvement in early visual analysis. The second pattern presentation of the letter string. It has also been suggested took place ~150 ms after stimulus onset and was that this activation may differ between fluent and dyslexic concentrated in the inferior occipitotemporal region with readers. Using whole-head magnetoencephalography, we left-hemisphere dominance. This activation showed a studied the spatiotemporal dynamics of brain processes preference for letter strings, and its strength and timing evoked by visually presented letter strings in 12 healthy correlated with the speed at which the subjects were able adult subjects. Our achromatic stimuli consisted of to read words aloud. The third pattern also occurred in rectangular patches in which single letters, two-letter the time window ~150 ms after stimulus onset, but syllables, four-letter words, or symbol strings of equal originated mainly in the right occipital area. Like the length were embedded and to which variable noise was second pattern, it was modulated by string length, but added. This manipulation dissociated three different showed no preference for letters compared with symbols. response patterns. The first of these patterns took place The present data strongly support the special role of ~100 ms after stimulus onset, originated in areas the left inferior occipitotemporal cortex in visual word surrounding the V1 cortex and was distributed along the processing within 200 ms after stimulus onset. Keywords: visual cortex; letter string; magnetoencephalography; word reading; stimulus degradation Abbreviations: BA Brodmann area; ECD equivalent current dipole; EOG electro-oculogram; fmri functional MRI; MEG magnetoencephalography; SQUID superconducting quantum interference device Introduction The importance of the lateral occipital cortex for processing experimental conditions are used to locate the brain areas visually presented words and letters has been revealed by that are presumably associated with particular cognitive lesion studies. For example, left occipital lesions can cause subprocesses. However, neither technique has the temporal pure alexia (Ajax, 1967; Damasio and Damasio, 1983; resolution necessary to uncover the time course of events Henderson, 1986). Further evidence for letter string and/or within the neuronal networks. word-specific processing in the occipital cortex has been In trying to understand visual word recognition, the ability obtained from functional imaging studies, using PET to follow in time the sequence of cortical events is particularly (Petersen et al., 1988, 1989, 1990; Price et al., 1994, 1996; helpful. For example, one might expect to find that brain Rumsey et al., 1997) and functional MRI (fmri) (Puce et al., areas responsible for the processing of letter strings are active 1996; Pugh et al., 1996). However, some studies have failed before those areas that are involved in lexical semantic to show this kind of letter string-specific occipital activity processing. Accordingly, Nobre and colleagues and Allison (e.g. Howard et al., 1992; Bookheimer et al., 1995). and colleagues performed intracranial recordings that PET and fmri measure the changes in blood flow that are unequivocally identified responses to letter strings in the triggered by activity within large populations of neurons. inferior temporal sulcus/fusiform gyrus bilaterally (Allison Subtractions of the flow patterns observed under different et al., 1994; Nobre et al., 1994). According to these studies, Oxford University Press 1999

2 2120 A. Tarkiainen et al. letter string-specific activation peaked ~ ms after (one of 25 different letters) or geometrical symbols (a stimulus onset, and was followed ~200 ms later by diamond, triangle or square); (iii) two-element stimuli: two- semantically sensitive activation in the medial temporal areas. letter Finnish syllables (25 different syllables) or two-item Magnetoencephalography (MEG) is well suited for studies symbol strings (four different combinations of two symbols: of language processing because it allows non-invasive brain a circle, diamond, triangle or square); and (iv) four-element recordings in neurologically normal subjects with good spatial stimuli: four-letter Finnish words (50 different words) or resolution and excellent temporal resolution. Salmelin and four-item symbol strings (four different combinations of four colleagues and Kuriki and colleagues found MEG responses symbols). All word stimuli were common Finnish nouns, consistent with word/letter string-specific tuning in the e.g. RAHA (money), TALO (house) and VELI (brother). occipitotemporal/extrastriate cortex (Salmelin et al., 1996; Letters/syllables/words were embedded in one of four Kuriki et al., 1998). Since the timing of these responses was levels of Gaussian noise labelled 0, 8, 16 or 24 (for details similar to that reported by Nobre and colleagues in epileptic see Appendix I). Symbol strings were always presented patients (Nobre et al., 1994), it is plausible that both the without noise (level 0), and served as controls for letter MEG studies and the intracranial recordings detected signals strings of equivalent length. Figure 1A shows examples of generated in the same cortical areas. the different stimuli. The crucial role of occipitotemporal areas in word The noise levels were selected so that at zero (level 0) or recognition is further supported by observations in subjects low (level 8) noise subjects could see letters, syllables and with developmental disorders of reading. Salmelin and words easily. Moderate noise (level 16) made identification colleagues found diminished and delayed left of letter strings more difficult and high noise (level 24) made occipitotemporal activation in developmentally dyslexic it extremely difficult. subjects during silent reading compared with healthy control In two subjects we repeated word measurements with both normal and distorted text in which we perturbed letter spacing subjects (Salmelin et al., 1996). Their data suggest that left and orientation. Although the distorted text looked wordoccipitotemporal mechanisms may be critical for fluent, like, these novel symbol strings were not readable in the automatized word recognition. If so, it is important to same way as words. Figure 1B illustrates the appearance of characterize such responses in both time and space, because the distorted letter strings. they may provide an objective means for evaluating reading abilities. As MEG allows the detection of small changes in the Magnetoencephalography timing and strength of brain responses, we decided to base When a large population of neurons becomes simultaneously our approach on the well-known behavioural effects of active, the small postsynaptic currents in parallel-oriented stimulus degradation: decreasing stimulus visibility increases pyramidal nerve cells create a net magnetic field that can be subjects reaction times in word-reading and lexical decision measured outside the head using SQUID (superconductive tasks (e.g. Besner and Smith, 1992; Holcomb, 1993). quantum interference device) sensors. From the distribution Accordingly, we expected to see systematic changes in the of the measured magnetic fields, the location, orientation and amplitude and/or latency of letter string-specific activity as strength of the underlying currents can be estimated using the visibility of the stimuli was reduced. equivalent current dipoles (ECDs), thereby giving a In the present study, we determined (i) whether letter stringby satisfactory representation of activity within that area. Thus, specific MEG responses can be identified with systematic measuring its associated magnetic field patterns, the locations and time windows across subjects, and (ii) whether underlying brain activity can be located in time and, with there is any correlation between such early MEG responses some restrictions, also in space. (For a thorough review of and reaction times for reading the stimuli aloud. magnetoencephalography, see Hämäläinen et al., 1993.) In a magnetically shielded room, we measured the magnetic fields generated by the subjects cortical activity. We used a Neuromag-122 TM neuromagnetometer (Neuromag Ltd, Material and methods Helsinki, Finland), which employs 122 SQUID sensors Subjects arranged in a helmet-shaped array (Ahonen et al., 1993). The Twelve healthy, right-handed, Finnish-speaking adults (four planar gradiometers of the device detect maximum signal females, eight males) gave their informed consent to just above the activated cortical area. Because of the participation in this study. Their ages ranged from 21 to 42 approximately spherical symmetry of the human brain, MEG years (mean age 29 years). They were all university students is most sensitive to currents tangential to the skull, and the or graduates, and had normal visual acuity. main contribution of MEG signal thus arises from neurons within the fissural cortex. Stimuli Subjects were presented with four categories of stimulus: (i) pure Gaussian noise; (ii) single-element stimuli: single letters Alignment of MEG and anatomical data First, we determined a head coordinate system, to which the coordinate systems for both the MEG measurement and the

3 Dynamics of letter string perception 2121 Fig. 1 (A) Examples of four different stimulus categories: pure noise, single-element stimuli (letters or symbols), two-element stimuli (syllables or symbol strings) and four-element stimuli (words or symbol strings). Within each category the letter strings were masked with four levels of Gaussian noise (levels were designated 0, 8, 16 and 24). Symbol strings (symb) were always presented without noise (level 0). (B) Examples of letter-like symbol strings tested with two subjects. subject s MRIs were aligned. The head coordinate system front of the subjects. Stimuli appeared on the screen in a was defined in relation to three anatomical landmarks: the centrally placed rectangular patch (~5 2 ). All images nasion and points just anterior to the left and right ear canals, were shown on a large background of uniform grey. The easily localizable also on the subjects structural MRIs. The grey level of the background was set to 150 on a scale of x-axis of the head coordinate system runs through the points from black to white, based on the mean level of located in front of the ear canals, with positive values towards the different stimulus images, to keep the luminance level the right side. The y-axis passes through the nasion and runs relatively constant and to reduce the eye stress caused by perpendicular to the x-axis from the back of the head long periods of viewing of the stimuli. (negative) to the front (positive). The z-axis runs The subjects MEG responses were recorded in four perpendicular to both the x- and the y-axes. It passes through experimental conditions: (i) pure noise (levels 0, 8, 16 and the origin defined by the intersection of the x- and y-axes, 24), i.e. four stimulus types; (ii) single letters in noise (levels with positive direction towards the top of the head. 0, 8, 16 and 24) plus single symbols (no noise), i.e. five Before the MEG measurement, small coils were attached stimulus types; (iii) syllables in noise (levels 0, 8, 16 and to the subject s head and the locations of the coils were 24) plus double symbols (no noise), i.e. five stimulus types; determined with a 3D digitizer (Isotrak 3S1002; Polhemus and (iv) words in noise (levels 0, 8, 16 and 24) plus four Navigation Sciences, Colchester, Vt., USA) together with the symbols (no noise), i.e. five stimulus types. three anatomical landmarks defined above. Once the subject All stimulus types (letter strings at different noise levels was sitting with his or her head inside the measurement and noiseless symbol strings) appeared with equal probability. helmet of the neuromagnetometer, a small electric current The order of stimulus presentation was randomized within was fed to the coils to induce a measurable magnetic field conditions but was the same for all subjects. For words, extra pattern. This allowed the coils to be located with respect to care was taken to ensure that at least 30 s elapsed between the neuromagnetometer. Since the coil locations were also consecutive presentations of the same word in order to reduce known in head coordinates, all MEG measurements could be repetition effects. Measurements were carried out on two transformed onto the head coordinate system. Furthermore, days so that responses to pure noise and words were always since the head coordinate system could be mapped onto the measured on one day and responses to letters and syllables subject s structural MRIs (using the nasion and the left and on another day. right ear canals), individual MEG responses could also be Each experimental condition involved a min MEG mapped onto the subject s structural MRIs. recording session, which was divided into four blocks with intervening pauses of 1 3 min to allow the subjects to rest. During a recording session, each stimulus was displayed for Procedure 60mswitha2sinterstimulus interval. MEG signals were During MEG measurement, subjects sat in a dimly lit, passband-filtered to Hz, sampled at 397 Hz and magnetically shielded room. The stimuli were generated by averaged on-line in separate bins, one for each stimulus type. a Macintosh Quadra 840AV computer and projected (Sony Signal averaging began 0.2 s prior to stimulus onset and LCD Data Projector VCL-350QM) onto a screen ~1 m in continued for 0.8 s after stimulus onset. The horizontal and

4 2122 A. Tarkiainen et al. vertical electro-oculograms (EOG) were monitored also used a realistically shaped conductor model consisting continuously and epochs contaminated by eye-blinks and eye of ~1200 triangles to test the validity of the spherically movements were excluded from the on-line averages. To symmetrical conductor model. The results from the spherical achieve an acceptable signal-to-noise ratio, a minimum of and realistically shaped conductor models were in good 70 trials was averaged for each bin, though typically this agreement, thus justifying the use of the mathematically total exceeded 100. simpler spherical model for the bulk of our analyses (for During a recording session, subjects were instructed to details, see Appendix II). fixate the central region of the projection screen and to pay Equivalent current dipoles representing active source areas attention to the stimuli. Every so often a question mark were determined using the data from a minimum of six appeared for 2 s (1.5% probability), prompting the subject sensor pairs surrounding the local magnetic signal maximum, to report the preceding stimulus. These probe trials ensured at time points when visual inspection revealed clear dipolar that subjects maintained concentration. The responses to the field patterns with minimum interference from other active question mark and to the stimulus following it were excluded brain areas. If it was necessary to scrutinize the field patterns from the averages. The pure noise condition contained no with the activity of specific source areas removed for clarity, probe trials. Since the recording session was 20% shorter for we employed the signal space projection method (Uusitalo this condition, it was easier for the subjects to maintain and Ilmoniemi, 1997). For the early time interval ms, concentration source areas were identified in each subject. For the later time interval, up to 600 ms, 0 6 mainly temporal sources were found for each subject. Because the active source areas Reaction times were similar in the different stimulus conditions, we were Seven out of the 12 subjects took part in a behavioural word able to select a single set of ECDs for each subject which pronunciation task. We presented half of the same stimuli as had appeared in the word condition during the MEG recording session but with the interstimulus interval increased to 3 s to allow enough time for the vocalization. The test was conducted in the magnetically shielded room to make sure that the conditions were identical to the corresponding MEG measurement. Subjects were instructed to read the words aloud as quickly as possible. If they could not identify a word, they were instructed to remain silent. The reaction times to different stimuli were measured using the signal recorded from a microphone attached close to the subject s face and the EMG signal measured from two electrodes placed in the opposite corners of the subject s mouth, and analysed off-line. Subjects responses were recorded with a DAT (digital audio tape) recorder and evaluated for correctness. Data analysis Averaged MEG responses were digitally low-pass filtered at 40 Hz. The baseline for the signals was calculated over the period 200 to 0 ms before stimulus onset. For signal analysis, the shape of the conducting volume, i.e. the brain (Hämäläinen and Sarvas, 1989), has to be defined. In our studies we approximated the brain as a spherically symmetrical conductor. In each subject, the posterior part of the brain was modelled by a sphere adjusted to the local curvature with the help of the subject s structural MRIs. This model was then used when estimating the source areas in the time window ms after stimulus onset, because this early activity was concentrated in the occipital and occipitotemporal regions. After 300 ms, activity was seen mainly in the temporal cortices. These source areas were analysed separately using a sphere model adjusted to the curvature of the temporal regions. In four subjects, we gave a good account of the data in all conditions. This enabled the direct comparison of dipole amplitudes and latencies within and across the different conditions. Results Averaged MEG signals As illustrated in Fig. 2, activity within 250 ms after stimulus onset was concentrated in the occipital cortex. Figure 2A shows the averaged MEG responses to words (noise levels 0 and 24) and four-item symbol strings in one subject. The neuromagnetometer sensors located over the right occipital cortex showed early (~ ms) responses, which increased in amplitude with increasing noise level; Fig. 2B shows one sensor s record magnified. This response type was seen even more clearly for pure noise stimuli. Over the left occipitotemporal cortex, a later (~ ms) response type could be seen which showed markedly greater activity for noise-free words than for high-noise words and noiseless symbol strings (Fig. 2C). These two prominent response patterns were evident in most subjects. In addition, many other source areas could be identified, but most of them failed to show systematic stimulus-dependent behaviour. Therefore, we decided to focus only on those patterns of activity that behaved in a systematic way, and which were thus interpretable. To draw further conclusions from the spatial locations and behaviour of the source areas responsible for the two response patterns mentioned above, they had to be reliably identified from among all the other sources. To do this, we set certain objective criteria describing the behaviour seen in the averaged MEG signals. These criteria were then applied to the amplitude waveforms of all ECDs in all subjects. We were able to classify the sources responsible for the early

5 Dynamics of letter string perception 2123 noise-sensitive behaviour into a category called Type I and the sources responsible for the later word-responsive behaviour into a category called Type II. The use of selection criteria enabled us to identify also a third pattern (Type III), which had not been readily recognizable from the averaged MEG responses. Analysis of early signals (0 300 ms) Type I activation We called the early activation, which increased with increasing noise level, the Type I response. To be included in this category a source had to fulfil all the following criteria. In the pure noise condition, (i) a clear activation peak was found at the highest noise level; (ii) the source waveforms showed a systematic increase in peak amplitudes as a function of noise: noise 0 noise 8 noise 16 noise 24, i.e. activity at noise level 0 was significantly (P 0.05) smaller than that at higher noise levels (a difference of at least 1.96 times the baseline standard deviation); (iii) peak latencies at the highest noise levels were similar to or shorter than those at the lowest noise levels. ECDs that fulfilled Type I criteria were found in 10 out of 12 subjects. Results in nine of these 10 subjects were quite consistent, showing Type I activity that peaked within 125 ms after stimulus onset (in pure noise level 24 condition). In one subject, the Type I responses were clearly delayed (peaks at ms) and showed quite strong activation even for the noiseless condition, raising doubt as to whether his activity represented neural processing that was functionally similar to that found in the other subjects. Therefore, we set an upper limit of 130 ms for Type I peak latencies (in pure noise level 24 condition), excluding the Type I sources of this one subject. A total of 16 Type I sources from nine subjects were accepted for further analysis. Figure 3 illustrates how the peak amplitudes and latencies of Type I sources, averaged over all nine subjects, were modified across all experimental conditions. The peak amplitudes are expressed relative to the pure noise level 24 condition. If no clear peak was apparent (usually in the pure noise level 0 condition, i.e. after presentation of an evenly Fig. 2 Averaged MEG signals from one subject in the time window mswith respect to stimulus onset in the grey rectangle), the baseline standard deviation was used to words condition. Only responses to words and four-item symbol define peak amplitude, and no peak latency was obtained for strings at noise level 0 and words at noise level 24 are presented. that situation. The mean standard error of the mean peak The helmet-shaped sensor array of the Neuromag-122 TM is latency for the pure noise level 24 condition was 107 flattened to a plane and viewed from above with the subject s 4 ms and the mean amplitude 15 3 nam (nanoamperenoise pointing upwards. Variation in the magnetic field was measured at 61 locations over the head both latitudinally and metres) (with mean baseline standard deviation 0.8 nam). longitudinally, as shown by the small schematic heads. An Consistent with the criteria used to define Type I activity, enlarged view of two clear types of stimulus-related response Fig. 3A shows that the amplitude of the subjects responses modulation can be seen in B and C. The corresponding field increased as a function of noise when they viewed pure noise distributions at activation peaks (vertical dashed lines) are shown patches. This same pattern was apparent also for the other below the enlarged sensor outputs. The letter P (positive) denotes the magnetic field emerging from the brain and the letter N experimental conditions (letters, syllables and words). (negative) the re-entering field; the back of the helmet-shaped Moreover, Fig. 3A shows an additional feature of Type I sensor array is viewed slightly from the right and left, activity: activation increased with increasing length of letter respectively. or symbol string. These qualitative impressions were

6 2124 A. Tarkiainen et al. bordering V1 and extending laterally as far as V4v. The mean standard error of the mean distance of Type I sources from the midline was 18 3 mm. Fig. 3 The mean Type I dipole (A) amplitudes and (B) latencies (mean SEM) calculated across all Type I sources (16 sources from nine subjects) and measurement conditions. Amplitudes are shown relative to the pure noise 24 condition (set equal to 1). (C) Locations of all Type I sources are presented on the surface rendition of one subject s MRI. The brain is viewed from the back. All sources were projected onto the brain surface for easy visualization. confirmed by a two repeated measures ANOVA (analysis of variance) of Type I amplitudes. Both main effects of stimulus (four levels: pure noise, letters, syllables and words) and noise (four levels: 0, 8, 16 and 24) were significant [F(3,45) 3.3, P 0.05 and F(3,45) 28.6, P , respectively]. The two-way interaction stimulus noise was also significant [F(9,135) 8.8, P ]. Crucially, symbol strings and equally long noiseless letter strings evoked practically identical responses. Unlike the peak amplitudes, Type I peak latencies showed no clear modulation (Fig. 3B). As illustrated in Fig. 3C, Type I responses originated in occipital areas Type II activation Type I activity was followed by a response that was strongest for visible words. This later activity, which we call Type II, is clearly a candidate for letter string-specific activation. To be included in this category, ECDs had to fulfil all the following criteria for peak amplitudes and latencies. They had (i) a clear peak in the words 0 condition which had longer latency than Type I activity in the same subject, (ii) significantly stronger activation and/or shorter latency (by at least 5 ms, which is twice the sampling interval) for words at noise level 0 than at noise level 24, (iii) significantly more activity for words than for pure noise patches at noise level 0, and (iv) stronger activity and/or shorter latency for words at noise level 0 than for four-item symbol strings. Thus, the classification was based on word and pure noise conditions. We identified at least one Type II dipole in 11 out of 12 subjects. The first Type II sources usually peaked ~ ms after stimulus onset (words 0 condition) with no clear exceptions. However, some later Type II sources were also observed. To include all the Type II sources that we thought were likely to represent the same kind of neural processing, we set the upper limit of 180 ms for Type II peak latencies (in the words 0 condition). The total number of accepted Type II sources, gathered from 11 subjects, was 15. Mean Type II peak amplitudes (relative to noiseless words) and latencies across all subjects are illustrated in Fig. 4. For the noiseless word condition the mean amplitude was 18 2 nam and mean latency ms. Even though we demanded only that the activation to words at noise level 0 was stronger than the activation to pure noise at noise level 0, Fig. 4A shows that the amplitudes of Type II sources increased systematically with the length of the stimulus strings both for letters and for symbols. In every situation, the noiseless letter strings evoked stronger activity than symbol strings of equal length (P for letters, P 0.01 for syllables and P for words; paired two-tailed t test). In a similar fashion, the highest noise level (24) decreased the mean amplitude relative to letter strings at noise levels 0 and 8. This reduction was largest for words (P 0.01; paired t test: words 0 versus words 24). The effect of increasing noise on Type II activation in the word condition was variable across individuals. In some subjects the peak latency was delayed, in others the signal amplitude was decreased (at least for the highest noise level) and in the remainder both effects were observed. Interestingly, some subjects showed first a small increase in Type II amplitudes from noise level 0 to levels 8 and 16 and then a clear decrease for the highest noise level. On the other hand,

7 Dynamics of letter string perception 2125 responses were measured to low-noise words (levels 0 and 8) and responses were delayed for symbols and high-noise words (levels 16 and 24) (Fig. 4B). Note also that the peak latencies of noise-free letter strings decreased systematically as a function of string length (P 0.01; paired t test: letters versus words), whereas the peak latencies for symbol strings remained at ~155 ms (P 0.38; paired t test: one-item versus four-item symbol strings). The locations of all Type II dipoles are shown in Fig. 4C. (It should be noted that two of these dipoles, both of which were located in the right hemisphere, could also be classified as Type I sources.) In seven of these 11 subjects, all Type II sources were located in the left hemisphere close to the border of the temporal and occipital cortices, 36 2mm from the midline. Three subjects showed bilateral Type II sources and one subject showed a Type II source in the right hemisphere only. For this source the criterion (iv) was only barely fulfilled. Type III activation Only a minority of all occipital sources showed modulation consistent with classification as Type I or Type II. Most of the remaining sources were active at some point after stimulus presentation but did not show any systematic stimulus-related modulation. However, we were able to isolate a third pattern of stimulus-dependent behaviour (Type III), which was not immediately obvious from the averaged data. Many sources showed a clear peak for noiseless words but did not fulfil all the criteria for Type II classification. Nevertheless, some of these sources did show modulation according to string length. We classified such Type III activity as showing (i) a clear peak for words at noise level 0, (ii) significantly larger amplitude for four-item symbol strings than for one-item symbol strings, and (iii) failure to fulfil Type II classification. The time window for Type III sources was set to be exactly the same as for Type II sources, i.e. Type III activity had to Fig. 4 The mean Type II dipole (A) amplitudes and (B) latencies occur after Type I activity but before 180 ms after stimulus (mean SEM) calculated across all Type II sources (15 sources from 11 subjects) and measurement conditions. Amplitudes are onset. Equivalent current dipoles satisfying all these criteria shown relative to the words 0 condition (set equal to 1). were found in nine out of the 12 subjects. The total number (C) Locations of all Type II sources are presented on the surface of Type III sources was 15. rendition of one subject s MRI. The back of the brain is viewed Figure 5 shows the mean peak amplitudes (relative to fourslightly from the left and from the right. All sources were projected onto the brain surface. item symbol strings) and latencies of Type III dipoles averaged across all subjects. The mean amplitude in the fouritem symbol string condition was 18 3 nam and the mean latency ms. Type III amplitudes increased with the in some subjects the responses decreased monotonically with noise. length of the stimulus string (Fig. 5A), as with Type II A two repeated measures ANOVA of Type II amplitudes sources, but letters did not evoke stronger activity than showed significant main effects of stimulus (four levels: pure symbols. This was confirmed by a two-factor (stimulus type: noise, letters, syllables and words) and noise (four levels: 0, letter string versus symbol string; string length: 1, 2 and 4 8, 16 and 24) [F(3,42) 33.9, P and F(3,42) elements) repeated measures ANOVA of peak amplitudes. 5.5, P 0.005, respectively]. The two-way interaction The main effect of string length was significant [F(2,28) stimulus noise was also significant [F(9,126) 8.2, 19.2, P ], whereas the main effect of stimulus type P ]. was not [F(1,14) 0.01, P 0.5]. As with the Type I pattern, latency modulation was not as The estimated locations of Type III sources are shown in clear as amplitude modulation. Nevertheless, the fastest Fig. 5C. Most of them were found in the right hemisphere,

8 2126 A. Tarkiainen et al. Fig. 6 Amplitude waveforms of the Type II dipoles in two subjects who viewed geometric and letter-like symbols (Fig. 1B). The thin black lines indicate responses to noiseless words (measured twice). The dashed line indicates responses to geometric symbols and the thick black line responses to letter-like symbols. multiple sources acted independently of each other, they could be included in a global analysis without biasing the results. In practice, it is difficult to determine the independence of multiple sources. Therefore, we applied either an amplitude or a latency criterion in order to select just one Type I, Type II and Type III source per individual. The details of these criteria are given in Appendix III. To test whether analyses based on the complete data set were likely to be biased compared with analyses based on either of the restricted data sets, we carried out a one between groups (selection criterion: all, amplitude, or latency), two repeated measures ANOVA (four levels of stimulus; four levels of noise) of Type I, Type II and Type III amplitudes. In each case the main effect of criterion had no significant effect on outcome [F(2,31) 0.34, P 0.5, F(2,34) 0.42, P 0.5 and F(2,30) 0.02, P 0.5, respectively]. These results suggest that our data were not significantly distorted by the problem of source independence. Some effect of selection criterion can be seen with Type III source locations (Fig. 5C). When only the dipole showing the largest difference in strength between one-item and four- item symbol strings was selected for each subject (Appendix III, Amplitude criterion), the dipoles were located more uniformly in the right occipital cortex. This selection did not, however, markedly change the peak modulation shown in Fig. 5 and the interpretations based on that. Fig. 5 The mean Type III dipole (A) amplitudes and (B) latencies (mean SEM) calculated across all Type III sources (15 sources from nine subjects) and measurement conditions. Amplitudes are shown relative to the four-item symbol string condition (set equal to 1). (C) Locations of all Type III sources are presented on the surface rendition of one subject s MRI. The brain is viewed from the back. All sources were projected onto the brain surface. Sources marked in white are those that were included when only one source per subject was selected based on the amplitude criterion (see text for details). unlike Type II sources. A tighter dipole cluster (white circles in Fig. 5C), 22 5 mm from the midline, was obtained when only one dipole per subject was selected, based on source amplitude (Appendix III). Considerations about multiple sources Multiple Type I, Type II and Type III sources were identified in five, three and six subjects, respectively. In principle, if Letter-like symbols The effect of manipulating the symbol type on Type II activity was studied in two subjects. To do this we repeated the word measurements with letter-like symbols (Fig. 1B). Figure 6 shows the amplitude waveforms of these subjects Type II sources for geometric and letter-like symbols and for two repetitions of responses to words. Since the magnetic field patterns measured using the control stimuli were similar to those measured originally, the same sets of dipoles were used to explain the data. The activity evoked by distorted

9 Dynamics of letter string perception 2127 Fig. 7 Locations of all temporal and frontal sources showing word/letter string-specific (Type II) behaviour ms after stimulus onset presented on the surface rendition of one subject s MRI. The brain is viewed from the left and right. All sources were projected onto the brain surface. text was stronger than that for geometric symbols and showed a closer resemblance to, though it was not identical with, the activity evoked by words at noise level 0. These control measurements also illustrate the reproducibility of the responses to words, at least in these two subjects. Fig. 8 Mean reaction times in the word pronunciation task for seven subjects (thick grey line). Estimates of reaction times when explained by the Type II peak latencies (dashed line) or amplitudes (dotted line), averaged over the seven subjects, are also shown. To match the measured reaction times, a combination of latency and amplitude modulation was created (solid black line). See text for details. Analysis of late signals ( ms) Inspection of the averaged data showed that most of the signals occurring 300 ms after stimulus presentation were located in the temporal and frontal brain areas. As many as six new ECDs were identified per subject (mean 3.3) to account for the magnetic field patterns. Of these, a total of resembled the effect of noise on Type II amplitude waveforms 22 sources fulfilled selection criteria for Type II behaviour (Fig. 4): cortical activity was very similar for noise levels 0 (11 subjects, one to four sources for each subject). As can and 8, differences started to appear at noise level 16 and at be seen in Fig. 7, most of these late Type II sources were noise level 24 the activation was clearly reduced or delayed. found in the left hemisphere (16 sources), clustering mainly This suggests that subjects behaviour in the reading task in the left superior temporal cortex (nine sources). Activation can be correlated with Type II activity. detected in this area peaked ms after the presentation We attempted to explain the mean reaction times using of noiseless words, and tended to be delayed for words at the averaged (across seven subjects) Type II peak latency or the highest noise level (P 0.054; paired two-tailed t test). amplitude modulation alone. For latency modulation we used The peak amplitudes for noiseless words were significantly the differences relative to the noiseless words condition, and larger than those both for words at the highest noise level for amplitude modulation the inverse of amplitudes relative (P 0.05) and for four-item symbol strings (P 0.05). to the noiseless words condition. Both models were scaled so that they matched the measured reaction times at noise levels 0 and 24. As shown in Fig. 8, neither of these Reaction times in the behavioural pronunciation task Figure 8 shows the mean reaction times as a function of noise for the seven subjects who read word stimuli aloud. Reaction times were calculated from the onset of the microphone signal. Only correct answers were included. The subjects mean reaction times to words at noise levels 0 and 8 were very similar. At noise level 16, the reaction times were delayed despite almost perfect identification (93 3% correct). At noise level 24, identification was noticeably impaired (19 7% correct) and subjects reaction times were markedly longer ( ms) than at noise level 16. The shape of the reaction time curve in Fig. 8 approaches gave a good account of all the measured reaction times; using Type II response onset latencies instead of peak latencies did not improve the situation. On the other hand, we were able to create a combined measure that resulted in an almost perfect fit to the reaction times (Fig. 8). This successful measure combined both the peak amplitude and the latency modulation of the Type II ECDs: it was calculated by dividing the mean latency difference (relative to noiseless words) by the mean relative amplitude (relative to noiseless words) raised to the power of 3.5. When analysed on the individual level, it was possible in one or two cases to achieve a reasonably good fit to the reaction time curve by using only the Type II peak amplitudes. However, as seen in

10 2128 A. Tarkiainen et al. Fig. 8, the best results were generally attained by combining and language domains, rather like a complex filter. It is likely the noise level-dependent amplitude modulation with latency that the properties of such a filter would be developed or modulation. tuned by the extensive exposure to printed words that The composite measure we created has no obvious skilled readers experience. physiological basis. Intuitively, it results from the individual If Type II activity is specifically associated with the variation in the trade-off between amplitude and latency processing of letter strings, it is likely that damage to the modulation. Therefore, we do not expect the exact formula underlying population of neurons will affect the fluency of to be of significance, but we emphasize the idea that reading. This idea is supported by studies of patients with interaction of activation strength with timing correlates with pure alexia, who have to resort to slow letter-by-letter reading behavioural measures. (e.g. Warrington and Shallice, 1980; Coslett and Saffran, 1989; Behrmann et al., 1998). In cases where pure alexia is observed without hemianopia, the critical lesion site has been Discussion localized either to the left inferior occipitotemporal region In this MEG study, we investigated the early cortical (Ajax, 1967; Greenblatt, 1973; Henderson et al., 1985), processing of visually presented letter strings the visibility of coincident with the origin of Type II activity, or to areas which was systematically varied. We isolated three systematic subjacent to the left angular gyrus (Greenblatt, 1976). patterns of stimulus-modulated activity distinguished by their The noise-dependent behaviour of Type II activation was behaviour, anatomical location and timing. The effects were strongly correlated with the subjects reaction times in the seen either in the activation strength or in the timing, or in word pronunciation task. Bearing in mind that this result is both, on an individual basis. based on only four data points, the fact that we found such Type I activity occurred ~ ms after stimulus onset a good fit suggests that the subjects behavioural responses and originated bilaterally in occipital areas bordering V1 and to visually presented words may depend directly on Type II extending laterally as far as V4v. The amplitude of Type I neural processing; it may represent a key rate-limiting step. responses increased both with increasing noise, i.e. with The third response pattern (Type III activity) was not as increasing luminance contrast of adjacent pixels in a noise readily detectable in the 122-sensor data sets as Type I and patch (Fig. 1), and with the number of elements in letter/ Type II activity. Nevertheless, it showed enough systematic symbol strings embedded in the patches. However, Type I stimulus-related modulation to be identified reliably. Type III sources did not show object specificity as the amplitudes activity was located mainly in the right extrastriate cortex, were the same for letter and symbol strings of equal length. unlike Type II activity. Although the time course of activation Moreover, at the highest noise levels, all stimulus types was similar to Type II, Type III activity showed no specificity evoked equally strong Type I responses. This non-specific for stimulus type (i.e. letter versus symbol strings). The pattern of behaviour suggests that Type I activity reflects the activation strengths of Type III sources were modulated by kind of low-level processing that is common to all visual string length. One possible interpretation of Type III activation stimuli, such as the extraction of oriented contrast borders. is that it represents some attribute of object processing that Our finding is in line with functional imaging studies reporting is common to both letter and symbol strings but is not a increased activation in areas V1, V2 and V3 in response to scrambled objects compared with clearly delineated objects, necessary component for the visual language filter properties probably due to additional luminance contrast borders created that we propose for the left occipitotemporal cortex. by scrambling (Allison et al., 1994; Malach et al., 1995; Generally speaking, we found reasonable agreement Grill-Spector et al., 1998). between the locations of letter string-specific Type II sources The second distinct response pattern, Type II, occurred and studies that suggest that extrastriate regions are involved within 180 ms after stimulus onset and was found in the in word- and letter string-specific processing (e.g. Petersen occipitotemporal cortex with left-hemisphere dominance. et al., 1988, 1989, 1990; Price et al., 1994, 1996; Puce et al., Unlike Type I activity, Type II responses were diminished at 1996; Pugh et al., 1996; Salmelin et al., 1996; Rumsey et al., the highest noise level. This suggests that Type II responses 1997; Kuriki et al., 1998). In particular, the location of the reflect the processing of stimulus attributes more complex present Type II sources in the left hemisphere agrees with than oriented contrast borders, for example. Type II activity our earlier MEG results of single-word reading (Salmelin was strongest for letter strings, especially for words, and was et al., 1996). Furthermore, the time window around 150 ms, clearly reduced for geometric symbols. We also found that in which we observed Type II and Type III sources, agrees Type II responses to strings of rotated letters were quite with those reported elsewhere (Allison et al., 1994; Nobre similar to those evoked by words. Together, these findings et al., 1994; Salmelin et al., 1996; Kuriki et al., 1998) suggest that Type II activity may not be specific for words (Appendix IV). It is worth noting that Allison and colleagues per se. Instead, it may reflect processing of multi-element have reported specific responses to such stimulus categories strings whose image characteristics resemble letter strings. as letters, numbers and faces within 200 ms after stimulus We suggest that Type II signals may even reflect the activity onset, and originating in the inferior temporo-occipital cortex of a module that acts as an interface between the visual (Allison et al., 1994). The time window around ms

11 Dynamics of letter string perception 2129 may thus be particularly important in the analysis of visual masking the visually presented letter strings affected the time objects exceptionally relevant to human behaviour. course and strength of letter string-specific activation. This Büchel and colleagues recently reported activation of the modulation was correlated with the subjects reaction times left basal posterior temporal lobe [Brodmann area (BA) 37] in the word pronunciation task. The present data speak in a visual reading task for sighted subjects as well as in a strongly for a special role for the left inferior occipitotemporal tactile reading task for congenitally blind and late-blind region in the neural processing of letter strings. subjects (Büchel et al., 1998). Interestingly, they also stated We suggest that the neural population underlying this that developmental dyslexics, when compared with control response represents a mechanism that acts as an interface subjects, showed reduced activation in the same area. A between visual and language domains. The properties of such similar difference between fluent and dyslexic readers was a mechanism are likely to be developed with constant also reported by Salmelin and colleagues in the left inferior exposure to printed text. The analysis of this letter stringspecific temporo-occipital cortex (Salmelin et al., 1996). It is thus response may eventually provide a tool for evaluating plausible that our letter string-specific Type II activity may the acquisition and fluency of reading. be associated with the BA 37 activity reported by Büchel and colleagues (Büchel et al., 1998). Closer comparisons between our data and those in the Acknowledgements published literature reveal interesting discrepancies. In part We wish to thank Riitta Hari and Katri Kiviniemi for this may be due to differences between experimental comments on the manuscript and Kimmo Uutela for paradigms and in part to small errors in source localization comments and expert technical help. This work was supported or uncertainty in the exact source locations. Nevertheless, by the Academy of Finland, the Human Frontier Science the left medial extrastriate response, specific for word-like Program, the EC Human Capital and Mobility Program stimuli, that was reported by two other groups (Petersen [through the Neuro-BIRCH II (large scale facility of et al., 1990; Pugh et al., 1996) seems to be more medial neuromagnetism) facility in Helsinki] and the Oxford MRC than our Type II sources. Interestingly, the existence of the IRC (Interdisciplinary Research Centre) for Cognitive medial extrastriate word-specific source has been questioned Neuroscience. The MRI scans were obtained at the recently by Indefrey and colleagues, who suggested that Department of Radiology, Helsinki University Central medial occipital responses are associated with the length and Hospital. not with the lexicality of the strings (Indefrey et al., 1997). On the other hand, when presented with pseudo-words some of the subjects of Indefrey and colleagues showed left- References lateralized activity at the occipitotemporal junction and along Ahonen AI, Hämäläinen MS, Kajola MJ, Knuutila JET, Laine the posterior superior temporal sulcus, which was not PP, Lounasmaa OV, et al. 122-Channel SQUID instrument for observed with false fonts. These observations are in good investigating the magnetic signals from the human brain. Physica agreement with our results Type I and Type III amplitudes, Scripta 1993; T49: originating in areas close to the midline, increased with string Ajax ET. Dyslexia without agraphia. Arch Neurol 1967; 17: length but were not specific for letters, whereas the letter- Allison T, McCarthy G, Nobre A, Puce A, Belger A. Human specific Type II sources were located more laterally in the extrastriate visual cortex and the perception of faces, words, left occipitotemporal cortex. numbers, and colors. [Review]. Cereb Cortex 1994; 4: Manipulating the visibility of a word naturally affects not only the orthographic prelexical stage but also other Behrmann M, Plaut DC, Nelson J. A literature review and new data subcomponents of reading after this stage. Accordingly, we supporting an interactive account of letter-by-letter reading. In: also identified brain areas sensitive to the visibility of words Coltheart M, editor. Pure alexia (letter-by-letter reading). Hove (UK): Psychology Press; p in the more anterior brain regions. Specifically, the left superior temporal cortex displayed a preference for clearly Besner D, Smith MC. Models of visual word recognition: when visible words compared with symbol strings and heavily obscuring the stimulus yields a clearer view. J Exp Psychol Learn degraded words. The signals in this area peaked ~300 ms Mem Cogn 1992; 18: after stimulus presentation for non-degraded words. As the Bookheimer SY, Zeffiro TA, Blaxton T, Gaillard W, Theodore W. same cortical area has been shown to be involved in analysis Regional cerebral blood flow during object naming and word of word meaning between 250 and 350 ms after word reading. Hum Brain Mapp 1995; 3: presentation (Helenius et al., 1998), it is likely that these Büchel C, Price C, Friston K. A multimodal language region in the signals reflect semantic analysis. ventral visual pathway. Nature 1998; 394: In conclusion, we report the existence of an early, letter string-specific MEG response, identified reliably in 11 out of Coslett HB, Saffran EM. Evidence for preserved reading in pure 12 subjects. These responses took place within 200 ms after alexia. Brain 1989; 112: stimulus presentation and were concentrated mainly in the Damasio AR, Damasio H. The anatomic basis of pure alexia. left inferior occipitotemporal regions. The level of noise Neurology 1983; 33: