SECTION II. Understanding Emotional Language Content (Edited by Johanna Kissler)

Transcription

1 SECTION II Understanding Emotional Language Content (Edited by Johanna Kissler)

2

3 Anders, Ende, Junghöfer, Kissler & Wildgruber (Eds.) Progress in Brain Research, Vol. 156 ISSN Copyright r 2006 Elsevier B.V. All rights reserved CHAPTER 8 Emotional and semantic networks in visual word processing: insights from ERP studies Johanna Kissler, Ramin Assadollahi and Cornelia Herbert Department of Psychology, University of Konstanz, P.O. Box D25, D Konstanz, Germany Abstract: The event-related brain potential (ERP) literature concerning the impact of emotional content on visual word processing is reviewed and related to general knowledge on semantics in word processing: emotional connotation can enhance cortical responses at all stages of visual word processing following the assembly of visual word form (up to 200 ms), such as semantic access (around 200 ms), allocation of attentional resources (around 300 ms), contextual analysis (around 400 ms), and sustained processing and memory encoding (around 500 ms). Even earlier effects have occasionally been reported with subliminal or perceptual threshold presentation, particularly in clinical populations. Here, the underlying mechanisms are likely to diverge from the ones operational in standard natural reading. The variability in timing of the effects can be accounted for by dynamically changing lexical representations that can be activated as required by the subjects motivational state, the task at hand, and additional contextual factors. Throughout, subcortical structures such as the amygdala are likely to contribute these enhancements. Further research will establish whether or when emotional arousal, valence, or additional emotional properties drive the observed effects and how experimental factors interact with these. Meticulous control of other word properties known to affect ERPs in visual word processing, such as word class, length, frequency, and concreteness and the use of more standardized EEG procedures is vital. Mapping the interplay between cortical and subcortical mechanisms that give rise to amplified cortical responses to emotional words will be of highest priority for future research. Keywords: emotion; semantics; word processing; event-related potentials; healthy volunteers; clinical populations Introduction Influential dimensional approaches to the study of emotion derive their basic dimensions from analyses of written language. Osgood and collaborators, using the semantic differential technique, were the first to empirically demonstrate that affective connotations of words are determined by three principal dimensions, namely evaluation, potency, Corresponding author. Tel.: ; Fax: ; johanna.kissler@uni-konstanz.de and activity (Osgood et al., 1957), the first two accounting for the majority of the variance. On the semantic differential, a word s evaluative connotation is determined by ratings on a multitude of seven-point scales, spanned by pairs of antonyms such as hot cold, soft hard, happy sad, etc. Factor analyses of the judgments of many words on such scales, given by large subject populations, reveal a three-dimensional evaluative space, whose structure has been replicated many times and across different cultures (Osgood et al., 1975). Figure 1 provides an illustration of the evaluative space determined by the semantic differential. DOI: /S (06)56008-X 147

4 148 Fig. 1. A three-dimensional affective space of connotative meaning as postulated by Osgood (1957, 1975). The figure depicts the three orthogonal bipolar dimensions, evaluation (E), activity (A) and potency (P), and gives examples of prototypical words for each dimension and polarity. (Adapted from Chapman, 1979.) Osgood s principal dimensions are at the core of other circumplex theories of affect (Lang, 1979; Russel, 1980). For instance, Lang and colleagues posit that human affective responses are determined by the dimensions valence, arousal, and dominance; again the first two having the largest impact, leading these authors to propose model of affect defined by the dimensions of arousal and valence. Within such a two-dimensional affective space, different classes of stimuli such as pictures, sounds, and words cluster in a u-shaped manner. Highly arousing stimuli usually receivevalenceratingsaseitherhighlypleasantor highly unpleasant, low arousing material is generally regarded as more neutral with regard to valence (see Fig. 2 for an illustration derived from the ratings of German word stimuli). The impact of valence and arousal on various central nervous and peripheral physiological indicators of affective processing has been repeatedly validated using picture and sound media (Lang et al., 1993; Bradley and Lang, 2000; Jungho fer et al., 2001; Keil et al., 2002). Word stimuli assessed for perceived arousal and valence such as the Affective Norms for English Words, ANEW (Bradley and Lang, 1998) have also been put to use in physiological research (see e.g., Fischler and Bradley, this volume), although so far the resulting evidence seems more restricted than the one for pictorial material.

5 149 Fig. 2. Illustration of a two-dimensional affective space spanned by the dimensions arousal and valence. Examples of the relative position of some German adjectives and nouns used in our studies are given in English translation. Along the x- and y-axis are depicted the arousal and valence scales of the Self-Assessment Manikin (SAM, Bradley and Lang, 1994) used to rate individual emotional responses to the words. Emotions are generally viewed as culturally universal, largely innate evolutionary old signaling and activation systems, residing in old, subcortical parts of the brain. They are designed to promote survival in critical situations, i.e., to signal and activate fight, flight or feeding, attachment, and sexual behavior. Reading and writing, by contrast, represent comparatively recent developments in the history of mankind, and in individual development these are acquired much later than oral language. Consequently, these skills are often regarded as cultural achievements, but during the acquisition of written language considerable regional specialization emerges in the human brain (Warrington and Shallice, 1979, 1980; Dehaene et al., 2005). Reading acts as a secondary process that utilizes the processing capabilities of the earlier acquired auditory language system once the analysis of visual word form is completed (Perfetti, 1998; Everatt et al., 1999; Perfetti and Sandak, 2000). Language and emotion share a communicative function but linguistic communicative functions are obviously not restricted to the communication of affect.

6 150 How do the emotional brain and the linguistic brain interact, when written words with emotional connotations are encountered? Emotion theories posit that linguistic expressions are stored within semantic networks that encompass links to all aspects of their linguistic and pragmatic usages and emotional connotations (Lang, 1979; Bower, 1981). Thus, the word gun, for example, not only represents the object itself, but also includes links to its operations, use, purposes, and their consequences as well as their emotional evaluation (Bower, 1981). A converging view is shared in neurolinguistics (Pulvermu ller, 1999) and cognitive semantics (Barsalou, 1999): All information related to a word is stored in a dynamic network. Recent evidence suggests that subnetworks 1 representing different aspects of a word s lexical representation can be separately and dynamically activated. For instance, differential neuromagnetic activations of semantic subnetworks have recently been shown for subclasses of plant or animal names (Assadollahi and Rockstroh, 2005). Moreover, biasing contextual constrains can affect the timing of access to the dominant vs. subordinate meaning of homonyms (Sereno et al., 2003), challenging the modular view that in word processing initially all lexical entries have to be exhaustively accessed. Functional divisions of the semantic system mirroring functional divisions in the organization of the cortex have repeatedly been shown for verbs denoting different types of actions (Pulvermu ller et al., 2001b; Hauk et al., 2004). Investigating verbs pertaining to movements carried out with different parts of the body, these authors demonstrate that the meaning of action words is reflected by the correlated somatotopic activation of motor and premotor cortex. These patterns of coactivations presumably reflect individual learning history, where the so-called referential meaning has been acquired by repeated coactivation of the body 1 The terms sub-network or sub-representation as used here are not necessarily intended to imply a fixed hierarchical ordering of the neural networks coding for different aspects of semantics, although a certain degree of hierarchical ordering may indeed exist. Instead, sub-network or sub-representation refers to the fact that different neural networks are likely to code for different aspects of a word s meaning, such as animacy, emotional connotation, grammatical gender, etc. movement and the descriptive speech pattern, for instance when a child observes or carries out a gesture such as throwing and simultaneously hears the caregiver say the respective word. Later, in the acquisition of written language, this phonological code is mapped onto the visual word form (Perfetti and Sandak, 2000). For emotional concepts, Lang et al. (1993, 1994) assume that not only associated semantic but also physiological and motor response information is coactivated in associative networks. Figure 3 illustrates such a multilevel network representation of an emotional scene, encompassing a semantic code of the given situation as well as associated motor and physiological responses. Thus, the semantic network that codes for emotional semantics could include the neuronal circuitry processing the associated emotion (see also Cato and Crosson, this volume, for a related suggestion). How does emotional content influence different stages of visual word processing? Here, the literature is sparse. Event-related brain potentials (ERPs), the scalp recorded averaged synchronized activity of several thousands cortical pyramidal cells, have successfully been used to delineate different stages of visual word processing (for reviews see e.g., Posner et al., 1999; Tarkiainen et al., 1999). A closer look also reveals that a considerable number of electrophysiological studies of emotional processing have employed visually presented word-stimuli. Some, particularly early, studies have used the semantic differential as their theoretical vantage point. In fact, in the wake of Osgood s studies, the examination of ERP correlates of emotional semantics generated substantial research interest (Chapman et al., 1978, 1980; Chapman, 1979; Skrandies, 1998; Skrandies and Chiu, 2003). More recently, two-dimensional valence arousal models have been used as a framework for research (see Fischler and Bradley, this volume). However, many studies used words with emotional connotations as experimentally convenient instances of a broader class of emotional events (Anderson and Phelps, 2001; Dijksterhuis and Aarts, 2003) or conversely, as a semantic class without much reference to any particular theory of language and/or emotion (Begleiter and Platz, 1969). So far, little systematic knowledge has been gathered on the relationship between the emotion and language

7 151 Fig. 3. A network representation of a complex emotional scene (exam situation) illustrates how in dynamic emotional processing perceptual, semantic and response systems are interactively linked. Activation on any level of this system can spread to other subsystems. (Adapted after Lang, 1994.) systems in visual word processing and possible implications for the underlying neural implementation of meaning. ERP recordings have an excellent temporal resolution, allowing for a fine-grained analysis of the temporal sequence of different processing stages. Their spatial resolution is more restricted, and inferences from the spatial distribution of scalp measured ERPs to their neural generators can only be made with caution. The number of electrodes and the recording reference used influence the probability of finding effects, the generalizability of these findings, and the accuracy of spatial localization (see also Jungho fer et al., this volume). The present review will summarize and systematize existing studies on the role of emotion in visual word processing, both in healthy volunteers and in clinical populations and relate this evidence to the available knowledge on ERP indices of stages of visual word processing. First, we will address studies reporting early effects of emotional content on ERP responses, occurring within the first 300 ms after stimulus onset, separately for healthy volunteers and clinical populations; second, we will review effects of emotional content on the late ERPs to visual words; and third, discuss how an interplay of subcortical and cortical mechanisms of emotion and word processing may give rise to the observed effects. To facilitate comparisons between studies, main methodological parameters and results concerning effects of emotional content, as stated in the reviewed studies, are summarized in two tables in the appendix. Table A1 describes studies with healthy volunteers; Table A2 describes work with clinical populations. There, it becomes immediately apparent that the studies

8 152 described vary considerably in theoretical approach and methodology. Early effects occurring within 300 ms after stimulus onset Healthy volunteers The extent to which early exogenous components of the human event-related potential are subject to modification by nonphysical stimulus characteristics is a matter of ongoing controversy. In visual word processing, a traditional view holds that within the first ms after a word has been presented, specific perceptual features of written words but no meaning-related attributes are extracted (Schendan et al., 1998; Posner et al., 1999) and for many years the N400 potential, a centro-parietal negativity arising around 400 ms after stimulus onset has been viewed as the index of semantic processing (Kutas and Federmeier, 2000). Using other types of visually presented stimuli with emotional content, such as faces or pictures, remarkable ERP differences between emotionally significant and neutral stimuli have been found within the first 300 ms after stimulus onset, some even within the first 100 ms. In his single subject study of ERP responses to emotional words, Lifshitz (1966) failed to find a visually impressive differentiation between emotional and neutral words within the first 500 ms after word onset, although upon visual inspection the difference between the likewise presented erotic and neutral line drawings was sizeable. However, in the meantime there is a considerable body of evidence indicating that even very early ERP responses can diverge between emotional and neutral words. Thus, the probably first quantitative study on the effects of emotional content on ERP indices of word processing found differences between negative taboo words and neutral words already within the first 200 ms after word presentation (Begleiter and Platz, 1969). Notably, this study was entitled: Cortical evoked potentials to semantic stimuli, expressing a view of emotion as a vital part of semantics. The technological standards at the time were not very sophisticated and the authors recorded only from a single right occipital electrode (O2), but ERP differences due to emotional content appeared in the ERP tracings already in the P1 N1 complex: twenty minimally above threshold presented repetitions of each of two clearly negative four-letter words led to larger responses than twenty repetitions of each of the words tile and page. This pattern held for both a passive viewing condition and a naming condition. Several years later, Kostandov and Azurmanov (1977) contrasted ERP responses to both subliminally and supraliminally presented conflict and neutral words. In the subliminal condition, the earliest differences between conflict and neutral appeared around 200 ms after stimulus onset. Subliminally presented conflict words apparently relating to the subjects relationship conflicts caused by jealousy led to larger N200 responses than neutral words. Supraliminally presented words led to later differentiations starting around 300 ms, in the P3a window. Unfortunately, the description of the study is somewhat sparse on details regarding the materials used. Chapman et al. (1978) sought to electrophysiologically validate Osgood s connotative dimensions of meaning (see Fig. 1), namely evaluation (E), potency (P), and activity (A). They recorded from one midline electrode (CPz) referenced to linked mastoids while subjects had to name briefly flashed (17 ms) words. Twenty words each from the extreme ends of the three connotative dimensions, E, P, and A, yielding six semantic classes (E+/-, P+/-, and A +/-) were used as stimuli. Random sequences of all these words were presented times to the subjects (until a set criterion of artifact-free trials was entered into the average). Although the exact multivariate analysis of the data in the original report is somewhat hard to reconstruct, in essence Chapman et al. (1978) were able to statistically differentiate between all six connotative dimensions using ERP data from a single central channel. Whereas the different connotative dimensions are not mapped onto any specific ERP components, in the original grandaverages a second positive peak at around 260 ms after word onset is noticeable that clearly differentiates between the extreme ends (+ and -) of all three dimensions but not so much among E, P, and

9 153 A. In general, the material in this study was well controlled for physical aspects but apparently not for other linguistic attributes such as word frequency, word class, or concreteness. Replication studies by the same group investigated the effects of explicit affective rating tasks on ERPs (Chapman, 1979) and directly compared the effects of affective rating and word naming in one study, yielding similar results (Chapman et al., 1980). Thus, a series of studies demonstrated reliable statistical differentiation between six connotative categories within 500 ms after stimulus onset on the basis of ERP data from a single channel. Extending their 1969 study across a larger selection of stimuli, Begleiter et al. (1979) recorded ERPs elicited by unpleasant, pleasant, and neutral words, as 10 subjects had to either identify the last vowel in a presented word or give their personal affective evaluation of the word. Recordings were made from three electrodes over each hemisphere, referenced to linked mastoids. However, results are reported only for two of these electrodes, namely P3 and P4. Stimuli were the 62 most unpleasant, 62 most neutral, and 62 most pleasant five-letter words derived from a larger pool of words previously assessed with the semantic differential (Osgood et al., 1957). Words were presented very briefly (20 ms). The words emotional meaning affected N1 P2 peak-to-peak amplitude during emotional evaluation. ERP responses to words evaluated as pleasant, neutral, or unpleasant could be distinguished statistically at both electrodes. Overall, effects of emotional content were somewhat more pronounced over the left hemisphere and were restricted to the affective evaluation condition. At the left hemispheric electrode, ERPs were also generally larger when the words were shown in the emotional evaluation than in the letter-identification task. Unlike the previous studies, Begleiter et al. (1979) was the first to provide evidence for a major impact of task, showing early ERP effects of emotional content only during active evaluation. Skrandies (1998), studying ERP correlates of semantic meaning, used an approach conceptually similar to Chapman s (1978, 1979, 1980). A pool of 60 nouns representing the bipolar extremes on Osgood s E, P, and A dimensions were selected and presented in a rapid serial visual presentation (RSVP) design, i.e., a continuous stream of alternating words without interstimulus interval. Skrandies (1998) used a comparatively slow RSVP design presenting each word for 1 s. Ten stimuli per category and polarity were selected and each stimulus was repeated 40 times, yielding 400 averages per category, thus optimizing the signal-to-noise ratio. Subjects were instructed to visualize the words and to remember them for a subsequent memory test in order to ensure active engagement with the stimuli. EEG was recorded from 30 channels referenced to an average reference. Brain responses between the emotional word categories differed in six distinct time windows, with respect to either their peak latency, associated global field power or the location of the centroids of the scalp distribution. Remarkably, most of the differences occurred within the first 300 ms after word presentation, starting with P1 at around 100 ms. These results were recently extended cross-culturally in a virtually identical study of affective meaning in Chinese, yielding a somewhat different but equally complex pattern of differences depending on emotional word content, which were restricted to the first 300 ms after word onset (Skrandies and Chiu, 2003). Together with Begleiter and Platz (1969), Skrandies studies report probably the earliest meaning-dependent differentiation between words. Consequently, these studies are frequently cited in the visual word processing literature, albeit without reference to the particular emotional semantic contents used. Schapkin et al. (2000) recorded ERPs as subjects evaluated nouns as emotional or neutral. The stimuli consisted of a total of 18 words, 6 pleasant, 6 unpleasant, and 6 neutral, which were matched across emotional categories for word length, frequency, concreteness, and initial letters. Words were presented peripherally, to the left and right visual fields for 150 ms, while the EEG was recorded from 14-linked mastoid referenced electrodes, eight of which were consistently analyzed. Stimuli were repeated 32 times, 16 times in each visual field. The earliest effect of emotional significance of the words on ERP responses was observed in the P2 window, peaking at 230 ms. At bilateral central sites, P2 responses to pleasant words were larger than responses to unpleasant and neutral ones. While the P2 response was generally larger

10 154 over the left hemisphere, the emotion effect was not lateralized. Similar effects of emotional content on ERP responses to words were also observed in later time windows (see below). As already suggested by Kostandov and Arzumanov (1977), the powerful effects of emotional connotation on brain responses appear to extend even below the limits of conscious perception. Bernat et al. (2001), recording from six-linked mastoid referenced scalp electrodes, report a differentiation between both subliminally and briefly (40 ms) presented unpleasant and pleasant adjectives at lefthemispheric electrode positions already in the P1 and N1 time ranges, as participants simply maintained fixation on a central cross and viewed the computer screen without an explicit behavioral response being required. In the P1 and N1 windows, unpleasant adjectives lead to larger ERP responses in the left hemisphere. Overall larger responses to unpleasant as compared to pleasant adjectives were obtained in the subsequent P2, P3a, and LPC time ranges, the main effects of emotional content having earlier onsets in the subliminal condition. The affective valence of the stimuli had been determined by assessing a larger pool of words on five bipolar scales from the evaluative dimension of the semantic differential. Both the subsequent ERP study subjects and an independent sample had repeatedly rated the stimuli. ERPs to 6 repetitions of the 10 most extremely pleasant and unpleasant as well as to 12 neutral words were recorded. Thus, study subjects had considerable experience with the words. Also, it is unclear, whether the stimuli were assessed on other potentially relevant emotional or linguistic dimensions such as arousal and dominance or word length, frequency, and abstractness. Still, these resultsaswellasdatafromkostandovandazurmanov (1977) or Silvert et al. (2004) and Naccache et al. (2005) in principle support the possibility of measurable physiological responses to subliminally presented emotional words and add to the evidence of emotional content-dependent P1 differences (Begleiter, 1969; Skrandies, 1998; Skrandies and Chiu, 2003). Recently, Ortigue et al. (2004) also reported a very early effect of word emotionality in a dense array ERP study recording from 123 scalp channels. The task consisted of a lexical decision to very briefly flashed (13 ms) stimuli. Subjects had to indicate which of two simultaneously presented letter combinations in both visual fields constituted an actual word. Stimuli were half neutral and half emotional nouns of both pleasant and unpleasant valence. They were matched for word length and frequency and selected from a larger pool of words pre-rated on a bipolar seven-point scale spanning the neutral-emotional continuum. Overall, emotional words presented in the right visual field were classified most accurately and fastest. However, the relative advantage for emotional words was larger for words presented in the left visual field. Using a source estimation approach (LAURA) the authors identified a stable topographic pattern from the spatiotemporal distribution of the ERP data that accounted for the processing advantage of emotional words in the right visual field. This pattern emerged between 100 and 140 ms after stimulus onset, i.e., mostly in the P1/N1 window. Curiously, it was localized to primarily right-hemispheric extra-striate cortex. Surprisingly, no specific neurophysiological correlate of the even more pronounced advantage for emotional words in the left visual field was identified within the 250 ms after stimulus presentation that this study restricted its analysis to. Recent data from our own laboratory also produced evidence for early differences between cortical responses to emotionally arousing (both pleasant and unpleasant) and neutral adjectives and nouns. The word s emotional content had been predetermined in a separate experiment, obtaining valence and arousal ratings on two ninepoint rating scales (see Fig. 2) from 45 undergraduate students. According to these ratings, highly arousing pleasant and unpleasant and low arousing neutral words were selected. Different subsets of these words were used in three studies where ERPs from 64 scalp sites were measured. Neutral and highly arousing pleasant and unpleasant words matched for word length, frequency, and in one experiment also for concreteness were presented in RSVP designs to subjects instructed to read the words. Across three different stimulus presentation durations (333, 666, 1000 ms) and regardless of word type (adjectives or nouns), a lefthemispheric dominant occipitotemporal negativity

11 155 differentiated emotional (both pleasant and unpleasant) from neutral words. This negativity had its maximum around 260 ms after stimulus onset. The influence of stimulus repetition was assessed, but neither habituation nor sensitization was found for the emotional-neutral difference within the five repetitions used (Fig. 4 illustrates the effect for the 666 ms presentation rate). In one of the studies we also manipulated task demands, instructing subjects to attend to and count one of the two word classes (adjective or noun). Interestingly, this manipulation did not affect the enhanced early negativity to emotional words but had a significant impact on the later positivity. Herbert et al. (2006) also recorded ERPs from 64 average-reference linked scalp channels, as 26 subjects evaluated the emotional significance of highly arousing pleasant and unpleasant as well as neutral adjectives. The affective content of the stimuli had been predetermined in a separate population using the above-described procedure. Words were presented for a relatively long period, namely 5 s. In this study the P2 component was the first index of differential processing of emotional vs. neutral words. This P2 component primarily responded to perceived stimulus intensity/arousal and did not differentiate brain responses to pleasant from those to unpleasant words. The same was true for the subsequent P3a component, but the picture changed for a later LPC component and the simultaneously recorded startle response that were more pronounced for pleasant than for unpleasant and neutral words. This sequence of effects is depicted in Fig. 5. Fig. 4. Early arousal-driven enhancement of cortical responses to emotional words. Uninstructed reading of both pleasant and unpleasant words in a rapid serial visual stimulation paradigm (RSVP, 666 ms stimulus duration) leads to larger occipitotemporal negativities than reading of neutral words. The effect is illustrated at two occipital sensors (O9, O10) and the scalp topography of the difference potential emotional neutral words is depicted. Grand-averages from 16 subjects are shown.

12 156 Fig. 5. Difference maps of cortical activation for emotional minus neutral words in a covert evaluation task. Averaged activity in three time windows is shown: P2 ( ms), P3 ( ms), and LPC ( ms). For P2 and P3 both pleasant and unpleasant words are associated with larger positivities than neutral ones. In the LPC window only processing of pleasant words diverges from neutral. The time course of the activity is shown at electrode Pz. Grand-averages from 26 subjects are shown. Clinical studies One of the first studies to use emotional words as a tool to address processing biases (or a lack thereof) in clinical populations is Williamson et al. (1991) who investigated behavioral and cortical responses to pleasant, unpleasant, and neutral words in psychopathic and nonpsychopathic prisoners. Subjects had to detect words in a sequence consisting of words and nonwords while their EEG was being recorded from five scalp positions referenced to linked mastoids. Stimuli were presented vertically for 176 ms, separately to either visual field, and repeated three times. Stimuli had been matched for length, number of syllables, word frequency, and concreteness but differed in emotional connotation. Nonpsychopathic subjects had faster reaction times and larger P2 responses to both pleasant and unpleasant emotional words than to neutral ones. These differences induced by emotional arousal extended into the late positive component (LPC) time range in controls but were completely absent in psychopaths. Weinstein (1995) assessed correlates of enhanced processing of threatening and nonthreatening verbal information in university students with elevated or normal trait anxiety levels. Subjects read sentences with threatening and pleasant content that served as primes for subsequently presented threat related,

13 157 neutral, or pleasant words. ERPs in response to the target words were assessed at Fz, Cz, and Pz, as subjects had to decide whether the target word contextually fit the previously shown sentence. Highly anxious subjects were reported to exhibit a larger frontal N1 in the threat-priming condition and an enhanced P400 (i.e., reduced N400, see below) in a later time window. In retrospect, a number of methodological problems seem to exist in this study or the data presentation may contain errors. For instance, a large ERP offset at baseline is shown, the presented condition means occasionally do not seem to correspond to the ERPs displayed and information on linguistic properties of the stimuli used is missing. However, taken at face value, the results indicate heightened selective attention to (N1) and facilitated semantic integration of (N400/P400) threat-related information in students with elevated trait anxiety levels. A similar pattern of early ERP differences was also found in a study investigating the processing of pain-, body-related, and neutral adjectives in healthy volunteers and prechronic pain patients. Knost et al. (1997) report enhanced N1 responses to the pain-related stimuli at a left frontal sensor (F3) in the patient group. ERPs had been recorded from 11 mastoid linked scalp positions while subjects had to name words that were presented at the individually determined perceptual threshold. In both groups, pain- and body-related words produced larger positivities than neutral ones in a later time window ( ms) and were also associated with larger startle eye-blink responses on separately administered startle probe trials. The authors interpret their findings as evidence for preconsciously heightened attention to unpleasant, pain-related stimuli in the patients. These results were paralleled in an analogous study with chronic pain patients (Flor et al., 1997). Chronic pain patients had a larger frontal N1 response to pain-related words than comparison subjects. Additionally, a general hemispheric asymmetry emerged: N1 responses were larger over the right for pain-related words and over the left side of the head for neutral words. The enhanced responses for painrelated words in the patient group were also visible in a centro-parietally maximal N2 component. In the P2 window, right hemispheric responses to pain words were likewise larger in the patients. In contrast to the first study, no differential effects of emotional category were observed in subsequent time windows (P3 and LPC). The ERP results are taken to reflect heightened preconscious allocation of attention (N1) and stimulus discrimination (N2) to disorder-related words in pain patients, but show no evidence of further evaluative processing. In a similar vein, Pauli et al. (2005) studied cognitive biases in panic patients and healthy volunteers analyzing ERP responses from nine scalp channels to panic-related unpleasant and neutral words that were presented, in separate runs, at individually determined perceptual thresholds and for 1000 ms. Early ERPs differentiated panic patients from comparison subjects. At threshold presentation, patients showed two enhanced early frontal positivities in response to panic words, one between 100 and 200 ms (P2) and the other (P3a) between 200 and 400 ms post-stimulus onset. Both early effects were absent at the longer exposure duration and did not occur at all in the control group. Interestingly, subsequent positivitiesbetween400and600msaswellasbetween 600 and 1000 ms differentiated between panic words from neutral words in both groups and for both presentation durations. This pattern of data resembles the results by Knost et al. (1997). Kissler and colleagues (in preparation) assessed processing biases in depressed patients and comparison subjects using the above-described RSVP paradigm, recording from 256 scalp electrodes and comparing the amplitude and scalp distribution of the previously described early negativity to pleasant, unpleasant, and neutral adjectives matched for length and frequency. Around 250 ms after word onset (see Fig. 3), comparison subjects displayed the above-described left-hemispheric dominant enhanced negativity for emotional words, pleasant and unpleasant alike. Depressed patients, by contrast, exhibited this enhanced negativity solely in response to the unpleasant words and only in the right hemisphere. Comparing early emotional and early semantic processing In sum, numerous studies have found early (o300 ms) amplifications of ERPs in response to

14 158 words with emotional content compared with neutral words. The occurrence of such effects is remarkable since controlled conscious processing has been suggested to arise only with the P3/N400 components (Halgren and Marinkovic, 1995), implying that emotion can affect preconscious stages of word processing. Such effects appear to be more pronounced in various clinical populations, reflecting heightened sensitivity and orienting to unpleasant, disorderrelated material (Weinstein, 1995; Flor et al., 1997; Knost et al., 1997; Pauli et al., 2005). Other patients groups, by contrast, seem to selectively lack processing advantages for emotional words, psychopaths showing no cortical differentiation between emotional and neutral words (Williamson et al., 1991) and depressed patients showing preferential processing of unpleasant but not pleasant words (Kissler et al., in preparation). At any rate, processing biases in a number of clinical populations are reflected in their patterns of early responses to pleasant, unpleasant, and neutral words. A debated issue in emotion research pertains to whether, when, and how cortical responses differ as a function of arousal, valence, and additional factors such as dominance, or complex interactions of these. Here, the data must remain somewhat inconclusive, as the studies discussed differed vastly on the dimensions included and assessment methods used. Studies that assessed their materials with the semantic differential found that brain responses differentiate between all dimensions and polarities within the first 300 ms after stimulus onset (Chapman et al., 1978, 1980; Skrandies, 1998; Skrandies and Chiu, 2003). However, the arising pattern of results is so complex that it is hard to gauge the effect of each individual dimension on brain responses. The vast majority of studies report generally larger ERP responses to emotional than to neutral words, with some studies reporting these effects even in the absence of a task that would explicitly require processing of emotional content or other types of semantic access (Begleiter and Platz, 1969; Bernat et al., 2001; Kissler et al., submitted manuscript). However, occasionally, early emotion effects in word processing were found restricted to situations where explicit processing of the emotion dimension is required by the task (Begleiter et al., 1979). Directly comparing the impact of pleasant vs. unpleasant word content yields mixed results, with some studies finding larger early effects for pleasant words (Schapkin et al., 2000) and others larger effects of unpleasant ones (Bernat et al., 2001). As mentioned above, the subjects clinical or motivational status may bias their cortical responses in either direction. Also, task characteristics as well as the timing of stimulus presentation may have an additional impact but so far the influence of these parameters is not well understood. Of note, some of the described effects occurred even before 200 ms after word onset, in a time range in which from a traditional theoretical standpoint meaning-related processing differences would not be expected (Schendan et al., 1998; Cohen et al., 2000). These very early effects of emotional content on ERP indices of visual word processing are rather heterogeneous with regard to timing, locus, and direction. Some of the inconsistencies are probably related to differences in instrumentation and recording methodology, number of electrodes, and choice of reference electrode(s) representing but the most obvious differences. The described studies also differ vastly in the way emotional content of the stimulus material was assessed as well as in the extent to which other, nonemotional, linguistic factors such as word class, length, and frequency or concreteness were controlled. Nevertheless, the bulk of the evidence suggests that, indeed, under certain circumstances the emotional connotation of words can affect even the earliest stages of preconscious sensory processing. Thus, the challenge is to specify under which circumstances such emotional modulation of earliest processing may occur and what the underlying mechanisms are. Two experimental factors arise from the reviewed studies that may contribute to the emergence of very early emotion effects. First, very brief stimulus presentation, near or even below the perceptual threshold (Begleiter and Platz, 1969; Kostandov and Arzumanov, 1977; Chapman et al., 1978, 1980; Flor et al., 1997; Knost et al., 1997; Bernat et al., 2001; Ortigue et al., 2004; Pauli et al.,

15 ) and second, repeated presentation of comparatively small stimulus sets (Begleiter and Platz, 1969; Chapman et al., 1978, 1980; Skrandies, 1998; Skrandies and Chiu, 2003; Ortigue et al., 2004). None of the cited studies have explicitly assessed the effect of stimulus repetition on the latency of emotional-neutral ERP differences. Our own studies of repetition effects on negative difference waves distinguishing emotional from neutral content around 250 ms after stimulus onset show no evidence of change within five repetitions. However, some of the cited studies used by far more than five stimulus repetitions and studies of early semantic processing indeed suggest an effect of stimulus repetition on the timing on meaning-related differences in cortical activity: Pulvermu ller and colleagues report neurophysiological evidence of differences in semantic processing from 100 ms post word onset (Pulvermu ller et al., 2001a). They used a task in which a single subject was repeatedly, over several days, presented with a set of 16 words that she had to monitor and hold active memory as responses to occasionally presented new words were required. Thus, in above threshold presentation, preactivation of the cortical networks coding for meaning by using tasks that require continuous attention to and working memory engagement with the stimuli as well as use of many repetitions may foster earliest semantic processing differences, nonemotional and emotional alike. Further, a recent study on repetition effects in symbol processing found an increase in N1 amplitude (around 150 ms) across three repetitions of initially unfamiliar symbol strings (Brem et al., 2005), supporting the view that stimulus repetition can amplify early cortical responses to word-like stimuli. Thus, repetition effects affecting emotional stimuli more than neutral ones as a consequence of differential initial capture of attention and rapid perceptual learning may account for some of the very early ERP effects in emotional word processing. Early effects of emotional content on brain responses to subliminally or near-subliminally presented stimuli have occasionally been accounted for by fast, subcortical short-cut routes (see Wiens, this volume, for a discussion of issues of subliminal stimulus presentation). Evidence from animal experiments and functional neuroimaging indeed reveals the existence of such subcortical short-cut routes in emotional processing, particularly of fear-relevant stimuli (Davis, 1992; LeDoux, 1995; Morris et al., 1999). Direct pathways from the superior colliculi and the thalamus to the amygdala and the cortex allow for the automatic processing of relevant stimuli outside of conscious awareness, preparing rapid behavioral responses. In humans, this subcortical pathway has been mapped by functional neuroimaging during fear conditioning of subliminally presented faces (Morris et al., 1999) as well as during the subliminal presentation of faces with fearful expressions (Liddell et al., 2005). On a cortical level, its activity may be reflected in transient early responses. Brief, subliminal stimulation with fearful faces has recently been shown to result in a transient enhancement of the N2 and early P3a components, which, however, did not continue in later N4/P3b/LPC windows. For supraliminal stimulation, conversely, N4/P3/LPC but not N2 components responded to emotional content (Liddell et al., 2004), suggesting the operation of a slower, conscious processing and evaluation route. Conceivably, subliminal stimuli receive a temporally limited amount of processing that wanes if it is not confirmed by further supraliminal input, much like in the case of subliminal priming (Greenwald et al., 1996; Kiefer and Spitzer, 2000). Recording ERPs during subliminal and supraliminal semantic priming, Kiefer and Spitzer observe decay of subliminal semantic activation within 200 ms, a delay at which supraliminal priming effects can still be robustly demonstrated. A plastic, maladaptive downregulation of subcortical excitability may account for early responsiveness to unpleasant and disorder-related words in clinical populations (see e.g. Pauli et al., 2005 for supportive evidence). Clearly, at present the operation of a fast subcortical route from the thalamus and the amygdala in emotional word processing that could account for near or subthreshold emotion effects in visual word processing remains a speculative conjecture. A most critical point is that such a mechanism would require at least basic reading abilities in the thalamus. While the case for stimuli such as faces or threatening scenes that by some are

16 160 assumed to be part of our evolved fear module (O hman and Mineka, 2001) can be made much more easily, many would have a hard time believing in rather sophisticated subcortical visual capacities allowing for the discrimination of written words. On the other hand, subcortical structures are also subject to modifications by learning, and by the time people take part in experiments they will usually have had about two decades of reading expertise. So far, most of the evidence for the subliminal processing of emotional stimuli is based on studies with aversive material. Accordingly, the abovereviewed studies evidence extremely early effects primarily for unpleasant words (Flor et al., 1997; Knostetal.,1997;Bernatetal.,2001). An alternative explanation of some of these early effects of enhancement by emotional content in visual word processing that would not rely on subcortical by-pass routes and therefore on subcortical vision is reentrant connections between the so-called visual word form area (VWFA) and the emotion processing system. During visual word recognition the earliest activation of an invariant version of the visual word form (i.e. the font-, size-, position-invariant representation of the string of letters) occurs from about 100 ms after stimulus onset (Sereno et al., 1998; Assadollahi and Pulvermuller, 2003). Form invariant, abstract representations of highly overlearned visual objects such as words and faces have been found to originate in the fusiform gyrus (Haxby et al., 1994; Chao et al., 1999; Cohen et al., 2000; Dehaene et al., 2002). Electrophysiological evidence with regard to the onset of word-specific effects of fusiform activity varies, with some authors reporting onsets around 120 ms (Tarkiainen et al., 1999; Assadollahi and Pulvermu ller, 2001) and others somewhat later around 170 ms (Bentin et al., 1999; Cohen et al., 2000). Timing differences may be partly attributable to differences in word familiarity across experiments (King and Kutas, 1998). Immediately after access of the visual word form, meaning can be activated: Assadollahi and Rockstroh (2005) showed that activation differences due to super-ordinate categorical differences (animals vs. plants) can be found in left occipitotemporal areas between 100 and 150 ms after word onset, whereas activation differentiating between subordinate categories was evident only from 300 ms on. Dehaene (1995) observed the earliest ERP differences between words of different categories (verbs, proper names, animals), ms after word onset. Semantic category differences were reflected in the scalp distribution of a left occipitotemporal negativity. Using RSVP designs a similar occipitotemporal negativity has been identified. This negativity has been termed the recognition potential (RP). It is sensitive to semantic aspects of visual word processing and has its maximum around 250 ms after word (Rudell, 1992; Martin-Loeches et al., 2001; Hinojosa et al., 2004). The RP responds to manipulations of depth of semantic analysis, its amplitude increasing with the meaningfulness and task-relevance of the presented word. Source analysis has placed the origin of the RP in the fusiform gyrus (Hinojosa et al., 2001). Results from our laboratory are consistent with the view that a word s emotional connotation enhances the associated recognition potential (see Fig. 4). Thus, a word s emotional connotation could be directly connected to the abstract representation of its visual form. Moreover, the combined evidence suggests that emotional content amplifies early stages of semantic analysis in much the same way an instructed attention enhancing processing task would. If enhanced semantic processing is an important mechanism by which emotional content affects visual word processing, again, the question arises as to the causative mechanism: back-projections from the anterior cingulate and the amygdala may give rise to such processing enhancements. In support, amygdala lesions impair the enhanced detection of unpleasant words in an RSVP attentional blink paradigm but not of identification enhancements caused by manipulation of target color (Anderson and Phelps, 2001). Thus, assuming that the amygdala plays a pivotal role in the preferential processing of emotional words as recently suggested by several neuroimaging and lesion studies (Isenberg et al., 1999; Anderson and Phelps, 2001; Garavan et al., 2001; Hamann and Mao, 2002; Naccache et al., 2005), an alternative model to the above described thalamo-amygdalo-cortical route

17 161 could account for most of the data. Emotional amplification of semantic processing would occur after initial stimulus identification, caused by bidirectional reentrant communication between cortical regions and the amygdale (Amaral et al., 2003). Crucially, cortical analysis would precede and spark subcortical amplification of cortical processing. Clearly, a theoretically crucial priority for future research is to determine the timing of subcortical mechanisms in relation to cortical enhancement of ERP responses to emotional words. Unlike for other semantic categories such as movement-related verbs (Pulvermüller et al., 2000, 2001b), ERP data for emotional words so far suggest little consistent emotion-specific change in topography (but see Skrandies, 1998; Skrandies and Chui, 2003; Ortigue, 2004). A distinct, emotion-associated topography might point to the existence of a homogeneous emotion lexicon localizable in distinct neuronal populations of the brain as has been suggested for other semantic categories (Martin et al., 1996). Rather, emotional content seems to amplify cortical word processing, much in the same way as it enhances picture (Jungho fer et al., 2001) or face processing (Schupp et al., 2004). However, functional neuroimaging techniques with better spatial resolution of especially deep cortical and subcortical structures (see Cato and Crosson, this volume) and the more consistent use of dense array EEG studies (Ortigue et al., 2004) may provide additional information. Late components (after 300 ms) Healthy volunteers In relation to traditional stages of visual word processing, effects occurring later than 300 ms after word onset are less puzzling than the previously discussed early ones. Enhanced late positivities in response to emotional word content have most often, but not invariably, been found. In several of the already discussed studies reporting early ERP modulations as a function of emotional content, later enhanced positivities associated with the emotional content of the word stimuli are also apparent. For instance, in the data shown by Chapman et al. (1978, 1979, 1980; see above) a positivity occurring around 300 ms is discernible and appears to be primarily related to the potency dimension extracted from their data. Using materials from the Begleiter et al. (1969, 1979) and Chapman et al. (1978, 1979) studies, Vanderploeg et al. (1987) assessed ERP responses to visually presented emotional (20 pleasant, 20 unpleasant) and 20 neutral words and face drawings (two per emotion category), which were evaluated during viewing. The EEG was recorded from six electrodes referenced to linked ears in 10 male subjects. During viewing, the visual stimuli were presented for either 80 ms (words) or 100 ms (faces). In the conditioning phase, the face drawings were shown for 1500 ms. For both faces and words clearly discernible emotion-category dependent differences in ERP tracings appear from around 300 ms after stimulus onset as parietal positivities. A small but significant effect of emotional connotation of words but not faces on the spatial distribution of the ERP was also evident in the P2 window. Interestingly, although sizeable in appearance, the P3 effect of emotional connotation did not reach statistical significance for words. For a later positivity (positive slow wave/late positive complex) a similar result was obtained; although visible in the presented grand-averages, the difference in parietal positivity between emotional, pleasant and unpleasant, and neutral words does not reach significance in an analysis of the corresponding PCA factors while it does for faces. The authors, in line with Lifshitz (1966) early finding, suggest that words may be less powerful (or more heterogeneously evaluated) emotional stimuli than pictures (Vanderploeg et al., 1987). Thus, ERPs from10 subjects may not yield enough statistical power to assess connotation-dependent ERP differences, particularly in studies using comparatively sparse electrode arrays. Also, the perceptual variance between 20 words of a category may be higher than among two faces. Differential effects may, therefore, also result from the greater consistency or higher frequency of occurrence of the faces. Indeed, subsequent studies have found robust effects of emotional connotation on later ERP components. For instance, Naumann et al. (1992) investigated late positive potentials to adjectives