EMOTIONAL CONNOTATION AS SEMANTIC ATTRIBUTE: A WHOLE BRAIN FMRI STUDY OF EMOTIONAL VALENCE AND SEX EFFECTS ON WORD GENERATION

Transcription

1 EMOTIONAL CONNOTATION AS SEMANTIC ATTRIBUTE: A WHOLE BRAIN FMRI STUDY OF EMOTIONAL VALENCE AND SEX EFFECTS ON WORD GENERATION By MARGARET ALLISON CATO A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2001

2 Copyright 2001 by Margaret Allison Cato

3 This dissertation is dedicated in loving memory to my grandmother, Dorcas Pritchard.

4 ACKNOWLEDGMENTS I would like to acknowledge my family and friends, as well as my dissertation committee and labmates for their help and support. I extend special thanks to David Soltysik, Kaundinya Gopinath, Didem Göçkay, Nathan Himes, and Heather Belanger for assistance with aspects of data acquisition and analysis. iv

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS... iv LIST OF TABLES... vii LIST OF FIGURES...viii ABSTRACT...ix INTRODUCTION...1 SEMANTIC PROCESSING...7 Modular Models... 9 Unitary Models Conclusion With A Controversy Matrix Theory Imaging Studies of Semantic Processing EMOTIONAL PROCESSING...35 Definitions of Emotions Peripheral Correlates of Emotions Central Correlates of Emotions Hemispheric Asymmetries Related to Emotional Processing Right Hemisphere (RH) Model Valence Model Imaging Studies Relevant to the RH-Bivalent Debate Sex Differences in Emotional Processing EMOTIONAL SEMANTICS...49 Experimental Findings Concerning Emotional Semantics Previous Imaging Studies of Emotional Connotation as Semantic Attribute HYPOTHESES AND PILOT STUDIES...55 Hypothesis Hypothesis v

6 Hypothesis Hypothesis Pilot I: Pilot II Conclusions Behavioral Pilot Study METHODS...71 Participants Experimental Tasks and Manipulation Check Image Acquisition Image Analysis RESULTS...77 Behavioral Results FMRI Results Positive, Negative, and Neutral Category Generation Versus Neutral Word Repetition Direct Comparisons of Positive, Negative and Neutral Word Generation DISCUSSION...95 LIST OF REFERENCES BIOGRAPHICAL SKETCH vi

7 LIST OF TABLES Table Page 1. Selective semantic impairments and corresponding lesion sites Comparison of functional neuroimaging findings of unique activation with animal versus tool comparisons using picture presentation Summary of fmri pilot results Positive, negative and neutral categories selected for the final fmri study Pilot results collapsed across sex of participants (N = 12) Behavioral results collapsed across sex of participants (N = 26) Volumes of tissue (> 200 µl) showing significant activity changes (p <.001) for word generation tasks versus neutral word repetition Volumes of tissue (> 200 µl) showing significant activity changes (p <.005) for emotional (positive and negative categories collapsed) versus neutral word generation Volumes of tissue (> 200 µl) showing significant activity changes (p <.005) for positive and negative versus neutral word generation vii

8 LIST OF FIGURES Figure Page 1. Model of lexical and semantic functions (Ellis & Young, 1988) Model of matrix theory Sample portion of pseudorandomized block run alternating with rest Sample portion of pseudorandomized event-related run alternating with rest Arousal ratings for each category type by sex of participant Sample portion of pseudorandomized block run alternating with repetition Cortex near the left frontal pole activated by negative, positive, and neutral categories relative to neutral word repetition Right temporal pole activated only by negative emotional categories relative to neutral word repetition Activity in cortex near the frontal pole and in retrosplenial cortex associated with the main effect of valence, across participants, along with corresponding hemodynamic response functions (HRFs) for positive, negative and neutral categories for these areas Activity in cortex near the frontal pole and in retrosplenial cortex in direct task comparisons Positive versus negative word generation...93 viii

9 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy EMOTIONAL CONNOTATION AS SEMANTIC ATTRIBUTE: A WHOLE BRAIN FMRI STUDY OF EMOTIONAL VALENCE AND SEX EFFECTS ON WORD GENERATION By Margaret Allison Cato August 2001 Chairman: Bruce Crosson, Ph.D. Major Department: Clinical and Health Psychology How the brain organizes knowledge has long been an enigma. Case studies of patients with selective loss of semantic information after brain injury have provided some insight into this organization. Recently, functional magnetic resonance imaging (fmri) has been used to investigate semantic organization in the brains of neurologically normal individuals. This noninvasive procedure has the capability of localizing functions within the brain s cerebral hemispheres. The current study represents a test of certain tenets of a theory of semantic organization, the matrix theory. Specifically, emotional connotation is investigated as a potential semantic attribute that cuts across semantic categories, and leads to unique patterns of activation. A whole brain fmri imaging study of 26 individuals (13 males and 13 females) revealed unique activation bilaterally near the frontal poles and bilaterally in the retrosplenial area for word generation to categories with emotional connotation relative to word generation to neutral categories. These areas ix

10 of activation, while bilateral, were more lateralized to the left than right hemisphere. In addition, examination of valence (positive or negative) and sex (males versus females) effects revealed emotional valence type (positive versus negative), but not sex as a further determinant of the pattern of activation associated with emotional connotation. This study provides strong support for the importance of item attributes in semantic processing. x

11 INTRODUCTION Numerous reports of patients with category-specific deficits have sparked much interest in the structure of knowledge within the brain. In the lesion literature, deficits for living things are the most frequently reported category-specific deficit (Caramazza & Shelton, 1998; Damasio, Grabowski, Tranel, Hichwa & Damasio, 1996; Farah, Hammond, Mehta & Ratcliffe, 1989; Farah, McMullen & Meyer, 1991; Gainotti & Silveri, 1996; Hart & Gordon, 1992; Hillis & Caramazza, 1991; McCarthy & Warrington, 1988; Ralph, Howard, Nightingale, & Ellis, 1998; Silveri & Gainotti, 1988; Tranel, Damasio & Damasio, 1997). Other categories reported as specifically impaired include nonliving things (Hillis & Caramazza, 1991; Sacchett & Humphreys, 1992), body parts (Dennis, 1976; Shelton, Fouch & Caramazza, 1998), tools (Damasio et al., 1996; Lu et al., 2001; Tranel et al., 1997), fruits and vegetables (Farah & Wallace, 1992; Hart, Berndt, & Caramazza, 1985), and medical items and conditions (Crosson, Moberg, Boone, Rothi & Raymer, 1997). While reports of semantically-bounded deficits serve to inform and constrain theories of the organization of semantic information in the brain, their contribution to these theories is limited by a number of factors. First, the lesions of the reported patients are the results of various cerebral insults, including stroke, herpes simplex encephalitis, and temporal lobe epilepsy. The extent of damage contributing to the resulting language or semantic deficit is not always clear. Table 1 shows the variability of anatomic lesions, both focal and widespread, that has led to 1

12 2 Table 1: Selective semantic impairments and corresponding lesion sites. Nature of Specificity Reference Etiology Lesion Location Living Things/Animals Impairment in knowledge of living things across verbal and visual modalities. Impairment in naming animals. Impairment in recognition of animals. Information on animals impaired across visual, functional attributes, and visual, verbal modalities. Loss of information about animate objects across visual and functional attributes. Nonliving Things Semantic sparing of animals with impairment of inanimate categories. Naming and comprehension impairment for artefactual objects and body parts. Body Parts Selective sparing of body parts. Selective impairment of body parts. Warrington & Shallice, 1984 Damasio et al., 1996 Tranel et al., 1997 Caramazza & Shelton, 1998 Ralph et al., 1998 Hillis & Caramazza, 1991 Sacchett & Humphreys, 1992 Shelton et al., 1998 Dennis, 1976 Category-Specific Two patients post herpes simplex virus encephalitis Focal brain lesions (N = 16) Focal brain lesions (N = 28) Cerebral infarction Dementia of the Alzheimer s type Thromboembolic stroke Cerebral infarction Stroke Temporal lobe epilepsy Bitemporal damage Greatest lesion overlap in left inferior temporal lobe Maximal lesion overlap in right mesial occipital/ventral temporal region and in left mesial occipital region Left posterior frontal and parietal lobes Global cortical atrophy, particularly to medial temporal structures bilaterally Large area of acute infarction in left temporal lobe and two focal infarcts in left basal ganglia region Left fronto-parietal region Left occipital infarct and left posterior temporal cortices Left anterior temporal lobectomy

13 3 Table 1 continued Nature of Specificity Reference Etiology Lesion Location Tools Impaired naming of items (nouns and verbs) related to human action. Impaired naming of tools. Impaired recognition of tools. Impaired naming of tools. Fruits and Vegetables Selective anomia and partial comprehension deficit for fruits and vegetables. Selective anomia for fruits and vegetables across modalities despite spared knowledge. Medical Items and Conditions Selective inability to name to definition medical items and conditions. Loss of attributes for visual information only, across modalities and categories. Selective loss of visual attributes. Lu et al., 2001 Left Anterior temporal lobe epilepsy (N = 15) Cappa et al., 1998 Haemorrhage due to an arteriovenous malformation Tranel et al., 1997 Focal brain lesions (N = 116) Damasio et al., 1996 Hart et al., 1985 Farah & Wallace, 1992 Crosson et al., 1997 Coltheart et al., 1998 Ralph et al., 1998 Focal brain lesions (N = 16) Stroke, several transient ischemic attacks (TIA s), occlusion of left internal carotid Arterial venous malformation Hemorrhage Attribute-Specific Left-hemisphere stroke; middle cerebral artery distribution Semantic dementia Left anterior temporal lobectomy Left anterior temporal lobe Maximal lesion overlap in the lateral occipitaltemporal-parietal junction of the left hemisphere Maximal lesion overlap in the posterolateral sector of the inferior left temporal lobe Left frontal lobe and basal ganglia Left occipital region Left pulvinar and internal capsule Diffuse lesions in both hemispheres Atrophy of left temporal lobe

14 4 Table 1 continued Nature of Specificity Reference Etiology Lesion Location Semantic impairment with spoken word input but not picture presentation for animals. Visual recognition of living things selectively impaired. Knowledge deficit for visual attributes of living things. Loss of visual attributes of living things. Input Modality- and Category-Specific McCarthy & Semantic Warrington, 1988 dementia Farah et al., 1991 Closed head injury Category- and Attribute-Specific Farah et al., 1989 Closed head injury Gainotti & Silveri 1996; Silveri & Gainotti, 1988 Herpes simplex virus encephalitis Left temporal lobe Bilateral inferior temporo-occipital, right temporal and right frontal; left temporal region Involvement of bilateral temporooccipital regions and the right inferior frontal lobe Bilateral lesions involving all temporolimbic structures, with more impact on the left side; involvement of entire inferior temporal cortex on left Impairment for perceptual attributes of animals, especially with verbal input. Category-, Input Modality- and Attribute-Specific Hart & Gordon, Parneoplastic 1992 syndrome Diffuse, mild inflammation including bilateral temporal lobes category-specific deficits or preservations. Further, the reports are usually either single case reports or reports of a few patients, with notable exceptions (Damasio et al., 1996; Tranel et al., 1997). This work is described in greater detail shortly.

15 5 Recently, functional neuroimaging has been used to examine how the brain organizes semantic information using hemodynamic activity or metabolic changes as an indication of neuronal activity. Functional neuroimaging is noninvasive and has the potential for precise intrahemispheric localization. Imaging studies of neurologically normal individuals reveal unique patterns of activation for categories of objects, such as tools and animals (Chao, Haxby & Martin, 1999; Damasio et al., 1996; Ishai, Ungerleider, Martin, Schouten & Haxby, 1999; Martin, Wiggs, Ungerleider, & Haxby, 1996; Thompson-Schill, Aguirre, D Esposito & Farah, 1999). Among imaging studies conducted to date, however, results are not sufficiently in agreement to draw even preliminary conclusions, other than the involvement of cortex in the fusiform gyrus for processing of visual attributes (Cato et al., 2000; Chao et al., 1999; Ishai et al., 1999; Martin, et al.,1996; Thompson-Schill et al., 1999). Nevertheless, a number of competing theories exist to explain the category-specific findings of the case report and imaging literature. Chapter 2 gives competing theories of the organization of semantic information in the brain, along with supporting and refuting evidence for each. The evidence provided by case reports will be presented first, followed by imaging studies. Chapter 2 ends by offering the matrix theory (Crosson, Cato, Sadek & Lu, 2000) as an attempt to better account for the lesion and imaging findings to date. The fundamental dimensions of semantic processing proposed by the matrix theory are modality of processing, semantic category and semantic attributes. The current study represents an empirical exploration of the attribute dimension of the semantic matrix. Specifically, the role of emotional connotation as semantic attribute was examined using a word generation paradigm.

16 6 Furthermore, the roles of emotional valence (positive versus negative) and sex effects were explored. The latter were two potential sources of variability in the pattern of brain activation associated with processing information with emotional connotation. To build a rationale for the hypotheses regarding unique brain activation related to processing words with emotional connotation, what is currently known regarding the biological and neurological substrates of emotional processing will be presented in Chapter 3, with evidence cited from imaging studies as well as numerous other experimental methodologies. In addition, sex differences in physiological and evaluative responses to emotional stimuli will be described. Emotional semantics will be explored in Chapter 4 with emphasis placed on the processing of single words with emotional connotations. Existing evidence suggestive of unique processing in the brain for words with emotional connotations will be reviewed. Chapter 5 describes the hypotheses for the study as well as the pilot studies that were conducted to determine appropriate methodology and parameters for the final study. Chapter 6 describes the methodology used in the final study. As a preview, the primary experimental question is whether generation of words with emotional connotations, positive or negative, leads to unique brain activation relative to generation of neutral words. Another question will be whether males and females differ in either the patterns or extent of activation for the positive, negative and neutral conditions.

17 SEMANTIC PROCESSING Before engaging in a discussion about the organization of semantic information in the brain, it is important to distinguish semantic processing from other related processes. Distinctions between lexical and semantic processing are crucial in order to interpret findings from the lesion and imaging literature. The term lexicon refers specifically to word form while the semantic system stores meanings of words and objects. Ellis and Young (1988) proposed one prominent model of lexical and semantic functions. This model (Figure 1) proposes independent input and output lexicons for spoken and written words. Input lexicons allow recognition of heard and read word forms, while output lexicons supply word forms for speaking and writing. One other input system consists of viewed objects, called the structural description system. According to this model, the input lexicons and input from seen objects independently access a unitary semantic system where meaning is stored. While this is one popular model of the nature of the lexical-semantic interface, other competing models argue against separate input and output lexicons. Much evidence points toward separate input and output lexicons (Berndt & Mitchum, 1997), and this will be a working assumption made by this author. Figure 1 depicts the semantic system as unitary, with all input modalities having equal access to the monolithic system. This depiction of the semantic system is certainly over-simplified. Many authors claim that findings of category-specific deficits with spared functioning for other categories of objects negate the possibility that the semantic system is unitary. The 7

18 8 Heard Words Seen Objects Read Words Phonological Input Lexicon Structural Description System Orthographic Input Lexicon Semantic System Phonological Output Lexicon Orthographic Output Lexicon Figure 1. Model of lexical and semantic functions (Ellis & Young, 1988). rest of this section will be devoted to more detailed discussion of potential models of the organization of semantic information in the brain. Theories of the organization of semantic information in the brain include modular, unitary and distributed models. Models of semantic organization have largely been formulated by process of deduction, with the largest body of evidence coming from case reports of modality- or category-specific naming deficits. More recently, functional

19 9 neuroimaging has been used to investigate the organization of semantic information in neurologically normal individuals. The current theories of semantic organization in the brain do not adequately account for the entire body of literature on modality-specific, category-specific and even attribute-specific deficits. This section describes the patient literature and the theories of semantic organization that have been used to account for the case findings. Next, the contribution of functional neuroimaging studies to the semantics debate is described, along with accompanying theoretical standpoints. Finally, the matrix theory (Crosson et al., 2000) is proposed to more fully account for the variety of deficits that have been reported. Modular Models Some authors who have discovered category-specific effects have assumed a compartmentalization of knowledge with independent subsystems that interact in the normal brain. With the assumption that subsystems can run independently in the face of partial disruption, modular theorists can account for circumscribed areas of spared knowledge in the face of partial injury to the system. In addition, modular models can account for imaging findings of category-specific activation in neurologically normal individuals. A number of modular models have been applied to explain findings of category-specificity. One such model holds that knowledge is stored by semantic category. Proponents of such a model argue that category-specific effects are a result of disruption at the level of the semantic system rather than a result of access to it, which would indicate disruption at an earlier level of processing. A general method used to establish a deficit of meaning rather than a deficit of access is a response-consistency

20 10 analysis. The assumption is that if the same items tend to be failed across testing sessions, the representations of those items have been damaged. If different items are failed on different testing sessions, but general level of impairment remains the same, the problem is one of access (Shallice, 1987). This consistency rule has been applied to the interpretation of patient s deficits to either confirm or reject a determination of a category-specific semantic deficit (Farah & Wallace, 1992; McCarthy & Warrington, 1988). Certainly, the consistency rule cannot be an absolute, as degradation in functioning does not always imply precisely consistent outcomes. Nevertheless, the literature does contain numerous case findings of categoryspecific deficits that conform to the consistency rule. Based on these findings, some theorists argue that the semantic system is divided into semantic categories. The most frequently cited dissociation is between animate and inanimate entities. Warrington and Shallice (1984) were among the first to discover an animate-inanimate distinction in four patients naming, where living entities were the selectively impaired category for all four patients. In other cases the opposite has been found: that is, that animate entities are selectively spared. Sacchett and Humphreys (1992) reported a single case of a patient with a category-specific deficit for naming of man-made objects and body parts along with intact naming of natural objects. On picture-word matching tasks, the patient could not distinguish between close semantic distracters of man-made objects and body parts. Such a matching task does not require lexical retrieval of the object, and provides further evidence that the disruption was at the level of semantics. These authors provided a number of interesting arguments for why category-specific deficits might occur. One

21 11 reason may be the relative importance of different attributes for the definition of a given concept (e.g., synthetic objects such as implements may be more strongly associated with fine hand manipulations or other motoric sequences, while associations of sounds, tastes, or types of motion may figure more prominently in definitions of natural objects). Another possible explanation is that object classes may differ by what information is minimally required to select the appropriate object name (e.g., function may be necessary to distinguish man-made items and body parts; perceptual characteristics such as color, and texture may weigh more heavily for natural objects). Finally, many man-made objects contain the same local parts, but are distinguished by the spatial configuration of these parts (e.g., automobile and bus). On the other hand, many natural objects have the same parts with similar spatial relationships among them so that more fine visual discrimination is required to discriminate among close associates (e.g., dog and cat). Farah et al. (1991) raised the issue that semantically-bounded naming impairments may be attributable to some other confound such as visual complexity, similarity or specificity. These authors performed a regression analysis on the results of two patients given the full set of Snodgrass and Vanderwart (1980) line drawings. They found that the living-nonliving distinction accounted for a large degree of variance in responding when other factors were controlled. These authors concluded that selective impairment of living things is not likely a result of confounding variables. The authors further concluded that a system within the brain is either devoted to or biased for the processing of living things. The authors further conceded the possibility of dissociable subdivisions within the organization of semantic information in the brain. Other authors

22 12 have looked at this issue more recently with similar conclusions (Bunn, 1998; Sartori, Miozzo & Job, 1993; Tranel et al., 1997). A double dissociation has been reported along the lines of an animate-inanimate distinction, providing convincing evidence that this is one organizing principle for how semantic information is stored in the brain. Hillis and Caramazza (1991) cited a double dissociation between two patients, both with lesions in the left temporal lobe. Using a corpus of 144 pictures from 10 semantic categories, with frequency and word length controlled, two patients were tested with opposite results. In one patient, production and comprehension of animals were disproportionately spared over knowledge of nonliving items such as types of furniture and foods. The second patient s deficit was specific to living things with animals and vegetables equally affected. The latter patient s left temporal lobe lesion was more anterior and did not extend as far superior as that of the patient who had unique sparing of naming and comprehension of animals. Deficits in both patients were found for both naming and comprehension. Findings of deficits across naming and comprehension are a strong indication that the deficit is semantic, rather than lexical in nature. Spoken word form retrieval is only necessary in naming; not for other tasks of comprehension that were given in this study, such as word-picture matching and providing definitions of words. In addition, deficits persisted in both patients regardless of input and output modalities, providing further evidence that the deficits could be placed squarely at the level of semantics. These authors concluded that the observed double dissociation could only be explained by a category-specific organization of semantic information in the brain.

23 13 Perhaps the most convincing evidence for category specificity in the organization of knowledge in the brain comes from two studies from the same lab: one examining word retrieval (Damasio et al., 1996) and the other, word recognition (Tranel et al., 1997). In the Damasio et al. study, 29 patients with visual naming difficulties secondary to left-hemisphere lesions were studied. Similarly in the Tranel et al. study, 40 patients with visual recognition difficulties secondary to lesions (left and/or right hemisphere) were studied. Patients in both studies had nami ng problems in one or more of three semantic categories; naming of famous faces, animals or tools. Both sets of authors used a volumetric mapping algorithm to plot the lesion sites for the patients with categoryspecific naming deficits. In the Damasio et al. study, lesion overlap among patients with the same category-specific visual naming deficit revealed strong evidence for spatial mapping of semantic categories in the left temporal lobe. These authors found that a specific naming deficit for famous faces was correlated with a lesion overlap that clustered in the left temporal pole. A naming deficit specific to animals was correlated with a lesion overlap in the anterior sector of left inferior temporal lobe. Patients with a naming deficit specific to tools had lesions that clustered within the posterolateral sector of the inferior temporal lobe along with the lateral temporo-occipito-parietal cortices. The Tranel et al. study included only those patients whose naming errors were secondary to faulty recognition (operationally defined as naming attempts that did not provide any indication of knowledge of the item that they could not name). In other words, Tranel et al. examined lesions of patients who appeared to have semantic rather than lexical retrieval deficits, though these authors did not make this distinction explicit.

24 14 Tranel et al. similarly found a unique convergence of lesions for tools versus animals. Lesion overlap among the patients with deficient knowledge of animals was maximal in a right mesial occipital area extending to the ventral temporal region, as well as a secondary focus in the left mesial occipital region. Patients with deficient knowledge of tool names had maximal overlap similar to that found by Damasio et al.; that is, in the left lateral temporo-occipito-parietal junction. Dissociations found with such relatively large patient samples appear to support the notion that storage of information is, at least in part, determined by semantic category. In addition, these studies with a large number of patients appear to provide convincing localization information. However, other lesion studies do not agree with the above findings. Lu et al. (2001) found that patients with left anterior temporal lobectomy for intractable epilepsy experienced relatively more difficulty naming objects related to human actions relative to other objects. Similarly, Cappa, Frugoni, Pasquali, Perani & Zorat (1998) reported a single case of a patient with a category-specific naming deficit for nonliving things (i.e., tools and furniture) subsequent to a hemorrhage in the left anterior temporal lobe. These findings are not consistent with the Damasio et al. and Tranel et al. studies that ascribe the left temporal pole to naming of famous faces and the right temporal pole to the recognition of famous faces. Such discrepant findings must be reconciled before firm conclusions about localization can be drawn. Reported deficits discussed thus far have fallen along the lines of a general animate-inanimate distinction; however, a variety of more specific semantic categories have been reported to be selectively impaired. These categories include body parts (Dennis, 1976; Shelton et al., 1998), fruits and vegetables (Farah & Wallace, 1992), and

25 15 medical items and conditions (Crosson et al., 1997). While the numerous findings of category-specific deficits have sparked much curiosity and further investigation of the organization of semantic information in the brain, debate persists with respect to the actual dividing lines, if any, along which semantic information is organized. Another modular theory holds the assumption that dissociable meaning systems might divide along the modalities by which we acquire information. In such a case there may be a visual semantic system, a verbal semantic system, and so on. An assumption of this hypothesis is that in the normal brain, the modality-specific systems are connected in some way to allow comprehensive retrieval of knowledge about an entity. If the semantic system were organized by input modality, it would then be possible to lose verbal but not visual comprehension or the reverse. Evidence of modality-specific impairments is well characterized in the literature, providing considerable support for this particular model. A documented modality-specific disorder is visual associative agnosia, in which a patient is unable to assign meaning to objects or pictures presented visually but may be able to perform normally when asked to define verbally presented items. While visual agnosia is the most frequently reported modality-specific disorder, other agnosias are documented in the literature including, auditory agnosia, (an impaired capacity to recognize sounds despite adequate hearing and an absence of aphasia); auditory agnosia for speech (with a comprehension deficit specific to spoken language despite intact reading, writing and speaking); auditory affective agnosia (in which one is unable to appreciate affective prosody); and somatosensory agnosia (a loss of higher-order tactual recognition and an inability to identify the nature of tactually presented objects despite otherwise adequate functioning) (Bauer, 1993).

26 16 Arguably, the presence of modality-specific naming or comprehension deficits does not necessarily imply multiple meaning systems. Instead, an observed modality impairment could just be a result of a disconnection to one unified meaning system. This argument becomes less convincing when a semantic impairment is both modality- and category-specific. McCarthy and Warrington (1988) reported such a case of a categoryspecific impairment affecting only knowledge of spoken names of living things. With visual comprehension intact, there appeared to be a degradation of meaning that was bounded by both modality and semantic category. These authors claimed that this was sufficient evidence for multiple modality-specific meaning systems. An attribute-specific theory, the sensory/function theory (SFT), proposes that category-specific findings are actually an emergent result of noncategorical attributes shared among exemplars of the same category. An additional assumption of the SFT is that knowledge is organized into modality-specific subsystems. Warrington and Shallice (1984) originally proposed this theory. They propose two major divisions among attributes, one consisting of the visual components and another, the functional/ associative components. Although not delineated by many other investigators, presumably other modality-specific stores exist corresponding to the other sensory modalities. In support of this theory are findings of both attribute-specific impairments, combined with category- and/or modality-specific impairments (Farah et al., 1989; Gainotti & Silveri,1996; Hart & Gordon, 1992; Silveri & Gainotti, 1988) (Table 1). This theory strongly advocates that the meaning of living things depends more on visual attributes than the meaning of nonliving things. Thus, an assumption of the SFT is that damage to the visual knowledge subsystem results in a disproportional deficit within the

27 17 category of living things. Unfortunately, this tenet has not been borne out in the case study literature. Ralph et al.(1998) reported a patient with semantic dementia who did not exhibit a category-specific impairment for living things but did demonstrate a selective impairment in knowledge of visual attributes. This evidence runs counter to the argument that deficits in visual feature discrimination are always accompanied by a categoryspecific deficit for living things. Unitary Models Unitary models assume that what appears to be category-specific deficits are either artefactual or are due to lost attributes rather than semantic categories. Funnell and Sheridan (1992) reported a patient who initially appeared to have a category-specific deficit for living things on tasks of naming pictures and defining words. This apparent semantically-bounded deficit disappeared when familiarity of the items was equated between the sets of living and nonliving things. Snodgrass and Vanderwart (1980) reported sub-categories among their set of line drawings with varying degrees of familiarity, here listed from highest familiarity to lowest: 1) human body parts; 2) furniture and kitchen utensils; 3) fruits, vehicles and clothing; 4) carpenter s tools and vegetables; 5) musical instruments, insects, and birds. Funnell and Sheridan argued that familiarity has important implications for the determination of a category-specific deficit. For example, items in low-familiarity groups, while common to a subject, are not usually examined closely and may never be handled. If physical features (attributes) do not become intimately known, questions about perceptual features of low-familiarity items could conceivably yield more errors. These authors concluded that apparent categoryspecific deficits for processing of living things could be a function of a lesser degree of

28 18 familiarity. Other possible confounds they listed included frequency and complexity of visual form. In a similar vein, Stewart, Parkin and Hunkin (1992) reported that familiarity accounted for an apparent category-specific deficit for animals. A patient was tested on naming using the entire corpus of Snodgrass and Vanderwart (1980) line drawings. Six individuals rated the drawings for visual complexity and familiarity. Using these ratings, familiarity and visual complexity were examined as influencing variables in naming performance. Word finding deficits were positively correlated with low ratings on the degree of familiarity and negatively correlated with high ratings of visual complexity. Once familiarity was controlled for, errors in naming were no longer category-specific. The authors argued that controlling simultaneously for frequency, familiarity, and visual complexity could eliminate apparently selective deficits. Using a subset of the pictures with these parameters equated, these authors grouped the remaining pictures by categories with higher dependence on physical attributes (i.e., animals, vegetables, fruits, large buildings, and natural features) versus categories with relatively more functional attributes (i.e., tools and kitchen utensils). In this case, the patient s performance was equal across the two sets of stimuli, providing evidence in this instance that the form/function distinction was not an important one in predicting this patient s naming accuracy. Similarly, the control participants mirrored what seemed to be a dissociation in the patient s relative weakness in the ability to name to visual definitions versus functional definitions. In other words, the visual definitions used in this experiment were more difficult compared to the functional definitions. Thus, these authors found no

29 19 support for a visual-verbal distinction in semantics. The authors suggested that their findings support the presence of a single, amodal semantic system. One unitary theory argues that the semantic system is composed of attributes, and that semantic categories are an emergent property of connections among similar attributes. Damasio (1990) proposed that particular neural regions are dedicated to specific sensory feature fragments, so that the organization of semantic information reflects attributes that combine to define entities. Thus, overlapping subsets of semantic attributes result in emergent categories. According to this theory, clusters of attributes do not have a one-to-one correspondence with boundaries of conceptual categories. Two recent findings of attribute-specific impairment without either category-specificity or input modality-specificity are listed in Table 1 (Coltheart et al., 1998; Ralph et al., 1998). Caramazza, Hillis, Rapp and Romani (1990) argue that it is unnecessary to assume a modality-based or category-based organization of semantic information in the brain in order for category-specific deficits to emerge. Their modality-independent, unitary semantics theory is referred to as the Organized Unitary Content Hypothesis (OUCH). The theory assumes that semantic representations have an internal structure consisting of various attributes, which are linked, but with unequal strength. In addition, strongly linked properties are stored in adjacent areas of the brain (Caramazza & Shelton, 1998). The core dictum of this theory is that semantic space in the brain is lumpy such that semantic attributes do not spread uniformly across categories but lump together (presumably based on strengths of associations), regardless of their modality. This produces a likelihood that focal damage leads to category-like effects because of similar and/or shared properties among members of categories. Caramazza and colleagues (1990)

30 20 further propose the Privileged Relationship and Access Unitary Content Hypothesis (PRAUCH) which means that a given input modality allows direct access to modalitycongruent information. For example, a visually presented object would directly access visual semantic information; other sensory information (verbal, tactile, olfactory and kinesthetic) would be accessed secondarily through links within the system. Thus, visuoperceptual attributes are more easily accessed by visual input than via lexicalorthographic or lexical-phonological input. Thus, these authors maintain that findings of category-specific and modality-specific deficits do not necessitate multiple semantic systems. They do concede that input modalities access the semantic system in different ways, leading to facilitated access to modality-congruent information about an object before modality-incongruent information. Computational models share an assumption with the OUCH, which is that semantic categories differ by the nature and degree of associations among its members. Computational models uniquely focus on the manner in which these associations interact and predict deficits based on a mathematical model of associations among attributes, sometimes referred to as features. Sproat (1995) argues that computational models force detailed mechanistic explanations behind theory. Sproat further states that a theory without computational appeal poses problems of likelihood, of accuracy, and of credibility. Such a statement demarcates the importance of an explanation of how, not just where, semantic information is laid down and accessed. Small, Hart, Nguyen, and Gordon (1995) used artificial networks and showed that features alone could account for category-specific deficits. These authors encoded line drawings by 77 semantic features, including perceptual qualities, physical attributes, motor associations, and

31 21 functional and associative attributes. From this input, fruits and vegetables clustered most coherently. Other emergent categories included animals, transportation terms, and inanimate items. These authors proposed that category-specific deficits may just be a byproduct of representations built by features that cluster. A dependency analysis revealed what features contributed the most to categorizations: for fruits and vegetables color discrimination contributed the most. For vehicles, composition and characteristics of movement weighted the most heavily. For tools and kitchen items, the number of limbs and movement were most important. Interestingly for animals, a variety of features loaded, with no one feature predominating. A potential criticism of computational models in general, is the leap that must be made from machine to brain. That is, with enough manipulation of the inputs, and the weights among the inputs, computational models may eventually produce the output expected by their makers. However, does this mechanistic process really mirror neuronal processes in the brain? Furthermore these models do not attempt to localize any of the processes involved, usually referring to the process as widely distributed. Conclusion With A Controversy Two case reports with very similar methodologies led to directly opposite findings regarding the organization of semantic information in the brain. Hart and Gordon (1992) reported a patient who following diffuse, mild inflammation involving both temporal lobes, had a naming deficit for animals across modalities of presentation (auditory, visual) with no problems for other living or nonliving things. This deficit in naming animals was observed with possible confounds of frequency, visual complexity and familiarity controlled for. In addition, this patient could not verbally describe the

32 22 physical attributes of animals, while functional properties of the animals were intact verbally and visually. Physical attributes were impaired with verbal input but spared with visual input. According to the PRAUCH such selective sparing would occur due to the privileged relationship between visual input and visual semantic information. However, Hart and Gordon argued that the findings mandate at least two distinct representations: one language based and one visually based. They argued that a dual representation of physical attributes must exist to be spared in one modality (visual) and impaired in another modality (verbal). The authors further concluded that because the naming deficit was limited to animals, sub-domains of knowledge representations exist within distinct modality systems. Finally, because errors were consistent, the authors assumed a representational deficit rather than one merely of access. Caramazza and Shelton (1998) similarly reported a patient who had a deficit in naming animals. However, in this case, both functional and featural aspects of living entities were lost. In addition, the finding was not restricted to one input modality: both featural and functional attributes were disproportionately worse for animals across both verbal and visual modalities. While McCarthy and Warrington (1988) presented a case of a deficit that was both modality- and category-specific, Caramazza and Shelton s patient had a category-specific deficit that spanned across modalities. These authors interpreted their patient s findings as support for a single semantic system with some attributes that cluster. According to these authors, the living-nonliving distinction may be one internal organizational principle, with adaptive implications for survival. Unfortunately, no one theory mentioned above can explain sufficiently semantic deficits that are triply specific,

33 23 that is, category-specific, modality-specific and attribute-specific, e.g., those of Hart and Gordon s (1992) patient. In sum, patient findings have led to entirely opposing conclusions with respect to the organization of semantic information in the brain. One theory is that there are multiple semantic systems with distinct subsystems (Hart & Gordon, 1992; Warrington & Shallice, 1984). Divisions proposed to exist within the semantic system have included either or both modality and semantic category. Others broadly define one semantic system with an internal organization by clustering of like features (Caramazza et al, 1990). Some investigators continue to doubt that findings thus far adequately control for extraneous variables. They argue for a single, amodal semantic system based on the absence of credible evidence to the alternative (Stewart et al., 1992). Matrix Theory The matrix theory of semantic processing has recently been proposed (Crosson et al., 2000) in an attempt to best explain the seemingly discrepant findings arising from the literature on category-, modality- and attribute-specific naming deficits. A visual schematic of this theory is provided using four sample categories (Figure 2). According to this theory, the matrix includes two content modalities, verbal and visual, as proposed by McCarthy and Warrington (1988). In order to explain Hart and Gordon s (1992) findings of a patient who could not provide verbal descriptions of visual information, one must assume two dissociable systems that are most easily accessed by their corresponding input modality. This patient could not access visual information when using the verbal modality. Within the two modalities, this theory postulates a parallel

34 24 internal organization by category. The category-specific deficits reported either span verbal and visual modalities or they are confined to one. Coltheart et al. (1998) have also argued for three classes of semantic impairment that can occur either independently or simultaneously. That is, they predict that it is possible to have a singly, doubly or triply specific impairment with respect to attribute, Category A Category B Category C Category D Category A Category B Category C Category D Visual form Color Emotional Attributes Motoric Action Verbal Modality Visual Modality Category A = Things to do at an Amusement Park Category B = Natural Disasters Category C = Animals Category D = Tools Figure 2. Model of matrix theory. category and modality. A triply specific semantic deficit would be one for a sole category, confined to one modality and specific to only one class of attributes. In partial support of their model, they report a patient whose deficit was specific to visual semantic

35 25 attributes (Table 1). The deficit was unusual in that it was neither category- nor modalityspecific. Their conclusion was that two functionally and neurally distinct subsystems exist in the semantic system, one that contains information about visual attributes of objects (corresponding to visual modality) and another that mediates non-perceptual properties (corresponding to verbal modality). The matrix theory argues for a similar dichotomy, which we label as verbal and visual content modalities. The matrix theory also provides a topographical explanation that attempts to resolve the seemingly disparate findings of Hart and Gordon (1992) and Caramazza and Shelton (1998). According to the matrix theory, attributes, including but not limited to visual form, color, emotional attributes, and motoric action are incorporated into object representations in the temporal lobe via converging input from cortical, and limbic structures. This incoming information converges in the temporal lobe in a way that is not random. The placement of categorical information is driven by the convergence of the incoming information streams (consisting of attributes) that are most salient for distinguishing a class of items. The categorical division along the temporal lobe emerges as common defining attributes cluster based on inputs to heteromodal cortex. An approximate matching of categorical information is stored within the neighboring verbal and visual modalities. This topography allows the possibility for deficits that are singly, doubly or even triply specific (Table 1 gives examples of such deficits). Results from imaging studies are another line of evidence by which the matrix hypothesis may be tested.

36 26 Imaging Studies of Semantic Processing Observations of patients have direct implications for theories of the functional organization of the brain, but a problem with case studies is that lesions usually are not circumscribed to brain structures of interest for experimental hypotheses. Second, the extent of damage contributing to the resulting language deficit is not always clear. Third, with notable exceptions (Damasio et al., 1996; Tranel et al., 1997) most patient studies are reported on a case-by-case basis, which makes generalization difficult. Finally, lesion studies cannot implicate areas outside the damaged area that are also involved in the impaired function under normal circumstances. In comparison to the contribution from lesion studies, functional neuroimaging provides a broader perspective with respect to the network of brain structures that are involved in language processes. Binder et al. (1997) expanded the classical model of brain areas mediating semantic processing that were originally identified using patients with language deficits subsequent to brain insult. Areas outside the classical model identified by Binder and colleagues (1997) included left temporoparietal activation outside the traditional Wernicke s area (BA 22) and angular gyri; including middle and inferior temporal gyri, the fusiform gyrus, as well as extensive left prefrontal activation outside the traditional Broca s area (BA 44,45). The task used to map semantic processing in this study was not designed to identify differences in patterns of semantic processing based on semantic category, or modality of input. The task involved listening to animal names and making semantic decisions about them. The control task was a tone discrimination task used to subtract out prelinguistic auditory processing. Neither category-specific effects nor the effect of emotional connotation of the stimuli were considered in this study.

37 27 Numerous attempts to further characterize the mapping of semantic information using functional imaging have been performed, but with a disappointing lack of convergence among results. All find unique patterns of activation associated with the semantic categories examined. However, little consistency emerges across studies such that definitive conclusions cannot yet be drawn. Damasio et al. (1996) using PET revealed distinct patterns of activation in the temporal lobe for word retrieval of familiar faces, animals and tools. In comparison to a control task involving passive viewing of unfamiliar faces, either upside down or right side up, naming of line drawings of animals and tools activated left inferior temporal lobe posteriorly. In addition, naming from both categories activated a portion of the left temporal lobe. Activation during tool naming was seen in posterior middle and inferior temporal gyri, 12 mm posterior and 15 mm lateral to the region activated by nami ng animals. Martin et al. (1996) also used PET to study category effects on the pattern activity that occurs during naming of line drawings and similarly revealed unique patterns of activation associated with naming animals and tools. However, ventral temporal activation did not differ in the Martin et al. study between animals and tools as it did in the Damasio et al. (1996) study. In comparison to viewing nonsense objects, both naming of tools and animals led to bilateral fusiform gyrus activity in the ventral temporal lobes. Direct task comparisons revealed tool naming to be uniquely associated with left premotor activation and activation in the left middle temporal gyrus. This area was similar to one that was found in another study by Martin, Haxby, Lalonde, Wiggs, and Ungerleider (1995) during which participants generated action words associated with objects. Despite similar methodologies, the Martin et al. (1996) and the Damasio et al.

38 28 (1996) studies show little convergence. For an illustration of differences in findings using similar methodologies to study animal versus tool processing confined to visual input modality (Table 2). Table 2 clearly shows what little overlap occurs in comparison of the peaks of activation from similar direct task versus task comparisons. Such inconsistency is present as well for several other studies which have investigated category differences using either pictures (Moore & Price, 1999; Perani et al., 1995) or words (Cato et al., 2000; Mummery, Patterson, Hodges & Price, 1998; Mummery, Patterson, Hodges, & Wise, 1996). For example, activation in the medial temporal lobe has been reported during naming of animals versus tools (Damasio et al., 1996), during naming of tools versus animals (Martin et al., 1996) and during semantic processing of man-made versus natural objects (Mummery et al., 1996; Mummery et al., 1998). Some sources for discrepancies among imaging studies include: type of imaging performed (thus, differences in temporal and spatial resolution), number of participants, and data analysis procedures. A difference among studies with potentially confounding outcomes is the control task selected to subtract away activity non-specific to the task of interest. For example, the control task used by Damasio et al. (1996) involving viewing of faces was certainly not free of semantic processes, the cognitive domain of interest. Some functional imaging studies have examined the impact of input modality on the pattern of brain activation. Vandenberghe, Price, Wise, Josephs, & Frackowiak (1996) reported that inquiries about knowledge of visual attributes versus knowledge of associations between concepts did not lead to significant differences in the pattern of brain activation; while type of input modality, word versus picture, did lead to some differences. For pictures only, the left posterior inferior temporal lobe was activated;

39 29 while for words only, left superior temporal and middle inferior frontal regions were activated. Thompson-Schill et al. (1999) investigated category-specific processing from a modality-specific theoretical standpoint whereby categories are emergent properties of Table 2: Comparison of functional neuroimaging findings of unique activation with animal versus tool comparisons using picture presentation. Task Reference Imaging Maximal Peaks of Functional Activity Sig. Naming line drawings: animals tools Naming line drawings: animals tools Naming black and white photographs: animals>tools; control: pixelated pictures Naming silhouettes: animals>tools Object recognition: animals>tools; control task: animals vs. shapes Martin et al., Damasio et al., 1996 Animals > Tools PET: whole brain PET: ROI bilateral temporal lobes Chao et al., FMRI: whole brain Martin et al., 1996 Perani, Schnur, Tettamanti, Cappa, & Fazio, PET: whole brain PET: whole brain Left calcarine sulcus (-4,-80, 8) Left frontal lobe (-26,-6,-24) Left frontal lobe (-26, 28, 16) Left posterolateral inferotemporal cortex (-57, -54, 0) Right inferior occipital gyrus (41, -80, -10) Left inferior occipital gyrus (-36, -80, -10) Right lateral fusiform gyrus (37, -52, -20) Left lateral fusiform gyrus (-37, -55, -20) Left medial occipital region (-18, ) Left medial occipital region (-2, -84, -4) Left and right fusiform gyrus/inferior occipital cortex (-44, -82, -32), (-46, -82, -20) z = 3.12 z = 2.50 z = 2.41 p =.04 12/14 9/14 14/14 12/14 p <.05 p <.05 z = 3.1 z = 3.1

40 30 Table 2 continued Task Reference Imaging Maximal Peaks of Functional Activity Naming line drawings: tools animals Naming line drawings: tools animals Naming black and white photographs: tools>animals; control: pixelated pictures Naming silhouettes: tools>animals Object recognition: tools>animals; control task: animals vs. shapes Martin et al., Damasio et al., 1996 Tools>Animals PET: whole brain PET: ROI bilateral temporal lobes Chao et al., FMRI: whole brain Martin et al., 1996 PET: whole brain Perani et al., PET whole brain Note. In parentheses are the Talairach coordinates. 1 z > 2.32 (p <.01, 1-tailed); z > 3.09 (p <.001, 1-tailed) 2 p <.05 for all task comparisons. Left middle temporal gyrus (-36, -50, 4) Left anterior cingulate (-6, -38, 2) Right supramarginal gyrus (48, -50, 24) Left lateral inferior frontal area (-52, 10, 20) Left premotor cortex (-48, 0, 20) Left inferotemporal cortex (-37, -40, -7) Right medial fusiform gyrus (26, -47, -16) Left medial fusiform gyrus (-27, -50, -15) Left middle temporal gyrus (-45, -57, 7) Left middle temporal region (-40, -62, 4) Left premotor region (-42, 0, 20) Left inferior frontal gyrus (-54, 8, 8) Sig. z = 2.90 z = 2.78 z = 2.50 z = 2.85 z = 2.74 p =.006 8/14 13/14 12/14 p <.05 p <.005 z = 3.2

41 31 modality-specific representations. They assume an object s semantic representation is widely distributed. According to their conceptualization, if any part of the representation is disrupted, it may affect the remaining components. They also propose that different categories of objects differ by the predominant modality by which it is processed. If access to modality-specific information is disrupted, the authors speculate that an entire category with high dependence to this modality may be disrupted, regardless of input modality. That is, one type of knowledge, whether visual or verbal, may be obligatory for the retrieval of certain categories of objects. They refer to this hypothesis as the interactive modality-specific hypothesis. In their fmri study, Thompson-Schill et al. (1999) specified the left fusiform area as a region of interest on an a priori basis. Questions were posed about either visual or non-visual characteristics for living and nonliving entities. These authors found increased fusiform gyrus activity in response to queries about the visual characteristics of either living or nonliving entities. However, while left fusiform activity was found for retrieval of non-visual knowledge of living things it was not present during retrieval of non-visual information of nonliving things. These authors concluded that semantic processing of living things (animals) will always lead to left fusiform activity this may not be so during non-visual semantic processing of nonliving entities. Such a conclusion is based on an assumption that while visual information is more critical for the discrimination among close semantic relatives, this is not true to the same extent for nonliving things. This conclusion may not be valid, however, as activation was in fact present in the inferior temporal lobe for non-visual information about nonliving things. This area of activity was

42 32 very close to the fusiform gyrus activity seen during non-visual semantic processing for living things. Among these neuroimaging studies which attempt to validate and explain case reports of semantically- or modality-bounded deficits, findings have been quite disparate with some exceptions. One very consistent finding is that semantic processing of concrete (visualizable) objects (living and nonliving) most often activates areas near the fusiform gyrus. In a left hemisphere fmri study (Cato et al., 2000) in which participants heard objects from three different semantic categories and made decisions about them, fusiform gyrus activity was present relative to the baseline task for all three categories (tools, animals, and words with emotional connotations). This finding suggests that semantic processing of all highly visualizable entities, animate or inanimate, depends in part on visual form. Two other studies discuss the involvement of the fusiform gyrus during semantic processing. These studies provide evidence for fractionation of the function of the fusiform gyrus based on semantic category. Chao et al. (1999) argue for a medial/lateral semantic distinction in the fusiform gyrus. Using a number of language and picture tasks that compared processing of animals versus tools, these authors found that while animal stimuli activated greater lateral fusiform gyrus activity bilaterally relative to tools, presentation of tool stimuli was related to greater activation of the medial portion of the fusiform gyrus bilaterally relative to animals. Chao et al. also found that pictures of houses activated the medial portion of the fusiform gyrus. In a study from the same laboratory, Ishai et al. (1999) provided yet another look at what may be a topographical organization of semantic information in the inferior temporal lobe. These authors, using

43 33 fmri, found circumscribed regions in bilateral ventral temporal cortex that differentially responded to processing pictures of faces, houses and chairs. Ishai and colleagues reported a medial fusiform area responding maximally to houses, in agreement with their colleagues, Chao et al. While activation associated with processing of these objects was not limited to ventral temporal cortex, support was strong from this group for a topographical organization at least within the ventral temporal lobe, along the ventral visual pathway. Such support, if bolstered by converging evidence from other groups and other methods of study, would provide strong evidence for a fractionation along the temporal lobe that can account for category-specificity. However, it should be noted that in both studies, authors argued that category-specific effects in the fusiform gyrus were related to the differences in the attributes distinguishing items within the various categories. If there are consistent differences in brain activation for different semantic categories, the areas for these differences have not been illuminated with any convincing regularity. More importantly, a resolution between modular and unitary theories of semantic processing must be offered in order to better conceptualize the findings to date in both the lesion and imaging literature. The matrix theory may provide a framework to begin to map findings to neuroanatomy and to begin to make more specific conclusions with respect to the organization of semantic information in the brain. In addition to investigating brain activation associated with semantic processing for animal and tool names, Cato et al. (2000) compared items with emotional connotations (living and nonliving) to prototypical living (animals) and nonliving (tools) items using fmri. Participants listened to words and made semantic judgments about

44 34 them. A major purpose of the study was the delineation of brain regions involved in processing emotional connotation as a semantic attribute. Before revealing the unique substrates associated with semantic processing of words with emotional connotations found in this and other related functional neuroimaging studies, a review of current knowledge regarding the biological substrates, both peripheral and central, of emotional processing will be offered, as well as a number of theories of the mapping of emotions in the brain.

45 EMOTIONAL PROCESSING In this chapter, the focus will be on the perception (input) of emotional stimuli rather than the expression (output) of emotions per se. The term perception of emotional stimuli refers here to the recognition that a given stimulus input has emotional connotation. This rather broad area includes input at the object and at the symbolic level, but emphasis will be placed on the symbolic level (i.e., pictures and words). The nature of emotional processing will be explored in order to arrive at specific hypotheses regarding differences in brain activation when generating words to categories with either positive, negative or neutral emotional connotation. Emotions will first be defined and differentiated from the related concepts of affect and emotional connotation. The biological correlates of emotions will then be briefly offered with a focus on the neuroanatomy of emotional processing. What has been proposed to date of hemispheric asymmetries with respect to emotion will be reviewed. Finally, findings of sex differences in physiological and evaluative responses to emotional stimuli will be covered to serve as rationale for hypotheses about sex differences in processing of words with emotional connotations. Definitions of Emotions Rolls (1995) defines emotions as states produced by instrumental reinforcing stimuli (stimuli that if their occurrence, termination, or omission is made contingent upon 35

46 36 the making of a response, alter the probability of future emission of that response). According to Rolls, emotions can be classified by reinforcement contingencies: Positive reinforcers may elicit pleasure, elation, ecstasy; negative reinforcers may lead to apprehension, fear, terror; removal or omission of a positive reinforcer may be followed by frustration, sadness, anger, grief, rage; removal or omission of a negative reinforcer may result in feelings of relief. Rolls (1995) cited factors that may relate to the arousal level and expression of emotions including, reinforcement contingency, intensity of reinforcer, and whether the reinforcer is primary or secondary. According to Rolls (1995), to understand the neural bases of emotions, one must consider what brain areas are involved in reward and punishment. He cites some of the functions of emotions as: elicitation of autonomic responses, flexibility of behavioral responses to reinforcing stimuli, nonverbal communication, social bonding, and facilitation of episodic memory storage. Emotions are also conceptualized using dimensional models. Valence (ranging from positive to negative) and arousal (ranging from low to high) are two frequently considered dimensions (Lang, 1995). Heilman, Bowers, and Valenstein (1993) posit motor intention-activation as another continuum that is important to consider. Davidson (1992) heralds the dimension of approach/withdrawal as the most basic due to its phylogenetic primacy. Discussion of dimensional models will be discussed in detail later in this chapter under the bivalent hypothesis. An emotion, therefore, can be viewed as a reaction or action disposition (Lang, 1995) to an evocative stimulus or stimulus-complex. The reaction that ensues can lead to emotionally-related cognitions (appraisal, perception), as well as subjectively

47 37 experienced feelings, autonomic and neural arousal, expressive behavior and goaldirected activity (Borod, 1992). The degree and direction (appetitive or aversive) of the reaction can be predicted somewhat by the normative ratings of valence and arousal level, as well as by normative information regarding changes in physiological arousal related to exposure to the stimulus. However, past experience with the stimulus determines the extent and type of reaction in each individual. Affect, unlike emotion, pertains to a mood state, rather than a discrete response to an emotionally evocative stimulus. Emotional connotation refers not to any emotional reaction or state but to the cognitive recognition that an object or action is considered by most individuals (or specific individuals) to have emotional significance consisting of an emotional valence and a potential to evoke some level of arousal. Emotional entities, such as roaches, babies, vomit, and desserts most likely lead to emotional reactions at the object level. Pictures of such objects may also be emotionally evocative. A less obvious question is whether processing such objects at the word level differs in any way from processing of emotionally neutral words. Peripheral Correlates of Emotions Feedback theorists assume a covariance of peripheral physiology and verbal reports of emotion. Some feedback theorists argue that physiological arousal not only precedes but determines subjective emotional experience. One such line of evidence comes from a study by Hohmann (1966) in which 25 male spinal cord injured patients with reduced visceral feedback reported reduced emotional experience in the areas of sexual excitement, anger, fear, and self-estimation of overall emotional feeling. Interestingly, all but two patients reported an increase in sentiment, perhaps meaning that these patients were experiencing depressive adjustment reactions.

48 38 Biological support does exist for at least a partial role of peripheral contributions to emotional experiences. Vagus nerve projections to cortex via the nucleus of the solitary tract do provide a source of feedback from the viscera to the brain (Heilman, Bowers, & Valenstein, 1993). The role of autonomic activity in emotion is a much studied area, although still a source of debate. Such psychophysiological indicators as facial electromyographic (EMG) measures, phasic galvanic skin response (GSR), and heart rate (Greenwald, Cook, & Lang, 1989) have been identified to vary with emotional valence. Greenwald et al. monitored participants with psychophysiological equipment while they viewed pictures and rated them for valence and arousal. They found that corrugator EMG increased with lower valence ratings (increased EMG with increasingly negative valence). Zygomatic EMG was found to increase with ratings of either positive or negative valence, but more so with pleasant ratings. In addition, cardiac acceleration was significantly related to more pleasurably rated pictures, with an accompanying deceleration to more negatively rated pictures. Finally, a greater GSR was associated with higher arousal judgments. Together, these findings suggest a relationship between valence, arousal and certain physiological indices. Another line of evidence comes from startle probe experiments. Modulation of the startle reflex has been demonstrated using emotionally evocative stimuli. A startle reflex can be evoked by a number of stimuli and using a number of input sensory modalities, including auditory (a sudden loud noise), tactile (a puff of air to the eye) and visual (a flashed light stimulus). Stimulus presentation is brief (~50 ms), and the startle response consists of a complex reflex reaction including a GSR and a reactive eyeblink (measurable by EMG). Startle response has been found to vary with the affective valence

49 39 of a pictorial foreground. This has been interpreted by some as a sort of emotional response priming (Bradley, Cuthbert, & Lang, 1996; Lang, Greenwald, Bradley, & Hamm, 1993). That is, startle stimuli are assumed to be noxious, eliciting a defensive response. Such a reflex is enhanced by a concordant ongoing response (as inducted by viewing a picture with negative valence in the foreground). Furthermore, such reflexes are diminished when the reflex is a mismatch with an ongoing positive response to a picture with a pleasant valence. In fact, relative to viewing neutral pictures, the amplitude of the eyeblink reflex and the skin conductance response is increased when viewing unpleasant pictures and attenuated when viewing pleasant pictures. Such evidence provides support for a covariance between valence of stimuli and degree of visceral response, with a greater magnitude of arousal for negative emotionally evocative stimuli. However, much of the debate over the role of autonomic activity in emotions persists due to tremendous variability across studies in the patterns of activity associated with discrete emotions (Davidson, 1995). Central Correlates of Emotions Paul Broca identified (1878) the ring of structures, including parahippocampal gyrus, cingulate gyrus, subcallosal gyrus and underlying hippocampal formation, surrounding the brain stem, and he named these le grand lobe limbique. At that time, the role of these structures was not yet considered to be related to emotions. Bard, in 1928, first delineated the role of the diencephalon (thalamus and hypothalamus) in the rage response of decorticate cats. He later surmised that the excessiveness of the sham rage, along with its easy elicitation was suggestive of disinhibition due to a release from higher (cortical) control (Bard, 1934). Cannon (1927) cited the hypothalamus as a key

50 40 structure for emotional responses, as it modulates endocrine and autonomic nervous system responses following presentation of emotionally salient stimuli. While he did not implicate cerebral cortex, Cannon argued that vasomotor processes are too diffuse to serve any other function than a generalized mobilization of energy. In 1937, Klûver and Bucy highlighted the amygdala as an emotionally relevant brain structure. They identified a syndrome in monkeys that follows bilateral anterior temporal lobectomy. Among numerous consequences, loss of fear and flat affect ensue. In humans, the effect of damage limited to the amygdala has been studied in patients suffering from Urbach-Wiethe disease, a condition that leads to bilateral destruction of this brain structure. In one such patient reported by Adolphs, Tranel, Damasio, and Damasio (1994), impairments in the recognition of facial expressions of fear was demonstrated. Human studies reveal lesions of the amygdala abolish or reduce fear conditioning (LaBar, LeDoux, Spencer, & Phelps, 1995; LeDoux, 1992). LaBar, Gatenby, Gore, LeDoux, & Phelps (1998) used echoplanar fmri to investigate the role of the amygdala in simple discrimination conditioning to a visual cue predicting shock or another presented alone. Activation was observed during both conditioned fear acquisition and extinction in periamygdaloid cortex and was biased toward the right hemisphere. These authors concluded that the amygdala may be involved in the detection of emotional stimuli when they are novel as it may be involved in the initial encoding of emotionally salient stimuli. Papez (1937) defined a loop demonstrating an interaction between limbic structures and other brain areas that are now known to mediate emotional processing. Cortex acts upon the hypothalamus via connections from the cingulate gyrus to the

51 41 hippocampal formation. Information processed in the hippocampus is projected via the fornix to the mammilary body, and then via the mammilothalamic tract, to the anterior thalamic nucleus. The loop is closed with the projections from the anterior thalamic nucleus back to the cingulate gyrus. Maclean and Yakovlev have each been given credit for incorporating other loops of brain structures to the neural circuitry of emotions, including the basolateral components proposed by Yakovlev (1948; orbitofrontal, insular, and anterior temporal lobe cortex, the amygdala, and the dorsomedial nucleus of the thalamus). Maclean (1952) incorporated the septal area and the nucleus accumbens of the archistriatum into the constituents of limbic circuitry. Thus, cortical and subcortical brain structures contribute to systems that are considered to mediate emotional experience. The greater contribution of the brain than the peripheral nervous system in the perception of emotions is well accepted and is buffered by many arguments. One such argument highlights the discrepancies between physiological response and self-report. Mandler, Mandler, & Uviller (1958) examined two groups; one group of participants reported a tendency to perceive a high level of autonomic reactivity in fear situations and another group tended to perceive a low level of autonomic reactivity. These authors found a tendency for the high perceivers to overestimate their autonomic responses and for the low perceivers to underestimate their autonomic responses. Likewise, Lang and associates have established some discordance between physiological correlates and verbal reports of emotion. Lang s research with anxious/fearful patients has yielded discordance between judgments of fear, fear behaviors and physiological reactivity (Lang et al., 1993).

52 42 Central theorists argue that emotional cognition is a process in which cortex appraises and interprets stimuli before emotional experience. Heilman et al. (1993) argue that a plausible central model is one in which cortex appraises and interprets stimuli, and based on this process, either activates or inhibits the reticular system via direct connections or via the limbic system. Heilman and colleagues (1993) posit unique neural processes associated with each emotion, and that neocortex derives meaning from the stimulus. Further, four axes of determinants combine to create a particular emotional experience; 1) emotional cognition, 2) arousal, 3) motor intention-activation and 4) approach-avoidance. Heilman et al. posit that almost all emotional experience is a product of top-down process, unless the reaction is a conditioned response, which would be subcortically mediated. Further, cognitive interpretation helps determine valence. Since arousal is associated with valence; cognitive interpretation can mediate arousal (Heilman et al., 1993). Hemispheric Asymmetries Related to Emotional Processing Hemispheric asymmetry refers to the notion that certain higher cortical functions are differentially represented in the two hemispheres. Cerebral dominance refers to a relative, rather than an absolute relationship. Findings of hemispheric asymmetries related to emotional processing lack the consistency of those related to cognitive processes (e.g., language). Methods that have been used to investigate emotional asymmetry in normals include dichotic listening tasks and tachistoscopic visual presentation to either hemispace. Clinical studies of patients with focal lesions restricted to either hemisphere are abundant as well. There are two general models of emotional processing: the right-hemisphere model (RH) and the valence model.

53 43 Right Hemisphere (RH) Model The RH model proposes a right-hemisphere dominance in the expression and perception of emotion, regardless of valence. A rationale for such a model is that the right hemisphere is presumably superior in nonverbal, integrative perceptions, as well as in visuospatial organization and visual imaging. All these right hemisphere processes would appear to be important abilities for accurate perception of emotions (Borod, 1992). The right hemisphere (specifically, right parietotemporal regions) is also presumed to have a greater involvement with mechanisms of autonomic arousal and behavioral arousal in emotional states (Heller, 1993). In normal individuals, a right-hemisphere advantage has been reported for the perception of facial emotional expressions (Bowers & Heilman, 1984). Dichotic listening experiments have demonstrated a left-ear advantage for processing the emotional contour of speech, nonverbal vocalizations, and musical passages (Borod, 1992). Clinical evidence in support of this model comes largely from patients who have suffered right hemisphere strokes, or other focal cerebral lesions. Heilman, Schwartz, and Watson (1978) found that the galvanic skin response to noxious stimuli was decreased in right-hemisphere lesioned patients. Right-hemisphere lesioned patients are also found to be impaired with nonverbal aspects of emotional processing including comprehension of prosody (Blonder, Bowers, & Heilman, 1991) and facial expressions (Blonder, Bowers & Heilman, 1991; Peper & Irle, 1997). Examination of effect of lesion site revealed in one instance that patients with right temporal and parietal lesions were more impaired in decoding emotions than patients with right frontal lesions (Peper & Irle, 1997).

54 44 Valence Model Two versions of the bivalent hypothesis exist. One version of the bivalent hypothesis is that the two cerebral hemispheres are specialized for processing a particular emotional valence. According to this valence model, the left hemisphere mediates positively-valenced emotional processing while the right hemisphere mediates processing of negative valence. Another version of the bivalent model contends that while hemispheric specialization can be applied to the expression and experience of emotion as a function of valence, the right hemisphere is dominant in the perception of emotions of both valences (Borod, 1992). If different brain systems are invoked by the quality of response engendered by a negative versus a positive stimulus, then differences in brain involvement by valence would be expected. Many negative emotions are linked to survival (e.g., fear, disgust), and perhaps fewer positive emotions have such a strong link with survival. A response to a life-threatening stimulus would require quick resolution with greater attention to the gestalt (right-hemisphere-mediated) than to fine detail. Positive emotions, on the other hand, may be more mediated by linguistic and communicative skills and are not so immediate. Another dimension that has been related to emotions, approach-withdrawal (Davidson, 1984), may also serve to support a bivalent model. According to Davidson, withdrawal behaviors are linked with the right hemisphere, which is specialized for undifferentiated automatic movement. Approach behaviors, on the other hand, are linked with left-hemisphere mediated processes such as sequentially executed movements, and fine manual control. Davidson s model (1984) is one that advocates the second version of the bivalent model and stipulates that only frontal regions show differential specialization

55 45 with respect to valence. According to his theory, in posterior regions, the right hemisphere predominantly mediates emotional processing. Davidson and colleagues have used EEG to examine asymmetric brain activation in response to stimuli evoking positive and negative emotions. Induced negative emotions (disgust, fear) have been associated with right-sided anterior activation whereas positive emotions lead to left-sided increases (Davidson, 1995). Davidson (1995) posits that activation in the left prefrontal region may be part of a mechanism that inhibits negative emotions. Imaging Studies Relevant to the RH-Bivalent Debate Lane and colleagues (1997a) used positron emission tomography (PET) to examine the neural correlates of happiness, sadness and disgust in 12 neurologically normal females. Each of these emotions differ either in valence and/or action tendencies (approach/withdrawal). Lane s group attempted to induce each of the three types of emotion in two ways, by film and by autobiographical recall. All three emotional states were associated with increases in activity in the thalamus (sadness and disgust bilaterally, happiness only on left) and medial prefrontal cortex (sadness and disgust only on left, bilaterally for happiness). Anterior temporal cortex and posterior temporal cortex were activated by all three emotions during film-induced recall. Recall-induced sadness was associated with increased activation in the anterior insula. Happiness was distinguished from sadness by greater activity in both ventral anterior cingulate and ventral mesial frontal cortex. Although unique patterns of activation were found for each emotion, the patterns of activity were generally symmetrical, with no significant asymmetries noted. Therefore, this study failed to support either the bivalent or the RH hypothesis.

56 46 Lane et al. (1997b) examined appetitive (pleasant) versus aversive (unpleasant) motivational states with PET. Twelve neurologically normal females viewed sets of pictures [International Affective Picture System (IAPS), Lang, Bradley & Cuthbert, 1995] that were either pleasant, unpleasant or neutral. Overall, viewing of emotional pictures regardless of valence led to increased cerebral blood flow in medial prefrontal cortex, head of the left caudate nucleus, left thalamus, left hypothalamus, and left midbrain. Unique to viewing unpleasant pictures was associated bilateral activation of occipito-temporal cortex and cerebellum, left parahippocampal gyrus, left hippocampus, and left amygdala. Thus, primarily left-lateralized activation was found for both positive and negative pictures, with no trend towards lateralization by valence. Interestingly, in this study, skin conductance was measured and revealed significantly higher arousal for negative than for positive pictures (a possible confound). Significant asymmetry was observed during unpleasant picture viewing relative to neutral picture viewing that centered on the left hippocampal gyrus. Overall, there was substantial but not complete overlap between pleasant and unpleasant emotions. As in Lane et al., (1997a), the thalamus and medial prefrontal areas were activated by emotional stimuli relative to neutral stimuli. A recent whole brain fmri study attempting to induce emotional experience, either positive or negative, with both pictures and captions, also did not lend support for the valence hypothesis (Teasdale et al., 1999). These authors found similar areas of activation in the medial frontal gyrus (BA 9) and right anterior cingulate gyrus (BA 24 and 32) for emotional picture-caption pairs regardless of valence. Lang et al. (1998) examined brain activation in posterior cortex using fmri. Participants viewed pleasant, neutral and unpleasant pictures from the IAPS (Lang,

57 47 Bradley & Cuthbert, 1995) system. Twelve men and eight women were studied, allowing examination of potential sex differences. Lang et al. found that functional activity was greater overall when processing emotional versus neutral pictures. In addition, functional activity associated with processing pictures with emotional connotations was greater in the right hemisphere, lending clear support for the RH hypothesis. Emotional relative to neutral pictures led to bilateral sizable clusters in occipital gyrus, right fusiform gyrus and right inferior and superior parietal lobules. Canli, Desmond, Zhao, Glover, and Gabrieli (1998) is the sole group to find support for the valence hypothesis in an imaging study. Using whole brain acquisition, Canli et al. found support for the valence hypothesis only for that subset of participants for which arousal ratings were equivalent across positive and negative pictures. This suggests that arousal may be a confounding factor in previous null findings regarding differences in activation between positively and negatively valenced stimuli. When arousal was equated across positive and negative pictures, Canli et al. found greater activation in the left hemisphere during processing of positive pictures and greater activation in the right hemisphere during processing of negative pictures. Thus, the findings were consistent with the valence hypothesis when arousal was equated across emotional valences. Sex Differences in Emotional Processing Some researchers avoid the potential for sex differences in emotional research by selecting one sex to study. Therefore, although it is generally accepted that some differences may exist, it is not yet clear how what differences occur physiologically and in brain activation. In the research that does examine relatively equal numbers of both

58 48 sexes, differences in reactions have been reported. More research is available in the area of peripheral findings than differences in brain activation. However, in the Lang et al. (1998) fmri study investigating brain activation in posterior cortex during viewing of pleasant versus unpleasant pictures, sex differences were found in response to emotional stimuli in the more active right hemisphere. Women showed significantly more functional activity for unpleasant than pleasant stimuli, while men showed the opposite pattern. Many more reports in the literature reveal sex differences in peripheral reactions to emotional stimuli. Lang, et al. (1993) found that significantly more women than men showed concordance between valence judgments of pictorial stimuli and facial tension as measured by corrugator and zygomatic EMG. Men showed greater concordance between skin conductance and viewing time and a closer relationship between arousal ratings and skin conductance changes. Greenwald et al. (1989) found that relative to males, females valence judgments were more broadly distributed: they rated danger-related materials as more unpleasant and appetitive materials as more pleasant. There was a trend, though not significant, for males to rate most danger-relevant and erotic slides as significantly more arousing than females. Also in this study, females showed a lower correlation between valence and arousal compared with males. In this study, females were found to produce greater mean zygomatic response than males over all levels of valence. Again males showed greater concordance between skin conductance magnitude and arousal ratings.

59 EMOTIONAL SEMANTICS Lang et al. (1993) suggested that pictures lead to greater autonomic, myographic and visceral responses than words due to the closer semantic distance pictures have to the original object. Words, on the other hand, are mere symbols of the object and as such are further removed from any emotional response (Lang et al, 1993). Therefore, a distinction must be made between emotional stimuli and words with emotional connotation, the latter being the focus of the current study. The words that were used in the current study have a valence and an arousal rating that indicate an emotional connotation. However, an emotional reaction to these words, if expected at all, is anticipated to be markedly attenuated from that which would be expected from its pictorial representation. In the following section, research examining the differences between neutral words and those with emotional connotations will be examined. Experimental Findings Concerning Emotional Semantics While it is commonly accepted that comprehension of nonverbal affective signals such as facial expression, gestures and prosody are mediated by the right hemisphere (i.e., Blonder, Bowers & Heilman, 1991), less is known about the lateralization pattern, if any, associated with processing of emotional verbal stimuli. In fact, Blonder, Bowers & Heilman (1991) found that right-hemisphere damaged patients were able to understand sentences with emotional content so long as facial, prosodic or gestural expression were 49

60 50 excluded from the content of the sentence. Cicero et al. (1999) examined lexical emotional perception in patients with either left or right brain injury and found support for the right hemisphere (RH) hypothesis. Specifically, patients with right brain injury were significantly impaired relative to those with left brain injury and neurologically normal participants on emotional identification tasks such as identifying the emotional category that matches with a group of words with emotional connotation (e.g., pairing the stimuli putrid, slime, stench to the category disgust ). Much of the research on perception of emotional verbal stimuli has been done on memory for words with emotional connotations, and great attention has been placed on patient populations. Phelps, LaBar and Spencer (1997) found that both neurologically normal subjects as well as patients with unilateral temporal lobectomies (studied 13 left and 13 right) have enhanced recall for words with emotional connotations, and enhanced recall for neutral words embedded in sentences with emotional contexts. Thus, neither patients with left or right unilateral temporal lobectomies experienced difficulty processing emotional content. With respect to neurologically normal individuals, tachistoscopic procedures have been used to examine the speed and accuracy of processing words with emotional connotations. Ali and Cimino (1997) found support for the valence model in the pattern of performance of participants on a recognition memory task. Given the verbal nature of the stimuli, there was a right visual field (RVF)/left hemisphere (LH) advantage in perception and recall of all words, emotional and neutral. Nevertheless, response accuracy for words with positive connotations was greater than words with negative connotations when flashed to the RVF/LH. This study failed to replicate a previous

61 51 finding (Graves et al., 1981) of a relative left visual field/ right hemisphere advantage for the perception of emotional words relative to neutral words. Thus, tachistoscopic studies using neurologically normal individuals lend partial support for the valence hypothesis in processing of words with emotional connotations. One potential reason for the heterogeneity of findings even in the area of tachistoscopic studies, is potential interaction effect between sex of participants and valence of words. Using similar procedures as Ali and Cimino (1997), Coney and Fitzgerald (2000) found that while women showed no differences in lateral asymmetry as a function of the emotional connotation of nouns presented, the lateral asymmetry (left hemisphere bias) was significantly greater in men for positive words only, revealing a confirmation of the valence hypothesis, when sex of participants is considered. Previous Imaging Studies of Emotional Connotation as Semantic Attribute Maddock and Buonocore (1997) used fmri to compare brain activation while participants listened to threat-related versus neutral words. Relative to neutral words, listening to threat-related words led to activation of left posterior cingulate gyrus in eight out of 10 participants. The activation was most prominent in the retrosplenial region. While the precise location of activity in this region is questionable due to its proximity to a large vein, this study raises the possibility that a consistent and unique pattern of brain activation occurs when processing words with emotional connotations. While the words with emotional connotations used in the Maddock and Buonocore (1999) study were all threat related words, other findings of retrosplenial activity corresponding with processing of emotional stimuli would suggest that the activity occurs with emotional stimuli, regardless of valence. Following this study, Maddock (1999) performed a review

62 52 of the functional neuroimaging studies that examined brain activation associated with emotionally salient stimuli. Maddock found that among the 20 studies reviewed, the retrosplenial cortex was one of the cortical regions most consistently activated by stimuli with emotional connotations. Beauregard et al. (1997) used PET to investigate brain activation during viewing of concrete nouns, abstract nouns and nouns with emotional connotations in contrast to viewing random-letter strings. Baseline tasks were passive viewing of plus signs or passing viewing of plus signs with a preceding instructional set. Concrete words consisted entirely of animal names. Words with emotional connotations were selected for high emotional valence but it is not clear if arousal or type of valence was equated. Furthermore, it is not clear if all emotional words were abstract (Examples given: sex, murder and sadness). Viewing words with emotional connotations subtracted from the baseline task led to unique activity in midline orbital frontal gyrus (BA 11), the left inferior frontal area (BA 11/47), and left medial frontal gyrus (BA 8). A direct comparison was also made between words with emotional connotations and animal names. Although some animal names may also have had emotional connotations, significant activity was associated with the words with emotional connotations compared to the animal names, primarily in midline and left medial frontal gyrus (BA 8, 10), and midline orbital frontal gyrus (BA 11). These authors concluded that processing of words with emotional connotations leads to further processing in the limbic brain structures of the frontal lobes that does not occur for words without emotional connotations. Crosson et al. (1999a) used fmri to investigate left-hemisphere processing during generation of words with emotional connotations relative to generation of emotionally

63 53 neutral words. In accordance with the Beauregard et al. (1997) study, Crosson et al. found activity near the frontal pole (an area richly connected to the limbic system) when contrasted to generation of emotionally neutral words. This comparison also yielded unique activation near the temporal pole during generation of words with emotional connotations. Canli et al. (1998) similarly found activation of cortex near the temporal pole (in both hemispheres) during viewing of pictures with positive valence relative to negative valence. In comparison between generating words with emotional connotations to repeating emotionally neutral words, Crosson et al. found that limbic structures were activated including the hypothalamus, amygdala and hippocampus. Crosson et al. did not compare differences in brain activation between generation of words with positive versus negative connotations. In addition, Crosson et al. did not report activity that was found in the superior edge or the thalamus and around retrosplenial cortex as the activity was in large veins for more than a third of the subjects. In another recent study (Cato et al., 2000), participants monitored words with emotional connotations for semantic characteristics. This study confirmed activity increases near the frontal pole during the processing of words with emotional connotations relative to neutral words. The current study, outlined in the chapters that follow, represents an extension of the literature examining emotional connotation as a semantic attribute. While several studies mentioned in this chapter have examined the role of emotional connotation using neuroimaging techniques, the current study is designed to uniquely contribute to the current literature in several ways. This study, using a word generation paradigm, examines without precedence, differences in hemispheric processing of categories with emotional connotations (positive and negative) versus emotionally neutral categories

64 54 using whole brain fmri. This imaging study is also designed to determine whether emotional valence (positive, negative or neutral) of categories influences the pattern of activation in either or both hemispheres when rated arousal of the categories are equated across positive and negative valences. Finally, this study explores whether sex of the participant influences the pattern of activation during word generation to categories with positive, negative or neutral emotional connotations. The hypotheses for the study will be presented in the next chapter, followed by a brief explanation of the pilot studies that were conducted to establish optimal methodologies for the current study.

65 HYPOTHESES AND PILOT STUDIES Hypothesis 1 All word generation tasks, with and without emotional connotation, will result in some common areas of activation relative to a baseline of repetition, including Broca s area, pre-supplementary motor area, and the inferior frontal sulcus. These common language areas will be lateralized to the left hemisphere for word generation to categories with and without emotional connotation. Tachistoscopic evidence points to a left-hemisphere dominance for processing words, even those with emotional connotations (Ali & Cimino, 1997). Evidence from patients with right hemisphere damage reveals that the left-hemisphere can mediate processing of emotional verbal content so long as emotional facial expressions or prosody are not described (Blonder et al., 1991). Crosson et al. (1999a), using fmri, examined activation within the left hemisphere during word generation to categories with and without emotional connotation. Common language areas related to generation of words (both neutral and with emotional connotation) relative to a baseline of neutral word repetition included: Broca s area, involved in language production; pre-supplementary motor area (pre-sma) which is presumably involved in aspects of initiating activities, such as monitoring for conflict between competing responses (Barch, Braver, Sabb, & Noll, 2000; Carter et al., 2000); and areas along the inferior frontal sulcus that are probably related to semantic 55

66 56 processing. These areas of activation were present during generation to categories with both emotional and neutral connotations, compared to a baseline of repetition. Crosson et al. (1999a) also found that both generation tasks (neutral and emotional) led to activation in striate and peristriate areas, relative to a baseline of neutral word repetition. In the current study, activation in these common areas found by Crosson et al. are expected to be lateralized to the left hemisphere, as they are presumably related to the generation and semantic processing of verbal stimuli. Hypothesis 2 Generation of words to categories either with positive or negative emotional connotations will activate bilaterally cortex near the frontal and temporal poles as well as limbic areas, including the amygdala, the hippocampus and the hypothalamus. Crosson et al. (1999a) reported unique neural substrates in the left hemisphere related to generation of emotionally connotative words. Their findings varied somewhat based on the comparison task used. Relative to a baseline of neutral word repetition, Crosson et al. found that during word generation to categories with emotional connotation, the following limbic structures were activated: the hypothalamus, amygdala and hippocampus. In addition, two cortical areas heavily connected to limbic regions were activated: cortex near the frontal pole, and inferior medial temporal cortex. Activation in these limbic and cortical limbic association areas was not found when generation of neutral categories was compared to repeating emotionally neutral words. With a direct comparison between generation of words with emotional connotations and generation of emotionally neutral words, Crosson et al. (1999a) found that generation of words with emotional connotation uniquely led to engagement of

67 57 cortex near the frontal and temporal poles, areas richly connected to the limbic system. While Crosson et al. confined their investigation to the left hemisphere, it is an assumption that homologous areas in the contralateral hemisphere will be engaged as well. Bilaterally, these areas are hypothesized to be involved in the processing of emotional connotations, either positive or negative. Hypothesis 3 In limbic and cortical limbic association areas, the relative contribution of the left and right hemispheres will be influenced by the valence (positive or negative) of the categories with emotional connotation. Thus, in those areas uniquely activated by categories with emotional connotation, (frontal and temporal poles; amygdala, hippocampus, and hypothalamus), right hemisphere activation will be greater than left hemisphere activation during generation to negatively valenced categories. For categories with positive emotional connotations, the opposite pattern will occur, in which these structures in the left hemisphere will show greater activation relative to the right hemisphere. Crosson et al. did not investigate the effect of valence on pattern of activity during word generation. However, other functional imaging studies have examined the impact of valence during semantic processing. Canli et al. (1998), using fmri whole brain acquisition, found support for the valence hypothesis, when arousal was equated across positive and negative pictures. As the valence hypothesis would predict, greater activation occurred in the left hemisphere during processing of positive pictures and greater activation was present in the right hemisphere during processing of negative pictures. Unlike the current study, Canli et al. (1998) considered all significantly

68 58 activated structures when determining hemispheric asymmetry, rather than limiting the asymmetry analysis to specific areas. On the other hand, Lane et al. (1997b), using PET found primarily left-lateralized activation for viewing of both emotionally positive and negative pictures, with no trend towards lateralization by valence. These authors did not attempt to equate for arousal and the pictures with the emotionally negative connotations were rated as significantly more arousing than the pictures with emotionally positive connotations (possible confound). A recent whole brain fmri study attempting to induce emotional experience, either positive or negative, with both pictures and captions, also did not find support for the valence hypothesis (Teasdale et al., 1999). These authors found similar areas of activation in the medial frontal gyrus (Brodmann s area 9) and right anterior cingulate gyrus (areas 24 and 32) associated with viewing both negative and positive picture-caption pairs. The ambiguities in the previous research merit further investigations employing imaging techniques. Hypothesis 4 The pattern of activation may differ by sex of participants. Lang and his colleagues (1998) examined brain activation in posterior cortex using fmri and found that the extent of brain activation in the sampled brain region was greater in the right hemisphere when processing pictures with emotional connotations versus neutral pictures. Furthermore, within the more active right hemisphere, women showed significantly more functional activity for unpleasant than pleasant stimuli, while men showed the opposite pattern, with more activation for pictures with positive connotations (Lang et al., 1998). Based on findings of Ali and Cimino (1997), it is hypothesized that

69 59 words will have a similar effect, such that men will show more functional activity related to word generation to positive categories, and women will show more functional activity to negative categories. Pilot studies were performed using a 3.0 Tesla (T) and a 1.5 T instrument to determine optimal parameters for this experiment. For the sake of brevity, only two phases of the pilot performed at the 1.5T scanner will be described. An initial question was whether a statistical technique, deconvolution, could be used for reliable imaging of word generation. The deconvolution procedure uses the data itself, rather than a prespecified waveform, to determine the form of the functional response in each voxel for each condition. Deconvolution provides a best linear least-squares fit between the acquired time series and the estimated time series that includes the following model parameters: constant baseline, linear trend in the time series, and the BOLD (Blood Oxygen Level Dependent) response to each condition. In contrast, the cross-correlation technique of data analysis previously used in our laboratory is one with greater limitations. Cross-correlation analyses require a fixed ideal reference waveform (e.g., square or sinusoidal) to determine the relationship between the acquired time series and that of the task manipulations; in the case of deconvolution, the data determines the functional form of the estimated response. Therefore, no assumptions need be made regarding the shape of the ideal signal waveform. In addition, deconvolution has the capability to fit a number of stimulus waveforms. Therefore, a number of manipulations can occur within a single imaging run. Finally, deconvolution has the flexibility to model any variability in the form of the hemodynamic response from voxel to voxel.

70 60 The brain region of interest for the pilot phase was left medial frontal cortex (Brodmann s Area 6/32). This area has been reliably activated during previous word generation experiments using cross-correlation analyses (Crosson et al., 1998; Crosson et al., 1999a; Crosson et al., 1999b) and served as a benchmark for comparison. A second question was whether a pseudorandomized event-related paradigm (PER) versus a pseudorandomized block (PRB) presentation would allow for maximal sensitivity in medial frontal cortex. For all phases of fmri pilot studies, the same verbal stimuli were used. Positive, negative and neutral categories were derived from the Affective Norms for Emotional Words (ANEW) (Bradley, Cuthbert, & Lang, 1988). This corpus provides words with ratings on a valence dimension from negative to positive (range, 1.00 to 9.00) and an arousal dimension from low to high arousal (range, 1.00 to 9.00). A brief behavioral pilot study was conducted prior to the fmri pilot studies to roughly equate rate of word generation for the positive, negative and neutral categories. Nine pilot participants (5 females, 4 males) generated words to the categories and rated the words generated for valence and arousal, with the same rating scales as those used for the ANEW corpus. Efforts were made to equate generation rate across valence type, and to equate positive and negative categories on arousal level. Following the fmri pilot studies, a more thorough behavioral pilot study was conducted to select the categories for the final fmri experiment. Results of this behavioral pilot will be offered at the end of this chapter. For both phases of the pilot study presented here, potential participants were excluded if they had a history of neurological disease, major psychiatric disturbance, learning disability, attention deficit disorder, or substance abuse, or if they were taking

71 61 psychoactive medications. Informed consent was obtained from participants according to institutional guidelines established by the Health Center Institutional Review Board at the University of Florida. Pilot I: Participants. Four male volunteers ranging in age from 18 to 27 (M = years, SD = 4.19 years) and ranging in education from 13 to 18 years (M = 16 years, SD = 2.45 years) participated in the first fmri pilot study. All participants were strongly right handed as measured by the Edinburgh Handedness Inventory (Oldfield, 1971; M = 58; SD = 7.12). Experimental tasks. Participants performed three pseudorandomized block (PRB) runs. The PRB runs consisted of an active period of fixed duration at 14 seconds, alternating with pseudorandomized rest durations of 8, 14 and 20 seconds, with a total of 9.5 cycles beginning and ending with rest. (See Figure 3 for a schematic of a PRB run). During rest periods, participants remained silent. During active periods of the PRB runs, participants heard a positive, negative or neutral category, followed by the cue, begin. They responded by silently generating as many exemplars of the category as possible before the cue end, occurring approximately 14 seconds after the onset of the trial. The participants also engaged in four runs of a pseudorandomized event related paradigm (PER) in which participants silently generated a single exemplar for each category aurally presented (See Figure 4 for a schematic of an PER run). The rest period began immediately after the subject silently generated the single item. As with the PRB design, category cues to generate words with positive, negative, or neutral connotations

72 Predators Desserts Shapes Diseases Positive Connotations Negative Connotations Neutral Connotations Figure 3. Sample portion of pseudorandomized block run alternating with rest. gifts chores tools toys Positive Connotations Negative Connotations Neutral Connotations Figure 4. Sample portion of pseudorandomized event-related run alternating with rest. were presented in a pseudorandomized order. For the PER runs, rest intervals were either 4, 6 or 10 seconds. Image acquisition. Imaging was performed with a 1.5 T GE Signa scanner using a 2-

73 63 spiral gradient echo sequence. Medial frontal cortex was imaged bilaterally using sagittal slices 6.8 mm thick. Slices were centered at the interhemispheric fissure for equal coverage of both hemispheres. For the PRB, 8 contiguous slices were obtained (TR = 1000 ms, FA = 60 deg., 133 images per run, temporal resolution = 2000 ms). For the PER, 6 contiguous slices were obtained (TR=500 ms, FA = 45 deg., 168 images per run, temporal resolution = 1000 ms). Image analysis. Data analyses were performed with Analysis of Functional Neuroimaging (AFNI; Cox, 1996) software. Signal intensities in the 399 serial PRB images obtained after concatenating three runs of 133 serial images were deconvolved with the stimulus time series to produce a hemodynamic response function (HRF) on a voxel-wise basis for each condition (positive, negative, and neutral). The coefficient of determination, full R 2, was evaluated in the ROI (medial frontal cortex) to quantify the activation associated with the three category types (positive, negative and neutral) combined. A similar analysis was performed on the voxels in the 480 serial PER images (obtained after concatenating four runs of 120 serial images). Results. Substantial activation in medial frontal cortex (Brodmann s areas 6 and/or 32) was found for the first phase of the pilot study. However, the robustness (magnitude of full R 2 ) of the estimated response was larger and more consistent for the blocked case. The maximal full R 2 in medial frontal cortex for the PRB paradigm ranged from.32 to.45. The maximal full R 2 in medial frontal cortex for the PER paradigm was consistently lower, ranging from.20 to.29. Conclusions. The results of Pilot I demonstrated that deconvolution could be used to reliably image medial frontal cortex. In addition, the results for the blocked paradigm

74 64 were consistently more robust than those for the single event paradigm. Therefore, a blocked paradigm was selected for Pilot II. An additional question was whether whole brain acquisition (thus, reduction in temporal resolution) would also lead to similar robust results in medial frontal cortex. In Pilot II, presented below, a blocked paradigm was tested using whole brain acquisition and deconvolution. Pilot II Participants. Ten volunteers (4 male, 6 female) ranging in age from 22 to 47 (M = years, SD = 7.03 years) and ranging in education from 15 to 22 years (M = years, SD = 2.36 years) participated in the second fmri pilot study. All participants were strongly right handed as measured by the Edinburgh Handedness Inventory (Oldfield, 1971; M = 80; SD = 14.45). Experimental tasks. Participants performed four, rather than three pseudorandomized block (PRB) runs and did not perform any PER runs. In this last and final phase, the PRB runs consisted of a slightly longer active period of fixed duration at 16.5 seconds, alternating with pseudorandomized rest durations of 9.9, 16.5 and 23.1 seconds, with a total of 9.5 cycles beginning and ending with rest. During rest periods, participants remained silent. During active periods, participants heard a positive, negative or neutral category, followed by the cue, begin. They responded by silently generating as many exemplars of the category as possible before the cue end, occurring approximately 16.5 seconds after the onset of the trial. Image acquisition. As with the previous fmri pilot, imaging was performed with a 1.5T GE Signa scanner using a 2-spiral gradient echo sequence. Unlike Pilot I, whole brain coverage was used during this phase (22 slices) with thick ( mm) sagittal

75 65 slices (TR = 1650 ms, FA = 65 deg., 95 images per run, temporal resolution = 3300 ms). Slices were centered at the interhemispheric fissure for equal coverage of both hemispheres. Image analysis. Data analyses were performed using the same methodology as given for Pilot I. Data analyses were performed with Analysis of Functional Neuroimaging (AFNI; Cox, 1996) software. Signal intensities in the 380 serial PRB images obtained after concatenating four runs of 95 serial images were deconvolved with the stimulus time series to produce a separate hemodynamic response function (HRF) on a voxel-wise basis for each condition (positive, negative, and neutral). The coefficient of determination, full R 2, was evaluated in the ROI (medial frontal cortex) to quantify the activation associated with the three category types (positive, negative and neutral) combined. Results. Activation in medial frontal cortex (Brodmann s areas 6 and/or 32) was consistently found for the second phase of the pilot study. In this case of whole brain coverage, with four, rather than three, blocked runs, the robustness of the estimated response was superior to the previous pilot. The maximal full R 2 in medial frontal cortex for the PRB paradigm ranged from.31 to.82. Conclusions An initial question was whether a new statistical technique, deconvolution, could be used for reliable imaging of word generation. The results of the pilot studies provided a definitive affirmation of the use of this technique. All iterations of the pilot study showed activation in medial frontal cortex (See Table 3 for summary of fmri pilot results). A second question was whether the employment of an event-related paradigm

76 66 (PER) versus a pseudorandomized block (PRB) presentation would allow for maximal sensitivity. Though activity in medial frontal cortex was found using PER and PRB paradigms, Pilot I revealed that results were more robust with the PRB format. Because of the advantages of deconvolution over cross-correlation techniques, this analysis was selected for the experiment, and the scanning parameters of the final phase of the pilot experiment were chosen due to the superior results in medial frontal cortex in the second and final phase of the fmri pilot. Table 3: Summary of fmri pilot results. Full R 2 PRB Full R 2 PER Pilot I: Thick Slices N = 4 Pilot II: Whole Brain N = 10 Behavioral Pilot Study Emotional and neutral categories for the final fmri experiment were derived from the Affective Norms for Emotional Words (ANEW) (Bradley et al., 1988) corpus. ANEW was also used to select emotionally neutral words for repetition. A pilot study with 12 subjects (6 males, 6 females; age years, mean 28.5 years; education years, mean 19.0 years) was conducted to equate number of words generated in a 16.5 second time period across valence of categories and to equate arousal ratings across positive and negative categories. As a result of the pilot, 45 of 80 piloted categories were

77 67 selected. All selected categories are listed in Table 4. Categories were given in their most abbreviated form, and all participants were instructed that for any category (e.g., funeral) they were to generate words associated with, or words that belong to, the category given. Table 4: Positive, negative and neutral categories selected for the final fmri study. Positive Categories Negative Categories Neutral Categories love execution buildings tropical island funeral parts of speech sexy characteristics violent crimes chemical elements attractive qualities weapons kitchen utensils scenic landscapes natural disasters human dwellings vacations death tools relaxation disasters types of ships signs of affection prejudice reading materials loved ones diseases roads celebrations insults sources of light amusement park jail farm animals romance torture parts of buildings comfort items slums types of ships dating illnesses containers success ways to die metals Three separate repeated measures analyses of variance (ANOVA) were performed examining generation rates, valence ratings and arousal ratings given by the 12

78 68 behavioral pilot participants on the final 45 categories, with sex of participants as the between subjects factor and assumed valence of categories as the within subjects factor. Table 5 lists the outcome of this pilot, collapsed across sex of participants. The sex of participants did not significantly influence the number of words generated per category [F(1,10) = 3.39, p =.1], the valence ratings for each category type [F(1,10) =.16, p =.70] or the arousal ratings for each category type [F(1,10)= 1.65, p =.23]. Collapsed across sex of participants, number of words generated in a 16.5 s period did not significantly vary by valence type [F(2, 20) = 1.61, p =.23]. However, collapsed across sex of participants, valence ratings did significantly differ between categories Table 5: Pilot results collapsed across sex of participants (N = 12). Positive Negative Neutral M SD M SD M SD Words Generated Valence Rating Arousal Rating assumed to be positive, negative, and neutral in the expected directions (Table 5), with ratings for assumed positive categories the highest and ratings for assumed negative categories the lowest [F(2, 20) = , p <.001]. Arousal ratings differed significantly between categories [F(2, 20) = 41.47, p <.001]. Post hoc comparisons revealed significantly higher ratings for positive and negative

79 69 categories than for neutral categories and no significant difference between arousal ratings of positive and negative categories, as expected. The interaction effect between sex of participants and generation rate was not significant [F(2, 20) = 1.07, p =.36], nor was the interaction effect between sex and valence ratings [F(2, 20) = 1.57, p =.23]. However, the interaction effect between sex and arousal ratings was significant [F(2, 20) = 6.11, p <.01], with females arousal ratings of negative categories as significantly higher than males arousal ratings of negative categories (Figure 5) MALE FEMALE Average Arousal Rating POS NEG NEU Category Type Figure 5. Arousal ratings for each category type by sex of participant.

80 70 Thus, for the 45 categories selected, generation rates were balanced across conditions and the arousal ratings were balanced across the two conditions with emotional connotation, positive and negative. While there was a significant interaction between sex of participants and arousal ratings, this pattern has been cited in previous literature (Coney & Fitzgerald, 2000; Lang et al., 1998) and may reflect sex differences in appraisal of stimuli with positive versus negative emotional connotation. Thus, this significant interaction did not preclude continuation with the selected categories, particularly as arousal ratings, overall, were equated across positive and negative categories. The methods outlined in the next chapter are based on the outcomes of the aforementioned pilot studies. Task parameters and analyses of each individual s data described in the next chapter closely follow those of the third and most successful phase of the imaging pilot study. One important change from the final pilot to the current paradigm is the use of repetition, rather than rest, as the baseline task. In pilot studies for the Crosson et al. (1999a) study, the investigators found that maximal discrimination in group comparisons between generation of words with neutral versus emotional connotations was attained with a baseline of repetition rather than rest. The use of rest in the preliminary pilot studies was intentional; at the individual level, past findings have been most robust when active tasks were compared to rest.

81 METHODS Presented below are the methods used for the final fmri experiment. The parameters of the final experiment represent a culmination of the findings of the pilot studies presented in Chapter 5. A change from the final pilot study is that the baseline of repetition, rather than rest was used. The results of the final fmri experiment are presented in Chapter 7. Participants Twenty-six (13 males, 13 females) strongly right-handed [Edinburgh Handedness Inventory (Oldfield, 1971), M = 81.88, SD = 14.34] volunteers participated in the study (age years, M = years, SD = 9.86; education years, M = years, SD = 1.74 years). Male and female participants did not significantly differ in terms of years of education [t(24) = -.56, p =.58] or age [t(24) = -.95, p = 35]. Potential participants were excluded if they reported a history of neurological disease, major psychiatric disturbance, learning disability, attention deficit disorder, or substance abuse, of if they reported current use of psychoactive medications. Potential risks were explained, and informed consent was obtained from participants according to institutional guidelines established by the Health Center Institutional Review Board at the University of Florida. 71

82 72 Experimental Tasks and Manipulation Check During each of four functional imaging runs, blocks of silently generating words to categories with positive, negative or neutral emotional connotation alternated with silent repetition of neutral words. During category generation, a category was given at the beginning of a 16.5 second word generation period, and participants silently generated as many exemplars as possible. Categories with either positive (e.g., scenic landscapes), negative (e.g., natural disasters), or neutral (e.g., containers) emotional connotations were presented in a pseudorandom order, such that each functional run consisted of three categories from each valence level (positive, negative and neutral). During alternating baseline periods of variable length (either 9.9, 16.5 or 22.1 seconds, distributed within runs in a pseudorandomized order), participants silently repeated neutral words, with 4 words to repeat during the 9.9 s baseline; 7 words to repeat during the 16.5 s baseline; and 10 words to repeat during the 22.1 s baseline (Figure 6). jail roads love Word Generation: Negative Connotation Word Generation: Positive Connotation Word Generation: Neutral Connotation Word Repetition: Neutral Connotation Figure 6. Sample portion of pseudorandomized block run alternating with repetition.

83 73 The onset of generation blocks was marked by the cue, generate, while repetition blocks commenced with the cue, repeat. All words, cues, and categories were delivered binaurally via an air conduction tube connected to hollow foam earplugs (Kenwood KR- A4070 amplifier, Realistic Tenband stereo frequency equalizer, JBL 16 Ù speaker, and insulated air conduction transducer with foam insert earphones). Volume was adjusted by adding a constant 35 db to each subject's auditory threshold. Auditory threshold was determined by engaging each subject in a task in which they responded to two target words, happy and calm with a button press. The volume of the words was gradually reduced until the participant no longer responded to the target stimuli. The point at which participants discontinued responding to the targets was considered their auditory threshold. Four of five possible runs were administered to each subject, with the order of these runs randomized across participants. Following scanning, the fifth run was conducted outside of the scanner to provide information about the generation rate for categories of positive, negative, and neutral valence. Next, participants rated all 45 categories for valence and arousal. Image Acquisition Each experimental run had nine 16.5 second periods of word generation. The baseline state was repetition of emotionally neutral words for 9.9, 16.5, or 22.1 seconds. Baseline periods were varied in a pseudorandom fashion to mitigate low frequency artifacts that can occur with fmri (Zarahn, Aguirre, & D Esposito, 1999). Each length of baseline was used an equal number of times, except that an extra 16.5 second baseline period was added to the end of each run. Thus, for each functional imaging run, there was

84 74 an average of 10 images per word generation-baseline cycle, with 95 images total per functional run. Whole brain imaging was performed on a 1.5 Tesla GE Signa scanner using a dome-shaped quadrature radio frequency head coil. The head was aligned such that the interhemispheric fissure was within 1º of vertical. For fmri sequences, 22 slices ( mm thick) were acquired, centered at the interhemispheric fissure for equal coverage of both hemispheres. Images were obtained using a 2-spiral gradient echo scan with the following parameters: TE=40 ms, TR=1650 ms, FA=65º, FOV (field of view)=180 mm, matrix size = 256 x 128. After functional image acquisition, structural images were acquired for mm thick sagittal slices, using a 3D spoiled GRASS volume acquisition (TE=7 ms, TR=27 ms, FA=45º, NEX = 1, FOV = 24 cm, matrix size = 256 x 192). Image Analysis Functional images were analyzed and overlaid onto anatomic images with Analysis of Functional Neuroimages (AFNI) software (Cox, 1996). To minimize effects of head motion, time series images were spatially registered in 3-dimensional space. For each subject, mean slice signal intensities were normalized to the grand mean of slice intensity across functional runs. Voxels where the standard deviation of the signal change exceeded 5 percent of the mean signal were set to zero to attenuate large vessel effects and residual motion artifact. Images were visually inspected for gross artifacts and were viewed in ciné loop to detect residual motion. If any time series of a subject was judged to contain a significant number of images with gross artifacts or residual motion, the subject s data was removed from further analyses. Data from three of 29 subjects who

85 75 participated were eliminated because of motion artifact leaving the 26 subjects who were described above. Signal intensities in the 380 serial pseudorandomized block (PRB) images obtained after concatenating four runs of 95 serial images were deconvolved with each of the three stimulus time series (positive, negative and neutral) to produce corresponding hemodynamic response functions (HRFs) of the fmri signal on a voxel-wise basis. For purposes of comparison, magnitude of response to each stimulus type was operationally defined as area under the curve of each HRF, positive, negative and neutral, on a voxelwise basis, averaged across time. Anatomic and functional images were interpolated to volumes with 1mm 3 voxels, co-registered, and converted to stereotaxic coordinate space of Talairach and Tournoux (1988) using AFNI. Functional image volumes were spatially smoothed using a 3 mm Gaussian full-width half-maximum (FWHM) filter to compensate for variability in structural and functional anatomy across participants. Student's t-tests were performed on a voxel-by-voxel basis to compare each word generation type to the baseline of repetition. For these comparisons to word repetition, a threshold volume of 200 ìl and a statistical threshold of p <.001 were adopted. Direct comparisons were also performed using Analyses of Variance (ANOVAs) in which the three category generation types (positive, negative and neutral) were compared directly with one another. A voxel-wise two-factor ANOVA, with a fixed condition factor (valence: positive versus negative versus neutral) and a random factor (participants) was performed to determine the effect of valence on the pattern of brain activation during word generation across participants. A three-factor ANOVA was also performed, with

86 76 two fixed conditions, valence (within subjects factor) and sex (between subjects factor) and one random participant condition, to determine the effect of sex and the interaction between sex and valence on brain activity associated with word generation. Posthoc pooled variance t-tests were performed following each ANOVA evaluating differences in HRFs (area under the curve) between the conditions. Cluster analyses on resulting statistical parametric maps were conducted. For all ANOVAs and their subsequent posthoc t-tests, a volume threshold of 200 ìl and a statistical threshold of p <.005 were used.

87 RESULTS Behavioral Results Three separate repeated measures analyses of variance (ANOVA), with sex of participants as the between subjects factor and assumed valence of categories as the within subjects factor, were performed examining generation rates, and arousal and valence ratings as dependent variables, performed by the 26 fmri participants that were included in the fmri analyses. As with the pilot, the goal was that number of words generated in a 16.5 second time period would be equivalent across valence of categories and that arousal ratings across positive and negative categories would be equivalent. The effect of sex on generation rates, valence and arousal ratings was also examined. The sex of participants did not significantly influence the number of words generated per category [F(1, 24) =.33, p =.57], the valence ratings for each category type [F(1, 24) = 3.02, p =.1] or the arousal ratings for each category type [F(1, 24)= 2.88, p =.1]. Collapsed across sex of participants, number of words generated in a 16.5 s period did not significantly vary by valence type [F(2, 48) = 1.05, p =.36]. As expected, collapsed across sex of participants, valence ratings did significantly differ between categories assumed to be positive, negative, and neutral [F(2, 48) = , p <.001] in the assumed directions, with ratings for assumed positive categories the highest and ratings for assumed negative categories the lowest (Table 6). Also as 77

88 78 expected, arousal ratings differed significantly between categories [F(2, 48) = 85.22, p <.001]. Post hoc comparisons revealed significantly higher ratings for positive and negative categories than for neutral categories and no significant difference between arousal ratings of positive and negative categories. The interaction effect between sex of participants and generation rate approached significance [F(2, 48) = 2.70, p =.08]. The direction of the interaction (non-significant) was that men generated less words to positive categories (M = 5.90; SD =.37) than women (M = 6.97; SD =.37).The interaction effect between sex of participants and valence ratings was also not significant [F(2, 20) =.01, p =.99]. Unlike the findings from the pilot, the interaction effect between sex and arousal ratings was additionally not significant [F(2, 48) =.02, p =.99]. This finding indicates that in the final sample of fmri participants, the sex of the participant did not significantly alter arousal ratings. Therefore, with this sample, any sex differences in functional activity cannot be attributed to differences in arousal ratings between categories with positive versus negative emotional connotations. Table 6: Behavioral results collapsed across sex of participants (N = 26). Positive Negative Neutral M SD M SD M SD Words Generated Valence Rating Arousal Rating

89 79 FMRI Results The fmri results from the comparisons of each word generation type to the baseline of repetition will be considered first. These comparisons were performed with student's t-tests. Next, the comparisons of each of the word generation tasks (positive, negative and neutral) directly to one another, will be presented. These comparisons were performed with repeated measures ANOVAs and follow-up t tests. Positive, Negative, and Neutral Category Generation Versus Neutral Word Repetition The first hypothesis for the fmri results (Hypothesis 1, Chapter 5) was that relative to a baseline of repetition, activation of certain common language areas lateralized to the left hemisphere were expected to be found for all three word generation tasks. The expected language areas included: Broca s area, pre-supplementary motor area, and the inferior frontal sulcus. These areas were found in Crosson et al. (1999a) with the same task comparison. In addition, Crosson et al. (1999a) found that each generation task (neutral and emotional) led to activation in areas of striate and peristriate cortex. At a threshold volume of 200 ìl and a statistical threshold of p <.001 per voxel, each generation task, when compared to repetition of emotionally neutral words (Table 7), did lead to some common areas of brain activation, lateralized to the left hemisphere: left medial frontal activity, including left pre-supplementary motor area [pre-sma, anterior Brodmann s Area (BA) 6]; and some areas of activity along the ventral visual stream, including activity in the lingual gyrus and other primary and secondary visual areas. At this statistical threshold, however, activity in Broca s Area was only found in the case of word generation to categories with negative emotional connotations. In

90 80 addition, only the neutral word generation task led to activity in the left inferior frontal sulcus. Table 7: Volumes of tissue (> 200 µl) showing significant activity changes (p <.001) for word generation tasks versus neutral word repetition. Neutral Positive Negative Connotations Connotations Connotations Location Anatomic Area (max t loc) max t, volume in µl Anatomic Area (max t loc) max t, volume in µl Anatomic Area (max t loc) max t, volume in µl Hypothesized Common Areas Broca s Area L BAs 44/45 (-54, 18, 13) t = 4.91, 237 µl Pre-SMA/ Other Medial Frontal Inferior Frontal Sulcus Lingual Gyrus/ Other Visual Areas L and R BA 6 (-12, 6, 60) t = 8.03, 2595 µl L BA 9 (-50, 25, 28) t = 5.47, 391 ìl L Bas 19, 37 (-13, -47, 1) t = 7.04; 1702 ìl R BA 17 (7, ) t = 5.85; 404 ìl Other Limbic or Limbic Association Areas Insula Retrosplenial Cortex Thalamus: Dorsomedial Nucleus (DM)/Pulvinar L BA 6, 8 (-10, 26, 56) t = 7.10, 2925 µl L BA 18 (-11, -63, 6) t = 6.50, 413 ìl R Insula (39, 12, 0) t = 6.92, 615 ìl L and R BA 30 (-19, -47, 5) t = 5.95, 641 ìl L DM, Pulvinar (-6, -22, 11) t = 6.46, vol = 1228 L BA 6 (-3, 12, 48) t = 10.07, 3556 µl L Bas 18/19 (-5, -72, -7) t = 5.14, 211 ìl L DM (-7, -8, 17) t = 6.74, 1032 ìl

91 81 Table 7 Continued Location Anatomic Area (max t loc) max t, volume in µl Other Frontal Cortex Neutral Positive Negative Connotations Connotations Connotations Anatomic Area (max t loc) max t, volume in µl Anatomic Area (max t loc) max t, volume in µl Other Premotor Cortex Other Frontal Cortex Inferior Frontal Gyrus Temporal Cortex L BA 6 (-28, 13, 57) t = 6.33, 750 ìl L BA 6 (-28, -1, 61) t = 4.99, 315 ìl L BA 6 (-47, 5, 34) t = 5.69, 788 ìl R BAs 45/47 (29, 30, 1) t = 6.53, 778 ìl L BA 6 (-34, -1, 52) t = 6.42, 594 ìl R BA 32, 8, 9 (25, 33, 29) t = 5.21, 216 ìl Inferior Temporal Gyrus L BA 37 (-49, -48, -12) t = 5.08, 203 ìl Note. BA = Brodmann's Area (according to Talairach & Tournoux, 1988); max t = maximum t value within given cluster of activity. Other abbreviations are as follows: L = Left, R = Right, SMA = Supplementary Motor Area. Based on the apriori hypothesis (Hypothesis 1, Chapter 5), comparisons of activity in Broca s area and in the inferior frontal sulcus were conducted with a slightly less stringent significance threshold of p <.005 per voxel and the same volume threshold of 200 ìl. When each generation task was compared to neutral word repetition at this significance level of p <.005, functional activity in Broca s area was additionally found for the other two word generation tasks (Positive Connotations, Maximum Intensity (MI) xyz = -54, 17, 10; Neutral Connotations, MI xyz = -51, 31, -4). In addition at a

92 82 significance level of p <.005, functional activity in the left inferior frontal sulcus was additionally found for negative connotations (MI xyz = -43, 20, 31), but not positive connotations. The second hypothesis (Hypothesis 2, Chapter 5) stated that generation of words with either positive or negative emotional connotations would activate cortex near the frontal and temporal poles bilaterally, as well as limbic areas, including the amygdala, the hippocampus and the hypothalamus. Crosson et al. (1999a) found that relative to neutral word repetition, word generation to categories with emotional connotations led to regions of activity near the left frontal pole, inferior medial temporal cortex, the amygdala, hippocampus, and the hypothalamus. Crosson et al. (1999a) found clusters of activity both near the left frontal and temporal poles only with direct task comparisons between generation tasks. As can bee seen in Table 7, when each generation task was compared to neutral word repetition, no activation was found in cortex near the frontal or temporal poles with the significance level of p <.001. Based on the apriori hypothesis (Hypothesis 2, Chapter 5), planned comparisons of activity near the frontal and temporal poles were conducted with a slightly less stringent significance threshold of p <.005 (rather than p <.001) and the same volume threshold of 200 ìl. When each generation task was compared to neutral word repetition at this significance level of p <.005, activity near the left frontal pole was seen exclusively in the case of generating words with positive and negative emotional connotation and only in the left hemisphere (Figure 7; MI for positive categories, xyz = -3, 58, 25; MI for negative categories, xyz = -3, 58, 28). The frontal pole areas found in this study, relative to a baseline of neutral word repetition, closely match the area near the frontal pole found in Crosson et al., 1999a, when generation of

93 83 categories with emotional connotations was compared to neutral word repetition (MI for emotional categories, xyz = -7, 60, 28). X = -3 X = -3 Positive Categories Negative Categories X = -3 Neutral Categories Figure 7. Cortex near the left frontal pole activated by negative, positive, and neutral categories relative to neutral word repetition, red = p <.005; yellow = p <.001. Activity near the right temporal pole (Figure 8; MI xyz = 48, -1, -32) was also found when generation of words with negative emotional connotations was compared to a baseline of repetition. Unlike the current finding, Crosson et al. (1999a) only found temporal pole activity when emotional word generation was compared directly to neutral word generation. The results of direct task comparisons will be presented shortly.

94 84 X = -3 X = -3 Negative Generation versus Neutral Repetition Positive Generation versus Neutral Repetition X = -3 Neutral Generation versus Neutral Repetition Figure 8. Right temporal pole activated only by negative emotional categories relative to neutral word repetition, red = p <.005; yellow = p <.001. In addition to the apriori examination of cortex near the frontal and temporal poles, some additional limbic areas of activity were found when comparing generation with either positive or negative connotations to neutral word repetition (Table 7). Activity in the left dorsomedial nucleus of the thalamus, implicated in emotional processing (Alexander, DeLong & Strick, 1986; Yakovlev, 1948), was present for both positive and negative, but not neutral category generation. In addition, activity in another limbic area,

95 85 the right insula, was found for generation to categories with positive but not negative or neutral connotations. Finally for categories with positive emotional connotations, activity was also found in left retrosplenial cortex, an area hypothesized by Maddock (1999) to be an important area for processing emotionally salient stimuli, including words. Despite these findings of activation in limbic association areas, even at a less stringent significance threshold of p <.005, activation was not found in hypothesized limbic areas (i.e., amygdala, hippocampus and hypothalamus) for either positive or negative connotations relative to a baseline of word repetition. The third hypothesis (Hypothesis 3, Chapter 5) involved predictions about the influence of valence of categories on the relative contribution of the left and right hemispheres. Hypothesized asymmetric activation was expected to be found in limbic association cortices, including cortices near the frontal and temporal poles, and other hypothesized primary limbic areas (amygdala, hippocampus, and hypothalamus). Partial support for Hypothesis 3 was found among this first set of comparisons in which each generation task was compared to a baseline of repetition. While the current finding of activity near the left frontal pole for categories with both positive and negative connotations is not consistent with the valence hypothesis, the finding that only word generation to categories with negative emotional connotations led to right temporal pole activity is partially supportive of the prediction of greater right hemisphere activation for stimuli with negative connotations. On the other hand, the finding that the right insula was activated only for generation to categories with positive connotations is contrary to the prediction of a relative right hemisphere advantage for processing of negatively valenced information.

96 86 The final hypothesis (Hypothesis 4, Chapter 5) regarding differences by sex of participants was not explored with the indirect comparisons of each word generation task versus neutral word repetition. In the next section, direct comparisons among the word generation tasks are presented. Presented below are the results of an ANOVA that examines the effect of valence among the three generation tasks (positive versus negative versus neutral) collapsing across sex. In addition the results of another ANOVA will be presented in which both the effect of valence and sex are examined. Direct Comparisons of Positive, Negative and Neutral Word Generation Direct comparisons of word generation tasks collapsed across sex of participants were performed with a voxel-wise two-way (fixed condition factor = valence; random factor = participants) repeated measures ANOVA, followed by posthoc t-tests. In addition, the effect of valence and sex of participants were considered together in a voxel-wise three-way (fixed condition factors = valence, sex; random factor = participants) ANOVA. Results of the two-way (valence x participants) voxel-wise ANOVA identified significant areas of activation associated with the main effect of valence. Significant clusters were identified at a significance threshold of p <.005 and a volume threshold of 200 ìl. Figure 9 depicts the two largest clusters of activity for the main effect of valence: one cluster extends bilaterally in cortex near the frontal pole (MI xyz = -2, 60, 12) and the other cluster extends bilaterally in the retrosplenial area (MI xyz = 0, -48, 30).

97 87 1 X = Neutral Categories Positive Categories Negative Categories Sig nal Image Number Neutral Categories Positive Categories Negative Categories Sig nal Image Number Figure 9. Activity in cortex near the frontal pole and in retrosplenial cortex associated with the main effect of valence, across participants, red = p <.005; yellow = p <.001, along with corresponding hemodynamic response functions (HRFs) for positive, negative and neutral categories for these areas.

98 88 Corresponding averaged HRFs for each category type (positive, negative and neutral) are also presented in Figure 9 to show differences in the time course of the hemodynamic response corresponding with each stimulus type. An examination of the response curves reveals different time courses for the area near the frontal pole and the retrosplenial area. For the former, the rise to peak occurs gradually with a peak at Image 4 for positive and negative categories, falling to baseline at Image 8. In retrosplenial cortex, the peak occurs earlier, at Image 3, falling to near baseline levels at Image 5 or 6. These differences in time courses may provide clues to the functions of these two areas in mediating word generation to categories with emotional connotation. The finding of a robust cluster of activation in cortex near the frontal pole serves to partially confirm Hypothesis 2,which predicted unique activation bilaterally of the frontal and temporal poles for generation of words with either positive or negative emotional connotations. Crosson et al. (1999a) also found a cluster of activity near the left frontal pole when directly comparing emotional and neutral word generation (MI: xyz = -13, 62, 27). To examine differences by valence, pairwise contrasts of the average HRF were conducted within the significant clusters associated with the main effect of valence, to yield the direction of significant differences between emo tional (positive and negative valences collapsed) and neutral categories (Table 8), as well as between positive and negative categories relative to neutral word generation (Table 9 and Figure 10). As the averaged HRFs in Figure 9 reveal, as well as the findings in Tables 8 and 9, it is clear that bilateral areas of retrosplenial cortex and bilateral areas near the frontal pole are more activated by generation to categories with emotional connotations (positive and negative, but not neutral).

99 89 X = -2 Y = 58 Emotional versus Neutral Word Generation X = -3 Y = 60 Positive versus Neutral Word Generation X = -4 Y = 60 Negative versus Neutral Word Generation Figure 10. Activity in cortex near the frontal pole and in retrosplenial cortex in direct task comparisons, red = p <.005; yellow = p <.001. Clusters of activity for emotional (positive and negative categories collapsed), positive, and negative categories relative to neutral categories that were found to be

100 90 significant in the two-way repeated measures ANOVA are depicted for the frontal pole and retrosplenial areas (Figure 10).With respect to the predictions made in Hypothesis 3 and consistent with the valence hypothesis, one would expect to see a Table 8: Volumes of tissue (> 200 µl) showing significant activity changes (p <.005) for emotional (positive and negative categories collapsed) versus neutral word generation. Location Cortex near the Frontal Pole Emotional > Neutral Anatomic Area (max t loc) max t, volume in µl L and R BAs 9, 10 (-2, 58, 25) t = 5.42, 3505 ìl Retrosplenial Cortex L and R BAs 23, 31 (0, -48, 30) t = 5.76, 3168 ìl Middle Frontal Gyrus Middle/Superior Temporal Gyrus Temporoparietal Junction Lingual Gyrus/ Other Visual Areas L BAs 21, 22 (-50, -30 1) t = 4.85, 596 ìl L BAs 22, 40 (-65, -52, 32) t = 4.53,1210 ìl L BAs 22, 39 (-41, -55, 24) t = 4.32, 987 ìl Neutral > Emotional Anatomic Area (max t loc) max t, volume in µl BAs 23, 30, extending to BA 19 (-12, -55, 12) t = -4.83,1021 ìl L BA 6 (-43, 4, 36) t = -4.23, 586 ìl L BA 37 (-59, -49, -9) t = -5.13, 1667 L BAs 19, 17 (-29, -68, 32) t = -5.63, 970 ìl BAs 18, 19 (-15, -74, 7) t = -4.15, 439 Note. BA = Brodmann's Area (according to Talairach & Tournoux, 1988); max t = maximum t value within given cluster of activity. Other abbreviations are as follows: L = Left, R = Right.

101 91 greater right hemisphere advantage for negative category generation and a greater left hemisphere advantage for positive category generation in areas related to emotional semantic processing. It is apparent in Tables 8 and 9 that the generation tasks with emotional connotation still largely led to left lateralized brain activity. Only two areas Table 9: Volumes of tissue (> 200 µl) showing significant activity changes (p <.005) for positive and negative versus neutral word generation. Positive > Neutral Location Anatomic Area (max t loc) max t, volume in µl Cortex near the L and R BAs 9, 10 Frontal Poles (-3, 60, 12) t = 6.12, 3035 ìl Retrosplenial Cortex L and R, BAs 23, 31 (0, -48, 30) t = 5.53, 3422 ìl Middle Frontal Gyrus Temporoparietal Junction/ Superior Temporal Gyrus Middle Temporal Gyrus L BA 9 (-22, 42, 31) t = 3.83, 244 ìl L BAs 22, 40 (-57, -51, 24) t = 4.44, 772 ìl L BAs 22, 39 (-48, -63, 16) t = 4.32, 595 ìl L BAs 21, 22 (-50, -24, 0) t = 4.25, 518 ìl Negative > Neutral Anatomic Area (max t loc) max t, volume in µl L and R BAs 9, 10 (-4, 60, 290 t = 5.17, 1807 ìl L and R BAs 23, 31 (0, -47, 29) t = 4.50,1042 ìl L BA 22 (-54, -44, 23) t = 3.96, 300 ìl L BAs 22, 39 (-40, -55, 26) t = 4.84, 872 ìl L BA 21 (-50, -30, 1) t = 4.16, 413 ìl Note. BA = Brodmann's Area (according to Talairach & Tournoux, 1988); max t = maximum t within given cluster of activity. Other abbreviations are as follows: L = Left, R = Right. showed bilateral activity: cortex near the frontal poles, and the retrosplenial area. The relative contribution of the left and right hemisphere for positive versus neutral categories and negative versus neutral categories is provided in Table 10 for these two areas of

102 92 bilateral activation. An examination of relative volume size in the left and right hemispheres reveals that for both areas, and across emotional valences, the activity is predominantly in the left hemisphere. In addition, an examination of the relative contribution of the right hemisphere in these two areas is also inconsistent with the valence hypothesis. This table clearly shows a pattern opposite to what the valence hypothesis would predict, with a slightly greater relative contribution of the right hemisphere for positive (22% total cluster in right frontal pole; 35% total cluster in right retrosplenial area), rather than negative categories (4% total cluster in right frontal pole; 18% total cluster in right retrosplenial area). A direct comparison of positive and negative category generation across participants revealed only two significant clusters of activity, both lateralized to the left hemisphere. Activity for positive categories was significantly Table 10: Relative contribution of the left and right hemispheres for positive and negative versus neutral word generation (p <.005). Location Positive > Neutral Negative > Neutral L Frontal Pole (BAs 9, 10) 2366 ìl; 78% total cluster 1726 ìl; 96% total cluster R Frontal Pole (BAs 9, 10) 669 ìl; 22% total cluster 81 ìl; 4% total cluster L Retrosplenial (Bas 23, 31) 2211 ìl; 65% total cluster 858 ìl; 82% total cluster R Retrosplenial (BAs 23, 31) 1211 ìl; 35% total cluster 184 ìl; 18% total cluster greater than activity for negative categories (significance threshold of p <.005; volume threshold of 200 ìl) in a retrosplenial region centered in the left hemisphere with some activity crossing over to the right hemisphere (MI: xyz = -2, -59, 14). In an area of the left inferior frontal sulcus (MI: xyz = -46, 18, 28), activity for negative categories was

103 93 significantly greater than for positive categories. These activity clusters are depicted in Figure 11. X = -2 X = - 46 Figure 11. Positive versus negative word generation. Red and blue = p <.005; Yellow and light blue = p <.001. The cluster in red and yellow was more activated by positive categories; the cluster in blue and light blue was more activated by negative categories. Finally, to address Hypothesis 4 regarding sex differences, activation differences according to sex of participants was determined by performing a voxel-wise three-way (fixed factors = valence, sex; random factor = participants) ANOVA followed by posthoc t-tests. Results of this voxel-wise analysis identified significant areas of activation associated with the main effect of sex of participants and the interaction effect of sex of participants and valence of categories. Two clusters of activity survived a p threshold of.005, with a volume threshold of 200 ìl for the main effect of sex. These areas were right precentral gyrus (MI xyz = 50, -4, 27) and left cerebellum (MI xyz = -28, -42, 22). In both cases, these areas were more active for men than for women. The interaction effect of sex of participants and valence of categories showed no significant areas of activity with a p threshold of.005 and a volume threshold of 200 ìl. Therefore, regardless of valence type, men showed more activity in the right precentral gyrus and the left