Studies of Neural Processing in Deaf Signers: Toward a Neurocognitive Model of Language Processing in the Deaf

Transcription

1 Empirical Articles Studies of Neural Processing in Deaf Signers: Toward a Neurocognitive Model of Language Processing in the Deaf David P. Corina University of Washington The ability to comprehend and produce language stands as a defining characteristic of human cognition and enables the transfer of knowledge and culture within human society. A proper characterization of the human capacity for language is required for the development of interventions that may be used to assist those individuals who have failed to achieve, or who have lost competence in, language behaviors. For signed languages, models of competent language use are lacking. This lack of knowledge hampers the development of effective assessment measures for deaf children who may be experiencing learning problems beyond those confronting the normal deaf child. I discuss two research avenues that have begun to provide a window into the neural systems involved in sign language processing: studies of language disruptions in adult deaf signers who have suffered brain injury, and studies of functional brain imaging in normal deaf signers. This research provides a basis for the development of a comprehensive neurocognitive model of sign language processing. The ability to comprehend and produce language stands as a defining characteristic of human cognition and enables the transfer of knowledge and culture within human society. A proper characterization of the human capacity for language is required for the development of interventions that may be used to assist those individuals who have failed to achieve, or who Thii work w»s supported in part by «grant from the University of Washington, RRF md a grant from NIDCD R29 DC A1 awarded to the author. I thank my collaborator! on the fmri studies, Drs. Helen Neville and Daphne Bavelier. I thank Connie Schachtel for editorial assistance. Figure 2: copyright Dr. Ursula Bellugi, the Salk Institute, La Jolla, California Figures 3 and 4: copyright DT. David Corina, all righo reserved, University of Washington, Correspondence should be sent to David P. Corina, Department of Psychology, Univenity of Washington, Seattle WA Copyright O 1998 Oxford University Press. CCC have lost competence in, a full range of language behaviors (e.g., effective interpersonal communication, reading, writing, etc.). Cognitive psychologists have made great strides in understanding the functional and neural mechanisms underlying the use of spoken language. These studies have led to a wide range of effective educational and clinical programs for enhancing language behaviors (Tallal, Miller, Bedi, Byma, Wang, Nagarian, Schreiner, Jenkins, & Merzenich, 1996; Merzenich, Jenkins, Johnston, Schreiner, Miller, & Tallal, 1996). For signed languages however, models of competent language use are lacking. This lack of knowledge hampers the development of effective assessment measures for deaf children who may be experiencing learning problems beyond those confronting the normal deaf child. Moreover, our lack of knowledge stifles the development of effective intervention strategies for these children. The development of a comprehensive neurocognitive model of sign language processing is crucial if we are to properly serve the needs of the deaf signing community. Development of such a model would also benefit basic science, providing insight into how altered sensory experience affects the development of neural systems underlying cognitive functions. Finally, this model will benefit cognitive scientists interested in functional models of human language. Within the last decade there has been a monumental increase in our knowledge of cognitive processing in deaf signing individuals (see, for example, Hanson, 1990; Hanson, Lichtenstein, 1990; Neville, Mills, & Lawson, 1992; Parasnis & Samar, 1985, Marschark,

2 36 Journal of Deaf Studies and Deaf Education 3:1 Winter a, 1993b; Emmorey, Corina, & Bellugi 1995; Emmorey, Kosslyn & Bellugi, 1993). In addition, the body of literature exploring online processing of sign language is growing (Emmorey, 1993; Emmorey & Corina 1990; Mayberry & Eichen, 1991; Mayberry & Fischer, 1989). Advances in sign language linguistics have provided a basis of comparison for cross-language and cross-modality studies (Perlmutter, 1993; Corina, 1990; Corina & Sandier, 1993). Taken together these behavioral studies provide a foundation for the development of functional models of cognitive and language processing in deaf signers. An equally important goal is to understand the neural systems that underlie these functional characterizations. Research in this direction is required in order to develop a comprehensive neurocognitive model of sign language processing. In this article I discuss two research avenues that have begun to provide a window into the neural systems involved in sign language processing: studies of language disruptions in adult deaf signers who have suffered brain injury, and studies of functional brain imaging in normal deaf signers. Sign Language Aphasia In the 1800s noted neurologist Hughlings Jackson broached the issue of neural control of signed language. In a much quoted musing, Jackson said: "No doubt by disease of some part of his brain the deafmute might lose his natural system of signs" (Jackson, 1878). Since this time researchers have looked to case studies of deaf signing individuals to answer two broad questions: first, whether left hemisphere structures mediate the signed languages of deaf individuals, and second, whether deaf individuals show complementary hemispheric specialization for language and nonlanguage visuospatial skills. More recent investigations have focused on the question of intrahemispheric specialization for sign language systems. Important generalizations regarding these two questions are beginning to emerge from this complicated literature. Of the case studies of brain-damaged deaf and/or signing individuals reported to date, roughly 20 involve left-hemisphere damage, while 8 involve right hemisphere damage (see Corina, in press, for a recent review). These case studies vary greatly in their ability to explain underlying brain processes involved in signing. Many of the early case studies were hampered by a lack of understanding of the relationships among systems of communication used by deaf individuals. For example, several of the early studies discussed disruptions of fingerspelling and only briefly mentioned or assessed sign language use. Anatomical localization of lesions was often lacking or confounded by the existence of multiple infarcts. Rarely were etiologies of deafness or audiological reports presented. Despite these limitations, with careful reading general patterns do emerge. More recently, well-documented case studies have started to provide a clearer picture of the neural systems involved in language processing in users of sign languages. One conclusion that can be drawn from the sign aphasia literature is that right-handed deaf signers, like hearing persons, exhibit language disturbances when critical left hemisphere areas are damaged. Of 16 left hemisphere cases reviewed by Corina (in press), 12 provide sufficient detail to implicate left hemisphere structures in sign language disturbances. Five of these cases provide neuro-radiological or autopsy reports to confirm left hemisphere involvement and provide compelling language assessment- to implicate aphasic language disturbance. In hearing individuals, severe language comprehension deficits are associated with left hemisphere posterior lesions, especially posterior temporal lesions. Similar patterns have been observed in users of signed languages. For example, in the case of WL, reported by Corina, Kritchevsky, and Bellugi (1992), the subject had damage to posterior temporal structures and evidenced marked comprehension deficits. WL showed a gradation of impairment across tasks, with some difficulty in single sign recognition, moderate impairment in following commands, and severe problems with complex ideational material. In contrast, a right hemisphere-damaged signer, SM, who also suffered a large right hemisphere lesion with parietal extension, showed only mild impairment on only the most difficult of comprehension tests (see Figure 1). In users of spoken languages, impairment in language production with preserved comprehension is associated with left hemisphere anterior lesions. The execution of speech movements, for example, involves the

3 Studies of Neural Processing in Deaf Signers 37 Sign Comprehension Tests 100H o <D O LHD W.L CJ RHD S.M. CD O at Q_ single signs commands ideational material Figure 1 Sign language comprehension data comparing left hemisphere-damaged signer WL with a right hemisphere control subject SM. Comprehension measures are from the Salk Institute adaptation of the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1972). Single sign comprehension requires patients to match a sign to a picture, with appropriate target and foils (eg., SHOW-ME TREE). The command test requires subjects to follow one-, two-, or three-part commands (eg., POINT-TO DOOR, FINISH, POINT-TO TABLE). Ideational questions ask patients to judge the truthfulness of complex sentences (e.g., Will a rock float on water?). cortical zone encompassing the lower posterior portion of the left frontal lobe (Goodglass, 1993). Left-frontal anterior regions are also implicated in sign language production impairment (Poizner, Klima, & Bellugi, 1987). A representative case is that of Poizner et al.'s (1987) subject GD, who experienced damage to Broadman's areas 44 and 45 of the left frontal lobe. This subject's signing was effortful and dysfluent and reduced largely to single sign utterances, yet GD had intact sign language comprehension. In summary, evidence from lesion studies of deaf signers indicates left hemisphere dominance for sign language processing. In addition, familiar global patterns of comprehension and production deficits following posterior and anterior damage, respectively, are found. However, a more challenging question concerns the extent to which the identical neuroanatomical systems that are responsible for mediating auditory languages are involved in the mediation of visual sign lan- guages. Ultimately, language modality will influence the neural systems controlling peripheral motor and perceptual systems involved in the use of language. However, whether the exact neural substrates underlying the symbolic systems supporting language comprehension and production are shared by spoken and signed languages remains to be determined. Scrutiny of individual case studies suggests that there may be subtle differences in the specialization of neural systems underlying signed and spoken language. Several researchers have suggested subtle left hemisphere reorganization based upon evidence from cases involving deficits in sign comprehension (Leischner, 1943; Poizner et al., 1987; Chiarello, Knight, & Mandel, 1982). For example, while both the cases of KL (Chiarello et al., 1982; Poizner et al., 1987) and WL (Corina, Poizner, Feinberg, Dowd, & O'Grady, 1992) revealed severe sign comprehension difficulties, neither subject's lesions occurred in cortical Wernicke's area, an

4 38 Journal of Deaf Studies and Deaf Education 3:1 Winter 1998 area classically associated with speech comprehension disturbance. Research also suggests that the equivalent of a sign language Broca's aphasia may involve cortical areas that differ from those associated with Broca's aphasia in spoken language. The argument is based upon the observation that cases of Broca's-like sign aphasia are consistently reported with accompanying agraphia (i.e., written language impairment), as well as with fingerspeuing disturbances. In contrast, spoken language Broca's aphasia may or may not co-occur with agraphia (Levine & Sweet, 1982). This dissociation has not yet been observed in signers. Based on this observation, Corina (in press) speculates that nonfluent signing aphasias require involvement of classic Broca's area and encroachment upon cortical and subcortical motor areas of the precentral gyrus involved in hand and arm representations. Thus, while at a general level the anterior/posterior and nonfluent/fluent dichotomies hold for spoken and signed languages, there are some indications that within-hemisphere reorganization may be present in the deaf. Data suggest possible subtle differences in the cortical organization of sign and speech that must be further validated with both lesion studies and functional in vivo imaging studies. Hemispheric Specialization Cases of right hemisphere-damaged signers provide an opportunity to assess hemispheric specialization for nonlinguistic visuospatial function. All of these cases to date report moderate to severe degrees of visuospatial impairment in signers with right hemisphere damage (Poizner et al. 1987; Corina, Kritchevsky, & Bellugi, 1996; Corina, Bellugi, Kritchevsky, O'Grady-Batch, & Norman, 1990; Kegl & Poizner, 1991). In contrast, none of the left hemisphere-damaged signers tested on visuospatial tests showed significant impairment. Thus, damage to critical left hemisphere structures produces sign language aphasia in deaf signers. Right (but not left) hemisphere lesions produce visuospatial impairments in deaf signers. Taken together, these findings suggest that deaf signers show complementary specialization for both language and nonlanguage skills. A recent group level comparison yields a similar conclusion (Hickock, Bellugi, & Klima, 1996). These studies demonstrate that development of hemispheric specialization is not dependent upon exposure to oral/ aural language. Neurolinguistics of Sign Language Aphasia Studies of lifelong signers who have incurred brain damage have established the importance of the left hemisphere in the mediation of sign language. An important question then is what is the manifestation of these sign language impairments? Spoken language breakdown following left hemisphere damage is not haphazard, but affects independently motivated linguistic categories. There is now ample evidence that sign language also breaks down in a linguistically significant fashion. The best documented work concerns impairments in sign production. Below I present descriptions of sign language phonemic paraphasias and impairments of lexical and inflectional morphology. Spoken language phonemic paraphasias arise from the substitution or omission of sublexical phonological components (Blumstein, 1973). In American Sign Language (ASL), sublexical structure refers to the formational elements that comprise a sign form: handshape, location, movement, and orientation. In signed languages, paraphasic errors result from substitutions within these parameters. For example, Poizner et al.'s (1987) subject KL produced paraphasic signing errors in which substitutions were found in all four major parameters. For example, the sign ENJOY, which requires a circular movement of the hand, was articulated with an incorrect up and down movement, indicating a substitution in the formational parameter of movement. Substitutions in the parameters of orientation, handshape, and location were also reported. In principle, selectional errors could occur among any of the four sublexical parameters (and they do). However, the most frequently reported errors are those affecting the handshape parameter. Corina et al. (1992) describe in some detail the phonemic errors produced by WL, errors which almost exclusively involved handshape specifications. For example, WL produced the sign TOOTHBRUSH with the Y handshape rather than the required G handshape, and produced the sign SCREWDRIVER with an A handshape rather than the required H handshape (see Figure 2). Based upon a linguistic analysis of these errors, Corina et al. (1992) have presented evidence that these handshape substitutions

5 Studies of Neural Processing in Deaf Signers 39 "FINE" "SCREWDRIVER 1 "WHITE" "TOOTHBRUSH" Figure 2 WL's Handshape-specific errors. Examples of sign language phonemic paraphasias in patient WL. Most of WL's errors consisted of incorrect selection of handshapes with correct place of articulation and movement. Copyright, Dr. Ursula Bellugi, The Salic Institute, La Jolla, California. Reprinted with permission. are phonemic in nature, rather than phonetic misarticulations. Finally, phonological paraphasias in sign aphasia do not compromise the syllabic integrity of a sign (Brentari, Poizner, & Kegl, 1995). For example, we observe substitutions of movements rather than omissions (the latter would violate syllable well-formedness in ASL). Morphological and syntactic errors. A common error pattern in spoken language aphasia is the substitution and omission of bound and free morphemes. Because languages differ in the degree to which they use morphology to mark obligatory grammatical distinctions (e.g., case and gender, subject and object agreement, etc), patterns of impairment may be more striking in some languages than in others (Bates, Wulfeck, & MacWhinney, 1991; Menn & Obler, 1990). ASL is a highly inflected language; in addition to temporal and adverbial inflections, ASL has a class of verbs that inflect for person and number agreement. Morphosyntactic agreement distinguishing grammatical subject and object requires directional movement trajectories. In the absence of grammatical movement trajectories, a verb sign will be produced in an uninflected "citation" form. Poizner et al. (1987) have investigated morphosyntactic impairments in their patients. Poizner's pa-

6 40 Journal of Deaf Studies and Deaf Education 3:1 Winter 1998 tient GD consistently omitted required inflectional morphemes in her spontaneous signing and instead produced uninflected "citation" verb forms. Poizner's patient PD produced both omissions in inflectional morphology and inconsistent verb agreement substitutions. That is, PD failed to maintain consistent verb movement trajectories to spatial locations, as is required by syntactic and discourse conventions. GD had a large left hemisphere lesion that involved most of the convexity of the left frontal lobe, including Broca's area. PD had a subcortical lesion in the left hemisphere, with anterior focus deep to Broca's area and posterior extension into the white matter in the left parietal lobe. The general pattern of omissions versus substitutions in signers GD and PD is consistent with profiles of agrammatic and paragrammatic impairment, respectively, reported for users of spoken language. The use of facial morphology has recendy been investigated as well. Facial expressions serve a dual function in ASL, conveying both affective and linguistic information. Separate classes of facial expressions subserve these two distinct functions. Differential impairment in these two classes of facial expressions have been reported (Zein, Say, Bellugi, Corina, & Reilly, 1993). Recent recognition and production studies official expression in normal deaf signers indicate bilateral mediation of facial expressions. This pattern contrasts with the strong right hemisphere advantage shown by hearing persons, for whom facial expressions serve mainly affective purposes (Corina, 1989). In summary, production of sign language morphology is vulnerable following left hemisphere damage in deaf signers. Taken together, these results demonstrate that language abilities in deaf signers break down in linguistically significant ways. These findings provide evidence that language impairments following stroke in deaf signers are aphasic in nature and do not reflect a general problem in symbolic conceptualization or motor behavior. Right Hemisphere and Language While the left hemisphere plays a crucial role in core aspects of language comprehension and production, there is a growing awareness that both hemispheres are required for language usage in a social context. For hearing persons, right hemisphere impairments disrupt the meta-control of language use, as evidenced by disruptions of discourse abilities (Brownell, Simpson, Bihrle, & Potter, 1990; Kaplan, Brownell, Jacobs, & Gardner, 1990; Rehak, Kaplan, Weylman, Kelly, Brownell, & Gardner, 1992). There is growing evidence that the right hemisphere plays a crucial role in the discourse abilities of deaf signers. Analysis of language use in right hemisphere-lesioned subject JH (Corina et al., 1992; Corina et al., 1996) revealed occasional non sequiturs and abnormal attention to details, which are characteristic of the discourse of hearing patients with right hemisphere lesions (Delis, Robertson, & Balliet, 1983). Subject DN, reported in Poizner and Kegl (1992), showed another pattern of discourse disruption. While DN was successful at spatial indexing within a given sentence, several researchers noted that she was inconsistent across sentences 1 (Emmorey et al., 1995; Poizner & Kegl, 1992). That is, she did not consistently use the same spatial index from sentence to sentence. In order to salvage intelligibility, DN used a compensatory strategy in which she restated die noun phrase in each sentence, resulting in an overly repetitive discourse style. An English equivalent might involve noun repetitions such as: "Tom went to die store; Tom looked for eggs; Tom paid for eggs; Tom went home" versus "Tom went to the store; he looked for eggs; he paid for them and then went home." The cases of JH and DN suggest diat right hemisphere lesions in signers can differentially disrupt discourse content (as in die case of JH) and discourse cohesion (as in the case of DN). Lesion site differs significantly in these two patients; JH's stroke involved central portions of the frontal, parietal, and temporal lobes, whereas DN's lesion was predominandy medial and involved the upper part of the occipital lobe and superior parietal lobule. Whether similar lesions in hearing individuals would result in differential discourse patterns awaits furdier study. In the cases of impaired discourse production, it is reasonable to suppose that general right hemisphere cognitive deficits manifest themselves in aspects of language use. For example, an attentional deficit might manifest itself as a perseveration of topics of conversation, while a spatial memory impairment could result in an inability to keep track of references across stretches of discourse. For users of ASL, diese types of

7 Studies of Neural Processing in Deaf Signers 41 deficits might manifest themselves as impairments in language continuity and spatial reference. It seems reasonable, then, to entertain the possibility that right hemisphere damage does not disrupt linguistic function per se, but rather impairs the execution and processing of linguistic information in sign language, in which spatial information plays a particularly salient role. However, the issues become more complicated when we consider the syntactic aspects of ASL. Disturbances in syntactic processing in ASL have been reported in patients with both left hemisphere and right hemisphere damage. In the case of left hemisphere damage, problems in the production and comprehension of spatialized syntax have been noted. For example, patient GD evidenced production problems such as omitting necessary inflections, whereas subject PD was inconsistent in his use of spatial referencing in the service of grammatical relations. Subjects PD, WL, and KL all demonstrated problems in the comprehension of syntactic relationships expressed via the spatial syntactic system. More surprising, however, is the finding that some signers with right hemisphere damage also exhibited problems. Two of the right hemisphere-damaged subjects, tested by Poizner et al. (1987), showed performance well below controls on two tests of spatial syntax (see description of SM and GG). Indeed, as pointed out in Poizner et al. (1987), "Right lesioned signers do not show comprehension deficits in any linguistic test, other than that of spatialized syntax." Poizner et al. (1987) speculated that the perceptual processing involved in the comprehension of spatialized syntax involves both left and right hemispheres; certain critical areas must be relatively intact for accurate performance. Recent evidence from fmri studies of deaf signers attests to the importance of both left and right hemisphere function in normal deaf signers. Functional Neuroimaging Studies Functional neuroimaging studies provide another avenue for investigating the underpinnings of language processing in the deaf. Functional neuroimaging, broadly defined, is the measurement of brain activity during task performance. Common techniques that have been successfully used to study language and cognitive function include Positron Emission Tomography (PET) (e.g., Fox, Raichle, Mintun, & Dence 1988; Petersen, Fox, Posner, Mintun, & Raichel, 1988; Habib, Demonet, & Frackowiak, 1996), Event-related potentials (ERP) (e.g., Osterhout, 1994), regional cerebral bloodflow (rcbf), Magnetoenchelagram (MEG) (e.g., Lounasmaa, Hamalainen, Han, & Salmelin, 1996) and Functional Magnetic Resonance Imaging (fmri) (Cohen, Noll, & Schneider 1993; Binder, 1995). One advantage of these techniques is that they provide information about brain function from awake human beings while they are performing controlled experimental tasks. Neville et al. have used ERP methodology to study language processing in hearing and deaf subjects. The ERP technique provides millisecond temporal resolution and thus is highly suitable for studying the rapid on-line processing involved in language behavior. The ERP technique, however, is less well suited for establishing the anatomical location of neural areas mediating these behaviors. One important question addressed in these studies concerns whether the various subcomponents of linguistic structure (semantics, syntax, and phonology) have identifiable signatures in the eventrelated brain potentials. Research from written language paradigms shows evidence for differential processing of "open" versus "closed" class items (Neville, Mills, & Lawson, 1992). Closed-class items are those grammatical formatives that convey structural or relational aspects of sentence meaning (for example, articles and auxiliaries in English), while open-class elements (such as nouns, verbs, and adjectives) make reference to specific objects and events. In normal hearing adults the ERP response to meaning-bearing, open-class words is characterized by a prominent negative component maximal around 350 msec after word onset. This component was larger over posterior brain regions of both hemispheres. In contrast, ERP responses to closed-class words displayed a prominent negative potential at 280 msec that was localized to the anterior and temporal regions of the left hemisphere. When this same series of studies was conducted with deaf native signers processing English, ERP responses to open-class words were virtually identical to those observed in the normal hearing subjects. However, the responses to the closed-class words were markedly different. Deaf subjects (for whom English was a second language) lacked the negative potential over ante-

8 42 Journal of Deaf Studies and Deaf Education 3:1 Winter 1998 rior regions of the left hemisphere, and no lateralized asymmetry was observed between the hemispheres (Neville, Mills, & Lawson, 1992). These results suggest that the neural systems that mediate syntactic processing between the hemispheres are more vulnerable to variability of the nature and timing of early language experience than are the neural systems linked to lexical-semantic processing. Importantly, a follow up study (Neville, Coffey, Lawson, Fischer, Emmorey, & Bellugi, 1996) showed that for native deaf signers processing ASL sentences, the characteristic differences between open-class and closed-class grammatical elements were observed (Neville et al., 1996). Specifically, native deaf signers showed differential specialization of anterior and posterior cortical regions for aspects of grammatical and semantic processing, respectively. These results suggest similarities in the organization of the neural systems that mediate formal languages, independent of the modality through which language is acquired. Recently, functional imaging techniques have been used to examine sign language representation in the brain. These studies, while providing good spatial resolution, are less well suited for establishing the temporal time-course of processing involved in language behaviors. A series of studies by Soderfeldt (1994) and Soderfeldt, Ronnberg, and Risberg (1994) used rcbf and PET to look at functional representations of sign. Studies of hearing bilingual speaker-signers revealed bilateral posterior temporal activation for both spoken language comprehension and sign comprehension. In a separate study with native deaf signers, a similar bilateral pattern was found, but with increased right hemisphere parietal-occipital activation. Soderfeldt (1994) suggested that this increased activity reflected the greater spatial processing required for perception of a signed language. Hickock et al. (1995) reported on an fmri production study of covert naming and word (sign) generation. Analogous tasks with spoken language have shown significant activation in Broca's and Wernicke's areas in hearing individuals. Two native deaf signers showed activation in Brodmann areas 44/ 45 (Broca's area) and posterior area 22 (Wernicke's area) predominantly on the left, although right hemisphere homologues did show significant activation. Additional sites of activation included premotor areas, cerebellum, and prefrontal cortex. These authors concluded that the similar patterns of left perisylvian activation in deaf and hearing subjects indicates that neural organization for language processing is modality independent. Hickock, Bellugi, and Klima (1996) contrasted activation for an ASL rhyming test with a working memory task. Previous investigations of rhyming versus verbal working memory tasks in spoken language have identified activation in the inferior frontal gyrus, predominantly on the left for rhyming tests (see, Zatorre, Meyer, Gjedde, & Evans, 1996) and implicate dorsolateral prefrontal cortex for verbal working memory (Awh, Jonides, Smith, Schumacher, Koeppe, and Katz, 1996; Cohen, Forman, Braver, Casey, Servan- Schreiber, & Noll, 1994; Fiez, Raife, Balota, Schwartz, Raichle, & Pertersen, 1996). Hickock et al. (1996) report patterns of activation from a single subject that suggest differences in the distribution of activation for rhyming and memory tasks. This finding suggests that subregions within classical Broca's area may contribute differentially to different language-related tasks. fmri Study of ASL Sentence Processing One of the most comprehensive fmri projects involving deaf signers has recently been reported by Corina, Bavelier, and Neville (Bavelier, Corina, Clark, Jezzard, Padmanhaban, Prinster, Kami, Rauschecker, Turner, & Neville, in press; Neville, Bavelier, Corina, Rauschecker, Kami, Lalwani, Braun, Clark, Jezzard, & Turner, in press). These studies examined neural activation patterns as hearing and deaf subjects read English words or viewed signed ASL sentences. These studies provide further evidence for common neural areas in the left hemisphere that subserve language processing regardless of language modality. These studies also revealed language-specific activation in the right hemisphere of native hearing and deaf signers. (An overview of the procedure and data analysis can be found in Bavelier et al., in press). Three groups of subjects were tested: hearing individuals who did not know sign language, native deaf signers, and normally hearing individuals bom to deaf parents for whom ASL was a first language. Table 1 illustrates the subject characteristics for these three groups. Each population was imaged 2 while processing sen-

9 Studies of Neural Processing in Deaf Signers 43 Table 1 Subject characteristics English exposure English proficiency ASL exposure ASL proficiency Hearing Mean age Handedness Number of subjects Hearing Deaf Hearing native signers Birth Native None None Normal 26 Right 16 School age Moderate Birth Native Profound deafness 23 Right 23 Birth Native Birth Native Normal 35 Right 16, 18' '16 subjects participated in the English condition and 18 in the ASL condition. Table 2 Performance on fmri task (percentage correct) English Sentences Consonant strings ASL Sentences Nonsigns Hearing Deaf Hearing native signers DPLC Angular Gyrus Broca's Wernickc' tences in both English and ASL. The English stimuli were presented in cycles defined as 32-second blocks of sentences that alternated with 32-second blocks of consonant strings. The ASL stimuli consisted of 32second blocks of videotape of a native deaf signer producing sentences in ASL. These alternated with 32-second blocks in which the signer made nonsign gestures that were physically similar to ASL signs. Four cycles of alternations were presented in each of four runs, two for each language. At the end of each run, subjects indicated whether or not specific sentences and nonword/nonsign strings had been presented. The behavioral data indicated that subjects were attending to the stimuli and were better at recognizing sentences than nonsense strings. Hearing subjects who did not know ASL performed at chance in recognizing ASL sentences and nonsense signs. All subjects performed equally well on simple declarative English sentences, and deaf and hearing native signers performed equally well on ASL sentences (see Table 2). We examined the effects of language type for these three groups of subjects in four language areas typically associated with language processing: Broca's area, Wernicke's area, the angular gyrus, and the dorsolateral precentral cortex (DPLC) (see Figure 3). Broca's Figure 3 Left hemisphere regions of interest evaluated for functional activity. Right hemisphere homologues are the equivalent right hemisphere anatomical areas. area is involved in language production abilities; Wernicke's area is considered important in language comprehension abilities; classically, the angular gyrus has been implicated in reading; and, as previously stated, the DPLC has been implicated in working memory behavior (Awh et al., 1996; Cohen et al., 1994; Fiez et al., 1996). fmri results: English. When hearing subjects read English sentences, we observed robust activation in all four language areas3 in the left hemisphere. In contrast, within the right hemisphere none of the homologous language areas were reliably active (see Figure 4, upper left panel). Thus, the data from neurologically intact users of spoken language are consistent with the ubiquitous left hemisphere asymmetry described by over a century of language research. An analogous pattern of activation was observed when hearing signing subjects read English sentences (see Figure 4, upper middle panel). Based on the analy-

10 44 Journal of Deaf Studies and Deaf Education 3:1 Winter 1998 English Hearing Hearing Native Signers Deaf ASL Hearing Hearing Native Signers Deaf p<.025 I K Figure 4 Composite representations of functional activation in response to linguistic stimuli in three subject groups: hearing nonsigners, hearing native signers, and deaf native signers. sis of these four language areas, a clear left hemisphere lateralization emerges, a pattern that is comparable to that observed for hearing nonsigners reading English. In contrast, for the deaf signing subjects reading English, a very different pattern emerged. Deaf subjects did not display left hemisphere dominance when reading English. The only left hemisphere language area reliably active was Wernicke's area. Interestingly, we observed reliable right hemisphere activation in the posterior temporal and parietal areas, the right hemisphere homologues of Wernicke's area, and the angular gyrus (see Figure 4, upper right panel). fmri results: ASL. As might be expected, hearing subjects who did not know ASL did not show any reliable activation in response to the difference between meaningful and nonmeaningful signs. This lack of activation is consistent with the behavioral data (see Figure 4, lower left panel). Deaf subjects processing ASL displayed significant left hemisphere activation for their native language (see Figure 4, lower right panel). Significant activation is observed within Broca's area, Wernicke's area, the DPLC, and the angular gyrus. Note that this activation pattern is similar to that observed in hearing subjects processing English. This result suggests that acquisition of a spoken language is not a prerequisite for establishment of language systems within the left hemisphere. Interestingly, the processing of ASL sentences in deaf subjects also strongly recruited right hemisphere structures. Reliable activation was observed in homologous right hemisphere structures including Broca's area, Wernicke's area, the DPLC, and the angular gyrus. This bilateral pattern of activation differs significantly from that observed when hearing subjects read English sentences. Hearing native signers processing ASL displayed left hemisphere activation similar to that of the deaf signers. All four classic language areas were highly active (i.e. Broca's area, Wernicke's area, the DPLC, and the angular gyrus). Interestingly, these same subjects also showed right hemisphere activation similar to that of the deaf subjects. However, the hearing native signers' right hemisphere activation was primarily limited

11 Studies of Neural Processing in Deaf Signers 45 to posterior temporal and parietal sites (e.g., the right hemisphere homologues of Wernicke's area and the angular gyrus) (see Figure 4, lower middle panel). The findings of right hemisphere activation for ASL processing by deaf and hearing native signers are particularly important and suggest that these effects are not attributable to hearing loss per se, but may reflect the processing demands imposed by ASL. Discussion Several important findings emerge from these studies. We have observed (1) differences in brain activation between deaf and hearing subjects reading English, (2) consistent left hemisphere activation for subjects processing a native language (either English or ASL), and (3) robust right hemisphere parietal activation in hearing and deaf signers processing ASL. As noted, hearing nonsigning subjects displayed robust activation within standard language areas of the left hemisphere when reading English sentences. In contrast, for deaf subjects, left hemisphere activation was limited to Wernicke's area. There are several aspects of these deaf subjects' varied experiences with English that might account for these differences. One possibility is that the bilingual status of the deaf signers has contributed to this pattern. Specifically, learning ASL as a first language may have altered neural representations for English. This hypothesis can be discounted, however, by considering the data from the hearing native signers, all of whom also learned ASL as a native language. These subjects, like hearing nonsigners, displayed the expected left hemisphere dominance while reading English. Thus, the lack of left hemisphere activity and the presence of right hemisphere activation when deaf subjects read English was probably not due to the acquisition of ASL as a first language, since hearing native signers did not display this pattern. Another possibility is that deaf individuals' acquisition of English does not benefit to the same degree from the sound-based strategies available to hearing individuals, A third and related possibility is that many deaf individuals do not obtain full grammatical competence in English. However, it should be noted that on tests of English grammaticality (Linebarger, Schartz, & Saffran, 1983), our subject pool scored reasonably well (mean 80%, range 6%-26% errors). These results suggest that the cortical organization for written English is highly specific to the language experience of the subjects. For native hearing signers, who are also fluent in English, the cortical patterns of activation for written English shared the main features of those observed in monolingual hearing subjects. However, in deaf subjects, the cortical patterns of activation were more similar to those observed for ASL (see below). These results suggest that the processing of English in deaf subjects may build on the same systems as those that mediate ASL processing, while in hearing subjects, the processing systems for written English are most probably guided by those that mediate spoken English processing. An important finding is the consistent left hemisphere activation observed for subjects processing their native language (either English or ASL). Deaf subjects, when processing ASL, displayed significant left hemisphere activation. Activation within Broca's area, Wernicke's area, the DPLC, and the angular gyrus was similar to that observed in hearing subjects processing English. This result suggests that acquisition of a spoken language is not necessary for establishing language systems within the left hemisphere. Robust left hemisphere activation of Broca's area, DPLC, and Wernicke's area was also observed in native hearing signers. Hence, in every subject (hearing or deaf) processing their native language (English or ASL), these left hemisphere structures were recruited. These results imply that there are strong biological constraints that render these particular brain areas well-suited for the processing of linguistic information, independent of the structure or the modality of the language. Finally, a surprising finding was the extent of right hemisphere activation observed while deaf signers processed ASL. One question that arises is whether this activation is attributable to auditory deprivation or to the acquisition of a language that depends on visual/ spatial contrasts. Fortunately, data from hearing native signers provide insight into this question. Hearing native signers processing ASL displayed right hemisphere parietal activation similar to that of the deaf subjects. These data suggest that the activation of right parietal structures is not a consequence of deafness per se, but may be a neural signature of the linguistic pro-

12 46 Journal of Deaf Studies and Deaf Education 3:1 Winter 1998 cessing of ASL. The right hemisphere activation that has been observed may be a response to certain modality-specific characteristics of signed languages namely, the fact that linguistic and visuospatial information temporally coincide. Finally, it is of interest to note that differences between hearing native signers and deaf native signers were observed in right hemisphere frontal areas (Broca's homologue and the DPLC). It is possible that these frontal activations reflect processing areas that have become specialized as a result of auditory deprivation. For example, the right hemisphere DPLC has been implicated in visuospatial working memory (Awh et al., 1996). Whether the recruitment of these frontal areas reflects a dependence upon visual processing in profoundly deaf persons awaits further study. Taken together, these studies provide evidence for the role of the left hemisphere in processing earlyacquired, fully grammatical linguistic systems. Moreover, these studies support the claim that common neural areas in the left hemisphere subserve language processing, regardless of language modality. These studies also indicate that sign language processing requires participation of the right hemisphere to a greater extent than does the processing of written English. The co-occurrence of visuospatial and linguistic information may result in the recruitment or maintenance of these areas in the language system for native deaf and hearing signers. Additionally, these studies suggest that deaf signers' processing of written English is markedly different from that of hearing individuals. These differences suggest that deaf individuals may be making use of ASL-based neural systems in the encoding of written English. This study of sentence processing across populations is an attempt to explore neural correlates of signed and written language processing. However, the research discussed above provides only a first glimpse at the neural systems involved in these language behaviors. Many more questions remain to be answered. For example, it will be important to determine the processing contributions of the common left hemisphere language areas that are activated in signers and users of spoken languages. In addition, we need to explore the role of the right hemisphere areas active during sign comprehension. A first step in this direction will be to examine patterns of neural activation for specific subcomponents of language and nonlanguage behaviors (e.g., processing of phonology versus syntax, processing of linguistic visuospatial properties versus nonlinguistic visuospatial processes) using tasks specifically designed for these purposes. Additional work is needed to explore language representations in populations of normative signers and late learners of sign language to further explore the roles that language experience and biological endowments play in language processing. Conclusion The development of a comprehensive neurocognitive model of sign language processing is crucial if we are to properly serve the educational and therapeutic needs of the deaf signing community. Studies of cognitive and language processing in the deaf are beginning to provide a foundation for development of a functional model of ASL processing. An equally important goal is to understand the neural systems that underlie these behavioral functions. This article has reviewed recent findings from studies of language abilities in adult deaf signers who have suffered brain injury and studies of functional imaging in deaf signers. These studies have begun to specify the neural machinery required for competent use of sign language. The combination of behavioral and neural imaging techniques provides the necessary groundwork for the development of a comprehensive neurocognitive model of sign language processing. Notes 1. It should be noted that the characterization of "withinsentential co-reference" as the province of syntax and "betweensentential co-reference" as the province of discourse is a simplification of the complexity of language structure. I thank an anonymous reviewer for this comment. 2. Gradient-echo echo-planar (EPI) images were obtained using a 4T whole body MR system, fitted with a removable z- axis head gradient coil. Eight para-sagittal images, positioned from the lateral surface of the brain to a depth of 40 mm, were collected (TR = 4 sec, TE = 20 mns, resolution 2.5 X 2.5 X 5 mm, 64 times-points per image). For each subject, only one hemisphere was imaged in a given session since a 20 cm diameter transmit-receive radio-frequency surface coil was used. In addition, at the beginning and end of each run, high resolution gradient-echo GRASS reference scans corresponding to the EPI

13 Studies of Neural Processing in Deaf Signers 47 slices were obtained (TR = 20 ms, TE = 10 ms, flip angle 15 degrees). These reference scans give good gray/white/csf contrast and permitted identification of activated areas in relation to sulcal anatomy. 3. Data were analyzed by performing correlations, voxel by voxel, between the MR signal time series and a sine wave that modeled the alternations between sentences and nonwords/nonsigns. This correlation map was thresholded to retain only voxels whose activity over time correlated with the stimulus alternation (r 2: 0.5; p = ). For each subject, anatomical regions were delineated according to sulcal anatomy (Rademacher ct al., 1992); active voxels were classified according to these anatomical regions. Averages across subjects were claculated for each of these anatomical regions. Activation measurements were made on the following two variables for each region and dataset: (a) the mean percent change of the activation for active voxels in a region, and (b) the mean spatial extent of the activation in the region. For each population and hemisphere, activation within a region was assessed by MANOVA (BMDP statistical software) on these variables (see Bavelier et al., 1997, in press, for further discussion of these procedures). Unless otherwise noted, significance levels are specified at/> <.025. References Awh, E., Jonides, J., Smith, E. E., Schumacher, E. H., Koeppe, R. A., & Katz, S. (1996). Dissociation of storage and rehearsal in working memory: PET evidence. Psychological Science, 7, Bates, E., Wulfeck, B., & MacWhinney, B. (1991). Crosslinguistic studies in aphasia: An overview. Brain and Language, 41(1), Bavelier, D, Corina, D. P., Clark, V. P., Jezzard, P., Padmanhaban, A., Prinster, A., Kami, A., Rauschecker, J., Turner, R., & Neville, H. (in press). Sentence reading: An fmri study at 4T. Journal of Cognitive Neuroscience. Binder, J. R. (1995). Functional magnetic resonance imaging of language cortex. InternationalJournal of Imaging Systems and Technology, 6, 2-3. Blumstein, S. E. (1973). A phonological investigation of aphasic speech. The Hague: Mouton. Brentari, D, Poizner, H., & Kegl, J. (1995). Aphasic and Parkinsonian signing: Differences in phonological disruption. Brain and Language, 48(\), Brownell, H. H., Simpson, T. L., Bihrle, A. M., Potter, H. H., et al. (1990). Appreciation of metaphoric alternative word meanings by left and right brain-damaged patients. Neuropsychologia, 28(4), ChiareUo, C, Knight, R., & Mandel, M. (1982). Aphasia in a prelingusdly deaf woman. Brain, 105, Cohen, J. D, Forman, S. D., Braver, T. S., Casey, B. J., Servan- Schreiber, D., & Noll, DC (1994). Activation of the prefrontal cortex in a nonspatial working memory task with functional MRI. Human Brain Mapping, 1, Cohen, J. D, Noll, DC, & Schneider, W. (1993). Functional magnetic resonance imaging: Overview and methods for psychological research. 22nd Annual Meeting of the Society for Computers in Psychology: Symposium on brain imaging methodology for the study of cognitive psychology. Behavior Research Methods, Instruments, and Computers, 25(2), Corina, D. P. (1989). Recognition of affective and noncanonical linguistic facial expressions in hearing and deaf subjects. Brain and Cognition, 9, Corina, D. P. (1990). Handshape assimilations in hierarchical phonological representations. Sign-language research: Theoretical issues. Washington, DC: Gallaudet University Press. Corina, D. P. (1997). Sign language aphasia. In P. Coppens (Ed.), Aphasia in atypical populations. Hillsdale, NJ: Lawrence Erlbaum Corina, D. P., Bellugi, U, Kritchevsky, M., O'Grady-Batch, L., & Norman, F. (1990). Spatial relations in signed versus spoken language: Clues to right parietal functions. Paper presented at the Academy of Aphasia; Baltimore, Maryland. Corina, D. P., Kritchevsky, M., & Bellugi, U. (1992). Linguistic permeability of unilateral neglect: Evidence from American Sign Language. Proceedings of the 14th Annual Conference of the Cognitive Science Society (pp ). Hillsdale, NJ: Lawrence Erlbaum. Corina, D. P., Kritchevsky, M., & Bellugi, U. (19%). Visual language processing and unilateral neglect: Evidence from American Sign Language. Cognitive Neuropsychology, 13(1), Corina, D. P., Poizner, H. P., Feinberg, T, Dowd, D, & O'Grady, L. (1992). Dissociation between linguistic and nonlinguistic gestural systems: A case for compositionality. Brain and Language, 43, Corina, D. P., & Sandier, W. (1993). On the nature of phonological structure in sign language. Phonology, 10(2). Delis, DC, Robertson, L. C, & Balliet, R. (1983). The breakdown and rehabilitation of visuospatial dysfunction in braininjured patients. International Rehabilitation Medicine, 5(3), Emmorey, K. (1993). Processing a dynamic visual-spatial language: psycholinguistic studies of American Sign Language: Journal of Psycholinguistic Research, 22(2), Emmorey, K., & Corina, D. P. (1990). Lexical recognition in sign language: Effects of phonetic structure and morphology. Perceptual and Motor Skills, 7/(3), Emmorey, K., Corina, D. P., & Bellugi, U. (1995). Differential processing of topographic and referential functions of space. In K. Emmorey & J. Reilly (Eds.), Language, gesture and space (pp ). Hillsdale, NJ: Lawrence Erlbaum. Emmorey, K., Kosslyn, S. M., & Bellugi, U. (1993). Visual imagery and visual-spatial language: Enhanced imagery abilities in deaf and hearing ASL signers. Cognition, 46(2), Fiez, J. A., Raife, E. A., Balota, D. A., Schwarz, J. P., Raichle, M. E., & Petersen, S. E. (19%). A positron emission tomography study of the short-term maintenance of verbal information. Journal of Neuroscience, 16, Fox, P. T, Raichle, M. E., Mintun, M. A., & Dence, C (1988). Nonoxidative glucose consumption during focal physiologic neural activity. Science, 241(48(4), Goodglass, H. (1993). Understanding aphasia. San Diego: Academic Press.

14 48 Journal of Deaf Studies and Deaf Education 3:1 Winter 1998 Goodglass, H., & Kaplan, E. (1972). The assessment of aphasia and related disorders. Philadelphia: Lea & Febiger. Habib, M., Demonet, J. E, & Frackowiak, R. (1996). Cognitive neuroanatomy of language: Contribution of functional cerebral imaging. Revue Neurologiquc, 152(4), Hanson, V. L. (1990). Recall of order information by deaf signers: Phonetic coding in temporal order recall. Memory ( Cognition, 75(6), Hanson, V. L., & Lichtenstein, E. H. (1990). Short-term memory coding of deaf signers: The primary language coding hypothesis reconsidered. Cognitive Psychology, 22(2), Hickok, G., Bellugi, U, & Klima, E. S. (1996). The neurobiology of sign language and its implications for the neural basis of language. Nature, 381(6584), Jackson, J. H. (1878). Brain 1, 64. Jackson, J. H. (1932). Selected writings of Hughlings Jackson. Vol 2. Ed. J. Taylor. London. Kaplan, J. A., BrowneU, H. H., Jacobs, J. R., & Gardner, H. (1990). The effects of right hemisphere damage on the pragmatic interpretation of conversational remarks. Brain ( Language, 38(2), Kegl, J., & Poizner, H. (1991). The interplay between linguistic and spatial processing in a right-lesioned signer. Journal of Clinical and Experimental Neuropsychology, 13, Leischner, A. (1943). Die "aphasie" der taubstummen. Archivfiir Psychiatry und Nerenkr, 115, Levine, D. N., & Sweet, E. (1982). The neurological basis of Broca's aphasia and its implications for the cerebral control of speech. In M. Arbib, D. Caplan, & J. C Marshall (Eds.), Neural models of language processes (pp ). San Diego: Academic Press. Linebarger, M. C, Schwartz, M. F, & Saffron, E. M. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13(3), Lounasmaa, O. V., Hamalainen, M., Hari, R., & Salmelin, R. (1996). Information processing in the human brain: Magnetoencephalographic approach. Proceedings of the National Academy of Sciences of the United States of America, 9J(17), Marschark, M. (1993a). Psychological development of deafchildren. New York: Oxford University Press. Marschark, M. (1993b). Psychological perspectives on deafness. Hillsdale, NJ: Lawrence Erlbaum. Mayberry, R. I., & Eichen, E. B. (1991). The long-lasting advantage of learning sign language in childhood: Another look at the critical period for language acquisition. Journal of Memory & Language, 30(4) Mayberry, R. I., & Fischer, S. D. (1989). Looking through phonological shape to lexical meaning: The bottleneck of nonnative sign language processing. Memory and Cognition, 17(6), Menn, L., & Obler, L. K. (1990). Agrammatic aphasia. Amsterdam: Benjamins. Merzenich, M. M., Jenkins, W. M., Johnston, P., Schreiner, C, Miller, S. L., & Tallal, P. (1996). Temporal processing deficits of language-learning impaired children ameliorated by training. Science, 27/(5245), Neville, H. J., Bavelier, D., Corina, D. P., Rauschecker, J., Kami A., Lalwani A., Braun. A., dark, V., Jezzard, P., & Turner, R. (in press). Cerebral organization for language in deaf and hearing subjects: Biological constraints and effects of experience. Proceedings of the National Academy of Sciences. Neville, H. J., Coffey, S. A., Lawson, D. S., Fischer, A., et al. (1997). Neural systems mediating American Sign Language: Effects of sensory experience and age of acquisition. Brain and Language, 57(3), Neville, H. J., Mills, D. L., & Lawson, D. S. (1992). Fractionating language: Different neural subsystems with different sensitive periods. Cerebral Cortex, 2(1), Osterhout, L. (1994). Event-related brain potentials as tools for comprehending sentence comprehension. In C. Clifton, L. Frazier, & K. Rayner (Eds.), Perspectives on sentence Processing. Hillsdale, NJ: Lawrence Erlbaum. Parasnis, I., & Samar, V. J. (1985). Parafoveal attention in congenitally deaf and hearing young adults. Brain and Cognition, 4(3), Perlmutter, D. (1993). Sonority and syllable structure in American Sign Language. In G. Coulter (Ed.,), Phonetics and phonology (pp ). San Diego: Academic Press. Petersen, S. E., Fox, P. T, Posner, M. I., Mintun, M., Raichle, M. E. (1988). Positron emission tomographic studies of the cortical anatomy of single-word processing. Nature, 331 (6157), Poizner, H., & Kegl, J. (1992). Neural basis of language and motor behavior. Perspectives from American Sign Language. Aphasiology6(3), Poizner, H., Klima, E. S., & Bellugi, U (1987). What the hands reveal about the brain. Cambridge: MIT Press. Rademacher, J., Caviness, V. S. J., Steinmetz, H., & Galaburda, A. M. (1993). Topographical variation of the human primary cortices: implications for neuroimaging, brain mapping, and neurobiology. Cerebral Cortex, 3(4), Rehak, A., Kaplan, J. A., Weylman, S. T, Kelly, B., BrowneU, H. H., & Gardner, H. (1992). Story processing in righthemisphere brain-damaged patients. Bruin and Language, 42(3), Soderfeldt, B. (1994). Signing in the brain. Acta Universitatius Upsaliensis, Uppsala, Sweden. Soderfeldt, B., Ronnberg, J., & Risberg, J. (1994). Regional cerebral blood flow in sign language users. Brain and Language, 46, Tallal, P., Miller, S. L., Bedi, G., Byma, G, Wang, X., Nagarajan, S. S., Schreiner, C, Jenkins, W. M., & Merzenich, M. M. (19%). Language comprehension in languagelearning impaired children improved with acoustically modified speech. Science, 271(5245), Zatorre, R. J., Meyer, E., C-jedde, A., & Evans, A. C (1996). PET studies of phonetic processing of speech: Review, replication, and reanalysis. Cerebral Cortex, 6(1), Zein, G, Say, K., Bellugi, U, Corina, D, & Reilly, J. S. (October 1993). The role of the right hemisphere for extra-syntactic aspects ofasl. Paper presented at the Academy of Aphasia; Tucson, Arizona.