Nonverbatim Captioning in Dutch Television Programs: A Text Linguistic Approach

Transcription

1 Nonverbatim Captioning in Dutch Television Programs: A Text Linguistic Approach Joost Schilperoord Tilburg University Vanja de Groot Nic van Son Viataal In the Netherlands, as in most other European countries, closed captions for the deaf summarize texts rather than render them verbatim. Caption editors argue that in this way television viewers have enough time to both read the text and watch the program. They also claim that the meaning of the original message is properly conveyed. However, many deaf people demand verbatim subtitles so that they have full access to all original information. They claim that vital information is withheld from them as a result of the summarizing process. Linguistic research was conducted in order (a) to identify the type of information that is left out of captioned texts and (b) to determine the effects of nonverbatim captioning on the meaning of the text. The differences between spoken and captioned texts were analyzed on the basis of on a model of coherence relations in discourse. One prominent finding is that summarizing affects coherence relations, making them less explicit and altering the implied meaning. The Netherlands has a long tradition of using open subtitles to translate foreign television programs and cinema movies for audiences. About 20% of all programs on Dutch public and commercial television are foreign, most of them in English. Since 1980, the year when teletext was introduced, closed captions for the deaf and hard of hearing also have been available on The research reported in this article is an elaboration of a master s thesis written by the second author, which was part of a 3-year research project into various linguistic and psycholinguistic aspects of television captioning for hard-of-hearing and deaf people (de Groot, 2001). The research was supported by grants from Viataal (Sint-Michielsgestel, the Netherlands), the Mgr. Van Overbeekstichting foundation ( s-hertogenbosch, the Netherlands), and the Stichting Niemeyer Fonds foundation (Rotterdam, the Netherlands). Correspondence should be sent to Joost Schilperoord, Tilburg University, P.O. Box 90153, 5000 LE Tilburg, the Netherlands ( j.schilperoord@uvt.nl). Dutch programs. In that year, 22 programs were captioned via teletext. Since then, there has been a gradual increase in the number of programs being captioned, but only on the public broadcasting stations (Van Son, Verboom, & Van Balkom, 1998). They now produce captions for slightly more than half of their Dutchlanguage programs, or up to 16,000 programs a year. 1 In the Netherlands, as in most European countries, open subtitles and closed captions provide a summary of what is being said in the program. Caption editors apply the so-called 6-s rule when producing captions: two full lines of text (64 characters) are projected onto the screen for a maximum period of 6 seconds; shorter captions are projected for a proportionally shorter period (Gielen & d Ydewalle, 1989). Using this rule, captions are presented at a rate of about words per minute, and editors argue that this gives the viewer ample time to both read the text and watch the program. For normal readers, who read at an average rate of 250 words per minute (Gibson & Levin, 1975), this is undoubtedly true, but it does not necessarily hold for deaf readers too. In the early days of teletext captioning, the text was heavily edited for the deaf audience and adapted to the expected level of language competence of prelingually deaf television viewers. The same approach was taken in the United States (Braverman, 1981; Braverman & Hertzog, 1980). In the early 1990s, however, this policy was changed owing to pressure from deaf people themselves. In the United States, this has led to a complete switch towards verbatim captioning Ó The Author Published by Oxford University Press. All rights reserved. doi: /deafed/eni038 For Permissions, please journals.permissions@oupjournals.org Advance Access publication on July 21, 2005

2 (Robson, 1997). 2 In the Netherlands and other European countries, heavy editing has been abandoned, but the regular practice of summarizing is still applied. For years, representatives of deaf people in the Netherlands have argued strongly in favor of verbatim subtitling, claiming that vital information is withheld from them as a result of the summarizing process. In their turn, caption editors claim that the process of summarizing does not impair the general message (the meaning) of the program. Neither party, however, makes clear exactly what information is left out and what effect this deletion has on the deaf reader in terms of both understanding and enjoying the program. The present research was aimed at providing this information, in both a qualitative and a quantitative sense. The goals of this research were, first, to describe the differences between caption texts and the original source texts and, second, to shed light on how the practice of captioning might affect the meaning of the original text. It was our aim to determine whether verbatim or nonverbatim captions are more desirable. A description of the differences between actual discourse and nonverbatim captions should help subsequent researchers to better specify factors that affect textual meaning and the comprehension processes of deaf readers. From these considerations follow the two main questions addressed in the study: 1. What is left out in the process of captioning texts? 2. What effect does summarizing have on the meaning of the original text? Whereas research on captioning normally addresses the deaf viewers general language and reading skills and message comprehension (see Braverman & Hertzog, 1980; Jelinek Lewis & Jackson, 2001; Verfaille & d Ydewalle, 1987), we approached the issue from a text linguistic perspective. A central topic in the field of text linguistics is textual coherence: the idea that successive utterances constitute textual units because they are somehow related to each other. Because coherence is a vital part of the text s overall meaning, understanding a text involves understanding the way its constituent parts are related to each other. Hence, text linguistics is concerned with Nonverbatim Captioning in Dutch Television Programs 403 (a) characterizing the various forms of coherence between successive utterances and (b) classifying the various linguistic devices that occur in texts in order to mark coherence relations. In comparing caption texts with the source texts (the original spoken words of the program), we take the latter as a base, and we analyze how the captions relate to that base in terms of textual coherence and the way it is linguistically marked. In the next section, we describe and justify this theoretical framework, and we demonstrate its potential to characterize captioning practices. The first research question what is left out in the process of captioning texts? is addressed by providing the reader with statistical information about coherence relations and the way in which their linguistic marking is treated by caption editors. The second question, concerning the effects of summarizing, is addressed by providing the reader with text analytical information about captioning practices. We will discuss various cases of captioning practices. The analyses also focus on how the practice of captioning affects the meaning of the source texts. The results give rise to the postulation of four underlying captioning principles. Text Linguistics Text Meaning The starting point of this study was the claim of Dutch caption editors that their captions leave the meaning of the source text unaffected. Examining to what extent this claim can be maintained requires us to be as precise as possible about what is meant by the meaning of a text. Consider the following utterance. (1) Is that a hummingbird in that tree? Having understood this utterance means that we have built a mental representation of it, which, at the very least, involves a hummingbird and a tree. Building this representation is guided by both our knowledge of the meaning of the words in (1) and our knowledge of the syntactic relations between its constituent parts. Let us now see what an understanding of two successive utterances might entail. Consider (2).

3 404 Journal of Deaf Studies and Deaf Education 10:4 Fall 2005 (2) a. Jim was a heavy drinker. b. He died at the age of 27. In order to arrive at the full meaning of (2), the reader has to work out the meaning of each utterance, but, in addition, he has to infer that the two utterances are related to each other in a certain way: the two utterances show coherence. A central issue in text linguistics is how readers make such inferences. Two notions that are central in text linguistics are vital to understanding this: referential coherence and relational coherence. With respect to (2), referential coherence refers to the proper name Jim in (2)a and the pronominal expression he in (2)b. By processing the pronominal expression, the reader interprets the incoming information ( died at the age of 27 ) as a predication about the person who was already introduced in the text (i.e., Jim ). In other words, both Jim and he are understood to refer to the same person, which accounts for the fact that we understand the two utterances as being related to each other (i.e., they form a text). This is referential coherence. Referential coherence thus informs us that two (or more) utterances are related. To arrive at a full understanding of (2), however, we also have to understand what kind of relation holds between the utterances. In the case of (2), the reader will, in all likelihood, relate Jim s drinking habits to his (untimely) death in a causal way; that is, he infers that Jim died relatively young because he drank too much. If our mental representation of (2) involves the idea that (2)a expresses a cause and (2)b its consequence, we have understood (at least part of) the text s relational coherence. In this study, we are concerned only with relational coherence. Although at first glance it may seem so, referential coherence and relational coherence are not the same. Consider (3). (3) Last month, the weather was very hot. The supermarkets sold huge amounts of ice cream. Note that the two sentences in (3) do not contain even a single coreferring element (as was the case in (2)). Hence, there is no referential coherence in (3). Nevertheless, the two sentences are clearly connected, and the fact that we understand them as such is fully accounted for by relational coherence: the supermarkets sold huge amounts of ice cream because it was so hot. Relational coherence can be expressed more formally as follows (see 4). (4) [...u1...]r[...u2...] There are two utterances, u1 and u2, between which a certain coherence relation R holds. We can, therefore, define relational coherence, the element R, as that specific part of the meaning of two (or more) utterances that cannot be described in terms of the meaning of the utterances in isolation (see especially Sanders, 1992, 2 ff.). In sum, coherence relations are a vital, if not defining, part of the notion of text meaning. In text linguistics, two questions are central with regard to this issue: (a) how can coherence relations be described? and (b) what means do languages have to mark coherence relations explicitly in texts? Question (a) pertains to the nature of coherence relations, whereas question (b) refers to the linguistic means that speakers/writers use in order to signal coherence relations, for example, the use of the connective because to signal a causal relation between two utterances. Both questions will be discussed in the next two sections. The Nature of Coherence Relations What kinds of coherence relations can be present in texts? The text linguistic literature provides much information on this issue (see for example, Mann & Thompson, 1988; Pander Maat, 2002), but a particularly useful account is offered by Sanders, Spooren, and Noordman (1992). They propose that relational coherence should be categorized according to a set of four so-called primitives, three of which are relevant to our issue. These three are listed in (5). (5) causal versus additive positive versus negative semantic versus pragmatic These three primitives allow for eight types of relations, but we shall discuss them by providing some prototypical examples. The first primitive categorizes

4 coherence relations as either causal or additive. We have already come across coherence relations that could be labeled causal. In general, a causal relation between u1 and u2 exists if it is possible to interpret either u1 or u2 as an implication of the other. Selling huge amounts of ice cream can be interpreted as an implication of hot weather, just as dying young can be interpreted as an implication of excessive drinking. Such a causal implication can, however, not be deduced from (6). (6) While John was sleeping, Mary took care of the dishes. Text (6) is said to express a temporal relation between two events, with the two events occurring simultaneously. Neither event can be interpreted as an implication of the other, and, therefore, the coherence relation between the two events is additive rather than causal. Other additive relations are lists, exceptions, and oppositions. Examples are shown in (7). (7) a. In Amsterdam, four people died in a car crash. In Rotterdam, two people were severely injured when their motorbike hit a tree (list). b. During the first quarter of this year, all major banks made profits, except for the Rabobank (exception). c. John bought a new suit, but Paul bought an entire wardrobe (opposition). The second primitive categorizes coherence relations as either positive or negative. Examples (7)b and (7)c are not only additive but also negative coherence relations, whereas (7)a is additive and positive. Acausal relation is negative if one of the two utterances denies an implication that can be deduced from the other one. An example is given in (8). (8) Although the weather was fine, we didn t go to the beach. Negative causal relations often express a notion of denied expectations : under normal circumstances, pleasant weather may lead us to go to the beach, but in this case, it did not. Finally, the third primitive categorizes coherence relations according to the semantic versus pragmatic distinction. Consider once again the example shown in (2), repeated here as (9). Nonverbatim Captioning in Dutch Television Programs 405 (9) Jim was a heavy drinker. He died at the age of 27. This text is coherent because it is part of our world knowledge that heavy drinking may cause an early death. Hence, the text describes a state of affairs and an event that are causally related. Such causal relations are, therefore, labeled semantic. However, using a causal relation may serve a different goal. Consider the example in (10). (10) John should be home right now. It s already 10 PM. The interpretation seems obvious: the speaker does not know whether John has already arrived home, but he expresses his belief that this is the case because it is already 10 p.m. In this case, a (causal) relation is labeled pragmatic. Pragmatic coherence relations often appear in discourse contexts where a speaker is arguing some point or where he/she is explaining something. We have arrived at a plausible answer to the question that was raised at the beginning of this section: how can we characterize the meaning of a text? A necessary condition for having understood the meaning of a[...u1...]r[...u2...] complex is that we have built a mental representation of both u1 and u2 and that we have been able to infer the (implied) coherence relation R that holds between u1 and u2. We should be able to characterize R in terms of the primitives additive versus causal, positive versus negative, and semantic versus pragmatic. And so, the relation between (2)a and (2)b may be characterized as R (U1,U2) / fcausal, positive, semanticg, whereas the relation between the two utterances in (10) can be characterized as R (U1,U2) / fcausal, positive, pragmaticg. We now turn our attention to the ways coherence relations can be linguistically marked. Linguistic Markers of Coherence Relations Text (2)/(9) contains two utterances that are causally connected: Jim died young because he drank too much. Note, however, that in this text, the causal relation has to be inferred from the meanings of the two parts. Our knowledge of the world tells us that heavy drinking is bad for one s health and that an unhealthy lifestyle may cause an early death. Indeed, world knowledge is crucially important in understanding the meaning of texts.

5 406 Journal of Deaf Studies and Deaf Education 10:4 Fall 2005 Languages contain many devices that speakers/ writers may use in order to explicitly mark coherence relations. For example, the coherence relation in (2)/ (9) may be marked using a connective like because: Jim died at the age of 27 because he was a heavy drinker. Linguistic markings of coherence can thus be considered means to guide the comprehension process. Languages contain various types of connectives, for example, conjunctions ( because, but ), descriptive constituents ( the reason for this is... ), serial markers ( first, second, moreover, in addition ), explicitly stated questions ( what may be the reason for this? ), punctuation marks (semicolons, for instance), and lexical signals ( In consequence,... ). The use of markers of coherence relations is sometimes crucial to a proper understanding of a text. Although text (2)/(9) can do without a connective, many contexts more or less require a writer to explicitly express the implied coherence relation. Consider (11). (11) Last month the weather was very hot. Many people suffered from insomnia. In this case, knowledge of the world will likely fall short when it comes to inferring the implied coherence relation. It may be that hot weather facilitates insomnia, but, by the same token, it is possible that hot weather does precisely the opposite. Hence, the implied causal coherence relation may be positive or negative. A writer can avoid a misunderstanding by using a proper connective, as shown in (12). (12) a. Despite last month s hot weather, many people suffered from insomnia. b. Because of last month s hot weather, many people suffered from insomnia. Hence, connectives may contribute to constructing mental representations of a text s meaning. Previous research shows that connectives play an important facilitating role in text processing and text understanding: they facilitate the online processing of coherence relations (Cozijn, 2000; Sanders, 1992; Spooren, Hoeken, & Mulder, 1998), they facilitate text understanding if the text is regarded as difficult by readers (Admiraal & Schilperoord, 2001; Lorch & Lorch, 1996), and they facilitate text understanding for readers with low reading proficiency (Romein, 2003). This latter result seems especially relevant to our research because, generally speaking, deaf readers have suboptimal reading skills. In sum, relational coherence is of crucial importance in assessing the notion of text meaning. Languages have means to explicitly mark coherence relations, and these markings may affect the process of text understanding. This enables us to evaluate the central claim of Dutch caption editors: caption texts can be said to leave the meaning of the source text unaffected if, and only if, a deaf reader is able to build a mental representation that is essentially similar to the one underlying the original text, that is, it should involve the meanings of the utterances and the coherence relation that ties them together. In order to show the potential of our approach, we apply this criterion to an actual instance of captioning. Applying the Criterion The (slightly adjusted) excerpts in (13) are taken from one of the television programs that we analyze in this study: a satirical soap series. The text originates from the series narrator, who introduces this week s episode by summarizing last week s events. Excerpt (13)a shows the original and (13)b the caption text. (13) a. u1 Patty sends her doctor Willemien to the overworked Max m which doesn t prevent him u2 from suspecting that Patty thinks he is lazy. b. u1 Patty sends her doctor Willemien to the overworked Max. u2 He suspects that Patty thinks he is lazy. If we have correctly understood (13)a (the source text), our mental representation reflects our understanding of the two utterances u1 and u2, together with the marked (m) coherence relation holding between them. This aspect R of the meaning of (13) is shown in (14). (14) R (U1,U2) / fcausal, negative, pragmaticg Our interpretation of R may be in need of some clarification. The relation R between u1 and u2 is interpreted as negative owing to the so-called descriptive constituent which doesn t prevent him. This

6 constituent explicitly marks the implied coherence relation (hence the m ) as being both negative and causal (or adversative to use the more common term). This implies that under normal circumstances, the event of Patty sending Willemien to Max should have led him not to suspect that Patty thinks he is lazy. But in this particular case, it did just that. Moreover, the causal relation is pragmatic because it relates an event (u1) with a particular conclusion on the part of the narrator. Note that the marking expression which doesn t prevent him is deleted from the caption text. One immediate consequence is that, although a deaf reader will be able to construct the proper mental representation of both u1 and u2, he/she will, most probably, fail to build the intended mental representation of R. In fact, based on the available evidence in the caption text, R will, in all likelihood, be represented as (15). Note that each primitive receives a different value. (15) R (U1,U2) / fadditive, positive, semanticg This brief case analysis shows the potential of our relational coherence approach to the problem of caption texts, and it also demonstrates its ability to detect changes in the meaning of the source text. Note that the analysis focused on the removal of the relational marker which doesn t prevent him. If a coherence relation is explicitly marked in the source text, this provides us with a maximally objective way of determining the implied coherence relation. For this reason, the analyses that were conducted reported in this study focus on those coherence relations in the source text that were indeed explicitly marked, and we examine how caption editors handle these markings. We are now in a position to rephrase the research questions much more precisely. 1. How are coherence relations and their linguistic markers treated in the process of captioning? 2. How does this practice affect the meaning of the source text? These questions were addressed in an analysis of three television programs (see next section). Nonverbatim Captioning in Dutch Television Programs 407 Method Materials Three Dutch television programs were analyzed. We selected programs that show typical captioning practices. We decided not to analyze live programs (which are also captioned in some cases) in order to circumvent the possibility that caption characteristics resulted from time pressures on the caption editor. Two of the selected programs were comedies: one episode of a sitcom series called Oppassen! (literally, Taking care ) and an episode of a satirical soap series called Hertenkamp (literally, Deer park ). Parts of comparable size were selected from both programs. Because the speech rates of the programs differ substantially (see Table 1), we determined the sizes using the number of words rather than minutes. The analyzed parts were roughly half of the original programs. The third program was a documentary program called Zembla. A shorter part of this program was analyzed; our main aim was to get an idea of how programs of a different nature (informative rather than entertaining) are captioned. Table 1 shows some statistics of the three programs, comparing the number of spoken and captioned words per minute. All three programs were broadcast in the year 2000 and videotaped. The tape and the filed captioned texts were made available by the caption service of the Dutch public broadcasting company, NOS (Nederlandse Omroep Stichting). The spoken texts were transcribed verbatim. Procedure The research method consisted of three steps: (a) all marked coherence relations in the original texts were tracked down and (b) all relations were categorized Table 1 Words spoken per minute and captioned in three Dutch television programs Program Words/minute spoken Words/minute % captioned a Captioned Hertenkamp Oppassen! Zembla a The upper limit of words per minute in captions is when using the 6-s rule (see Introduction section).

7 408 Journal of Deaf Studies and Deaf Education 10:4 Fall 2005 according to the classification of coherence relations based on the three primitives described in the previous section. Coherence relations were thus labeled causal or additive, positive or negative, and semantic or pragmatic; (c) all connectives were classified as conjunctions, descriptive constituents, serial markers, explicitly stated questions, punctuation marks, or lexical signals. Model of Analysis Each coherence relation in the source texts was compared to its counterpart in the caption text. Differences were classified using an analytical model that will be described in this section. The model is based on Sanders (1992, p. 14) definition of coherence relations: an aspect of the meaning of two or more segments that cannot be described in terms of the meaning of the segments in isolation. As mentioned above, this definition can be represented as R (U1,U2), with U1 being the meaning of utterance 1, U2 the meaning of utterance 2, and R the coherence relation. By augmenting the conceptual representation of a coherence relation, it can be connected it to its linguistic counterpart. The linguistic representation may or may not include a coherence marker. Because we were only interested in cases of explicit marking, the augmented model can be depicted as in (16), with u1 and u2 being the linguistic surface forms of U1 and U2 and m the marker of R. (16) (u1 m u2) The analytical goal of this study was to find out how instances of (16) in the source texts are handled by caption editors. In theory, there are eight possibilities for captioning (u1 m u2) complexes. These are presented in Table 2 (examples of all caption patterns observed will be provided in the next section). If an element of the source text (u1, u2, or m) is excluded, it is given between brackets. Not all of these possibilities make sense, for example, pattern 7 ([u1] m [u2]), which indicates that both u1 and u2 are removed from the source text, while the element m remains part of the caption text. However, as the data in the results section indicate, on a few occasions this captioning operation does occur (albeit exceptionally), especially Table 2 Possible patterns in captions with regard to marked coherence relations Type Captions 1 u1 m u2 2 u1 [m] u2 3 [u1] m u2 4 [u1 m] u2 5 u1 m [u2] 6 u1 [m u2] 7 [u1] m [u2] 8 [u1 m u2] in the case of additive relations involving more than two members. Results The first part of this section presents some quantitative analyses, which provide statistical information on the frequencies of the types of captioning operations that were distinguished in the previous section. The second part reports a number of qualitative analyses, which will illustrate how the treatment of coherence relations and their markers affects the text meaning. Frequencies Table 3 shows the frequencies of the various types of captioning operations in relation to the nature of the coherence relations. As no differences were found between the programs, all observations are taken together. In total, 185 marked coherence relations were found in the corpus. Because no instances of caption types 3 and 8 were found, these are excluded from the table. Moreover, we found no examples in the corpus of fnegative, semantic, causalg relations or fadditive, positive/negative, pragmaticg relations. Table 3 shows that out of the total of 185 coherence relations, 77 (41.6%) fully survived captioning. The remaining 108 relations (58.4%) were in some way affected by captioning. By far the most common type of captioning operation concerns deleting only the coherence marker: u1 [m] u2. This occurred 65 times (35.1%). Other relatively frequent types involve deleting the relational marker together with one of the two connected utterances: [u1 m] u2 and u1 [m u2]. These

8 Table 3 Operation type Frequencies of types of captions in relation to type of coherence relation Causal, positive, semantic Causal, positive, pragmatic Nonverbatim Captioning in Dutch Television Programs 409 Causal, negative, pragmatic Additive, semantic Totals (%) 1. u1 m u u1 [m] u [u1 m] u u1 m [u2] u1 [m u2] [u1] m [u2] Totals (%) 50 (27.0) 69 (37.3) 48 (25.9) 18 (9.7) 185 make up almost 20% of all affected relations (36 instances). Together, these three types of deletions, involving the relation marker, account for 93% of all deletions. This seems to warrant the conclusion that the practice of captioning does indeed show a robust tendency to influence coherence relations and their linguistic markings. One might wonder whether a relationship exists between caption patterns, on the one hand, and relation types, on the other. In other words, is it true that particular types of relations are more prone to being affected by captioning than other types? If this were the case, this tendency would surface as a significant interaction between the variables caption types and relation types. A common way of testing whether such an interaction exists is to run a chisquare test on the frequencies reported in Table 3. The test, however, revealed no such interaction (v 2 (15) ¼ 14.96, p ¼.45). This picture did not change when we left out all cases of nonaffected coherence relations (v 2 (12) ¼ 12.14, p ¼.44). Table 4 shows how various types of markers of coherence relations are treated by caption editors. The distribution of types of markers across the 185 cases speaks for itself. We investigated whether caption editors show a preference for maintaining or deleting certain types of markers. This tendency would show itself as a significant relationship between the variables types of markers, on the one hand, and types of operation, on the other hand. However, the chisquare test revealed no such tendency (v 2 (5) ¼ 8.45, p ¼.13). The data further show that more than half of all markers were in fact deleted (111 vs. 84). The difference between deleted and maintained coherence markers is significant (Wilcoxon signed rank test ¼ 8.85, p,.001): markers are more often deleted than they are maintained. Qualitative Analyses Several examples of captioning practice will be discussed in this section. The main goal is to explore what effects the practice of captioning has on the meaning of the source texts. However, we first explore the possible reasons behind the practice of captioning. The starting point for the editor is simply reducing the number of words, which is dictated by the demands of the 6-s rule (see the Introduction section and Table 1). In choosing the words, however, the caption editors seem to be led by a number of principles. As mentioned above, the results of our analyses will lead to the postulation of four hypothetical underlying captioning principles, which seem to guide caption editors practices. Whether or not caption editors actually apply these principles in a conscious way is a matter still open to research. Table 4 Frequencies of types of coherence relation markers in relation to type of operation: deleting (operation types 2, 3 and 5) or maintaining (operation types 1, 4 and 6) Type of marker Operation Deleted Maintained Totals (%) Conjunctions (67.0) Lexical signals (8.1) Serial markers (6.5) Punctuation marks (5.4) Explicit questions (7.0) Lexical utterances (5.9) Totals (%) 101 (54.6) 84 (45.4) 185 (100)

9 410 Journal of Deaf Studies and Deaf Education 10:4 Fall 2005 Below are two examples demonstrating captioning operations type: type 4 ([u1 m] u2, first utterance left out, together with marker) and type 2 (u1 [m] u2, only the marker is left out). They are presented in (17)a and (17)b, respectively. The analysis of both cases focuses on one possible reason for caption editors to delete markers of coherence relations. (17) a. Source: En omdat mevrouw Meijer er nu niet is, bent u nu de akela. And because Mrs. Meijer isn t here now, you re the akela now. Caption: U bent nu de akela. Now you re the akela. b. Source: Nou, het verbaast me gewoon, omdat ik jou eigenlijk ken Well, I am just surprised, because I just know you Caption: Het verbaast me gewoon. Ik ken jou I am just surprised. I know you Example (17)a shows a causal coherence relation, which is also positive and semantic. Mrs. Meijer isn t here (u1) expresses a cause for a certain consequence you re the akela now (u2). The causal relation is marked by the conjunction because. In the caption text both the first expression and the relational marker are deleted. Example (17)b shows a relation that is causal, positive, and pragmatic. In this text, the second utterance Because I just know you serves to justify the very act of expressing amazement. The causal marker omdat (because) is deleted from the source. As shown in the previous section, the two patterns illustrated here are very common. Take a close look at the second utterance in both examples (see 18). (18) a. bent u nu de akela / U bent nu de akela are you now the akela / You are now the akela b. omdat ik jou eigenlijk ken / ik ken jou because I you actually know / I know you The effect of the captioning operations bears directly on the syntax of both utterances. In (18)b, for example, deleting the marker omdat alters the word order ik-jou-ken (I you know) into ik ken jou (I know you). In (18)a, deleting both the marker and the first utterance alters the word order bent u de akela (are you the akela) into U bent de akela (you are the akela). These alterations have to do with two specific characteristics of Dutch grammar: (a) only one constituent of a sentence is allowed to appear before the main verb, and (b) Dutch subordinate sentences have a subject object verb order. In addition, as in English, the canonical word order of declarative sentences in Dutch is subject verb object. Some evidence exists to indicate that deaf readers prefer to read sentences according to this canonical word order (Slijkerman, 1988) and that they have difficulties processing sentences that deviate from the canonical order. We believe this may be the reason behind the two captioning operations illustrated here. Let us briefly explain why we believe this to be the case. In (17)a, the first constituent position is occupied by the entire first utterance (u1), and because only one constituent is allowed before the main verb, the resulting word order is verb subject object (bent u de akela). By deleting the entire utterance, the caption editor is able to restore the canonical order subject verb object ( U bent de akela ). The same thing is effected in (17)b. Because the vast majority of Dutch connectors, such as omdat, tend to subordinate the sentence in which they appear, the presence of the connector in (17)b leads to a subject object verb order ( ik jou ken ). By deleting the marker, the caption editor is once again able to restore the canonical subject verb object order. In both cases, he/she ends up with utterances showing the word order that deaf readers are believed to prefer or are likely to process best. If this line of reasoning is correct, it reveals an important captioning principle: Restore, if possible, the canonical main clause word order of Subject Verb Object. Note that, in doing so, the caption editor has established yet another characteristic: in both examples the complex grammatical structure of two syntactically connected clauses is entirely disentangled. In (17)b, deleting the marker omdat (because) has the effect that two distinct clauses remain: I am surprised and I know you. Available evidence in the literature suggests that processing two distinct clauses rather than one compound sentence is easier for deaf readers.

10 Therefore, a second principle underlying captioning practices may be disentangle complex sentences. Many cases in our corpus testify to these principles. Finally, as (17)a demonstrates, disentangling a complex sentence sometimes forces the caption editor to dismiss an entire utterance. If this is the case, then which of the two utterances is deleted? Arguably, this will be the less important one. In a cause consequence pair, such as (17)a, the consequence is more important than the cause (i.e., this utterance carries the more important information). Inspection of the corpus suggests that this indeed is the underlying rule. Consequences are more important than causes, and, likewise, solutions are more important than problems, points of view more important than arguments, and so on. In all these cases, caption editors delete the less important utterance and preserve the more important one. Hence, a third principle underlying captioning operations may be give more important utterances preference over less important ones. We now turn our attention to the central issue of meaning preservation. Do captioning operations affect the meaning of the source text? We found that they do. For example, (17)a alters the meaning of the original. But the way it changes the meaning of the source text is not so interesting, simply because an entire fragment is removed. The coherence relation is not changed; it simply no longer exists. Consider example (19), which shows a far more subtle change in meaning. (19) Source: Prikt u een vorkje mee? (...) Nee, dank u. Nee, nee, t is heel vriendelijk van u, maar ik kom alleen om de uniformen af te leveren. Speaker 1: Would you like to join the meal? (...) Speaker 2: No, thank you. No, no, it s very kind of you, but I only came here to deliver the uniforms. Caption: Prikt u een vorkje mee? (...) Nee, dank u. Ik kom alleen om de uniformen af te leveren. Speaker 1: Would you like to join the meal? (...) Speaker 2: No, thank you. I only came here to deliver the uniforms. Nonverbatim Captioning in Dutch Television Programs 411 The source text is produced by two speakers, rather than one. One speaker invites the other one for lunch, and the second one rejects the invitation. We first focus on the (deleted) conjunction maar (but). In this context, maar can be interpreted as marking an additive, negative relation of contrast (as in John is big but Jim is small ). An alternative reading is also possible: maar marks a pragmatic, causal, negative relation (or an adversative relation). Depending on the reading, different consequences follow as to the meaning of the source text. If maar marks a contrast, it is the contrast between accepting the invitation to stay for lunch and turning it down. If we interpret maar this way, the captioning operation is one of type 2 (u1 [m] u2). Because it is clear, also from the captioned text, that Speaker 2 rejects the invitation to stay for lunch, one is led to conclude that this captioning operation leaves the meaning of the source text intact. Under the alternative reading, however, maar marks an adversative relation. It connects the segment I only came here to deliver the uniforms with the preceding segment It is very kind of you. In his response to the invitation, Speaker 2 starts by expressing his appreciation for being invited for lunch. As this may lead Speaker 1 to expect that he will accept the invitation, Speaker 2 marks his refusal to stay for lunch with an adversative use of maar. Interpreted in this way, the captioning operation is one of Type 4 ([u1 m] u2) because not only the connective maar is deleted but also the utterance It is very kind of you. More importantly, under this reading, the captioning operation has changed the meaning of the source text. In the caption text, Speaker 2 simply announces that he only came to deliver the uniforms. In other words, the pragmatic stance of his utterance is severely altered. Although the reasons for this particular captioning operation can easily be connected with Principles 2 and 3 (disentangle compounds, and save the more informative segment), the analysis leaves room for a fourth captioning principle: delete conversational markers. Although conversational markers fall outside the immediate scope of the approach we take here, they do have an important feature in common with markers of coherence relations. Instead of (explicitly)

11 412 Journal of Deaf Studies and Deaf Education 10:4 Fall 2005 connecting utterances to each other, conversational markers tend to connect an utterance to the speaker or to the conversational setting. Speakers use such markers to structure the conversation and to modify the pragmatic stance of their contributions (Schilperoord & Verhagen, 1998). In addition, speakers use conversational markers to express their attitudes towards other speakers and their contributions. In this way, they establish and maintain the relationships between participants in social interaction. As the example in (19) indicates, conversational markers often occur together with markers of coherence relations. Some additional examples are shown in (20), with conversational markers given in italics. (20) a. Source: Ik vind juist dat Wiebe heel goed in z n vel zit sinds ik er ben. I think, on the contrary, that Wiebe looks very well since I ve been here. Caption: Wiebe zit juist heel goed in z n vel sinds ik er ben. Wiebe just looks very well since I ve been here. b. Source: Kijk, ik doe mijn best van je te houden zoals je bent. Look, I do my best to love you as the way you are. Caption: Ik doe mijn best van je te houden zoals je bent. I do my best to love you as the way you are. c. Source: Ja kijk, toen mijn vrouw meneer Buys aan de lijn had, toen bood hij aan om haar te helpen. Ja maar er moet toch een misverstand in het spel zijn, hoor meneer Meyer. Speaker 1: Well look, when my wife phoned Mr. Buys, then he offered to help her. Speaker 2: Yes, but after all there has to be a misunderstanding here Mr. Meyer. Caption: Toen mijn vrouw meneer Buys aan de lijn had, bood hij aan om haar te helpen. Er is een misverstand in het spel. Speaker 1: When my wife phoned Mr. Buys, he offered to help her. Speaker 2: There is a misunderstanding. In (20)c, all four caption principles are at work. In the source text, five relational markers are present (all given in italics). Only one of them returns in the caption text (the conjunction toen, when). Toen is a temporal marker. By using it, Speaker 1 marks the positive, additive coherence relation between his two utterances as one of simultaneousness. This relation and its marker are left intact. The second toen (then in translation) is deleted, probably because it repeats the first one. The caption of Speaker 2 s response lacks the conjunctions maar and toch, which, in the source text, both mark the relation between this utterance and that of Speaker 1 as one of contrast. Therefore, this aspect of meaning is deleted. In addition, the complex verb phrase in the utterance er moet een misverstand in het spel zijn (literally, there must a misunderstanding in a matter are / this must be a matter of a misunderstanding) is disentangled and altered to er is een misverstand (there is a misunderstanding). This might free the deaf reader from processing the relation between the main verb be and the subject misunderstanding, which are far apart (Principle 1). In addition, the clause yes but after all there has and the second clause are disentangled (Principle 2). Furthermore, in preserving the second clause there is a misunderstanding, the more informative utterance is saved (Principle 3). Finally, in deleting the conversational marker ja (yes), the relation of concession between the utterances of the two speakers is deleted (Principle 4). Below are examples of the most frequent captioning operation: delete the coherence marker and leave the connected segments intact. The examples demonstrate that deleting the connective alters the meaning of the coherence relation expressed in the source text. In (21), the person who is asking the question breeds his own roses. In Utterance 1, he asks for an opinion on the color of his roses. In Utterance 2, he adds to this the information that he has bred the roses himself.

12 (21) Source: Wat vinden jullie van deze kleur rood? Want die heb ik zelf gekweekt What do you guys think of this color red? Since I ve bred it myself. Caption: Wat vinden jullie van deze kleur rood? Die heb ik zelf geweekt. What do you guys think of this color red? I ve bred it myself. At first sight, the difference between source and caption text may seem trivial. The connector want (since) marks a causal relation, but it does not seem that difficult to reconstruct the intended relation if want is lacking. If we take a closer look at the example, however, there turns out to be more at stake. The reason for want to be included in the second utterance of the source text is that it marks the causal coherence relation as pragmatic. Want relates the second utterance not to the contents of the first one but to the speaker s act of asking his interlocutors for an opinion about the color red. It is possible to reconstruct his utterance as in (22). (22) What do you guys think of this color red? I ask you this question since I ve bred it myself. Note that (as one reviewer pointed out to us) want may even have the social pragmatic effect which could be interpreted also as a caution: Be careful how you respond in that I am particularly sensitive because I have a personal investment in this one. Be that as it may, the pragmatic aspect of the meaning of the source text is entirely deleted in the caption. Close inspection of various deleted connectives reveals basically the same picture. Consider (23), in which two speakers (S1 and S2) are discussing a greenhouse. (23) Source S1 Eh... dat is een kas. Eh... that is a greenhouse. S2 Ja, maar hij zit vol giftige planten! Yes, but it is full of poisonous plants! Caption S1 Dat is een kas. That is a greenhouse. S2 Die zit vol giftige planten. It is full of poisonous plants. Nonverbatim Captioning in Dutch Television Programs 413 Once again, at first sight, deleting maar (but) appears to only mildly alter the meaning of the source text. If we look more closely to its pragmatic consequences, however, we see that the deletion seriously distorts the intended meaning. We argue that the caption of the utterance of Speaker 2 has changed its original meaning from starting an argument to observing an empirical fact. As already noted, maar is often used to mark an adversative relation (causal, negative): the utterance following the connective denies an expectation or conclusion that can be based on the utterance preceding the connective. This specific kind of relation can best be acknowledged if we assume the message to be expressed by the same speaker (see 24). (24) That is a greenhouse but it is full of poisonous plants. The speaker in (24) apparently believes that the idea of a greenhouse would lead his interlocutor to exclude the presence of poisonous plants in it. The second part of the utterance denies this to be the case: contrary to what one might expect, there are in fact poisonous plants in it. In (24), however, two speakers are having a conversation. The connective maar is used by the second speaker and is put in front of his contribution. In a conversational setting, this particular use of maar has the effect of attributing an argumentative stance to the utterance. After all, this speaker denies the validity of the utterance of the first speaker. In brief: a quarrel may well be underway in (24). Speaker 2 s contribution to the conversation can thus be reconstructed as (25). (25) You are wrong in calling this a proper greenhouse, because it is full of poisonous plants. Nothing of the exchange of points of view is preserved in the caption text. Rather than questioning (part of) the validity of the assertion of the first speaker, Speaker 2 merely adds some additional information on the state of affairs. Rather than quarrelling, the two speakers now cooperate so as to sketch an adequate picture of the topic they are talking about. The hugely different messages conveyed by the source text and the caption text can be rephrased as in (26).

13 414 Journal of Deaf Studies and Deaf Education 10:4 Fall 2005 (26) Source: This is a greenhouse,. No, it cannot be, because it is full of poisonous plants Caption: This is a greenhouse,. Yes it is, and it is full of poisonous plants In terms of the coherence primitives introduced earlier, we may conclude that deleting maar from the source text alters the coherence relation from causal, negative, pragmatic to additive, positive, semantic. Not all deletions of relational markings have such implications for the text meaning. Take, for instance, excerpt (27), in which a woman is visiting a doctor. (27) Source: Ik ben zo blij dat ik even tussendoor kon. Want als werkende vrouw heb ik het toch druk! I am so glad that you could find time for me at such short notice. Because as a working woman I am so busy! Caption: Ik ben blij dat ik even tussendoor kon. Als werkende vrouw heb ik het druk! I am glad that you could find time for me at such short notice. As a working woman I am busy! In her second utterance, the woman explains why she is so glad: having a full-time job, she is always busy. This explicative coherence relation is marked with the connective want in the source text. In the caption text, want is deleted. In this case, however, as soon as the two utterances are understood, it should not be difficult to infer the implied coherence relation holding between them. Conclusions and Discussion Conclusions The aim of this study was to provide answers to the following questions: 1. How are coherence relations and their linguistic markers treated in the process of captioning? 2. How does this practice affect the meaning of the source text? With regard to the first question, our results allow us to draw the following conclusions: a. Almost 60% of all coherence relations of the type (u1 R u2) are affected in some way by captioning operations (see Table 2). b. The most common type of captioning operation is the type u1 [m] u2, i.e., the marker of the coherence relation is deleted. Other important types are [u1 m] u2 and u1 [m u2], in which the marker is removed together with one of the constituents (Table 2). c. Markers of coherence relations that are subjected to this captioning operation are almost exclusively subordinating conjunctions (Table 3). d. Captioning operations can, tentatively, be classified in terms of four underlying principles of caption editing: I. Restore, if possible, the canonical, main clause word order of Subject Verb Object II. Disentangle clause complexes III. Give more important utterances preference over less important ones IV. Delete conversation markers Principles I, II, and, in some cases, IV underlie captioning operation u1 [m] u2; principle III underlies operations [u1 m] u2 and u1 [m s2]. e. Neither are there systematic differences in the treatment of different types of coherence relations nor are there systematic differences in the treatment of various kinds of markers of coherence relations. With respect to the second question, it seems safe to conclude that captioning practices, on many occasions, distort the meaning of the source text. Coherence relations are either completely removed from the source text or their meaning is changed during the process. The latter is especially true in the case of coherence relations that operate on the pragmatic level of texts. In many cases pragmatic relations are altered into semantic relations. The practice of captioning also has the effect of changing causal relations into additive relations. Discussion The central claim of Dutch caption editors is that captioning practices do not affect the meaning of the original texts. Our results indicate that this claim can