Reduction strategies and accuracy rate in live subtitling of weather forecasts: a case study

Transcription

1 Annalisa Sandrelli Facoltà di Interpretariato e Traduzione LUSPIO Roma Reduction strategies and accuracy rate in live subtitling of weather forecasts: a case study

2 Overview: aims and objectives Analysis of live subtitles produced via respeaking (weather forecast) Aims: - to contribute to the further validation of the NER model - to classify text reduction strategies in the subtitles (training purposes) 2

3 Thanks to Stefania Monsorno MA in Interpreting and Translation, LUSPIO Rome March 2012 Swiss TXT Zurich, Gion Linder and all the respeakers who took part in the survey and interviews Juan Martínez and Pablo Romero-Fresco for helping Stefania in the application of the NER model Brigitte Stanglemeier, LUSPIO colleague who helped Stefania with the German part of the analysis 3

4 Methodology small survey at SRF on respeakers (June 2011) - 6 questionnaires (background, training, experience, required skills and competences ) - 8 interviews study on the product of respeaking, i.e. the subtitles broadcast by SRF (Swiss German state television channel) during 7 episodes of the SF Meteo weather forecast, September 2011 analysis of the subtitles - quantitative analysis (accuracy rate) - qualitative analysis (condensation strategies) Focus on respeakers and their work, to find out more about strategies and difficulties for possible training implications 4

5 Speech rate and TV genre «a subtitler listens to the original soundtrack of a live programme and respeaks it repeats it or reformulates it, depending on whether it is possible to keep up with the original speech rate, including [oral] punctuation marks to a speech recognition program which turns the recognised utterances into subtitles displayed on the screen with minimum delay» (Romero Fresco 2010: 175) original speech rate (OSR) is a key factor spontaneous speech rate generally lower than scripted Romero Fresco (2009): different speeds for different TV genres (UK data). - sports programmes average 160 wpm - news programmes average 180 wpm - interviews and weather reports average 230 wpm 5

6 How do respeakers manage? respeaking speed generally lower than OSR because respeaking includes oral punctuation (about 0-20 wpm behind) in order to produce verbatim subtitles, respeakers should speak faster than original speakers: almost impossible «as the average reading speed of adult viewers is estimated at 66% of the average speaking speed (ITC-URL 1997), each subtitle has to be reduced by approximately a third. This is the main reason why there cannot be verbatim subtitles» (De Linde & Kay 1999: 11) In other words, most respoken subtitles are edited or quasiverbatim hence, the importance of editing decisions: what strategies are used by professional respeakers? 6

7 Choice of TV genre Our selection criteria for the study. We wanted a programme with the following characteristics: - entirely live (no recorded parts) - no overlapping speech - strong link between speech and images - high OSR - specialised and restricted vocabulary (homogeneous) Therefore: no TV news (they contain recorded news items) no talk shows (several speakers, overlapping speech) no sports programmes (OSR tends to be slower) no current affairs programmes (topics are too varied) Choice: the weather forecasts 7

8 The weather forecasts: key features high OSR: e.g. BBC wpm prepared in advance, so high lexical density and few features of oral speech (not spontaneous speech) respeakers must edit the text quite heavily strong link with images: - negative aspects: deictic references («here», «over there» «in this area» ) accompanied by speakers gestures pointing to maps and pictures result in respeakers having to ADD information; need to preserve audiovisual coherence, bearing in mind subtitle delay - positive aspect: visual information (graphics and maps) means that subtitles do not need to contain redudant information difficulty: place names (if not in vocabulary) Romero Fresco (2011), Eugeni (2009) 8

9 Corpus data duration words rate (w/m) subtitles Programme 1 3 min 3 sec Programme 2 2 min 47 sec Programme 3 3 min 17 sec Programme 4 4 min 9 sec Programme 5 3 min 33 sec Programme 6 3 min 21 sec Programme 7 3 min 25 sec Average 3 min 22 sec 553 / 35 TOTAL 23 min 35 sec

10 weather programmes in Swiss German, subtitles in standard German. In this case, there is a degree of translation in transforming Swiss German into standard German. 3 respeakers at work: A, B, and C Comments small but very homogeneous corpus of data (7 episodes of the same programme) OSR much lower than what is reported in the literature. More similar to TV news OSR (English). Perhaps syllables per second is a more appropriate measurement for German? In terms of difficulty, a comparison with simultaneous interpreting (Pöchhacker 2004): comfortable OSR for simultaneous : wpm fast OSR for simultaneous: > 160 wpm Simultaneous interpreting has the translation element BUT Respeaking has (additional) oral punctuation 10

11 The NER model NER model for German (Martínez & Romero Fresco 2011) Media 4All 4 - London Accuracy = N E R x 100 N N : number of words in respoken text, including commands (punctuation) E: edition errors (omission of idea unit; addition of idea unit; paraphrasing of idea unit which introduces wrong information) - serious (1) - standard (0.5) - minor (0.25) R: recognition errors (mispronounciations; insertions; deletions; substitutions) - serious (1) - standard (0.5) - minor (0.25) CE: correct editions (no loss of information; omission of redundancies, hesitations non-verbatim subtitles) Assessment: comments on the analysis 11

12 Quantitative analysis: errors E edition errors R recognition errors P1 P2 P3 P4 P5 P6 P7 P1 P2 P3 P4 P5 P6 P7 serious standard minor TOTAL score edition errors prevail over recognition errors in all programmes; 64.4% of all errors found in the programmes are human errors same number of serious errors for both recognition and edition (6) minor errors take the lion s share in both categories (63 E; 35 R) 12

13 A quick comment on errors Overall 119 edition errors: 6 serious, 50 standard, 63 minor minor edition errors > standard errors in all programmes except P1 95% of all edition errors are either standard or minor Overall, 87 CE (correct editions) in the seven programmes Overall 64 recognition errors: 6 serious, 23 standard, 35 minor minor recognition errors > or = standard errors in all programmes except P % of all recognition errors are either standard or minor key problems: - recognition errors (the majority) - spelling errors - proper nouns (place names) 13

14 Accuracy rate P1 P2 P3 P4 P5 P6 P7 Av OSR (wpm) Accuracy rate CE Respeaker A, Respeaker B, Respeaker C average accuracy rate 95.6 %: not hitting the 98% target Respeaker C closer to target, with higher-than-average rates in all three programmes Respeaker A: higher-than-average rates in all three programmes Respeaker B falls short of the target by a long shot and is below average in the only programme attempted (P1) 14

15 A comment on accuracy rate respeaker B was the least experienced of the three: had recently joined SWISS TXT and this was one of his/her first live programmes data on P1 can be considered less significant in terms of the NER model, because the respeaker only managed to produce 19 subtitles (as opposed to an average of 35 per programme): the bulk of the errors is content omission, as the respeaker was having trouble keeping up less experienced respeakers take a little longer to warmup, but the weather forecasts are very short: by the time you ve managed to get going, the programme is over! a positive aspect is the ability to edit the text strategically, in all 3 respeakers: even respeaker B managed 13 CE, above average 15

16 A closer look at some editions: reduction In the NER model: edition errors include: omissions of idea units; additions of idea units; paraphrasing of idea units which introduce wrong information correct editions include: omissions of redundancies, hesitations with no loss of information text reduction (both as a result of a strategy and as a result of an error) is a key feature of respeaking and of all kinds of subtitling! - Reduction of the original message is necessary for two reasons: to adapt the message to the reading speed of the viewers, and to avoid delay» (Geens 2011) - Reduction is crucial to take into account slower readers and speech recognition constraints (Remael 2009) Aim: to analyse more in detail some of the editing decisions made by respeakers (both errors and correct editions) 16

17 Text reduction in respeaking and in subtitling variously referred to as text reduction, compression, omission, condensation, summary, synthesis. Various labels and classifications in the literature: text reduction (Romero Fresco 2009, 2011) reduction by substitution and reduction by omission (Geens 2009) reduction to prevent delay (e.g. omitting redundant information); forced reduction ; time-induced reduction (Remael 2009) qualitative (meaningful words or phrases) and quantitative (grammatical words, discourse markers, features of oral speech) omissions ( Eugeni 2009) Our choice: - adaptation of the model suggested by Gottlieb (1992) for the subtitling of prerecorded programmes - importance of the principle of relevance as in Kovaćić (1994, 1996) 17

18 Condensation, Deletion, Decimation Our categories (Gottlieb s original definitions + our subcategories) Transfer: both meaning and form can be fully reproduced (in respeaking: verbatim subtitles) Condensation: condensed expression, concise rendering (normal speech and standard speech rate) - CON-A: more concise expression used (language-related) - CON-B: caused by time or space constraint - CON-C: more concise subtitle because information is expressed in images (visual source, e.g. graph or picture) In both transfer and condensation, there is no information loss 18

19 Condensation strategies: qualitative analysis Decimation: abridged expression, reduced content (fast speech of some importance) Dec-A: language Dec-B: time or space constraint Dec-C: visual source Deletion: omitted expression, no verbal content (fast speech of less importance) Del-A: language Del-B: time or space constraint Del-C: visual source In both decimation and deletion, there is some information loss, but less serious in the C sub-categories (some information can be retrieved from the images, some is lost) 19

20 Condensation strategies: qualitative analysis subtitles percentage Transfer % Condensation % Decimation % Deletion % over 63% of subtitles are either transfers or condensations (no information loss) results in line with studies on interlingual subtitles for recorded programmes or films (Gottlieb 1992, Sandrelli 1996, 2002, 2006) Results are also similar to Eugeni (2009), despite the fact that his categories are different: «reductions» are the vast majority of all alterations (especially omissions ); images almost wholly compensate what is omitted. 20

21 Condensation strategies: qualitative analysis A - language B time/space C -visual B & C Condensation Decimation Deletion the visual element plays a key role in all three strategies decimations and deletions in this programme genre are less serious, thanks to the abundance of visual information (graphs and pictures) that ensures that deaf and hard-of-hearing viewers can access certain data anyway «visual redundancy» : relevance is not a purely linguistic principle but must be applied in relation with the MACRO-SIGN, i.e. speech and visual elements in the original 21

22 Conclusions subtitles are not an autonomous text, they are integrated with the images: visual information can be exploited to avoid repeating the same things in the subtitles difficult to make decisions of this kind because you must bear in mind the subtitle delay (the graph might have disappeared by the time the subtitle appears on the screen): audiovisual coherence Training implications: - genre-based approach to training: different OSRs, different content, different number of speakers involved, etc. - exercises for coping with speed (see Respeaker B s experience) - exercises aimed specifically at highlighting «visual redundancy» and maintaining audiovisual coherence - exercised aimed at enhancing trainees awareness of the difference between deliberate text reduction (i.e. correct editions) and loss of information: condensation can be used as a positive strategy 22

23 THANK YOU GRÀCIES GRACIAS GRAZIE 23

24 Selected references Eugeni, Carlo (2009) "Respeaking the BBC News. A strategic analysis of Respeaking on the BBC" The Sign Language Translator and Interpreter (SLTI), Vol. 3, n. 1, Manchester, St. Jerome Publishing: Geens, Rita (2011) How to train an interpreter into a respeaker, Paper presented at the Third international symposium on live subtitling with speech recognition, Exploring new avenues and new contexts: live subtitling and other respeaking applications for (media) accessibility, Antwerp - Friday October Gottlieb, Henrik (1992) «Subtitling - A New University Discipline», in Cay Dollerup & Anne Loddegaard (eds) Teaching Translation and Interpreting: Training, Talent and Experience, Amsterdam & Philadelphia: John Benjamins Publishing Company Kovačič, Irena (1994) Relevance as a Factor in subtitling reductions. In: Dollerup, Cay & Lindegaard, Annette, (eds.), Teaching Translation and Interpreting 2: insights, aims, visions. Amsterdam: John Benjamins Publishing, pp Kovačič, Irena (1996), Subtitling strategies: A flexible hierarchy of priorities, In: Heiss, Christine & Bollettieri Bosinelli, Rosa Maria, (eds.), Traduzione multimediale per il cinema, la televisione e la scena: atti del convegno internazionale: Forli, ottobre Bologna: CLUEB, pp Monsorno, Stefania (2012) Il respeaking in televisione. Strategie di condensazione e valutazione del tasso di accuratezza dei sottotitoli, Unpublished MA dissertation, LUSPIO University. 24

25 Pöchhacker, Franz (2004) Introducing Interpreting Studies London/New York, Routledge. Remael, Aline (2009) "Respeaking at the University of Antwerp and Antwerp University College: Research and training", Paper delivered at the 2nd International Seminar on Real- Time Intralingual Subtitling, Universitat Autònoma de Barcelona, 19 June Romero Fresco, Pablo (2009) «More Haste Less Speed: edited vs verbatim respeaking, in special issue of Vigo International Journal of Applied Linguistics (VIAL), VI, Romero Fresco, Pablo (2010) «Standing on Quicksand: Hearing Viewers? Comprehension and Reading Patterns of Respoken Subtitles for the News». In: Díaz-Cintas J, Matamala A, Neves J, editors. New Insights into Audiovisual Translation and Media Accessibility. Media for All 2. Rodopi. Romero Fresco, Pablo (2011) Subtitling through Speech Recognition: Respeaking Manchester, St Jerome Publishing. Sandrelli, Annalisa (1996) Caro diario di Nanni Moretti. Analisi delle versioni sottotitolate in inglese e spagnolo, Unpublished MA dissertation, University of Trieste. Sandrelli, Annalisa (2002), Comicità italiana e humour inglese: un matrimonio felice?, Il traduttore nuovo, Multimedia Atti del convegno, vol LVII, ottobre 2002: Sandrelli, Annalisa (2006), La traduzione dei proverbi in sottotitolazione: I cento passi, in G. Benelli & G. Tonini (eds) Studi in ricordo di Carmen Sánchez Montero, volume II, Dipartimento di Scienze del Linguaggio, dell Interpretazione e della Traduzione, Università degli Studi di Trieste: