AUTOMATED CHARACTERIZATION OF ESOPHAGEAL AND SEVERELY INJURED VOICES BY MEANS OF ACOUSTIC PARAMETERS

Transcription

1 AUTOMATED CHARACTERIZATIO OF ESOPHAGEAL AD SEVERELY IJURED VOICES BY MEAS OF ACOUSTIC PARAMETERS B. García, I. Ruz, A. Méndez, J. Vcente, and M. Mendezona Department of Telecommuncaton, Unversty of Deusto Avenda de las Unversdades 24, 48007, Blbao, Span phone: + (34) , fax: + (34) , emal: mbgarca@esde.deusto.es web: ABSTRACT In the scope of dgtzed speech sgnals wth severe pathologes, as the esophageal voces are, t s mpossble the objectve assessment of the voce based on a set of parameters accepted by the scentfc communty lke ptch, jtter, shmmer or HR. Ths mpossblty s due to healthy voces or wth slght pathologes can be only evaluated wth commercal applcatons. However, ths paper presents a robust and precse algorthm that allows the automated calculaton of ptch and jtter on very nosy and low qualty voce sgnals. Therefore, both the mprovement acheved due to medcal treatment and the evaluaton of dgtal sgnal processng algorthms wll be measurable based on objectve crtera. ITRODUCTIO Speech sgnal processng plays an mportant role wthn the dgtal processng projects and nvestgatons. Wthn ths feld, the esophageal voces are beng objectve of analyss and transformaton [2],[3] but these have the lmtaton of measurng ther qualty only wth subjectve crtera as hearng tests. Ths s because an evaluaton based on the calculaton of objectve parameters lke ptch, jtter, shmmer or the harmonc to nose rato HR demands a hgh precson n the defnton of the begnnngs and ends of cycle n the voce sgnal. The esophageal voce s generated usng the ar pass across the esophagus but wthout the modulaton possblty by the vocal fold because they have been removed due to, generally, a larynx cancer. Because of ths ther tme-spectral characterstcs are atypcal and nclude levels of nose, fundamental frequency asymmetry and formant unstructuraton. Ths leads to wrong measures n commercal applcatons and therefore s mpossble to assess the qualty of esophageal voces. The same s applcable to voces wth severe pathologes. In ths sense, the algorthm presented n ths paper offers a soluton to the exact calculaton of the marks that correspond to each cycle of the sgnal of esophageal or pathologcal voces so that the calculaton of ptch s exact and, wth t, the measures of jtter, shmmer or sgnal to nose rato. 2 OBJECTIVES The presented algorthm was developed n the context of the research work around the area of measurement and mprovement of pathologcal voces. The man am of the proposed algorthm s to characterze pathologcal voces usng objectve parameters such as ptch or jtter. Wth ths general perspectve we can establsh the followng specfc objectves: Medcal objectves: the desgned algorthm has the double purpose of evaluatng objectvely the ntal status of a patent and hs evoluton e.g. durng both rehabltaton and esophageal voce learnng stages. Research objectves: the tool wll allow evaluatng certan algorthms performance such as the pathologcal voce mprovement usng sgnal processng technques [2]. As t can be apprecated, the set of objectves wll not only allow to carry out a basc research n sgnal processng for the desgn of algorthms specalzed n the measurement of esophageal or pathologcal voces, but adds a research applcablty n the medcal and rehabltaton scopes. 3 METHODOLOGY In ths secton some concepts wll be presented. Some of them are used to characterze voces and measure accuracy whle others represent the mathematcal background employed n the desgn and development of the algorthm explaned n ths paper: 3. Ptch (or fundamental frequency, F0) It s the property of a sound or muscal tone measured by ts perceved frequency. Due to de pseudo-perodc nature of the voced speech, there are varatons n the nstantaneous frequency f so the ptch can be defned as Ptch( Hz) f beng the number of extracted ptch perods. The key ssue at ths pont s to determne the exact tme nstants where the ptch s produced n a sgnal, and that s what ths paper analyzes. () 2007 EURASIP 2224

2 3.2 Sonorty It s defned as a knd of measure of the energy contaned wthn a wndowed part of the sgnal and s used as the reference to fnd the ptch nstants: Sonorty( k) n B( k, n) Beng B(k,n) a matrx contanng n each column the FFTs of ponts of the k wndows of the sgnal. 3.3 Jtter It s the varaton of one or more sgnal characterstcs; n ths case t s related to the fundamental frequency: Jtter(%) f f f + Jtter( Hz) f f + (2) (3,4) f + f + f+ f 2 3 RAP 2 (5) (%) 00 f There are many parameters whch represent the jtter beng the number of extracted ptch perods. The prevous are the most mportant ones defned n [] but whle the jtter(%) s the most common measure, the MDVP software, whch s used to test the results n secton 5, uses the Relatve Average Perturbaton (RAP) as the jtter value. 3.4 MATLAB and jopas/octave All the desgnng stage was made wth Matlab from Mathworks Inc. but when mplementng a software tool n Java to apply the algorthm, Octave (the free software verson of Matlab) was used together wth jopas API ([7], a Java to Octave brdge). 4 DESIG The algorthm s based on a core procedure desgned to extract the ptch wndowng the sgnal and checkng whch frames are the most energetc ones. Furthermore, correctng actons are performed to mprove the accuracy of the results. 4. Basc Procedure The ptch extracton s done by means of a tme doman algorthm whch takes nto account the energy (the sonorty n ths case) wthn each wndowed frame. The hgher the sonorty wthn one frame, the hgher the probablty of contanng a ptch nstant on t. Applyng ths to the negatve sde of the sgnal (whch has more energy n most of the cases), the ptch nstants can be estmated and ft n the correct place of the voce sgnal consderng that they must fulfll the constrant of beng the absolute mnmum of the frame below a gven threshold. The fgure shows the whole process whch calculates the vector wth the nformaton of the ptch nstants. Ths vector s called the base. Fgure : base calculaton dagram Once the base has been calculated the ptch can be drectly measured. Frst of all the ptch profle s calculated, the ptch profle s the varaton of the nstantaneous frequences (represented as the nverse of the tme dfference between two consecutve ptch peaks) along the tme. Fnally, the mean of the ptch profle s taken as the ptch value. 4.2 Correctve Actons The core procedure becomes a very sutable algorthm to work wth all knd of voces (healthy, pathologcal and esophageal) but must be mproved to obtan the best possble results n some cases. In order to acheve the amed mprovement three correctve functons were developed: base correcton, ptch profle correcton and nterpolaton. An example of a sgnal and ts base s shown n the fgure 2. Fgure 2: Sgnal and ptch marks of the base 2007 EURASIP 2225

3 The base correcton avods beng two consecutve peaks separated less than nmn samples whle the ptch profle correcton removes from t all the nstantaneous frequences correspondng to peaks separated more than nmax samples. To adjust these two parameters an estmaton of the range where the ptch s supposed to be must be done. Ths estmaton s performed calculatng the ptch value of the speech sgnal n the cepstrum doman. Wth these correctons the accuracy of the algorthm s hghly mproved but sometmes there are peaks whch reman undetected. Despte the mssng peaks the ptch value s stll accurate but knowng the ptch value, mssng peaks can be calculated. In ths way, the nterpolaton searches for dstance between peaks that are too bg for the gven ptch and tres to fnd all the peaks mssng between them. 4.3 The whole process The am of the research was a to desgn a ptch detecton algorthm sutable for all knd of voces (healthy, pathologcal and esophageal) whch also allowed the drect estmaton of other acoustc parameters such as jtter or RAP. In fact, the algorthm was developed to measure correctly the ptch n esophageal voces whch s the most dffcult ssue to deal wth because there s lttle work around ths. Once ths was acheved t was adapted to be applcable to the rest knd of voces. The frst teraton of the ptch estmaton performs an ntal calculaton wthout correctve actons to check f the voce s esophageal by means of a prevous value (called pre-ptch). Then a second step s done calculatng and correctng as explaned before n order to obtan an accurate result as t s shown n the next fgure: whch represents the key ssue of the algorthm. In fact, the whole process s desgned to perform a correct assgnaton of these parameters dependng on the voce type and estmated ptch range. The block dagram showng the full algorthm s presented next: Fgure 4: Hgh-level block dagram The only fxed parameters are the ntal ones, those that are used to calculate the pre-ptch whch have been estmated expermentally to get the results whch lead to dstngush the esophageal voces. It has been proven that a voce s esophageal f ts preptch s below 0 Hz. Once the voce s classfed the parameters are assgned as explaned above. There s a set of the presented parameters for analyzng esophageal voces (defned as Low ptch parameters n fgure 4) and wth other knd of voces the parameters are assgned dependng on the band n whch the ptch s supposed to be. After all, the peak nterpolaton s performed for all the cases.. Havng ths, the calculaton of the jtter s performed as follows: Fgure 3: Ptch and Pre-ptch block dagram As t can be seen n ths fgure there are many parameters (wn, shft, thres, nmn and nmax) to be set Fgure 5: Jtter calculaton process Thus, takng the ptch profle nformaton (the nstantaneous frequency values correspondng to each cycle) the jtter s obtaned applyng the expresson (5) for the RAP parameter. The justfcaton of ths ambguty s presented later n the next secton EURASIP 2226

4 Esophageal Voces Pathologcal Voces Healthy Voces # Praat MDVP Proposed Algorthm Exact Value Ptch(Hz) RAP(%) Ptch(Hz) Jtter(%) Ptch(Hz) RAP(%) Ptch(Hz) RAP(%) A A 92,08 0,399 04,575 0,237 04,587 0, ,96 3,844 82,003 4,067 69,609 3,847 69,60 4,65 3 A A 63,584 0,774 60,962 7,065 58,746 7,09 4 A A 60,386 A 6,539 6,849 6,533 7, ,08 A 89,405,32 89,34 3,225 89,308 3, ,367 0,637 83,95 4,725 72,492 5,336 72,479 5, ,28 5,85 79,508 5,57 62,726,5 62,534, ,77,37 85,265 2,522 62,86 4,62 62,820 3, ,79 3,69 88,487 2,360 59,46 3,98 59,423 3, ,266,307 62,735 5,545 59,543,799 59,549,95 35,68 0,380 35,543,27 35,628 0,442 35,6 0, ,605 2,67 23,79 3,064 23,684,543 23,778, ,49,33 202,3, ,04, ,905, ,339 0,853 84,74 0,558 84,94 0,776 84,39, ,939 3,242 8,274 0,699 8,226 0,95 8,204, ,652 0,922 59,82 0,356 59,70 0,422 59,69 0, ,24 0, ,799 0, ,339 0, ,277 0, ,552 0,370 47,56 0,237 47,598 0,435 48,2 0, ,263 0,25 49,254 0,8 49,265 0,346 49,266 0, ,665 4,706 57,455 0,89 57,542,56 57,550,83 07,447 0,693 03,50 6,56 03,57 5,498 03,684 5, ,580 0,605 36,792 0,802 37,384 2,922 36,469 2, ,502 0,224 24,740 0,344 25,356 0,577 25,409 0, ,537 2,234 92,53 0,662 92,893,362 92,77, ,622 0,325 0,497,59 0,505 0,433 0,638 0,970 6,888 0,454 99,00 0,630 99,634 3,4 99,864 2, ,4 0,96 93,50 0,49 93,384 0,929 93,8 0, ,67 0,447 33,539 0,389 34,802 0,433 34,385 0, ,859,073 03,68,238 02,737 0,957 02,828,36 0 3,232 0,96 3,408 0,93 3,03 0,396 3,246 0,38 Fgure 6: Comparatve table wth ptch and jtter/rap measurements 5 RESULTS It has proven to be a very accurate algorthm wth all knd of voces. For that, a large database was compled: 85 utterances were recorded beng 64% of them female voces and 36% male. Three knds of voces were recorded (healthy, pathologcal and esophageal) so that a complete corpus was acheved: 65 of them were of patents wth slght pathologes, 0 were of esophageal voces (patents wth severe pathologes), and 0 of healthy people. Only ten voces of healthy people were taken because ther acoustcal propertes are very smlar to the pathologcal ones and these are the most numerous of the corpus. In order to compare the obtaned results n the research and measure the accuracy of the results, many tests have been performed checkng the results aganst well-known software packages. For the testng stage two references were taken, n one hand the MDVP software from Kay Elemetrcs [4] and n the other the Praat software from the Insttute of Phonetc Scences of the Unversty of Amsterdam [5]. Although t s known that they are not sutable for automatc characterzaton of esophageal voces (see Fgure 7), they can be used to perform the measurements 2007 EURASIP 2227

5 manually. The man drawback of usng MDVP (and Praat) s that s not possble to measure esophageal voces ptch wthout a manual adjustng of the frequency range and voced perod marks. Furthermore, wth a manual confguraton of both applcatons the measured ptch s stll naccurate. An example of ths naccuracy can be checked n the st and 7 th tems correspondng to esophageal voces on fgure 6. Fgure 7: Esophageal voced measured n MDVP To show ths, n fgure 7 can be seen a vocng analyss of an esophageal voce wthout a manual setup. As t s shown, not enough voced perod marks are added and almost all of them are wrong placed. So as to llustrate the obtaned results, the table n the prevous page (Fgure 6) shows examples of the dfferences of the ptch and jtter measurements n some cases for the three knds of voces. Here must be ponted out that whle the proposed algorthm measures automatcally samples of each voce group, to measure esophageal voces ptch n MDVP and Praat, a manual setup was performed adjustng the ptch range. As ntroduced n secton 3 frequency perturbatons measurements were also performed. In the begnnng the jtter parameter was chosen to study ths knd of perturbaton but n the end was dscovered that MDVP uses the value of RAP to report ths parameter, thus the same value s achevable wth the same ptch marks. So as t can be seen, regardless of the column ttles whch just represent the name gven by the applcaton, the table n fgure 6 only shows the RAP measurements for each applcaton. 6 COCLUSSIOS The proposed algorthm has overcome the amed objectves. The followng conclusons can be rased from the analyss of the results prevously descrbed: Esophaegal voces can only be evaluated objectvely by means of ptch and jtter usng the algorthm proposed by authors. It s verfed that the acoustc parameters ptch and jtter are vald to evaluate the qualty of the voces snce the values obtaned for esophageal voces are outsde the range of normalty accepted by the scentfc communty. Ptch measurements of healthy and pathologcal voces can be performed both wth commercal applcatons and wth the algorthm proposed by the authors. However, the jtter(rap) values obtaned by Praat are not vald The results of the research descrbed n ths paper open new future horzons n the optmzaton of the accuracy of the calculus of the objectve parameters n the scope of ether esophageal or pathologcal voces wthn a telemedcne applcaton. In fact, authors are workng at the moment n shmmer and harmoncs-to-nose rato (HR, [6]) measurements. In the case of telemedcne, t would be necessary to study the effects of the transmsson on the sgnals and, f necessary, to adapt the algorthm of measurement to the modfcatons that the channel carres out. Telephonc communcatons, submarne and other contexts are sutable to take advantage of the algorthm to objectfy ts qualty. 7 ACKOWLEDGEMETS Ths research was partally carred out under grant TEC C02-02 from the Mnstry of Scence and Technology of Span. Authors also wsh to thank the support of IRIA s EUROMED 2007 project. REFERECES [] P.J. Baken, R.F. Orlkoff, Clncal Measurement of Speech and Voce. 2nd Edton, CA:Sngular Publshng Group, San Dego, [2] B.García, J.Vcente, I.Ruz, A.Alonso and E.Loyo, Esophageal Voces: Glottal Flow Regeneraton, n Proc. ICASSP Phladelpha, USA, March [3] Y.Q, B.Wenberg, and.b, Enhancement of female esophageal and tracheoesophageal speech, J. Acoust. Soc. Am., vol 98, no. 5, pp , ovember 995. [4] D.D.Delysk, MDVP Acoustc Model and Evaluaton of Pathologcal Voce Producton, Eurospeech, Berln, Germany, September 993. [5] P. Boersma, PRAAT, a system for dong phonetcs by computer Glot Internatonal, vol. 5, no. 9/0, pp , 200. [6] E.Yumoto and W.J.Gould, Harmoncs-to-nose rato as an ndex of the degree of hoarseness, J. Acoust. Soc. Am., vol 7, no. 6, pp , June 982. [7] J.Vcente, B.García, A.Mendez, I.Ruz and O.Lage, Teachng Sgnal Processng Applcatons Wth jopas: Java To Octave Brdge, n Proc. EUSIPCO 2006, Florence, Italy, September EURASIP 2228