Constriction Goals of Gestures

Transcription

1 Constriction Goals of Gestures Define constriction goal with respect to a control model that can generate patterns of kinematic, acoustic change over time. Model should account for: Regular relations among multiple kinematic observations for a a given phonological category. Differences in kinematic observations across contexts. Regular differences between contrasting phonological categories in these kinematic observables.

2 Elements of Model Toolkit Virtual Targets Relation of target to acoustic consequence Activation Interval Relation of activation interval to stiffness Release Gestures Coordinative Structures Articulator Weights

3 Profile of Lip Closure Gestures (Westbury & Hashi) 1. No systematic differences between lip closures for /pbm/. 2. Less movement of Upper Lip than Lower Lip. 3. Inflection in movement histories around the time of acoustic closure (t c ). 4. Substantial movement during acoustic closure interval (t c -t min - t r ). 5. LA greater at acoustic release than acoustic closure.

4 Profile of Lip Closure Gestures (Westbury & Hashi) 6. Speed movement maximum within 30 ms of acoustic closure. 7. Release speed greater than closure speed (English).

5 Movement during Coronal stops (Gracco & Löfqvist, 2002) Despite the hard palate, coronal stop show considerable tongue movement during closure. even of the tongue tip.

6 Articulator Weights AUDIO AUDIO GLA GLA LA LA ULy LLy ULy LLy vlly vlly vll vll 'LA' JA=32,UH=6,LH= 'LA' JA=6,UH=5,LH= more realistic!

7 Manipulation of Dynamical Parameters Changing a parameter can effect multiple kinematic and acoustic consequences Target (ata_targ) Frequency (ata_freq) Activation Interval (ata_long) Utterance: (AA)(TAA)(AA)(TAA)

8 Target 'TTCD' 'ons1_clo2' 'TTCD' 'ons1_clo4' Note effects: Peak Velocity Displacement CLO duration AUDIO GTTCD TTy vtt TTCD

9 Freq (ω) 'TTCD' 'ons1_clo2' 'TTCL' 'ons1_clo2' 'TTCD' 'ons1_clo4' 'TTCL' 'ons1_clo4' Note effects: Peak Velocity CLO duration AUDIO GTTCD TTy vtt TTCD

10 Activation Interval 'TTCD' 'TTCL' 'TTCD' 'TTCL' AUDIO Note effects: Displacement CLO duration GTTCD TTy vtt TTCD

11 Virtual Targets (Löfqvist) Virtual goal of the lips is to penetrate each other, thus forming an airtight seal. Explains Changes in LA after acoustic closure Differences in LA at closure and release Trajectory inflections at closure (e.g. protrusion) Tight, short temporal coupling between deceleration and closure: deceleration of lips is actually caused by lip contact, rather than the dynamical control.

12 BUT: Another account of short temporal coupling Not necessary to appeal to mechanical forces, though they presumably are involved. Time from Vp to Acons will be a function of x o Why? Vp occurs at constant lag following Gestons For a given activation interval and ω, as the target (xo) specification for a constriction becomes more and more extreme (more compression), the time to acoustic closure gets earlier and earlier, as acoustic closure gets to be a smaller and smaller proportion of the way to the target (xo) AUDIO GLA LA ULy LLy vlly 5000 vll LL Vmax= 29 'LA' JA=32,UH=5,LH= x o (LA)= -10

13 Stops vs. Fricatives Virtual target for fricatives? What is goal for fricatives? Observed effects: stops vs. fricatives voiced vs. voiceless fricatives coronals vs. labials

14 Stops vs. Fricatives Fricative lag from closing Vp to frication onset > stop lag from closing Vp to closure onset Time from Peak Velocity to CLO/FRIC Fricatives show lower Vp than stops. Are effects due to more precise production of fricatives. ms Alternative: Assume dynamical systems with the same value of ω, d. Assume different targets for stops (virtual) than fricatives (actual) mm/sec 10 8 p b m f v t d n s z Value of Peak Velocity at CLO/FRIC System will produce differences in Vp and lag p b m f v t d n s z

15 Velocity and lag effects fall out of x o differences AUDIO LL AUDIO GLA LA ULy LLy GLA LA ULy LLy vlly 5000 vll vlly vll Vmax= 29 'LA' 'LA' JA=32,UH=5,LH= Vmax= 16 'LA' JA=32,UH=5,LH= STOP x o (LA)= -10 FRICATIVE x o (LA)= 1

16 Voiced vs. voiceless fricatives Lag: Generation of turbulence is a function of both constriction size and airflow Change in Constriction from CLO/FRIC to min Same gesture for voiced, voiceless will generate turbulence later (narrower constriction) for voiced. because a more narrow constriction is required to generate turbulence when the airflow is less. mm Predictions: p b m f v t d n s z narrower constriction at turb onset for z it will take time to narrow that much more longer acoustic duration for /s/

17 Voiced vs. voiceless fricative models /asa/ AUDIO /aza/ AUDIO GTBCD GTTCD TTCD=4 GTBCD GTTCD TTCD=1.8 TTCD TTCD TTy TTy vtty vtty Vp to turb ons = 35ms Vp to turb ons = 70ms X o (TTCD) = 1 X o (TTCD) = 1

18 Coronal vs. labial effects Vp effect can be due to initial conditions, assume same targeted constriction degree for coronals and labials. x init -x 0 for coronals > labials

19 Models of /t,d/ vs. /p,b/ /apa/ AUDIO /ata/ AUDIO GLA LA=14.2 GTBCD TTCD=20 LA GTTCD ULy TTCD LLy V p (LLy)=28.5 TTy 5000 V p (TTy)=60 vlly vtty X o (LA) = -2 X o (TTCD) =

20 Additional Tests How much of close temporal coupling of Vp and closure in stops due to the relation of the virtual target to closure; how much is due to the passive forces? Does deceleration peak coincide with closure? If due to learning of (particular) virtual target we should see similar patterns for closure and release. How much change in goal variables during constriction intervals in stops vs. fricatives?

21 Example Token AUDIO TTy TTv (tangential) TTa (tangential)

22 Different control for stops and fricatives? (Fuchs et al) Fricatives show lower magnitude deceleration peaks than stops (Hoole, 1996) Passive forces decelerate stops? Deceleration coincides with onset of oral closure Fricatives involve more precise positioning acquired later cause problems in speech impairements lateral positioning must be controlled to produce grooving in /s/

23 Experiment: German Hypothesis: control of fricatives differs of that from stops Predictions: less overall movement in fricatives lower lower deceleration magnitudes longer durations less movement during constriction interval Ich habe gecvce nicht gecvc erwähnt. C = /t, z/, V = /a, u/

24 Movement amplitude Movement amplitude t > z

25 Closing gesture duration Gesture duration z > t, only for /a/

26 Acceleration and Deceleration dec (ons) >> dec (rel) for /t/, but much less so for /z/ evidence for mechanical forces

27 Horizontal movement during closure/ movement during clo/fric t > z frication

28 Conclusion Fuchs et al. conclude there is a different type of control for stops than for fricatives stops: target planned beyond contact location fricatives: target planned at lateral margins to produce midsagittal channel Just this difference in target seems sufficient to account for all observed kinematic differences, given a gestural dynamical model in the presence of mechanical effects of collision.

29 Geminate vs. Single Cs Löfqvist(2005) If targets are virtual, then perhaps geminates are produced with more extreme constriction targets. Why would this produce longer durations? Predictions: Peak position of LL is higher in CC than C Positive correlation of LLy and ULy at point of max LL height Higher peak velocity for CC

30 LLy max

31 ULy at LLy max

32 Peak Velocity during Closing

33 Averaged Time Functions

34 Conclusions Löfqvist(2005) concludes that both spatial and temporal control for C and CC must differ. TD model: same spatial targets same gesture frequency Different Activation Intervals (longer for CC) For this frequency, target will be substantially undershot in the shorter activation interval. Frequency could also be lower for CC. This would model Löfqvist s results that V p /D differ

35 Activation Interval 'TTCD' 'TTCL' 'TTCD' 'TTCL' Note effects: Displacement CLO duration More extreme-> stronger burst AUDIO GTTCD TTy vtt TTCD

36 Berber Ridouane shows that lexical geminates 1. are longer than single Cs 2. have a stronger burst than single Cs 3. shorten the preceding vowel, compared to CC clusters. Gestural Japanese model Could account for 1 and 2, but not 3.

37 AUDIO Japanese model 'TTCD' 'TTCD' GTTCD TTy vtt TTCD

38 Berber Model for Lexical Geminates Gestural model 1.Activation Interval longer for Geminates 2.Target is more extreme Will produce stronger burst and shorten preceding V.

39 Berber Lexical Model 'TTCD' 'TTCD' GTTCD TTy vtt Also Predicts Vp TTCD Difference Not observed in Japanese, but Berber?

40 Fake Geminates Ridouane shows that geminates arising from concatenation across morphemes show the increased length of true geminate fail to show the stronger burst that true geminates exhibit. Gestural model Two adjacent (non-overlapping) constrictions Final position would not be as extremely constricted as in true geminates This could account for the absence of stronger burst

41 Berber: lexical vs. fake TTCD' 'TTCD' 'TTCD'

42 'TTCD' AUDIO Single vs. fake 'TTCD' 'TTCD' GTTCD Predicts Burst TTy difference But no V duration or vtt Vp Difference TTCD

43 Juncture geminates May exhibit multiple peaks in EMA data, even though there is no release acoustically.

44 Italian Geminates (Smith, 1994)

45 French juncture geminates (Smith, 1996)

46 Fricative action goals Hypothesis: Goal is to generate turbulence. Consequences: TTCD goal value is crit, that degree of constriction that will produce turbulence. Does the actual degree of constriction vary with contextual conditions to ensure turbulence? Goal will not be a virtual one like stops, but the actual constriction size necessary But is the goal specifiable in mm, or is it functionally specific to generate turbulence?

47 Wake vs. channel turbulence Wake Sibilants (/s/, /ʃ/) are produced by nozzling airsteam against the teeth (including the lower teeth) (Ladfoged & Maddieson, Catford). Channel For non-sibilant fricatives (θ,ç, x), turbulence is generated as air passes through narrow channel. Possible controlled variables to distinguish them: TTLT (distance from tongue tip to lower teeth), or LTH (lower teeth height).

48 Goal to generate wake turbulence?

49 /s/ - /ʃ/ contrast Possible Goal variables: TTCL dental alveolar postalveolar Gafos (1999) argues that these are all variable across speakers proposes Channel Area (TTCA) as controlled variable (takes into account the cross-sectional shape) TB may be involved in controlling TTCA. Other possibilities TTLT (needed anyway to control sibilants) sublingual cavity (also can be modeled as TTLT). /ʃ/ in English may also involve TB gesture?

50 /s/ - /ʃ/ Contrast Groove (Narayanan, 1995) /s/ /ʃ/

51 /s/ - /ʃ/ Contrast Blue = asha; red = asa. Images courtesy of Michael Proctor.

52 Area Functions (Narayanan et al, 1995) /s/ /ʃ/ / /z/ /Z/

53 TTCL Possible goal variables /s/ - /ʃ/ (Narayanan et al, 1995) alveolar vs. postalveolar TTCD (TTCA) does not appear to differ systematically apical-laminal /s/ more likely apical, but speakers differ. GROOVE /s/ has concavity behind constriction Wake Location between teeth for /s/, upper teeth for /ʃ/. TB Constriction Palatal for /ʃ/, possible velar for /s/ for some speakers TB Constriction palatal for /ʃ/, possible velar for /s/ (Sublingual Cavity) Larger for /ʃ/

54 TTCD TTCL

55 /s/-/ʃ/ contrast

56 /s/-/ʃ/ contrast s ʃ Area btw tongue & palate Curvature wrt palate Groove depth

57 Testing Goal Variables Goal variables will tend to be achieved despite perturbations Use context variability as a kind of natural perturbation Variables that exhibit relative invariance across contexts are plausible goals.

58 Gestures for stops and fricatives Goal: Characterize the gestural control for the tongue in English /t,s,ʃ/ Limited to mid-sagittal plane (TADA, EMMA) Expectations: TT controlled for each (TTCD? TTCL? TTLT?) More TB control for fricatives than stops (TB for s and ʃ should differ from one another; /t/?) Context-dependence (vowels) More for stops than fricatives For stops, only TT is controlled, so TB is completely controlled by the co-produced V. Can context-dependence in fricatives be modeled with invariant TB gesture that blends with TB gesture with vowel?

59 Data EMMA (Jaw, TT, TB1, TB2, TD) Control utterances from speech error experiment: Two-word phrases read to a metronome (about 15 reps) Two words identical: sip-sip sip-sip sop-sop sop-sop Vowels: /i/, /a/ Consonants: /t/, /s/, /ʃ/ Accent: Iambic and trochaic Rate: Fast and Slow Four Subjects Measurements at time of max constriction

60 Results: Differences among coronals Horizontal position (all sensors): s > t > ʃ (s is most anterior) Vertical position (all sensors): ʃ > t > s TB: Large difference between and ʃ and t,s Others: small differences

61

62 Horizontal: Results: Context Effects (i-a) /s/ most invariant All effects are small Vertical TB2: Large effects for s, t, but ʃ is pretty invariant TD: Large effect for t, ʃ is pretty invariant (s invariant at fast rate) Others: smaller effects Context effects always smallest for ʃ.

63

64 Implications for gestural control Small differences in TTx, TBx could result from differences in targets for TTCL, TTCD s - t differ in TTCD s - ʃ differ in TTCL, TTLT Large differences in TBy Could result from TB gesture for ʃ. Blending wt of this TB gesture is stronger than for Vs, preventing blending.

65 TADA modeling t s ʃ TTCD TTCL 56 o 56 o 56 o TBCD 10 6 TBCL 125 o 65 o alpha 5 10

66 TADA results t s ʃ a i

67 Aerodynamic or acoustic goals? Aerodynamic Compensation for velic leakage? Acoustic Lip-rounding for /ʃ/ Articulatory variability across subjects Acoustic feedback for altered palate conditions

68 Acoustic Goals

69 Perturbed Auditory feedback: compensation for auditory perturbation Human cortical sensorimotor network underlying feedback control of vocal pitch Edward F. Chang,a,1,2 Caroline A. Niziolek,b,1 Robert T. Knight,a Srikantan S. Nagarajan,c and John F. Houdeb,2 Proc Natl Acad Sci U S A Feb 12; 110(7):

70 Application to Fricatives (Casserly, 2011) (A) Final trial of baseline, without acoustic alteration. Speaker produces a vowel-[ ]-vowel nonword. (B) Production is recorded and the voiceless fricative is automatically excised from the file (vertical lines). (C) Power spectrum is computed from the fricative and its frequency bandwidth (schematic cutoffs shown by vertical lines) is noted. (D) Synthetic fricative containing turbulent noise starting at the topmost edge of the observed frequency bandwidth is created. (E) Highfrequency fricative noise is played over headphones to the speaker during production of their next fricative. That natural production starts the cycle over at (A). 9 subjects imitate shift 6 counter it (compensate) 5 showed no change

71 Multi-gestural segments and goals For segments that are composed of single constriction gestures, it is possible to consider a goal for the segment defined in auditory or acoustic coordinates. For multi-gestural segments (liquids, nasals, round vowels) consideration of an acoustic goal leads to the question of how the multiple gestures are coordinated. If the gestures are coordinated sequentially (anti-phase), then it is difficult to see how the goal could be a static acoustic target: e.g., laterals and nasals in coda. If the gestures are coordinated relatively synchronously, a static acoustic goal can be considered: nasals and laterals in onset only, /r/, fricatives?

72 American English /r/ Initial /r/ have three gestures: lip, TT/TB, TR Speakers differ as to the nature of the middle gesture. Delattre and Freeman (1968)

73 /r/ constrictions and cavities The three constrictions divide the vocal tract tube into three cavities, regardless of how the middle constriction is formed. Different subjects produce similar constrictions, but they do differ in terms of the organ employed for the medial constriction. The position of the three constrictions all have the effect of lowering the third vocal tract resonance (F3). Differences show up in F4 and F5. There have to be some, as there are different /r/ variants tend to occur in different regions of the country (Delattre & Freeman, 1968)

74 Alwan et al. /r/ area functions from MRI

75 Alwan et al.

76 Goal of /r/? The three constrictions line up with regions of tube that are anti-nodes of the first formant standing wave. Is the goal of /r/ a reduced F3? (Gestures have been reported as relatively synchronous). Tested by Nieto-Castanon et al. (2005).

77 A modeling investigation of articulatory variability and acoustic stability during American English Õ.Õ production Alfonso Nieto-Castanon a) and Frank H. Guenther Department of Cognitive and Neural Systems, Boston University, 677 Beacon Street, Boston, Massachusetts Joseph S. Perkell Research Laboratory of Electronics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Room , Cambridge, Massachusetts Hugh D. Curtin Department of Radiology, Massachusetts Eye and Ear Infirmary, 243 Charles Street, Boston, Massachusetts Basic idea: Variability is lower along an articulatory dimension that affects a goal variable than along a dimension that is orthogonal to a goal variable, The orthogonal dimension(s) constitute what has been called the uncontrolled manifold in hyperspace.

78 Goal Variables and articulators Goal Variables 12 articulator dimensions

79 Testing Possible Goals Do the articulator dimensions whose variability correlates highly with variability of a hypothesized goal variable show less variability than ones that do not correlate with the goal variable. Divide the articulatory dimensions into the six that correlate most with goal variables and the six that correlate least. What percent of the total variability is in the dimensions that correlate least?

80 Results While individual subjects show significant evidence for tract variable control, the tract variables are different across subjects. F3 is significant for all subjects and is the only goal variable significant in the pooled data.

81 Nieto-Castanon s conclusion Evidence that goal is lowered F3. Problem: We expect the goal variables to differ across subjects because of the known articulatory differences in the palatal construction. FIG. 5. Diagram summarizing the main results in this section. The plot The pharyngeal goal variable should be relatively consistent across subjects but there is no pharyngeal data here.

82 Additional Evidence: Subject specific models factor analysis models of static MRI postures. Correlate factors with F1, F2, F3 Oddly, no pharyngeal constriction shoes up in the modeling of F3.

83 Subject-specific DIVA models of /r/ Goal is a particular formant configuration. Again, no pharynx, as this is EMA data that is being modeled.

84 Conclusions? Perhaps lowered F3 is a goal variable. Constrictions may also be goals. System is hierarchical. Evidence for constriction goals: Sound change in which the gestures de-synchronize: BIRD > BOID One way of seeing the the effect of the higher-level goal (F3) is if there are negative correlations (trading relations among construction variables).