JOURNL OF THE EXPERMENTL NLYSS OF BEHVOR VOLUME 7, NUMBER JNURY, 1 964 DFFERENTL RENFORCEMENT OF VOCL DURTON1 HRLN LNE UNVERSTY OF MCHGN The effects of differentil reinforcement of vocl durtion were exmined in series of experiments in which ech of 28 subjects (Ss) emitted vowel whenever light ws flshed. n the first phse of ech experiment, penny ws dispensed fter ech of 2 responses. n the second nd subsequent phses, only those responses whose durtions exceeded criterion were reinforced; when 1 successive reinforcements were presented, one phse ws terminted nd the next begun. The criterion for reinforcement in ech phse ws determined by different schedule in ech of six experiments; it rnged from 8 to 12 per cent of the men durtion of the 1 terminl responses in the prior phse. Differentil reinforcement effected lrge nd systemtic chnge in the durtion of vocl responses s long s the responses selected for reinforcement hd sufficiently high probbility of occurrence. This requirement ws formulted s the difference between the criterion durtion nd the men durtion of the terminl responses in the prior phse, divided by their stndrd devition. This sttistic, nmed the shping index, ws correlted with the number of responses emitted before ech phse ws terminted. t ws found to be lrge whenever the shping process filed. Mny Ss filed to tct the reinforcement contingency despite mrked chnges in their vocl behvior nd extensive probing by questionnire, dministered t the end of ech session. n excellent description of the procedure for differentil reinforcement of responding hs been given by Keller nd Schoenfeld (195): "We select one (or more) of the 'nturl' vritions of well-conditioned response nd give it exclusive reinforcement; the remining vritions re subjected to extinction. f we pick out, in dvnce, vrition tht hs been of firly frequent occurrence, nd if we pply this selective reinforcement rigorously, we cn soon produce n increse in the frequency of the response tht possesses the property... we hve chosen. t the sme time we decrese the frequency of those responses tht do not meet our specifictions." n the experiments to be reported, the well-conditioned response selected ws the emission of the vowel /u/ by humn dults. The nturl vrints, given exclusive reinforcement, were durtions of the vocl re- 'This reserch ws performed pursunt to contrct with the Lnguge Development Section, U. S. Office of Eduction. The ssistnce of Mr. T. M. nderson is grtefully cknowledged. Reprints my be obtined from Hrln Lne, Behvior nlysis Lbortory, University of Michign, 1315 Hill Street, nn rbor, Michign. sponse greter thn criterion durtion. The vlue of the criterion ws n experimentl vrible; firly frequent vrints were picked out in some studies, less frequent ones in others. Selective reinforcement ws pplied rigorously to every vrint bove criterion. There ws n increse in the frequency of responses whose durtion exceeded criterion nd decrese in the frequency of those vrints tht were not reinforced. METHOD Techniques of mesuring the prmeters of the vocl opernt hve been discussed previoulsy (Lne nd Shinkmn, 1963). The method employed to mesure response durtion will be summried here. The vocl response ws trnsduced by dynmic throt microphone (Rye TM 1), mplified, nd processed by n verge speech power circuit (Peterson nd McKinney, 1961). The output wveform ws pplied to dc mplifier (Krohn-Hite) nd then to the timeintervl section of frequency counter (Hewlett-Pckrd 523D). The system ws clibrted with pure-tone pulses of known durtion, prepred with electronic timers nd switches. The 17
18 HRLN LNE durtion ws red in msec by the counter nd recorded by prllel printer nd crd punch (BM). Subjects nd Procedure Twenty mle nd eight femle subjects served individully in sessions lsting pproximtely 2 hr. The S ws seted in soundttenuting booth in front of pnel contining light nd coin dispenser. These instructions were red to S: "n this experiment you cn ern money simply by sying /u/ when the light in front of you flshes. You will know tht you hve erned money when penny flls into the try; t the end of the experiment you my tke home ll the pennies you hve erned. This is your only py. Occsionlly the light will not flsh for period of time. When the light is off, plese do not respond." n electronic timer flshed the stimulus light t 5-sec intervls. fter ech of the first 2 responses emitted by S, the experimenter (E) presented reinforcement nd entered the response durtion on desk clcultor. Then, E reset the timer nd computed the reinforcement criterion for the next phse of the experiment. n the second nd lter phses, only those responses with durtion exceeding the criterion were reinforced. Ech phse ws continued until 1 successive reinforced responses occurred; the timer ws reset, new criterion computed, nd the next phse initited. Criteri for differentil reinforcement. The durtion criterion for reinforcement vried ccording to the schedule shown in Tble 1. n Exp., the criterion for ech phse equlled the men durtion of the 1 responses tht terminted the preceding phse, plus their verge devition. Following this experiment, the reserch detoured to obtin preliminry estimte of the Weber frction for produced vocl durtion. Five Ss were instructed to produce 9 pirs of the vowel /u/ such tht the responses in ech pir hd equl durtion. Thirty pirs were obtined from ech S t "short" durtions, nd like number t "moderte" nd t "long" durtions. When the bsolute vlue of the difference in durtion between the responses in ech pir ws plotted s function of the shorter response, the 45 determintions were well fit by stright line with slope of.1 (determined by the method of lest squres).2 n Exp. through V, the criterion for reinforcement in ech phse equlled the men durtion of the 1 terminl responses in the preceding phse plus (or minus) some per cent of tht men. The step sie or increment in the criterion rnged from hlf the vlue of the Weber frction (5 per cent) to twice tht vlue (2 per cent). Probing for Tct of the Reinforcement Contingency. t the end of ech session in Exp. 2The verge vlue of the Weber frction, D 2=.1, turned out to coincide with the verge coefficient of vrition (/M) for the durtion of responses emitted under continuous reinforcement. ble 1 Successive Criteri for Differentil Reinforcement of Vocl Durtion Exp. Step Phse Number No. Subjects Sie 1 2 3 4 5 6 7 8 9 1-1 CRF M1+ M2+* Ms+ M4+ M,+ M6+ + D1 D2 D3 D, D5 D, 1 1-15 CRF M1+ M2+ M,+ M4+ M6- M6- M7- M8- & 2 +, +.5M1.1M2.15M3.2M4.5M5.1M6.15M7.2Ms 16-19 CRF M1+ M2+ M3+ M4+ M5- Me- M7- M8- +,_.5M1.1M2.15M3.2M4.2M5.15Me.1M7.5M8 V 21-23 CRF M1+ M2+ M3+ M4+ M6- M6- M7- Const..5M1.5M2.5MS.5M4.5M5.5Ms.5M7 V 24-25 CRF Ml+ M2+ M3+ Const..2M1.2M2.2M3 V 26-28 CRF Ml- M2- M3- +.5M1.1M2.15M3 *Men durtion of the 1 terminl responses in phse 2 plus their verge devition.
DFFERENTL RENFORCEMENT 19 -V, S ws given three-item questionnire. The response to ech item ws completed before the next ws presented. The questions were: (1) Wht do you think ws the point of this experiment? (2) Did you know wht it ws bout your responding tht erned pennies for you? (3) Did you know tht you received pennies depending on the length or durtion of your response? f the words "durtion" or "length" never ppered in S's response, nd if the word "no" ws prt of the response to questions (2) nd (3), the questionnire ws scored "negtive". Extperiment RESULTS ND DSCUSSON U N' 6 5k Shping Longer Durtions; Step Sie Constnt. The reltion between the criterion for differentil reinforcement nd the men response durtion in ech phse is shown in Fig. 1 for representtive S. The terminl 1 reinforced responses, which ended ech phse (filled tringles), the non-terminl reinforced responses (open tringles) nd the unrein- 4f- forced responses (inverted tringles) re plotted seprtely. t is pprent tht the durtion of vocl responding by S5 ws under the control of the reinforcement contingencies. s the criterion ws incresed, in steps of one verge devition, from 27 to 57 msec, the men durtion of vocl responding ws shped from 258 to 61 msec. n ech phse, the men durtion of unreinforced, non-terminl nd terminl reinforced responses incresed in tht order. This finding lrgely reflects the incresing durtion of vocl responding within the phse, which is shown for typicl phse (the lst) in the insert. Note tht the lst 1 responses in this phse hd men durtion 4 msec greter thn the criterion. This "overshoot" is chrcteristic of the other phses s well. f the men response durtions precisely equlled the criterion, they would pper in the grphs long line with unit slope, which pssed through the intersections of corresponding points on the x- nd y-xes. n generl, the non-terminl reinforced responses fll ner this line, tht is, t criterion vlues, while reinforced responses lie bove, nd unreinforced responses lie below the line. Frequency distributions of response durtions in ech phse re shown for S5 in Fig. 2. Under continuous reinforcement, response durtions re short nd their vrince is reltively smll. The criterion for ech phse nd for the one tht follows re indicted by the symbols C nd C1, respectively. With differentil reinforcement, the vrince increses 3[_ 3 4 5 CRTERON RESPONSE DURTON (M SEC) Fig. 1. Shping longer vocl durtion; step sie constnt (Exp., S5). The reltion between the criterion response durtion for differentil reinforcement nd the men response durtion. Unreinforced responses (inverted tringles), non-terminl reinforced responses (open tringles) nd the 1 terminl reinforced responses (filled tringles) re plotted seprtely for ech phse of the experiment. The bseline durtion under continuous reinforcement is shown by the symbol t lower left. The verticl lines represent one stndrd devition. The insert shows the men durtion of successive blocks of 1 responses in the lst phse. $5 25 MSEC 22 2SOi 1 2 3 4 275 325 ' 35 5 6 7 Fig. 2. Frequency distributions of response durtions in ech phse, for S5. The criterion durtion for phse is indicted by C, tht for the next phse by C1.
11 HRLN LNE 7- sook 5o- 41-3- L~ u 4 5 CRTERON RESPONSE OURTO (M SEr.) Fig. 3. The reltion between the criterion nd men response durtion, for S8 (Exp. ). The insert shows the men durtion of successive blocks of 2 responses in the lst phse. nd, in the lter phses, the frequency distributions become bimodl. For most of the Ss, the second pek lies t durtions greter thn the criterion nd reflects the "overshoot" occurring towrd the end of ech phse, described bove. Figure 2 lso shows tht S5 reched the successive criteri rpidly; the number of unreinforced responses rnged from five in phse 2 to mximum of 2 in the lst phse. useful sttistic to consider in this regrd is the difference between the criterion durtion nd the men durtion of the terminl responses in the prior phse, divided by their stndrd devition: (C-M)/o-. This sttistic, which my be clled the shping index, expresses the step sie (C-M) in terms of the vribility (-) of current behvior. n infrequent vrint of behvior hs been selected for differentil reinforcement if the shping index is lrge, either becuse the step sie is lrge, or the vribility is smll, or both. For the 16 Ss whose dt re presented in this rticle, the shping index computed for ech phse correlted.4 with the number of unreinforced responses in tht phse (Spermn rho, p <.1). The shping index for S5 is roughly constnt t bout.7; this follows from the fct tht the criterion increment ws one 4 4 1 CRTER EUNSE GRPTN P PECJ Fig. 4. Shping longer durtions, step sie incresing, then shorter durtions, step sie incresing (Exp., S12). The reltion between criterion nd response durtion. n phse 6, the sign of the criterion ws chnged 6 j (-), nd ll responses with durtion less thn the criterion were reinforced. verge devition. (For norml distribution, the verge devition is pproximtely seventenths of the stndrd devition.) The reltion between response durtion nd the reinforcement criterion is shown for second S in Fig. 3. The shping index ws constnt t bout.7; the criterion rnged o5 1 2 3 6 dlt 25 M = S/2 3 4 7 7b 75. Fig. 5. Frequency distributions of response durtions for S12. The negtive symbols denote reinforcement of responses with durtion less thn C. 9
DFFERENTL RENFORCEMENT from 272 to 66 msec while the verge response durtion rnged from 256 to 736 msec. Overshoot of the criterion ws prticulrly mrked in the lst phse, which is shown in greter detil in the insert. There were 72 unreinforced responses, ll of which hd reltively short durtions, continuing trend estblished erlier in the session. The men of the non-terminl reinforced responses ws 2 msec greter thn criterion, s in erlier phses. The terminl responses in the lst phse hd men durtion tht exceeded criterion by 13 msec. Experiment Shping Longer Durtions, Step Sie ncresing, Then Shorter Durtions, Step Sie ncresing. Men response durtion s function of criterion durtion is shown for representtive S in Fig. 4; the corresponding frequency distributions of response durtions pper in Fig. 5. The men durtion grows stedily during the first five phses nd the vribility increses proportionlly (C-/M =.5). n phse 6, the criterion for reinforcement chnged sign: ny response durtion less thn 446 msec ws followed by reinforcement. The reltive vribility doubled (c/m =.1); the men durtion of the unreinforced responses climbed bruptly to 57 msec, while tht of the terminl responses fell to 334 msec, n overshoot of 112 msec. The frequency distribution of response durtions spred nd becme trimodl, with peks corresponding, roughly, to the terminl, non-terminl reinforced nd unreinforced responses, respectively. Order ws restored rpidly in the next phse, however, where the criterion increment ws smll nd of the sme sign. The shping index for this S vried from 1.1 in phse 2 to mximum of 3. in the lst phse. The ltter vlue is lrge becuse the vribility in the preceding phse ws smll nd the step sie ws 2 per cent of the men. There were nerly four times s mny unreinforced responses in this phse s in ny other. The questionnire, which probed for tct of the reinforcement contingency, ws scored "negtive" for this S. The dt for S13 (Fig. 6) provide similr picture of the shping of vocl durtion. The criterion rnged upwrd from 369 to 1,353 msec nd then downwrd to 433 msec; the men durtion of the terminl responses Omohoow 1H ow + Om SW, O Soo,o CRtVON PONUE DURTN (MC.) Fig. 6. The reltion between criterion nd response durtion for S13 (Exp. ). rnged from 348 to 1,424 to 285 msec. The lrgest shping index (2.8) occurred in phse 5 nd there ws corresponding increse in the number of unreinforced responses (135, or 1 times more thn in ny other phse). When the sign of the criterion chnged in phse 6, n brupt increse in the durtion of unreinforced responses occurred. This finding is obtined whenever the sign of the criterion is chnged; it my be relted to the observtion, reported by Lne nd Shinkmn (1963), tht severl prmeters of the vocl opernt, in. cluding durtion, increse in extinction fter intermittent reinforcement. n ll phses but the fifth, criterion ws reched rpidly by S13, who described the reinforcement contingency on the first sheet of the questionnire. The findings obtined with three Ss who did not rech criterion in ll phses re exmined next (Fig. 7 nd 8). The dt for S15 re nomlous in severl respects. The men durtion of the non-terminl reinforced re- 2 5 F4 2 i S/S ~ / ~~~~~~ oo 3 4 5 @ CRTERON RESPONSE OJRTON (M SEC.) Fig. 7. The reltion between criterion nd response durtion for S15 (Exp. ). n phse 5, S15 filed to rech criterion.
112 HRLN LNE sponses lies bove tht of the terminl responses in phses 2 nd 3. The shping index ws quite lrge (3.2) in phse 4, but criterion ws reched within 5 responses. The 2 per cent increment in criterion in phse 5 ws not lrge step, in view of the vribility in the preceding phse, but S15 filed to receive single reinforcement in this phse. The filure to shpe the behvior of S14 (Fig. 8) my be lrgely ttributed to the sie of the shping index in the uncompleted phse: 3.8. t is interesting to note tht the men durtion of the 29 unreinforced responses in the lst phse fell below tht men for the preceding phse. The sie of the shping index similrly ccounts for the brekdown of reinforcement control with S2. The step sie in phse 4 ws modertely lrge (15 per cent) nd there ws little vribility in the preceding phse; hence, the shping index took on the exceedingly lrge vlue, 8.4. totl of 28 unreinforced responses occurred before the session ws terminted by E. The questionnires for Ss 14, 15, nd 2 were ll scored "negtive". Experiment Shping Longer Durtions, Step Sie ncresing, Then Shorter Durtions, Step Sie Decresing. The first hlf of this procedure (phses 1-5) replictes tht for Exp. nd the behvior of S16 (Fig. 9, 1) is similr in these phses to tht of S12 nd S13, presented erlier. The frequency distributions of response durtions hve greter vrince for S16 thn for erlier Ss. n phse 6, when the step 14 en. 49 3 2 _ F-i S2 $14 L-4 12 3 3 CRTERON RESPONSE DURTON (MSEC.) Fig. 8. The reltion between criterion nd response durtion for Ss 14 nd 2 (Exp. ). S14 filed to rech criterion in phse 3, S2 in phse 4. The symbols t the brek in the bsciss indicte bseline durtions for the two Ss. 15 12! 9 < 2 6 4c i - -Kmr L- err 3 6 9 6 3 CRTERON RESPONSE DURTN (USEC) Fig. 9. Shping longer durtions, step sie incresing, then shorter durtions, step sie decresing (Exp., S16). The reltion between criterion nd response durtion. sie ws both lrge (2 per cent) nd of opposite sign, the frequency distribution seprted into two clusters. One reflects the mrked increse in the durtion of unreinforced responses, the other the very short durtions of the terminl responses, whose men, 279 msec, exceeded the criterion by 487 msec. S16 ~~~~~2 2c 1 2 3 25 MSEC.g e el 675.C 4 5 C 1 6 25U/< 2s--C 25 -C 7 8 9 Fig. 1. Frequency distributions tions for S16. of response dur-
DFFERENTL RENFORCEMENT 113 The shping index of 3.3 nd the number of unreinforced responses were both lrger for phse 6 thn for ny other phse. The terminl response durtion in phses 7-9 ws less thn the men durtion observed under CRF; it reched s low s 59 msec in the lst phse, 157 msec less thn the bseline determintion. The findings for S19 (Fig. 11) re similr, lthough the rnge of response durtions is much smller: 165 to 532 to 94 msec. The shping index ws pproximtely 1.2 except in phse 5 where, coupled with 2 per cent step sie, the index rose to 4.1; the number of unreinforced responses rose from n verge of 25 to high of 185 in this phse. The men durtion of the terminl responses in phse 9 ws 94 msec or 71 msec below the bseline estblished under CRF. Both S16 nd S19 described the reinforcement contingency on the questionnire. Experiment V Shping Longer, Then Shorter, Durtions; Step Sie Smll nd Constnt. When the shping index remins smll, s in the present procedure, vribility remins smll, shping proceeds grdully nd the number of steps required to chnge response prmeters over given rnge is incresed. Subject 21, for exmple, shows stedy increse in men durtion (Fig. 12) during the first five phses nd then rpid decline when shorter durtions were reinforced. The step sie ws five per cent. The shping index rnged from.4 in the first nd lst phses to pproximtely 1. in the others. Vribility remined reltively smll, cr/m =.7, with the notble exception 5 F 4 w 23 2w d 1 Lc1m, - -. *1 +1 - -- 2 3 4 4 3 2DO CRTERON RESPONSE DURTON CM SEC.) Fig. 11. The reltion between criterion nd response durtion for S19 (Exp. ). 51-4Ww to 31-2 3 4 3 CRTERON RESPONSE DtRTON CM SE Fig. 12. Shping longer, then shorter durtions; step sie smll nd constnt (Exp. V, S21). The reltion between criterion nd response durtion. of phse 6, where the sign of the criterion chnged. Subject 21 responded "yes" to items (2) nd (3) on the questionnire. The findings for S23 (Fig. 13) provide second exmple of the result of mintining smll shping index. n this cse, the index rnged from.3 to mximum of 1. in phse 6, where the criterion chnged sign. Vribility remined smll, y/m =.6, nd the rnge of Q* 2 4OC e 3C <3O 2W_ cm. - so cnr + L-4.-- + 5 4 o3 4 CRTERON REONE DURTON (MSECJ Fig. 13. The reltion between criterion nd response durtion for S23 (Exp. V).
114 HRLN LNE durtions for the reinforced responses ws limited: 246 to 519 to 3 msec. Shping took plce quite rpidly. n the finl phse, the reinforcement criterion ws reched immeditely, with no unreinforced or non-terminl reinforced responses occurring. The questionnire for this S ws scored negtive. Experiment V Shping Longer Durtions; Step Sie Lrge nd Constnt. When step sie s lrge s 2 per cent ws employed t the outset of the shping procedure, Ss 24 nd 25, whose dt re presented in Fig. 14, filed to rech criterion fter the first phse of differentil reinforcement. The bseline durtion for S24 ws 174 msec; he reched criterion in the second phse, when the shping index ws 2.6. n the third phse, the shping index rose to 5. nd no reinforced responses occurred. The findings for S25 re similr, lthough ll the vlues re lrger since his bseline durtion ws 327 msec. The shping index in the third phse ws 3.8. totl of 273 responses were emitted in this phse, including 24 reinforced responses. t is interesting to note tht the lst 1 of these reinforced responses hd 5o --- -- 3 1 5m t *K. ow - - - cw - - - ew - - -..-...,2 3 2 2 3 COTERmO V3POhhE DUtTON (MUEC Fig. 15. Shping shorter durtions; step sie incresing (Exp. V, Ss 26, 27, 28). The reltion between criterion nd response durtion. None of the Ss reched criterion in the fourth phse. men durtion 26 msec greter thn criterion. However, numerous unreinforced responses were interspersed nd the criterion of 1 successive responses with durtion greter thn 52 msec ws neverxeched. The questionnire for S24 ws scored negtive wheres tht for S25 ws not. Experiment V Shping Shorter Durtions; Step Sie ncresing. The ttempt to shpe downwrd from 11 S 27 st 4 3 o i 21 S24 ~ S15 2 3 5 25 MSEC Cm, + + cm +..... 2 3 r'' 4O SOO CRTERON RESPONSE DURTON (MSEC) Fig. 14. Shping longer durtions; step sie lrge nd constnt (Exp. V, Ss 24, 25). The reltion between criterion nd response durtion. S24 emitted no reinforced responses in the third phse. S25 emitted 24 reinforced responses, including 1 tht exceeded criterion (rrow) but unreinforced responses were interspersed nd criterion ws not reched. 175 ' 1 71 C. -C,C ego 1 2 3 4 Fig. 16. Frequency distributions of response durtions for S27 (Exp. V).
DFFERENTL RENFORCEMENT 115 the opernt bseline ws successful only over limited rnge. The findings for three Ss re shown in Fig. 15. n phses 2 nd 3 the shping indices for ll three Ss rnged from.5 to 1.5 nd differentil reinforcement ws effective. t the end of phse 3, the men durtion nd vribility were quite smll for ll Ss. This finding is shown grphiclly for S27 in Fig. 16. s result, the shping index for the next phse ws reltively lrge; pproximtely 2.5 for the three Ss. No reinforced responses were emitted by Ss 26 nd 28 in phse 4. Three responses by S27 were reinforced but this did not led to control over the durtion of vocl responding. The questionnires for ll three Ss were scored negtive. REFERENCES Keller, F. S. nd Schoenfeld, W. N. Principles of psychology. New York: ppleton-century-crofts, 195. Lne, H. L. nd Shinkmn, P. G. Methods nd findings in n nlysis of vocl opernt. J. exp. nl. Behv., 1963, 6, 179-188. Peterson, G. E. nd McKinney, N. P. The mesurement of speech power. Phonetic, 1961, 7, 65-84. Received My 16, 1963