Published in Public Opinion Quarterly, 39, 1975, Monetary Incentives in Mail Surveys

Transcription

1 Publishd in Public Opinion Quartrly, 39, 1975, Montary Incntivs in Mail Survys J. Scott Armstrong Th Wharton School, Univrsity of Pnnsylvania Abstract Eightn mpirical studis from fourtn diffrnt rsarchrs provid vidnc that prpaid montary incntivs hav a strong positiv impact on th rspons rat in mail survys. On of ths studis is dscribd hr and an attmpt is mad to gnraliz from all ightn about th rlationship btwn siz of incntivs and rduction in nonrspons. Ths gnralizations should b of valu for th dsign of mail survy studis. Prvious Rsarch On of th mail survys, conductd in 1969, consistd of I00 popl randomly pickd from th Philadlphia mtropolitan-ara tlphon books. 1 Th qustionnair, which was rathr long, includd a dscription of a nw form of transportation calld th Minicar Transit Systm and 61 qustions, som of thm opn-ndd. Fifty of th potntial rspondnts rcivd a dollar honorarium with th qustionnair. Th othr fifty rcivd no mony. Svnty prcnt of th formr group and only 22 prcnt of th lattr group rspondd. Whil th sampl siz is small, th diffrnc btwn th two sub-sampls was statistically significant (p <.05). Tabl 1 [nxt pag] summarizs rsults of this study and th svntn prvious ons. 2 Th conclusions ar rathr clar: (1) prpaid montary incntivs yild larg incrass in rspons rats, and (2) th largr th montary incntiv, th gratr th incras in th rspons rat. Th only xcption to an othrwis prfct ranking is th fiv-cnt incntiv in Kphart and Brsslr's study. 3 This rvrsal is quit likly du to sampling rror. Thus, th rankings wr in th xpctd dirction on 24 of th 25 possibl comparisons: an incras in montary incntivs ld to an incras in rspons rat 96 prcnt of th tim, a statistically significant rsult (sign tst). Unfortunatly, only six of th studis tstd multipl lvls of incntivs, but ths studis also support th hypothsis that th gratr th incntiv, th gratr th rspons rat. It is important to not that all of th studis in Tabl I rfr to prpaid montary incntivs. Th promis of a rward dos not hav as strong an ffct. Hads and Thrift, in a study on a British population, dmonstratd that thr was som incras in rspons rat as th siz of th promisd rward was incrasd. 4 Howvr, as shown in th studis in Tabl 2, th ffct of a promisd incntiv is small in comparison to that of a prpaid incntiv. It is also intrsting to not that th paymnt of an incntiv only on follow-up mailings has an ffct, but on not quit as strong as whn mony is snt in th first mailing. Huck and Glason obtaind a rspons rat of 94 1 Scott Armstrong and Trry S. Ovrton (1971). Brif vs. comprhnsiv dscriptions in masuring intntions to purchas, Journal of Markting Rsarch, 8, Paul L. Erdos (1970), Profssional Mail Survys. Nw York: McGraw-Hill, provids a partial rviw of th litratur. 3 William M. Kphart, and Marvin Brsslr (1958), Incrasing th rspons to mail qustionnairs, Public Opinion Quartrly, 22, Howvr, thr wr only thr comparisons covring th rang from $0.25 to $1.00, a rang that would sm to b of particular intrst. 4 J. Hads, and H. J. Thrift (1966), Nots on a study in postal rspons rats, Commntary, 8,

2 Tabl 1 Mail Rspons Rats for Various Prpaid Montary Incntivs; Entris Rprsnt Prcntag of Sampl That Rspondd Incntiv Study Non $.01 $.05 $.10 $.25 $1.00 Armstrong and Ovrton a Blumbrg t al. b Erdos c Erdos d Erdos Erdos f Hcklr and Bourgtt g Huck and Glason h Hancock i Kphart and Brsslr j Kimball l Malony m Nwman n Nwman o Shuttlworth p Watson q Wisman r Wotruba s a Armstrong and Ovrton, op. cit., n. 1. b Hrbrt H. Blumbrg, Carolyn Fullr, and A. Paul Har (1974), Rspons rats in postal survys; Public Opinion Quartrly, 38, c Erdos, op. cit., n. 2, p. 97. d Ibid., pp Ibid., p. 98. f Ibid.. p. 99. g Jams C. Hacklr and Patricia Bourgtt (1973), Dollars, dissonanc and survy rturns, Public Opinion Quartrly, 37, h Huck and Glason, op cit. I John Hancock (19), An xprimntal study of four mthods of masuring unit costs an obtaining attitud toward th rtail stor, Journal of Applid Psychology, 24, j Kphart and Brsslr, op. cit, n. 3. k This figur is th only xcption to othrwis prfct rankings. l Andrw E. Kimball (1961), Incrasing th rat of rturn in mail survys, Journal of Markting, 25, m Paul W. Malony (1954), Comparability of prsonal altitud scal administration with mail administration with and without incntiv, Journal of Applid Psychology, 38, Th no-incntiv form of Malony had two follow-ups whil th group that got a quartr rward had only on follow-up. n Shldon W. Nwman (1962), Diffrncs btwn arly and lat rspondnts to a maild survy, Journal of Advrtising Rsarch, 2, (2-pag qustionnair) o Ibid. (4-pag qustionnair) p Frank K. Shuttlworth (1931), A study of qustionnair tchniqu; Journal of Educational Psychology, 22, q John I. Watson (1965), Improving th rspons rat in mail rsarch, Journal of Advrtising Rsarch, 5, r Frdrick Wisman (1973), Factor intraction ffcts in mail survy rspons rats, Journal of Markting Rsarch, 10, s Thomas R. Wolruba (1966), Montary inducmnts and mail qustionnair rspons, Journal of Markting Rsarch. 3, k

3 prcnt whn a quartr was snt on th first mailing, 92 prcnt whn th sam amount was snt on first follow-up, 78 prcnt whn it was snt on th scond follow-up, and 71 prcnt whn no mony was snt. 5 Tabl 2 Mail Rspons Rats for Prpaid vs. Promisd Incntivs; Entris Rprsnt Prcntag of Sampl That Rspondd Typ of Incntiv No Promisd Prpaid Study a Incntiv $.25 $.50 $1.00 $.25 $1.00 Blumbrg Hancock Wotruba 10 a S footnots in Tabl I for study sourcs, Estimating th Rlationship Btwn Prpaid Incntivs and Rspons What gnralizations can b drawn about th rlationship btwn prpaid montary incntivs and nonrspons? Considration has bn givn only to thos studis in which th mony was snt with th initial mailing. In ordr to utiliz th prvious studis, th qustion was formulatd as follows: What prcntag rduction in nonrspons might b xpctd from various lvls of prpaid montary incntivs? By stating th problm in trms of th prcntag rduction in nonrspons, on is abl to control for th fact that som studis ar inhrntly mor intrsting to th rspondnts than othrs and thus draw largr rspons rats. (Th rspons rats varid from 10 to 65 prcnt whn no montary incntivs wr providd.) This statmnt of th problm liminatd from th sampl on study, th four-pag on by Nwman, sinc it providd no data for th cas with no montary incntiv. 6 On furthr chang was rquird. Sinc th studis wr drawn from diffrnt tim priods, ranging from 1931 to 1973, it was ncssary to convrt to constant dollars. Th 1973 dollar was usd as th standard hr. 7 In currnt dollars, th incntivs varid from on cnt to on dollar, as shown in Tabl 1. In 1973 dollars, th incntivs varid from 15 cnts to $1.56. This providd substantial variation. Figur 1 provids a summary of th data. Twnty-four obsrvations wr drawn from th svntn studis sinc som of th studis providd data on diffrnt lvl of incntivs. 8 A crud rlationship was obtaind by fitting th data to a curv that was consistnt with th finding notd abov, that ach incrmnt of incntiv lads to an incras in rspons rat. In addition, th following a priori assumptions wr mad: 1. Th curv should go through th origin to rflct, by dfinition, that no rduction in non-rspons is obtaind with no incntiv. 2. Th curv should approach an asymptot. Obviously, th asymptot cannot xcd 100 prcnt. An asymptot of 100 was slctd undr th assumption that "vryon has his pric." Howvr, it should b notd that this dcision was subjct to much uncrtainty. 5 Schuylr W. Huck and Edwin M. Glason (1974), Using montary inducmnts to incras rspons rats from mail survys, Journal of Applid Psychology, 59, (2 wavs). 6 Nwman, op. cit.. Tabl 1. 7 Th Consumr Pric Indx was obtaind from th 1973 Economic Rport of th Prsidnt, and this was convrtd to a January I, 1973, bas. Th yar prior to th publication dal was usd as th yar of th study unlss information was availabl on th yar in which th study was actually don. Excptions wr th Erdos study don in 1952, th othr Erdos studis don in 1956, and th Armstrong and Ovrton study don in 1969, finally, th indx from 1933 was usd for th 1931 publication by Shuttlworth, op. cit., Tabl 1. 8 On rational for trying to assss th rlationship by using diffrnt studis was providd by Wisman, whos study suggstd that th various tchniqus usd to rduc nonrspons bias oprat indpndntly of on anothr; Wisman, F. (1973), "Factor intraction in mail survy rspons rats," Journal of Markting Rsarch, 10,

4 3. Thr should b diminishing marginal rturns: ach additional amount of mony should hav lss of an impact. 4. Th rlationship should b simpl. For this rason, a rlationship was slctd that rquird th stimation of only on paramtr. Figur 1 Prcntag Rduction in Nonrspons vs. Prpaid Montary Incntiv (1973 Dollars) whr On way to captur th abov assumptions is to fit th data to th following functional form: Y is th rduction in non-rspons rat a is th asymptot M is th montary incntiv in dollars, and b is th paramtr to b stimatd. This functional form may b transformd to: Assuming an asymptot of 100-prcnt, Y a a = (1) bm In (a) - In (a - Y) = bm (2) In (100) - In (100 - Y) = bm (3) Th b was thn stimatd via rgrssion analysis using th 24 obsrvations to b 0.6 [missing footnot] Thus, th rduction in th prcntag of nonrspons, R, can b prdictd: = (4) R 0. 6M 4

5 Th curv rprsntd by Equation 4 is shown in Figur I. Th adjustd R 2 btwn th prdictd and actual figurs for Y was only Th poor fit is du largly to th studis by Huck t al. and by Malony. Also, th prdictions obtaind from th curv ar lowr than th actual rductions in non-rspons bias on of th 24 obsrvations. On th avrag, th curv undrstimats by about ight prcnt. This consrvativ bias is spcially tru for small montary incntivs (thos lss than fifty cnts). Whil prpaid montary incntivs ld to a rduction in th nonrspons rat in all 24 cass, and by almost on third on th avrag, th attmpt to rlat this rduction to th montary incntiv was rathr unsatisfactory. Th us of quation 4, along with an 8 prcnt compnsation for bias, ld to only a modst improvmnt ovr a modl that assumd that an incntiv of any siz would rduc nonrsponsc by th avrag of 32 prcnt. Th man absolut prcntag rror (MAPS) was 14.9 for th formr mthod and 11.4 for th lattr. This providd an indx of prdictiv fficincy of about 23 prcnt. 9 Th vidnc for th a priori assumptions bhind quation 4 was not, of cours, strong vn though th data wr consistnt with th hypothsis that ach incras in incntivs lads to a rduction in nonrspons. Furthrmor, Hacklr and Bourgtt spculatd that vry larg incntivs would lad to lowr rspons rats. 10 By dropping this a priori assumption, it is possibl to dvlop a simpl rul of thumb that provids a bttr fit to th data: Thr is a I prcnt dcras in th nonrspons rat for ach on-cnt incras in th prpaid incntiv up to a maximum of prcnt. This rul of thumb rducd th MAPE for th sampl from 11.4 (with th curv) to 9.4. Compard to a prdiction basd solly on th sampl man, th indx of prdictiv fficincy was 37 prcnt. Whn will montary incntivs b most ffctiv? In viw of th wid and unxplaind variations among th studis, it sms worthwhil to control for th dsign of th study and th targt population. Crtainly, mor data of th typ xamind hr would b usful. Mor usful, howvr, would b larg-scal survys, ach xamining (1) a larg numbr of diffrnt incntivs (.g., lvls), and (2) a wid rang of incntivs (.g., on cnt to twnty dollars). Anothr promising rout would b to conduct in-dpth studis on how ths montary incntivs affct popl. This might hlp dtrmin which aspcts of th survy dsign or th targt population ar important. For xampl, on th dsign qustion, it may b mor ffctiv to call th incntiv an honorarium than a paymnt. Summary What hav w larnd aftr four dcads of rsarch on montary incntivs in mail survys? Th only typ of montary incntiv that has an impact on th nonrspons rat is th prpaid montary incntiv (modrat support). Th rductions in nonrspons ar substantial. In this sampl, thr was a rduction of about on-third (vry strong support). Also, w hav larnd that it is vry difficult to quantify th rlationship btwn siz of incntiv and rduction in nonrspons. A rul of thumb that thr is a 1 prcnt dcras in nonrspons rat for ach on cnt incras in th prpaid incntiv up to prcnt sms to b of som hlp (modrat support). Finally, progrss ovr th nxt four dcads is likly to b minimal unlss larg-scal survys and/or in-dpth studis ar dirctd at this issu. 9 This indx of prdictiv fficincy was calculatd by dividing th improvmnt (3.5 in this cas), by th total rror using th sampl man only (14.9). 10 Hacklr and Bourrtt, op. cit., Tabl 1. 5