Addressing empirical challenges related to the incentive compatibility of stated preference methods

Transcription

1 Addressng emprcal challenges related to the ncentve compatblty of stated preference methods Mkołaj Czajkowsk 1, Chrstan A. Vossler 2,, Wktor Budzńsk 1, Aleksandra Wśnewska 1 and Ewa Zawojska 1 The fnal publcaton s forthcomng n Journal of Economc Behavor & Organzaton 1 Department of Economcs, Unversty of Warsaw, Długa 44/50, Warsaw, Poland Correspondng author, cvossler@utk.edu 2 Department of Economcs and Howard H. Baker Jr. Center for Publc Polcy, Unversty of Tennessee, Knoxvlle, Tennessee, USA 1

2 Addressng emprcal challenges related to the ncentve compatblty of stated preferences methods Abstract: An emergng theoretcal lterature focused on the ncentve compatblty of stated preference surveys offers a new lens through whch to vew extant evdence on external valdty, and provdes gudance for practtoners. However, crtcal theoretcal assumptons rest on latent respondent belefs, such as the belef that respondents vew surveys as potentally nfluencng polcy (.e., polcy consequentalty), whch gves rse to pressng emprcal challenges. In ths study, we develop a Hybrd Mxed Logt model capable of ntegratng multple latent belefs, and subjectve measures of these belefs, nto a dscrete choce model of stated preferences. Planned use of a resource, whch can also be consdered a latent varable, s frequently an mportant consderaton when modellng stated preferences for a change n a good, and we demonstrate how our framework can be used to ncorporate ths nformaton smultaneously. We further explore whether smple nformaton treatments, whch vary the degree to whch the potental role of surveys n nformng polcy s emphaszed, can nfluence respondent belefs. Our results suggest that latent belefs over consequentalty, current use and, to a much lesser extent, the nformaton treatments sgnfcantly nfluence elcted wllngness to pay. Keywords: dscrete choce experment; stated preferences; consequentalty; feld experment; hybrd mxed logt model JEL classfcaton: C93, C35, H41, Q51 2

3 1. Introducton Stated preference surveys contnue to be the leadng approach for estmatng the economc value of products not avalable n markets, ncludng proposed publc polces wth passve use value. Although the methodology has been n use for over ffty years, concerns over the ablty of surveys to provde vald welfare measures reman, servng as an obstacle to wdespread adopton n the legal and polcy arenas. Recently, theoretcal work has dentfed condtons for a stated preference survey to be ncentve compatble n the sense that t provdes ncentves for truthful preference revelaton. These condtons for ncentve compatblty rely heavly on latent (.e., unobserved) respondent belefs. For nstance, when a sngle bnary choce (SBC) queston s used, respondents must perceve that the stated cost can be coercvely collected upon mplementaton of the project (.e., payment consequentalty) and that a response n favor of the proposal weakly monotoncally ncreases the chance of ts mplementaton (.e., polcy consequentalty) (Carson and Groves, 2007; Vossler, Doyon and Rondeau, 2012). In addton to these belefs, ncentve compatblty for the ncreasngly popular repeated bnary dscrete choce experment (bnary DCE) requres respondents to beleve that at most one of the proposed polces can be mplemented, and that the perceved mplementaton rule nduces ndependence between choce sets (.e., that a vote on one polcy n one choce set has no effect on the mplementaton probablty of another polcy from another choce set) (Vossler, Doyon and Rondeau, 2012). Our study proposes methods for addressng two of the sgnfcant challenges that arse when undertakng emprcal work that endeavors to satsfy and/or test the theoretcal assumptons ted to respondent belefs. One emprcal challenge s how to ncorporate stated measures of latent/unobservable belefs, such as Lkert-scale responses to a queston about perceved polcy consequentalty, nto models of stated preferences. Drect ncluson of stated measures of belefs may be problematc for two reasons. Frst, stated belefs are measured mprecsely, gvng rse to ssues of measurement error. Second, stated belefs may be correlated wth other unobserved factors that nfluence respondents choces. In pror work, Herrges et al. (2010) develop a Bayesan treatment effect model for SBC data that uses nstrumental varables to dentfy the relatonshp between stated polcy consequentalty and wllngness to pay (WTP). Vossler, Doyon and Rondeau (2012) and Vossler and Watson (2013) brefly menton bnary probt nstrumental varable models, wth the former study suggestng statstcal evdence that measured belefs can be consdered exogenous and the latter ctng a weak nstruments problem. Here, we propose a 3

4 Hybrd Mxed Logt (HMXL) approach, whch models an unobserved belef as a latent varable n a Random Utlty Maxmzaton (RUM) framework, and specfes a measurement equaton where a stated measure of the belef s a functon of the latent varable and an error term, thus recognzng the presence of measurement error. 3 Relatve to pror approaches, the proposed HMXL model can: (1) be appled generally to both SBC and DCE data; (2) accommodate multple latent factors n flexble ways; and (3) ncorporate flexble specfcatons for measurement equatons (e.g., ordered choce, multnomal choce, count data models, etc.). Further, as wth standard mxed logt models, the HMXL allows the analyst to ncorporate varous forms of preference heterogenety. Identfcaton reles on there beng avalable measures of the latent varables, rather than nstrumental varables n the case where stated belefs are drectly ncluded n the choce model. A second challenge, assumng the theoretcal ncentve compatblty condtons ted to belefs are not unversally met, s how to modfy survey desgn to nduce desred belefs (.e., make respondents beleve n real consequences followng from a survey outcome). In most studes, asde from controlled experments, researchers do not have the ablty to manpulate the actual consequentalty of a survey. Further, whether and to what extent a survey s actually consequental s rarely known ex ante. Under these crcumstances, belefs over consequentalty reman mportant theoretcally, and these belefs are lkely nfluenced by many aspects of survey desgn. In ther crtcal revew of the lterature, Klng, Phaneuf and Zhao (2012) pont out that the effect of consequentalty scrpts n stated preference surveys s n ts nfancy. Usng as a case study a bnary DCE survey focusng on the publc fnancng of muncpal theaters n Warsaw, Poland, we employ a splt-sample approach to nvestgate four nformaton scrpts that vary n ther emphass of polcy consequentalty. 4 The baselne treatment provdes nformaton at a level that s common n stated preference surveys, wth addtonal emphass placed n the other treatments. As an ancllary beneft, ths exogenous varaton allows us to dentfy whether there s a causal effect of polcy consequentalty on elcted values. As acknowledged n pror work, follow-up consequentalty questons are themselves 3 Many researchers state that hybrd choce models address general endogenety ssues (e.g., Daly et al., 2011), such as may arse when unobservables underlyng a stated belef measure are correlated wth the choce model errors. Budzńsk and Czajkowsk (2017) undertake a Monte Carlo analyss, demonstratng that ths s not necessarly the case, and propose an extenson to the HMXL that allows errors to be correlated across equatons. 4 Our applcaton s of potental nterest n ts own rght, as few non-market valuaton studes have examned the value of the performng arts (Forrest, Grme and Woods, 2000; Hansen, 1997; Wlls and Snowball, 2009; Grsolía and Wlls, 2010; Grsolía and Wlls, 2012; Wlls et al., 2012). 4

5 nconsequental constructs. Ths opens up the possblty that dentfed correlatons may be spurous and, smlarly, that drvers of responses to consequentalty questons may have lttle to do wth actual belefs. In the feld survey context, and n a smlar sprt to our research, a few pror studes exogenously vary nformaton provded to respondents wth the ntent of alterng perceptons over polcy consequentalty. 5 Bulte et al. (2005) fnd that provdng a statement that alerts respondents that the results of the study wll be made avalable to polcymakers, and could serve as a gude for future decsons decreases WTP. Oehlmann and Meyerhoff (2017) and Drchouts et al. (2015), n contrast, observe no WTP changes resultng from the ncluson/excluson of a consequentalty scrpt. Herrges et al. (2010) make use of a publshed artcle that provded evdence that survey results drectly affected related polcy decsons n the past, and fnd that stated belefs over polcy consequentalty ncrease when respondents are provded ths nformaton. All of these studes used SBC elctaton, and as such our exploraton provdes prmary evdence on ths type of nducement for DCE surveys. Further, the scrpts we explore can further be easly ncorporated nto general practce. Indeed, t s presumably rare to have relevant meda coverage avalable that provdes a clear thrd-party lnk between surveys and polcy. An ancllary beneft of the HMXL framework n ths context s that t allows one to not only measure whether nformaton treatments alter stated belefs but also whether such treatments nfluence stated WTP. The emprcal study provdes several mportant nsghts. Frst, smlar to Vossler and Watson (2013), for our applcaton we are not able to dentfy (strong) nstrumental varables from the extensve nformaton collected through the survey. 6 Ths provdes further mpetus for the proposed HMXL estmator. Second, we fnd that latent belefs over polcy consequentalty have a dscernble effect on elcted WTP for the proposed dscounted theater tcket programs. Importantly, these latent belefs are strongly correlated wth stated belefs, usng as the measurement devce a Lkert-scale polcy consequentalty queston now prevalent n the 5 A handful of controlled experment studes exogenously vary the level of actual polcy consequentalty, for example, by manpulatng the probablty a vote s bndng (Landry and Lst, 2007; Mtan and Flores, 2012; Carson, Groves and Lst, 2014), or by ntroducng treatments the vary the proporton of respondent and regulator votes (Collns and Vossler, 2009; Vossler and Evans, 2009). 6 In partcular, we estmated a verson of the mxed logt model that ncorporates a control-functon approach to deal wth the potental endogenety of stated perceptons of polcy consequentalty. We refer the nterested reader to Guevara and Ben-Akva (2010) for a comparson of control-functon and latent-varable approaches. 5

6 lterature. 7 Thrd, WTP s sgnfcantly correlated wth the nformaton treatments, whch emphaszes the emprcal mportance of the theoretcal assumpton regardng polcy consequentalty; ndeed, ths can be taken as evdence n favor of construct valdty. 8 Fourth, somewhat surprsngly, the nformaton treatments have no sgnfcant effect on stated belefs. Thus, although the econometrc results provde emprcal support that a responses to a followup queston about polcy consequentalty carres useful nformaton, the fndngs emphasze the mportance of developng follow-up questons that elct belefs more precsely. We further hghlght the need to use multple belef elctaton questons n order to help dscrmnate underlyng motves. Our econometrc framework can be appled to such an nvestgaton. 2. Econometrc approach the Hybrd Mxed Logt model Hybrd choce models are very flexble tools that allow analysts to ncorporate perceptons and cogntve processes nto a RUM model framework. They thus provde a lnk between behavoral scences (e.g., psychology) and felds orented on estmaton such as engneerng and economcs. As stated n Ben-Akva et al. (2002), hybrd choce models (also known as Integrated Choce and Latent Varable models) are a general class of models whch may nclude addtonal random dsturbances n the form of error components or random parameters, psychologcal factors n the form of latent varables and latent classes, wth possble non-rum decson processes n some of them. Of course, latent varables do not need to be lmted only to psychologcal constructs, but can also be used for other features whch are not straghtforward to measure, such as socal nteractons (Kamargann, Ben-Akva and Polydoropoulou, 2014), socal nfluences (Km, Rasoul and Tmmermans, 2014) or response qualty (Hess and Stathopoulos, 2013). In ths study we develop a HMXL model whch combnes the framework wdely adopted for analyzng DCE data, the Mxed Logt (MXL) model (Greene, 2011), wth the Multple Indcators and Multple Causes (MIMIC) model (Jöreskog and Goldberger, 1975). 7 There are multple possble nterpretatons of what latent varables n hybrd choce models are actually capturng. Followng the lterature, and for ease of exposton, we nterpret a latent varable n a manner consstent wth the (observed) varable that we assume provdes a measurement of the latent varable; e.g., the latent varable we measure through stated responses to a polcy consequentalty queston s labelled latent consequentalty. In the Dscusson secton we elaborate on ssues of nterpretaton. 8 Gven we do not have a crteron measure of actual demand for the proposed programs, we can make no clam that condtonng WTP estmates on the latent belef or on the nformaton treatments produces more accurate or externally vald measures. 6

7 Connectng dscrete choce models wth a MIMIC model s an emergng approach for ncorporatng psychologcal factors n the RUM framework. Most of the applcatons to date appear n the transportaton lterature (e.g., Vredn Johansson, Heldt and Johansson, 2006; Hess, Hensher and Daly, 2012; Dazano and Bolduc, 2013). Applcatons n the economcs lterature nclude Hess and Beharry-Borg (2012), who ncorporate latent atttudes towards coastal water qualty protecton n a model of stated preferences regardng watery qualty mprovements n Tobago, and Dekker et al. (2016), who treat preference uncertanty as a latent varable n a choce model of flood rsk polces n the Netherlands. Further, Hoyos, Marel and Hess (2015) employ latent constructs to explan preferences regardng land management n Span, and Czajkowsk, Hanley and Nyborg (2017) use the approach for nvestgatng motves for household recyclng. Vj and Walker (2016) analyze the possble advantages of employng a hybrd framework and provde general crtera for assessment of whether ts use s justfed. Correspondng wth ther second crteron, n our settng we use the HMXL n order to fnd a relatonshp between survey consequentalty and respondents preferences, whch can be used to nform polcy and practce. In the context of our applcaton, we assume there are two latent factors n the dscrete choce model. The frst s a psychologcal factor assumed to explan responses to a Lkert-scale queston gaugng belefs over polcy consequentalty. For ease of exposton, we nterpret ths factor as latent consequentalty. The second factor s assumed to explan the varaton n past vsts to local theaters, whch s presumably an mportant underlyng drver of demand for proposed programs, whch we nterpret as latent theater use. The frst latent factor s of course motvated by theory work related to stated preferences, whereas the second captures heterogenety from possble changes n theater tcket demand due to proposed dscount tcket programs. Hybrd choce models can consst of up to three parts: a dscrete choce model, structural equatons, and measurement equatons. Below we descrbe each part n turn. In dong so, we descrbe the general framework as well as provde estmaton detals relevant for our partcular applcaton Dscrete choce model The theoretcal foundaton for the dscrete choce model s RUM theory, whch assumes that the utlty a person derves depends on observed characterstcs of choces and unobserved 7

8 dosyncrases, represented by a stochastc component (McFadden, 1974). As a result, ndvdual s utlty from alternatve j n choce set t can be expressed as: V a c bx e, (1) jt jt jt jt where utlty s assumed to be addtvely separable n the cost of the alternatve, c jt, and other attrbutes, X jt ; a and b denote estmable parameters; and e jt s a stochastc component allowng for factors not observed by the econometrcan to affect utlty and choces. There are two aspects of the specfcaton to emphasze. Frst of all, a and b are ndvdual-specfc, thus allowng for heterogeneous preferences amongst respondents and motvatng the MXL model. Assumng nstead that parameters are the same for all respondents mples homogenous preferences and results n the multnomal logt (MNL) model as a specal case. Second, the stochastc component of the utlty functon ( e jt ) s of unknown, possbly heteroskedastc varance e 2 var jt s normalzng ths varance, such that the error term wth constant varance 2. Identfcaton of the model typcally reles on jt ejt s..d. type I extreme value 6s var 6. Ths leads to the followng specfcaton: jt U jt jt jt jt a c b X. (2) Note that due to the ordnal nature of utlty, specfcaton (2) represents the same preferences as (1). The estmates a and b do not have a drect nterpretaton anyway, but f nterpreted n relaton to each other the scale coeffcent ( 6s X jt ) cancels out. Fnally, gven the nterest n establshng estmates of WTP for the non-cost attrbutes,, t s convenent to ntroduce the followng modfcaton whch s equvalent to usng a money-metrc utlty functon (a.k.a. estmatng the parameters n WTP space; see Scarpa, Thene and Tran, 2008): U a b c X c βx. (3) jt jt jt jt jt jt jt a Under ths specfcaton the vector of parameters nterpreted as a vector of mplct values for the attrbutes, β s now scale-free and can be drectly X jt. In MXL models, an addtonal advantage of ths formulaton s that the econometrcan can specfy a partcular dstrbuton of 8

9 WTP n the populaton, by defnng the dstrbuton of β, rather than the dstrbuton of the underlyng utlty parameters, b. 9 We assume there s a vector of ndvdual-specfc latent varables, denoted by LV, that depend on the random parameters of the utlty functon. The functonal form of ths dependence may vary due to dstrbutonal assumptons. In the analyss we use two dstrbutons, normal (for all non-cost attrbutes) and log-normal (for the cost attrbute). For a normally dstrbuted β, ths dependence s of the form: β ΛLV ΞSD β, (4) where Λ and Ξ are matrces of estmable coeffcents and β has a multvarate normal dstrbuton wth a vector of means and a covarance matrx to be estmated. 10 Ths specfcaton allows for ncluson of soco-demographc or other drectly observable varables (such as dfferent treatments n the survey) n the vector coeffcent depends on latent varables n the followng way: SD. Smlarly, we assume that the cost exp τlv ζsd, (5) where τ and ζ are vectors of estmable coeffcents and follows a normal dstrbuton wth the parameters descrbng ts mean and standard devaton to be estmated. 11 As a result, the condtonal probablty of ndvdual s choces y, for all T choce tasks, s gven by: P y X, β,,,, LV Λ Ξ, τ, ζ, θ T C t1 k 1 exp exp c where θ s a vector of parameters on whch and β depend. jt c βx kt jt βx kt, (6) 9 As translatng utlty parameters nto a money equvalent requres dvdng them by a (possbly also random) cost coeffcent, ths can mply what are often mplausble assumptons about the dstrbuton of WTP (Carson and Czajkowsk, 2013). 10 The number of columns n s equal to the number of latent varables, and the number of rows s equal to the number of non-cost attrbutes. 11 can also be correlated wth β. 9

10 2.2. Structural equatons Latent varables can depend on exogenous factors, such as soco-demographc varables, whch are stacked n the vector str X. Vector relatonshp s descrbed by the followng structural equatons: str X may, n prncple, overlap wth vector SD. Ths LV ΨX ξ, (7) str wth a matrx of coeffcents Ψ and error terms ξ whch are assumed to come from a multvarate normal dstrbuton. 12 Generally, lnkng soco-demographc varables wth latent varables through structural equatons s not necessary. In the absence of such structural equatons, latent varables become smlar to random parameters they capture the correlaton between ndvduals preferences and measurement varables. In order to make dentfcaton of hybrd choce models possble, the scale of every latent varable needs to be normalzed (Hess, Hensher and Daly, 2012). Ths can be done by normalzng varances of the error terms n the structural equatons or by normalzng some coeffcents for each latent varable n the measurement equatons. Raveau, Yáñez and Ortúzar (2012) conducted a smulaton study whch ndcates that normalzng varances leads to better convergence of the models and results n the recovered estmates beng closer to the underlyng data generatng process. Normalzng the varances of the error terms s thus the approach we adopt here. Contrary to most studes conducted to date, we do not normalze the varance of ξ to one. Instead, we use normalzaton to assure that the varance of every latent varable n s equal to one. Although such an approach ntroduces addtonal nonlneartes nto the model, t s very useful, as now all latent varables have the same scale (even wth soco-demographc varables n structural equatons) and therefore ther relatve mportance (e.g., n measurement equatons) can be easly assessed. We do not observe any addtonal ssues wth convergence due to ths normalzaton. LV Formally, we defne LV Ψ ' X ξ, wth str Ψ beng a matrx of parameters to be estmated and ξ beng a vector of ndependent normally dstrbuted varables wth mean zero and unt standard devaton. Wth LV k representng a vector of values of the 12 Ths s a common assumpton, although Bhat, Dubey and Nagel (2015) ntroduce a specfcaton that allows for non-normal error terms. 10

11 k-th non-normalzed latent varable for all ndvduals, and k std k LV ts standard devatons, we have LV LV, k k k Ψ Ψ and k k k ξ ξ. 13 k k k 2.3. Measurement equatons The man purpose of ncludng latent varables n the models s that they descrbe psychologcal or other factors that cannot be measured n a drect way, unlke other ndvdual characterstcs such as age and gender. Instead, a researcher must use varous ndcator questons n a survey, responses to whch are hypotheszed to be determned by the latent varables. The model choce for the ndcator equatons depends on the partcular applcaton. The measurement equatons can be lnear, ordered, bnary, multnomal or count regressons whatever fts an nterpretaton of each ndcator best. In ths study we nclude one ndcator for the latent belef over the polcy consequentalty of a survey, measured on a fve-pont Lkert scale. The measurement equaton s modelled usng ordered probt. We nclude a second ndcator, whch s a count varable of pror theater vsts. We model ths as a Posson regresson. functon: Frst, consder the case of the ordnal ndcator. We can specfy the followng ndex I ρlv υx, (8) Mea 1 1 where ρ 1 and υ 1 are vectors of coeffcents and denotes an error term assumed to have a normal dstrbuton wth zero mean and unt standard devaton. 14 Mea X s a vector of socodemographc varables, whch drectly explan the ndcator varable. Such a formulaton allows for greater flexblty of the model and may provde addtonal nsght nto how socodemographc varables nfluence respondents choces modeled through the HMXL. For our applcaton, n the frst measurement equaton, we defne I as the ndcator for polcy consequentalty for ndvdual. Polcy consequentalty s measured through a Lkert- Ψ denotes k-th row of Ψ matrx, and 13 k equaton for all ndvduals. ξ k denotes stacked values of the random term n the k-th structural 14 It s mportant to note that the number of measurement equatons does not need to equal the number of latent varables. For nstance, cases may arse where more than one ndcator for a latent varable s avalable (e.g., there may be two survey questons measurng belefs over polcy consequentalty). Ths framework can accommodate such a settng by specfyng multple measurement equatons for a sngle latent varable. 11

12 scale queston to gauge the degree of perceved polcy consequentalty, gvng rse to an ordnal measurement varable that takes on fve values. Accordngly, we adopt a model of ordered choces. 15 Under ths specfcaton, we defne the relatonshp between I and I I I I I 1 f 1 1 I 2 2 I 3 3 I 4 4 I 2 f 3 f 4 f 5 f I I as follows:, (9) where the s are the threshold parameters to be estmated. Assumng a normal dstrbuton for, ths leads to the well-known ordered probt lkelhood form for I : str Mea Mea Mea,,, 1, 1, l 1 ' 1 ' l 1 1 ' 1 ' P I X ξ X ρ υ α ρ LV υ X ρ LV υ X, (10) where denotes the normal cumulatve dstrbuton functon and I l. 16 For the second measurement equaton, the dependent varable s a count of past theater attendances, denoted as J. Employng a Posson model, we defne Mea 2 2 exp ρ ' LV ν ' X, where ρ2, υ 2 are sets of vectors to be estmated, and the correspondng probablty s: P J str Mea,,, 2, 2 J exp X ξ X ρ υ. (11) J! 2.4. Hybrd Mxed Logt Model estmaton Fnally, we combne the dscrete choce model specfed n (6), the structural equatons descrbed n (7) and the measurement equatons defned n (10) and (11) to obtan the fullnformaton lkelhood functon for our HMXL model (for ease of exposton, we stack the parameter vectors Λ, Ξ, τ, ζ, θ, Ψ, ρ1, υ1, ρ2, υ2, α nto the sngle vector Ω ): 15 Many early hybrd choce model applcatons used a smple, lnear regresson even n cases where the dependent varable was clearly ordered (e.g., Daly et al., 2012). Although we specfy ths measurement equaton as an ordered probt, nothng precludes one from assumng a dfferent error dstrbuton, gvng rse to alternatve ordered choce models (e.g., ordered logt). 16 We assume 0 and 5. 12

13 As random dsturbances of str str Mea,, β,,,,,, str Mea X, ξ, X, Ω β,, ξ θ β,, ξ L P y X X ξ Ω P I X ξ X Ω P J f d,. (12) β and (non-normalzed) error terms n structural equatons are not drectly observed, they must be ntegrated out of the condtonal lkelhood. Ths multdmensonal ntegral can be approxmated usng a smulated maxmum lkelhood approach. 17 As can be seen, we use one-step estmaton. Ths approach has two man advantages over a two-step (or mult-step) method. Frst, t s more effcent and, second, t allows for dentfcaton of more flexble specfcatons because t has more degrees of freedom. Unfortunately, the exact condtons for HMXL dentfcaton are not known ths depends on the number of latent varables and measurement equatons. We follow Bollen and Davs (2009) to ensure the necessary condton for dentfcaton of structural equaton models holds; n partcular, our specfcaton satsfes the 2+ emtted paths rule (we assume that each latent varable has exactly one unque ndcator n the measurement equaton and s lnked wth sx preference parameters n the dscrete choce component). In addton, we tested our model usng smulatons we generated artfcal datasets and valdated our model by recoverng the underlyng parameters. Our model encountered no problems n dentfcaton and produces 18, 19 stable results. ξ 17 Our model assumes no correlatons between error terms n the measurement, structural and choce components. Ths assumpton has been relaxed n some related albet smpler models (e.g., Bhat, Varn and Ferdous, 2010), and t s possble that allowng for some of these correlatons can mprove model performance or better address specfcaton ssues. The tradeoff n ntroducng correlatons, however, s that ths mposes estmaton challenges to what s already a complex estmaton problem. 18 Econometrc models estmated usng maxmum smulated lkelhood are known to be relatvely senstve to startng values, optmzaton technques and selecton of convergence crtera. Our model s no excepton n ths respect and to make sure we reached the global maxmum n optmzaton, we used dfferent optmzaton methods, derved gradents analytcally and used multple startng ponts. In addton, snce usng longer low-dscrepancy sequences (as opposed to shorter sequences or usng pseudo-random draws) s found to facltate reachng the global optmum or revealng dentfcaton problems (Chou and Walker, 2007; Czajkowsk and Budzńsk, 2015) n smulaton of the log-lkelhood functon, we used 10,000 Sobol draws wth a random lnear scramble and a random dgtal shft. 19 The models were estmated usng a DCE package, whch among other thngs can be used to estmate HMXL models. The package has been developed n Matlab and s avalable at The code and data for estmatng the specfc models presented n ths study are avalable from 13

14 3. Stated preference survey and nformaton treatments 3.1. Survey desgn Our applcaton focuses on the publc fnancng of muncpal theaters n Warsaw, Poland. The survey scenaros descrbe proposed programs for dscounted theater tckets for Warsaw ctzens. In partcular the consdered programs would result n a unform tcket prce of 5 Polsh złoty (PLN) for a sngle admsson, 20 for up to four dfferent categores of theaters defned by the type of productons they offer. Entertanment theaters stage lght comedes and musc shows. Drama repertory theaters am at combnng readng classcs wth ambtous comedes. Chldren s theaters are focused on the youngest audence, whle expermental theaters reject typcal forms to ntroduce an alternatve perspectve nto the performng arts. The proposed payment vehcle s an addtonal annual tax leved on ctzens of Warsaw. Our use of a coercve payment vehcle s mportant from an ncentve standpont. Moreover, as all of about 50 muncpal theaters n Warsaw are already subsdzed by the cty on the order of 40 to 60%, the scenaros can be seen as ncreasng the provson of a publc good. The cost amounts consdered are 10, 20, 50 and 100 PLN per year. The development of the survey was nformed by n-depth ntervews and a plot study, whch confrmed respondents found t easy to dstngush between theater types, and that the tax payment vehcle was credble. Moreover, the majorty of respondents had vsted one or more types of theaters, suggestng that the possble polces descrbed n the survey were straghtforward to understand. A detaled descrpton of the development of the study, the polcy context and polcy relevance of the results are provded n Wśnewska and Czajkowsk (2017). Wth the objectve of estmatng the demand for programs nvolvng varyng opportuntes for dscounted theater tckets, we adopt a bnary DCE approach. Specfcally, each choce set ncludes the status quo (no program of theatre tcket reducton mplemented) and one polcy alternatve. 21 Fgure 1 presents an example choce set. 22 Wth four theater categores and 20 Durng the tme frame the survey was admnstered, 1 PLN 0.25 EUR 0.33 USD. 21 Vossler, Doyon and Rondeau (2012) dentfy condtons for ncentve compatblty of ths elctaton approach, and through ther complementary feld experment fnd favorable evdence of external valdty condtonal on polcy consequentalty holdng emprcally. However, gven the scant evdence avalable, there s not yet consensus regardng external valdty of ths preference elctaton format. 22 The mean number of theater vsts of the respondents n the 12 months before the survey was almost three, wth approxmately 20% of the sample not havng vsted a theater. Ths leads us to beleve that respondents were famlar wth the status quo, such as current tcket prces. 14

15 cost, there are fve choce set attrbutes. Wth two levels for each theater attrbute (.e., the reduced tcket prcng s ether offered or not), and four possble cost amounts, the assocated full factoral desgn gves rse to = 64 choce sets. The polcy alternatves were generated usng Ngene software followng a Bayesan (medan) D-effcent desgn optmzed for the MNL model (Scarpa and Rose, 2008), wth prors for the choce parameters obtaned from a plot study admnstered to a sample of 119 respondents. In partcular, we generated three blocks of 12 choce sets, and respondents were randomly assgned one of the blocks. To control for order effects, we randomzed across respondents the order the choce sets were presented as well as the order n whch the theater attrbutes appeared n the choce set. The survey begns wth warm-up questons gaugng respondents nterest n culture and partcpaton n cultural events, ncludng past vsts to Warsaw theaters. Ths s followed by a dscusson of possble dscounted theater programs, ncludng descrptons of the four theater types, the proposed fundng mechansm and a typcal remnder about budget constrants. A sequence of 12 choce tasks s then presented, where respondents are nstructed to ndcate ther preference between the proposed program and the status quo for each. At the end of the survey, partcpants are asked to ndcate ther belefs concernng polcy consequentalty (hereafter, stated consequentalty), n addton to typcal follow-up questons (e.g., gaugng motvatons for choces, atttudes towards the fnancng of Warsaw theaters, etc.). The wordng of the polcy consequentalty queston s (translated from Polsh): To what extent do you agree wth the statement that the results of the survey wll nfluence future decsons regardng fnancng muncpal theaters n Warsaw?. The level of respondents agreement s measured on a fve-pont Lkert scale (from 1 defntely dsagree to 5 defntely agree ) Expermental nformaton treatments In a feld settng such as ours, t s not possble to vary the actual consequentalty of the survey as t pertans to ts actual role n formulatng polcy. Instead, to the extent that survey desgn can nfluence belefs over consequentalty, we ntroduce four treatments to examne the effect of consequentalty scrpts, whch vary the degree of emphass placed on the potental role that the survey plays n nformng polcy regardng dscounted theater programs. Table 1 presents 15

16 the nformaton provded n the varous treatments. Each respondent partcpated only n one, randomly assgned treatment. Consstent wth survey desgn prncples (Dllman, Smyth and Chrstan, 2014), t s common for surveys to emphasze the mportance of the ndvduals response, as well as the socal usefulness of the survey, whch often ncludes statements that results wll be provded to government offcals. In other words, common survey practce nvolves usng language that may promote belefs of polcy consequences. In accordance wth standard practce, regardless of a treatment, the preamble scrpt ndcates that the survey s beng conducted by researchers of the Unversty of Warsaw and that the purpose of the survey s to collect nformaton on opnons regardng the fnancng of Warsaw theaters. Treatment 1 (T1) does not provde any addtonal nformaton. The survey nstrument n T2 states at the very begnnng of the survey that respondents choces may nfluence future polces. As such, ether T1 or T2 s lkely to characterze the typcal wrtten (or verbally artculated) nformaton related to polcy consequentalty provded n most stated preference surveys. The remanng treatments, T3 and T4, further emphasze potental polcy nfluence. Smlar to common advertsng technques, these treatments nvolve repetton and alternatve phrasng n attempt to ncrease awareness and mportance. In addton to the nformaton provded n T2, and n the context of provdng background nformaton relevant for the proposals, T3 remnds respondents of possble tes to actual polcymakng. T4 ncludes all the nformaton from T3 and adds an nducement mmedately before the value elctaton questons and after the budget constrant remnder. We note that none of the nformaton provded s deceptve. Relevant nsttutes (e.g., Offce of Culture of Warsaw) were made aware of the study and our results were dssemnated to them Survey mplementaton The survey nstrument s the outcome of extensve pretestng ncludng ndvdual ntervews wth potental survey respondents (verbal protocols) and a plot study performed on a group of 119 Warsaw ctzens. The survey was admnstered by a professonal pollng agency usng Computer Asssted Web Intervews. Screenng questons were used to restrct respondents to nclude only adult ctzens of Warsaw who lve and pay taxes n Warsaw. Quotas were mplemented n order to obtan a close match of the treatment subsamples wth the adult populaton of Warsaw wth respect to gender, age, educaton and household sze. 16

17 Data was collected n February and March In total, there are 1,700 respondents. 23 Detaled soco-demographc characterstcs for each treatment sample and the adult populaton of Warsaw are presented n Table 2. As evdent from the table, the targeted samplng strategy worked well to match the survey sample and populaton n terms of key demographcs. However, snce we cannot rule out mportant dfferences n unobservable factors between the sample and the populaton, we make no clam regardng the sample representatveness. We nevertheless have clean dentfcaton of nformaton treatment effects gven random treatment assgnment. 4. Results Usng data from the stated preference survey, we apply the HMXL model detaled n Secton 2. The structure of the model s llustrated n Fgure 2 and the estmaton results for the prmary specfcaton are presented n Table Overall, the model explctly ncorporates the lnk between the nformaton treatments, stated consequentalty, two latent varables and respondents observed preferences, and possbly also ther soco-demographc characterstcs nto one, jontly estmated, framework. The frst latent varable consdered, latent consequentalty, s the belef over polcy consequentalty of the survey. The second latent varable, latent theater use, represents respondents unobserved profle of theater use as t pertans to proposed scenaros. The ordered probt measurement equaton lnks latent consequentalty wth respondents stated consequentalty, whle the Posson regresson lnks latent theater use wth the reported number of theater vsts n the past 12 months As reported by the pollng agency, the response rate to our survey was 29%. 24 It s somewhat common n the lterature to modfy analyses based on protest respondents, such as those who ndcate an unwllngness to pay any amount n response to some unantcpated characterstc of the valuaton exercse, such as ncomplete nformaton. Adoptng common conventon n expermental economcs, we do not elmnate partcpants all were treated as legtmate respondents and left n the sample. We note that 16.18% of respondents consstently selected the no-cost status quo alternatve; however, only 3.47% could be classfed as protest zero responses when takng ther stated motves nto account. As a robustness check, Model 1.5 n the Onlne Appendx (avalable at presents the results from our prmary specfcaton when these possble protest responses are removed. 25 Model 1.1 n the Onlne Appendx presents a parallel model where nstead of controllng for the nfluence of the theater use profle va a latent varable, we nclude two ndcators of theater use drectly (whether a respondent made any theater trp at all and the log of the number of theater vsts f a respondent made at least 1 trp). We fnd ths specfcaton to outperform many other non-lnear drect use specfcatons. None of the alternatve specfcatons qualtatvely change our results. 17

18 The dscrete choce model component of the HMXL explans stated choces from the bnary DCE as a functon of the attrbutes, latent consequentalty, latent theater use and the nformaton treatments. Gven that each of the DCE attrbutes has two levels, we use bnary ndcators for dscounted theater tckets n a gven theatre type. An ndcator, status quo, s ncluded to allow for dfferences n unobserved utlty assocated wth choosng the status quo opton relatve to a proposed program. The latent varables and the nformaton treatments appear as lnear functons of the random coeffcents ted to the polcy attrbutes and the status quo ndcator, as well as n an exponental functon defnng the random coeffcent. The varaton n nformaton treatments s collapsed nto a sngle lnear varable and normalzed (.e., the orgnal varable takng on the values 1 through 4, ndcatng the treatments from T1 to T4, respectvely, s normalzed for zero mean and unt standard devaton), and henceforth referred to as consequentalty scrpt. 26 The frst panel of Table 3 presents the results for the ordered probt measurement equaton for stated consequentalty. In addton to the estmates of the thresholds of the ordered probt model, we fnd that latent consequentalty sgnfcantly ncreases respondents reported level of agreement wth the statement that the survey results wll ndeed be used n desgnng future polces. Ths result thus suggests there s a causal lnk between latent belefs over polcy consequentalty and stated assessments of t. At the same tme, we fnd an nsgnfcant relatonshp between consequentalty scrpt and stated consequentalty. Congruent wth ths result, as llustrated n Table 7, the raw data suggests that the nformaton treatments dd not systematcally alter stated belefs. In partcular, usng a Pearson ch-squared test, we fal to reject the null hypothess that the frequency dstrbutons for the four treatments are equal (χ 2 (12) = 6.759, p = 0.873). In the second panel of Table 3, the results for the Posson measurement equaton for reported theater trps are reported. We observe that latent theater use s postvely correlated wth the reported number of theater vsts, thus lendng tself to beng nterpreted as unobserved theater use ntensty Model 1.2 n the Onlne Appendx presents a parallel model where nstead of treatng consequentalty scrpt as a contnuous varable, we nclude the nformaton treatments nto the choce model as separate treatment-specfc ndcators. The results justfy the use of a sngle (lnear, normalzed) varable, whch allows for easer nterpretaton of our results. Ths specfcaton does not qualtatvely change our results. 27 We consder the possblty that the latent varable we label as latent consequentalty n fact captures some other latent construct, whch would also manfest tself by gvng hgher scores to the queston about survey consequentalty. One thng we can test for wth our data s whether beng a theater enthusast (an actve user) has 18

19 The thrd panel of Table 3 reports the results of the dscrete choce (MXL) component of the model. The model s specfed n WTP space and, accordngly, preference parameters can be drectly nterpreted n terms of WTP dfferences, n 100 PLN. 28 Notng that the expected value of latent consequentalty and consequentalty scrpt are normalzed to equal zero, the man effect for a partcular theater type s the estmate of the mean change n WTP assocated wth ncluson of the theater type n the proposed program. Specfcally, the model suggests that respondents, on average, are wllng to pay the most 33 PLN more per year for a program that ncludes dscounted tckets to entertanment theaters, followed by 21 PLN for drama repertory theaters and about 10 PLN for chldren s and expermental theaters each. The man effect correspondng wth the status quo opton s less sgnfcant, although the large and sgnfcant estmated standard devaton of ths random parameter suggests substantal heterogenety n ndvdual preferences n ths respect. In fact, all estmated standard devatons are both relatvely large and statstcally dfferent from zero, provdng evdence of sgnfcant preference heterogenety and, accordngly, justfcaton for the mxed logt relatve to the restrctve multnomal logt. From the perspectve of evaluatng the nfluence of consequentalty on respondents observed preferences, the most nterestng results are ted to the effects that latent consequentalty and consequentalty scrpt have on the WTP for program attrbutes. For all theater types, the nteractons wth latent consequentalty are statstcally sgnfcant and postve, ndcatng that a stronger belef n polcy consequentalty s correlated wth hgher WTP for these attrbutes. Stronger latent belefs over polcy consequentalty has the addtonal effect of decreasng the scale of the unobserved component of utlty. The nteractons wth consequentalty scrpt show that emphaszng the potental role surveys have n shapng new polces sgnfcantly ncreases WTP for entertanment theaters and drama theaters, decreases the scale of unobserved utlty and has a weak effect for reducng such an effect. To explore ths, we use the responses about ndvduals past use of theaters as an explanatory varable n the measurement equaton for stated consequentalty ether through the ncluson of latent theater use (model 1.3 n the Onlne Appendx) or drectly, usng two ndcators of theater use (model 1.4). The latter specfcaton performs better. We fnd that respondents who are more actve theater users tend to respond that they perceve the survey as more consequental. Even wth ths effect controlled, however, our man results reman qualtatvely the same. 28 The cost parameter nstead represents the scale of the unobserved component of utlty a 19 and hence does not have a money-metrc nterpretaton. The cost parameter s assumed be log-normally dstrbuted, but the reported results are the mean and the standard devaton of the underlyng normal dstrbuton, whch are expressed n negatve 100 PLN.

20 the preferences for the status quo alternatve. These qualtatve fndngs thus largely mrror those based on latent consequentalty. However, the magntudes of the effects of nformaton sgnals are much smaller. One nterpretaton of ths result s that belefs over polcy consequences may largely be homegrown and entrenched, provdng lttle room for the researcher to sgnfcantly nfluence them. Further, gven our earler fndng of no sgnfcant correlaton between stated consequentalty and consequentalty scrpt, the mplcaton here s that the Lkert-scale queston dd not capture adequately the effects of the nformaton treatments. One possblty s that the fve-pont Lkert scale we used may not be precse enough to dentfy mportant dstnctons. To place the effects of the latent belefs and the nformaton treatments nto better perspectve, Tables 4, 5 and 6, and accompanyng Fgure 3 provde the smulated WTP changes assocated wth consequentalty scrpt, stated consequentalty and latent consequentalty, respectvely. The results ted to latent consequentalty are qute dramatc. Even when comparng the WTP for program attrbutes across respondents n the 25 th and 75 th percentles, WTP ncreases by a factor of two for chldren s theaters, by a factor of three for drama theaters, by a factor of fve for entertanment theaters and by a factor of four for expermental theaters. Although stated consequentalty does not drectly appear n the dscrete choce component of the model, through smulaton we can determne how WTP vares based on stated belefs over polcy consequentalty. These results are provded n Table 5. Overall, the results suggest that consderable nformaton on preferences s conveyed through ths smple ndcator for polcy consequentalty. Indeed, WTP for program attrbutes s negatve or statstcally zero at the two lowest levels of stated consequentalty and become postve and larger for hgher levels. Ths postve relatonshp between stated consequentalty and elcted WTP mrrors a common but not unversal fndng n the lterature (e.g., Herrges et al., 2010). Smlar to Vossler, Doyon and Rondeau (2012), we fnd a contnuous relatonshp, rather than the knfeedge dstncton between those wth nconsequental belefs and those wth consequental belefs (regardless of ntensty). Such a knfe-edge result s predcted by the theory and found n the feld survey studes of Herrges et al. (2010) and Vossler and Watson (2013). One possble explanaton for ths dvde s the elctaton format;.e., the latter two studes nvolved SBC rather than bnary DCE elctaton. In Fgure 4, we present the effect of latent theater use on WTP. Generally, the effect s around two to three tmes weaker than the effect of latent consequentalty, although stll sgnfcant for all attrbutes, except for chldren s theaters. Importantly, even for ndvduals 20

21 whose latent theater use s n the bottom 2.5%,.e., for the ones who wth hgh probablty wll not vst any theater after program mplementaton, we stll observe a postve WTP for most attrbutes. Expermental theaters are the excepton, and ths type of entertanment may only be valuable to theater users. Fnally, we note that whle movng from a MXL to a HMXL model allows for rcher specfcatons, potentally avodng estmaton bases, the analyst must make addtonal choces as t pertans to the specfcaton of the measurement and structural equatons. In order to help assess the senstvty of our results to these specfcaton choces, as well as to gan further nsght, we estmate several addtonal models and nclude them n the Onlne Appendx. 29 Gven that there are no meanngful dfferences n the qualtatve results relatve to those n our prmary model, ths provdes evdence that the prevously llustrated results regardng latent consequentalty are not smply an artefact drven by a partcular model specfcaton. In fact, the results of all our models consstently pont to the mportance of stated consequentalty, latent consequentalty and consequentalty scrpt, whle demonstratng that these three varables are not necessarly fully correlated nor easly measured. 5. Dscusson Usng a novel Hybrd Mxed Logt (HMXL) model, ths study provdes prmary emprcal evdence that, consstent wth mechansm desgn theory, latent belefs regardng the polcy consequentalty of a bnary DCE survey are an mportant drver of stated preferences. In fact, we fnd the effects to be very pronounced, wth belefs supportng polcy consequentalty leadng to substantally hgher estmates of wllngness to pay. Importantly, we observe that the emergng approach for dentfyng belefs over polcy consequentalty.e., a smple Lkert-scale survey queston provdes a measure that s, n fact, strongly correlated wth latent belefs. Recognzng that the drect ncluson of stated measures of the belefs n choce models can gve rse to econometrc ssues, for example measurement error, 29 In partcular, models 2.0 and 3.0 n the Onlne Appendx present the results of specfcatons whch nclude soco-demographc controls as explanatory varables n the measurement equatons, and as addtonal latent varables capturng the combned effects of soco-demographc characterstcs on preferences (Pakalnete et al., 2017). We have consdered other specfcatons of the count data model, namely zero nflated Posson, negatve bnomal and zero nflated negatve bnomal. The results of these specfcatons were not, however, relable enough for presentng here, ndcatng convergence problems we were not able to trace. 21

22 the HMXL model provdes an alternatve, flexble framework from whch to te stated measures and preferences. We further fnd that the use of consequentalty scrpts, whch emphasze the potental role of surveys n formulatng polcy, lead to meanngful ncreases n elcted WTP. However, the magntude of the effect on WTP s small n comparson to the effect of latent consequentalty belefs. That our nformaton treatments have only a small effect suggests some possbltes. Frst, t may be the case that n ths and other feld applcatons belefs over polcy consequentalty naturally emerge. Stated preference surveys are often of hgh qualty, relate to socally mportant topcs and descrbe plausble polcy nterventons. It thus may be nstnctve for respondents to beleve that, even f left unstated, polcymakers are fundng the study or otherwse care about ts outcomes. Second, even n settngs where belefs over polcy consequentalty are weak, the potental role that consequentalty scrpts may play s lmted. That s, barrng some exceptonal crcumstances, survey researchers cannot assure respondents that results wll absolutely nform polcy nor precsely defne the mechansm by whch ths would occur. These arguments, n turn, suggest that t may be nterestng to explore consequentalty n opportunstc settngs. Ths may nclude settngs where t s possble to establsh and convey a lnk between survey results and polcy decsons (see Johnston, 2006). It s further possble that, at least for some people and some settngs, belefs about potental ctzen nvolvement n polcy makng may be very entrenched, and researchers are thus unlkely able to alter them. It bears notng that stated belefs about polcy consequences were somewhat weak n ths study, wth just 5.82% of respondents ndcatng they defntely agree, 21.21% statng they rather agree and almost half (49.53%) nether n agreement or dsagreement wth the statement that the results of the survey wll nfluence future decsons. Klamer (2016) argues that economsts have had lttle nfluence on cultural polcy, and thus our respondents appear to have reasonable belefs. From a theory standpont, respondents need only have a non-zero probablty of nfluencng agency acton. On the other hand, an elctaton mechansm may only work as expected f ths probablty s hgh enough. Ths remans an mportant emprcal ssue. Our exploraton of polcy consequentalty targets just one of the belefs dentfed by theory as mportant for truthful preference revelaton. Another mportant, latent belef s that payment can be coercvely collected from respondents upon program mplementaton. Ths assumpton has receved less attenton, although efforts have begun to explore the use of followup questons to elct nformaton ted to ths belef. The HMXL model developed here provdes 22

23 an estmaton framework that allows the analyst to ncorporate many latent belefs smultaneously. It accommodates the jont modellng of choces and responses to a multtude of survey questons charged wth measurng belefs, regardless of whether these questons are targeted towards the same belef or dfferent sets of belefs. In fact, related to the development of belef elctaton questons, the HMXL model can be used to determne the magntude of the relatonshp between elcted and latent belefs, thus provdng a platform from whch to dscrmnate and evaluate alternatve approaches. The econometrc framework further provdes a way to determne the relatve mportance of dstnct latent factors ted to dfferent belefs relevant for ncentve compatblty or otherwse on elcted values. In mplementng hybrd models t s common for researchers, as we have done here, to hypothesze that certan latent factors exst, proceed to measure them based on avalable data, and nterpret results as f they are n fact drven by the hypotheszed latent varables. We note, however, that there are multple possble nterpretatons of the latent varables n these models and our partcular nterpretatons are thus subjectve. The measurement varables may nadequately explan the latent varables or otherwse the latent varables dentfed n a model may be attrbutable to factors not consdered by the analyst. From a techncal perspectve, n our model what we label and nterpret as latent consequentalty s merely some unobserved construct that s postvely correlated wth responses to the Lkert-scale survey queston on polcy consequences and postvely correlated wth the WTP for attrbutes of the proposed dscounted theater tcket programs. For nstance, the latent factor could nstead be ted to forms of socal desrablty bas (e.g., yea-sayng or nay-sayng), or nternally motvated expressons of atttudes that are dsjont from unobserved demand for proposed projects. Other econometrc approaches that ncorporate stated belefs, such as nstrumental varables methods or smply treatng belef ndcators as exogenous varables, are of course not mmune to nterpretaton ssues. Although emprcal evdence has demonstrated that condtonng on stated belefs over polcy consequentalty can enhance external valdty, more evdence s needed and t remans an open queston as to what exactly s beng captured by belef questons n ths context. We conclude our paper wth a proposed approach for nvestgatng ths. Specfcally, one can nclude multple survey questons to measure a partcular belef. The lterature on consequentalty has reled on a sngle queston, whereas t s more common n the broad lterature on hybrd models to use multple ndcators to measure a latent factor. The use of multple questons gves more opportuntes for researchers to support or refute ther proposed nterpretaton. Related to ths, to help address concerns over socal 23

24 desrablty bas or nternal motvatons, one can construct questons assumed to be correlated wth for example yea-sayng and ncorporate these other possble factors as separate latent varables (wth assocated measurement varables) wthn our proposed econometrc framework. 24

25 References Ben-Akva, M., McFadden, D., Tran, K., Walker, J., Bhat, C., Berlare, M., Bolduc, D., Boersch- Supan, A., Brownstone, D., Bunch, D., Daly, A., De Palma, A., Gopnath, D., Karlstrom, A., and Munzaga, M., Hybrd Choce Models: Progress and Challenges. Marketng Letters, 13(3): Bhat, C. R., Dubey, S. K., and Nagel, K., Introducng non-normalty of latent psychologcal constructs n choce modelng wth an applcaton to bcyclst route choce. Transportaton Research Part B: Methodologcal, 78: Bhat, C. R., Varn, C., and Ferdous, N., A comparson of the maxmum smulated lkelhood and composte margnal lkelhood estmaton approaches n the context of the multvarate orderedresponse model. In: Advances n Econometrcs, W. Greene and R. C. Hll, eds., Emerald Group Publshng Lmted, Bollen, K. A., and Davs, W. R., Two Rules of Identfcaton for Structural Equaton Models. Structural Equaton Modelng: A Multdscplnary Journal, 16(3): Budzńsk, W., and Czajkowsk, M., Addressng Endogenety n Hybrd Choce Models. Paper presented at the Conference of the European Assocaton of Evronmental and Resource Economsts, Athens, Greece. Bulte, E., Gerkng, S., Lst, J. A., and de Zeeuw, A., The Effect of Varyng the Causes of Envronmental Problems on Stated WTP Values: Evdence from a Feld Study. Journal of Envronmental Economcs and Management, 49(2): Carson, R., and Czajkowsk, M., A New Baselne Model for Estmatng Wllngness to Pay from Dscrete Choce Models. Paper presented at the Internatonal Choce Modellng Conference, Sydney, Australa. Carson, R., and Groves, T., Incentve and nformatonal propertes of preference questons. Envronmental and Resource Economcs, 37(1): Carson, R. T., Groves, T., and Lst, J. A., Consequentalty: A Theoretcal and Expermental Exploraton of a Sngle Bnary Choce. Journal of the Assocaton of Envronmental and Resource Economsts, 1(1/2): Chou, L., and Walker, J. L., Maskng dentfcaton of dscrete choce models under smulaton methods. Journal of Econometrcs, 141(2): Collns, J. P., and Vossler, C. A., Incentve compatblty tests of choce experment value elctaton questons. Journal of Envronmental Economcs and Management, 58(2): Czajkowsk, M., and Budzńsk, W. (2015). "An nsght nto the numercal smulaton bas a comparson of effcency and performance of dfferent types of quas Monte Carlo smulaton methods under a wde range of expermental condtons." In: Envronmental Choce Modellng Conference, Copenhagen. Czajkowsk, M., Hanley, N., and Nyborg, K Socal Norms, Morals and Self-nterest as Determnants of Pro-envronment Behavours: The Case of Household Recyclng. Envronmental and Resource Economcs, 66(4): Daly, A., Hess, S., Patrun, B., Potoglou, D., and Rohr, C., Usng ordered atttudnal ndcators n a latent varable choce model: a study of the mpact of securty on ral travel behavour. Transportaton, 39(2): Dazano, R. A., and Bolduc, D., Incorporatng pro-envronmental preferences towards green automoble technologes through a Bayesan hybrd choce model. Transportmetrca A: Transport Scence, 9(1): Dekker, T., Hess, S., Brouwer, R., and Hofkes, M., Decson uncertanty n mult-attrbute stated preference studes. Resource and Energy Economcs, 43: Dllman, D. A., Smyth, J. D., and Chrstan, L. M., Internet, Phone, Mal, and Mxed-Mode Surveys: The Talored Desgn Method. 4 Ed., Wley. Drchouts, A. C., Vasslopoulos, A., Lusk, J. L., and Nayga, R. M., Jr. (2015). "Reference Dependence, Consequentalty and Socal Desrablty n Value Elctaton: A Study of Far Labor Labelng." Paper presented at the 143rd Jont EAAE-AAEA Semnar, Naples, Italy. 25

26 Forrest, D., Grme, K., and Woods, R., Is t worth subsdsng regonal repertory theatre? Oxford Economc Papers, 52(2): Greene, W. H., Econometrc Analyss. 7 Ed., Prentce Hall, Upper Saddle Rver, NJ. Grsolía, J., and Wlls, K., A latent class model of theatre demand. Journal of Cultural Economcs, 36(2): Grsolía, J. M., and Wlls, K. G., An evenng at the theatre: usng choce experments to model preferences for theatres and theatrcal productons. Appled Economcs, 43(27): Guevara, C. A., and Ben-Akva, M., Addressng Endogenety n Dscrete Choce Models: Assessng Control-Functon and Latent-Varable Methods. In: Choce Modellng: The State-ofthe-Art and the State-of-Practce. Proceedngs from the Inaugural Internatonal Choce Modellng Conference, S. Hess and A. Daly, eds., Emerald Group Publshng Lmted, Hansen, T., The Wllngness-to-Pay for the Royal Theatre n Copenhagen as a Publc Good. Journal of Cultural Economcs, 21(1):1-28. Herrges, J., Klng, C., Lu, C.-C., and Tobas, J., What are the consequences of consequentalty? Journal of Envronmental Economcs and Management, 59(1): Hess, S., and Beharry-Borg, N., Accountng for Latent Atttudes n Wllngness-to-Pay Studes: The Case of Coastal Water Qualty Improvements n Tobago. Envronmental and Resource Economcs, 52(1): Hess, S., Hensher, D. A., and Daly, A., Not bored yet Revstng respondent fatgue n stated choce experments. Transportaton Research Part A: Polcy and Practce, 46(3): Hess, S., and Stathopoulos, A., Lnkng response qualty to survey engagement: A combned random scale and latent varable approach. Journal of Choce Modellng, 7:1-12. Hoyos, D., Marel, P., and Hess, S., Incorporatng envronmental atttudes n dscrete choce models: An exploraton of the utlty of the awareness of consequences scale. Scence of The Total Envronment, 505: Johnston, R. J., Is hypothetcal bas unversal? Valdatng contngent valuaton responses usng a bndng publc referendum. Journal of Envronmental Economcs and Management, 52(1): Jöreskog, K. G., and Goldberger, A. S., Estmaton of a Model wth Multple Indcators and Multple Causes of a Sngle Latent Varable. Journal of the Amercan Statstcal Assocaton, 70(351a): Kamargann, M., Ben-Akva, M., and Polydoropoulou, A., Incorporatng socal nteracton nto hybrd choce models. Transportaton, 41(6): Km, J., Rasoul, S., and Tmmermans, H., Expandng scope of hybrd choce models allowng for mxture of socal nfluences and latent atttudes: Applcaton to ntended purchase of electrc cars. Transportaton Research Part A: Polcy and Practce, 69: Klamer, A., The value-based approach to cultural economcs. Journal of Cultural Economcs, 40(4): Klng, C., Phaneuf, D. J., and Zhao, J., From Exxon to BP: Has Some Number Become Better than No Number? Journal of Economc Perspectves, 26(4):3-26. Landry, C. E., and Lst, J. A., Usng Ex Ante Approaches to Obtan Credble Sgnals for Value n Contngent Markets: Evdence from the Feld. Amercan Journal of Agrcultural Economcs, 89(2): McFadden, D., Condtonal Logt Analyss of Qualttatve Choce Behavour. In: Fronters n Econometrcs, P. Zarembka, ed., Academc Press, New York, NY, Mtan, Y., and Flores, N. E. (2012). "Robustness Tests of Incentve Compatble Referenda: Consequental Probablty, Group Sze, and Value-cost Dfference." Paper presented at the Conference of the European Assocaton of Envronmental and Resource Economsts, Prague, Czech Rebuplc. Oehlmann, M., and Meyerhoff, J., Stated Preferences Towards Renewable Energy Alternatves n Germany Do the Consequentalty of the Survey and Trust n Insttutons Matter? Journal of Envronmental Economcs and Polcy, 6(1):

27 Pakalnete, K., Agars, J., Czajkowsk, M., Strake, S., Zawojska, E., and Hanley, N., Understandng the dstrbuton of economc benefts from mprovng coastal and marne ecosystems. Scence of The Total Envronment, 15: Raveau, S., Yáñez, M. F., and Ortúzar, J. d. D., Practcal and emprcal dentfablty of hybrd dscrete choce models. Transportaton Research Part B: Methodologcal, 46(10): Scarpa, R., and Rose, J. M., Desgn Effcency for Non-Market Valuaton wth Choce Modellng: How to Measure t, What to Report and Why. Australan Journal of Agrcultural and Resource Economcs, 52(3): Scarpa, R., Thene, M., and Tran, K., Utlty n Wllngness to Pay Space: A Tool to Address Confoundng Random Scale Effects n Destnaton Choce to the Alps. Amercan Journal of Agrcultural Economcs, 90(4): Vj, A., and Walker, J. L., How, when and why ntegrated choce and latent varable models are latently useful. Transportaton Research Part B: Methodologcal, 90: Vossler, C. A., Doyon, M., and Rondeau, D., Truth n Consequentalty: Theory and Feld Evdence on Dscrete Choce Experments. Amercan Economc Journal: Mcroeconomcs, 4(4): Vossler, C. A., and Evans, M. F., Brdgng the gap between the feld and the lab: Envronmental goods, polcy maker nput, and consequentalty. Journal of Envronmental Economcs and Management, 58(3): Vossler, C. A., and Watson, S. B., Understandng the consequences of consequentalty: Testng the valdty of stated preferences n the feld. Journal of Economc Behavor & Organzaton, 86(0): Vredn Johansson, M., Heldt, T., and Johansson, P., The effects of atttudes and personalty trats on mode choce. Transportaton Research Part A: Polcy and Practce, 40(6): Wlls, K. G., and Snowball, J. D., Investgatng how the attrbutes of lve theatre productons nfluence consumpton choces usng conjont analyss: the example of the Natonal Arts Festval, South Afrca. Journal of Cultural Economcs, 33(3): Wlls, K. G., Snowball, J. D., Wymer, C., and Grsolía, J., A count data travel cost model of theatre demand usng aggregate theatre bookng data. Journal of Cultural Economcs, 36(2): Wśnewska, A., and Czajkowsk, M., Desgnng a socally effcent cultural polcy: the case of muncpal theaters n Warsaw. Internatonal Journal of Cultural Polcy:

28 Table 1. Informaton treatments ted to polcy consequentalty Informaton statement T1 T2 T3 T4 At the very begnnng of the survey: Your answers may be used n plannng future polces of fnancng theaters n Warsaw. By the general nformaton on fnancng the cty theaters: Warsaw ctzens mght have an mpact on the work of cty theaters [as they are subsdzed from the cty budget]. Ths survey ams at fndng out Your opnon on the cty theaters, and ts results wll allow to determne how the cty of Warsaw should fnance them. Near the descrpton of the proposals: Your answers n ths survey wll allow us to assess whether ntroducton of such a program s a good dea. Drectly before the value elctaton exercse, presented n a frame: ATTENTION Please, remember that Your answers mght be used for plannng the future fnancng of Warsaw theaters. In the case of an ntroducton of any polcy, whch can be nfluenced by the survey results, the changes wll nvolve all ctzens (ncludng You). Note: all passages translated from Polsh. ncluded ncluded ncluded ncluded ncluded ncluded ncluded ncluded 28

29 Table 2. Soco-demographc characterstcs of survey nformaton treatment samples and target populaton Varable T1 T2 T3 T4 Female (proporton) 0.55 [0.50] 0.52 [0.50] 0.53 [0.50] 0.55 [0.50] Age (n years) [16.79] [16.00] [16.46] [15.83] Educaton attanment (share) Elementary school [0.25] [0.23] [0.23] [0.25] Vocatonal pre-hgh school [0.29] [0.32] [0.28] [0.29] Hgh school [0.46] [0.46] [0.47] [0.47] Vocatonal school [0.32] [0.32] [0.32] [0.30] Unversty [0.49] [0.49] [0.49] [0.49] Household sze (ndvduals) [1.20] [1.22] [1.21] [1.18] Number of respondents Note: standard devatons appear n square brackets. Adult populaton of Warsaw

30 Table 3. HMXL reference model Measurement equaton 1 (ordered probt) dependent varable: stated consequentalty Coeffcent Std. err. Latent consequentalty Consequentalty scrpt Threshold parameters: α α α α Measurement equaton 2 (Posson regresson) dependent varable: number of theater vsts n the last 12 months Coeffcent Std. err. Constant Latent theater use Wllngness to pay model (mxed logt) Status quo Entertanment theaters Drama theaters Chldren s theaters Expermental theaters Cost (scale) Means man effects Coeffcent (s.e.) (0.0140) (0.0121) (0.0100) (0.0094) (0.0093) (0.0645) Means nteractons wth latent consequentalty Coeffcent (s.e.) (0.0161) (0.0119) (0.0129) (0.0121) (0.0120) (0.0854) Means nteractons wth latent theater use Coeffcent (s.e.) (0.0143) (0.0135) (0.0135) (0.0120) (0.0109) (0.0610) Means nteractons wth consequentalty scrpt Coeffcent (s.e.) (0.0124) (0.0098) (0.0088) (0.0087) (0.0084) (0.0452) Standard devatons Coeffcent (s.e.) (0.0147) (0.0111) (0.0163) (0.0101) (0.0079) (0.0718) 30

31 Model dagnostcs Log-lkelhood (constant only) -20, Log-lkelhood -15, McFadden s pseudo R AIC/n n (observatons) 20,400 k (parameters) 38 Note:,, denote sgnfcance at the 10%, 5% and 1% levels, respectvely. 31

32 Table 4. Smulated margnal wllngness to pay (PLN per year) assocated wth nformaton treatments [95% confdence nterval] Consequentalty scrpt = 1 Consequentalty scrpt = 2 Consequentalty scrpt = 3 Consequentalty scrpt = 4 Status quo Entertanment theaters Drama theaters Chldren s theaters Expermental theaters [-4.23, 3.64] [24.42, 30.91] [13.81, 19.62] [7.47, 13.03] [7.45, 12.73] [-1.01, 4.57] [28.62, 33.43] [17.45, 21.55] [8.73, 12.55] [8.05, 11.78] [0.76, 6.93] [31.76, 37.00] [20.13, 24.45] [8.99, 13.09] [7.69, 11.79] [1.36, 10.45] [34.02, 41.46] [21.92, 28.21] [8.37, 14.53] [6.53, 12.59] 32

33 Table 5. Smulated margnal wllngness to pay (PLN per year) assocated wth stated consequentalty levels [95% confdence nterval] Stated consequentalty = 1 Stated consequentalty = 2 Stated consequentalty = 3 Stated consequentalty = 4 Stated consequentalty = 5 Status quo Entertanment theaters Drama theaters Chldren s theaters Expermental theaters [-31.71, 35.41] [ , ] [ , ] [-56.90, -6.68] [ , ] [-22.17, 26.38] [ , ] [ , ] [-38.31, -2.03] [-76.51, ] [0.18, 5.47] [18.37, 22.83] [12.65, 16.49] [7.54, 11.08] [4.64, 8.27] [-20.91, 27.97] [245.95, ] [123.03, ] [21.81, 58.21] [55.83, 91.87] [-30.87, 38.49] [340.14, ] [168.21, ] [27.01, 78.72] [76.29, ] 33

34 Table 6. Smulated margnal wllngness to pay (PLN per year) assocated wth latent consequentalty [95% confdence nterval] 2.5 th percentle 25 th percentle 50 th percentle 75 th percentle 97.5 th percentle Status quo Entertanment theaters Drama theaters Chldren s theaters Expermental theaters [-3.31, 8.65] [-32.87, ] [-15.93, -6.44] [-1.26, 7.56] [-11.61, -2.63] [-0.14, 5.72] [9.35, 14.09] [7.71, 12.07] [6.18, 10.21] [1.91, 6.06] [0.11, 5.60] [30.41, 35.14] [19.00, 22.93] [9.02, 12.68] [7.99, 11.65] [-1.06, 6.89] [50.58, 57.10] [29.07, 34.98] [10.72, 16.30] [12.93, 18.38] [-4.45, 10.58] [88.16, 99.81] [47.26, 58.95] [13.04, 24.07] [21.37, 32.17] 34

35 Table 7. Cross-tabulaton of nformaton treatments versus stated consequentalty number of cases Consequentalty scrpt Stated consequentalty Total Total

36 Alternatve B Alternatve A (contnuaton of the current polcy) Entertanment theaters No change No change Drama repertory theaters Tckets for 5 PLN No change Chldren s theaters No change No change Expermental theaters Tckets for 5 PLN No change Annual cost for You 100 PLN 0 PLN Your choce Fgure 1. An example choce set (translated from Polsh) 36

37 Fgure 2. Overvew of the HMXL model framework used for analyzng stated preferences for proposed dscounted theater programs 37