Incentive compatibility in stated preference valuation methods

Transcription

1 Incentive compatibility in stated preference valuation methods Ewa Zawojska Faculty of Economic Sciences University of Warsaw Summary of research accomplishments presented in the doctoral thesis Assessing values of goods that are not bought or sold in markets is challenging and controversial, yet often inescapable for informing many public policy decisions. For these goods, there is no natural mechanism, such as a market, that could indicate their value. At the same time, values of non-market goods measured in monetary terms are needed in a wide range of contexts. The values provide estimates of benefits from considered public policy projects, which are essential for the cost-benefit analyses of these projects. The values also help measure losses from natural damages necessary in litigation processes. Economics has developed tools for non-market value assessment, which can be grouped into two categories: revealed preference methods and stated preference methods. To gauge values of non-market goods, revealed preference approaches rely on observed behaviour of individuals, such as how often and how far they travel to make use of non-market goods (for example, national parks). Stated preference approaches ask individuals about their preferences towards given non-market goods in specially designed surveys. This thesis focuses on the use of stated preference methods for valuation of non-market goods. Since the time of early applications of stated preference methods in the field in the 1960s (Davis, 1963; Luce and Tukey, 1964), the methods have been substantially advanced. One of the main drivers for the advancements has been the goal to make the methods generate valid value estimates, which will truthfully represent public preferences. In order to achieve this goal, non-market valuation literature suggests designing stated preference surveys so that they are incentive compatible, that is, they make truthful response the single best strategy for survey participants (Johnston et al., 2017). Making a survey incentive compatible requires satisfying several conditions. Non-market valuation literature from the past decade (e.g., Carson and Groves, 2007; Herriges et al., 2010; Vossler, Doyon and Rondeau, 2012) has devoted particular attention to two of these conditions, which are an explicit link of the survey outcome to actual consequences (henceforth referred to as the survey consequentiality) and the question format used for value elicitation. This thesis provides new insights and empirical evidence regarding the importance of these two conditions for truthful preference revelation in surveys. 1

2 The thesis comprises eight papers inquiring into the issue of truthful value disclosure in valuation surveys employing stated preference methods. The papers apply a wide spectrum of approaches to examine this issue. They range from a broad review of the literature devoted to testing the methods validity and a theoretical model depicting the role of survey consequentiality, to empirical investigations in both laboratory and field settings, which separately address various aspects related to incentive compatibility of stated preference surveys. The first four papers presented are concentrated around survey consequentiality. The next four papers focus on the role of the format used for value elicitation. The underlying research question shared by each of these papers is whether, and if so, how, stated preference methods can provide valid economic value measures that closely reflect public true preferences. Paper 1 reviews abundant empirical literature investigating the validity of stated preference methods. The mixed findings arising from these studies have usually led to lack of consensus regarding the methods validity. Paper 1 re-evaluates this evidence taking into account incentive compatibility of the studies. Interestingly, when only studies employing an incentive compatible mechanism of value elicitation are considered, the evidence becomes unambiguous, clearly pointing to validity of stated preference methods. Paper 2 addresses the issue of survey consequentiality in a field valuation study. There are two major lines of the paper s contribution to stated preference literature. First, since incentive compatibility requires that a respondent views the survey as consequential, the paper proposes an econometric framework for including (self-reported by respondents) measures of unobservable perceptions about consequentiality in models of stated preferences. Advantages of this framework lie in that it accounts for imprecision of the self-reported measures of the perceptions and that, in comparison with common, stepwise approaches, it improves efficiency of preference modelling through simultaneous estimation of a structural model. Second, the paper explores links between stated preferences, self-reported consequentiality perceptions and the degree of survey consequentiality conveyed through the survey script. The findings show that while consequentiality perceptions significantly affect stated preferences, the effect of the consequentiality exposition in the script on respondents behaviour is, in the case of this study, negligible. Also, the consequentiality script appears to weakly impinge on subjective perceptions about the survey consequentiality, pointing to difficulties associated with influencing the perceived consequentiality of a survey through survey scripts. Robustness of these findings should be tested in future research involving alternative wordings in the scripts emphasising survey consequentiality. Building upon existing empirical evidence, Paper 3 deconstructs the general concept of consequentiality into several elements. The suggested consequentiality components include a respondent s beliefs over: (1) her response influencing the survey outcome, (2) the evaluated public good being actually provided and (3) the payment related to the provision of the good being actually collected. A theoretical model developed in the paper illustrates the importance 2

3 of each of these beliefs for incentive compatibility of value elicitation. In particular, the analysis demonstrates that relationships between the strengths of these beliefs matter for incentive compatibility of a stated preference survey. For example, even if a respondent views a survey as consequential, when she perceives the actual provision of the good to be more likely to happen than the actual collection of the payment, then she may be inclined to overstate the value of the good to her. The paper further reveals that the impacts of the consequentiality components on survey incentive compatibility properties can vary depending on a respondent s attitude towards risk. These findings are especially important in light of usual practices for controlling for respondents perceptions about survey consequentiality, which rely on answers to a single question asking whether a respondent believes that the survey outcome will be considered by policy makers; typically, the questions eliciting a respondent s perceptions about the survey consequentiality do not take into account various aspects of consequentiality. Paper 4 brings the theoretical considerations implied by Paper 3 to the field. It empirically verifies both the effects of the beliefs over different components of consequentiality for stated preferences and the hypothesised link between the beliefs and risk attitudes. The paper observes the impacts of the beliefs over the good s actual provision and the payment s actual collection on stated preferences to be divergent: the former is found to increase the stated values, while the latter is found to decrease them. This constitutes an important result for stated preference research, as it may help develop guidelines for measuring and controlling for consequentiality perceptions in surveys. The remaining papers included in the thesis pertain to the issue of the question format used for eliciting preferences. Paper 5 inquires whether such formats as single binary choice, doublebounded binary choice, payment card and open-ended yield statistically different value estimates when the implementation of each of these formats assures incentive compatibility of the value elicitation procedure. The research is conducted in a laboratory, but the experiment retains important field-study characteristics, such as funding of an environmental public good. The main finding is that the four formats considered generate statistically equivalent value measures. This stays in contrast to myriad previous studies; however, these studies are unlikely to have maintained incentive compatibility of the value elicitation procedure. To the extent that the incentive compatibility of the formats can be assured in the field, the paper provides an encouraging result for practitioners. Naturally, the result should be further verified under conditions that extend the experimental design to better characterise field settings. This includes, for example, broadening the study population from a pool of college students to a more heterogeneous sample from a more representative population. Paper 6 builds upon the ample literature that examines whether two main approaches within stated preference methods, namely contingent valuation and choice experiments, yield equivalent value measures. Although the nomenclature in stated preference research is far from being uniform, contingent valuation is commonly seen as an assessment of a good treated as an indivisible whole, while choice experiments involve valuation of a good defined through 3

4 separate characteristics (Champ, Boyle and Brown, 2017; Johnston et al., 2017). Based on an extensive literature review, Paper 6 argues that the underlying difference between the two approaches is the format of information presentation in the valuation question(s): in contingent valuation the information is typically displayed as textual descriptions, while in choice experiments it is shown as tables. The paper empirically tests the role of the information presentation format for stated preferences in an incentive compatible laboratory experiment. The results indicate that preference disclosure may not be affected by the information presentation format in a sequence of valuation questions, but it can be impacted when a single valuation question is used. This constitutes an interesting finding encouraging further research, given that recommendations for stated preference research suggest the use of a single valuation question rather than the sequence of questions (Johnston et al., 2017). Paper 7 evaluates the impact of varying the number of choice alternatives provided per valuation question on stated preferences. Specifically, it compares values elicited in a field survey through formats with two and three choice alternatives. The two-alternative format is often viewed as the benchmark from which to gauge properties of other elicitation formats, because a binding voting in a binary choice question is well known to be incentive compatible under weak assumptions (Farquharson, 1969). The results presented in Paper 7 imply that there are no statistically significant differences in the mean values elicited through two- and threealternative questions. Further, the results show that, compared with two-alternative questions, the value estimates derived from three-alternative questions lead to smaller standard errors as they are based on broader information. This finding may provide some support for the use of three rather than two alternatives per choice question as a way to increase efficiency without significantly biasing the results. However, in the light of vast mixed evidence on the impact of the number of choice alternatives on stated preferences, the result reported in Paper 7 needs further validation, in particular through surveys that will assure consequentiality of preference elicitation. Paper 8 describes a field valuation study designed to verify the practical importance of theorybased conditions for incentive compatibility of the open-ended elicitation format. The study in Paper 8 implements a few versions of the questionnaire which differ in the degree of their compliance with the incentive compatibility conditions specific to the open-ended format. Precisely, one version provides full information needed for incentive compatibility, while other versions limit the information at the advantage of increasing simplicity of the survey script. The findings show that the values derived from the different versions of the questionnaire are statistically equivalent. This evidence raises the question of the benefit gained from increased script complexity related to conveying all information needed for incentive compatibility of value elicitation through an open-ended question. Nevertheless, because the theoretical conditions for an open-ended format have been recently developed and, hence, the number of studies testing their practical importance is very limited, more research is needed before any definite conclusions are made. In particular, future studies should employ larger samples for 4

5 evaluating statistical equality of estimates, and laboratory experiments could be used to control for other aspects of incentive compatibility, such as consequentiality. The research presented in this thesis indicates that in general, incentive compatibility of stated preference surveys plays a crucial role for the validity of elicited values. The thesis contributes to the existing literature by adding both theoretical and empirical evidence on the importance of incentive compatibility in non-market valuation surveys. This evidence may help advance stated preference methods, as well as, provide some guidance for practitioners about how to both design valuation surveys and analyse stated preference data so that the value estimates will closely match true preferences. The papers presented above delineate many pathways for subsequent research, which could further broaden researchers knowledge and understanding of various issues related to incentive compatibility of stated preference methods. In what follows, several general ideas for possible extensions are discussed. However, given the number of papers included in the thesis and the variety of study approaches employed in them, plenty of other next research steps, specific to every paper, may be considered (and some of them are already under way in my ongoing research). A substantial part of the papers in the thesis use follow-up questions to elicit respondents perceptions about the survey incentive compatibility, in particular, about the survey consequentiality. Although it is a standard (and advised; Johnston et al., 2017) practice to control for the perceived incentive compatibility of a survey based on respondents answers to such questions, the literature does not provide clear recommendations on the design of these questions. Unsolved issues include how to incentivise truthful responding to the follow-up questions, how to formulate the questions, and what response scales to employ, among others. The follow-up questions are also potentially related to another important shortcoming, namely to endogeneity of the self-reported measures of the perceptions. Surveys typically elicit the perceptions at the end of the questionnaire and, thus, the self-reported measures may be a function of the individual s earlier responses in the survey. Standard modelling approaches that use the measures of the perceptions do not account for this possible problem. Also, it remains unknown how respondents treat their answers to these questions. As a result, the literature acknowledges that, for instance, self-reports about a degree of perceived consequentiality could be a proxy for something else that implicitly reflects preferences (Hwang et al., 2014; p. 485). Furthermore, it is likely that external factors exist that affect both stated preferences and self-reported measures of the perceptions. Investigations helping to guide stated preference researchers in this area would constitute a valuable addition to the methodology of non-market valuation. The papers included in the thesis investigate incentive compatibility and truthfulness of stated preferences within a relatively small set of possible formats used for value elicitation. Valuation studies vary in a wide number of other characteristics, which include the number of 5

6 valuation questions given per respondent, the framing of questions and the set of attributes characterising choice alternatives in questions, to name a few. A natural extension of this line of research would be to compare respondents behaviours across valuation surveys differing in other dimensions than those examined in this thesis, while at the same time assuring incentive compatibility of each of the survey formats used. As the incentive compatibility is a theorybased feature of a survey, this research could inform whether other behavioural aspects related to value formation and elicitation should be taken into account in order to enhance validity of stated preference methods. In conclusion, stated preference methods play a unique role in welfare analysis. They are the only known tool to estimate values of goods when data on real-market choices and actual behaviour, which could provide insight regarding the value, is not available. Estimates of values of non-market goods provide crucial information for public decision-making, including assessments of new public policies and evaluation of losses of public goods. Consequently, non-market economic valuation continues to be an important, policy-relevant research field. Studies conducted within this thesis constitute a part of on-going efforts in the area of nonmarket valuation aimed at enhancing validity and reliability of stated preference methods. The theoretical and empirical investigations presented show that incentive compatibility is important for validity of elicited values. References Carson, R. T. and Groves, T. (2007). Incentive and informational properties of preference questions. Environmental and Resource Economics, 37(1), Champ, P. A., Boyle, K. J. and Brown, T. C. (2017). A Primer on Nonmarket Valuation. Dordrecht, The Netherlands: Springer. Davis, R. K. (1963). The Value of Outdoor Recreation: An Economic Study of the Maine Woods. Doctoral thesis. Cambridge, MA: Harvard University. Farquharson, R. (1969). Theory of Voting. New Haven, CT: Yale University Press. Johnston, R. J., Boyle, K. J., Adamowicz, W. L., Bennett, J., Brouwer, R., Cameron, T. A., Hanemann, W. M., Hanley, N., Ryan, M., Scarpa, R., Tourangeau, R. and Vossler, C. A. (2017). Contemporary guidance for stated preference studies. Journal of the Association of Environmental and Resource Economists, 4(2), Luce, R. D. and Tukey, J. W. (1964). Simultaneous conjoint measurement: A new type of fundamental measurement. Journal of Mathematical Psychology, 1(1), Herriges, J., Kling, C., Liu, C.-C. and Tobias, J. (2010). What are the consequences of consequentiality? Journal of Environmental Economics and Management, 59(1), Hwang, J., Petrolia, D. R. and Interis, M. G. (2014). Consequentiality and opt-out responses in stated preference surveys. Agricultural and Resource Economics Review, 43(3), Vossler, C. A., Doyon, M. and Rondeau, D. (2012). Truth in consequences: Theory and field evidence on discrete choice experiments. American Economic Journal: Microeconomics, 4(4),