Detecting, Non-Transitive, Inconsistent Responses in Discrete Choice Experiments

Transcription

1 Deecing, Non-Transiive, Inconsisen Responses in Discree Choice Experimens Ali Rezaei Zachary Paerson July 2015 CIRRELT

2 Deecing, Non-Transiive, Inconsisen Responses in Discree Choice Experimens Ali Rezaei 1,*, Zachary Paerson 1,2 1 Deparmen of Geography, Planning and Environmen, Concordia Universiy, 1455 de Maisonneuve W., H (Hall Building), Monreal, Canada H3G 1M8 2 Ineruniversiy Research Cenre on Enerprise Neworks, Logisics and Transporaion (CIRRELT) Absrac. Surveys focusing on choice behaviour, and in paricular, Discree Choice Experimens (DCEs) are widely used in sudies across many disciplines, including ransporaion, markeing research, healh economics, labour economics and environmenal sudies. Invesigaion of he raionaliy of responses in choice experimens has received a fair bi of aenion from researchers. Mos of his research has focused on he idenificaion of irraional behaviour as i relaes o non-saiaion or lexicographic behaviour. A he same ime, irraional behaviour indicaed by non-ransiive choices (ofen referred o as inconsisen behaviour) has received less aenion by researchers. Unil now he idenificaion of non-ransiive inconsisen behaviour has concenraed on relaively simple choice experimens. This research aims o exend previous work by developing a mehod o idenify non-ransiive inconsisen behaviour in more complex experimens. In paricular, a sysemaic es procedure o deec inconsisen behaviour is developed and applied o hree DCEs. The consisency es is implying ha each responden has a given preference srucure and ha her/his choices should be consisen wih his srucure across heir choices, and herefore saisfy he axiom of ransiiviy. As such, choices ha are no consisen wih an individual s observed preference srucure are idenified as inconsisen wih his/her own choices. Our analysis shows ha inconsisen choices commonly occur in DCEs wih muliple asks and aribues. Moreover, more inconsisen behaviour is deeced in more complex experimens. Also, such behaviour has a significan impac on he valuaion of responden sensiiviy o aribues in models esimaed from DCE daa. Finally, excluding inconsisen responses resuls in significan improvemens in models fi. Keywords. Discree Choice Experimens (DCE), raionaliy, ransiiviy, inconsisen behaviour, dominance-based approach. Acknowledgemens. The auhors would like o acknowledge suppor from he Canada Research Chairs Program, he Canadian Social Sciences and Humaniies Research Council (SSHRC) as well as he Canadian Foundaion for Innovaion (CFI). Resuls and views expressed in his publicaion are he sole responsibiliy of he auhors and do no necessarily reflec hose of CIRRELT. Les résulas e opinions conenus dans cee publicaion ne reflèen pas nécessairemen la posiion du CIRRELT e n'engagen pas sa responsabilié. * Corresponding auhor: a.rezaaei@gmail.com Dépô légal Bibliohèque e Archives naionales du Québec Bibliohèque e Archives Canada, 2015 Rezaei, Paerson and CIRRELT, 2015

3 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens 1. INTRODUCTION Choice behaviour modelling aims o undersand people s behaviour by saisically analysing heir choices. Random uiliy heory is he mos commonly used basis on which o model and predic individual choice behaviour. Choice daa is ypically colleced using eiher Revealed Preference (RP) or Saed Preference (SP) approaches. SP approaches, and Discree Choice Experimens (DCEs) more specifically, use specialized surveys where respondens are asked o choose beween hypoheical alernaives in a series of differen scenarios, or choice asks. An imporan mehodological issue in he use of hese surveys is wheher preferences elicied via hese experimens are consisen wih he axioms of preference-based consumer heory. Recenly, a fair bi of research has been published focusing on he issue of esing for he violaion of normaive axioms ha hypohesize how raional individuals should make choices (Lancsar & Louviere, 2006). The compleeness axiom sipulaes ha each decision maker has a well-defined preference beween any wo possible alernaives A and B (Lancsar & Louviere, 2006), ha is eiher A>B or A<B. This axiom can be esed by repeaed choice ses in experimens (Ryan & San Miguel, 2003). The axiom of ransiiviy saes ha for alernaives A, B and C in a choice se, if A>B and B>C, hen A>C (Rulleau & Dachary-Bernard, 2012; McInosh & Ryan, 2002). Choices ha violae he ransiiviy axiom, and are herefore no consisen wih previous or subsequen choices are considered as inconsisen choices (Sælensminde, 2002). Research in he ransporaion lieraure o have considered he issue of inconsisency in choice behaviour and is effecs on models esimaed wih such daa include Sælensminde (2001), Sælensminde (2002), Hess e al. (2010), Rose e al. (2013), and Rezaei & Paerson (2015). The axiom of monooniciy explains ha, more is preferred o less and i implies ha he uiliy funcion is increasing (Lancsar & Louviere, 2006). A es for violaion of his axiom could be including a dominan alernaive in one choice se (Burge & Rohr, 2004). The coninuiy axiom saes ha respondens are assumed o consider all aribues of he opions in a choice se and o choose he opion hey prefer CIRRELT

4 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens (Rulleau & Dachary-Bernard, 2012). However, some respondens may rank he aribues and make choices based on he aribue wih which hey associae he highes prioriy, which is referred as lexicographic behaviour (Rulleau & Dachary-Bernard, 2012). Some exploraion of lexicographic behaviour in DCEs has been done by Hess e al. (2010) and by Rose e al. (2013). Lexicographic behaviour is non-compensaory, so ha respondens do no consider all aribues bu raher adop an aribue processing sraegy o ease heir decisionmaking, such as always choosing he cheapes alernaive (Campbell & Lorimer, 2009). Finally, non-rading choice behaviour occurs, especially in he case of labelled choice experimens, when a responden always chooses he same alernaive across choice ses (Hess, e al., 2010). Despie ineres in deecing and analysing he source of irraionaliy in responses, some researchers have argued agains he removal of he irraional choices from SP daa. Lancsar and Louviere (Lancsar & Louviere, 2006) believe ha deleing responses/individuals ha seem o be irraional may resul in he removal of valid preferences; impose sample selecion bias; and reduce he saisical efficiency and power of he esimaed choice models. They sae ha several facors may make raional behaviour appear irraional, such as shorcomings in he design and implemenaion of choice experimen. In his case choice may be influenced by aribues ha are no included in a choice experimen (Viney, e al., 2002). For example, if qualiy is no explicily included in he experimen, respondens could infer ha a higher qualiy is associaed wih a higher price. Also, a number of alernaive approaches o consumer heory have been proposed o accoun for violaions of he sandard preference-based axioms (Chorus & Bierlaire, 2013). Tha is, wha may appear irraional using he sandard preference-based approach may equally be explained as raional using an alernaive approach o consumer heory (Lancsar & Louviere, 2006). The conexual concaviy, and random regre models explicily allow for paricular ypes of reference dependencies and choice se composiion effecs ha are considered irraional under he classical uiliy-based model. For example, ake he 2 CIRRELT

5 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens compromise effec; his effec can be judged as irraional under he classical model paradigm, bu has been found o be robus in oher choice conexs (Chorus & Bierlaire, 2013). Research considering he issue of raionaliy in DCE responses, have mosly focused on deecing non-rading and, lexicographic behaviour. However, hese behaviours don conradic raionaliy nor do hey cause problems in represening preferences in erms of a uiliy funcion (Lancsar & Louviere, 2006). Also, evidence suggess random uiliy heory (RUT) can cope wih such preferences (Lancsar & Louviere, 2006). So, i is suggesed ha if raionaliy is of ineres and if one inends o employ a preference-based view of consumer heory, hen research migh be beer direced owards he axioms of ransiiviy and compleeness, raher han focusing on non-saiaion (dominance) and lexicographic preferences (Lancsar & Louviere, 2006). The aim of his paper is no o deec and remove responses deemed o be irraional. Insead, i is o propose a sysemaic approach o es he ransiiviy of responden choices as an axiom of raionaliy and check he effecs ha he removal of inconsisen responses may have on model esimaion resuls. As noed by (Samuelson, 1938), ransiiviy is a he cenre of he heory of choice and has he greaes empirical conen of hose axioms responsible for he exisence of preferences.. However, very few sudies have acually esed for consisency in his way (McInosh & Ryan, 2002; Lancsar & Louviere, 2006). Previous lieraure invesigaing inconsisencies in responses can be spli beween parameric, and simple inspecion approaches for deecing inconsisen behaviour across responden choices. In parameric approaches, one migh allow for differen error variances wihin a single model, such as using he scaling approach (Ben-Akiva & Lerman, 1985; Rose & Black, 2006); or esimae a panel model wih responden-specific scale parameers for he laen random uiliy disribuion, in which each responden is reaed as his/her own individual daa se wih is own scale facor; or use each responden s muliple observaions o esimae a separae model (Johnson & Desvousges, 1997). Anoher parameric approach is o include decision sraegy selecion as an explici facor in he choice model (Swai & Adamowicz, 1997). In he simple inspecion CIRRELT

6 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens approach, one deecs he occurrence of choices ha violae he axiom of ransiiviy resuling in inconsisency across individual responden choices (Hess, e al., 2010). According o his approach, in he case of only wo aribues, if respondens are observed in one choice ask o choose an alernaive wih a subsiuion raio benefi (beween wo aribues) relaive o all oher alernaives of a given value (e.g. X), bu hen laer rejec an alernaive wih a subsiuion raio benefi relaive o all oher alernaives of a value greaer han X in a subsequen choice ask (Hess, e al., 2010; Rose, e al., 2013), hey are idenified as having behaved inconsisenly. This consisency es is based on he assumpion ha each individual responden has a given preference srucure and ha her/his choices should be consisen wih his srucure across her/his own choices, and herefore saisfy he axiom of ransiiviy (Sælensminde, 2002). While deecing inconsisen choice behaviour can be easily performed in simpler experimens (e.g. experimens using only wo aribues such as ime and cos) difficulies arise in experimens wih more aribues (Hess, e al., 2010).The es proposed in his paper uses a sysemaic decision rule model ha focuses on he axiom of ransiiviy, which is considered a necessary condiion for a preference-based view of consumer heory (Ben-Akiva & Lerman, 1985; Lancsar & Louviere, 2006). Transiiviy implies, for example in he case of a binary choice, ha if an alernaive A is seleced in one choice, ha same opion should, ransiively, be chosen in any oher choice where i is beer in a leas one aribue and no worse on he ohers (McInosh & Ryan, 2002). Adoping an approach developed by Greco e al. (2001), we find a subse of aribues and heir associaed hresholds so ha if a responden is faced wih a choice ask where a given difference in aribue values across alernaives is exceeded, he responden should (according o heir preference srucure as suggesed from oher choices) choose he ask. The cenral idea of his approach is he represenaion (approximaion) of upward and downward unions of decisions, by granules of knowledge generaed by aribues. These granules (or condiion profiles) are dominance cones in aribue value space. Each condiion profile defines a dominance cone in n-dimensional (n being he 4 CIRRELT

7 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens number of aribues) condiion space R n, and each decision profile defines a dominance cone in one-dimensional decision space {selec, rejec}. In general, his consisency es is based on he assumpion ha each responden has a given preference srucure and ha her/his choices should be consisen wih his srucure across heir own choices, and herefore saisfy he axiom of ransiiviy (Sælensminde, 2002). The paper sars wih a simple (wo alernaives, wo aribues) example o explain wha inconsisency across individual choices means. Also, an approach o ransform a saed choice daa se so ha inconsisencies can be easily deeced is presened. In secion hree, he dominance approach o find dominance cones and condiion profiles, and how hey are used o develop individual decision rules, is explained. In secion four, we ouline hree case sudies and discuss he resuls derived from deecing and excluding inconsisen responses in choice model esimaion. Finally, secion five provides concluding remarks and suggesions on how his migh be able o influence fuure research in he design and analysis of SP sudies. 2. MODELLING INCONSISTENT BEHAVIOUR In his paper we ry o idenify inconsisen choice behaviour across choice asks. Inconsisencies are considered o occur when a violaion of he axiom of ransiiviy in he dominance principle is observed. As an example, we may hink of a binary choice siuaion (i.e. wo alernaives) wih wo aribues, a.1, a.2, each of which includes hree levels, L 1, L 2 and L 3 (where L 1 < L 2 < L 3 ). Suppose also ha he uiliy of an alernaive increases if he level of any of is aribues increases. The firs five columns of Table 1 presen an example of aribue levels and decisions made by a responden in four differen asks. Each of hese decisions seems o be raional (aken on is own), ye here migh be inconsisency beween he differen decisions. Like previous research, our inconsisency es is based on he differences beween he aribue levels of alernaives (Sælensminde, 2002; Hess, e al., 2010; Rose, e al., 2013). In CIRRELT

8 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens he example below, here are five possible aribue level differences for each aribue: wo levels beer (2; e.g. L 3 -L 1 ), one level beer (1; e.g. L 2 -L 1 ), no difference (0; e.g. L 3 -L 3 ), one level worse (-1; e.g. L 2 -L 3 ) and wo levels worse (-2; e.g. L 1 -L 3 ). Also, he decision made can be classified in wo differen ways: he alernaive is seleced meaning ha he alernaive was considered beer (b) compared o he oher alernaive; or i can be expressed as he rejecion of he oher alernaive meaning ha he alernaive was considered worse (w). The differences beween aribues levels are presened in he las hree columns of Table 1. For each choice ask, he firs line presens he aribue levels of alernaive one minus he aribue levels of alernaive 2 (Al.1-Al.2). The second line presens (Al.2-Al.1). Evidenly, he componens of hese wo lines are symmeric. TABLE 1 An Example Of Aribue Levels And Decisions Made Alernaive levels Difference beween aribue levels Tas k Al. A.1 A.2 Decision A.1 A.2 Decision 1 1 L 3 L 1 Selec 1-1 b 2 L 2 L 2 Rejec -1 1 w 2 1 L 3 L 1 Selec 1-2 b 2 L 2 L 3 Rejec -1 2 w 3 1 L 3 L 1 Selec 2-2 b 2 L 1 L 3 Rejec -2 2 w 4 1 L 3 L 2 Rejec 2-1 w 2 L 1 L 3 Selec -2 1 b 6 CIRRELT

9 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens A close look a he daa in Table 1, focusing on he firs line of each ask (i.e. Al.1-Al.2), shows he following: in all asks Al.1 dominaes Al.2 wih respec o a.1; in all asks Al.2 dominaes Al.1 wih respec o a.2. So, in all cases here is a rade-off beween he wo alernaives. In he firs case, he responden chose an alernaive ha is one level beer wih respec o a.1, bu one level worse wih respec o a.2. This implies ha his responden values a.1 more han a.2. This choice is no inconsisen wih he choice made in ask 2. There, he responden chose he same alernaive when a.2 is wo levels worse. Indeed, ask 2 provides analyss wih more informaion han ask 1. Tha is, he responden values a.1 much more han a.2. The choice in ask 3 is consisen wih he preference srucure implied by he firs wo choices asks. The siuaion unil now, is represened in Figure 1A. This figure is symmeric since Al.1- Al.2 = - (Al.2-Al.1). All poins represening Al.1-Al.2 fall in he lower righ quadran (4 h quadran), and he poins associaed wih Al.2-Al.1 fall in he upper lef (2 nd ) quadran. Because resuls are symmeric in he wo quadrans, we concenrae he following explanaion on he 4 h quadran, recognizing he resuls are generalizable o he 2 nd as well. The 2D cone drawn by he wo coninuous lines emanaing from he poin represened by he choice made in ask 2 shows an area inside which all poins dominae ask 2. Based on he decision made in ask 2, Al.1 was he implied beer alernaive (b). In all asks for which he poin represening Al.1-Al.2 falls inside he cone, he responden should make he same decision, i.e. choose Al.1. As will be seen in he following secion, he Dominance-based rough se approach, describes a responden s choice behaviour srucure and finds cones (described by decision rules) in which all alernaives are dominan wih respec o chosen alernaives (or dominaed wih respec o rejeced alernaives) like he cone referred o above. In ask 4, however, he firs alernaive is wo levels beer and one level worse han he second one wih respec o a.1 and a.2, respecively. This is shown in Figure 1B. This ask dominaes all oher asks wih respec o boh aribues (i.e. i falls inside he cone). In he oher word, in he ask 4 Al.1 is CIRRELT

10 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens more superior o Al.2, compared o ask 2, wih respec o he boh aribues. So, based on he axiom of ransiiviy, and given he knowledge inferred from he firs hree asks, he responden should choose Al.1. However, s/he seleced Al.2, which is inconsisen wih he oher choices. (A) (B) FIGURE 1 - Dominance cones and inconsisen behaviour (A consisen, B inconsisen) Wha has been described above, is anoher way of represening wha ohers have done in he pas when considering he quesion of inconsisen choice behaviour in saed preference experimens in he field of ransporaion (i.e. (Hess, e al., 2010; Rose, e al., 2013)). Using heir erminology, one can infer from ask 2 ha he responden is willing o choose an alernaive wih an a.1-a.2 raio benefi relaive o he oher alernaive of 1/2. In ask 4, however, he responden rejecs an alernaive wih a.1-a.2 raio benefi relaive o he oher alernaive of 2/1. As such, hey would also idenify his responden as having behaved inconsisenly. I is worh noing ha in he graphical represenaion above, for each choice, he raio benefi value is equal o he negaive of he inverse of he slope of he line connecing he origin of he coordinae sysem o he poin associaed wih ha choice. 8 CIRRELT

11 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens The reason for represening choices as we have described above, is ha i allows us o exend he approach o more complex experimens. In paricular, and as will be explained in he following secion, we can increase complexiy along he following dimensions. Firs we can include more aribues, which can be reaed by mapping he informaion in a higher level coordinae sysem (e.g. md coordinae sysem wih md cones for an experimen wih m aribues). Second, we can include more alernaives per choice ask. In his case, imagine here are l alernaives al 1, al 2 al l. As noed by Louviere, e al. (2000), any given choice ells us ha he responden prefers one alernaive, al k, o he oher l-1 alernaives. Tha is, al k > al 1,, al k > al k-1, al k > al k+1,, al k > al l. In his case, each muual dominance relaion beween al k and he oher alernaives can be reaed as a binary choice and mapped on he md coordinae sysem. I is worh noing ha his ype of experimen (i.e. a sandard saed choice experimen wihou ranking of alernaives) canno provide he analys wih informaion on he muual dominance relaion beween oher l-1 alernaives (Louviere, e al., 2000). Finally, and a he same ime, he approach can be exended o experimens ha do include he ranking of alernaives in a saed preference seing. In his case, any given response ells us ha he responden prefers he h h alernaive, h={1,..., l}, in he preference ranking, o he l-h alernaive ha are less preferred. This could be shown by l-h binary choices, and herefore all muual dominance relaions beween alernaives in a ask can be reaed as (l) (l 1) 2 binary choices and mapped on he md coordinae sysem. I is worh noing ha in his case, inconsisencies could occur even among he pieces of informaion drawn from he same response. 3. METHODOLOGY This secion explains he Dominance-based Rough Se Approach (Greco, e al., 2001), which we propose as a ool o sysemaically deec inconsisen behaviour in complex experimens. CIRRELT

12 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens 3.1. Dominance Approach Generally, Rough Se (RS) is a mahemaical framework ha deals wih vagueness and uncerainy in he fields of Arificial Inelligence (AI), knowledge discovery in daabases and Daa Mining (DM) (Wilox & Tindesman, 2004). The goal of his approach is reasoning from imprecise daa, or more specifically, discovering relaionships in daa. Wilox and Tindemans work {, 2004 #15} was he firs o employ RS heory for ravel choice paern modelling. Classic RS heory considers aribues wihou preference ordering of aribues. DRSA is an exension of RS ha explicily akes ino accoun he preference ordering of aribues (Greco, e al., 2001) which has allowed i o be applied in several fields such as he analysis of cusomer saisfacion (Greco, e al., 2007), Kansei engineering (cusomer psychological impressions or feelings abou produc) (Zhai, e al., 2009), and he predicion of airline passenger (Liou, 2009; Nassiri & Rezaei, 2012). The laes applicaions have benefied from an advanced version of DRSA called he Variable Consisency Dominance-based Rough Se Approach (VC-DRSA). This version allows some inconsisencies in he lower approximaions of ses by defining a parameer called he consisency level. Is predicion model is in he form of decision rules (Liou, 2009). The basic conceps of DRSA are described as follows (Dembczyński, e al., 2009; Zhai, e al., 2009; Liou, 2009) Dominance-Based Rough Se Approach According o DRSA heory (Greco, e al., 2001), informaion regarding choice is represened in he form of an informaion able. The rows of he able refer o disinc objecs (acions), while he columns refer o aribues ha are considered. Each cell of he able indicaes a quaniaive or qualiaive evaluaion of he objec aribue placed in he corresponding row and column, respecively. Formally, an informaion able is he 4-uple informaion sysem IS = (U, Q, V, f), where U is a finie se of objecs (universe), Q={q1, q2,...,qm} is a finie se of aribues, V= qq Vq in 10 CIRRELT

13 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens which Vq is he domain of aribue q, and f: U Q V is a oal funcion so ha f(x,q)vq for each q Q, x U, called he informaion funcion. The se Q is, in general, divided ino se C of condiion aribues and se D of decision aribues Rough Approximaion by Means of he Dominance Relaionship Le q be an ouranking (also called weak preference) relaion on U wih reference o crierion qq, so ha x q y means ha wih respec o crierion q, x is a leas as good as y. Suppose ha q is a complee pre-order, i.e., a srongly complee (which means ha for each x, yu, a leas one of x q y and y q x is verified, and hence wih respec o crierion q, x and y are always comparable) and ransiive binary relaion. Moreover, le Cl = {Cl, T}, T = {1,..., n}, be a se of classes of U, so ha each xu belongs o one and only one class Cl Cl. We assume ha all r, st, so ha r s, each elemen of Cl r is preferred o each elemen Cl s. Tha is, if is a comprehensive ouranking relaion on U, hen i is supposed ha (x Cl r, y Cl s, r> s) x > y, (1) where x>y means x y and no y x. In he example presened in he previous secion Cl included wo classes ha is Cl = {b, w}, so ha b w. Le s define unions of classes by a specific dominaed or dominaing class hese unions of classes are called upward and downward unions of classes, respecively. The upward union of classes is defined as: Cl, Cls s 1,..., n.; and (2) The downward union of classes is defined as: Cl, Cls s 1,..., n. (3) The saemen x Cl means ha x belongs a leas o class Cl, while x Cl means ha x belongs a mos o class Cl. To clarify, he union Cl is he se of objecs belonging o CIRRELT

14 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens class Cl or a more desired class, whereas he union Cl is he se of objecs belonging o class Cl or a less desired class. I should be noed ha Cl 1 = Cl n = U, Cl n = Cl n and also Cl 1 = Cl 1. Consequenly, for 2,..., n, we have Cl U Cl 1, ha is, all he objecs belonging o class Cl or more desirable belong o class U minus Cl -1 or less desirable and similarlycl 1 U Cl. In DRSA approaches, where among condiion aribues here is a leas one crierion, and decision classes are preference-ordered, he knowledge approximaed is a collecion of upward and downward unions of decision classes and he granules of knowledge are ses of objecs being defined using a dominance relaion insead of he indiscernible relaion. This is he main difference beween he classical RS approach and DRSA approaches. I is said ha objec x P-dominaes objec y (or, x P-dominaes y) wih respec o PC, denoed as xd P y, if x q y for all qp, and D P = q P q, hen he dominance relaion D P is a parial preorder. Given PC and xu, he granules of knowledge are: D ( x) { y U : yd x}, (4) p P D ( x) { y U : xd y} (5) p P called he P-dominaing se (a se of knowledge dominaing x) and he P-dominaed se (a se of knowledge dominaed by x), respecively. For any se of crieria P C, we say ha he inclusion of objec xu o he upward union of classescl, for 2,..., n, makes an inconsisency if one of he following condiions happens: (1) x belongs o class Cl or beer while being P-dominaed by an objec y belonging o a class worse han Cl, in oher words, x Cl bu Dp( x) Cl 1 ; or (2) x belongs o a worse class han Cl while i P-dominaes an objec y belonging o class Cl or beer, in oher words, x Cl bu D ( x) Cl. p 12 CIRRELT

15 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens In ha case, i is said ha x belongs o Cl wih some ambiguiy. In conras, if xcl and here is no inconsisency, i is said ha x belongs o objecs P-dominaing x belong o Cl, namely, D ( x) Cl. p Cl wihou any ambiguiy. Tha is, all Then, for P C, he se of all objecs belonging o Cl wihou any ambiguiy forms he P-lower approximaion of Cl, denoed by P( Cl ), and he se of all objecs ha have he possibiliy of belonging o Cl consiues he P-upper approximaion of Cl, which is denoed by P( Cl ). These approximaions are defined as follow: P( Cl ) xu : D ( x) Cl, (6) p P( Cl ) xu : D ( x) Cl D ( x), 1,..., n. p p xcl (7) Analogously, he P-lower approximaion and P-upper approximaion of Cl can be defined as follows: P( Cl ) xu : D ( x) Cl, (8) p P( Cl ) xu : D ( x) Cl D ( x), 1,..., n. p p xcl (9) Also he P-upper approximaions of Cl and Cl, by complemen of P( Cl ) and P( Cl ) wih respec o U can be obained as follows: P( Cl ) U P( Cl ), (10) 1 P( Cl ) U P( Cl ). (11) 1 Therefore, he classificaion paerns o be discovered in he dominance-based rough ses are funcions represening Cl and Cl by granules D p + (x) and D p (x). CIRRELT

16 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens Decision Rules The ulimae resul of he DRSA is obaining some simple if..., hen... decision rules from he informaion conained in he daa able. For a given upward union of classes Cl, he decision rules, inferred under a hypohesis ha acions belonging o P( Cl ) are posiive and all he ohers are negaive, sugges an assignmen o a leas class Cl. Analogously, for a given downward union Cl, he rules inferred under a hypohesis ha acions belonging o P ( Cl ) are posiive and ha all ohers are negaive sugges an assignmen o a mos class Cl s. On he oher hand, he decision rules inferred under a hypohesis ha acions belonging o he l inersecion P( Cl ) P( Cl ) are posiive and ha all he ohers are negaive sugges an assignmen o some class beween Cl s and Cl (s<). Each rule has hree pars in he premise. The firs one relaes o dominance on a subse of crieria, he second o indiscernibiliy on a subse of qualiaive aribues, and he las o similariy on a subse of quaniaive aribues. The following hree ypes of decision rules can be considered: 1. D -decision rules suppored only by objecs from P-lower approximaions of he upward unions of classes Cl, ha is P( Cl ). They have he following form: If f(x, q 1 ) r q1 and f(x, q 2 ) r q2 and... f(x, q p ) r qp, hen xcl. In our example (see Figure 1A), his is represened as he cone wih solid lines. 2. D -decision rules suppored only by objecs from he P-lower approximaion of he downward unions of classescl, ha is P( Cl ). They have he following form: if f(x, q 1 ) r q1 and f(x, q 2 ) r q2 and... f(x, q p ) r qp, hen xcl. In our example (see Figure 1A), his is represened as he cone wih doed lines. 14 CIRRELT

17 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens 4. EMPIRICAL STUDIES This secion presens he findings from hree empirical sudies looking a he applicaion of he proposed approach o deec inconsisen choices. In each case, he following mehodology was used o idenify inconsisen choices ha a responden may have made. Firs, we esimaed a base basic mulinomial logi wih he enire daase in Saa (SaaCorp, 2011). Second, he Saed Choice daa was ransformed in he same way as described in he secion Modelling Inconsisen Behaviour, o derive muual dominance relaions, so ha he individual decision cones could be idenified. Third, he ransformed daa were used as inpus o he process whereby he proposed approach produced decision cones for each individual. A code wrien in Visual Basic (available from he auhors), using he proposed approach, was hen used o produce decision rules for each individual. By examining he individual decision rules and individual responden choices, i was possible o idenify hose choices ha were inconsisen wih an individual s oher choices, as well as o idenify wih which oher choices he choice was consisen. In fac, i was possible o idenify, for each choice, wheher i was consisen or inconsisen wih all of he oher choices. As such, i was also possible o idenify he degree o which a given choice was consisen or inconsisen by idenifying wih how many oher choices i was inconsisen. So, for example, supposing a choice se wih wo alernaives and six choice asks, i is possible o esablish wheher a given choice is consisen wih all, all bu one, all bu wo, ec. oher choice asks. As a resul, a choice ask inconsisen wih hree oher choice asks is considered more inconsisen han a ask inconsisen wih only one oher ask. We hen removed responses using differen hresholds of inconsisency (e.g. inconsisen wih more ha 50% of all responses) and hen re-esimaed he basic logi models and compared hem wih he base model. CIRRELT

18 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens 4.1. Pedesrian Preferences Wih Respec o Roundabous Daa The firs empirical analysis makes use of daa colleced for a sudy of pedesrian preferences wih respec o roundabous (PPRR) carried ou in Canada (Perdomo, e al., 2014). The sudy was based on an unlabelled, video-based saed preference survey. Each ask showed wo alernaive roundabous ha were characerized by he following aribues: presence of signs (no sign, regular sign and flashing sign); number of lanes (one or wo); presence of a pedesrian island (presen or absen), presence of pedesrian crossing (no crossing, crossing a roundabou enrance, crossing five meers from enrance); raffic volume (100 and 500 vehicles per hour); and finally, raffic speed (average speed hrough roundabou of raffic of 22 and 65 km/h). Six choice asks were presened o each responden. The online survey was conduced during he firs week of July, The sample available for esimaion conains 3005 observaions colleced from 501 respondens Empirical Resuls As can be seen in Table 2, he resuls from 5 differen basic MNL models are presened. In each case, a simple linear-in-parameers specificaion of he MNL model was used. The firs model is esimaed using all he observaions originally colleced. The second is he model afer having removed observaions using he same daa cleaning sraegy as explained in Perdomo e al. (2014). For he res of he models, inconsisen responses were removed using differen hresholds. Considering model 1, all coefficien signs are inuiively reasonable. Also, hey are significan a 10% confidence level excep for he case of regular signs and raffic speed. Model 2 was esimaed afer removing 14% of respondens in he daa cleaning process. All coefficiens esimaed have inuiively reasonable signs and are significan a he 5% confidence level, excep he regular sign coefficien ha is significan a 10% confidence level. Also, he ρ 2 of he model is 0.43, showing an improvemen in he goodness of fi compared o Model 1. Model 3 was esimaed afer removing responses inconsisen wih more han 2/6 h (33.3%) of 16 CIRRELT

19 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens he responden s oher choices. This resuled in he removal of 1% of all responses. Model 4 is a model esimaed afer removing choices inconsisen wih more han 1/6 h (17.6%) of he responden s oher choices. Tha is, 2.2% of responses were removed. While he percenage of he daa removed o esimae his model was almos one sevenh of ha in Perdomo e al. (2014) (i.e. 2.2% vs. 14% in he Perdomo e al. sudy), he goodness of fi of he new model is almos he same, even hough he coefficien of Regular sign aribue is sill insignifican. Finally, he hreshold was se so ha all responses inconsisen wih any oher of he responden s choices were removed (6.4% of responses). This lef only responses wih enirely consisen choices. While he percenage of he responses removed o esimae his model is almos half of ha in Perdomo e al. (2014), he majoriy of coefficiens of he model using only he consisen choices are significan a higher confidence levels han he coefficiens of Model 2. Moreover, he resuling model provides by far he bes performance in erms of he ρ 2. Furhermore, a closer inspecion of he models esimaed shows differences in model coefficiens. Model 2 and model 5 resul in very similar coefficien esimaes for he presence of a pedesrian island and pedesrian crossing a he enrance. However, excluding inconsisen responses resuls in lower values of pedesrian sensiiviy o regular signs, raffic volume and raffic speed; and greaer sensiiviy o flashing signs, number of lanes and having a pedesrian crossing 5m from enrance Discussion The differen esimaed models highligh he significan effec ha inconsisen behaviour has on model esimaes for he PPRR daa. Furher, removing inconsisen responses leads o universal gains in model fi. As such, he evidence would speak in favour of removing such responses from he daa, given he poenial effec on model esimaes ha heir inclusion can produce. CIRRELT

20 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens TABLE 2 Esimaion Resul on PPRR Daa Model 1 Model 2 Model 3 Model 4 Model 5 Base logi Perdomo e al. Removing Removing Removing model 2014 responses wih responses wih responses wih All Removing 14.2% more han 2 more han 1 more han 0 observaions inconsisen inconsisen inconsisen Responses Responses Responses (1% of daa) (2.2% of daa) (6.4% of daa) Variables Coeff. -sa Coeff. -sa Coeff. -sa Coeff. -sa Coeff. -sa No sign Regular signs Flashing signs Number of lanes Presence of Island No pedesrian crossing Crossing a enrance Crossing 5m from enrance Traffic volume Traffic speed Observaions ρ Shipper Preferences Daa The second daa se considered is from a saed choice survey of shippers wih respec o carriers in he Quebec Ciy Windsor Corridor in Canada (Paerson, e al., 2007). The survey was adminisered online in he summer of There were 18 choice asks, each wih hree unlabelled alernaive carriers. The carriers were characerized by five aribues: cos of shipmen (low - 10% below; medium and high - 10% above); on-ime reliabiliy (85%, 92%, and 98%); damage risk (0.5%, 1%, and 2%); securiy risk (0.5%, 1%, and 1.5%); and wheher or no he shipmen would be carried by ruck only, or by ruck and inermodal rain. The sample available for esimaion conains 7,074 observaions colleced from 393 respondens. 18 CIRRELT

21 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens Empirical Resuls A linear-in-parameer uiliy funcion is again used. Also, addiional alernaive specific consans for he firs and second alernaives are included o capure order effecs. A deailed summary of he models esimaed using he shipper preference daa is presened in Table 3. The firs model was esimaed using all observaions, wihou implemening any daa cleaning. All coefficiens esimaed are significan a he 1% confidence level and have inuiively reasonable signs. The oher hree models were esimaed afer removing he mos inconsisen responses. As described in he secion Modelling Inconsisen Behaviour, he mehodology described here can be used wih experimens ha have choice asks wih more han wo alernaives, which was he case wih he shipper daa. To do so, each choice was ransformed ino wo binary choices. This allowed he deecion of inconsisencies across derived muual dominance relaions. Consequenly, in his case, each response can be inconsisen wih up o 72 muual dominance relaions. To be comparable wih wha was done wih he Roundabou daa, responses inconsisen wih 24/72 nd (33.3%), 12/72 nd (17.6%), 6/72 nd (8.8%) of a responden s oher muual dominance relaions were removed. Invesigaion and comparison of he models reveals ha removing more inconsisen respondens improves model performance in erm of ρ 2, so ha he ρ 2 of Model 4 is far beer han ha for Model 1. Also, some rends in coefficiens value are observed. Boh ASCs have become slighly lower when removing inconsisen respondens implying smaller order effecs in more consisen responden daa. A he same ime, however, here is a marked increase in he oher coefficien values when moving across he models, probably due o decreases in he relaive weigh of he unobserved uiliy componens, and consequenly increases in he scale parameer value. This is consisen wih he resuls obained in oher research (Hess, e al., 2010). However, he paricularly large increase in he securiy risk coefficien compared o oher coefficiens shows ha failing o exclude inconsisen respondens can resul in under esimaion his coefficien. CIRRELT

22 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens TABLE 3 Esimaion Resul on Shippers Daa Variables Model 2 Model 3 Model 4 Removing Removing Removing Model 1 responses wih responses wih responses wih Base logi more han 24 more han 12 more han 6 model inconsisen inconsisen inconsisen All observaions Responses Responses Responses (0.2% of daa) (1.1% of daa) (4.4% of daa) Coeff. -sa Coeff. -sa Coeff. -sa Coeff. -sa ASC ASC Cos level On-ime reliabiliy Damage risk Securiy risk Inermodal carrier Observaions ρ Discussion The analysis on he Shipper preferences daa has again highlighed he impac ha inconsisen behaviour can have on model resuls. The mos apparen change o model resuls relaes o gains in model fi resuling from he removal of inconsisen responses. Finally, as wih he PPRR daa, he model resuls show he advanage of removing inconsisen responses from he daa, given he poenial effec on model esimaes ha heir inclusion can produce Neighborhood Choice Projec Daa Se The hird analysis makes use of daa colleced for a neighbourhood locaion choice sudy in Monreal, Canada (Mosofi_Darbani, e al., 2014). Each ask showed wo alernaive neighbourhoods ha were characerized by he following aribues: Dwelling ype (Aparmen, 20 CIRRELT

23 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens Deached houses, Townhouses, and Triplexes); fron yard deph (6 fee, 9 fee), space beween buildings (no space, 20 fee); average home value (low - 20% below base price; medium and high - 20% base price); ravel ime o work by car (20, 35, 50 minues); ravel ime o work by ransi (5% below, 30% above ravel ime o work by car); and finally, ravel ime o nearby shops on foo (5, 15, 25 minues). The surveys were adminisered a coffee shops in June 2013 and also in February The sample available for esimaion conains 2,430 observaions colleced from 405 respondens Empirical Resuls Afer running he inconsisency deecion es only 47 responses (1.93%) were found o conradic muual dominance relaions. Table 4 presens he coefficiens of he models esimaed on he survey daa. The models were esimaed 1) using all he observaions originally colleced, 2) afer removing responses ha were inconsisen wih more han 2/6 h (33.3%) of a responden s oher choices, 3) afer removing responses ha were inconsisen wih more han 1/6 h (17.6%) a responden s oher choices and 4) afer removing all inconsisen responses. Considering model 1, all coefficien signs are inuiively reasonable and significan a 10% confidence level. The significan alernaive specific consan implies he exisence of an order effec in responses. While he percenage of he daa removed o esimae model 2 is very small, he goodness of fi, ρ 2, of he new model is slighly beer han he firs model. To esimae Model 3 only responses wih, a mos, one inconsisen choice wihin each individual s decision were used. All coefficiens of his model, excep he ASC, are significan a a higher confidence level compared o hose of he base model. Model 4 is a model esimaed afer removing all responses wih inconsisen dominance relaions. The performance of he model in erms of ρ 2 is much beer han he firs model. The insignifican ASC coefficien shows ha he order effec problem has been resolved. Also, in general, oher coefficiens are more significan (hey are all significan a 1% confidence level) compared o he model esimaed using all daa. Like he CIRRELT

24 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens previous case sudies, here is a sligh increase in he coefficien values when moving across he models, probably due o decreases in he scale parameer. Bu, a larger increase is observed in he case of he riplex and fron yard deph coefficiens ha could be a resul of underesimaing hese coefficiens when including he inconsisen responses in he esimaion Discussion As was he case wih he previous daa ses, he models showed increases in model fi and increasing significance of coefficiens afer removing responses idenified as being inconsisen wih respondens oher choices, wih respec o he dominance relaions. In paricular we found beer model and more significan coefficiens. We also found ha coefficien values in general increase, and some coefficiens change more han he res. TABLE 4 Esimaion Resul on Virual Realiy Daa Variables Model 2 Model 3 Model 4 responses wih responses wih responses wih Model 1 more han 2 more han 1 more han 0 Base logi model inconsisen inconsisen inconsisen All observaions responses responses responses (0.08%) (0.37%) (1.93%) Coeff. -sa Coeff. -sa Coeff. -sa Coeff. -sa ASC Dwelling ype Aparmen Deached houses Townhouse Triplex Average home value (housands CDN) -1.65e e e e Fron yard deph (fee) 8.75e e e Space beween buildings (in fee) Travel ime o work by car (minues) Travel ime o work by ransi (minues) Travel ime o nearby shops on foo (minues) Observaions ρ CIRRELT

25 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens 5. CONCLUSION This paper proposed a sysemaic approach o es he axiom of ransiiviy in daa derived from Discree Choice Experimens, which is essenial in consumer heory and ye lile considered in he lieraure. An approach using dominance rules (Greco, e al., 2001) was proposed o deec inconsisen choices of respondens in he case of more complex experimens han hose ha have been invesigaed previously in he lieraure. This provides he opporuniy o examine problemaic choices sysemaically in he conex of more complex experimens. The empirical analysis suggess ha inconsisen choices are common in SP surveys wih muliple asks and aribues. Moreover, more inconsisen behaviour is deeced in more complex experimens e.g. he shipper daase compared o he neighbourhood choice projec daase. The analysis also suggess ha such choices have a significan impac on he valuaion of responden sensiiviy o aribues in esimaed models. Anoher imporan finding is ha excluding inconsisen responses resuls in significan improvemen in model fi. Togeher, he resuls sugges ha removing inconsisen responses can resul in beer models. Furher invesigaion can use his approach o consider how he complexiy of experimens influences he share of inconsisen choices, and possibly opimal complexiy levels for hese surveys. Similarly, he approach could be used o evaluae opimal numbers of asks in hese surveys. 6. REFERENCES Ben-Akiva, M. E. & Lerman, S. R., Discree Choice Analysis, Theory and Applicaion o Travel Demand. Cambridge, Ma: MIT Press. Bierlaire, M., Axhausen, K. W. & Abay, G., The accepance of modal innovaion: The case of Swissmero. Ascona,, s.n., pp Błaszczyński, J. e al., jmaf - Dominance-based Rough Se Daa Analysis Framework- User's guide. Dominance-based rough se daa analysis framework. CIRRELT

26 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens Burge, P. & Rohr, C., DATIV: SP Design: Proposed approach for pilo survey. Tera-Plan in cooperaion wih RAND Europe and Gallup A/S. Campbell, D. & Lorimer, V. S., Accommodaing aribue processing sraegies in saed choice analysis: do respondens do wha hey say hey do?. Amserdam, s.n., pp Chorus, C. G. & Bierlaire, M., An empirical comparison of ravel choice models ha capure preferences for compromise alernaives. Transporaion, 40(3), pp Dembczyński, K., Greco, S. & Słowiński, R., Rough se approach o muliple crieria classificaion wih imprecise evaluaions and assignmens. European Journal of Operaional Research, 198(2), pp Greco, S., Maarazzo, b. & Slowinski, R., Rough ses heory for muli-crieria decision analysis. European Journal of Operaional Research, 129(1), pp Greco, S., Maarazzo, B. & Slowinski, R., Cusomer saisfacion analysis based on rough se approach. Zeischrif für Beriebswirschaf, 77(3), pp Hess, S., Rose, J. M. & Polak, J., Non-rading, lexicographic and inconsisen behaviour in saed choice daa. Transporaion Research Par D, 15(7), pp Johnson, F. R. & Desvousges, W. H., Esimaing Saed Preferences wih Raed-Pair Daa:Environmenal, Healh, and Employmen Effecs of Energy Programs. Journal of Environmenal Economics and Managemen, 34(1), pp Lancsar, E. & Louviere, J., Deleing irraional responses from discree choice experimens: a case of invesigaing or imposing preferences?. HEALTH ECONOMICS, 15(8), pp Liou, J. J., A novel decision rules approach for cusomer relaionship managemen of he airline marke. Exper Sysems wih Applicaions, 36(3), pp Louviere, J. J., Hensher, D. A. & Swai, J. D., Saed Choice Mehods Analysis and Applicaion. 1 ed. New York: Cambridge Universiy Press. McInosh, E. & Ryan, M., Using discree choice experimens o derive welfare esimaes for he provision of elecive surgery: Implicaions of disconinuous preferences. Journal of Economic Psychology, Volume 23, pp Mosofi_Darbani, J., Rezaei, A., Paerson, Z. & Zacharias, J., VIDEO GAME VS. TRADITIONAL TEXT-ONLY SP SURVEY OF NEIGHBORHOOD CHOICE. Washingon D.C., s.n., p CIRRELT

27 Deecing, Non-Transiive, Inconcisen Responses in Discree Choice Experimens Nassiri, H. & Rezaei, A., Air iinerary choice in a low-frequency marke: A decision rule approach. Journal of Air Transpor Managemen, January, 18(1), p Nible, A., Tracking inconsisen judicial behavior. Inernaional Review of Law and Economics, Volume 34, pp Paerson, Z., Ewing, G. O. & Haider, M., Shipper Preferences Sugges Srong Misrus of Rail Resuls from Saed Preference Carrier Choice Survey for Quebec Ciy Windsor Corridor in Canada. Transporaion Research Recoard, 26 December, Issue 2008, pp Perdomo, M. e al., Pedesrian preferences wih respec o roundabous A video-based saed preference survey. Acciden Analysis and Prevenion, 1 April, Volume 70, pp Rezaei, A. & Paerson, Z., Idenifying Inconsisen Responses in Saed Choice Surveys Using a Dominance-Based Approach. Washingon, DC, s.n. Rose, J. M. & Black, L. R., Means maer, bu variance maer oo: Decomposing response laency influences on variance heerogeneiy in saed preference experimens. Markeing Leers, 17(4), pp Rose, J. M., Hess, S. & Collins, T. A., Wha if My Model Assumpions are Wrong: The Impac of Non-sandard Behaviour on Choice Model Esimaion. Journal of Transpor Economics and Policy, 47(2), pp Rulleau, B. R. & Dachary-Bernard, J., Preferences, raional choices and economic valuaion: Some empirical ess. The Journal of Socio-Economics, 41(2), pp Ryan, M. & San Miguel, F., Revisiing he axiom of compleeness in healh care. Healh Economics, 12(4), pp Sælensminde, K., Inconsisen choices in Saed Choice daa Use of he logi scaling approach o handle resuling variance increase. Transporaion, 28(3), pp Sælensminde, K., The Impac of Choice Inconsisencies in Saed Choice Sudies. Environmenal and Resource Economics, Volume 23, pp Samuelson, P. A., A Noe on he Pure Theory of Consumer's Behaviour. Economica, 5(17), pp SaaCorp, Saa saisical sofware: release 12, College Saion: Saa Corporaion. Swai, J. & Adamowicz, W., Choice Task Complexiy and Decision Sraegy Selecion, Edmonon: Universiy of Albera. CIRRELT