TOPICS IN HEALTH ECONOMETRICS

Transcription

1 TOPICS IN HEALTH ECONOMETRICS By VIDHURA SENANI BANDARA WIJAYAWARDHANA TENNEKOON A dssertaton submtted n partal fulfllment of the requrements for the degree of DOCTOR OF PHILOSOPHY WASHINGTON STATE UNIVERSITY School of Economc Scences AUGUST 2012

2 To the Faculty of Washngton State Unversty: The members of the Commttee apponted to examne the dssertaton of VIDHURA SENANI BANDARA WIJAYWARDHANA TENNEKOON fnd t satsfactory and recommend that t be accepted. Robert E. Roseman, Ph.D., Char Ron C. Mttelhammer, Ph.D., Co-char Laura G. Hll, Ph.D. Bdsha Mandal, Ph.D.

3 ACKNOWLEDGMENT I wsh to gratefully acknowledge the nvaluable support and gudance of all members of my dssertaton commttee. Dr. Rosenman, n partcular, provded a contnuous stmulus throughout ths process to sharpen my capabltes and unleash hdden potentals by ntellectually challengng me. The drectons and practcal gudance of hm, together wth Dr. Mttelhammer s, helped me to master the art of fndng a good research topc, to pck the rght tools to analyze problems, to avod or overcome the obstacles when reachng the target and fnally to dsmount dead horses. Dr. Laura Hll s opnons as a non-economst always helped broaden my perspectve. I am grateful as well for the support of the other faculty and colleagues at the School of Economcs Scences for ther contnuous encouragement, apprecaton of work and many useful comments and crtcsms. Dr. Jll McCluskey, n partcular, deserves my grattude for her mentorng at tmes when I needed t most. Fnally, I wsh to express my ndebtedness for my wfe Mahesha for her enormous sacrfces, and the two boys Chrayu and Seth who helped dsspate the stress durng ths arduous journey.

4 TOPICS IN HEALTH ECONOMETRICS ABSTRACT By Vdhura Senan Bandara Wjayawardhana Tennekoon, Ph.D. Washngton State Unversty August 2012 Char: Robert E. Rosenman, Co-char: Ron C. Mttelhammer Ths dssertaton dscusses three topcs n health econometrcs focusng on msclassfcaton n bnary data, sources and the nature of bas, the mpact of usng msclassfed data n econometrc estmatons and methods to dentfy and correct the bas. The frst chapter proposes an econometrc estmator to estmate correct nferences when the dependent varable of the bnary choce model s endogenously msclassfed. The approach s valdated usng a smulaton study and appled to the analyss of a treatment program desgned to mprove famly dynamcs. The second chapter analyses the HIV-vrgn puzzle where a number of adolescents n Afrca who were found HIV postve report as never havng sex, whch ndcates ether the domnance of non-sexual modes of HIV transmsson or systematc msreportng of sexual behavor. A method s proposed to estmate the extent of msreportng and the contrbuton of sexual mode for HIV transmsson n Afrca after accountng for msreportng. The thrd chapter employs econometrc technques to compare self-reported and objectvely measured smokng data, takng nto account errors wth both methods, and cautons that objectve measures may not always be more relable than self-reported data. v

5 TABLE OF CONTENTS Page ACKNOWLEDGEMENTS... ABSTRACT...v LIST OF TABLES...v CHAPTER ONE: SYSTEMATICALLY MISCLASSIFIED BINARY DEPENDENT VARIABLES INTRODUCTION THE GENERALIZED MODEL TO CORRECT FOR COVARIATE-DEPENDENT MISCLASSIFICATION MONTE CARLO EXPERIMENT APPLICATION TO ESTIMATE THE EFFECIVENESS OF A FAMILY IMPROVEMENT PROGRAM Applcablty of the model Data Analyss of Results CONCLUSIONS...24 REFERENCE...26 APPENDIX...34 CHAPTER TWO: BEHOLD, A VIRGIN IS WITH HIV! MISREPORTING SEXUAL BEHAVIOR AMONG INFECTED ADOLESCENTS INTRODUCTION HIV TRANSMISSION IN AFRICA v

6 3. THE MODEL THE APPLICATION Data Sexual Actvty and the Extent of Msreportng Prevalence of HIV Populaton Level Estmates CONCLUSIONS...55 REFERENCE CHAPTER THREE: CAN AN OBJECTIVE MEASURE BE A BETTER ALTERNATIVE? SELF-REPORTED AND BIO-CHEMICAL DATA FOR IDENTIFYING ACTIVE SMOKERS INTRODUCTION THE STUDY SAMPLE THE MODEL TO IDENTIFY COVARIATE DEPENDENT MISCLASSIFICATION ESTIMATION RESULTS DISCUSSION AND CONCLUSIONS...81 REFERENCE...84 v

7 LIST OF TABLES CHAPTER ONE Table 1: Determnants of Pr (y=1) wth covarate dependent msclassfcaton (coeffcents)..28 Table 2: Determnants of Pr (yo=1 y=0) wth covarate dependent msclassfcaton...29 Table 3: Determnants of Pr (yo=0 y=1) wth covarate dependent msclassfcaton...30 Table 4: Varable Names, Descrptons and Summary Statstcs Table 5: Determnants of True Improvement n Famly Functonalty Table 6: Determnants of Probabltes of Msclassfcaton Table 7: Overall Comparson of three Models Table A1: Determnants of Pr (y=1) wth random msclassfcaton (coeffcents) Table A2: Determnants of Pr (yo=1 y=0) wth random msclassfcaton Table A3: Determnants of Pr (yo=0 y=1) wth random msclassfcaton CHAPTER TWO Table 1: Summary Statstcs...63 Table 2: Beng a Vrgn and Msreportng...64 Table 3: Prevalence of HIV...65 CHAPTER THREE Table 1: Descrptve Statstcs...86 Table 2: Comparson of each measure assumng the other measure s correct...87 Table 3: Estmaton results usng reported data Table 4: Estmaton results usng test results Table 5: Comparson of each measure relatve to the true value...90 v

8 CHAPTER ONE SYSTEMATICALLY MISCLASSIFIED BINARY DEPENDENT VARIABLES ABSTRACT When a bnary dependent varable s msclassfed, that s, recorded n the category other than where t really belongs, probt and logt estmates are based and nconsstent. In some cases the probablty of msclassfcaton may vary systematcally wth covarates, and thus be endogenous. In ths paper we develop an estmaton approach that corrects for endogenous msclassfcaton, valdate our approach usng a smulaton study, and apply t to the analyss of a treatment program desgned to mprove famly dynamcs. Our results show that endogenous msclassfcaton could lead to potentally ncorrect conclusons unless corrected usng an approprate technque. Key words: bnary choce model, msclassfcaton, response shft bas, Lkert scales. JEL codes: C01, C10, C18, C24, C50. 1

9 1. INTRODUCTION 1 Msclassfcaton of a dchotomous categorcal varable means that an observaton wth a true value of 0 s observed as 1 or an observaton that s truly a 1 s observed as a 0. When the msclassfed varable s the dependent varable, probt or logt estmates may lead to based and nconsstent estmates f the msclassfcaton s gnored or modeled ncorrectly (Hausman, 2001). Msclassfcaton of a varable can happen for varous reasons, although one can categorze them broadly nto two groups; response errors that are random n nature, and those that vary systematcally wth some respondent characterstc. The case we explore here s the latter, when the probablty of msclassfcaton s observaton specfc and dependent on covarates. The source of the error sometmes offers some nsght nto whether possble msclassfcaton s systematc or not. In labor market data, for example, some respondents may msreport ther employment status or a correctly reported labor status may be mstranscrbed (Chua and Fuller, 1987 ; Poterba and Summers, 1995). If the data s msreported because of language dffcultes or a lack of understandng, the probablty of msclassfcaton could vary systematcally wth educaton or prmary language. Moreover, whle we mght thnk mstranscrbed data s a random event, f the mstranscrptons are due to transcrber qualty and transcrbers are correlated wth locaton, then msclassfcaton probabltes could vary systematcally wth locaton. Prevous work on msclassfed dependent varables has taken two paths. The frst approach uses supplemental data to verfy the accuracy of responses. Chua and Fuller (1987) 1 Ths research was supported n part by a grant from the Natonal Insttute of Drug Abuse (R21-DA A1). We thank the parents and facltators who partcpated n the program evaluaton of SFP. 2

10 develop a parametrc model that ncorporated all J( J 1) msclassfcaton possbltes of an outcome varable wth J categores, but ther approach requres a mnmum of three ndependent sets of survey responses obtaned by re-ntervewng the orgnal respondents, and has a lmted practcal use. The condtonal logt procedure proposed n Poterba and Summers (1995) also ncorporates all possbltes of msclassfcaton and requres msclassfcaton probabltes found by analyzng the dscrepances between ntervew and re-ntervew outcomes. An alternatve path bulds the probablty of msclassfcaton nto the estmaton procedure, allowng for errors n the data, and usng statstcal methods to correct for t. Hausman et al. (1998) and Abrevaya and Hausman (1999) suggest both parametrc and semparametrc approaches for msclassfcaton probabltes that cannot be ndependently verfed and are ndependent of covarates. The focus of ther parametrc model s a dchotomous outcome varable wth two types of msclassfcaton, whch they denote as 0 (the probablty that a true 0 s recorded as a 1) and 1 (the probablty that a true 1 beng recorded as 0) 2. Wth ther parametrc approach the unknown msclassfcaton probabltes are estmated smultaneously wth the usual coeffcents of the bnary choce model. Ther sem-parametrc method provdes consstent estmates of the model parameters, but not of the msclassfcaton probabltes. Dustmann and van Soest (2000) extend the parametrc model of Hausman et al. (1998) to a trchotomous case. Lewbel (2000) allows the msclassfcaton probabltes to be covarate-dependent functons and shows that (gven some regularty) the bnary choce model wth covaratedependent msclassfcaton s completely dentfed even when the functonal forms of 0, 1 2 Throughout ths paper we use the same notaton. 3

11 and the dstrbuton of the error term are unknown. However, he also acknowledges that hs estmator s not lkely to be very practcal snce they nvolve up to thrd order dervatves and repeated applcatons of nonparametrc regresson (pp ). The lack of any emprcal work explotng hs estmator ndcates the need for a more practcal estmator n the case of covarate dependent msclassfcaton, even at the cost of some addtonal assumptons. Our paper extends the parametrc approach of Hausman et al. (1998) to the case, where the msclassfcaton probabltes are functons of one or more covarates 3. The parametrc estmator that we propose s a more tractable way to dentfy a model smlar to Lewbel (2000), but s condtonal on functonal form assumptons. The paper proceeds as follows. In secton 2, we present our structural approach to deal wth covarant-dependent msclassfcaton of the dependent varable and the dentfcaton requrements. Secton 3 has a Monte Carlo experment that compares our approach wth the ordnary probt model and the basc model presented n Hausman et al. (1998). In secton 4, we present an emprcal applcaton to demonstrate the applcablty of the model. Fnally, n secton 5 we dscuss mplcatons and conclusons from our generalzaton. 2. THE GENERALIZED MODEL TO CORRECT FOR COVARIATE-DEPENDENT MISCLASSIFICATION Assume, * y s an unobserved latent varable such that y X (1) * 3 Although Hausman, et al. (1998) brefly dscusses a lmted extenson of systematc msclassfcaton n secton 5.5, they do not fully characterze or mplement the approach. A sem-parametrc approach to deal wth covaratedependent msclassfcaton of the dependent varable s dscussed n detal n Abrevaya and Hausman (1999). Our nterest s n the parametrc model and n methods that provde msclassfcaton probabltes. 4

12 where, X s a vector of observed ndependent varables, s a vector of coeffcents to be estmated and s an d error term wth a known common dstrbuton. We observe y (2) * 1( y 0). If no msclassfcaton s present, we always observe the dchotomous outcome varable, y, correctly. However, f there s msclassfcaton, the outcome varable that we observe, ncludes some true 1 s classfed as 0 s and some true 0 s classfed as 1 s. As a result, n o y, general, y o o y. Accordngly, the bnary varable we observe y also ncludes an addtonal measurement error such that 1 wth probablty 1 f y 1 o y y where 1 wth probablty 0 f y 0 0 otherwse. (3) In other words, o 0 Pr y 1 y 0 (4) and o 1 Pr y 0 y 1. (5) The addton of not only ncreases the varance of the econometrc error term, but also adds heteroskedastty n a specfc way. The overall stochastc mechansm that determnes the values ultmately observed wth random msclassfcaton s a condtonal Bernoull process that can be characterzed va the followng data generatng process. y 1( y 0)1( u 1 ) 1( y 0)1( u ), where u ~ Unform 0,1, (6) 0 * * 1 0 5

13 We assume u and n (1) are ndependent. If the values of 0 and 1 are dependent on y as n (4) and (5), but ndependent of X, and the probablty dstrbuton functon of s F (.) then, as Hausman, et al. (1998) show, we can express the expected value of the observed dependent varable as o o E y X Pr y 1 X 1 F( X ). (7) When 0 and 1 are constants and 01 1, 4 the parameters of the above model can be consstently estmated ether by MLE or NLLS 5. Suppose nstead that the msclassfcaton probabltes 0 and 1 are functons of a set of varables, 0 Z and 1 Z respectvely as n Lewbel (2000). In partcular, the probabltes n (4) and (5) are now gven by Z 0 Pr o 1 0, 0 0 y y Z F Z (8) Z 1 Pr o 0 1, 1 1 y y Z F Z (9) where 0 Z and Z may be but are not necessarly subsets 6 of X, and F 0 and F1 are the cumulatve 1 dstrbuton functons of stochastc components that determne each type of msclassfcaton. 7 Insertng the precedng generalzed representaton of the msclassfcaton probabltes nto (7), the expected value of the observed dependent varable wth a covarate-dependent msclassfcaton can be expressed as 4 Ths condton, termed the monotonocty condton n Hausman et al. (1998) must be satsfed to dentfy (,, ) 0 1 separately from 1 0 (,1,1 ). 5 The relevant objectve functons are gven by equatons (6) and (7) n Hausman et al. (1998). 6 Ths allows one or both msclassfcaton probabltes to depend on varables that do not affect the outcome. 7 A generalzaton of the model could nclude a correlated error structure between the error terms of the latent varable equatons. 6

14 o 0 1 o 0 1,, Pr 1,, E y X Z Z y X Z Z F Z 1 F Z F Z F( X ) If the frst elements of vectors 0 Z and 1 Z are constants and the vectors 0 n (10) for 0,1 wth j 0 for j 0 and j 0 for j 0 we have F( 0) n (7). Accordngly, equaton (10) nests the basc parametrc model presented n Hausman et al. (1998), hereafter referred to as HAS1, allowng a statstcally testable proposton. 8 Assumng the functonal forms of F0, F 1 and F are known the parameters of the model can be estmated wth NLLS by mnmzng 2 n f,, y F Z 1 F Z F Z F( X ) (11) 1 over, 0, 1. Alternatvely, MLE can be appled to the followng log lkelhood functon: l,, 0 1 n y ln F Z 1 F Z F Z F( X ) (12) 1 y ln 1 F0 Z 0 1 F0 Z 0 F1 Z1 F( X ) n In the Monte-Carlo smulatons and the applcaton to real data that we present n subsequent sectons, all the parameter estmatons are based on MLE usng equaton (12) and also approxmate all three functons F0, F1and F above by a normal CDF. As we explaned earler, HAS1 s a specal case of our generalzaton, whch we refer to hereafter as GHAS, wthout any covarates affectng each type of msclassfcaton probabltes If F Z F Z (10) further collapses to a standard bnary choce specfcaton. However, as , dscussed n footnote 9, t s not possble to drectly test for ths condton. 7

15 The generalzaton of the Hausman et al. (1998) data generatng process n (5) that apples to the GHAS specfcaton s gven by y 1( y 0)1( u 1 F Z ) 1( y 0)1( u F Z ), 0 * 1 * where u ~ Unform 0,1, (13) and agan n (1) and u are ndependent. The nestng of HAS1 n GHAS and of the standard bnary choce model n HAS1 facltates statstcal testng for the most sutable model n a gven applcaton. The sgnfcance tests for parameters n 0 Z and 1 Z other than the constant terms serve as tests for the sutablty of GHAS over HAS1. Gven that no elements of 0 Z and 1 Z pass ths threshold, one may estmate HAS1 and the sgnfcant tests of the terms 0 and 1 serve as tests for the sutablty of HAS1 model over the standard bnary choce model 9. Identfcaton of the parameters of (12) stems from the non-lnearty of F. The frst order necessary condtons and the Fsher nformaton matrx of (12) can be expressed as below. L n 0 0 L 1 y 1 y g n C P P L 1 (14) 9 As noted above, the standard probt model, n general, s not nested n GHAS n a drectly testable manner and thus we propose ths sequental procedure. As the msclassfcaton probablty, k for k 0,1reaches 0, approaches the lower bound of F whch s n case of a normal dstrbuton, potentally leadng to convergence ssues. As such, convergence ssues of GHAS may ndcate a msspecfed model and that HAS1 could be a more approprate choce. Z k k 8

16 2 2 2 L L L ' ' ' n L L L 1 1 ' ' ' ' P 1 P I E n C C L L L ' ' ' where P F Z 1 F Z F Z F( X ), (15) 0 1 ' 1 F0 Z 0 F1 Z 1 F ( X ) X ' 0 0 C 1 F( X ) F0( Z 0) Z and ' 1 1 F( X ) F1( Z 1) Z and F respectvely. F, F and ' 0 ' 1 F, F ' F are the frst dervatves of 0 1 from When F 0 and F are symmetrc and F1 F0 dentfcaton requres that 1 Z. To demonstrate ths pont consder the case where 0 Z be dfferent Z Z Z, F ( v) F ( v) 1 F ( v) and F( v) 1 F( v). Then the log-lkelhoods, l 0 1 l 1 0 0,,,,. Hence, to dentfy,, from,, , we need 1 0 Z Z. However, f F s asymmetrc or F1 F0, we do not necessarly requre ths excluson 0 1 restrcton to dentfy the model parameters. 0 A mert of our estmator, however, s that the dentfcaton does not requre X Z or X 0 1 Z when Z Z and F (), 0 F () 1 and F() are non-lnear transformatons. Addtonal 1 exclusve restrctons wll help strong dentfcaton of parameters but are not necessary. 0 1 Moreover, f Z Z we no longer need as Hausman et al. (1998) requres. In spte of these advantages our estmator has certan lmtatons too. The Hausman et al. estmator 9

17 allows the msclassfcaton probabltes to be zero (but not 1, snce that would volate the monotoncty condton). If and/or F F F F ' 1 ' F F 0 F F 1 0 as ' 1 ' s the case wth most functonal forms 10, ours requre each type of msclassfcaton probablty to be bounded between 0 and 1, and not at the possble extremes, because f ether of the two types of msclassfcatons takes an extreme value, the matrx CCbecomes sngular. A related ' consequence would be large standard errors when msclassfcaton probabltes are too small or too large. In contrast to HAS1, our estmator performs best when the msclassfcaton probabltes are large n both drectons. If the msclassfcaton s known to be one-sded we can use a restrcted verson of our model by mposng F0 0 or F as approprate, 1 0 crcumventng ths dentfcaton ssue whle mprovng the effcency of the estmator. 3. MONTE CARLO EXPERIMENT In order to assess the mpact of covarate-dependent msclassfcaton on estmates wth and wthout an approprate correcton mechansm we set up a Monte Carlo experment whch mmcs the experment used n Hausman et al. (1998). We frst generated the X matrx n equaton (1) ncludng three random varables and a constant as covarates. Our X matrx s dentcal to the one they use n secton 4 of ther paper and comprses of x 1, drawn from a lognormal dstrbuton, x 2, a dummy varable equal to one wth probablty 1/3, x, a 3 unform 0,1 random varable, and a constant. The econometrc error term,, was drawn from a standard normal dstrbuton. The parameter vector also s dentcal to thers. Based on ths data generaton process, the latent dependent varable s gven by, 10 A notable excepton s the unform dstrbuton functon. 10

18 where X 1 x x x, * y X ' (16) In our experment the two types of msclassfcaton probabltes are functons of subsets of X. More specfcally, we have desgned our experment such that, the covarates n equatons (7) and (8) are gven by Z 1 x and Z 1 x x. Denotng , gven the dstrbuton of Z 0 and Z 1, the expected values of 0 and 1 n equatons (8) and (9) are, respectvely, E 0 E Z0 0 d 3 (17) E E Z d d (18) where denotes the normal dstrbuton functon. For consstency wth the Hausman et al. (1998) experment, we choose the parameter vectors 0 and 1 such that the expected value of each of the two types of msclassfcaton probabltes are both approxmately equal to the varous values they used for 0 and 1 (0.02, 0.05, and 0.2), by numercally ntegratng (16) and (17) usng Gauss-Legendre quadrature. We also ran an experment wth symmetrc mean msclassfcaton at 0.1, and two more sets of Monte Carlo experments wth asymmetrc and larger msclassfcaton probabltes, ( 0, 1) =(0.3, 0.75) and (0.75, 0.3). The observed and dependent varable, o y, was generated by addng msclassfcaton accordng to equaton (13). For each set of parameters, we generated a random sample, and used that sample to estmate the model parameters usng, () the standard probt model (Probt); () HAS1; and () GHAS. The results are based on 200 Monte Carlo runs, each wth a random sample of

19 observatons, for each of the sets of parameter values descrbed n the precedng paragraph. The standard errors reported are the standard devatons of each set of 200 estmates. Our fndngs wth regard to probt estmates, shown n table 1, though based on a dfferent data generatng process, are broadly n lne wth the fndngs of Hausman et al., (secton 4): () Even n the case of a small amount of msclassfcaton, ordnary probt produces estmates that are based by 15-25%; () The problem worsens as the amount of msclassfcaton grows; () Not only does probt yeld nconsstent estmates, but t can also overstate the precson of the estmates. Our results show that the three observatons are vald, not only for the case wth random msclassfcaton, but also for the more general case wth covarate-dependent msclassfcaton. The problems wth the ordnary probt model n the presence of a msclassfed dependent varable, whether random or covarate-dependent, are not small sample problems and thus cannot be overcome by ncreasng the sample sze. As the sample sze ncreases, Z and Z E approaches ther expected values E 0. The consstency of the ordnary probt estmator requres 0, E 1 ˆ MLE 0,0 ˆ MLE E whch s not the usual case. The overstated precson of estmates, together wth a sgnfcant bas of estmates s a more severe ssue than havng the based estmates alone. Even when the msclassfcaton probabltes are 5%, ordnary probt estmates are at least two standard devatons away from the true values, and any statstcally sgnfcant estmates are but a mere lluson due to the false precson, possbly leadng a researcher towards ncorrect conclusons. The problem worsens as the msclassfcaton probabltes ncrease. and 12

20 Despte not beng the correct model, one may expect HAS1 to perform better than the ordnary probt model n the presence of covarate-dependent msclassfcaton. As the result show, there s no guarantee that HAS1 wll perform better, even though t may partally correct the bas under certan condtons. More specfcally, when the msclassfcaton probabltes are small and only depend on one or few covarates whch are ndependent of the covarates of the man equaton, HAS1 s a better alternatve than the conventonal probt model. In real applcatons, however, msclassfcaton probabltes may be large and may depend on a large number of covarates; hence the random component of msclassfcaton may be much smaller relatve to the systematc component. Under such condtons HAS1 may ncrease the bas whle also reducng the effcency and thus may not be a better opton than ordnary probt. When comparng the coeffcent estmates of each of the three models presented n Table 1, one must be cautous of the scale nvarablty of the standard bnary choce model. As Mroz and Zayats (2008) show, drect comparsons and nterpretatons of arbtrarly scaled coeffcents from dfferent estmaton approaches may not be approprate; the relatve effects or the coeffcent ratos could be a better measure of comparson. The dea s that the rato cancels out the common scale factor. Although the magntude dfferences are less severe, the superorty of GHAS prevals even when evaluatng coeffcent ratos. For example, when the symmetrc msclassfcaton probabltes are 0.1, the average rato of the estmated beta1/ntercept n the Monte Carlo wth probt was , compared to for HAS1 and for GHAS, whle the true value of the rato s Ths pattern perssts over for all parameters and all msclassfcaton probabltes These results are avalable from the authors. 13

21 As our expermental results show, the superorty of GHAS over HAS1 and ordnary probt becomes more apparent both wth the ncreased msclassfcaton probabltes and wth the ncreased heterogenety of the dstrbuton of msclassfcaton probabltes. Ths holds when the probablty of msclassfcaton s symmetrc or asymmetrc. We ntentonally used the two last sets of parameters to show a potental outcome of probt estmates when the msclassfcaton probabltes (n expectaton) are so hgh that the HAS1 monotonocty condton s not satsfed. Msclassfcaton probabltes of these magntudes are not always unrealstc. In fact, we may not abandon a project due to large covarate dependent msclassfcaton probabltes, partcularly when one or both the msclassfcaton probabltes are very large wth a specfc sub group, but small wth others. If we gnore msclassfcaton and use probt estmates, as the results show, the co-effcent estmates are not only based downward but also may show up wth ther sgns toggled. Accordngly, large msclassfcaton probabltes could lead to one or more of the followng consequences wth regard to the probt estmates.. Downward based estmates wth the same sgn and wth reduced statstcal sgnfcance;.. v. The coeffcent of an mportant varable may appear to be nsgnfcant; The bas could be suffcently large to flp the sgn of the estmate; An nsgnfcant varable may appear to be sgnfcant f t affects msclassfcaton probabltes; and/or v. The estmates may show an mpact larger than the true mpact. The HAS1 model should not be employed when the msclassfcaton probabltes are large. When mean msclassfcaton probabltes sum to a value greater than 1, volatng HAS1 monotoncty requrement, as shown n tables 2 and 3, HAS1 n general predcts very low or zero 14

22 msclassfcaton. In addton, HAS1 coeffcent estmates are not qualtatvely dfferent from the based probt estmates. Typcally the magntudes of the msclassfcaton probabltes are not known; usng HAS1 when the means of the msclassfcaton probabltes are large and systematc may mslead a researcher nto belevng msclassfcaton s not a problem. In addton to ts superorty over other models n precsely estmatng the coeffcents of the man equaton, GHAS also helps to correctly and precsely estmate the mpact of each covarate on the two type of msclassfcaton. As the results reported n tables 2 and 3 ndcate, to precsely estmate the parameters of equatons (8) and (9) when the msclassfcaton probabltes are small, we need a szable sample. However, when the msclassfcaton probabltes are large, those parameters can be estmated wth a hgh precson even wth a relatvely small sample. As a fnal check, we tested what damage s done f we use GHAS when the probabltes of msclassfcaton are not covarate dependent, so HAS1 would be more approprate. Not surprsngly HAS1 s more effcent than GHAS. However, usng GHAS when the msclassfcaton s not covarate dependent does lttle harm. These results are reported n the appendx (Tables A1-A3). 4. APPLICATION TO ESTIMATE THE EFFECIVENESS OF A FAMILY IMPROVEMENT PROGRAM We demonstrate the applcablty of GHAS by usng t to estmate the determnants of mprovement n famly functonng after partcpatng n the Strengthenng Famles Program for Parents and Youth (SFP) n Washngton State and Oregon. For comparson we estmate the same model usng HAS1 and ordnary probt. The Strengthenng Famles Program (SFP) s an nternatonally recognzed parentng and famly strengthenng program for hgh-rsk famles. 15

23 The program s desgned to be delvered n local communtes for groups of 7-12 famles. Famles attend SFP once a week for seven weeks and partcpate n educatonal actvtes that brng parents and ther chldren together n learnng envronments desgned to strengthen entre famles through mproved famly communcaton, parentng practces, and parents famly management sklls Applcablty of the Model The dependent varable of our applcaton s a bnary ndcator equal to 1 f a partcpant s self-reported famly functonalty after the program s hgher than the pre-program functonalty. Ths ndcator varable s derved usng the pre-treatment and post-treatment scores measured on a Lkert scale. One fundamental assumpton that we make here s that there are true (latent) objectve scores before and after the treatment, but nether the researcher nor the respondent observes these true values. Each partcpant makes a subjectve assessment of her score and then translates t nto an nteger value wthn the range of the Lkert scale used by the researcher. Response bas s the dfference between the subjectve measures of same objectve outcome used by dfferent ndvduals, whle response shft bas comes from the response bas of the same ndvdual changng at two measurement ponts (Sprangers and Hoogstraten, 1989; Hll and Betz, 2005). Our study s essentally a before-after comparson at the surface. However, under certan assumptons the comparson s equvalent to true treatment effect. The famly functonalty of a household, the target of the nterventon that we dscuss here, n general s a slowly-changng varable and hghly unlkely to change autonomously wthn a 7-week perod, the duraton of the nterventon. Ths assumpton leads two more results. Frst, any change n the

24 famly functonalty of a partcpant s household s due to the program effect snce the mpact of any other potental factors s neglgble. Second, the famly functonalty of non-partcpants does not change durng ths short perod. The two results together mply that the before-after comparson s a good practcal measure of treatment effect n our case. Our concern here s the msclassfcaton of the ndcator varable of mprovement that we derve. Suppose both pre-treatment and post-treatment scores reported by each partcpant nclude response bas. If the magntude of the bas remans unchanged after the program the reported scores show the true change. The ssue we face here s that the nterventon not only changes the famly functonalty, but also the knowledge about what good functonalty s. As a result, partcpants may recalbrate ther metrcs used to measure and report famly functonalty after the program. As an example, suppose a partcpant reports her pre-treatment score s 2. After the program her famly functonalty has not changed but due to recalbratng her metrc she realzes that her ntal score should be 3, whch she reports as her post-treatment score, seeng no mprovement n her famly functonalty from the program. A researcher now observes an mprovement whle she really has not mproved, contrbutng to msclassfcaton probablty 0. Suppose another partcpant reports her pre-treatment score as 4 but after recalbratng the metrc she fnds that her true score before the program should have only been 3 and now t has mproved to 4. The researcher observed no mprovement whle she has really mproved and we have msclassfcaton type 1. Rosenman et al. (2011) has shown substantal response bas and response shft bas n SFP data. In addton to the msclassfcaton n our bnary varable due to response shft bas we suspect there s also Lkert mbalance bas (Tennekoon and Rosenman, 2012). Lkert mbalance 17

25 bas occurs when subjectve measures are translated to a Lkert scale value and may complement response shft bas. By ths nature the msclassfcaton n our varable s probably not random. Any response shft change of a partcpant after the treatment lkely depends on famly and socal background ncludng her demographcs and the characterstcs of her SFP group, makng HAS1, whch assumes constant msclassfcaton probabltes, a poor choce. The mpact of Lkert mbalance bas too s uneven across partcpants wth dfferent reported pre-treatment famly functonng levels. In partcular, the partcpants at one of the extremes of the Lkert scale pror to the program are more lkely to unntentonally msreport. Avalable SFP data are lmted and we only have some demographc nformaton and reported pre-treatment and post-treatment scores of partcpants, whch mpacts not only the mprovement n famly functonalty but also the msclassfcaton probabltes. Accordngly, we have no way to proceed wth the Lewbel (2000) approach, whch requres at least one contnuous varable affectng the mprovement but not msclassfcaton, even f we gnore the computatonal complexty of hs approach. In addton to the varables that we have at hand, unobserved ndvdual effects are lkely to affect the true mprovement n famly functonalty as well as the bas hence a normal dstrbuton appears to be the best functonal form choce for these unobserved effects. Ths motvates us to choose normal CDFs for F0, F 1 and F. We have one varable, a dummy equal to one f the pre-score s near the upper bound, to dfferentate 1 Z from 0 Z, whch s unlkely to 18

26 affect 0. Snce we assume F1 F0 and our F s symmetrc, we need ths excluson restrcton to dstngush 4.2 Data from,,, 0, Our data conssted of 1,437 observatons of parents who attended one of the 94 SFP cycles n Washngton and Oregon states through Varables used n the analyss, ncludng defntons, and summary statstcs are presented n Table 4. The average famly functonng, as measured by the change n self-assessed functonng from the pretest to the posttest ncreased from 3.98 to 4.27 after partcpaton n SFP. Seventy-one percent of the partcpants showed an mprovement n famly functonng. The remanng 29% showed ether a negatve or no change n famly functonng. Twenty-fve percent of the partcpants dentfed themselves as male, 72% as female, and 3% dd not report ther gender. Twenty-seven percent of the partcpants dentfed themselves as Hspanc/Latno, 60% as Whte, 2% as Afrcan-Amercan; 4% as Amercan Indan/Alaska Natve, and 3% as other or multple race/ethncty, whle 3% of the partcpants dd not report ther race/ethncty. Seventy-four percent of the partcpants reported that they are lvng wth a partner or a spouse, and 19% reported not havng a spouse or partner. Almost 8% of partcpatng parents dd not report whether they are lvng wth a partner or a spouse. The average of the wthn-cycle average pre-score was 3.99, not statstcally dfferent from the overall average prescore of The average of the wthn-program standard devaton of pre-score was 0.499, compared to the overall standard devaton of pre-score of The mplcatons of these 13 0 As explaned n secton 2, our model allows Z and Z 1 to be subsets of X and even one to be equal to X. Accordngly, our use of GHAS s not constraned by the unavalablty of addtonal excluson restrctons n vector X. 19

27 statstcs are that there does not seem to be much varaton n the attendees of dfferent cycles. Around 3% of the sample had reported pre-score values larger than 4.9. We used the two gender related varables, the fve varables related to race/ethncty, the two varables related to partner/spouse, age, pre-score, wthn-program average and standard devaton of pre-score (despte the seemng consstency n those attracted to the program whenever and wherever t was offered) and a constant as the covarates of the man equaton. Our covarates determnng the propensty to record mprovement as no-mprovement ( 0 ) were three race categores (natve and other categores were combned wth the category who dd not report ther race/ethncty) 14, age, pre-score, a dummy equal to 1 f the pre-score s larger than 4.9, and a constant. As the covarates determnng the propensty to record no-mprovement as mprovement ( 1 ) we used the same three race related varables, age, pre-score and a constant. The choce of these varables was partly motvated by the fndngs of Rosenman et al. (2011). The dummy varable pre-score 4.9 was used as a covarate because people wth very hgh ntal functonng have lttle room to show mprovement, even f they mprove. Ths varable helps specfcally to capture Lkert scale bas, whle servng as an excluson restrcton. A smlar varable was not ncluded among the covarates of equaton (6) because only 3 partcpants had pre-scores below 1.5 and the lowest value of the scale, unlke the hghest value, dd not appear to be bndng. 4.3 Analyss of Results The results from GHAS, together wth the results of HAS1 and tradtonal probt, are presented n tables 5 and 6. Accordng to the tradtonal probt model, mprovement after 14 Ths combned category was not sgnfcantly dfferent from whtes. The result was robust when we used the three categores separately but the standard errors were very large. 20

28 partcpatng n SFP s a functon of four covarates. Male partcpants are less lkely to mprove after the program than are females and those who dd not report ther gender; Afrcan Amercans are less lkely to mprove than are other race categores; those who dd not report whether they are lvng wth a partner or a spouse are less lkely to mprove than the partcpants who reported that nformaton; and, partcpants wth hgher pre-scores are less lkely to mprove than the partcpants wth lower pre-scores. HAS1 fnds mprovement covarates qualtatvely smlar to those found wth the ordnary probt, but predcts msclassfcaton probabltes as well. Accordng to the results, the probablty that a partcpant wth no mprovement reportng an mprovement ( 0 ) takes the lowest possble value, zero. The model also predcts a 3.2% probablty that partcpants who mproved ther famly functonng after the program may report that they have not mproved ( 1 ). The GHAS estmates are notceably dfferent from those found wth ordnary probt and HAS1, albet not wthout some smlartes. In contrast to HAS1, GHAS ndcates that the msclassfcaton probabltes n each drecton are substantal (based on model predctons) and depends on several covarates. When consderng 0, the coeffcents of Hspanc dummy, age and pre-score are sgnfcant. However, the coeffcent of the constant term s not sgnfcant confrmng that the random component of msclassfcaton s not sgnfcant. Older partcpants, partcpants wth Hspanc orgn and people wth self-perceved low ntal famly functonng levels are more lkely to show mprovement even when they do not mprove. Accordng to GHAS, the probablty that true mprovement would be reported as nomprovement ( 1 ) also depends on several covarates. Among the statstcally sgnfcant 21

29 determnants of 1 are the constant term, age, pre-score, and pre-score beng close to the upper bound. The results suggest that older people and people wth hgh ntal famly functonng levels are more lkely to msclassfy mprovement as not happenng. Consstent wth Lkert Scale Bas, people wth ntal functonng levels closer to the upper bound of the scale have very lttle or no room to show any mprovement and therefore are also lkely to be msclassfed. The coeffcent of the constant term, albet statstcally hghly sgnfcant, s small n magntude, suggestng that the random component of msclassfcaton n that drecton too s small, consstent wth the results of HAS1. Our most mportant result, especally n lght of the Monte Carlo analyss, s that the predctors of mprovement found wth GHAS model are not the same as those found consstently usng HAS1 and probt. The male and Afrcan Amercan dummes, whch were sgnfcant n HAS1 and probt, are not sgnfcant n GHAS. Pre-score and the constant term contnue to be sgnfcant, but wth opposte sgns. In addton, several varables that were ndcated not mportant by HAS1 and probt are sgnfcant at conventonal levels usng GHAS. GHAS ndcates that Hspancs are more lkely to mprove than Whtes, that the partcpants from twoparent famles are more lkely to mprove than sngle parents, as are the group that dd not report the detals of ther partner/spouse. Partcpants who do not report ther gender or race, however, are less lkely to mprove than the partcpants who report ther nformaton. Fnally, programs wth partcpants from ntally better functonng famles and programs wth more heterogeneous partcpants n terms of ther pre-scores are more successful than other programs. Of the dfferences, the most mportant s that GHAS ndcates that better functonng famles are more lkely to mprove than poor functonng famles, a fndng that contrasts wth what was found wth ordnary probt and HAS1. However, when the ntal functonng 22

30 ncreases, t ncreases not only the propensty to mprove, but also the propensty to be msclassfed and not to show the mprovement. Ths explans why ordnary probt, whch does not account for ths msclassfcaton, and HAS1, whch does not account for the dependence of msclassfcaton on ntal functonng, show the opposte. The expected values of msclassfcaton probabltes predcted by GHAS, E and E 1.339, are very large and sharply contrast wth the HAS1 estmates ( 0 0 and ). The results, however, are n conformty wth the fndngs of the Monte Carlo study, whch showed a severe underestmaton of msclassfcaton probabltes by HAS1 when they are systematc and of these magntudes. Gven the dfference n results, one must wonder whch model s the most approprate. Overall, GHAS has the best ft among the three models n terms of the log-lkelhood, adjusted pseudo R-squared (McFadden) and the number of successful predctons (Table 7). The model, successfully predcts 1,079 of 1,437 outcomes as reported by the data (75.1%), and estmates that 1,264 partcpants (88.0%) really mprove after the SFP program compared to the reported 70.8%. The probt estmate of the number of people mproved, for comparson, s 990 (68.9%) whch, perhaps not surprsngly, s very close to the observed number. HAS1 lags sgnfcantly n the number of correct predctons of the data as a whole and reports, by far, the smallest number of partcpants who actually mproved. Snce HAS1 reports there s no probablty of someone who mproved recordng themselves as not mproved and a postve probablty someone who mproved reportng that they dd not, ths ndcates that the man equaton serously underreports the predcted mprovement, callng nto queston the valdty of ts results. Accordngly, the 23

31 ultmate effect of msclassfcaton n our observed data could well be a serous underestmaton of SFP s effcacy, unless corrected approprately, wth systematc msclassfcaton. 5. CONCLUSIONS When the dependent varable s msclassfed, parameter estmates of the bnary choce model are based and nconsstent, a condton exacerbated f the msclassfcaton s systematc rather than random. Although nonparametrc methods can provde consstent estmates of model parameters, those that also provde estmates of msclassfcaton (whch may be of sgnfcant nterest to polcy makers) are cumbersome and often mpossble to mplement because of addtonal data needs. We provde a straghtforward method to properly account for endogenous msclassfcaton that provdes both consstent estmates of the model parameters and yelds estmates of msclassfcaton probabltes for the sample and for each ndvdual. Our expermental results document the mportance of controllng for endogenous msclassfcaton, and demonstrate that lttle harm s done f our approach s used for random msclassfcaton. Moreover, our results ndcate that possble systematc msclassfcaton s not a factor that a researcher can smply gnore. The presence of systematc msclassfcaton can toggle overall conclusons and lead analysts to substantally underestmate program benefts. Our applcaton to real data from the Strengthenng Famles Program shows how large msclassfcaton can be wth subjectve self-reported data, and how t can radcally affect parameter estmates. The ultmate goal of evaluatng the effcacy of a treatment s dentfyng ts costs and benefts, whether the treatment s preventve, curatve or educatonal. If the results produced are spurous, the researchers and any other users of such results may easly end up wth wrong conclusons, whch may have severe polcy mplcatons. The model presented here provdes an 24

32 effectve and easly mplemented way to deal wth the ssue and estmate treatment effects more accurately. The applcablty of GHAS to the research problem we explaned does not prove ts superorty under all stuatons. Snce MLE consstency s an asymptotc property, the relatve merts of GHAS and HAS1 are not clearly vsble when ether the sample sze s small or the msclassfcaton probabltes are small. In our applcaton we gnored the mpact of a potental selecton bas that could arse f the partcpants of SFP are systematcally dfferent from the non-partcpants. We can easly correct for selecton bas by combnng a selecton probt equaton wth equaton (12) and estmatng a modfed bvarate probt wth selecton. Lmtatons of our data dd not allow us to pursue ths extenson, although t s straghtforward. If there s reason to beleve that there are unobserved varables that affect the outcome as well as the msclassfcaton probabltes, t may be approprate to allow the error terms to be correlated, whch s also straght forward. Fnally, a msclassfed polychotomous varable can be dealt wth by enhancng the models presented n Abrevaya and Hausman (1999) and Dustman and van Soest (2000) n a manner smlar to ours. 25

33 REFERENCES Abrevaya J, Hausman JA Semparametrc Estmaton wth Msmeasured Dependent Varables: An Applcaton to Panel Data on Employment Spells. Annales D'Econome et de Statstque 55-56: Chua TC, Fuller WA A Model for Multnomal Response Error Appled to Labor Flows. Journal of the Amercan Statstcal Assocaton 82: Dustmann C, van Soest A Parametrc and Semparametrc Estmaton n Models wth Msclassfed Categorcal Dependent Varables. IZA Dscusson Papers no Hausman J Msmeasured Varables n Econometrc Analyss: Problems from the Rght and Problems from the Left. The Journal of Economc Perspectves 15(4): Hausman JA, Abrevaya J, Scott-Morton FM Msclassfcaton of the Dependent Varable n a Dscrete-Response Settng. Journal of Econometrcs 87: Hll LG, Betz D Revstng the Retrospectve Pretest. Amercan Journal of Evaluaton 26: Lewbel A Identfcaton of the Bnary Choce Model wth Msclassfcaton. Econometrc Theory 16(4): Mroz TM, Zayats YV Arbtrarly Normalzed Coeffcents, Informaton Sets, and False Reports of Bases n Bnary Outcome Models. The Revew of Economcs and Statstcs 90(3): Poterba JM, Summers LH Unemployment Benefts and Labor Market Transtons: A Multnomal Logt Model wth Errors n Classfcaton. The Revew of Economcs and Statstcs 77:

34 Rosenman R, Tennekoon V, Hll LG Bas n Self Reported Data. Internatonal Journal of Behavoural and Healthcare Research 2(4): Sprangers M, Hoogstraten J Pretestng Effects n Retrospectve Pretest-Posttest Desgns. Journal of Appled Psychology 74(2): Tennekoon, V., Rosenman, R Lkert Imbalance Bas, manuscrpt, School of Economc Scences, Washngton State Unversty, Pullman, WA. 27

35 Table 1: Determnants of Pr (y=1) wth covarate dependent msclassfcaton (coeffcents) Varable E 0 = 1 E =0.02 True Value Probt HAS1 GHAS Est. Std. Err. Est. Std. Err. Est. Std. Err. Intercept beta beta beta E 0 = 1 E =0.05 Intercept beta beta beta E 0 = 1 E =0.1 Intercept beta beta beta E 0 = 1 E =0.2 Intercept beta beta beta E 0 =0.3, E 1 =0.75 Intercept beta beta beta E 0 =0.75, 1 E =0.3 Intercept beta beta beta

36 Table 2: Determnants of Pr (yo=1 y=0) wth covarate dependent msclassfcaton Varable E 0 = 1 E =0.02 True Value HAS1 Std. Est. Err. Est. GHAS Std. Err. Intercept gamma E( ˆ0 ) E 0 = 1 E =0.05 Intercept gamma E( ˆ0 ) E 0 = 1 E =0.1 Intercept gamma E( ˆ0 ) E 0 = 1 E =0.2 Intercept gamma E( ˆ0 ) E 0 =0.3, 1 E =0.75 Intercept gamma E( ˆ0 ) E 0 =0.75, 1 E =0.3 Intercept gamma E( ˆ0 )

37 Table 3: Determnants of Pr (yo=0 y=1) wth covarate dependent msclassfcaton Varable E 0 = 1 E =0.02 True Value HAS1 GHAS Est. Std. Err. Est. Std. Err. Intercept gama gama E( ˆ1 ) E 0 = 1 E =0.05 Intercept gama gama E( ˆ1 ) E 0 = 1 E =0.1 Intercept gama gama E( ˆ1 ) E 0 = 1 E =0.2 Intercept gama gama E( ˆ1 ) E 0 =0.3, 1 E =0.75 Intercept gama gama E( ˆ1 ) E 0 =0.75, 1 E =0.3 Intercept gama gama E( ˆ1 )