How to Find the Poor: Field Experiments on Targeting. Abhijit Banerjee, MIT

Transcription

1 How to Find the Poor: Field Experiments on Targeting Abhijit Banerjee, MIT

2 Why is targeting hard? Targeting entails a different set of challenges in developing countries because governments lack reliable data on incomes Several methods used to address this problem entail a tradeoff between information and local preferences: Proxy-means testing (PMT): government collects data on hard-tohide-assets to proxy for consumption Community-based targeting: allow local community discretion to decide who is poor Self-selection: allow people to apply and make the application process costly May be then do a PMT

3 Which of these works best? The Indonesian government wanted to know Not an easy question: Different programs are targeted differently but they are also different in fifty other ways Also within each model what is the optimal design? Community targeting by the whole community Or just by the elites? How costly should the application process be? One extreme is National Rural Employment Guarantee Act (NREGA) Another extreme is a one time application cost Evidence from experiments

4 Project 1: PMT vs. Community Targeting This study examined a special, one-time real transfer program operated by the government of Indonesia Beneficiaries would receive a one-time, US$3 transfer (PPP$6) Goal of program was to target those with per-capita consumption less than PPP$2 per day Sample consists of 640 sub-villages (rural and urban) across 3 provinces in Indonesia

5 The PMT Method Government chose 49 indicators, encompassing the household s home (wall type, roof type, etc), assets (own a TV, motorbike, etc), household composition, and household head s education and occupation Use pre-existing survey data to estimated districtspecific formulas that map indicators to PCE Government enumerators collected asset data doorto-door PMT scores calculated, and those below villagespecific (ex-ante) cutoff received transfer

6 The Community Method Goal: have community members rank all households in sub-village from poorest ( paling miskin ) to most welloff ( paling mampu ) Method: Community meeting held, all households invited Stack of index cards, one for each household (randomly ordered) Facilitator began with open-ended discussion on poverty (about 15 minutes) Start by comparing the first two cards, then keep ranking cards one by one Also varied who was invited (elites or everyone)

7 Hybrid Hybrid combined community with PMT verification of very poor The idea was that the community could pre-screen those to be put in to the PMT

8

9

10 Time Line Baseline Survey Nov to Dec 2008 Targeting Dec 2008 to Jan 2009 Fund Distribution, complaint forms & interviews with the subvillage heads Endline Survey late Feb and early Mar2009 Feb 2009

11 Distribution of Per Capita Cons. PMT centered to the left of community methods better performing on average However, community methods select slightly of the very poor (those below PPP$1 per day) On net, beneficiaries have similar average consumption

12 MISTARGET ivk = α + β 1 COMMUNITY ivk + β 2 HYBRID ivk + γ k + ε ivk (1) Full Sample: population Community treatment 0.031* (0.017) Hybrid treatment 0.029* (0.016) Observations 5753 Mean in PMT treatment 0.30 Using the $2 per capita expenditure cutoff per day, 3 percentage point (or 10 percent) increase in mistargeting in community and hybrid over the PMT

13 Community Satisfaction: Endline Is the method applied to determine the targeted households appropriate? (1=worst,4=best) Are you satisfied with P2K08 act ivities in this sub-village in general? (1=worst,4=best) Are there any poor HH which should be added to the list? (0=no, 1 = yes) Community treatment 0.161*** 0.245*** *** (0.056) (0.049) (0.040) Hybrid treatment (0.055) (0.049) (0.042) Observations Mean in PMT treatment Number of HH that should be added from list Number of HH that should be subtracted fro m list Number of complaints in the comment box Community treatment *** *** *** (0.158) (0.112) (0.286) Hybrid treatment ** (0.188) (0.129) (0.285) Observations Mean in PMT treatment

14 Understanding the differences Summary of results so far: Community methods did slightly worse based on PPP$ poverty line. No differences in average consumption Hybrid and community do exactly equally well Villagers overwhelmingly happier with community Why might PMT be different? 1. Elite capture No 2. Effort (community gets tired) Yes 3. Different concept of poverty Yes 4. Different information (community less accurate that PMT) Unlikely

15 1. Elites Elite subtreatment: In half of community/hybrid villages, only local small group of elites (neighborhood leader + a few others) invited to meetings In other villages, whole community invited Results: No change in average mistargeting rates No change in mistargeting rates for households who are family members of elites Only result is that in community treatment, elite connected households even less likely to be on list than PMT this reverse discrimination is eliminated to some degree in elite treatment

16 2. Community Effort To rank 75 households, must make 365 pairwise comparisons Community members may get fatigued throughout the meeting To investigate this, we randomized the order in which households were considered We find that those ranked early in the meeting were ranked more accurately in fact, more accurately than PMT

17 3. Does the Community Have a Different Concept of Poverty? Communities could be identifying who is poor, but have a different view of what constitutes poverty We therefore estimate: RANKCORR vkw = α + β 1 COMMUNITY vk + β 2 HYBRID vk + γ k + ε vkw where RANKCORR is the rank correlation in each village between targeting rank list and: Consumption (u g ) Community survey ranks (u c ) Sub-village head ranks (u e ) Self assessments (u s )

18 Hybrid closer to community preferences than PMT, but less so than community method Impact on different welfare metrics Community survey ranks (u c ) Subvillage head survey ranks(u e ) Self Assessment (u s ) Consumption (u g ) Community ** 0.246*** 0.248*** 0.102*** (0.033) (0.029) (0.038) (0.033) Hybrid ** 0.143*** 0.128*** 0.075** (0.033) (0.029) (0.038) (0.033) Observations Mean in PMT treatment Community methods have lower correlation between targeting rank list and consumption, but higher correlation between targeting rank list and self-assessment

19 Unpacking the local welfare metric To investigate this, we regress survey ranks and community ranks in a wide variety of household characteristics, conditional on per-capita consumption

20 Key findings Household equivalence scales: Community accounts for household economies of scale Community treats kids as more costly than adults (e.g., greater burden ) Networks for smoothing shocks Community counts elite-connected households as better off than indicated by consumption Community counts those who have high share of savings in banks as better off, even though total savings doesn t affect rank Those with family outside the village also counted as richer

21 Key differences Discrimination No discrimination against ethnic minorities Laziness or deservingness Those with only primary education treated as poorer, conditional on actual consumption Widows, disabled, and those with serious illness all treated as poorer, conditional on actual consumption

22 4. Is community information worse? Survey rank Survey rank (1) (2) Rank per capita consumption within 0.132*** 0.088*** village in percentiles (0.014) (0.012) Rank per capita consumption from 0.368*** PMT within village in percentiles (0.014) Individual PM Variables YES Unlikely: in fact, community members have additional information about consumption over the PMT Controlling for PMT score, a one percentile increase in consumption rank is associated with a percentile increase in individual household ranks of the community

23 Conclusions Decentralizing to the community: Increased local satisfaction with the outcomes Produced only slightly worse targeting based on percapita consumption Reason is primarily that local communities have a different welfare function than the central government

24 PMT vs. Self-selection + PMT One way to do so is to impose program requirements that are differentially costly for the rich and the poor Welfare programs with labor requirements (WPA, NREGA) Food schemes with lower quality food Unemployment schemes with weekly requirements to visit unemployment office during work hours Many self selection mechanisms rely on the idea that time is more costly for the rich than the poor

25 Other Factors that May Worsen Selection Introducing realistic features into the model (e.g. concave utility) may result in worse targeting, if poor have higher money costs, but lower utility costs Behavioral arguments: self control (Madrian and Shea, 2001); stigma (Moffitt, 1983); information (Daponte, Sanders and Taylor, 1999) Thus, whether self-section improves targeting is ultimately an empirical question

26 Setting for Project 2: Experiment Takes place in the context of Indonesia s Conditional Cash Transfer Program, PKH Must be very poor, defined as < 80% of poverty line High stakes: household annual benefits between Rp. 600,000 (US$66) and Rp. 2,200,000 (US$245) per year (11% consumption for a typical beneficiary) We examine the expansion of the program to 400 new villages in 3 provinces in Indonesia

27 Experimental Design To qualify for PKH: means test is determined by a PMT survey based on assets and demographics Randomize targeting method: Automatic Enrollment System (200 villages): Status Quo in Indonesia Self-Targeting (200 villages) Households must come to application site to apply, then take asset test Randomly varied travel time, opportunity cost of signing up

28 Explaining the Program

29 Application Process

30 Timeline Baseline Survey Consumption Travel costs to locations Variables for PMT formula Targeting and Intervention Government conducts targeting PKH funds begin to be distributed Endline Satisfaction Process questions: e.g. wait time during selftargeting

31 Who Self-Selects? Self Targeting may differ than an automatic enrollment system on two dimensions 1. Observable Characteristics: Households that would fail PMT test anyway may be less likely to come save government money since no longer have to interview them 2. Unobservable Characteristics: Noise in PMT test, and so richer households may select out of being tested can result in a poorer group of beneficiaries Decompose consumption into PMT score and residual

32 Observable Characteristics

33 Unobservables

34 Comparing across treatments Selection on both observable and unobservable components But, still in self-targeting, only 60% of those who are eligible show up! How does this compare against: Status Quo of Automatic Enrollment on those who have been selected to be interviewed Hypothetical Universal Automatic Enrollment

35 Self Targeting leads to a poorer distribution of beneficiaries

36 ST reduces both exclusion and inclusion error: 16 percent of those who are in the bottom 5 percent receive benefits in ST, as opposed to 7 in AE (sig at 10% level) Households in top 50 percent of consumption are more

37 Taking Stock of Experimental Results Self-targeting leads to a poorer distribution of beneficiaries, both because the poor are more likely to receive benefits and the rich are less likely Compare against hypothetical where everyone is surveyed for automatic enrollment leads to similar results: ST leads to a poorer distribution of beneficiaries (although significance depends on specification) Probability of poor being selected is similar under both systems, but wealthier people more likely to receive it under AE

38 Costs of Alternative Approaches ST places a greater total cost on households: $70,000 compared to $9300 in AE and $32,403 for universal AE In sample: Administrative costs for ST is about $171,000: it is about 4.5 times that cost for AE and about 13 times that cost for AE Assuming we treat costs by households and administrative costs the same, ST leads to a better distribution of beneficiaries at total lower costs

39 What is the Marginal Effect of Increasing the Ordeal? Difference between one sign up cost and NREGA Experimental Evidence: ST Villages were allocated to the following two sub-treatments: Application Site Close or Far Increasing distance does not improve selfselection just reduces massively application rates, even for the poorest

40 Conclusions These two projects investigated alternative approaches to identifying poor households Found that: Community targeting did about the same as PMT in terms of identifying people based on per-capita consumption, but much better in terms of local poverty metrics. Self-targeting did a much better job at differentiating between poor and rich than automatic PMT, although it does impose costs on applicant households However all the approaches miss a very large proportion of the poor Making the ordeal harder does not help.