Can a social media campaign increase the use of Long-Acting Reversible Contraception? Evidence from a randomized control trial using Facebook

Transcription

1 Can a social media campaign increase the use of Long-Acting Reversible Contraception? Evidence from a randomized control trial using Facebook Tanya Byker, Caitlin Myers and Maura Graff* April 2017 We implement a cluster randomized control trial testing the efficacy of a social media campaign on the use of long acting reversible contraception (LARC). Over 126,000 women ages 18 to 34 were exposed to an average of 14.4 Facebook advertisements describing the efficacy, ease-of-use and safety of IUDs and contraceptive implants. We find statistically significant but imprecise evidence that the information campaign increased LARC insertions by 6 percent, and discuss several possible interpretations including the possibility that the finding is spurious. Key words: Contraception, Social Media, LARC *Tanya Byker is Assistant Professor of Economics at Middlebury College ( tbyker@middlebury.edu). Caitlin Myers is Associate Professor of Economics at Middlebury College and a Research Fellow at IZA in Bonn, Germany ( cmyers@middlebury.edu). Maura Graff is Director of the Project to Reduce Unintended Pregnancy at Planned Parenthood of Northern New England ( Maura.Graff@ppnne.org). This experiment was reviewed and approved by the Middlebury College Institutional Review Board (Proposal 15687). The authors wish to thank members of the Middlebury College Department of Economics for useful feedback on the study design. We also thank staff members of Planned Parenthood of Northern New England: Meagan Gallagher, Donna Burkett, Jill Krowinski, and Yvonne Lockerby for feedback on the design, Amy Lafayette and Erica Viscio for assistance with the advertisement campaign, and Eve Benen for providing the health center data. We are grateful to Evan Deutsch and Jon Portman at Oxbow Creative for designing the advertisements, and also to Anna Cerf and Birgitta Cheng for expert research assistance. We also thank students in Caitlin Myers Unplanned Parenthood course at Middlebury College for additional helpful comments and feedback. The findings and conclusions in this article are those of the authors and do not necessarily represent the views of Planned Parenthood Federation of America, Inc.

2 Introduction Despite the widespread availability of safe and reliable forms of contraception, nearly half of all pregnancies in the United States are unintended (Finer and Zolna, 2016). The primary explanation is contraceptive failure due to inconsistent use. Nine out of ten sexually active women who wish to avoid pregnancy report that they use contraception (Jones et al., 2002), but the most popular methods the birth control pill and condoms are not the most effective. Women experiencing an unintended pregnancy often report that they did not take the pill consistently (Jones et al., 2002; Finer and Henshaw, 2006), human error that accounts for the gap between the pill s sub-one percent failure rate in clinical trials and 9 percent failure rate in typical use (Trussell, 2011). In contrast to the high typical-use failure rates of the contraceptive pill, long acting reversible contraception (LARC) including intrauterine devices (IUDs) and subdermal hormonal implants require no user action for three to twelve years, and clinical and typical-use failure rates for LARCs are virtually identical and extremely low (Trussell, 2011). For this reason, public health advocates have increasingly endorsed and encouraged LARC. The American Academy of Pediatrics recommends that LARC methods be considered first-line contraceptive choices for adolescents (AAP, 2014), while The American College of Obstetricians and Gynecologists endorses LARC as top-tier contraceptives (ACOG, 2012). Despite these endorsements and the potential to substantially reduce unintended pregnancy, LARC remains relatively uncommon in the United States. Among contracepting women in the United States, LARC use has increased from about 2 percent in 2002 to nearly 12 percent in 2012, but it remains low relative to other industrialized countries (Kavanaugh et al., 2015). LARC use is even lower among young women and adolescents who are at greatest risk of unintended pregnancy (Finer et al., 2012). The Contraceptive CHOICE Project in St. Louis (Secura et al., 2014) and the Colorado Family Planning Initiative (Ricketts et al., 2014) have demonstrated that major interventions designed to increase awareness and decrease the cost of LARC are effective. But these programs also illustrated the 1

3 financial and political costs of large-scale efforts to expand LARC access and use. Our goal is to test whether a different type of intervention, a simple and low-cost information campaign carried out on social media, can influence LARC awareness and use. Following random assignment of Northern New England zip codes to treatment and control groups, over 126,000 women residing in treated zip codes were exposed to an average of 14 Facebook advertisements sponsored by Planned Parenthood of Northern New England. These advertisements featured information on LARC efficacy, ease of use, and safety. We evaluated changes in LARC awareness and attitudes using a follow-up survey, also distributed via Facebook, to women in both treatment and control groups. Changes in LARC uptake were evaluated using a difference-in-difference research design and clinical data from Planned Parenthood clinics located in the three targeted states. The results suggest that the intervention increased LARC insertions by 5.5 percent (p=0.036). This result, however, is driven almost entirely by patients at a single large health center, leading to concerns that the findings may be spurious. We discuss possible contaminating effects, highlighting limitations of even gold-standard experimental approaches. Barriers to LARC Trussell (2009) estimates that the average upfront cost of LARC, including the cost of the device, office visit and insertion, was $724 for a Mirena IUD and $814 for an Nexplanon contraceptive implant in These one-time upfront costs are much higher than those for the contraceptive pill ($93 for an office visit and 1 month of pills), and can be prohibitive for women seeking contraception. However, when averaged over 5 years and taking into account the long duration of efficacy and also the medical costs of contraceptive failure, LARC is more cost-effective than any other form of reversible contraception (Trussell, 2009). Following the 2010 enactment of the Affordable Care Act, LARC coverage has expanded for women covered by Medicaid and private insurance. Women living in states 2

4 that have opted into the Medicaid expansion have access to LARC with no co-pay, and the majority of privately insured women seeking LARC also can now obtain it with no co-pay (Sonfield et al., 2015; Bearak et al., 2016). Survey evidence suggests that the second barrier to LARC use is a lack of accurate information about safety, efficacy and ease of use (Kaye et. al., 2009, Hladky et al., 2011, Kavanaugh et al, 2013). In a 2009 survey of unmarried men and women in their twenties, 75 percent had heard of IUDs, but the majority indicated that they knew little or nothing about them and only half correctly reported that IUDs are more effective than the pill (Kaye et al., 2009). In a separate survey of women in the St. Louis area, 51 percent of respondents stated that they did not believe IUDs were safe, 46 percent did not know that women who had not previously had children were candidates for IUDs, and 38 percent incorrectly believed that tampons could not be used with IUDs (Hladky et al., 2011). There is also a lack of knowledge among providers. Kavanaugh et al. (2013) surveyed 584 publicly-funded contraception providers and found that in initial contraception consultations, IUDs were routinely discussed with teens or young adults at only about half of locations. In recent years, two large-scale initiatives, both funded by the Susan A. Buffet Foundation, have targeted financial and knowledge barriers to LARC use. The Contraceptive CHOICE Project is a prospective cohort study of 9,256 women aged 14 to 45 in the St. Louis Area. The project provided nocost reversible contraception to participants, and women were read a script intending to increase awareness of LARC. Of the 1404 teens enrolled in CHOICE, 72 percent chose LARCs. These women experienced much lower pregnancy and abortion rates than national rates (Secura et al., 2014). In Colorado, the Colorado Family Planning Initiative trained providers and provided free LARC at Title-X clinics across the state. Between the inception of the program in 2009 and an evaluation in 2011, caseloads had increased by 23 percent and LARC use among young women had increased from 5 to 19 percent (Ricketts et al., 2014). Lindo and Packham (2015) compare the change in teen births in Colorado 3

5 counties with Title-X clinics to the change in teen births in counties with Title-X clinics in the rest of the country, and estimate that the program decreased teen births by about 5 percent. These two studies are proving influential. In 2016 the governors of Virginia and Delaware proposed Colorado-style programs. However, expanding access to LARC continues to be politically controversial in many states. Even in Colorado, where the program was shown to be highly effective, the state legislature considered but did not pass an appropriations bill to fund the program in This study seeks to evaluate a new, complementary approach to lowering barriers to LARC. We test whether a simple and low-cost informational campaign can increase LARC awareness, knowledge and use. Methods The authors implemented a stratified cluster randomized control trial to identify the effect of a LARC information campaign carried out using Facebook advertisements. The advertisements were designed by the researchers and sponsored by Planned Parenthood of Northern New England (PPNNE), the largest reproductive healthcare provider in the three states it services: Maine, New Hampshire and Vermont, which also are the location of the study. The informational advertisements appeared on Facebook during the month of October Attitudinal and knowledge outcomes were observed in a follow-up survey of Facebook users conducted in February 2016, and LARC insertions were observed both pre and post-treatment using PPNNE health center data covering January 2014 through April Randomization The United States Postal Service (USPS) database of zip codes lists 1,081 zip codes for the threestate study area. Each of these zip codes is associated with one of 907 unique primary cities, and many zip codes are additionally associated with smaller secondary towns sharing the same zip code as 4

6 the primary city. The researchers divided the primary cities into five strata according to linear distance to the nearest PPNNE health center, and treatment was then randomly assigned to primary cities within each stratum. Under this randomization strategy, in cases where a single primary city contains multiple zip codes as is the case for larger cities like Burlington, Vermont and Portland, Maine all zip codes within the primary city receive the same treatment status. In cases where multiple small neighboring towns share the same zip code, all of these towns receive the same treatment status. Randomizing in this way increases the accuracy of the advertisement targeting because it does not require Facebook to distinguish between zip codes within a city nor between small neighboring towns that share the same zip code. Figure 1 maps the locations of PPNNE health centers and illustrates the locations of treatment and control clusters in the study region as identified by zip code boundaries. 1 Visually, treated clusters appear to be randomly distributed across the region. As summarized in Table 1, the treatment and control groups contain approximately equal numbers of primary cities and similar population sizes. Intervention Women living in the treatment areas were eligible to receive five informational advertisements appearing on Facebook during October These advertisements are reproduced in Figure 2. The two efficacy ads provide infographics comparing the typical use failure rate of IUD and the birth control implant to that of the birth control pill as estimated by Trussell (2011). The two ease-of-use ads report the number of birth control pills a woman would have to take in the time that the Mirena 1 Some zip codes, particularly those for unique point locations such as some large companies and government offices, do not occupy two-dimensional space. These were assigned to treatment and control status according to the associated primary city, but do not appear on the map. 5

7 IUD remains effective (5 years) and the Nexplanon contraceptive implant remains effective (3 years). 2 The safety ad features the headline Today s IUDs and implants are safe for young women and their future fertility. This message is intended to address the misconception among survey respondents that LARC methods are dangerous and not appropriate for young women (Kaye et al., 2009; Hladky et al, 2011). Facebook offers advertisers options when designing a new campaign. Advertisers must first select from a series of limited advertisement objectives. The most appropriate for our purposes was Send people to your website. 3 This type of advertisement included a Call to action button inviting viewers to Learn more. Women who clicked this link were directed to the Planned Parenthood Federation of America s webpage providing information on the relevant form of contraception, IUDs or Implants, and information about the full range of contraceptive options. Advertisers on Facebook must also select a delivery method. The goal of the intervention was to expose as many treated women as possible to the information intervention. We selected the advertisement delivery method to optimize daily unique reach, causing Facebook to deliver the ads to individuals no more than once per day. These ads are presented to the maximum number of unique individuals in a day and repeat exposures to the ads are distributed across multiple days. Finally, advertisements can be targeted to select audiences through a rich set of user demographics and locational information. The advertisements for this study were targeted to individuals that Facebook identified as women aged 18 to 34 residing in zip codes or in the case of larger cities, clusters of zip codes selected for treatment. Facebook identifies where people live based on the current city that 2 Since this study was conducted, duration of effectiveness for Mirena IUD and Nexplanon subdermal implant has increased to six and four years respectively. 3 At the time the advertisements were posted, the available objectives were as follows: Boost your posts, promote your page, send people to your website, increase conversions on your website, get installs of your app, increase engagement in your app, reach people near your business, raise attendance at your event, get people to claim your offer, get video views, promote a product catalog, and collect leads for your business. 6

8 users enter in their profiles additionally validated with IP address and friends profile locations. 4 We are not aware of any public empirical evidence on the accuracy of this targeting, so we conducted our own survey to assess it; this is described in the following section. Table 2 summarizes the results of the campaign as reported to us by Facebook. In total, the advertisements were shown 1.8 million times during the month of October The ads were shown to 126,231 unique women, for an average of 14.4 advertisements per woman. Based on the USPS estimate of the population of women ages 18 to 34 residing in the treatment areas, roughly 80 percent of the targeted population saw at least one advertisement. 5 Facebook does not allow advertisers to control the total unique impressions for each advertisement within the campaign; in this case they ranged from 88,742 impressions for the ease-of-use advertisement for IUDs to 97,693 impressions for the efficacy advertisement for IUDs. Eighty-two percent of the advertisements appeared in the desktop right column, 12 percent in the mobile news feed, and 6 percent appeared on third-party mobile apps and websites. The total cost of the campaign was $1,968, which was paid for by PPNNE. The goal of the advertisement intervention was to expose Facebook users to information about LARC. The advertisements were designed to provide a simple, clear piece of information. It was not designed to encourage clicks to the PPFA website, but 1,007 women did click-through. The ads also were liked 38 times over the course of the month, and two users commented on the ads. All comments were immediately hidden from other ad viewers so as not to contaminate the treatment. PPNNE Clinic Data 4 Information on Facebook s advertisement targeting was collected from Facebook online help pages. We contacted Facebook employees on multiple occasions to request more detailed information about the algorithm used to identify users towns of residence, but were provided with no additional information. 5 The targeted units of geography are made up of five-digit zip codes. Detailed population estimates for zip codes are not readily available. The USPS zip code database includes estimated of the population aged 10-19, 20-29, and We assumed a uniform distribution of population within each category, and estimated the population of women aged 18 to 34 as being equal to half the total population aged 18 to 34. 7

9 The primary outcome of interest was LARC insertions at PPNNE s 21 health centers. PPNNE provided the researchers with anonymized patient-level health center data from January 2014 through April These data contain 233,214 visit records. Of these, 12,411 were dropped from the analysis because patient zip code was not in the three-state study area or was missing. An additional 56,477 visit records were dropped from the analysis because the patients were not in the targeted group of women ages 18 to 34. The resulting sample consists of 164,326 observations of visits made between January 2014 and April 2016 by women ages 18 to 34 residing in northern New England. Table 3 summarizes this sample. Of the 164,326 patients observed in the data, 79,095 live in control zip codes and 85,231 in treated zip codes. The average patient is a white, 25 year-old woman. Thirty-nine percent are covered by private insurance, 26 percent by public insurance such as Medicaid, and 35 percent self-pay for the service they are receiving. Eighty-one percent of patient visits are recorded by PPNNE as family planning visits. An additional 6 percent are related to abortion services, 4 percent are for pregnancy testing and 8 percent are for sexually transmitted infection testing. The accuracy of Facebook s geographic targeting We are unaware of published attempts to assess the accuracy of Facebook s geographic targeting of advertisements. Because this is crucial to the research design, we implemented a postintervention survey intended to compare respondents self-reported zip codes to their treatment assignment. The survey also included three questions regarding knowledge of and attitudes toward LARC. Survey participants were recruited via Facebook advertisements sponsored by a generic Birth Control Survey page that we set up on Facebook. This page provided no information other than the identities of the two academic researchers who conducted the survey and who are not affiliated with 8

10 PPNNE. The sponsored advertisements invited viewers to click to the survey page. Appendix A provides complete survey materials. The Facebook advertisements recruiting respondents were shown 243,372 times to 87,338 women ages 18 to 34 living in the study area. The link to the survey was clicked 3,285 times, but only 171 respondents began the survey. Of these, 10 elected not to participate on the informed consent page. We additionally dropped 7 respondents outside of the targeted age range of 18 to 34 and 10 respondents who indicated that the link was shared with them by a friend, precluding assessing the accuracy of Facebook s geographic targeting. The final sample size for the survey is 144 respondents. Of these, 125 provided their zip code of residence. This represents a low response rate, but a sufficiently large sample to evaluate the accuracy of Facebook s geographic targeting. Table 4 compares Facebook s geographic assignment to respondents self-reported zip codes. Eighty-two percent of respondents who viewed advertisements intended for the control group self-report that they live in a zip code assigned to the control group. The corresponding accuracy rate for the treatment group is 71 percent. Overall, 78 percent of participants self-reported zip codes in the intended treatment group. Dropping those who report living outside the three-state study region, the Facebook s accuracy in targeting advertisements to the treatment and control groups is 87 percent. Results The primary outcome of interest is LARC insertions observed at PPNNE reproductive health centers. Figure 2 summarizes trends in LARC uptake, which is defined as LARC insertions per 100 patients. Following a national trend, LARC insertions have increased at PPNNE health centers over the previous two years, from 3.4 percent of patient visits in January 2014 to 7.5 percent of patient visits in April However, the trend appears similar for both treatment and control groups, and there is no clear visual 9

11 evidence that the advertisement campaign led to a differential increase in LARC uptake for the treated group relative to the control. This conclusion is further supported by the results of a standard differences-in-differences Poisson model to fit the number of monthly LARC insertions to women living in experimental group e (treatment or control) in month t: (1) EE LLLLLLLL ee,tt tttttttttttttttttt ee, mmmmmmmmh tt = eeeeee ββ 0 + ββ 1 tttttttttttttttttt ee + tt=25 tt=1 αα tt mmmmmmmmh tt + ββ 1 (tttttttttttttttttt ee II(mmmmmmmmh tt > OOOOOOOOOOOOOO 2016)) The explanatory variable of interest is (tttttttttttttttttt ee II(mmmmmmmmh tt > OOOOOOOOOOOOOO 2016), which indicates the treatment group observed post treatment. The indicator tttttttttttttttttt ee controls for time-invariant differences in LARC insertions between treatment and control, and a vector of monthly fixed effects control for spatially-invariant monthly shocks in LARC insertions. We estimate Model 1 for all LARC insertions and for IUD and Implant insertions separately. The results, which are presented in Columns 1-3 of Table 5, suggest that the information campaign increased LARC insertions by 5.5 percent (p=0.036), IUD insertions by 2.8 percent (p=0.235), and implants by 14.0 percent (p=0.009). While two of the three results are statistically significant, all are imprecisely estimated. Moreover, when we add an exposure variable controlling for variation in the number of overall patient visits (Columns 4-6), the results no longer suggest a positive treatment effect on IUD or overall LARC insertions, though the coefficient on implants remains statistically significant. One explanation for the reduction in the estimated coefficients is that the information campaign itself may have caused more women to make appointments at PPNNE health centers. We also estimate a Poisson model as specified in Equation 1, where the outcome is the number of patient visits. The results suggest that the information campaign increased patient visits by 5.0 percent (p<0.001). We therefore consider 10

12 the first three columns of results, which do not account for differences in the number of patient visits, as the more appropriate measure of the effect of the information campaign. Panel B presents estimated treatment effects for the health center in Burlington, Vermont, which accounted for 13 percent of patient visits in our dataset. Panel C presents estimated treatment effects for all of the remaining clinics combined. The results suggest that the weak evidence of a treatment effect observed in Panel A is driven entirely by an estimated 25.3 percent (p=0.001) increase in relative LARC insertions for the treatment group at the Burlington health center. At this health center, seventy percent of clients resided in treated zip codes, including zip codes in the city of Burlington itself, while the remaining 30 percent of patients are from control zip codes, mostly in outlying towns. It is possible that women living in the city of Burlington which includes the University of Vermont were particularly receptive to the information campaign. We also learned from PPNNE that there was a change in providers at the Burlington health center during the study period, though we have no reason to believe this would have disproportionately affected women living in treated zip codes relative to control zip codes. We consider the estimated treatment effect for the Burlington health center to be implausibly large, leaving us concerned that the estimated effect of the campaign is spurious. We also used the survey responses to gauge any treatment effect on knowledge and attitudes toward LARC. Figure 4 presents the fraction of respondents in the control and treatment groups who had heard of each of three methods: the contraceptive pill, IUD, and implant. Awareness of all three methods was very similar in magnitude, and none of the differences are statistically significant. Respondents who reported awareness of the pill and each of the other LARC methods also were asked to rank their relative efficacy. These results are presented in Figure 5. Again, the differences between treatment and control are small and lack statistical significance. We caution that the survey was primarily intended to gauge the accuracy of the geographic targeting of the advertisements; selection 11

13 into responding to the survey could bias estimates of the effects of the information campaign. The survey participation rate was low, and those who completed it may have been motivated by a desire to share strongly-held views on LARC. This is a group that is not likely to be receptive to an information campaign. Conclusion To our knowledge, this is the first study to test the effect of a simple, low cost and easily-implemented information campaign on the use of LARC. While the aggregate findings suggest that an information campaign carried out on Facebook increased LARC insertions at PPNNE health centers by 5.5 percent, we are reluctant to interpret this as convincing evidence of the efficacy of such an approach because we observed that the result is driven entirely by patients at a single health center. From a scientific perspective, this finding illustrates the ease with which researchers might but we did not-- attempt to obfuscate results of randomized control trials to suggest a spurious treatment effect. It also illustrates the difficulty of identifying the causal effects of low-cost, diffuse and politically expedient information interventions designed to influence behavior. This project also illustrates the leap of faith that many organizations take when expending resources on advertising campaigns designed to influence reproductive knowledge and behaviors. To our knowledge this is the first experimental test of the effects of an intervention designed solely to provide information on LARC. We are, of course, unable to say to what extent choices about advertisement design, placement, targeting and duration may have affected the outcomes, limitations that prevent generalizing results to other types of information campaign. A potentially fruitful line of future research would be to not only test the efficacy of more such campaigns, but to compare it to more time-costly approaches in which information is provided in person by health care providers. 12

14 13

15 Works cited American Academy of Pediatrics (AAP) Policy Statement: Contraception for Adolescents. Pediatrics 134(4): e1244-e1256. The American College of Obstetricians and Gynecologists (ACOG) Adolescents and Long-Acting Reversible Contraception: Implants and Intrauterine Devices. Committee Opinions no. 539, October Reaffirmed Bearak, Jonathan, Lawrence Finer, Jenna Jerman, and Megan Kavanaugh Changes in out-of-pocket costs for hormonal IUDs after implementation of the Affordable Care Act: An analysis of insurance benefit inquiries. Contraception 93(2): Finer, Lawrence and Stanley Henshaw Disparities in Rates of Unintended Pregnancy in the United States, 1994 and Perspectives on Sexual and Reproductive Health 38(2): Finer, Lawrence and Mia Zolna Declines in Unintended Pregnancy in the United States, The New England Journal of Medicine 374: Finer, Lawrence, Jenna Jerman, and Megan Kavanaugh Changes in use of long-acting contraceptive methods in the U.S., Fertility and Sterility 98(4): Hladky, Katherine, Jenifer Allsworth, Tessa Madden, Gina Secura, and Jeffrey Peipert Women s knowledge about intrauterine contraception. Obstetrics and Gynecology 117(1): Jones, Rachel, Jacqueline Darroch, and Stanley Henshaw Contraceptive Use Among U.S. Women Having Abortions in Perspectives on Sexual and Reproductive Health 34(6): Kavanaugh, Megan, Jenna Jerman, and Lawrence Finer Changes in Use of Long-Acting Reversible Contraceptive Methods Among U.S. Women, Obstetrics & Gynecology 126(5): Kavanaugh, Megan, Jenna Jerman, Kathleen Ethier, and Susan Moskosky Meeting the Contraceptive Needs of Teens and Young Adults: Youth-Friendly and Long-Acting Reversible Contraceptive Services in U.S. Family Planning Facilities. Journal of Adolescent Health 52(3):

16 Kaye, Kellen, Katherine Suellentrop, and Corinna Sloup The Fog Zone: How misperceptions, magical thinking and ambivalence put young adults at risk for unplanned pregnancy. Washington, DC: The National Campaign to Prevent Teen and Unplanned Pregnancy. Lindo, Jason and Analisa Packham How Much Can Expanding Access to Long-Acting Reversible Contraceptives Reduce Teen Birth Rates? NBER working paper No Ricketts, Sue, Greta Klingler, and Renee Schwalberg Game change in Colorado: Widespread use of long-acting reversible contraceptives and rapid declines in births among young, low-income women. Perspectives on Sexual and Reproductive Health 46(3): Secura, Gina, Tessa Madden, Colleen McNicholas, Jennifer Mullersman, Christina Buckel, Qiuhong Zhao, and Jeffrey Peipert Provision of no-cost, long-acting contraception and teenage pregnancy. New England Journal of Medicine 371(14): Sonfield, Adam, Athena Tapales, Rachel Jones, and Lawrence Finer Impact of the federal contraceptive coverage guarantee on out-of-pocket payments for contraceptives: 2014 update. Contraception 91(1): James Trussell, Anjana Lalla, Quan Doan, Eileen Reyes, Lionel Pinto, and Joseph Gricar Cost effectiveness of contraceptives in the United States. Contraception 80(2): James Trussell Contraceptive failure in the United States. Contraception 83(5):

17 Figures Figure 1: Treatment and control areas 16

18 Figure 2: Facebook Advertisements Panel A: Advertisements highlighting the efficacy of LARC IUD Efficacy Implant Efficacy 17

19 Panel B: Advertisements highlighting the ease-of-use of LARC IUD Ease of Use Implant ease of use 18

20 Panel C: Advertisement highlighting the safety of LARC IUD and Implant Safety 19

21 Figure 3: Monthly LARC insertion rate, January 2014-February 2016 LARC insertions per 100 patient visits Ad campaign Control Treatment Note: The insertion rate is measured as total LARC insertions per 100 patient visits.

22 Figure 4: Fraction of respondents who have heard of each method of contraception Pill IUD Implant Pill IUD Implant control treatment Source: Analysis of results of online survey conducted in March 2016 by the authors. n=142 21

23 Figure 5: Fraction of respondents who know method is more effective than the pill IUD Implant IUD Implant control treatment Source: Analysis of results of online survey conducted in March 2016 by the authors. Respondents were asked to rate the relative efficacy of two methods if they reported in the previous question that they had heard of both methods. 129 respondents were asked to rate the relative efficacy of the pill and IUD, and 116 were asked to rate the relative efficacy of the implant and pill. 22

24 Tables Table 1: Number of towns and subject population by stratum and treatment status Number of towns Population of Women aged Distance to clinic Stratum (km) Control Treatment Total Control Treatment Total ,103 32,308 62,411 2 (0-20] ,082 56, ,988 3 (20-50] ,219 39,981 93,200 4 (50-100] ,671 17,146 34, ,259 13,862 36,121 Total , , ,537 *The population of women aged is based on zip code population estimates provided by the USPS. 23

25 Table 2: Advertisement exposure Total Impressions Unique Impressions Average impressions per woman Percent of population exposed to ad campaign Efficacy IUD 379,854 97, Implant 367,416 93, Ease of use IUD 353,286 88, Implant 357,600 91, Safety IUD & Implant 359,706 92, All 5 advertisements 1,817, , *Summary statistics for Facebook advertisement campaign targeted to women aged 18 to 34 living in the treated zip code clusters. "Impressions" is the total number of times each ad was served as reported by Facebook. "Unique impressions" is the total number of women who were served the ad as reported by Facebook. "Average impressions per woman" is the ratio of Total Impressions to Unique Impressions. "Percent of population exposured to ad campaign" is the ratio of unique impressions to the estimated number of women aged in treated zip codes. 24

26 Table 3. Summary statistics for PPNNE clinic visits Control Treatment Total Demographics female nonwhite age State of residence Maine New Hampshire Vermont Payment type private insurance public insurance self-pay Visit Purpose family planning abortion pregnancy test STI other <0.01 <0.01 <0.01 Sample Size 79,095 85, ,326 *Summary statistics for PPNNE clinic data. The unit of observation is a patient visit. The sample has been restricted to women aged 18 to 34 living in the three-state study area with nonmissing zip codes.

27 Table 4: Accuracy of geographic targeting of survey ads Facebook ad set Treatment Control Self-reported zip code Treatment 37 3 Control Outside study area 4 10 Accuracy

28 Table 5: Poisson model estimates of treatment effects dep. Variable: Number of monthly insertions Number of patient visits as No Exposure Variable exposure (1) (2) (3) (4) (5) (6) LARC IUD Implant LARC IUD Implant Panel A: All Clinics Estimated treatment effect ** *** * (0.026) (0.024) (0.054) (0.028) (0.028) (0.051) Pearson goodness-of-fit test (p-value) Panel B: Burlington, VT only Estimated treatment effect 0.253*** 0.310** (0.078) (0.128) (0.120) (0.053) (0.087) (0.143) Pearson goodness-of-fit test (p-value) Panel C: Not Burlington, VT Estimated treatment effect (0.036) (0.039) (0.074) (0.037) (0.042) (0.071) Pearson goodness-of-fit test (p-value) No. months No. experimental groups Note: Estimated coefficients from a Poisson Model. The dependent variable is the number of monthly LARC insertions for women living in the control and treated zip codes. A treatment indicator and monthly fixed effects are included. The exposure variable is the total number of monthly visits. Standard errors are robust. (***p<0.01 ** p<0.05 *p<0.10)