Appendix. Klar and Krupnikov, Appendix

Transcription

1 Appendix Klar and Krupnikov, Appendix

2 Table A1.1: All Experimental Studies Study Number Description of Sample Chapter 3: 1 Study 3.1* National American, N=826 2 Study 3.2 Internet N = Study 3.3*^ National American, N=404 Chapter 4: 4 Study 4.1 Internet N=156 5 Study 4.2 Internet-Skilled N=251 6 Study 4.3^ National American, N=550 7 Study 4.4 Internet, N=150 Chapter 5: 8 Study 5.1* Social Media Savvy, N = Study 5.2 Arizona Residents, N = Study 5.3 National American, N = 400 Chapter 7 11 Study 7.1 National American, A1: Experimental Studies and Survey Data Description of Result Demonstrates that participants believe selecting independent makes a better impression Demonstrates that the belief that being independent makes a better impression persists after exposure to partisan debate Demonstrates that a combination of partisan disagreement and a need to make a positive impression lead people to be more likely to avoid partisanship. Demonstrates that partisan disagreement leads people to avoid even those partisans who agree with them politically Demonstrates that partisan disagreement leads to more negative visualizations of partisanship. Demonstrates that partisan disagreement leads people to avoid neighborhoods with politically-active residents. Demonstrates that people view partisans as less attractive, competent, trustworthy and likable than independents. Demonstrates that political disagreement makes people less likely to reveal political preferences on a social media website. Demonstrates that political disagreement makes people less likely to take and display a partisan sticker. Demonstrates that political disagreement leaves people less likely to participate in political discussions or join political websites. Demonstrates that people most likely to be undercover partisans are dissatisfied when their preferred party makes concessions to reach compromise. N=2,400 * Replication of study with different sample included in appendix to chapter; Additional outcome measures included in appendix to chapter; ^ Pre-tests of stimuli with different samples included in appendix to chapter.

3 Table A1.2: All Surveys Survey Name Description of Survey Description of Data Use Chapter 1: Pew Politics Survey (2013) Survey of Political Independents (2007) SSI Open-ended survey (2014) Chapter 3: Pew, People-Press Surveys Chapter 6: American National Election Study, 2004 and 2006 Chapter 7: Survey Monkey Audience Survey, 2014 GfK General Population Survey, 2013 Landline and cell-phone survey, 1,480 American adults; conducted July 17 to 21, Telephone interviews with 2,140 randomly selected American adults. Oversample of 1,014 respondents who identify as independent. Survey conducted May 3 to June 3, Survey is a collaboration between the Washington Post, Kaiser Family Foundation and Harvard University. Internet survey conducted for specifically for this project in July of Sample collected by Survey Sampling International. 14 surveys from a series of Pew studies; all studies listed in Appendix A3. Telephone sample of American adults, 2004 and A subset of 2004 participants was reinterviewed in Internet survey of 2,915 Americans who are party of the Survey Monkey Audience panel. Survey 2,193 Americans, fielded online between March 6 and March 13, 2013 Descriptive patterns, independents and partisans. Descriptive patterns of independents demographics and reasons individuals offer for identifying as independent. Descriptive openended responses about reasons for selecting a partisan identification. Comparisons of proportions of independents in national survey samples pre and post- Todd Akin s comments in Estimates the effect of self-monitoring on indicators of party avoidance. Patterns in open-ended responses of messages to President Obama Patterns of identification across social and economic ideological positions.

4 A1.1 Descriptions and Demographics of Experimental Samples Our experimental studies deliberately rely on a variety of samples. The goal is to show that the results we find are not specific to one particular set of conditions, but rather can replicate across different samples and different experimental contexts. Since we use similar types of samples across chapters, we describe each of our sample types below. A1.1 National Adult Samples We rely on national adult samples recruited by different types of survey companies: Research Now (Study 3.1), Survey Sampling International (SSI)(Study 4.3, Study 7.1) and YouGov (Study 3.3, Study 5.3). The goal of these studies is to recruit a sample that is nationally-representative meaning that it matches the characteristics of the national population. The Research Now recruitment process begins with a large panel of people who have already agreed to take part in surveys. The members of this panel are then sampled for participation in particular studies. Study samples are based on the specification requested by a particular researcher. Since our goal is to produce a sample that best matches the characteristics of the general population of American adults, Research Now relied on intra-panel recruitment techniques targeting those panel members who are over 18. One of the benefits of Research Now is that the company deliberately attempts to reach population groups that are difficult to recruit for study participation. Although Research Now is a non-probability sample, it is a sample that is designed to match national demographic characteristics.

5 Two of our samples come from Survey Sampling International (SSI). SSI is similar to Research Now, although SSI has conducted more academic studies, while Research Now has largely worked on consumer-preference surveys. Like Research Now, SSI recruits individuals to join their panel via opt-in recruitment methods online. Subsequently, within-panel samples are produced to approximate a target sample of registered voters based on age, race/ethnicity, gender, educational attainment, geographic region, annual household income, homeownership status, and marital status. Although SSI samples are not as representative as national probability samples, they are more diverse than many types of convenience samples. Data collected through SSI have been used in a variety of political science studies, ranging from research by the Brookings Institute on the success or failure of Obamacare (Dropp et al. 2013) to research on experimental and survey methods (Berinsky et al. 2014). The YouGov approach is also based on a panel of participants who are then randomly invited to take part in various studies both political and non-political. This panel includes over two million Americans, and YouGov relies a process called sample matching to obtain nationally-representative samples from this panel (see Rivers (2006) for a description). Research suggests that the YouGov technique produces samples of similar or higher quality to traditional telephone surveys (Berrens, Bohara et al. 2003, Sanders, Clarke et al. 2007). As a result, numerous scholars (Barabas and Jerit 2010, Brooks 2011, Huber, Hill et al. 2012,) and surveys such as the Cooperative Congressional Election Study and the Cooperative Campaign Analysis Project (Jackman and Vavreck 2009) have relied on YouGov.

6 We present the demographic characteristics of the Research Now, SSI and You Gov samples relative to the 2012 American National Election Study (ANES), the US Census and Pew in Table A1.3. The 2012 ANES was conducted both, via the Internet and face-to-face (FTF); we present comparisons between our samples and both the web and FTF versions of the 2012 ANES. We use these samples for comparison as they are often considered the gold standard of sampling. Our samples are quite comparable to the US Census, ANES and Pew. The main difference is that there are more women in the Research Now sample relative to the other samples. Importantly, however, we use the Research Now sample in Study 7.1, which focuses on compromise. Generally, women are more in favor of compromise than men, so the very slight oversample of women is likely to attenuate our results making the study a more conservative test. Table A1.3: Sample Comparisons, National Adult SSI Research Now YouGov ANES 2012 ANES 2012 FTF Pew US Census 2 Web % Female 51.42% 60.8% 51.56% 52.1% 52.01% 51.08% 50.8% %18 to % 11.20% 6% 9.44% 12.39% 13.33% 7.2% % over % 25.6% 17.62% 19.93% 18.20% 17.26% 14.6% $75, % 29.20% % 34.5% BA % 25.58% 25.45% 30.37% 29.07% 27.10% 29.7% Democrats 39.55% 32.48% 33.91% 37.41% 35.74% 33.35% --- Republicans 23.14% 24.61% 23.88% 30.42% 25.77% 22.90% --- A1.2 Internet Samples 1 January 2014 Political Survey; results are weighted. 2 Based on the 2013 American Community Survey Estimates 3 Those 25 and over given Census data.

7 Our Internet samples were recruited via the crowdsourcing website Amazon Mechanical Turk. Mechanical Turk is an online web-based platform for recruiting and paying people to perform tasks. These samples have been used widely across the social sciences (Berinsky et al. 2012; Kriner and Shen 2012; Doherty 2013; Huber and Paris 2013; Mattes and Redlawsk 2014). Although Mechanical Turk samples are not nationally representative, they provide a diverse group of participants and a new context in which we are able to test hypotheses. Moreover, recent research has shown that Mechanical Turk participants behave much like participants in national adult samples (Leeper and Mullinix 2014; Mullinix, Druckman and Freese 2014). Furthermore, to the extent that Mechanical Turk participants behave unlike other types of participants, they are most likely to do so in studies that require them to place an extensive amount of trust in the researcher (Krupnikov and Levine 2014), something which we are not asking our participants to do. Since 2010, numerous articles have analyzed the effects of political information on individual preferences using Mechanical Turk. This subject population has been used to analyze topics such as cognitive biases (Arceneaux 2012), the role of negative advertising (Dowling and Wichowsky 2014; Krupnikov and Bauer 2014), ambiguity in campaign appeals (Hirsch and Schaffner 2013), and the effects of party reputation and brand (Butler and Powell 2014), as well as many other topics. Our reliance on this sample, however, is also deliberate given the tasks we are asking our participants to complete in certain studies. In particular, in Study 4.1

8 we ask participants to search for Internet images and provide us with the URL. This means that we must ensure that all of our participants are equally adept at using the Internet in this way, as systematic differences in Internet skill would confound our results. As a result, this particular subject population offers us the best means of ensuring that all participants are at a certain Internet use baseline. In particular, in Study 4.1 we even conduct an additional check to ensure that all our participants are Internet-savvy enough to complete the task. Although we do not require such a check in Study 4.3, this study nonetheless does ask people to categorize faces by moving images in a website. Again, this requires some familiarity with the Internet, therefore again, this particular sample is particularly beneficial. We present the demographics of this sample in Table A1.4. Compared to our national adult samples, the Internet Mechanical Turk samples are somewhat demographically different. This is, however, to be expected. Given that participating in MTurk requires some Internet skill, it follows that these samples skew younger. For example, according to recent US Census records, 78.7% of people aged 18 to 34 years access the Internet from their homes. In contrast, of people who are over 65 years of age, only 48.7% access the Internet from their own home. Overall, 51% of people aged 65 or older reported accessing the Internet from either a home or non-home location. In contrast, 86% of people aged 18 to 34 reported that they had accessed the Internet from some location. 4 As a result, when our goal is to obtain a sample that is familiar with the Internet, it is reasonable to 4 Data from the US Census Bureau, Current Population Survey, Internet release on January 2014.

9 expect that it would be weighted more heavily toward those in the younger category and have comparatively fewer people in the 65 and over category. Age aside, our Internet convenience samples recruited through Mechanical Turk are similar to our comparison samples. The convenience sample recruited for Study 3.2 is similar in gender and educational attainment to the Census sample. While the participants recruited for the studies 4.1 and the Internet savvy samples -- do skew more Democratic, this skew would only have made it more difficult for us to show the results we did. In fact, we see similar patterns across all parties in that sample. Table A1.4 Samples Comparison, Internet Samples Study 3.2 Internet Savvy (Study 4.1 and 4.3) ANES 2012 Web ANES 2012 FTF Pew 2014 US Census % Female 51.38% 42.94% 52.1% 52.01% 51.08% 50.8% %18 to % 25.47% 9.44% 12.39% 13.33% 7.2% % over % 3.73% 19.93% 18.20% 17.26% 14.6% $75, % 24.83% % 34.5% BA % 27.87% 30.37% 29.07% 27.10% 29.7% Democrats 31.74% 48.22% 37.41% 35.74% 33.35% --- Republicans 23.08% 16.75% 30.42% 25.77% 22.90% --- A1.3 Face-to-Face Samples Our final category is face-to-face samples. These, by definition, are samples from studies that relied on people directly interacting with researchers. For the sake of categorization, we also include samples in which the initial interaction between the researcher and the participant occurred face-to-face even if the bulk of the participation occurred over the internet. The samples in this category are by

10 necessity geographically limited. As a result, they are less diverse than the samples in our national Internet studies. Especially different is the sample that we use in our social media study. As we discuss in Chapter 5, the participants in this study were selected due to their familiarity with social media and the fact that they already formed a type of network. Furthermore, these participants were all members of an undergraduate class. As a result, this is the least diverse sample of all the samples we use. The characteristics of this sample allow us to retain the internal validity of the study (participants who are familiar with social media websites and networks). Moreover, this study also retains a baseline of external validity as many users of social media are quite similar to our participants. The sample demographics are shown in Table A1.5. We do not have income characteristics for these samples. The members of our social media sample are students, meaning their income levels would not be reflective of their actual socio-economic positions. These participants completed the entire study online, at their own time rather than in a lab. We include them in this category because the initial instruction to these participants occurred in a face-toface manner. Our adult Tucson sample, however, was approached on the street and income was not asked to retain privacy and trust between the interviewer and the participant. These types of face-to-face samples are common in political science experiments. For example, Brader (2005) relies on face-to-face studies with Massachusetts residents for his advertising experiments. Taber and Lodge (2006),

11 Lodge and Taber (2013) rely on face-to-face studies of undergraduate students to analyze the role of cognitive biases. Further, although undergraduate student samples have garnered criticism in the past (Sears 1986), recent research suggests that they can be useful (Druckman and Kam 2011) and often perform as well as national samples (Krupnikov and Levine 2014; Leeper and Mullinix 2014). Table A1.5 Samples Comparison, Face-to-Face Samples Social Media Tucson Sample ANES 2012 ANES 2012 Pew 2014 US Census Web FTF % Female 52.25% 53.98% 52.1% 52.01% 51.08% 50.8% %18 to % 19.53% 9.44% 12.39% 13.33% 7.2% % over % 19.93% 18.20% 17.26% 14.6% $75, % 34.5% BA % 30.37% 29.07% 27.10% 29.7% Democrats 29.63% 23.95% 37.41% 35.74% 33.35% --- Republicans 29.63% 21.02% 30.42% 25.77% 22.90% ---

12 A2. Chapter 2 Appendix A2.1 Debate Data In order to track the word patterns in Presidential debates from 1996 to 2000, a coder (who was not informed of the purposes of our research) first obtained full transcripts of the debates. The coder then conducted a word-count to obtain the total words used within the debate by the candidates. Subsequently, the coder read the transcripts for instances of insurmountable disagreement: language considered negative, confrontational or bickering as well as language that assigned fault or blame. We rely on coder analyzed transcripts rather than machine-based analyses because we analyze not only the words themselves, but also the context in which the words were used. The words that speak to insurmountable disagreement were totaled postcoding. The percentages shown in Figure 2.1 (in-text) reflect the number of words that make up instances of disagreement relative to all words used in the debate by candidates. A2.2 Instances of Polarization in the New York Times Using Levendusky s (2009) findings as a foundation, we track more recent media coverage of polarization. We also focus on the New York Times because, as many scholars have argued, its contents reflect the coverage of many other media sources (Baumgartner, De Boef and Boydstun 2008; Boydstun 2013; Gans 2004; Levendusky 2009; Soroka 2002). To obtain a total of the articles that discuss polarization, we use the New York Times Chronicle: a web-based tool that tracks the

13 number of times a certain word or phrase appears in the New York Times in a given time period. The Chronicle pulls the articles by year, and provides the user with a listing of all the articles, which can then be examined for content relevance. 5 We first replicate Levendusky s counts from 1980 to 2004 and then we extend the analysis to Data in Figure 2.2 comes from this polarization wordcount. We use the identical approach to calculate word totals that relate to partisan divide. The partisan divide is a count of partisan disagreement and partisan fighting. 5 The benefit of relying on Chronicle rather than on other databases such as Lexis/Nexis is that this tool also calculates not only the number of articles published that include the term polarizations, but also the percentage of these articles relative to all the articles published in the New York Times that year. This is an important check, as the sheer number of stories the Times is producing increases as the newspaper begins to introduce internet-only content.

14 A3. Appendix to Chapter 3 A3.1 Additional Information for Study 3.1 A3.1.1 The Sample Participants in this study were recruited through SSI. A discussion of SSI recruiting is in Appendix A1, and the demographics of the SSI samples are shown in Table A1.3. A3.1.2 The Treatments Participants who were instructed to make the worst impression were told: For the next question, please select the answer you believe will make the WORST impression on others even if this answer does not actually describe your true feelings. Participants who were instructed to make the best impression were told For the next question, please select the answer you believe will make the BEST impression on others even if this answer does not actually describe your true feelings. A3.1.3 Randomization Checks To ensure that we have appropriate randomization we estimated a model that predicted group assignment by a variety of demographic characteristics that were measured pre-treatment. These characteristics include gender, income, education, news interest and pre-treatment partisanship. We treat our group assignment as a binary dependent variable. We find that none of the variables

15 measured pre-treatment predict group assignment. Furthermore, the model produces a χ 2 = 2.67, with a p-value of 0.751, suggesting that the included factors cannot explain group assignment. A3.2. Additional Information for Study 3.2 A3.2.1 The Sample Participants in this study were recruited through the Internet using Mechanical Turk. This sample provided a diverse set of adults. Moreover, many participants recruited in this way tend to identify as independent, which leaves this sample as a conservative test of our predictions. We discuss additional sample recruitment details in our Appendix A1, and present the demographic characteristics of the sample in Table A1.4. A3.2.2 The Treatments This study asked people to either make the best or worst impression using the same wording as the treatments in Study 3.1. Prior to this treatment, all participants received the following text from a presidential debate: Transcript from the 2nd Presidential Debate between President Barack Obama and Governor Mitt Romney October 16, 2012 ROMNEY: In the last four years, you cut permits and licenses on federal land and federal waters in half. OBAMA: Not true, Governor Romney. ROMNEY: So how much did you cut (inaudible)? OBAMA: Not true. ROMNEY: How much did you cut them by, then? OBAMA: Governor, we have actually produced more oil -- ROMNEY: No, no. How much did you cut licenses and permits on federal land and federal waters? OBAMA: Governor Romney, here's what we did. There were a whole bunch of oil companies.

16 (CROSSTALK) ROMNEY: No, no, I had a question and the question was how much did you cut them by? OBAMA: You want me to answer a question -- ROMNEY: How much did you cut them by? OBAMA: I'm happy to answer the question. ROMNEY: All right. And it is -- OBAMA: Here's what happened. You had a whole bunch of oil companies who had leases on public lands that they weren't using. So what we said was you can't just sit on this for 10, 20, 30 years, decide when you want to drill, when you want to produce, when it's most profitable for you. These are public lands. So if you want to drill on public lands, you use it or you lose it. ROMNEY: OK, (inaudible) OBAMA: And so what we did was take away those leases. And we are now reletting them so that we can actually make a profit. ROMNEY: And production on private -- on government land -- OBAMA: Production is up. ROMNEY: -- is down. OBAMA: No, it isn't. ROMNEY: Production on government land of oil is down 14 percent. OBAMA: Governor -- ROMNEY: And production on gas -- (CROSSTALK) OBAMA: It's just not true. ROMNEY: It's absolutely true.. You'll get your chance in a moment. I'm still speaking. A Randomization Checks Much as we did for Study 3.1 we conducted a randomization check to ensure that variables measured pre-treatment did not affect randomization. Again, we use assignment to either the best or worst impression condition as the outcome variable in a model. Relying on the same set of pre-treatment variables as in our first study we show no evidence that pre-treatment variables affected group assignment. A3.3 Additional Information for Study 3.3 A3.3.1 The Sample

17 For this study, we employ a sample of respondents recruited via YouGov, which provides us with a geographically-representative sample of adults (via weighting). We include more information about the YouGov panel and recruitment approach in our Appendix A1 and present the demographics in Table A1.3. A3.3.2 The Treatments and Pre-Tests of Treatments A Wording of Simulated News Clips Our treatments were designed to appear as short news-briefs, and are similar in word-count and structure. Control Version Every February, Americans wait for Groundhog Phil in the little town of Punxsutawney, Pa. According to folklore, Phil s sighting of his own shadow means there will be 6 more weeks of winter. If Phil does not see his shadow, it means there will be an early spring. The official website of Punxsutawney Phil, perhaps not impartial, claims the Groundhog has issued a correct forecast 100% of the time. Unity Version As President Barack Obama begins his second term, the Democrats and Republicans in Washington appear to be more unified than ever. Political experts predict that Americans can expect a new era of bipartisanship in Washington. The profound debate that has raged between the two parties appears to be settling. The next two years may very well bring progress towards two parties cooperating in Washington. Disagreement Version As President Barack Obama begins his second term, the Democrats and Republicans in Washington appear to be as divided as ever. Political experts predict that Americans can expect even more of the partisan bickering that has characterized Washington in recent years. The profound debate that has raged between the two parties has not been settled in the least. The next two years may very well bring a continuous cycle of two parties battling it out in Washington. A Pre-Tests of Treatments

18 We conducted several pre-tests to ensure that there are no confounding differences between treatments used in this study that can jeopardize control and affect our results. These pre-tests were conducted at a different time than our main study and relied on a different subject population. The use of a different sample for the pre-tests was deliberate, with the goal being to ensure that there were no possible spillover effects from the pre-tests to our main study. Our first pre-test tracked the readability of our treatments and relied on a sample of 104 adults recruited via the Internet using Mechanical Turk. In this pretest, participants were randomly assigned to either the partisan unity or the partisan disagreement treatments. Using a scale where a score of 1 means that something is difficult to understand and 7 means it is easy to understand, participants assigned the partisan unity treatment an average score 6, and the disagreement treatment an average score of 6.1. The difference between these treatments is not significant, p=0.59. The second pre-test relied on N=139 of participants recruited using Mechanical Turk. In this second pre-test participants were also randomly assigned to one of three conditions: (1) no treatment, (2) the partisan unity treatment or (3) the partisan disagreement treatment. Since these treatments are designed to bring different partisan traits to mind, this pre-test ensured that each treatment does lead people to associate different traits with partisanship. The key post-treatment measure offered subjects a list of terms and asked them to select which terms they associated with the word partisanship. In total, subjects saw 16 words, 8 words

19 were positive, 8 words were negative. The words are shown below in Table A3.1; they were presented to all subjects in random order. Table A3.1: Terms a Participant May Associate With Partisanship Positive Terms Negative Terms Cooperative Inflexible Friendly Closed-Minded Positive Negative Well-liked Disliked Kind Angry Intelligent Stupid Flexible Stubborn Free-Thinker Follower We can then track, on average, how many negative words subjects select by treatment group. The results show (Table A3.2) that subjects who were randomly assigned to the partisan disagreement treatment are significantly more likely to select negative terms than subjects randomly assigned to the other two groups. We can also compare the number of negative words selected relative to the number of positive words. In this analysis we subtract the number of positive words selected from the number of negative words selected; the larger the value on this relative comparison, the more negative words a subject selected relative to positive words. Again, there are significant differences between groups (Table A3.2) as subjects in the partisan disagreement group are more likely to select more negative terms to describe partisanship than subjects in the other two groups. Table A3.2: Group differences in terms selected Control Unity Disagreement ANOVA Mean Negative F=7.37 (p=0.0009) Terms (raw) Mean Relative F=8.61 (p=0.0003)

20 (neg-pos) A3.3.3 The Measures: Self-Monitoring The measures of self-monitoring we use originate in previous research that relies on this characteristic (Terkildsen 1993, Huddy and Feldman 2005; Berinsky and Lavine 2012). These come from a shortened scale initial developed in psychology (Snyder 1979). Self-monitoring is based on the way people adapt to their outside environment, and is a concept distinct from other individual characteristics such as need for cognition or evaluation (Cacioppo et al. 1996). In most of our experimental studies we use both the 5-point versions of the self-monitoring questions (Berinsky and Lavine 2012) and the true-false versions of the questions (Terkildsen 1993; Huddy and Feldman 2005). We use the shortened version of the scale validated by Berinsky and Lavine (2012). Although Berinsky and Lavine (2012) argue that the 5-point version is a stronger measure, we use both of the versions to ensure robustness, as previous experimental results rely on the true/false version of the scale. We also discuss the extent to which our results are robust to both versions of the scale when appropriate. There is only one case where we obtain very slightly different results depending on the scale, and this case is when we rely on observational data rather than experimental data. The five-point version of the scale is as follows: 1. When you are with other people, how often do you put on a show to impress or entertain them? Always Most of the time Some of the time Once in a while

21 Never 2. When you are in a group of people, how often are you the center of attention? Always Most of the time Some of the time Once in a while Never 3. How good or poor of an actor would you be? Excellent Good Fair Poor Very poor The true- false version is as follows: 1. I would probably make a good actor. [True/False] 2. In groups of people, I am rarely the center of attention [True/False] 3. I guess I put on a show to impress or entertain people. [True/False] We treat self-monitoring as an individual trait and do not expect that it would be affected by our treatments. Nonetheless, in Study 3.3 and other studies in which we include measures of self-monitoring -- we measure self-monitoring prior to treatment exposure. Comparing the groups in Study 3.3 on self-monitoring shows that all three of our experimental groups have virtually identical means on the self-monitoring scale. Even more importantly, conducting Bartlett s test on the data produces a χ 2 of (p=0.204), which means that we cannot reject the null hypothesis that all our three groups have equal variances on self-monitoring. Selfmonitoring also does not predict group assignment in an ordered probit with group assignment as a dependent variable. A Randomization and Data Checks

22 An additional benefit to relying on the YouGov panel is that our study subjects had participated in other YouGov surveys prior to taking part in our experiment. As a result, we have measures of participants party identification obtained nearly one year prior to our survey experiment. 6 This allows us to ensure that the participants in the different experimental groups had, on average, the same levels of partisanship before the study began. We see that, one year prior to participating in our study, subjects across all three groups were equally likely to identify as partisans. Among the control group, 63% identified as partisans and, in both treatment groups, 62% identified as partisans. Furthermore, among the three groups we see nearly equivalent numbers of participants who identified as independents; ANOVA reinforces that there are no differences by group in these previous measures of partisanship (F=0.65, p=0.525). We also see no differences in the strength of ideology, again as measured a year ago, by our current treatment groups (F=0.67, p=0.511). We also conduct additional randomization checks using pre-treatment measures to predict group assignment. We find that a model predicting group assignment using age, gender, race, education, voter registration, church attendance and partisanship measured a year prior to our study produces a χ! of 4.35, with a p- value of A Additional Results 6 Our study was conducted in February 2013 and measures of partisanship are from January 2012.

23 The results we present in the chapter text are robust to a logistic regression used to predict the likelihood of identifying as a strong partisan. Although we conduct a randomization check, for the sake of additional robustness we estimate a model that includes additional factors that are known to influence partisanship, including gender, education, race, whether the subject lives in the south, 7 ideology (measured a year prior to our study and thus unaffected by our treatments) and partisan group (measured a year prior to our study). Just as we do in the analyses presented in Chapter 3, we consider the participants we term past partisans people who a year prior to our study identified as members of a party. In addition, while group comparisons in Figure3.3a and b used a categorical version of the selfmonitoring scale split at the median, in our model we use the full range of the scale. In the model higher values of the self-monitoring scale mean that the participant is higher in self-monitoring. Therefore, if results follow from our predictions, we should expect a negative coefficient on the interaction between selfmonitoring and exposure to disagreement. The results of this test (Table A3.3 ) reinforce our group comparisons, showing that the interaction between selfmonitoring and the exposure to disagreement decreases the likelihood of expressing of identifying as a strong partisan. 8 7 South in this case is identified following the American National Election Study s definition of the political south: Alabama, Arkansas, Florida, Georgia, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, Texas and Virginia 8 It is important to note, however, that due to the range of the self-monitoring variable, we cannot interpret the coefficients on the individual treatment variables directly (Brambor, Clark and Golder 2006) as they represent the effect of our treatment when the self-monitoring variable is at zero -- a case where a subject has absolutely no self-monitoring behavior and a rare occurrence not only in our sample, but in general studies of self-monitoring.

24 We also estimate the marginal effect of increases in self-monitoring by condition. We find that for participants in the disagreement condition, a one-unit increase in self-monitoring decreases the likelihood of identifying as a strong partisan by a decrease that is significant at p=0.03. In contrast, for participants in the unity condition, a one-unit increase in self-monitoring has a effect on the likelihood of identifying as a strong partisan, a change that does not reach conventional levels of statistical significance p= Although there is a slight overlap in confidence intervals, this is to be expected given the power of selfmonitoring. The key is the significant effects in the disagreement condition and the null effects in the unity condition. We reach similar findings when we estimate an identical model but one that uses independent identification as a dependent variable. Here, the interaction between self-monitoring and the disagreement condition is positive, however indicating that higher self-monitoring and the disagreement condition lead to a greater likelihood of identifying as a pure independent. The coefficient on the interaction between self-monitoring and the disagreement condition is p=0.09. Even more importantly, the marginal effect of self-monitoring in the disagreement condition on the likelihood of identifying as a pure independent is positive (+0.037) and significant at p=0.03.

25 Table A3.3: Likelihood of Identifying as a Strong Partisan Coefficient p-value (Standard Error) Disagreement Treat x Self p<0.05 Monitor (0.116) Unity Treat x Self-Monitor p=0.985 (0.118) Disagreement Treatment p=0.565 when s.m. scale =0 (0.494) Unity Treatment when p=0.415 s.m. scale =0 (0.494) Self-Monitoring Scale p=0.667 when treatment =0 (0.082) Ideology (one year ago) p=0.747 (0.101) Gender p=0.094 (0.230) Graduated College p=0.786 (0.249) Residence in the South p=0.559 (0.255) Race p=0.150 (0.081) News Interest p<0.001 (0.140) Democrat p<0.001 (0.278) Constant Pp=0.521 (0.692) N=400 A3.4. Replications of Studies 3.1 and 3.3 We conduct a number of replications of our experiments to ensure that the results we show translate to other variations on treatments and other samples. First, we replicate Study 3.1 with an additional sample (N=100, Internet sample). This study produces results that are virtually identical to those we show in Figure 3.1.

26 A3.4.1 Replication 1 of Study 3.3 We also replicate the results of Study 3.3, using different treatments than those we use in the in-text study. To carry out this test, we randomly assigned 156 participants from our online convenience sample to one of three groups. The first group read the news clip highlighting the importance of partisans ( partisan importance ); Group 2 read the news clip highlighting disagreement between political parties ( partisan disagreement ); and Group 3 read no news article at all ( control ). We then asked all respondents in each of the three groups to select their own party identification. In Table A3.4, we display the percentage of participants in each group who identified as independent (either leaning or pure), as opposed to with a party. As the table shows, among those respondents who read no article, 41% of respondents identified as independent. What happens when participants read a news-clip before reporting their own party identification? We first see that there is little substantive difference and no statistical difference when respondents read coverage highlight the importance of partisans 47.1% identify as politically independent. In contrast, reading about partisan disagreement causes an increase in the rate at which participants claim to be politically independent: 63.9% of respondents in the partisan disagreement group identify as independent. This is both a substantively large and statistically significant increase from the rate of identification in both, the control and partisan importance groups. 9 9 This is significantly (p=0.08) greater than among those who read an article about the importance of partisanship, and it is significantly (p=0.02) greater than among those who read no news article at all.

27 Table A3.4: Percentage of Independents by Treatment, Replication of Study 3.3 No News Clip (Control Group) % Independent (standard error) 41.2 (8.57) Partisan Importance (Positive) News Clip 47.1 (8.68) Partisan Disagreement (Negative) News Clip 63.9 (8.11) A3.4.1 Replication 2 of Study 3.3 We conduct still one more replication of Study 3.3, we discuss a part of this replication in Chapter 3 when explaining that treatments do not change issue positions. Here we recruited 641 participants (Internet sample) and invited them to participate in a study. In the first wave of the study we measured our participants ideological positioning. Then, we re-contacted our participants several weeks later, and invited them to take part in a second study. Of the 641 participants, 420 (about 65.5%) for this second study. In this second study, some of our participants were randomized to three treatment groups: our familiar control, partisan unity and partisan disagreement. After treatment, we again measured our participants issue positions, as well as their partisanship. The remaining participants were randomized in to separate groups, serving as checks for additional studies. We first ensured there were no systematic patterns to which participants from the first wave of our study returned for the second wave of our study. Indeed, in a model where participation in the second wave of the study is the binary outcome variable (1 = participated, 0 = did not participate), we show that wave 1 political positions have no predictive power in participation.

28 Next, we see that exposure to partisan disagreement does, indeed, lead more participants to identify as independent in wave 2 of our study. Indeed, the percentage of pure independents in the partisan disagreement treatment increases by 13 percentage points relative to the control. When we use a 4-point variable, where 4 means someone is a strong partisan and 1 means they are independent, we see that partisan disagreement significantly decreases the average partisan strength score relative to the control. Importantly we see no such changes in the partisan unity group. The goal of this study, however, is to compare pre- and post-treatment positions on issues. We want to show that our treatment is not moving people on issues, but rather is affecting their willingness to display their partisanship. To consider this possibility, we create a distance measure on our issues. We take the absolute value of the difference between the position at wave 1 and the position at wave 2. The greater this difference, the greater the change in positions; this difference can range from 0, meaning that the person s position stayed the same to 6, meaning that it change from one extreme to the other. Using the absolute value we can focus on the differences without making assumptions about the direction. We see that across all our issues there are few changes in issue positions. Most importantly, however, the partisan disagreement treatment does not produce disproportionately bigger shifts. We show this effect in Figure 3.4 of Chapter 3. We measure our issue positions by asking our participants to note on a seven-point scale how strongly they agree or disagree with the following statements:

29 1. There need to be stricter laws and regulations to protect the environment 2. Wealthy Americans should pay higher tax rates than they currently do. 3. The government should spend more on social services to help needy people even if it means going deeper into debt 4. The government needs to spend more on subsidies to make health care affordable and accessible 5. The choice to have an abortion should be legal in all or most cases. 6. Gays and lesbians should be allowed to marry legally. A3.5. Dates and details for the surveys used in Todd Akin example The data we use to track the percentage of independents pre and post-todd Akin comes from Pew. We list the dates and survey titles in Table 3A.5 below. We exclude the Pew survey that concluded on August 19 as this is the day on which Akin made his comments and some people may have been interviewed after Akin s comments were broadcast. Table A3.5: Surveys Used in Todd Akin Example Dates for 2012 Date N Survey Title August 2 August 5, ,005 Weekly News Interest Poll August 9 August 12, ,002 Weekly News Interest Poll August 23 August 26, ,010 Weekly News Interest Poll August 31 Sept. 2, ,008 Weekly News Interest Poll Additional Years for Comparison Date N Survey Title Change July 23 July 27, ,503 Political & Economic Survey Sept. 9 Sept. 14, ,982 Foreign Policy Poll 0 pct-points July 22 July 26, ,506 Political Survey August 27 August 30, ,005 Political Poll +2 pct points August 5 August 8, ,002 Weekly News Interest Poll August 12 August 15, ,005 Weekly News Interest Poll +1 pct points August 4 August 7, ,001 Political Poll August 25 - August 27, ,010 Political Poll -4 pct-points July 31 August 3, ,002 Political Poll August 20 August 24, ,501 Political Poll +3 pct points

30 We also include the dates and changes for the additional years we use as a baseline of comparison. Pew runs many surveys, and, as a result, there are many surveys we could have used. We used several characteristics when selecting surveys: (a) survey was fielded at relatively the same time as the pre and post-akin studies in 2012; (b) surveys have similar N-sizes, (c) surveys have a uniform study topic (meaning for example, that the goal is to compare two political surveys, rather than a political survey to a religion survey) and (d) surveys were conducted over a similar number of days. We can find such studies in all our cases except in 2008 and We include 2008 as an example because it is a Presidential election and thus provides an important comparison to In 2013, the closest dates we can find show a substantial decreases in the percentage of independents, suggesting that the increase we see in 2012 is unique.

31 A4. Appendix to Chapter 4 A4.1 Additional Information on Study 4.1 In this study we assigned members of an Internet convenience sample (demographics in Table A1.4) to either a partisan unity treatment, a partisan disagreement treatment or a control treatment. We use the treatments presented in Appendix A3, section A3.2. Following treatment, participants answered a series of distractor questions about airline policy. We ask the distractor questions between the treatments and the key outcome questions to ensure that people do not attempt to guess the goals of the study and adjust their responses accordingly. After the distractor tasks, we asked participants a series of questions that follow from Iyengar et al (2013). The questions were as follows: Coworker question: Imagine that there is a new co-worker at your job. This coworker voted prefers the same candidate as you for President. This co-worker also frequently discusses politics. How happy or unhappy would you be about this new co-worker? [Scale of 1 (very unhappy) to 10 (very happy)] Lawn-signs question: Imagine that you have a neighbor who likes to display political signs promoting candidates on his or her lawn during electoral campaigns. How happy or unhappy would you be about this neighbor? [Scale of 1 (very unhappy) to 10 (very happy)] We also measured a number of other political variables, including interest. We then considered our results by treatment (Table A4.1). Our results show that people who first read about partisan disagreement would be significantly less happy with having a co-worker who discussed politics and having a neighbor who displayed political lawn signs. However, we show that there are no differences in

32 political interest as a result of our treatments. These results suggest that the dissatisfaction is with partisanship, rather than with politics itself. Table 4A.1: Responses to Partisan Coworkers or Lawn Signs Control Disagreement Unity Coworker Lawn signs Political Interest (higher value = more interest) Cell values are means, bold values are significantly different form control at p<0.1 (coworker) and p<0.05 (lawn signs) A4.2. Additional Information for Study 4.2 In Study 4.2 we recruited a set of Internet savvy participants to take part in our study. These participants were deliberately selected because they could search for images online and produce the URL for our study. This result still produces a diverse set of participants. We include more details about this sample in Appendix A1, Table A1.4. A The Treatments Study 4.2 introduces a new treatment designed to eliminate the alternative explanation that our results are a function of people imagining politicians, rather than partisanship. This new treatment describes partisan disagreement at the mass level, rather than the elite level. The control treatment matched the treatment used in Study 3.3 (Appendix A3), the disagreement and unity treatments remained largely the same as those used in Study 3.3 with one change. The treatments in Study 3.3 began with As President Obama begins his second term the treatments in Study 4.2 were adjusted to state With President Obama well in to his second

33 term ; the adjustment is made to reflect the dates on which this study was fielded. The additional, mass partisanship treatment was as follows: With President Barack Obama well in to his second term, the Democrats and Republicans in Washington appear to be as divided as ever. Political experts predict that the partisan bickering which has characterized Washington in recent years will also spill over to ordinary Americans. The next year may very well bring a continuous cycle of bitter partisan disagreement among the American public. A The Images instructions: In order to capture the images, our participants were given the following We are interested in the kinds of images that come to mind when you think of the term "political partisan." Using Google Image Search (image.google.com) or another website of your preference, please find an image that you think BEST fits your vision of a "political partisan." Please paste the full URL (including http) for that image in the space below. (Opening a new browser window or browser tab will not disrupt this study). Then, they were given a text-box for the URL; the text-box was adjusted such that the response had to start with http, this limiting non-url responses. All images were retrieved using URLs that participants provided. Images were coded independently by two coders. The coders were trained and given samples of coded images. Of the total number of URLs provided by the participants, 16% led to broken links. We conducted a test to ensure that these links were randomly distributed among participants. We find that group assignment does not explain patterns of broken links. A probit that predicts the likelihood of a broken link does not produce

34 any significant coefficients on the treatments and leads to a χ 2 of 3.08 (p=0.38). We also find no evidence that partisanship predicts broken links (χ 2 = 0.06, p =0.96). A Randomization Checks We conducted a randomization check using pre-treatment characteristics, which in our case are partisanship. We find no evidence that these pre-treatment characteristics affected group assignment. An ordered probit that uses treatment assignment as a dependent variable produces a χ 2 of 1.05 (p=0.59). A4.3 Additional Information on Study 4.3 A The Sample The sample used in this study came from Survey Sampling International (SSI). We include more details about this sample, as well as the demographic details of the sample Appendix A1, Table A1.3. A The Treatments We include the neighborhood images in Table 4.2 of the text. The partisan information treatments are as follows: Unity Treatment With President Barack Obama well in to his second term, the Democrats and Republicans in Washington are showing signs of being more unified. Political experts predict that Americans can expect a new era of bipartisanship in Washington. The profound debate that has raged between the two parties appears to be settling. The next year may very well bring progress towards two parties cooperating in Washington. Disagreement With President Barack Obama well in to his second term, the Democrats and Republicans in Washington appear to be as divided as ever. Political experts predict that the partisan bickering which has characterized Washington in recent years will also spill over to ordinary Americans. The next year may very well bring a continuous cycle of bitter partisan disagreement among the American public.