Measuring the Impact of Cross-Cultural Exchanges. Juliana Schroeder Assistant Professor, UC Berkeley

Size: px

Start display at page:

Download "Measuring the Impact of Cross-Cultural Exchanges. Juliana Schroeder Assistant Professor, UC Berkeley"

Myron Smith
6 years ago
Views:

1 Measuring the Impact of Cross-Cultural Exchanges Juliana Schroeder Assistant Professor, UC Berkeley

2 Case Study Outline 1. The cross-cultural exchange we will evaluate today: Seeds of Peace 2. Focusing the evaluation: Big Questions 3. Understanding theory: Forming Hypotheses 4. Methods for evaluation: Measurement 5. Results: Answering Questions 6. Evaluating other questions

4 There s a saying at Seeds of Peace Treaties are made by governments Peace is made by people.

5 Conflict Regions Seeds of Peace focuses on three conflict areas: The Middle East South Asia Maine

6 Seeds of Peace Objectives To humanize the conflict through open dialogue To provide conflict resolution and leadership training To create a network of young leaders who will become a core constituency for peace

7 Seeds of Peace Camp 3 week overnight camp in Maine, USA Neutral-setting model

8 Selected on two primary criteria: 1) Leadership potential; 2) English-speaking ability

9 Water sports CANOEING, WATERSKIING, SAILING

10 Athletics SOCCOR, BASKETBALL, TENNIS, SOFTBALL

11 Arts PAINTING, DRAWING, DANCING, MUSIC

12 Seeds of Peace Camp But unlike any other summer camp...

13 Dialogue Sessions

14 Group Challenges

16 Seeds of Peace: Change Makers Friends Friends Parents Palestinian Seeds Israeli Seeds Parents Society/ Government Society/ Government

17 The Challenging Transition Home Friends Friends Parents Palestinian Seeds Israeli Seeds Parents Society/ Government Society/ Government

18 After Camp: Regional Programming Advanced Dialogue Groups Civic and Cultural Education Cross-Border Workshops Community Service Projects The Olive Branch Delegation Leaders Program Parents Program Model Schools Initiative

19 Case Study Outline 1. The cross-cultural exchange we will evaluate today: Seeds of Peace 2. Focusing the evaluation: Big Questions 3. Understanding theory: Forming Hypotheses 4. Methods for evaluation: Measurement 5. Results: Answering Questions 6. Evaluating other questions

20 Seeds of Peace Objectives To humanize the conflict through open dialogue To provide conflict resolution and leadership training To create a network of young leaders who will become a core constituency for peace

21 Focusing the Evaluation What questions need to be answered? To what extent & how is conflict alleviated by the program? Do campers humanize the outgroup more by the end of camp? Do networks form between conflict groups? Does this attitude change last after camp ends? Re-entry problem Does this attitude change spread to others?

22 Focusing the Evaluation What should we measure? Outgroup positivity : Humanization, empathy, overall feelings toward the outgroup Attitudes toward the conflict (willingness to compromise) Relationships among campers How to operationalize these constructs?

23 Focusing the Evaluation Who should we evaluate? Jewish Israelis & Palestinians participants* Sample size: at least 50 participants * Participants networks outside camp

24 Case Study Outline 1. The cross-cultural exchange we will evaluate today: Seeds of Peace 2. Focusing the evaluation: Big Questions 3. Understanding theory: Forming Hypotheses 4. Methods for evaluation: Measurement 5. Results: Answering Questions 6. Evaluating other questions

25 Hypothesis Formation For every research question, you will form & test a hypothesis Hypothesis generation comes from three sources: Theory Empirical studies Anecdotal data

26 Intergroup Contact Theory (Allport, 1954) Over 6,000 studies have tested contact theory (Pettigrew & Tropp, 2006) 4 optimal but not necessary conditions 1. Equal status between groups 2. Common goals 3. Intergroup cooperation 4. Support of authorities A fifth condition? Friendship Potential

27 Neutral Setting Interventions Friendship Potential vs. the Re-Entry Problem The contact situation must provide participants with the opportunity to become friends. (Pettigrew, 1998) vs. Neutral-setting encounters rely on the creation of artificial, mediated settings detached from the actual conflict context any effect of the encounter will be erased upon participants inevitable return to reality. (Hammack, 2006)

28 Anecdotal Evidence Make one friend.

29 Hypotheses H1: Contact will improve outgroup positivity from pre-camp to post-camp. H2: Making an outgroup friend will predict post-camp positivity.

30 Hypotheses H3: Outgroup positivity will fade after re-entry, but some may be maintained. H4: Outgroup friendship will predict positivity after re-entry.

31 Case Study Outline 1. The cross-cultural exchange we will evaluate today: Seeds of Peace 2. Focusing the evaluation: Big Questions 3. Understanding theory: Forming Hypotheses 4. Methods for evaluation: Measurement 5. Results: Answering Questions 6. Evaluating other questions

32 Testing Hypotheses 5 years:

33 Survey Questions Measuring Outgroup Positivity Humanization They are refined and cultured ; They are rational and logical ; They are unsophisticated (reverse-scored); They are my equal ; They are less than human (reverse-scored) Empathy If I saw a person from the other side was being treated unfairly, I think I would feel angry at the way they were being treated. Feelings How positively do you feel? ; How close do you feel? ; How similar do you feel? ; How much do you trust? Measuring Outgroup Friendships Think of the 5-10 people to whom you feel most close

34 Data Collection 3 weeks 6-9 months Pre-camp survey (First day of camp) Post-camp survey (Last day of camp) Follow-up survey (6-9 months after camp)* * Missing 2010 & 2014

35 Case Study Outline 1. The cross-cultural exchange we will evaluate today: Seeds of Peace 2. Focusing the evaluation: Big Questions 3. Understanding theory: Forming Hypotheses 4. Methods for evaluation: Measurement 5. Results: Answering Questions 6. Evaluating other questions

36 Did contact improve outgroup positivity from pre-camp to post-camp? Yes, in all years tested ( ).

37 Outgroup Positivity (Jewish Israelis) Outgroup Positivity (Palestinians) Pre-camp Post-camp 2 Pre-camp Post-camp

38 Who made an outgroup friend? 100% 80% 60% 40% 20% Percentage of campers who made at least one outgroup friend 74% 64% 54% 54% 63% 68% 63% 26% Jewish Israeli Palestinian 85% 52% 0%

39 Did making an outgroup friend predict outgroup positivity at postcamp? Yes, in all years tested (2010, 2011, 2012, & 2014).

40 The effect of having an outgroup friend before camp Who makes a friend at camp? The effect of making an outgroup friend at camp Attitudes generalize (from campers to entire group)

41 Did any outgroup positivity remain after re-entry? Yes, in all years tested (2011, 2012, & 2013).

42 2011 *(54% follow-up completion) Outgroup Positivity (Jewish Israelis & Palestinians) (n=76) 2011 completed follow-up (n=41) 5 4 Selection Bias Attitude Erosion Residual Positivity 3 Pre-camp Post-camp Follow-up

43 2012 *(56% follow-up completion) 6 Outgroup Positivity (Jewish Israelis & Palestinians) 2012 (n=81) 2012 completed follow-up (n=45) 5 4 Selection Bias Attitude Erosion Residual Positivity 3 Pre-camp Post-camp Follow-up

44 2013 *(41% follow-up completion) 6 Outgroup Positivity (Jewish Israelis & Palestinians) 2013 (n=79) 2013 completed follow-up (n=32) 5 4 Attitude Erosion Residual Positivity 3 Pre-camp Post-camp Follow-up

45 Did outgroup friendships formed during camp predict follow-up positivity? 3 possibilities: 1. Having an outgroup friend at post-camp predicts followup positivity. 2. Having an outgroup friend at follow-up predicts followup positivity. 3. The change in outgroup friends from post-camp to follow-up predicts follow-up positivity.

46 Evidence for each possibility: 1. Having an outgroup friend at post-camp predicts follow-up positivity. 2011: Yes, β = 0.31, p = : No, β = -0.19, p = : No, β = 0.15, p =.288

47 Evidence for each possibility: 2. Having an outgroup friend at follow-up predicts follow-up positivity. 2011: No, β = 0.09, p = : No, β = 0.12, p = : No, β = 0.14, p =.244

48 Evidence for each possibility: 3. The change in outgroup friends from post-camp to follow-up predicts follow-up positivity. 2011: No, β = -0.16, p = : Yes (marginally), β = 0.23, p = : No, β = 0.01, p =.979

49 What have we learned?

50 1) Outgroup positivity improved over camp Change from pre-camp to post-camp Support for Contact Theory Generalization of attitude change

2) Re-Entry is a problem but maybe a surmountable one 6 5 4 Positivity Israelis & Palestinians 2014 (n=69) 2013 (n=79) 2013 completed

51 2) Re-Entry is a problem but maybe a surmountable one Positivity Israelis & Palestinians 2014 (n=69) 2013 (n=79) 2013 completed follow-up (n=32) 2012 (n=81) 2012 completed follow-up (n=45) 2011 (n=76) 2011 completed follow-up (n=41) 2010 (n=122) 3 Pre-camp Post-camp Follow-up

52 3) Outgroup friendship is critical for attitude change

53 Disseminating Results Who needs to know & how to inform them? Program planners Internal memos, calls Potential donors Fundraising materials, funding agencies General public Press releases, New York Times op ed, etc. Scientific community Peer-reviewed journal article

54 Case Study Outline 1. The cross-cultural exchange we will evaluate today: Seeds of Peace 2. Focusing the evaluation: Big Questions 3. Understanding theory: Forming Hypotheses 4. Methods for evaluation: Measurement 5. Results: Answering Questions 6. Evaluating other questions

55 Other Questions 1. How do we know that the program causes attitude change? Comparison groups: applicants, national samples, parents Experiments

56 Other Questions 2. Does attitude change spread? Friends Friends Parents Palestinian Seeds Israeli Seeds Parents Society/ Government Society/ Government

57 Other Questions 3. Is there behavioral change in addition to attitude change? Social media (Facebook posts) Attendance at follow-up events Conversations with family & friends

58 Evaluating Your Own Programs 1. What are your big questions? Consider the objectives of your program 2. What are your hypotheses? Draw on 3 sources: theory, empirical research, anecdotal data 3. How will you test these hypotheses? Develop your measures, operationalize your questions, and collect data 4. To whom & how to disseminate results?

59 Questions or Comments? Contact me: For more information:

BUILDS CONFIDENCE, COMMUNICATION COMMUNITY

BUILDS CONFIDENCE, COMMUNICATION COMMUNITY & FE NBC NATURED ON THE TIGHTLY NEW OD S, CBS S AY SHOW, U MORNNDAY & MORING E! It s Truly a Gift Camp SAY is the gift that you give to a child who stutters the