Evaluating the Impact of Interventions Designed to Change Social Attitudes. Sequana Tolon, Donald Fischer & Adena Young-Jones

Transcription

1 1 Evaluating the Impact of Interventions Designed to Change Social Attitudes Sequana Tolon, Donald Fischer & Adena Young-Jones Missouri State University Tolon, S., Fischer, D., & Young-Jones, A. (2013, April). Evaluating the Impact of Interventions Designed to Change Social Attitudes. Paper presented at the 28 th annual meeting of the Society for Industrial and Organizational Psychology, Houston, TX

2 2 ABSTRACT The malleability of racial attitudes of students in courses with objectives that involve understanding and valuing diversity were assessed using both implicit and explicit measures. We found support for the malleability of bias assessed by explicit measures but no support for the malleability of automatic bias assessed by implicit measures. PRESS PARAGRAPH A decrease in the overt expression of racist attitudes over the past half-century has been accompanied by evidence of pervasive implicit racial bias. Diversity training interventions often endeavor to help participants learn about their own attitudes as a way of motivating positive change. Despite the widespread use of these programs, empirical evidence supporting their effectiveness has been lagging one survey of colleges and universities reported that 81% used diversity training programs to address racial discrimination without any evaluation of their effectiveness. This study examined an intervention designed to change social attitudes and found mixed evidence in support of its effectiveness.

3 3 Interventions designed to change social attitudes can be found in a broad range of settings. Attitude change initiatives are present in both public and private sector organizations seeking to create climates which respect diversity and value inclusion. In corporate settings globalization and an increasingly diverse workforce are often cited as factors that require cultural competence in effective human resource management (e.g., Cascio & Aguinis, 2011) and diversity training programs designed to achieve this objective focus upon social attitudes (Lee Mun Wah; Likewise, in higher education the curriculum often includes courses that have, among other objectives, the development of egalitarian values and respect for diversity. For example, it is easy to find courses in psychology departments that include the topics of stereotypes, prejudice and the importance of social attitudes in understanding and changing human behavior this is especially so in courses that focus upon personal adjustment, psychological growth/well-being, and understanding diverse populations. The pedagogy used in college courses frequently reflects what is used by organizational change agents in diversity training programs. These efforts often begin with the proposition that psychological health and personal effectiveness are fostered by self-awareness and experiential openness. Ironically, few of us have accurate self-knowledge when it comes to our social attitudes and how these affect our behavior and many of us do not accurately perceive the impact of our behavior upon others. Indeed, it is hard to find a chapter in an adjustment text that does not include a self-assessment activity that encourages students to learn more about their current status in order to identify the direction and nature of steps they might take to achieve positive change (Nevid & Rathus, 2010; Watson & Tharp, 2002). When the focus is specifically upon understanding and improving interpersonal relations with racial or ethnic minority individuals, the social attitudes involve automatic or implicit processes of which one may not be aware and

4 4 effortful or explicit processes of which one is more consciously aware (Fazio & Olsen, 2003; Greenwald & Banaji, 1995). A corresponding distinction is made between implicit and explicit measures of attitudes, where the latter are typically self-reports that provide an opportunity to reflect and deliberate before responding and the former attempt to reduce the effects that such reflection and deliberation might produce. In particular, Greenwald, McGhee, and Swartz (1998) suggest that the Implicit Association Test (IAT), which is based upon reaction times in classification tasks, is resistant to self-presentation artifacts and independent of a person s introspective ability or selfknowledge. For instance, people who espouse socially desirable values like fair-mindedness and overtly describe themselves as not the kind who judge people according to their race, but rather according to individual merit on a case-by-case basis, often have IAT scores that indicate a strong tendency to associate negative attributes with those identified as Black (Greenwald, McGhee, & Swartz, 1998). Consequently, implicit measures like the IAT are useful when assessing racial/ethnic attitudes in settings where demand characteristics (e.g., referent group norms) motivate people to respond in socially acceptable ways (Fazio & Olsen, 2003; Perugini, 2005). Empirical evidence links explicit and implicit measures of negative social attitudes with dysfunctional and harmful behavior when the attitudes involve racial and ethnic minorities. Allport s social psychology classic The Nature of Prejudice (Allport, 1954) and Pettigrew s work over the past half-century vividly illustrate how negative social attitudes impede effectiveness in racially diverse settings (Campbell & Pettigrew, 1959; Pettigrew & Martin, 1979; Pettigrew, 2011). More recent interest has focused upon the incremental contributions that implicit measures can make regarding the prediction of individual conduct in these settings

5 5 (Greenwald, Poehlman, Uhlmann, & Banaji, 2009). Furthermore, implicit measures of racial/ethnic attitudes have been found to better predict overt behavior that is consistent with these attitudes. For example, implicit measures of racial/ethnic attitudes have been found to better predict differences in police officers decisions to shoot/not shoot a suspect pulling an object from his pocket (a wallet or weapon) in a simulation exercise that varies the race/ethnicity of the suspect, relative to explicit measures of racial/ethnic attitudes (Eberhardt, 2007). Perhaps of more general interest in organizational contexts are studies that link IAT measures of racial attitudes with interpersonal space and collaboration initiatives with Blacks (Amodio & Devine, 2006), discomfort and negative interactions with Blacks (Dovidio et al., 1997), and negative evaluations of Black job applicants (Ziegert & Hanges, 2005). These studies provide a rationale for organizations, including universities, to be concerned about the nature and prevalence of negative racial attitudes among members, and they provide an incentive for these organizations to use implicit measures when assessing endeavors to change these attitudes. While it is easy to find empirical evidence linking social attitudes with interpersonal behavior, it is not as easy to find empirical evidence supporting the effectiveness of university courses or diversity training programs that attempt to change negative social attitudes, particularly when the evidence includes implicit measures (Gregg, Seibt & Banaji 2006; Joy- Gaba & Nosek, 2010). However, Rudman, Ashmore, and Gary (2001) report an effort at Rutgers University that represents an exception in this regard. This study compared pre-posttest changes in the attitudes of students in an upper-division prejudice and conflict seminar, where the course content involved relevant topics and activities, with those in a research methods course, where the content did not involve relevant topics. They found support for the malleability of students

6 6 attitudes in both explicit and implicit measures for students in the prejudice and conflict course but not for students in the experimental methods course. The present study attempts to replicate and extend the findings of Rudman et al. (2001). More specifically, we used multiple implicit and explicit measures of racial attitudes in a repeated measures design to assess the effectiveness of activities that are common components of diversity training programs and university courses where changing social attitudes are key objectives. These activities were embedded in two undergraduate psychology courses an adjustment course entitled Psychology for Personal Growth and a diversity course entitled The Psychology of Diverse Populations. A mid-term assessment of racial attitudes toward Blacks was used to introduce a unit that examined racial and ethnic attitudes in both courses. A second assessment was administered on the final day of the term. The assessments included multiple implicit (IAT) and multiple explicit measures. Like the Rutgers study, one of the course instructors was Black (the adjustment course) and the other was White (the diversity course). Unlike the Rutgers study, the curricular activity involving racial attitudes was common to both courses; however, this was the only unit that addressed social attitudes in the adjustment course, whereas it was one of many curricular units addressing these topics in the diversity course. Like the Rutgers study, we hoped to identify effective methods for developing attitudes that are more consistent with the university s mission, attitudes that better equip students to interact effectively in racially diverse environments, and perhaps point the way toward better evaluation designs of diversity training efforts in other organizational contexts. Hypotheses Hypothesis 1: The malleability of racial attitudes will be evident in the pre-post change for both the explicit and implicit measures of students attitudes. More specifically, White

7 7 students will explicitly express less resentment and hostility toward Blacks and more desire to control their prejudicial reactions toward Blacks on their posttest measures, relative to their pretest measures. Similarly, White students will implicitly express a stronger association of Blacks with positive evaluations and mental strength on the posttest measures, relative to their pretest measures. If the curricular activities in the diversity course increase students awareness of their racial attitudes and an understanding of the effects these attitudes have upon their well-being, then students should be motivated to change their attitudes in a positive direction and express this desire on explicit measures. To the extent the curricular activities provide opportunities for positive, prosocial interactions with racial minority individuals, these experiences should change the implicit associations and automatic responses students make to racial minority individuals, and this change should be evident in the implicit measures (Pettigrew, 2011; Rudman et al., 2001; Zajonc, 1968). Hypothesis 2: The malleability of racial attitudes will be more evident in the pre-post change for the explicit measures among students in the diversity course than for students in the adjustment course. We expect students in the diversity course to be more motivated to change the way they consciously think about Blacks and try to control their expression of prejudicial behavior toward Blacks. The greater breadth, intensity, and duration of the curricular activities in the diversity course should, if effective, produce larger gains than the single session activity in the adjustment course. However, we expect these gains to be more evident in the conscious, deliberative thinking of these students rather than in the implicit associations and automatic processes that are experientially conditioned from childhood to the present. We expect this because of the limited

8 8 exposure to positive, prosocial experiences with racially diverse others which a course with a White instructor provides, relative to the exposure that a course with a Black instructor provides (Pettigrew, 2011; Zajonc, 1968). Hypothesis 3: The malleability of racial attitudes will be more evident in the pre-post change for the implicit measures among students in the adjustment course than for students in the diversity course. The more frequent opportunities to interact with a Black person in the adjustment course should, if the experiences are positive and prosocial, produce larger gains than the more limited opportunities in the diversity course. However, these gains should be more evident in the implicit measures relative to those of students in the diversity course (Pettigrew, 2011; Zajonc, 1968). METHOD Participants Participants (N = 86) were recruited from psychology courses at a public university in the Midwest that has relatively few racial and ethnic minority students (about 10%). Originally the design included two sections taught by the same instructor in each of two courses an adjustment course (Psychology for Personal Growth) taught by a Black male and a diversity course (The Psychology of Diverse Populations) taught by a White female. However, we were unable to collect posttest data for one section of the adjustment course, which left us with two sections of the diversity course (n = 47) and one section of the adjustment course (n = 20). Missing data on one or more of our measures further reduced the number of participants for whom we had complete pretest and posttest data (n = 47). Measures

9 9 Explicit Racial Prejudice. This measure combined the seven items of the Modern Racism Scale (McConahay, 1986) with the four items of the Explicit Racial Resentment Scale (Wilson & Davis, 2008) to form an 11-item measure of explicit racial prejudice. This scale included statements like Discrimination against Blacks is no longer a problem in the United States and I resent all of the special attention/favors that Blacks receive; other Americans like me have problems, too scored on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). The respondents scale score was the average response, and higher scores indicate more negative attitudes toward Blacks. Wilson & Davis (2008) report alpha coefficients that range from.64 to.76 for the resentment scale and.67 to.79 for the racism scale. Motivation to Control Prejudiced Reactions Scale. This 17 item scale was developed by Dunton and Fazio (1997) to assess concern about acting prejudiced ( it s important to me that other people not think that I m prejudiced ) and the desire to avoid dispute ( I m not afraid to tell others what I think, even when I know they disagree [reverse scored]). Responses are scored on a scale from 1 (strongly disagree) to 5 (strongly agree), with the total score being the average response. Higher scores indicate a stronger motivation to control prejudiced reactions. Dunton and Fazio (1997) report alpha coefficients ranging from.74 to.77 from several samples. Racial Preference IAT. An IAT modeled after those used by Greenwald, Banaji, and others was used assess the degree of students implicit evaluative race bias. The seven block procedure and scoring algorithm (D measure) recommended by Greenwald et al. (2003) was used with images (cropped pictures of 12 faces; six Black male and female faces, six White male and female faces) and 16 word stimuli (see Table 1). The IAT stimuli were obtained from the Project Implicit web site where they have been used extensively to assess racial preference (Nosek et al., 2007; Nosek, Banaji, & Greenwald, 2002). The more negative the IAT effect, the

10 10 stronger the implicit association between negative valence and Blacks (and positive valence with Whites). Racial Stereotyping IAT. An IAT modeled after that developed by Amodio and Devine (2006) was used to assess the degree of students implicit racial stereotyping. The same seven block procedure, scoring algorithm, and face stimuli described above were used with 12 word stimuli to obtain this measure (see Table 1). The more negative the IAT effect, the stronger the implicit association of mental strength with Whites (and physical strength with Blacks). Metaanalytic studies report internal consistency coefficients for IATs like those used in this study to be approximately.80 (Lane et al., 2007). Procedure Software from Millisecond Software, Inc. was used to create racial preference and stereotyping IATs that could be administered through a web site hosted by Millisecond. InQsit software (Fortriede, 2005) was used to create a web based questionnaire that contained the explicit measures. All measures were self-administered by students during a single class period in a computer lab at approximately mid-term as part of an introductory activity for a unit that involved attitudes about racial and ethnic minorities. Students were provided immediate feedback regarding their scores, including interpretive suggestions about what scores within various ranges might mean and how students might use this information. The same measures were administered a second time during the final class period of the term. 1 RESULTS Descriptive statistics for the study variables are presented in Table 2. These results suggest that the measures provide adequate variance and internal consistency to support additional analyses. Table 3 contains zero-order correlations for study variables. These results

11 11 provide evidence of convergent validity for both the explicit and implicit measures in that the two measures within each category were significantly correlated on both the pretest and posttest. Consistent with others findings (Fazio & Olsen, 2003; Greenwald et al., 2002), these results also provide evidence of discriminant validity and dissociation between the two types of measures in that only one of the eight correlations involving an implicit and explicit measure was significant. With respect to sample demographics, 63% were female and 88% were White. Tests of Hypotheses Only non-black students (N = 44) were included in the sample used to test the hypotheses because our predictions and measures target attitudes toward Blacks. Consistent with the first hypothesis, a multivariate analysis of variance for a repeated measures design produced a significant within-subjects main effect for the pre-posttest factor (Wilks Lambda =.74, F (4, 38) = 3.30, p <.05, partial eta-squared =.26). Univariate analyses of the four dependent measures produced significant within-subject main effects for pre-posttest differences on only the Motivation to Control measure (F (1, 41) = 5.75, p <.05, partial eta-squared =.12) and Racial Preference IAT (F (1, 41) = 10.69, p <.01, partial eta-square =.21). An examination of cell means (see Table 4) indicated that the pre-posttest difference was in the predicted direction for the explicit measure (Motivation to Control) but not the implicit measure (Racial Preference IAT). Students expressed a greater desire to control their prejudicial behavior on the posttest but they exhibited a stronger association between positive valence and Whites (and negative valence with Blacks) on the posttest. Counter to the predictions of the second and third hypotheses, none of the interaction effects were significant for the within-subjects factor (pre-posttest) with the between-subjects factor (course/instructor), nor were there any significant main effects for the between-subjects factors (i.e., course/instructor or section effects).

12 12 DISCUSSION This study used both implicit and explicit measures to investigate the malleability of racial attitudes in the context of two university courses. The results indicated a significant increase in students explicit motivation to control their prejudice without a corresponding significant decrease in their explicit level of racial resentment and hostility. On the positive side, the magnitude of the change in students conscious motivation to control their prejudice approaches Cohen s threshold for being labeled moderate (Cohen, 1992) and, as such, represents evidence of the intervention activities effectiveness. Being aware of one s prejudices and motivated to regulate these beliefs is a necessary first step in the right direction for organizations that value inclusion and diversity. Oftentimes, at the onset of interventions to change social attitudes, participants do not believe they are prejudiced against those who are different. Results from this study provide evidence that, as a consequence of the intervention activities, students awareness of their prejudicial attitudes was heightened. Thus, the explicit and conscious manifestation of prejudice was addressed. Although this study did not find support for the malleability of automatic bias assessed by implicit measures, the value of the implicit assessments may have been to encourage students to confront the possibility they are at risk of being prejudiced in ways they are not consciously aware. The idea that one may be influenced in ways he or she is not consciously aware may lead to greater concern about monitoring one s conduct and a heightened explicit desire to control prejudicial reactions. On the other hand, this finding may partially be a consequence of the group assessment procedures that were used. Technical glitches and students completing the assessments at different times led to disruptions and distractions, and several students complained about difficulties concentrating and staying focused on the IAT s classification tasks.

13 13 Finally, the lack of support for the malleability of bias assessed by the implicit measures may partially be a function of the amount of time between pretest (midterm) and posttest (end of term) assessments. Rather than waiting until midterm to conduct the pretest, future research should evaluate students at the beginning of the semester to assess the maximum effect of the course content and delivery. In addition to creating distractions, technical problems compounded by the group administration procedure caused the loss of much data; over 40% of the students in our sample were missing values on one or more of the measures. This contributed to another problem: low power. The tests for our hypotheses were based upon a sample of only 44 students, which resulted in a probability of about.5 for finding significant results (p <.05) when a moderate effect exists in the population (Cohen & Cohen, 1983). In addition to addressing the technical problems associated with group administrations of the assessments, larger sample sizes should be used in future research. Although this was a quasi-experimental investigation, it has practical utility for organizations that are interested in establishing valid assessments of attitudes and using these baseline measures to evaluate the effectiveness of their training interventions. While there is an established need for empirically based training evaluations, a lack of sufficient research on the development and use of baseline measures exists. Additional research that emphasizes measurement outcomes of diversity training can provide better evidence of the return on investment (Paluck, 2006) and better index the real value organizations should place upon diversity training interventions.

14 14 Notes 1. The authors want to acknowledge the contributions of research team members Phil Thompson, Louis Oberdiear and Teanna Lucas; we want to express our gratitude for their efforts in making this research possible.

15 15 References Allport GW The Nature of Prejudice. Reading, MA: Addison-Wesley. Amodio, D. M. & Devine, P. G. (2006). Stereotyping and evaluation in implicit race bias: Evidence for independent constructs and unique effects on behavior. Journal of Personality and Social Psychology, 91, Campbell, E. Q., & Pettigrew, T. F. (1959). Christians in racial crisis: A study of Little Rock's ministry. Washington, DC: Public Affairs Press. Cohen, J. (1992). Quantitative methods in psychology: A power primer. Psychological Bulletin, 112(1), Cohen, J. & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences, 2 nd ed. Hillsdale, NJ: LEA Publishers. Dunton, B., & Fazio, R. H. (1997). An individual difference measure of motivation to control prejudiced reactions. Personality and Social Psychology Bulletin, 23, Cascio, W. F., & Aguinis, H. (2011). Applied psychology in human resource management (7th Edition). Upper Saddle River, NJ: Prentice Hall. Dovidio, J. F., Kawakami, K., Johnson, C., Johnson, B., & Howard, A. (1997). On the nature of prejudice: Automatic and controlled processes. Journal of Experimental Social Psychology, 33, Eberhard, J. (2007, August). Examining racial bias in the context of policing. Paper presented at the annual meeting of the American Psychological Association, San Francisco, CA. Fazio, R. & Olsen, M. (2003). Implicit measures in social cognition research. Annual Review of Psychology, 54,

16 16 Fortriede, D. (2005). InQsit Test Quiz Survey User Manual, Version 10. Muncie, IN: Ball State University. Greenwald, A. & Banaji, M. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes, Psychologyical Review, 102, Greenwald, A., Banaji, M., Rudman, L., Farnham, S., Nosek, B, & Mellot, D. (2002). A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept. Psychological Review, 109(1), Greenwald, A., McGhee, D. & Schwartz, J. (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, Greenwald, A, Poehlman, T., Uhlmann, E. & Banaji, M. (2009). Understanding and Using the Implicit Association Test: III. Meta-Analysis of Predictive Validity. Journal of Personality and Social Psychology, 97(1), Gregg, A. P., Seibt, B., & Banaji, M. R. (2006). Easier done than undone: Asymmetry in the malleability of implicit preferences. Journal of Personality and Social Psychology, 90, Joy-Gaba, J. & Nosek, B. (2010). The surprisingly limited malleability of implicit racial evaluations. Social Psychology, 41(3), Lane, A.M., Soos, I., Leibinger, E., Karsai, I. and Hamar, P. (2007) Validity of the Brunel Mood Scale for use with UK, Italian and Hungarian Athletes. In: Mood and human performance: Conceptual, measurement, and applied issues. Ed: Lane, A.M. Hauppauge, NY: Nova Science

17 17 McConahay, J. B. (1986). Modern racism, ambivelance, and the modern racism scale. In J. F. Dovidio & S. L. Gaertner (Eds.), Prejudice, discrimination and racism(pp ). New York: Academic. Nevid, J. S., & Rathus, S. A. (2010). Psychology and the challenges of life: Adjustment and growth (11th ed.). New York: Wiley. Nosek, B. A., Smyth, F. L., Hansen, J. J., Devos, T., Lindner, N. M., Ranganath, K. A., Smith, C. T., Olson, K. R., Chugh, D., Greenwald, A. G., & Banaji, M. R. (2007). Pervasiveness and correlates of implicit attitudes and stereotypes. European Review of Social Psychology, 18, Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002a). Harvesting implicit group attitudes and beliefs from a demonstration website. Group Dynamics, 6, Nosek, B. A., Smyth, F. L., Hansen, J. J., Devos, T., Lindner, N. M., Ranganath, K. A., Smith, C. T., Olson, K. R., Chugh, D., Greenwald, A. G., & Banaji, M. R. (2007). Pervasiveness and correlates of implicit attitudes and stereotypes. European Review of Social Psychology, 18, Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002a). Harvesting implicit group attitudes and beliefs from a demonstration website. Group Dynamics, 6, Paluk, E. L. (2006). Diversity training and intergroup contact: A call to action research. Journal of Social Issues, 62(3),

18 18 Perugini, M. (2005). Predictive models of implicit and explicit attitudes. British Journal of Social Psychology, 44, Pettigrew, T. F., & Martin, J. (1987). Shaping the organizational context for Black American inclusion. Journal of Social Issues, 43(1), Pettigrew, T. F. (1979). The ultimate attribution error: Extending Allport's cognitive analysis of prejudice. Personality and Social Psychology Bulletin, 5(4), Pettigrew, T. F. (2011). Toward sustainable psychological interventions for change. Peace and Conflict, 17(2), Watson, D. L. & Tharp, R. G. (2002). Self-directed behavior: self-modification for personal adjustment. Belmont, CA: Wadsworth. Wilson, D. C. & Davis, D. W. (2008) The measurement of racial resentment: Conceptualization and content. Paper presentated at the annual meeting of the American Political Science Association, Boston, MA. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology,9 (Monograph Supplement 2, Pt. 2). Ziegert, J. C., & Hanges, P. J. (2005). Employment discrimination: The role of implicit attitudes, motivation, and a climate for racial bias. Journal of Applied Psychology, 90(3),

19 19 Table 1. Concept Labels (in italics) and Word Stimuli for the Implicit Association Tests Race Evaluation Stereotype Black White Positive Negative Mental Strength Physical Ability 6 faces 6 faces joy terrible math athletic glorious nasty educated run wonderful evil scientist boxing love hurt smart dance happy horrible college jump laughter failure read rhythmic pleasure awful peace agony

20 20 Table 2. Descriptive Statistics for Study Variables. Variables N Mean SD Alpha Pretest Measures 1 Motivation to Control Racial Prejudice IAT-Preference IAT-Stereotype Posttest Measures 1 Motivation to Control Racial Prejudice IAT-Preference IAT-Stereotype All implicit measures are IAT effects expressed as D measures (Greenwald et al., 2003). 2 Larger negative values indicate a stronger association of Black+negative (and White+positive). 3 Larger negative values indicate a stronger association of Black+physical (and White+mental).

21 21 Table 4. Group Means for Study Variables Used in the Analysis of Variance. Course-Sec 1 N 2 RP 3 MC 3 IAT-P 3 IAT-S 3 RP 4 MC 4 IAT-P 4 IAT-S 4 PSY PSY PSY PSY PSY101 = adjustment course; PSY411 = diversity course 2 Non-Black students with complete data 3 Pretest measures 4 Posttest measures

22 22 Table 3. Zero-order Correlations for Study Variables Variables Pretest Measures 1. Motivation to Control - 2. Racial Prejudice -.53** - 3. IAT-Preference IAT-Stereotype ** - Posttest Measures 5. Motivation to Control.69** -.37* Racial Prejudice -.51**.70** ** - 7. IAT-Preference **.72** IAT-Stereotype **.39** *.57** - *p <.05; **p <.01 N = 44 (all non-black students with complete data)