An Exploratory Factor Analysis of the Global Perspective Inventory. Kelli Samonte and Dena Pastor. James Madison University

Transcription

1 Running Head: AN EXPLORATORY FACTOR ANALYSIS OF THE GLOBAL PERSPECTIVE INVENTORY 1 An Exploratory Factor Analysis of the Global Perspective Inventory Kelli Samonte and Dena Pastor James Madison University Presented at the 23 rd Annual Convention of the Association for Psychological Sciences. Correspondence regarding this paper should be addressed to Kelli M. Samonte, Center for Assessment and Research Studies MSC 6806, James Madison University, Harrisonburg, VA, samontkm@dukes.jmu.edu

2 An Exploratory Factor Analysis of the Global Perspective Inventory 2 An Exploratory Factor Analysis of the Global Perspective Inventory The importance of global perspective-taking has grown considerably over the years and continues to grow as technology continues to bring people from opposite sides of the world together. King and Baxter Magolda (2005) note that colleges are working to develop students global perspective-taking for several reasons. One of these reasons is the fact that corporations, especially ones that work internationally, value this skill. Also, there is still an unnerving amount of reports of racially-motivated crimes on many college campuses. With the increasing importance of global perspective-taking comes an increased need for a way of measuring this skill. The Global Perspective Inventory (GPI) was developed by Braskamp, Braskamp and Merrill (2010) for this purpose. The authors claim that the inventory is a holistic approach to the measurement of global perspective-taking because, unlike other scales, it takes into account intercultural maturity and intercultural communication (Braskamp, Braskamp & Merrill, 2010), which are described below. Intercultural maturity, as defined by King and Baxter Magolda (2005), is the process in which one becomes capable of functioning appropriately in culturally different atmospheres. King and Baxter Magolda provide a framework in which they include three domains that make up the concept of intercultural maturity. These three domains are the cognitive, intrapersonal and interpersonal domains. The cognitive domain is concerned with how people understand and conceptualize issues dealing with diversity. The intrapersonal domain focuses on self-concept and the way people see themselves and the interpersonal domain is concerned with how one interacts with people from other cultures.

3 An Exploratory Factor Analysis of the Global Perspective Inventory 3 Intercultural communication competence is discussed by Chen and Starosta (1996) as the way people are able to effectively understand and communicate in diverse situations. Chen and Starosta, like King and Baxter Magolda, proposed a framework comprised of three processes that make up intercultural communication: the affective process, the cognitive process and the behavioral process. These three processes seem to relate closely with the domains proposed by King and Baxter Magolda. The affective process focuses on emotions and/or things that can influence changes in emotions. The cognitive process is concerned with how one conceptualizes their own and others cultures and the behavioral process focuses on interactions with diverse individuals. The three domains of the intercultural maturity perspective overlap considerably with the three domains of the intercultural communication perspective and provide the framework for the GPI. The authors employ the domain names used by King and Baxter-Magolda and thus the three domains that make up the GPI are the cognitive, intrapersonal and interpersonal. Each domain captures aspects from both perspectives, creating six subdomains (see Table 1). The intercultural maturity aspect of the cognitive domain is known as the cognitive knowing subdomain while the intercultural communication component is known as the cognitive knowledge subdomain. In the intrapersonal domain, the intercultural maturity component is known as intrapersonal identity and the intercultural communication aspect is known as intrapersonal affect. The intercultural maturity and intercultural communication subdomains for the interpersonal domain are interpersonal social responsibility and interpersonal social interactions, respectively. The GPI provides scores for all six of these subdomains. Creation of the inventory was also based on how we answer three questions: How do I know? Who am I? and How do I relate to others? Though these are questions we continually ask

4 An Exploratory Factor Analysis of the Global Perspective Inventory 4 ourselves throughout our lifetime, the authors do note that these are especially relevant to individuals between the ages of 18 and 24. Although these questions might be particularly relevant to college-aged persons, the inventory was designed to measure levels global perspective-taking across all age groups. Because college students have several intercultural experiences and opportunities, much of the work that has been done on this inventory has utilized samples of college students. For example, Doyle (2009) used the GPI as a pre- and posttest to examine a study abroad program for college students. Engberg used college students in one study to examine the relationship between global perspective-taking and undergraduate student learning (Engberg & Fox, 2011) and in another study to examine the relationship between global perspective-taking and diversity engagement (Engberg, 2011). According to the GPI manual, the GPI has been administered to over 18,000 students from over 54 institutions. Although the exact reasons why the GPI was administered in these institutions are unknown, it is reasonable to believe that one purpose involved the use of the GPI scores to make inferences about the global perspective-taking of college students. However, in order to trust that the scores indeed reflect global perspective taking, validity evidence is needed. Although there are a variety of different kinds of validity evidence that should be acquired for an instrument such as the GPI, we focus our attention here on the structural validity evidence pertaining to the GPI scores. As described in the Standard for Educational and Psychological Testing, structural validity studies are used to understand if the pieces of an instrument, such as the items or subscales, are operating in ways anticipated by theory (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education [AERA, APA, NCME], 1999). For instance, a structural validity study might examine

5 An Exploratory Factor Analysis of the Global Perspective Inventory 5 the correlations among subscales with supporting structural validity evidence obtained if the subscales are found to relate to one another in expected ways. Similarly, the correlations among items can be examined. During scale development, all items might be written to tap into a single construct if the construct is considered to be unidimensional. With unidimensional scales all items should have similar correlations with one another because all items are indicators of the same construct. Conversely, sets of items might be created for various subscales if the construct being measured is considered to be multidimensional. In this situation, items associated with the same subscale should relate more strongly to one another than items on different subscales, particularly if the subscales are not expected to be highly related to one another. Factor analysis is a common tool use in structural validity studies to explore if the pattern of relationships among items conforms to the conceptualization of the construct (Benson, 1998). There are two different kinds of factor analytic techniques that can be employed in a structural validity study: confirmatory and exploratory. In a confirmatory factor analytic study the researcher specifies the factor model, meaning that they dictate how many factors underlie the responses to a set of items, whether factors are correlated with one another, and which items correspond to which factor. Fit indices are then used to gauge the extent to which the observed correlations among items correspond to the correlations that would be expected under the proposed factor model. Although finding that one s proposed model fits the data is encouraging, stronger structural validity evidence is obtained by also showing that other possible factor models do not fit the data. Thus, when using confirmatory factor analytic techniques, researchers fit their favored model to the data as well as other plausible factor models. Exploratory factor analysis differs in that no factor model is specified a priori. Instead, all items are allowed to correlate with each factor and the number of factors is permitted to

6 An Exploratory Factor Analysis of the Global Perspective Inventory 6 emerge from the data. The decision as to whether to use a confirmatory factor analysis (CFA) or an exploratory factor analysis (EFA) depends on several things. First, the use of CFA necessitates a strong conceptualization of the construct in order to specify a variety of different factor models. Second, to increase one s chances of achieving adequate fit in CFA the scale in question needs to have gone through several rounds of development so that only high quality items remain with characteristics likely to conform to the factor model. CFA provides a rigorous means by which to test one s factor model, however, if the conceptualization of the construct is not yet solidified or if the quality of items written to measure the construct has not yet been thoroughly scrutinized, EFA is the preferred methodology (Fabrigar, Wegener, MacCallum, & Strahan, 1999). The various EFA solutions (e.g., 1-factor, 2- factor, etc.) can be used to inform the conceptualization of a construct and also to identify problematic items, which includes those items that do not load on any factor or items that crossload, meaning they relate to multiple factors. The number of factors that emerge from EFA and the nature of those factors can also be used to inform one s understanding of the construct. To date, there has been little factor analytic work, confirmatory or exploratory, done on the GPI. The only known factor analyses of the GPI were conducted by the Engberg and Fox (2011) and the results of those analyses are reported in the GPI manual. This analysis was performed using a large sample from a variety of different institutions. The particular exploratory analysis employed by the authors was a principal components analysis with varimax rotation, which is an inappropriate method for examining factor structure, particularly when there is no reason to believe that the factors are uncorrelated (Fabrigar, Wegener, MacCallum, & Strahan, 1999). Another limitation of the approach is that the GPI scale as a whole was not analyzed; instead separate PCAs were conducted on each of the six subscales. Problems with the

7 An Exploratory Factor Analysis of the Global Perspective Inventory 7 methodology set aside, very little information is provided about the analysis. It is unclear what criteria (e.g., scree plot, interpretability) the authors used to arrive at single-component solutions for each subscale that according to Braskamp et al. (2010), confirms the contention that the GPI provides a reasonable structure of the three domains (pg.12). Although it is encouraging that factor analytic work has been conducted on the GPI, the results provided by the PCAs are based on questionable methodology and provide very limited information as to how the scale as a whole is operating. To our knowledge, no other factor analytic studies have been pursued except for those alluded to by the instrument s creators. Because further structural validity evidence is needed for the GPI, the purpose of our study was to conduct exploratory factor analysis using responses to the collected from 963 undergraduates at a mid-sized East Coast university. A wide variety of factor models are possible for the GPI, including one in which each subscale serves as its own factor and another in which factors are formed according to the cognitive, intrapersonal, and interpersonal subdomains. Given the existence of multiple plausible factor models, the use of CFA might be considered preferable to EFA. However, given the little factor analytic work that has conducted thus far on the GPI, it is likely that none of the many models would yield adequate fit to the data. For this reason, EFA was used in the current study not only to identify problematic items, but also to explore if other plausible factor models are appropriate for the data. Methods Participants and Procedure

8 An Exploratory Factor Analysis of the Global Perspective Inventory 8 The sample was comprised of 962 undergraduates from a mid-sized, southeastern university, who were required to take part in a semi-annual university-wide Assessment Day in August All incoming freshmen were required to participate in Assessment Day the week before classes begin. Students who do not attend Assessment Day were required to make up this day to avoid a hold being placed on their student account. It is made clear to the students that the purpose of this day is to aid the university in examining learning and development within the early stages of their college careers. Students were randomly assigned to classrooms based on the last two digits of their student identification number and a different battery of instruments was given in each classroom. Each student completed about five instruments totaling to about three hours of testing. Trained proctors reminded students of the purpose of the assessment, obtained informed consent from the students and then proceeded to administer the battery of tests assigned to that specific room. The random assignment of students to rooms where different tests were being completed ensured that the 962 students completing the GPI is representative of the population of incoming freshmen at the university in The resulting sample was 77.3% White and 62.4% female with an average age of (SD =.37). Data Screening Students were asked to respond to the items of the GPI using a 1 to 5 Likert scale and thus responses outside of this range were recoded as missing. Cases with missing data were removed from the data set using listwise deletion. The sample was screened for multivariate outliers using Mahalanobis distance estimates that were produced using DeCarlo s (1997) macro. Those cases with large Mahalanobis distance estimates relative to the rest of the sample were removed from the data. Because normality is an assumption of maximum likelihood estimation, the estimation method we employed for our EFAs, descriptive statistics were obtained for each

9 An Exploratory Factor Analysis of the Global Perspective Inventory 9 item of the GPI. These values were examined to determine the normality of the item responses with acceptable skewness and kurtosis values being set at less than 2 and 7, respectively (Kline, 2005). Because excessive relationships among items can be problematic in factor analysis, the multicollinearity among items was inspected using several different approaches. First, the inter-item correlations were inspected with values > 0.85 indicating problematic items. Second, tolerance values, which represent the proportion of variance for an item not shared with the remaining items, were examined and values < 0.10 were used to flag possibly redundant items. Exploratory Factor Analysis Procedure A maximum likelihood exploratory factor analysis with oblique rotation (specifically, promax rotation) was used. Maximum likelihood estimation was used because it provides fit indices that can aid in comparing the different factor solutions. The possibility of factors in this instrument being correlated led to the use of oblique rotation because it allows the different factors to correlate with each other, unlike orthogonal rotation in which correlations between factors are forced to be zero. Mplus, version 6.1 was used to estimate all models. The examination of two fit indices, the root mean square error of approximation (RMSEA) and the standardized root mean square residual (SRMR), were used to assess how well the factor solutions fit the data. The recommendations made by Hu and Bentler (1998; 1999) of using RMSEA values at or below.06 and SRMR values at or below.08 were used to determine model fit. In order to decide how many factors to retain, a parallel analysis (Horn, 1965) was conducted by first generating random data with similar characteristics to of the original data set.

10 An Exploratory Factor Analysis of the Global Perspective Inventory 10 An eigenvalue decomposition of the unreduced correlation matrix was then created and plotted against the scree plot obtained for the original sample. The factors that occurred before the intersection of the randomly generated data were favored in the decision of how many factors to retain. Plausible solutions identified by the fit indices and parallel analyses were further examined in order to identify the most interpretable, parsimonious solution. Pattern coefficients with an absolute value greater than.4 were used to designate an item as loading onto a particular factor. Results Data Screening Deletion of cases with missing data on all items resulted in the removal of 9 cases. When multivariate outliers were examined using Mahalanobis distance, three multivariate outliers were flagged and removed from the data set, resulting in a final sample size of 950. The examination of univariate normality using skewness and kurtosis values indicated that the assumption of univariate normality was not violated. Inter-item correlations did not exceed values of.85 and tolerance values were all less than.10, indicating that there were no issues with multicollinearity within our data. Exploratory Factor Analysis The results of the scree plot indicated that both a one and four factor solution were plausible for the GPI (see Figure 1). The parallel analysis, which shows which factors emerge beyond chance, demonstrated that a three factor model was also a possibility. Because the scree plot and parallel analysis indicated that a 1-, 3- and 4-factor solution were plausible for the data, solutions consisting of one to four factors were examined. We also were interested in the

11 An Exploratory Factor Analysis of the Global Perspective Inventory 11 interpretability of the 6-factor solution given that the instrument was created to consist of 6 subscales. Thus, the one to six factor models were all examined to determine how factor structure changed as more factors were allowed to emerge. Of the six solutions examined, the three factor model was the most interpretable solution with the least amount of problems (e.g. split loadings, factors with two or less items, etc). The RMSEA and SRMR values (see Table 2) for the three factor model were.052 and.047, respectively, indicating that the model provided adequate fit to the data, In the three factor model, 18 items did not load on any factor whereas the remaining 22 loaded explicitly on one of the three factors. Specifically, 8 items loaded onto factor one, 9 items loaded onto factor two, and 5 items loaded onto factor three as shown in Table 3. Factor one items focus on enjoyment, and acceptance of other cultures. This factor also consists of items pertaining to sensitivity towards those who are discriminated against, taking into account difference perspectives before drawing conclusions and behaving in terms of making a difference. Factor two items are concerned with one s knowledge and understanding of other cultural traditions, perspectives and current events and issues. Several of the items loading on factor two measure the extent to which one is informed of, and can function in another culture. Factor three encompasses items pertaining to one s identity. The items loading on factor three appear to measure how well defined one s identity is as well as how willing one is to defend or stand up for their beliefs. The correlations among the factors ranged from.31 to.35 indicating that the three factors were related, yet quite distinct from one another. Discussion Due to the overall lack of structural validity evidence using appropriate methodology that has been collected to date on the GPI, exploratory factor analytic techniques were used in this study to examine the structural validity of the instrument. Several EFA solutions were examined

12 An Exploratory Factor Analysis of the Global Perspective Inventory 12 based on the results of the scree plot and parallel analysis as well as the authors conceptualization of the scale. Specifically, the 1- to 6-factor solutions were examined using interpretability as the main basis for choosing among these models. Of these six solutions, the three-factor model was considered the most interpretable. Fit indices, which were obtained to examine how well each model fit the data, indicated adequate fit for the three-factor model. There are several reasons why the three-factor model was chosen over the other solutions. First, of those items that had pattern coefficients >.40, none loaded strongly on more than one factor. Second and most importantly, the content of many of the items associated with the same factor were similar. Although the three factor solution has some desirable properties, the large number of items (i.e., 18) that failed to load on any factor indicates that even the most interpretable factor solution for this instrument is far from ideal. Implications for the GPI For our results to provide supporting structural validity evidence for the GPI, the factors emerging from the analyses would have needed to align with either the two components (e.g., intercultural maturity, intercultural communication), three domains (e.g., cognitive, intrapersonal, interpersonal) or six subdomains (e.g., cognitive knowing, intrapersonal affect, social responsibility) central to the GPI authors conceptualization of the global perspective taking construct. Unfortunately, none of the factor models investigated in the current study, including the most interpretable three-factor solution, had a strong alignment with this conceptualization. Most notably, our favored solution has little correspondence with the six subdomains that are used in scoring the instrument.

13 An Exploratory Factor Analysis of the Global Perspective Inventory 13 There are several possible reasons why we failed to find supporting structural validity evidence for the GPI. One possibility is that the conceptualization of the construct proposed by the authors is not how the construct is truly structured. If this is the case, then the next step that would need to be taken is a reevaluation of global-perspective taking. Another possibility is that the conceptualization is correct, but that the items written by the authors do not fully or adequately represent the construct. If this is the issue, then the next step would consist of revising old items and adding additional items. It is our opinion that it would be worthwhile to revisit both the conceptualization of the construct and the GPI items. Although theory and previous research should heavily guide this process, the results from the current study can also be used. Future Revisions to the Global-Perspective Taking Construct and Instrument One way in which the current study s results could be used is to inform future revisions of the global-perspective taking construct and instrument is by examining if and how the current six subscales are represented in the three-factor solution. In the section that follows, we describe how each subscale is represented in the EFA results and what these results suggest about the instrument and the construct. Our suggestions are depicted in Table 4 and summarized at the end of this section. We organize this discussion by first describing what we consider to be the least problematic subscales and ending with the most problematic subscales. Cognitive Knowledge (CKe). The results indicate that the least problematic subscale might be the Cognitive Knowledge (CKe) subscale, where all subscale items loaded on Factor 2. This indicates that the items are related to one another, providing some justification that they can be combined to create a subscale score. Of course, items 15, 23, 17, 21 from three other subscales also loaded on Factor 2 and therefore contribute to the meaning of the factor. Items 17

14 An Exploratory Factor Analysis of the Global Perspective Inventory 14 and 23 appear similar in content to the other Cognitive Knowledge (CKe) subscale and thus, might be considered for inclusion on that subscale. Items 15 and 21, although similar in content to one another, appear to be tapping into something different from Cognitive Knowledge, making it harder to justify their inclusion on this subscale. Items 15 and 21 from the Interpersonal Social Interactions (ISI) subscale both load on Factor 2. As argued above, these items seem distinct in content from many of the other items associated with Factor 2. Items 15 and 21 both pertain to one s ability to function in diverse settings, which may very well be a skill associated with one s Cognitive Knowledge (CKe). Future revisions of the GPI might consider creating more items for this aspect of globalperspective taking and treating it as a separate subscale that is related to Cognitive Knowledge (CKe). Intrapersonal Identity (II). Three of the five Intrapersonal Identity (II) items loaded on Factor 3 while the remaining two items did not load on a factor. The three items (i.e., 2, 3, 9) that load on Factor 3 all appear to tap into self-awareness. An item similar in content, but with a pattern coefficient just shy of.40 is item 33, which might also be considered as aligning with Factor 3. The remaining Intrapersonal Identity (II) item, item 14, appears to differ in content from the other Intrapersonal Identity (II) items. Item 14 measures one s confidence in new situations as opposed to one s awareness of or comfort with their own identity. Therefore, there is some empirical and logical justification for including all Intrapersonal Identity (II) items on a subscale, with the exception of item 14. There were two other items (e.g., 22, 12) from different subscales that also loaded on Factor 3. Both items are measuring one s willingness to stand up for their beliefs and values. The fact that these two items aligned with the three Intrapersonal Identity (II) items on the factor indicates that awareness of one s identity is related to standing up

15 An Exploratory Factor Analysis of the Global Perspective Inventory 15 for one s beliefs. Of course, whether these two different types of items, however, should be included in the same subscale as the Intrapersonal Identity (II) items requires theoretical justification. Intrapersonal Affect (IA) and Interpersonal Social Interactions (ISI). Both subscales are discussed here because many of their items loaded on Factor 1. Items 29 (IA), 39 (ISI), 37 (ISI), 28 (IA), 34 (ISI) and 26 (IA) all measure acceptance of differences and one s preference for intercultural relationships. The alignment in the content of these items as well as their relationships with the same factor provides some justification that these items could be combined to form a subscale, perhaps named the Interpersonal Affect and Social Interactions subscale. Of course, the merging of these Intrapersonal Affect (IA) and Interpersonal Social Interactions (ISI) items calls into question the theoretical distinctiveness of these two aspects of global-perspective taking. The remaining items on Factor 1 from other subscales (i.e., 18, 38) appear different in content, making it hard to justify their inclusion on a new, combined subscale. There were several Intrapersonal Affect (IA) and Interpersonal Social Interactions (ISI) items that do not load on any factor. The only Interpersonal Social Interactions (ISI) item in this category is item 4, which asks whether the majority of the respondent s friends are of their own ethnicity. There are two possibilities as to why this item does not relate to the other IA and ISI items. First, a respondent might enjoy interacting with persons of different ethnic backgrounds, but has to respond disagree because either they don t have a lot of friends, or their opportunities for having friends of different ethnicities is limited. Another reason why this item might not relate to the other Factor 1 items is because it is negatively worded. The negative wording of an item can affect responses in such a way as to diminish the item s correlations with other, positively worded items measuring the same thing (Marsh, 1996). For this reason, it is

16 An Exploratory Factor Analysis of the Global Perspective Inventory 16 suggested that the items be revised with one possible revision being I enjoy interacting with persons of ethnicities different than my own. The Intrapersonal Affect (IA) items that did not load on any factor include items 10, 11, 20, 27, and 36. From these items, item 10 and 27 might be revised and considered for retention along with the Intrapersonal Affect (IA) and Interpersonal Social Interactions (ISI) items that loaded on Factor 1. This suggestion is based on the fact that items 10 and 27 are similar to these items in that they also measure acceptance of differences. As with item 4, these items might not be loading on Factor 1 because they are negatively worded. Given the similarity between items 10 and 27, it is suggested that only item 10 be retained and reworded positively on a future versions of the scale. It could also be argued that the reason why Intrapersonal Affect (IA) items 20 and 36 failed to load on any factor is because of their negative wording. Given the content of these items, however, we feel that their failure to load on a factor is more likely due to their strong wording, ambiguity, and unique content. These are the same reasons item 11, an IA item that also did not load on a factor, may not be functioning well. For these reasons, we suggest that items 11, 20 and 36 be considered for revision or deletion. Interpersonal Social Responsibility (ISR). When just considering the three-factor solution, the most problematic subscale on the GPI appears to be the Interpersonal Social Responsibility (ISR) subscale. Two ISR items loaded on different factors and the remaining four items did not load on a factor. As argued above, item 22, which is one of the two ISR items that loaded on a factor, more naturally aligns with item 12 from the CKg subscale and measures standing up for one s beliefs as opposed to making socially responsible commitments. The other item that loaded on a factor, item 38, seems distinct in content from the other items associated

17 An Exploratory Factor Analysis of the Global Perspective Inventory 17 with its factor and more aligned with the other four ISR items (i.e., 5, 16, 31, 40) that did not load on a factor. These items (e.g., 38, 5, 16, 31, 40) appear to measure one s desire to make a difference in society. Information as to why these seemingly related items did not load on the same factor can be gleaned from looking at the other factor solutions. All five of these items loaded onto the first factor (along with items 29, 39, 37, 28, 34, 26 and 18) in the 4- and 5-factor solutions, but did not load on the first factor in the 3-factor solution. When our results are expanded and more than just the three-factor solution is considered, our EFA results do indicate that these social responsibility items are related to one another and that they are also related items measuring acceptance of differences and the preference for intercultural relationships. We suggest that these items be retained in future revisions of the scale and their strong relationships with items measuring enjoyment of diverse others noted. Cognitive Knowing (CKg). Our results indicate that one of the more problematic subscales on the GPI is the Cognitive Knowing (CKg) subscale. The fact that three of the nine Cognitive Knowing items each loaded on a different factor in our three factor solution and the remaining six failed to load on any factor calls into question what the sum of these nine items really represents. The fact that six items did not load on any factor indicates that these items have little if any of a relationship with the remaining items on the GPI. A possibility for why these items are not functioning as intended has to do with their wording. Most of these items are so strongly worded that respondents might not answer truthfully. For instance, item 6 which states Some people have culture where others do not is such a strong statement that a respondent might be inclined to disagree with the statement, even if they believe otherwise. Another reason these items might be problematic is because many of them are negatively worded. For these reasons, it is suggested that these items be considered for revision or deletion.

18 An Exploratory Factor Analysis of the Global Perspective Inventory 18 Summary. Our EFA results suggest changes that could be made to many of the GPI subscales, problems with a few of the subscales, and characteristics of the construct that need to be clarified. The Cognitive Knowledge (CKe) and Intrapersonal Identity (II) subscales appear to be functioning fairly well, although it was suggested that one item (14) be dropped from the II subscale. There were a couple of items from different subscales that loaded on the factors with which the Cognitive Knowledge (CKe) and Intrapersonal Identity (II) items were affiliated, but because these items differed in content, it was suggested that theory and previous research be consulted to determine how these items should be handled in future revisions of the instrument. In other words, clarification as to what role one s ability to function in diverse settings" (items 15, 21) and one s willingness to stand up for their beliefs and values" (items 22, 12) play in global-perspective taking is needed. The alignment in the content of many of the Interpersonal Affect (IA) and Interpersonal Social Interactions (ISI) subscale items (items 29, 39, 37, 26, 28, 34) and the fact that these items aligned with the same factor begs the theoretical question of whether Interpersonal Affect (IA) and Interpersonal Social Interactions are distinct enough to be considered separate subdomains. If after consulting theory and literature it is concluded that they are not distinct, these items might be combined to form a single subscale, along with items 4 and 10 after revision. At first glance, the Interpersonal Social Responsibility (ISR) subscale appeared to be the worst functioning. However, more complex factor solutions indicated that the ISR items did align with the same factor, similar to Factor 1 in the three-factor solution. We believe the ISR items are worthy of further scrutiny and encourage researchers to keep in mind the relationships between one acceptance of differences, preference for intercultural relationships, and engagement in socially responsible commitments implied by our results.

19 An Exploratory Factor Analysis of the Global Perspective Inventory 19 There were many items, particularly from the Cognitive Knowing (CKg) and Interpersonal Affect (IA) subscale that we suggest revising or deleting from the scale. Many of these items are negatively worded, so revising these items such that they are positively worded might improve their functioning. However, we also noted that some of these items might be problematic because they are strongly worded, ambiguous, or measuring unique content. These potential problems should be kept in mind when considering these items in future scale revisions. Limitations There were several limitations of the current study. First, it should be acknowledged that we relied heavily on the authors six subscale conceptualization of the global-perspective taking construct, the empirical results from our EFA and our personal judgments the content alignment among items to make recommendations. Any action taken based on our recommendations should be done in unison with a careful consideration of theory and previous research as it pertains to global-perspective taking. We also did not consider whether the six subdomains in the original conceptualization of global-perspective taking are those that are needed to best represent the construct. We encourage future researchers to critically evaluate whether there are any sources of construct irrelevant variance operating in the GPI (i.e., subscale or items that don t belong). We also encourage researchers to consider whether the construct is underrepresented, meaning that there are aspects of global-perspective taking not being captured by the instrument. The second limitation was the sample employed for our analyses. Our sample consisted of predominantly Caucasian incoming freshman and it is possible that our results would have been different had we used a sample of more culturally and academically diverse individuals. Another limitation could have been the motivation levels of the students responding to the

20 An Exploratory Factor Analysis of the Global Perspective Inventory 20 instrument. On Assessment Day, students are given a battery of low stakes instruments with testing time lasting about 3 hours. It is possible that questionable motivation levels or fatigue adversely affected responses to the GPI. Confidence in the results found in this study would be increased if similar findings emerged in other EFA studies of the GPI, particularly those using more diverse samples with respondents completed the instrument under more ideal conditions. Future Directions It is our hope that the recommendations we have provided will be used along with theoretical considerations to redefine the construct and revise the scale. Incredibly useful information about how to revise those items we considered to be problematic could be obtained by conducting think alouds (van Someren, Banard, & Sandberg, 1994). The use of think-alouds may clarify whether respondents are interpreting the items in the manner intended by having respondents voice their thoughts while responding to the items. Useful information to supplement our recommendations could also be obtained by asking content experts to categorize the items into the various subdomains (Haynes, Richard, & Kubany, 1995). Items and subdomains might be identified as problematic if content experts align items with subdomains in a different manner than the instrument s creators. Content experts can also be used to critically evaluate the components of the GPI, providing their opinion as to whether there are some existing subdomains or items that don t belong or non-existing subdomains or items that need to be included. Of course, if any revisions are made to the GPI, it is imperative that new reliability and validity evidence be acquired to support the inferences being made from the new GPI scores. Even if our recommendations are not acted upon, we do hope that our results inform and encourage future structural validity studies of the GPI. One possible twist to the current study

21 An Exploratory Factor Analysis of the Global Perspective Inventory 21 would be use EFA techniques appropriate for categorical, as opposed to continuous, responses. We treated the 5-point Likert scale response as continuous, when it may have been more appropriate to consider the response scale as an ordinal, categorical scale (Finney & DiStefano, 2006). It would be of interest to know the extent to which the results differ when the responses are treated as categorical. Another direction for future research would be to pursue confirmatory factor analyses with the GPI. We chose to use EFA given that both the GPI and the globalperspective taking construct are in their early stages of development. Although we anticipated that CFA models would not fit the data for these reasons, it would be worthwhile to test this assumption empirically. There are a wide variety of CFA models that could be considered for the GPI, either using the original scale or a revised version of the scale based on this study s recommendations. These models include a 6-factor model, a higher-order model, a bi-factor model, as well as models generated from this study s results. The use of CFA techniques would also allow a more rigorous investigation into affects the negatively worded items (Marsh, 1996). Of course, the use of confirmatory approaches to examine the internal structure of the GPI is only warranted if the internal structure of the current or revised versions of the GPI is considered worthy of further examination. Conclusion Ultimately, our three-factor solution does not align strongly with the six subdomain structure proposed by the GPI creators. Thus our study does not provide supportive structural validity evidence for the GPI. If the recommendations made herein along with theory and previous research are used to revise both the GPI construct and scale, supportive structural validity may very well be possible. Before such revisions are made, it is important to proceed

22 An Exploratory Factor Analysis of the Global Perspective Inventory 22 with caution when using the GPI because an instrument with questionable structural validity can lead to inappropriate and unsubstantiated inferences.

23 An Exploratory Factor Analysis of the Global Perspective Inventory 23 References American Educational Research Association, American Psychological Association and National Council on Measurement in Education (1999) The Standards for Educational and Psychological Testing. Washington, DC: American Psychological Association. Benson, J. (1998) Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issues and Practice, 17, Braskamp, L. A., Braskamp, D. C., & Merrill K. C. (June 2010) Global Perspectives Inventory (GPI): Its purpose, construction, potential uses, and psychometric characteristics. Retrieved from Chen, G. & Starosta, W. J. (1996) Intercultural communication competence: A synthesis. Communication Yearbook, 19, Doyle, D. (2009) Holistic assessment and the study abroad experience. Frontiers: The Interdisciplinary Journal of Study Abroad, 18, Engberg, M. E. (2011) Examining the linkages between diversity engagement and global perspective-taking. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA. Engberg, M. E. & Fox, K. (2011) Exploring the relationship between undergraduate studentlearning experiences and global perspective-taking. Journal of Student Affairs Research and Practice, 48, Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4,

24 An Exploratory Factor Analysis of the Global Perspective Inventory 24 Finney, S. J. & DiStefano, C. (2006) Non-normal and categorical data in structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course ( ). United States of America: Information Age Publishing. Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995). Content validity in psychological assessments: A functional approach to concepts and methods. Psychological Assessment, 7, Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, Hu, L. & Bentler, P. M. (1998) Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, Hu, L. & Bentler, P. M. (1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, King, P. M. & Baxter Magolda, M. B. (2005) A developmental model of intercultural maturity. Journal of College Student Development, 46, Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling (2 nd Edition). New York: Guilford. Marsh, H. W. (1996). Positive and negative global self-esteem: A substantively meaningful distinction or artifactors. Journal of Personality and Social Psychology, 70, van Someren, M. W., Banard, Y. F., & Sandberg, J. A. C. (1994). The think aloud method: A practical guide to modeling cognitive processes. London: Academic Press.

25 An Exploratory Factor Analysis of the Global Perspective Inventory 25 Table 1 The Six Subdomains of the Global Perspective Inventory Intercultural Maturity Intercultural Communication Cognitive Cognitive Knowing Cognitive Knowledge Intrapersonal Intrapersonal Identity Intrapersonal Affect Interpersonal Social Responsibility Social Interactions Table 2 Fit Indices for the 1- to 6-Factor Solutions Number of Factors SRMR Lower Bound RMSEA Upper Bound

26 An Exploratory Factor Analysis of the Global Perspective Inventory 26 Table 3 Pattern Coefficients for the Three Factor Solution Pattern Coefficients Subscale Factor 1 29.) I am accepting of people with different religious and spiritual traditions. IA ) I am open to people who strive to live lives very different from my own life style. ISI ) I enjoy when my friends from other cultures teach me about our cultural differences. ISI ) I prefer to work with people who have different cultural values from me. IA ) I am sensitive to those who are discriminated against. IA I take into account different perspectives before drawing conclusions about the world around 18.) me. CKg ) I intentionally involve people from many cultural backgrounds in my life. ISI ) I consciously behave in terms of making a difference. ISR Factor 2 32.) I can discuss cultural differences from an informed perspective. Cke ) People from other cultures tell me that I am successful at navigating their cultures. ISI ) I understand how various cultures of this world interact socially. CKe ) I am informed of current issues that impact international relations. CKe ) I know how to analyze the basic characteristics of a culture. CKe ) I understand the reasons and causes of conflict among nations of different cultures. CKe ) I consider different cultural perspectives when evaluating global problems. CKg ) I see myself as a global citizen. IA ) I am able to take on various roles as appropriate in different cultural and ethnic settings. ISI Factor 3 3.) I can explain my personal values to people who are different from me. II ) I have a definite purpose in my life. II ) I know who I am as a person. II ) I put my beliefs into action by standing up for my principles. ISR ) I am willing to defend my own views when they differ from others CKg Items Not Loading on Any Factors 1.) When I notice cultural differences, my culture tends to have the better approach.* CKg ) Most of my friends are from my own ethnic background.* ISI ) I think of my life in terms of giving back to society. ISR ) Some people have a culture and others do not.* CKg ) In different settings what is right and wrong is simple to determine.* CKg ) I feel threatened around people from backgrounds very different from my own.* IA ) I often get out of my comfort zone to better understand myself. IA ) I am confident that I can take care of myself in a completely new situation. II ) I work for the rights of others. ISR ) I get offended often by people who do not understand my point-of-view.* IA ) I rely primarily on authorities to determine what is true in the world.* CKg ) I do not feel threatened emotionally when presented with multiple perspectives. IA ) Cultural differences make me question what is really true. CKg ) I put the needs of others above my own personal wants. ISR ) I am developing a meaningful philosophy of life. II ) I rarely question what I have been taught about the world around me.* CKg ) I constantly need affirmative confirmation about myself from others.* IA ) Volunteering is not an important priority in my life.* ISR

27 An Exploratory Factor Analysis of the Global Perspective Inventory 27 Table 4 Recommendations for Revisions of GPI Construct and Instrument Item Number Item Subscale * If Reverse Factor Scored Number 32 I can discuss cultural differences from an informed perspective. Cke 2 19 I understand how various cultures of this world interact socially. CKe 2 8 I am informed of current issues that impact international relations. CKe 2 25 I know how to analyze the basic characteristics of a culture. CKe 2 13 I understand the reasons and causes of conflict among nations of different cultures. CKe 2 23 I consider different cultural perspectives when evaluating global problems. CKg 2 17 I see myself as a global citizen. IA 2 15 People from other cultures tell me that I am successful at navigating their cultures. ISI 2 21 I am able to take on various roles as appropriate in different cultural and ethnic settings. ISI 2 3 I can explain my personal values to people who are different from me. II 3 2 I have a definite purpose in my life. II 3 9 I know who I am as a person. II 3 33 I am developing a meaningful philosophy of life. II 0 22 I put my beliefs into action by standing up for my principles. ISR 3 12 I am willing to defend my own views when they differ from others CKg 3 29 I am accepting of people with different religious and spiritual traditions. IA 1 39 I am open to people who strive to live lives very different from my own life style. ISI 1 37 I enjoy when my friends from other cultures teach me about our cultural differences. ISI 1 26 I am sensitive to those who are discriminated against. IA 1 28 I prefer to work with people who have different cultural values from me. IA 1 34 I intentionally involve people from many cultural backgrounds in my life. ISI 1 Recommendation These items appear to measure "one s knowledge and understanding of cultural difference" and should be retained as indicators on the Cognitive Knowledge subscale. Consider retaining for Cognitive Knowledge subscale. Consider retaining for Cognitive Knowledge subscale. These two items should be kept together. These items appear to measure "one s ability to function in diverse settings". Consider creating more items for this aspect of global-perspective taking and treating it as a separate subscale, keeping in mind its relationship with the Cognitive Knowledge (CKe) subscale. These items appear to measure "one s awareness of and comfort with their own identity" and should be retained as indicators of Inrapersonal Identity. Consider retaining for Intrapersonal Identity subscale. These two items should be kept together. What role "one s willingness to stand up for their beliefs and values" contributes to global-perspective taking should be carefully considered. These items appear to measure "acceptance of differences and preference for intercultural relationships". If theoretically justifiable, consider retaining to create a Interpersonal Affect and Social Interactions subscale. 4 Most of my friends are from my own ethnic background. ISI * 0 10 I feel threatened around people from backgrounds very different from my own. IA * 0 38 I consciously behave in terms of making a difference. ISR 1 5 I think of my life in terms of giving back to society. ISR 0 16 I work for the rights of others. ISR 0 31 I put the needs of others above my own personal wants. ISR 0 40 Volunteering is not an important priority in my life. ISR * 0 18 I take into account different perspectives before drawing conclusions about the world around me. CKg 1 30 Cultural differences make me question what is really true. CKg 0 1 When I notice cultural differences, my culture tends to have the better approach. CKg * 0 6 Some people have a culture and others do not. CKg * 0 7 In different settings what is right and wrong is simple to determine. CKg * 0 24 I rely primarily on authorities to determine what is true in the world CKg * 0 35 I rarely question what I have been taught about the world around me CKg * 0 36 I constantly need affirmative confirmation about myself from others. IA * 0 20 I get offended often by people who do not understand my point-of-view. IA * 0 11 I often get out of my comfort zone to better understand myself. IA 0 27 I do not feel threatened emotionally when presented with multiple perspectives. IA 0 14 I am confident that I can take care of myself in a completely new situation. II 0 Consider rephrasing as "I enjoy interacting with persons of ethnicities different than my own." and retaining for the new (if justifiable) Interpersonal Affect and Social Interactions subscale. Revise to be positively worded and consider retaining for the new (if justifiable) Interpersonal Affect and Social Interactions subscale. These social resposibility items should be retained in future revisions of the scale and their strong relationships with the Interpersonal Affect and Social Interactions subscale noted. Consider items for revision or deletion. Note. CKg = Cognitive Knowing, CKe = Cognitive Knowledge, II = Intrapersonal Identity, IA = Intrapersonal Affect, ISR = Interpersonal Social Responsibility, ISI = Interpersonal Social Interactions