Taylor University s Intercultural Inventory Analysis Presented by Natalie Nunes August 5, 2009
The Taylor University Intercultural Inventory (TUII) Purpose: TUII measures the intercultural development that occurs in a student due to an intercultural experience Description: Includes demographics section Includes 130 questions Uses a likert scale from strongly disagree (1) to strongly agree (6)
The Taylor University Intercultural Inventory (TUII) Development: Conducted a thorough literature review and an examination of all previous surveys measuring intercultural experiences Found significant categories and variables relating to intercultural development Generated questions within the framework of Personal Development Connections and Cognitive Understanding Levels Created unique questions regarding spiritual development
Intercultural Developmental Scales Targets the following areas called Intercultural Developmental Scales (IDS): Cognitive Understanding Levels Knowledge Awareness Attitudes Behavior Intrapersonal Growth Personal Development Connections Intellectual Interpersonal Spiritual Knowledge Intellectual (11) Awareness Intellectual (6) Attitudes Intellectual (4) Behavior Intellectual (18) Intrapersonal Growth Intellectual (20) Knowledge Interpersonal (3) Awareness Interpersonal (6) Attitudes Interpersonal (7) Behavior Interpersonal (10) Intrapersonal Growth Interpersonal (6) Knowledge Spiritual (10) Spiritual Awareness (4) Attitudes Spiritual (9) Behavior Spiritual (11) Intrapersonal Growth Spiritual (5)
The Taylor University Intercultural Inventory (TUII) Further Development: Conducted a Factor Analysis of each subscale and reduced the number of question per scale, reducing the instrument to 74 questions. Questions within the Cognitive Development Levels were randomly organized. An article is being submitted about the development of the instrument. The inventory will be published and marketed.
Intercultural Developmental Scales Number of questions in the 74 item inventory after factor analysis was conducted. Cognitive Understanding Levels Knowledge Awareness Attitudes Behavior Intrapersonal Growth Personal Development Connections Intellectual Interpersonal Spiritual Knowledge Intellectual (5) Awareness Intellectual (5) Attitudes Intellectual (4) Behavior Intellectual (6) Intrapersonal Growth Intellectual (7) Knowledge Interpersonal (3) Awareness Interpersonal (5) Attitudes Interpersonal (5) Behavior Interpersonal (5) Intrapersonal Growth Interpersonal (5) Knowledge Spiritual (5) Spiritual Awareness (4) Attitudes Spiritual (5) Behavior Spiritual (5) Intrapersonal Growth Spiritual (5)
Reliability of the TUII Strong Reliability One Test Administration Method Cronbach s Alpha coefficient-.933 (N=67) and.965 (N=419) Two Test Administration Method Test retest reliability coefficient-.915 (N=38)
Validity of the TUII Strong Validity Construct validity (Known-Group Difference Method) Discriminates between levels of intercultural experience (none, low, medium, high) Discriminates between pre- and post-test performance (i.e. Irish Studies Program, Lighthouse) Discriminates between gender Content validity Established through the literature review and review of previous surveys
Taylor University s Intercultural Inventory The TUII is a reliable and valid instrument It can be used in several ways: Understand and assess the holistic intercultural development of students due to intercultural experiences Analyze the specific developmental benefits of intercultural programs (i.e. Lighthouse) Compare intercultural programs through collaboration between CCCU schools (i.e. Orvieto)
IDS Analysis Purposes: 1) To discover the degree that students are developing Taylor University s Intercultural Development Scales (IDS) through their intercultural study abroad experiences by comparing their pre and post scores on the Taylor University Intercultural Inventory (TUII). 2) To compare the relative influence of going on a study abroad program with a freshman control group before entering Taylor University and a control group of seniors at Taylor University who have never gone on a study abroad experience in regard to Taylor University s IDSs.
Mean (1- Strongly Disagree, 6- Strongly Agree) Mean (1- Strongly Disagree, 6- Strongly Agree) Study Abroad Pre and Post: IDS An analysis was conducted on the spring and fall 2008 study abroad group comparing the students before and after their intercultural experience. The results showed a statistically significant increase (p<.05) in 7of the 8 Intercultural Developmental Primary Scales totals. All scales with statistical significance indicated practical significance. 6 5 4 3 2 1 Pre/Post Comparison Personal Development Scales Pretest 6 5 4 3 2 1 Posttest 0 Pre/Post Comparison Cognitive Understanding Levels Pretest Posttest 0 Intellectual Interpersonal Spiritual
Eta (.2-snall,.5 med,.8 large) Study Abroad Pre and Post: IDS ~Practical Significance Study Abroad vs. Control Effect Sizes 0.4 0.36 0.3 0.29 0.32 0.32 0.28 0.2 0.19 0.16 0.1 Eta2 0
Study Abroad vs. Control Group: IDS 6 5 4 3 2 1 0 For this study an analysis was conducted comparing the students who studied abroad in the spring and fall of 2008 with a control group of freshmen and senior students who had no intercultural experience. The results showed a statistically significant increase in each of the 19 Intercultural Developmental Scales (IDS) measured for those students who studied abroad, and in 11 of the 19 subscales there was a practically significant change in the students intercultural development due to their study abroad experience. Study Abroad vs. Control Group Personal Development Connections Intellectual Spiritual Interpersonal Study Abroad Control Group 6 5 4 3 2 1 0 Study Abroad vs. Control Group Cognitive Understanding Connections Study Abroad Control Group
Eta (.2-snall,.5 med,.8 large) Study Abroad vs. Control Group IDS ~Practical Significance Pre/Post Effect Sizes Common Learning Objectives 0.5 0.41 0.4 0.3 0.2 0.1 0 0.28 0.02 0.35 0.39 0.2 0.22 Eta 0.49
Taylor University Common Learning Objectives Analysis Common Learning Objectives: Critical Thinking Spiritual Activity Competent Communication Aesthetic Literacy Civic Mindedness Responsible Stewardship Lifelong Learning
Taylor University Common Learning Objectives Analysis Purposes: 1) To discover the degree that students are developing Taylor University s educational Common Learning Objectives through their intercultural study abroad experiences by comparing their pre and post scores on the Taylor University Intercultural Inventory (TUII). 2) To compare the relative influence of going on a study abroad program with a freshman control group before entering Taylor University and a control group of seniors at Taylor University who have never gone on a study abroad experience in regard to Taylor University s educational Common Learning Objectives using the (TUII).
Taylor University Common Learning Objectives Analysis Mean (1- Strongly Disagree, 6- Strongly Agree) There was a significant difference between students Pre and Post inventory scores on the TUII found for all seven of the objectives: Critical Thinking (p<.001), Competent Communication (p <.001), Spiritual Activity (p <.01), Civic Mindedness (p<.001), Responsible stewardship (p<.001), Lifelong Learning (p<.001), and Aesthetic Literacy(p <.001). 6 5 4 3 2 1 0 Pre/Post Comparison Pretest Posttest Common Learning Objectives
Taylor University Common Learning Objectives Analysis Eta (.2-snall,.5 med,.8 large) Five of these seven common learning objective areas showed practical significance. Critical Thinking (Eta 2 =.43), Competent Communication(Eta 2 =.22 ), Civic Mindedness(Eta 2 =.47), Aesthetic Literacy(Eta 2 =.28), and Responsible Stewardship(Eta 2 =.20). This means that an intercultural experience at Taylor University has practical value for significantly increasing these common learning objectives. Pre/Post Effect Sizes Common Learning Objectives 0.5 0.43 0.47 0.4 0.3 0.2 0.1 0.1 0.22 0.28 0.2 0.13 0 Eta2
Taylor University Common Learning Objectives Analysis Purpose 2 Results: We found that when comparing the control groups and students who had an intercultural experience in the Spring and Fall of 2008, that all of the 7 Common Learning Objectives were found significantly different (p<.001). 6.00 5.00 4.00 3.00 2.00 1.00 0.00 Study Abroad vs. Control Group Common Learning Objectives Study Abroad Control Group
Taylor University Common Learning Objectives Analysis Eta (.2-snall,.5 med,.8 large) Five of the seven Common Learning Objectives indicated practical significance when comparing the study abroad and control groups. Significant effect sizes were found for Critical Thinking (eta 2.33), Spiritual Activity (eta 2.35), Aesthetic Literacy (eta 2.26), Civic Mindedness (eta 2.26), and Responsible Stewardship (eta 2.22). Study Abroad vs. Control Group Effect Sizes 0.5 0.4 0.3 0.2 0.1 0 0.33 0.35 0.12 0.26 0.26 0.22 0.19 Eta
Taylor University Core Values Analysis Core Values: Christ Centered Biblically Anchored Liberal Arts Grounded Whole Person Focused Vocationally Equipped Servant Leadership Motivated World Engaging
Taylor University Core Values Analysis Purposes: 1) To discover the degree that students are developing Taylor University s Core Values through their intercultural study abroad experiences by comparing their pre and post scores on the Taylor University Intercultural Inventory (TUII). 2) To compare the relative influence of participating in a study abroad program with a freshmen control group before entering Taylor University and a control group of seniors at Taylor University who have never studied abroad.
Mean (1-Strongly Disagree, 6-Strongly Agree) Taylor University Core Values Analysis All seven of the Core Values increased significantly from the pre to the posttest. 6 5 4 3 2 1 Pre/Post Comparison Core Values Pre Post
Taylor University Core Values Analysis The 2008 Study Abroad group s intercultural experience had a total practical bearing (Eta 2 =.46) on the students development of the university s Core Values. There is a real practical value for the intercultural experience for accomplishing Taylor s Core Values.
Taylor University Core Values Analysis 6 5 4 3 2 1 Purpose 2 Results: Each of the Core Values was significantly greater (p<.001) for students who studied abroad compared with those who had no intercultural experience (Control Group). Pre/Post Comparison Core Values (1-3) 6 5 4 3 2 1 Pre/Post Comparison Core Values (4-7) 0 Liberal Arts Grounded Liberal Arts Grounded w/o a language Christ Centered Biblically Anchored 0 Whole Person Whole Person Focused (All) Focused (Selected) Servant Leadership Motivated Vocationally Equipped World Engaging
Lighthouse Program Analysis: Purposes Pretest Posttest Comparisons The first purpose of this study was to assess changes of students involved in the Lighthouse program during January 2009 by comparing their scores on a pretest compared to their posttest in regards to the IDS and Common Learning Objectives. Lighthouse Group& Control Group Comparison The second purpose was to compare those that went on Lighthouse trips in January of 2009 and a control group. The analysis focused on intercultural developmental growth of those going on a lighthouse experience (N=61) with a control group (N=116).
Results on IDS s Lighthouse 2009 IDS Pre and Post-Test Analysis All of the Intercultural Developmental Scales (IDS) increased significantly from pre-test to post-test. Of the 23scales that increased, 22 indicated practical significance (Eta 2.22-.71). The scale that was not practically significant was interpersonal attitude.
Results of IDS s Lighthouse and Control Group Comparison on IDS Scores Of the 19 IDS that were compared all were statistically significant at.000 level. All of 19 IDS were practically significant (.28-.60).
Implications The above results indicate that not only did the Lighthouse students grow significantly from before they went on their trip to when they came back, but their growth was significantly greater than those students without intercultural experience.
Results of Common Learning Objectives 6 5 4 3 2 1 0 Pre/Post Comparison Common Learning Objectives Pretest Posttest Lighthouse on Taylor University Learning Objectives Analysis Significant differences were found between the pre and post-tests for all of the 7 objectives. There was practical significance (.40-.70) for all of the 7 objectives.
Implications Data indicates that the Lighthouse Program meets and exceeds the Taylor University academic learning objective goals.
Assessing the Intercultural Development of Taylor University Students Longitudinal Investigation Update: Year Two Mean Score 58 of 369 students of the 2007 freshmen class had intercultural experience in their first two years at Taylor Of the eight IDS, seven were found to have statistical significance from pre to posttest (all but Attitudes) 6 5 4 3 2 1 0 Year Two of Longitudinal Study (2007-2009) Intercultural Developmental Scale Pre Post
Effect Size Assessing the Intercultural Development of Taylor University Students Longitudinal Investigation Update: Year Two 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Practical Significance of Taylor Students' Intercultural Experiences Practical significance was found in the strong range for all seven of the statistically significant IDS. An effect size must be greater than.20 to be considered significant. Intercultural Developmental Scale
Intercultural Developmental Scales Intellectual The formation of cognitive thoughts and processes concerning a unique culture Interpersonal The interaction occurring between individuals of different culture Spiritual The progressive and dynamic interaction with a personal God and spiritual community of a unique culture Knowledge Understanding of specific facts, truths and principles of different cultures Intellectual Knowledge The formation of cognitive thoughts and processes based on the understanding of specific facts, truths and principles of unique cultures Interpersonal Knowledge Understanding of specific cultural facts, truths and principles in order to effectively interact with people of different cultures Spiritual Knowledge Understanding of the culture s perspective on specific cultural facts, truths, and principles about God and the effect it has on the individual s interaction with God Awareness Perceiving differences between aspects of unique cultures Intellectual Awareness The formation of cognitive thoughts and processes by recognizing the existence of the facts, truths, and principles of unique cultures Interpersonal Awareness The interaction between individuals based on an accurate perception of unique cultures Spiritual Awareness Perceiving the spiritual significance between aspects of unique cultures Attitudes Personally held dispositions concerning different cultures Intellectual Attitudes Personally held dispositions concerning different cultures based on the formation of thoughts and processes due to a cultural experience Interpersonal Attitudes Personally held dispositions concerning different cultures that change the interaction occurring between people of different cultures Spiritual Attitudes Personally held dispositions concerning different cultures that change the individual s interaction with God and a spiritual community Behavior The ability to act effectively and appropriately on facts truths and principles of a unique culture Intellectual Behavior The use of abilities to act on the facts, truths, and principles found through the formation of cognitive thoughts and processes developed from experiencing a unique culture Interpersonal Behavior The ability to act effectively and appropriately during the interaction between individuals of different cultures Spiritual Behavior The interaction God and spiritual community demonstrated through the use of an individual s cultural skills Intrapersonal Growth Internal development due to knowledge, awareness, attitudes and behavior/skills from cultural experiences Intellectual Intrapersonal Growth The development of the cognitive thoughts and processes of the individual concerning different cultures Interpersonal Intrapersonal Growth The development within the individual due to the interaction with individuals of different cultures Spiritual Intrapersonal Growth The development that occurs within the individual due to their interaction with God