DEVELOPMENT OF THE PSYCHOLOGICAL SKILLS INVENTORY FOR CHINESE ATHLETES by Xiaochun Yang B.Sc, Wuhan Institute of Physical Education, 1993 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES (School of Human Kinetics) We accept this thesis as conforming to thej'equired standard THE UNIVERSITY OF BRITISH COLUMBIA April 1997 Xiaochun Yang, 1997
In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. ; Department of f t»ia> JL^ The University of British Columbia Vancouver, Canada Date DE-6 (2/88)
ABSTRACT An inventory for testing the psychological skills of Chinese Athletes was developed based on a two-order structure of both Howe's (1993), and Hardy and Jones' (1994) conceptualizations. This project consisted of two phases. In Study I the original version of the 47-item Psychological Skills Inventory for Chinese Athletes (PSICA) was administrated to 305 subjects ranging from college level to international level athletes. The PSICA contains five sport specific subscales: Anxiety Control, Concentration, Confidence, Mental Preparation, and Motivation. A confirmatory factor analysis (CFA) revealed some problems with the original 47-item PSICA, but a modified 23-item scale demonstrated a good factorial validity. In Study II the revised 23-item PSICA was administrated to 713 subjects ranging from college level to international level athletes. Cross validation test revealed some problems with the model, and /or the inventory. The overall fit indices, with exception of the RMSEA for the female subsample, when analyzed using weighted least square (WLS) procedure, indicated an adequate but not good fit of the five-factor model. Convergent and discriminant validity test, and the test-retest reliability results indicated that the PSICA is psychometrically strong in these aspects. The findings suggest that the PSICA has potential as a valid psychological scale. ii
TABLE OF CONTENTS ABSTRACT TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES ACKNOWLEDGMENTS ii iii iv v vi INTRODUCTION 1 Introduction to Psychological Skills 1 Components of the New Psychological Skills Inventory 5 Psychological Techniques 8 STUDY I: INITIAL INVENTORY DEVELOPMENT 12 Instrument 12 Subjects 12 Procedures 13 Results 15 STUDY BE: ASSESSMENT OF RELIABILITY AND VALIDITY 19 Instruments 19 Subjects 19 Procedures and Results 20 DISCUSSION 33 REFERENCES 38 LITERATURE REVIEW 43 APPENDICES I. Psychological Skills Inventory for Chinese Athletes (Original Version) 55 II. Psychological Skills Inventory for Chinese Athletes (Final Version) 59 III. Demographic Information of Subjects (Study II) 61 iii
LIST OF TABLES Table 1 Descriptive Statistics and Internal Consistency of PSICA subscales (Study I) 15 2 Model-Testing Results from Confirmatory Factor Analyses (Study I) 16 3 Interscale Correlations among the Subscales (Study I) 18 4 Descriptive Statistics, Internal Consistency, and Test-retest Reliabilities of the PSICA Subscales (Study II) 21 5 Model-Testing Results from Confirmatory Factor Analyses (Study II) 23 6 Factor Loadings of the PSICA Items Using a Polychoric Correlation Matrix as Data Input (Study II) 24 7 Factor Loadings of the PSICA Items Using a Pearson Product-Moment Correlation Matrix as Data Input (Study II) 26 8 Factor Score Correlations among the PSICA Subscales and CSAI-II Subscales (Study II) 28 9 Multivariate Summary and Univariate Follow-ups (Study II) 30 10 Post-hoc Results of the Sport Level by the PSICA Subscale Main Effect (Study II) 31 11 Demographic Information of Male Subjects (Study II) 61 12 Demographic Information of Female Subjects (Study II) 62 iv
LIST OF FIGURES Figure 1 Classification of Psychological Skills and Techniques based on a Literature Review 4 2 Psychological Skills and Techniques Assessed with the PSICA 10 3 Classification of Psychological Skills and Techniques based on a Literature Review 51
ACKNOWLEDGMENTS The completion of this thesis was a result of a successful group effort. I am greatly appreciative to a number of individuals who provided their guidance and support that made this research process a valuable learning experience. I would like to express my sincere appreciation to my committee members: Dr. Bleuler, Dr. Schutz, and Dr. Sinclair for their expertise and guidance. It is a pleasure to acknowledge those who have assisted me at different stages of this research. I would like to thank the efforts of Mr. Jie Ren, and some other Chinese sport psychologists who collected the valuable data for this study. The most valuable learning experience though has been through my association with Dr. Schutz. He was unfailing in his enthusiasm and support and, at all time, he was willing to discuss with me the numerous questions and problems that I encountered. His time, effort and thorough knowledge was very much needed and appreciated, and also his example as a statistician/scientist has left a lifelong impression. vi
INTRODUCTION Introduction to psychological skills Research on peak performance and ideal performance states is still in its relative infancy. However, there is sufficient evidence from retrospective studies to suggest that a number of psychological skills may contribute to peak performance (e.g., Gould, Hodge, Peterson, & Petlichkoff, 1987; Mahoney, Gabriel, & Perkins, 1987; Vealey, 1988). Psychological skills are psychological characteristics such as anxiety control and concentration that can be improved through practising. Consequently, psychological skills training (PST) programs have flourished since the 1980's. Regardless of the age or skill level of the athletes, individuals who have weak psychological skills will benefit from PST (Williams, 1986). There is a wide variation in what skills people consider to be important, and the inclusion of particular areas is somewhat arbitrary depending on the experience of the individual (Seiler, 1992). For example, Hall and Carron (1990) provided coaches with information on self confidence, mental imagery, performance analysis, goal setting, token rewards and variety in practice sessions. Gould, Petlichkoff, Hodge, and Simons (1990) offered a psychological skills training program to elite wrestlers which included relaxation, visualization/imagery, goal setting, and mental preparation techniques. Gould, Tammen, Murphy, and May (1989) noted that there was some agreement among sport psychologists working with Olympic athletes who were asked to list those areas they considered important. Consensus was obtained for the areas of goal setting, relaxation training, arousal regulation, visualization (imagery), and self-talk. These factors could be considered specific techniques although they might not necessarily be presented independently. A more general listing of mental skills was presented by Gould, Hodge, Peterson, and Petlichkoff (1987), derived from
their survey of wrestling coaches. They reported that these coaches were eager to develop programs for anxiety/ stress management, attention/concentration, and confidence building during competition. These areas were confirmed by Sullivan and Hodge (1991) in their survey of New Zealand athletes and coaches, who also added the area of precompetition preparation as a concern. These can be seen as broader areas of concern and do not reflect the techniques as directly. In Weinberg and Gould's (1995) summary of the topics coaches and athletes would find useful in PST programs, they listed arousal regulation, imagery (mental preparation), confidence building, increasing motivation and commitment (goal setting), and attention (concentration skills, self-talk, mental plan). When considering the underlying principles involved in PST, Vealey's (1988) distinction between advanced skills and basic techniques was considered to be an appropriate starting point: advanced skills are "qualities to be attained", while basic techniques are what "athletes engage in to develop advanced skills"(p. 326). In this context, advanced skills such as anxiety control, can be thought of as the desired outcomes associated with the implementation and practice of basic techniques, such as goal-setting and relaxation. Hardy and Jones (1994) classified the advanced skills and basic techniques as follows: basic techniques included Goal-setting, Imagery, Relaxation, and Self-talk; and advanced skills included Anxiety Control, Activation Control, Self-confidence, Maintaining Motivation, and Attention Control. Howe (1993) developed a two-order structure for psychological skills training. The higher-order component included Arousal Management, Confidence Building, Focusing/Concentration, Precompetition and Competition Strategies, and Leadership. The first-order component included Relaxation, Imagery, Self-talk, Reinforcement, Goal Setting, Communication, and Attention Strategies. This is similar to the advanced skills and basic 2
techniques classification. The differentiation between skills and techniques is quite important, because they are two different components. Therefore, the two-order structure classifications of Howe's (1993) and Hardy and Jones' (1994) are quite useful. A benefit of those structures is that they avoid the confusion in the development of PST programs. People often become confused when facing a PST program that contains both an anxiety control skill and a relaxation technique, because the later is one of the basic techniques of anxiety control skill. Figure 1 summarizes the classification of psychological skills and techniques based on a literature review. Silva (1982) and Seabourne, Weinberg, Jackson, and Suinn (1985) indicated that a comprehensive or abbreviated training program will be more effective if psychological skills training objectives appropriate to the athletes are identified before the program is planned. Therefore, the first step in planing any PST program should be to assess those psychological skills that may be deficient. A few instruments designed to assess psychological skills specific to sport have recently been developed, such as the Competition State Anxiety Inventory-II (Martens, Vealey, & Burton, 1990), the Sport Anxiety Scale (Smith, Smoll, & Schutz, 1990), and the Sport Competition Anxiety Test (Martens, 1977). However, there is still a need for a multidimensional scale instrument that assesses a broad range of psychological skills possessed by athletes as the presently available instruments are most often used to assess precompetition anxiety specifically. In an attempt to develop such a measure, Mahoney and his colleagues (Mahoney & Avener, 1977; Mahoney, Gabriel, & Perkins, 1987) developed the Psychological Skills Inventory for Sport (PSIS). The scale has undergone continuous development, and the most frequently studied version (PSIS R-5) consists of 45 items that are arranged into six subscales (Anxiety Control, Concentration, Confidence, Mental 3
Figure 1 Relationships among the Psychological Skills and the Techniques Self-talk Relaxation Imagery Reinforcment Competition Plan Goal Setting Psychological Skills Psychological Techniques Preparation, Motivation, and Team Emphasis). In its various forms, the PSIS has been successfully employed by a number of investigators to differentiate between elite and nonelite athletes, male and female athletes, nordic disabled and able-bodied athletes, athletes in 4
various sports and athletes of different nationalities (e.g., Cox & Liu, 1993; Cox & Yoo, 1995; Qiu, 1993; White & Croce, 1992). Despite its promise as a research instrument, the PSIS R-5 appears to have a number of serious psychometric shortcomings that limit its potential usefulness. Both Yang (1995) and Tammen, Murphy, and Jowdy (1990) were unable to replicate the factor structure established by Mahoney and his coworkers (1987). The result from Yang's study might be due to culture differences since Chinese athletes were used as subjects in her study. Chartrand, Jowdy, and Danish (1992) tested the hypothesized six-factor (subscale) model advanced by Mahoney and his coworkers (1987) using confirmatory factor analysis. They found no evidence for the factorial validity of the scale, nor for any of the alternative models that they tested. Many of the items loaded on several of the factors, indicating that the subscales were not measuring distinct constructs, and seven of the items failed to load on any factor. It thus appears that, in its present form, the PSIS R-5 does not meet the factorial validity standards required for a multidimensional instrument used for research or applied purposes. Components of the new psychological skills inventory It is thus appropriate to develop a new multidimensional instrument with good factorial validity for the assessment of the psychological skills of athletes, and this is the purpose of this study. There may be two reasons for the results of Chartrand, Jowdy, and Danish (1992) that showed the factorial validity problem of the PSIS R-5. One possibility was that the theoretical structure of the PSIS R-5 was wrong. The other possibility was that the test items were not properly written, and could not reflect the theory structure of the PSIS R-5. 5
Since the six-subscale structure of the PSIS R-5 is similar to the first-order classification of psychological skills by Howe (1993) and Hardy and Jones (1994), it is reasonable to believe that the theoretical structure of the PSIS R-5 is acceptable. Therefore, a five-subscale structure for a new psychological skills inventory was developed based on the PSIS R-5 structure and Howe's (1993) and Hardy and Jones' (1994) classifications. The five subscales are Anxiety Control, Concentration, Confidence, Mental Preparation and Motivation. Concentration and Confidence are two common psychological skills that were included in all of the three previous structures. The words "arousal" and "anxiety" are often used interchangeably, but it is important to distinguish between them. Arousal is "a general physiological and psychological activation of the organism [person] that varies on a continuum from deep sleep to intense excitement" (Gould & Krane, 1992, pp. 120-121), while anxiety is "a negative emotional state with feelings of nervousness, worry, and apprehension associated with activation or arousal of the body" (Weinberg, & Gould, 1995, p. 93). On the basis of those definitions, the Arousal Management skill in Howe's (1993) classification included Hardy and Jones' (1994) Anxiety Control skill and Activation skill. Anxiety Control skill aims at reducing the negative effect of anxiety and helps athletes to calm down when they are highly anxious. Activation control skill aims at helping athletes to arrive at the desired arousal level required by competition. Few studies have examined the effect of raising arousal level (Howe, 1993), and the lack of activation is not a common phenomenon in competition situations. On the other hand, many researchers have reported that athletes experience high levels of competition anxiety. As Howe (1993) said "Higher arousal may be necessary for several action-type sports. However, it is also likely that all sports will involve some situations that produce enough 6
anxiety to interfere with performance" (p. 34). Therefore, only anxiety control skill was included in the new inventory. Although there was inconsistency in Howe's (1993) and Hardy and Jones' (1994) classifications on Mental Preparation and Motivation factors, they were kept in the new structure for two reasons. One was a theoretical reason. Although some of the techniques for mental preparation are also used in anxiety control, concentration control, or even in obtaining confidence, there are some techniques that are used exclusively for mental preparation. It is an individual skill that can help athletes gain control before and during a competition. Motivation is a special field in sport psychology research, and it is widely agreed that motivation is an important component of human behavior. The important question here is "Is controlling motivation a psychological skill or not?". Since people can set appropriate levels of motivation through goal setting or some other techniques, it should be viewed as a kind of psychological skill. The second reason for retaining motivation was based on practical reasons. The purpose of the new inventory is to assess the needs in psychological skill training, so it should cover a wide variety of psychological skills. The Team Emphasis subscale in the PSIS R-5 was excluded from the new inventory not only because it was excluded in both Howe's and Hardy and Jones' classification, but also because Mahoney (1989b) himself mentioned that team emphasis varies among team and individual sports. Since the new inventory is for testing athletes' psychological skills, the leadership factor in Howe's classification was not included. Finally, the five theoretical psychological skills in the new inventory are similar to the five psychological skills Weinberg and Gould (1995) determined from their survey of athletes and coaches. Thus, the theoretical structure of the new inventory will have practical utility. 7
Psychological techniques An examination of the PSIS R-5 test items, revealed some items had problems. The items were written to test the psychological state of the athletes rather than their psychological skills. For example, a question like "I am more tense before I perform than I am during the performance", is testing the athletes' anxiety state, not testing the anxiety control skill. Although there are high correlations among psychological skills and psychological states, they are two constructs. Researchers have pointed out that while both elite and non-elite athletes have competition anxiety, the difference is that elite athletes know how to control their anxiety (Gould, Eklund, & Jackson, 1993; Troup, 1991). The purposed new inventory will focus on testing athletes' psychological skills. As classified by Howe (1993), and Hardy and Jones (1994), each psychological skill consists of many techniques. Relaxation (including breath control) and self-talk ( coping statements, cue words, and thought-stopping techniques) are techniques widely used for developing anxiety control, concentration, and confidence skills. Although some researchers included imagery as a technique for developing those three psychological skills, it is included in the Mental Preparation subscale in the new inventory. Not only because imagery is a very important technique for mental preparation ~ Weinberg and Gould (1995) even regarded it as an interchangeable word for "mental preparation", but also because pre-competition imagery is really a part of mental preparation. A special technique for confidence building is reinforcement, and research has shown that positive information can help athletes gain confidence. A technique referred to as a "competition plan" has been viewed as a very important technique for developing mental preparation skill since the 1980's. Although some researchers reported that it can help athletes with concentration control (Boutcher, 1990; 8
Jones & Hardy, 1990; Orlick, 1986; Orlick & Partington, 1988), it is based on the fact that athletes are mentally well prepared. This technique is included in the Mental Preparation subscale in the new inventory. One well-known technique for motivation control is goal setting. This technique was developed in management psychology, and is now widely used in sport psychology. Based on the fact that athletes with a proper goal for competition (that is, aiming at challenging the reachable personal potential) will be less distracted by noncompetition related things, and will develop their confidence little by little, this technique can also be used for developing concentration (Boutcher, 1990; Jones & Hardy, 1990; Orlick, 1986; Orlick & Partington, 1988) and confidence (Elliot & Dweek, 1988). In the new inventory, this technique is included only in the Motivation subscale. Figure 2 summarizes the factorial structure of the new inventory. Although those techniques are basic factors in developing psychological skills, the new inventory still aims at testing the psychological skills not the techniques. This is because the techniques are only the vehicles used to attain the target psychological skills. It is the absence of psychological skills that determines which kind of technique should be used. As Vealey (1988) stated, the sport psychologist should focus on the skill to be attained and to choose any technique or combination of techniques to use toward attaining and enhancing that skill. From indications above, one can see that the five psychological skills are naturally related. Not only are some of the underlying techniques common among the psychological skills, but also some psychological skills are the basis for other psychological skills. For example, Weinberg and Gould (1995) indicated that confidence can help individuals in concentration and goal setting (motivation). Other research revealed that less anxious 9
Figure 2. Psychological Skills and Techniques Assessed with the PSICA Self-talk Psychological Skills Psychological Techniques athletes have less concentration problems. Therefore, moderate correlations among those subscales are expected.
Although the five-subscale construct is based on research in North America, the successful use of the PSIS R-5 on Chinese athletes (Cox & Liu, 1993; Qiu, 1993) did provide evidence that the five-subscale construct fits this population too. The new inventory will be applied to Chinese athletes, and as culture differences are expected, the utility of the inventory is limited to that population. Therefore, the instrument is named as Psychological Skills Inventory for Chinese Athletes (PSICA), but a subsequent a North American version may be possible. The thesis project consisted of two studies. Study I was the initial inventory development, an examination of its psychometric characteristics, and modifications of the original inventory. Study II involved cross validation of the revised inventory (through administration to a new sample) and a testing of the psychometric characteristics of the revised inventory. Details will be discussed in the following sections. 11
STUDY I: INITIAL INVENTORY DEVELOPMENT The first two phases of Study I, the development of the instrument and its administration, were conducted prior to the initiation of this thesis and thus are only briefly described here. The second two phases, the psychometric analyses and inventory modification constitute the starting point for this specific research project. Instrument A total of 57 test items were written based on the theoretical structure of the new inventory, former research results and data collected from an open-ended psychological skills inventory handed out to 50 Chinese athletes ranging from college level to national level. An example of an item from the open-ended inventory was: "What will you do when you can not concentrate on your performance during the competition?" The 57 items covered psychological skills used before competitions and during competitions. The pool of 57 items was then given to three Chinese athletes to review for ambiguity and wording, and to two Chinese sport psychologists to review for adequacy of content coverage, and face validity. Based on their reviews, 10 items were eliminated and minor changes in wording were made. The 47-item original inventory was scored on a 5-point Likert-scale ranging from 1 to 5 (Always- Often- Sometimes- Seldom- Never). Demographic and training questions such as birth date, gender, sport experience, and best achievement in sport were included in the Psychological Skills Inventory for Chinese Athletes (PSICA). The original PSICA is presented in Appendix I. Subjects A total of 350 sets of the original inventory were sent to male and female adult athletes (older than 18 years old) from international level, national level, and college level.
Inventories in which over 80% of the items were completed were considered useful, resulting in a total of 305 inventories retained for analysis (male=209, female=96). Individuals who had been competitors of world competitions were considered international level subjects. China categorizes athletes from grade three to grade one and elite. We considered individuals at grade one or above, but having not taken part in international competitions as national level subjects, and individuals under grade one and studying at colleges as college level. Because of the limitation of the sample size, both male and female athletes were analyzed together. The athletes represented sixteen different sports. Procedures After contacting coaches and athletes and receiving their approval, PSICA tests were administered under the supervision of supervisors trained for this study. The supervisors were trained to explain the items according to a standard procedure. The finished inventories were collected by the supervisors directly, and then sent to the researcher. Athletes were assured that coaches had no chance to see the answers and that supervisors would not take part in the data analysis. Confirmatory factor analysis (CFA) was conducted for testing the factor validity of this inventory with LISREL 8 (Joreskog & Sorbom, 1993) and using maximum likelihood estimation procedures with a variance-covariance matrix as data input. Confirmatory factor analysis allows developers to evaluate the degree to which the structural characteristics of a scale conform to a hypothesized underlying model, as well as the degree to which each item maps appropriately on to the underlying subscale structure. The fit of each model was evaluated with a number of indices, including the p-value associated with the X 2 statistic, Steiger's (1990) Root Mean Square Error of Approximation (RMSEA), the Goodness-of-Fit 13
Index (GFI), the Parsimony Goodness-of-Fit Index (PGFI), and Bentler's 1990 Comparative Fit Index (CFI). A RMSEA of.05 or less indicates that the model based on the sample data represents a " close fit" to the population, and values less than.08 a reasonable fit (Joreskog & Sorbom, 1993). The GFI indicates the relative amount of variance and co variance jointly explained by the model, and a value close to 1.00 indicates a good fit (Byrne, 1989). The PGFI was selected because of its utility in comparing competing models (the larger the PGFI, the more parsimonious the model), and the CFI was chosen over other normed fit indices because it is contained in the 0~ 1 interval. CFI values of.90 and larger were deemed to indicate an adequate fit of the model to the data, even with a PGFI as low as.50 (Mulaik, James, Van Alstine, Bennett, Lind, & Stilwell, 1989). A X 2 /df ratio of around 2.0 or less is considered good, and may indicate an acceptable fit of the overall model. Cronbach's alphas were calculated for the five subscales to test their internal consistency. Item-total correlations were also calculated for each subscale to test the itemsubscale relationships. Some modifications were done to the items on the basis of the results from confirmatory factor analyses and item-total correlation analyses, providing that there was theoretical support. First, items that had a low correlation with the subscale they belonged to and loaded low on the completely standardized solution of confirmatory factor analyses (compared to the loadings of the other items on that scale), were deleted from that subscale. Second, modification indices for the factor loadings and completely standardized expected change for these loadings were checked. If an item loaded higher on more than one subscale, it was deleted, and if an item loaded higher on a subscale other than on the subscale it belonged to, it was switched to the subscale on which it loaded highest. Third, the largest 14
positive/negative standardized residuals were checked, and items that caused very large standardized residuals were considered for deletion. Confirmatory factor analysis was rerun after each modification, and the goodness of fit indices were checked to see if the modification led to a better fitting model. Finally, the subscale correlations were examined. Confirmatory factor analysis was run for a single-factor model to see if it was better than the 5-factor model. Cronbach's alphas were tested again for the modified PSICA inventories. Results As a preliminary step to the factor analysis, the distributional properties of the responses to the 47 items were examined. Skewness and kurtosis measures suggested that the marginal distributions of the data set were approximately normal; skewness values ranged from -1.00 to.71, and kurtosis values ranged from -1.00 to.49. Table 1 contains the number of items, means, standard deviations, and internal consistency statistics (Cronbach's alpha) for each of the five PSICA subscales. The CFA results of 11 different models are presented in Table 2. Table 1. Descriptive Statistics and Internal Consistency of PSICA Subscales (Study I) Subscales No. of Items M SD a Anxiety Control 10(4) 32.80 4.46.52 (.54) Concentration 8(4) 27.56 4.01.54 (.52) Confidence 9(3) 31.03 4.82.61 (.47) Mental Preparation 10(7) 31.76 6.27.71 (.69) Motivation 10(5) 34.04 5.81.70 (.63) Note. The parameters of the final model are in parentheses. 15
Table 2. Model-Testing Results from Confirmatory Factor Analyses (Study I) PSICA models X 2 /df RMSEA GFI PGFI CFI Original(47,5) 1.76.050.80.72.67 Revision 1(41,5) 1.75.050.82.74.73 Revision2(38,5) 1.75.050.84.74.76 Revision3(35,5) 1.61.045.86.75.81 Revision4(34,5) 1.58.044.86.75.82 Revision5(31,5) 1.56.043.88.75.84 Revision6(28,5) 1.56.043.89.75.86 Revision7(26,5) 1.58.044.90.74.87 Revision8(24,5) 1.63.046.90.73.87 Final(23,5) 1.60.045.91.73.88 Single-factor 5.48.120.53.49.00 Note. The first numbers in the parentheses are the number of items in the inventories, and the second numbers in the parentheses are the number of factors in the inventories. RMSEA= Root Mean Square Error of Approximation, GFI = Goodness-of-Fit Index, PGFI = Parsimony Goodness-of-Fit Index, CFI = Comparative Fit Index. In the first analysis, the 47-item, 5-factor PSICA was evaluated. The goodness-of-fit statistics indicated that the data did not conform well to the hypothesized structure. The CFI of the original inventory was only.67. A series of controlled sequential steps finally resulted in a new 23-item, 5-factor inventory that proved to have the acceptable dimensional structure. None of the negatively worded items in the original inventory held well on the sub scales, and were deleted from the sub scales. This might reveal that Chinese athletes had some reaction patterns to the negative items. As can be seen in Table 2, the statistics for the 16
final (23,5) model indicated an acceptable fit of the model to the data (X 2 /df=1.60, RMSEA=.045, GFI=.91, and CFI=88). Additionally, factor loadings were significant at p<.001, and 15 out of the 23 loadings exceeded.45 (others were over.40). Because alpha coefficients are highly influenced by the number of items in the scale, most alphas went down after deletion of several items. For example, the alpha value of the Confidence subscale went down to.47 from.63 after 6 items were deleted from the subscale and only 3 items were left. In this case, the goodness of fit between the subscales and the underlying model in a confirmatory factor analysis might be more indicative of the adequacy of construct measurement than a borderline level of internal consistency (Nunnally & Berntein, 1994; Pedhazur & Schmelkin, 1991). The only exception was the Anxiety Control subscale. Its alpha coefficient went up from.52 to.54 after the deletion of 6 items. This might mean that the items in the original Anxiety Control subscale had a really weak internal consistency, therefore, modifications deleting the weak items led to a better internal consistency among the remaining four items. Table 3 shows the interscale correlations of the final model, which were calculated from the raw score subscale item totals. The subscales were correlated at a low to moderate level, ranging from.34 to.55. The moderately correlated subscales were Anxiety Control and Concentration (r=55); Concentration and Confidence (r=.53); and Concentration and Motivation (r=.52). In order to test the possibility that moderate correlations among the subscales meant a single-factor structure, a single-factor model was tested. The single-factor model gave a very poor fit; the X 2 /df ratio went up to 5.48, the RMSEA value went up to.12, and the CFI value went down to.00. These results indicated that the construct of psychological skills is best regarded as a multifaceted construct having five underlying 17
psychological skills facets. The moderate to low correlations among the subscales also suggested that the subscales can be treated as measures of reasonably distinct psychological characteristics in multivariate analyses. Table 3. Interscale Correlations among the Subscales (Study I) Subscales Anxiety Concentration Mental Motivation Control Preparation Concentration.55 Confidence.43.53 Mental Preparation.39.47.48 Motivation.34.52.46.46 The results of Study I suggested that the five factors structure of the PSICA is correct, and the revised 23-item, 5-factor PSICA had an acceptable factorial validity. However, more tests of the PSICA's psychometric characteristics were conducted in Study II. The revised PSICA is presented in Appendix II. 18
STUDY II: ASSESSMENT OF RELIABILITY AND VALIDITY Instruments The Psychological Skills Inventory for Chinese Athletes (PSICA). This is the 23- item, 5-factor inventory developed from Study I. Since all items are positively stated, answers from "always" to "never" are scored from 5 to 1, respectively. Therefore, higher PSICA subscale scores indicate higher psychological skills. To test the convergent validity and discriminant validity of the PSICA several well developed inventories were needed to compare with the PSICA. The problem is that although there are many sport specific inventories, not many of them have Chinese versions. The only existing inventory suitable for this purpose is the Competitive State Anxiety Inventory (CSAI-II). Competition State Anxiety Inventory (CSAI-II). The CSAI-II is a multidimensional measure of state anxiety, assessing somatic and cognitive anxiety in sport situations as well as confidence. It is a well-developed inventory, and has been widely used in sport psychology research. The Chinese version of the CSAI-II is available with detailed psychometric information (Zhu, 1994). High somatic or cognitive anxiety scores indicate high anxiety, while high confidence scores indicate high confidence. Correlations among the PSICA subscale scores and the CSAI-II subscale score were calculated. Subjects A total of 800 sets of the revised inventory were sent to male and female Chinese athletes participating in 18 different sports at international, national, and college levels, (the definitions for the three levels are same as the definitions given in Study I). Only 713 sets of 19
the inventory were returned and useful (male=460, female=253, age 13 to 33). The subjects were chosen from typical cities in eight geographical regions of China: North East China, North West China, North China, Middle China, East China, South China, South West China, and South East China. Different participants were used in Study I and Study II. Detailed information is presented in Appendix III. Athletes were contacted through coaches. After obtaining the athletes' approval, the PSICA and CSAI-II tests were administered under the supervision of the supervisors trained for this study. The international and national level athletes were tested in the competition intervals during their competition seasons, and the college level athletes were tested in their rest time during the off-season. Because that the CSAI-II was not tested right before a competition, the test scores might be less accurate, and this in turn, would decrease the correlation among CSAI-II and PSICA subscales. The completed inventories were collected by the supervisors and sent to the researcher by mail. Athletes were assured that coaches would not see the answers and that supervisors would not take part in the data analysis. Confirmatory factor analyses and internal consistency (Cronbach's alpha) tests for each subscale were conducted. Test-retest reliability, convergent validity and discriminant validity, and criterion related validity were also examined. The details are discussed in the following section. Procedures and Results Psychological Characteristics. Table 4 shows the male and female means, standard deviations, and internal consistency statistics (Cronbach's alpha) for the 5-subscales. Only the Anxiety Control subscale showed a significant gender difference on the means (Hotelling's J 2 =10.46, p_<05). 20
Table 4. Descriptive Statistics, Internal Consistency, and Test-retest Reliabilities of the PSICA Subscales (Study II) Scale Male Female Total Test- M SD a M SD a M SD a Retest AX 14.66 2.95.52 15.39 2.82.53 14.92 2.92.53.87 CC 14.56 2.94.61 14.86 2.48.37 14.67 2.79.54.87 CF 11.36 2.28.37 11.50 2.14.39 11.41 2.23.37.88 MP 23.63 5.23.72 24.26 4.94.72 23.85 5.14.72.92 MV 18.93 3.80.73 18.80 3.65.71 18.88 3.75.72.93 Note. Descriptive statistics and alphas are based on a sample of 460 male and 253 female Chinese athletes. Test-retest coefficients are based on 140 athletes who were retested after four weeks. AX = Anxiety Control, CC = Concentration, CF = Confidence, MP = Mental Preparation, MV = Motivation. Athletes were grouped by their ages (group 1: age 13-17; group 2: age 18-21; group 3: age 22-25; group 4: age 26-29: group 5: age 30-35). Tests of the means showed that none of the subscales had significant difference among the age groups. Therefore, comparisons were not considered in the following analyses. It was not surprising that the alpha coefficients were not large, given that the subscales contained only 3 to 7 items, and sampled fairly broad psychological skills constructs. The Confidence subscale had the lowest internal consistency (a=.37), which was even lower than in study one (a=.47). Internal consistency statistics were similar for male and female athletes, with the exception of the Concentration subscale, which had a higher internal consistency for the male subjects (a=.61) than for the female subjects (a=.37). The item- 21
total score correlations of the subscale were quite low for the female subjects, ranging from. 11 to.29. This suggested that males and females might have different response modes on this subscale. Cross validation of the factorial structure. The distributional properties of the responses of the 23 items were examined before confirmatory factor analyses were conducted. Skewness values ranged from -1.10 to.12, and kurtosis values ranged from -1.05 to.62. This suggested that the marginal distributions of the data set were approximately normal. Confirmatory factor analysis (CFA).was conducted with LISREL 8 (Joreskog & Sorbom, 1993), using the weighted least squares procedure (WLS) with a polychoric correlation matrix as data input, and the asymptotic variance-covariance matrix elements as weights. This is a method recommended for analyzing ordinal data (Bollen, 1989; Muthen, 1993). The CFA analyses were run on the total group, and separately by gender. Separate analyses for males and females were deemed necessary because of the descriptive statistics results (differences in some means and alpha coefficients), and because males and females are known to differ in psychological traits. The CFA results are presented in Table 5. The 23- item, 5-factor model did not fit the data very well. All X 2 /df ratios were higher than 2.00. The female subsample had the highest X 2 /df ratio of 3.84, and the male subsample had the lowest X 2 /df ratio of 2.13. The female subsample also had the highest RMSEA value (. 106), this was beyond the acceptable value of.080. However, the RMSEA values for the male subsample(.050) and the total sample (.045) were acceptable. Although the GFI values were all very high, ranging from.93 to.95, only male (CFI=.91) and female (CFI^.92) subsamples had very high CFI values. Because that the GFIs can be influenced by sample size, the robust 22
Table 5. Model-Testing Results from Confirmatory Factor Analyses (Study II) PSICA models X 2 /df RMSEA SRMR GFI PGFI CFI Polychoric correlations analyzed* Males 2.13..050.265.94.75.91 Females 3.84.106.382.93.74.92 Total Group 2.45.045.214.95.76.87 Pearson correlations analyzed** Males 2.30.053.052.91.72.88 Females 1.63.050.058.89.71.87 Total Group 2.76.050.046.93.74.88 22-item, 5-factor model Males 2.18.050.046.93.72.87 Females 1.63.050.048.90.71.90 Total Group 2.59.047.042.94.74.90 Note. * Weighted least squares procedures with a polychoric correlation matrix as data input, and asymptotic variance-covariance matrix elements as weights were used. ** Maximum likelihood procedures with Pearson product-moment correlations as data input were used. RMSEA = Root Mean Square Error of Approximation, SRMR = Standard Root Mean Residual, GFI = Goodness-of-Fit Index, PGFI = Parsimony Goodness-of-Fit Index, CFI = Comparative Fit Index. CFI values were considered to be more indicative of a good model fit. The bad fit of the X 2 /df ratios, and the good fit of the CFI values were not consistent. An examination of the residuals revealed that almost all of the covariances between two variables were overpredicted in the three confirmatory factor analyses. This was unusual, as there is usually 23
Table 6. Factor Loadings of the PSICA Items for Using a Polychoric Correlation Matrix as Data Input (Study II) Items Anxiety Concentration Confidence Mental Motivation Control Preparation 8.74 (.66) 15.74 (.73) 16.64 (.78) 18.76 (.91) 6.72 (.63) 10.79 (.91) 12.76 (.82) 21.71 (.75) 5.68 (.91) 7.66 (.75) 11.71 (.82) 1.66 (.59) 3.65 (.78) 9.73 (.87) 13.77 (.83) 14.70 (.70) 17.73 (.89) 19.80 (.73) 2.77 (.80) 4.76 (.74) 20.81 (.92) 22.77 (.79) 23.79 (.66) Note. Factor loadings for the female subsample are in parentheses, and outside the parentheses are factor loadings for the male subsample. a balance between the number of overpredicted and underpredicted covariances. The overprediction of the covariances might be the reason that the X 2 /df ratios, and the RMSEA value of female subjects were very high. The factor loadings for the items were moderate to high, ranging from.55 to.77 for the total sample; from.64 to.81 for the male subsample; and from.59 to.92 for female 24
subsample. Table 6 presents the factor loadings of the PSICA items for the male and female subsamples. The female subsample had four items which loaded higher than.90 on their factors (item 5,10,18, and 20). These were also the items that had the highest number of overpredicted covariances. The male subsample had few overpredicted covariances and comparatively lower factor loadings. Therefore, the overpredictions of the covariances cast doubt on the CFA results. Because the CFA using polychoric correlation and WLS led to extreme overestimation of the variances and covariances, the validity of the results is questionable. Consequently the CFA analyses were repeated using maximum likelihood estimation procedures (ML) with Pearson product-moment correlation as data input. The CFA results are presented in Table 5. There was a balance between the overpredicted and underpredicted covariances in all cases. The X 2 /df ratios and the RMSEA values of the total sample (X 2 /df =2.76, RMSEA=.050) and the male subsample (X 2 /df =.2.30, RMSEA=.053) were slightly higher than the former results. All GFI values were lower than the former results, and the CFI values went down for the male (CFI=88) and female (CFI=.87) subsamples, but were still acceptable. The CFI went up slightly for the total sample (CFI=.88). Results for the female subsample indicated that further examination of these results was required. The X 2 /df ratio and RMSEA value went down to 1.63 and.050 respectively. This was quite different than the former results, and suggested a good fit of the model for the female data. The inconsistent changes of X 2 /df ratios from different CFA results requires further explanation. Since the Pearson product-moment correlation matrix generally leads to underestimates of factor loadings and overestimates of their standard errors (Babakus, Ferguson, & Joreskog, 1987), the factor loadings were much lower this time in all three samples (see Table 7). 25
Table 7. Factor Loadings of the PSICA Items Using a Pearson Product-Moment Correlation Matrix as Data Input (Study II) Items Anxiety Concentration Confidence Mental Motivation Control Preparation 8.56 (.43) 15.49 (.48) 16.38 (.51) 18.45 (.48) 6.56 (.31) 10.58 (.45) 12.48 (.30) 21.53 (.40) 5.45 (.38) 7.35 (.42) 11.49 (.50) 1.39 (.31) 3.48 (.54) 9.54 (.54) 13.59 (.70) 14.56 (.64) 17.47 (.41) 19.60 (.52) 2.59 (.55) 4.55 (.68) 20.67 (.51) 22.63 (.63) 23.56 (.49) Note. Factor loadings for the female subsample are in parentheses, and outside the parentheses are factor loadings for the male subsample. However, the items all held very well on the factors they belonged to. Modification indices indicated that no changes were required, and suggested that the items only tested the characteristics they were expected to reflect. Although neither method yielded reliable results, there were some consistencies, and two reasons to believe that the results from the second method were closer to the real relationships between the model and the data. The first reason was that the goodness-of-fit 26
statistics based on a balanced overprediction and underprediction of the covariances should be more accurate than the indices based only on an overprediction of the covariances. The second reason was that the ordinal data used in this research was normally distributed with no excessive kurtosis and skewness, and had sufficient number of categories. Data such as these were robust when treating them as if they were continuous (Bollen, 1989). Therefore, the results from the second method will be discussed instead of the results from the first method. The factor loadings on Anxiety Control subscale were not high. And the moderate internal consistencies for both male (a=.52) and female (a=53) subsamples suggested that this subscale could be improved with some modifications. Item 16 loaded the lowest on the subscale for the male subsample (.38; and the highest for the female subsample (.51). This item might test underlying difference between males and females. There were big differences on the alpha coefficients for the male (a=.61) and female (a=.37) subsamples on the Concentration subscale. Items also loaded high on this subscale for the male subsample, but quite low for the female subsample (3 out of four items loaded below.40). Obviously, this subscale was better for testing males than for females. The Confidence subscale had only three items, and was the weakest subscale. The factor loadings on this subscale were low for both male and female subsamples (none of them beyond.50), and alpha coefficients were all quite low (a=.37 for males, and a=.39 for females). Items in this subscale should be reanalyzed, and more items added. The subscale items loaded highly on the Mental Preparation and Motivation subscales, with the exception of item 1 in the Mental Preparation subscale, which loaded comparatively low (factor loading was.39 for males and.31 for females) on the subscale. The alpha coefficients of the Mental Preparation subscale were the same for both male and female subsamples (a=.72). Table 5 also includes the CFA results 27
after the deletion of item 1 from the Mental Preparation subscale. Most of the goodness of fit indices were improved, especially for the female subsample (X 2 /df=1.63, RMSEA=.050, GFI=.90, CFI=.90), and suggested that the Mental Preparation subscale would be a good subscale without item 1, and this, in turn, would improve the factorial validity of the whole inventory. The Motivation subscale had high internal consistencies for both male (a=.73) and female (a=71) subsamples, even though it only had five items. It was the strongest subscale in the PSICA. The moderate correlations among the PSICA subscales presented in Table 8 ranged Table 8. Factor Score Correlations among the PSICA Subscales and CSAI-II Subscales (Study II) Scales Anxiety Concentration Confidence Mental Motivation Control Preparation AX 1.00 CC.48 1.00 CF.46.50 1.00 MP.45.55.52 1.00 MV.41.57.49.51 1.00 COGNITIVE -.66 -.34 -.07 -.19 -.22 SOMATIC -.66 -.25 -.24 -.27 -.24 CONFIDENCE.28.19.53.24.26 Note. Sample size was 713 for the PSICA interscale correlations, and was 111 for bicorrelations between the PSICA and CSAI-II subscales. 28
from.41 to.57. The highest correlation was between Concentration subscale and Motivation subscale. This was consistent with the findings that athletes with a proper goal/ motivation for competition will be less distracted outside competition things (Boutcher, 1990; Jones & Hardy, 1990; Orlick, 1986; Orlick & Partington, 1988). Test-retest reliability. A total of 140 subjects (male=101, female=39) were retested after a 4-week interval. Pearson product moment correlations between the two sets of PSICA raw scores were calculated for each subscale, and the results are presented in Table 4. The test-retest reliability of the inventory was very high, ranging from.87 to.93. This suggested that in spite of its weakness in its factorial structure, the inventory was a reliable measure, and tested relatively stable characteristics. Convergent validity and discriminant validity. A total of 116 subjects completed both the PSICA and CSAI-II. In order to avoid a test order effect, 64 subjects answered the PSICA first, followed by the CSAI-II; and 52 subjects answered in the reverse order. Correlations among the PSICA and CSAI-II subscales were calculated, and are presented in Table 8. The Anxiety Control subscale in the PSICA correlated highly with the Cognitive Anxiety (r=-.66) and Somatic Anxiety (r=-.66) subscales in the CSAI-II, but lowly with the Confidence subscale in the CSAI-II (r=.28). Since higher scores in the PSICA subscales represent higher psychological skills, while higher scores in the CSAI-II Cognitive and Somatic Anxiety subscales represent higher anxiety, the negative relationships between the PSICA subscales and the CSAI-II Anxiety subscales were consistent with the assumption that athletes with less psychological skills have higher anxiety. On the other hand, the Confidence subscale in the PSICA had a high correlation with the Confidence subscale (r=.53) in the CSAI-II, but low correlations with the Cognitive Anxiety (r=-.07) and Somatic Anxiety 29