LISREL analyses of the RIASEC model: Confirmatory and congeneric factor analyses of Holland's self-directed search

Bond University From the SelectedWorks of Gregory J. Boyle 1992 LISREL analyses of the RIASEC model: Confirmatory and congeneric factor analyses of Holland's self-directed search Gregory J. Boyle Sergio Fabris Available at: https://works.bepress.com/greg_boyle/140/

1 Lisrel Analyses of the RIASEC Model: Confirmatory and Congeneric Factor Analyses of Holland s Self-Directed Search Gregory J. Boyle and Sergio Fabris

2 Abstract A sample of 401 apprentice plumbers was administered the Australian version of Holland's [(1977) Self-Directed Search: A guide to educational and vocational planning. Palo Alto, CA: Consulting Psychologists Press] Self-Directed Search (SDS), in an investigation of the construct validity of the multidimensional interest inventory. Both exploratory (iterative principal factoring with oblique simple structure rotation), as well as LISREL confirmatory factor analyses (CFA), provided only partial support for the six-factor RIASEC typological model on which the SDS instrument was structured. Indeed, only one RIASEC factor (Artistic) was supported unequivocally from the exploratory factor analysis while the CFA statistics indicated a poor fit overall of the data to the RIASEC model. More specific LISREL congeneric factor analyses for each RIASEC dimension, however, provided tentative support for five of the six RIASEC themes. Moderately acceptable Adjusted Goodness of Fit indices, and Root Mean Square Residual estimates were obtained in each instance, except for the Realistic dimension, wherein the fit of the empirical data w the congeneric model was not supported. A further finding was the predominance of the RSE summary code, compared with the predicted summary code for a plumber which is REI (U.S.A.) or REC (Australia). On the basis of the present findings, therefore, Holland's RIASEC model and SDS instrument appear to require extensive revision before being suitable for use in the Australian context. Holland's (1977-1979, 1985) RIASEC theory of vocational choice proposed that individuals could be classified as predominantly Realistic (R), Investigative (I), Artistic (A), Social (S), Enterprising (E), and Conventional (C).

3 Likewise, it was asserted that occupational categories could also be categorized as corresponding to a combination of these six "personality" types. Predictions regarding vocational choice could therefore be based on a matching of occupational and RIASEC profiles. Holland's (1977) Self-Directed Search (SDS) was structured in terms of the six-factor RIASEC model. However, perusal of the extant literature indicates that the construct validity of this model has not been subjected to critical scrutiny. Exploratory factor analytic (EFA) studies (cf. Boyle, 1988; Kline, 1987) of the SDS using a homogeneous sample (e.g. Rachman, Amernic & Aranya, 1981) have provided some support for the structure of the RIASEC model. Rachman et al. reported that the structure of the SDS instrument as a whole was quite clear, and that most of the items which were supposed to measure particular RIASEC personality types, tended to form unidimensional scales, as expected. The Australian version of the SDS has been subjected to EFA investigations. For instance, Lokan (1988) conducted a series of principal components analyses similar to those carried out by Keeling and Tuck (1982) in New Zealand. Using a sample of senior secondary school students, Lokan reported four of the six dimensions emerged for females (R, A, I, and C), whereas only three components were obtained for males (R, I, and A). The construct validity of the RIASEC model has not been addressed adequately (cf. Campbell, 1985). Consequently, the main aim of the present study was to test the adequacy of the RIASEC model using a combination of exploratory, congeneric and confirmatory factor analyses (CFA).

4 Method Subjects and procedure The sample comprised apprentice plumbers in their first, second and third years of training at the Royal Melbourne Institute of Technology, and the Northern Metropolitan College of TAFE. The sample also included fully qualified plumbers, working at randomly selected building sites in the central business district of Melbourne. Of the 401 individuals, 94 were first year apprentices, 113 were second year, 78 were third year, and 116 were qualified plumbers. The vast majority of EFA studies reported in the literature have employed orthogonal rotation methods (usually principal components, or iterative principal factoring, with Varimax rotation). Such approaches are potentially problematic. Using exploratory (unrestricted) methods of factor analysis results in arbitrary, datadriven factor solutions which merely conflate theory. Orthogonal methods assume that the factors are independent. However, there is considerable literature pertaining to overlapping dimensions (cf. Boyle, Stanley & Start, 1985; Dorans, 1977; Gilliland, 1980). Moreover, the Pearson product-moment correlation coefficient is often computed from responses measured on dichotomous or Likert-type ordinal scales. Yet, the underlying assumptions (normality of distribution and homogeneity of variance) are usually ignored. By using product-moment estimates for dichotomous or ordinal variables, instead of the less biased tetrachoric/polychoric

5 Table 1 Intercorrelation matrix for factor analysis (N = 401) Variable 4 7 10 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 2 26 26 25 36 IS 5 18 14 24 21 07 22 7 73 17 20 36 09 22 36 16 49 36 22 29 19 38 45 30 10 02 25 28 35 12 34 11 15 19 22 13 49 27 39 13 12 12 29 18 10 16 22 33 28 16 01 53 13 14 01 13 25 49 17 10 64 44 14 19 03 15 21 17 29 22 36 12 33 47 42 15 01 08 13 18 16 22 13 42 51 43 34 16 21 19 24 01 13 30 17 22 02 10 39 24 26 09 17 10 24 19 09 18 24 29 30 10 12 30 29 23 15 14 37 18 09 04 27 04 39 23 45 19 16 37 23 48 10 18 38 39 19 25 05 18 29 22 25 22 37 29 20 22 19 25 20 21 40 35 20 03 13 65 10 30 10 15 16 23 21 21 12 16 28 37 25 26 21 18 17 25 09 13 43 24 34 20 14 24 32 21 41 29 18 22 10 36 18 21 25 26 31 26 21 18 20 28 20 19 09 23 51 32 17 39 23 08 01 29 20 44 11 15 32 15 48 16 32 56 24 15 47 29 24 07 14 15 21 16 31 21 13 17 17 24 20 13 29 27 49 15 30 33 39 25 15 05 17 24 22 28 27 35 34 15 16 25 24 17 21 34 31 50 26 36 41 53 26 19 14 20 16 15 37 19 21 18 21 23 17 08 01 37 26 27 27 45 26 18 29 27 06 21 08 09 13 25 43 25 27 25 17 29 18 17 19 20 33 23 36 14 27 32 27 28 38 32 28 08 08 20 04 37 22 39 12 20 21 34 07 07 29 24 50 25 07 36 25 59 28 35 36 30 29 14 09 12 09 30 28 25 30 18 14 22 25 21 22 30 34 49 13 34 30 34 51 58 37 38 30 05 15 10 10 32 33 29 37 Ml 24 23 24 24 29 24 37 38 53 18 33 32 37 46 61 33 51 41 73 Correlations are reported to two decimal places only. Decimal points are omitted. Correlations > 0.10 are significant at the 5% level; correlations > 0.14 are significant at the 1% level, or better.

6 estimates, significant bias is introduced inadvertently in to the subsequent analyses (cf. Jöreskog & Sörbom, 1988). As Rowe and Rowe (1992) pointed out, failure to recognize the measurement and distributional properties of response variables, amounts to "an undisciplined romp through a correlation matrix (Hendrickson & Jones, 1987, p. 105)". Hence, claims about substantive knowledge may often be prefaced largely on statistical artifact. The 30 x 30 matrix of product-moment intercorrelation coefficients was computed via PRELIS (Jöreskog & Sörbom, 1986), and congeneric factor analyses via SIMPLIS (using a two-stage least squares method of parameter estimation, followed by a maximum-likelihood procedure) (Jöreskog & Sörbom, 1987) and LISREL 6 (Jöreskog & Sörbom, 1986) were undertaken for each RIASEC dimension. Finally, an overall CFA of the best 18 items for the SDS (best three items per RIASEC theme) was carried out using the full LISREL 7 statistical package, (using the maximum-likelihood estimation option (Jöreskog & Sörbom, 1989). Instrument The SDS consists of an Assessment Booklet and an Occupations Finder (Holland, 1977-1979, 1985). The Assessment Booklet consists of six scales labelled: Occupational Daydreams (up to 8 self-identified occupations can be listed), Activities (11 items for each of the six RIASEC personality themes), Competencies (11 items for each of the six RIASEC themes), Occupations (14 items for each of the six themes), and two Self-Estimates (two 7-point scales for each of the six types). Except for Occupational Daydreams, all of the scales are

7 used to calculate the total score for each RIASEC theme (each total score equals the sum of the five raw scores). Results and Discussion The 30 x 30 intercorrelation matrix (five variables for each of the six RIASEC categories-see Table 1) served as the starting point for the factor analysis. Examination of the eigenvalues (latent roots) for the unrotated principal components suggested that six factors could be extracted legitimately, on the basis of the Scree test (cf. Hakstian, Rogers & Cattell, 1982). An iterative principal axis factoring procedure starting with the lower-bound communality estimates (SMCs) was employed (only 14 iterations were required to reach convergence of communality estimates). Perusal of the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (an index of the observed versus partial correlations), indicated that the correlations between variables were appropriate for an EFA to be conducted. The KMO is defined algebraically in Norušis (1985, p. 129). In addition, Bartlett's Test of Sphericity (see Norušis, p. 128) indicated that the correlation matrix was not an identity matrix, and therefore that it was suitable for subsequent factor analysis. The direct Oblimin factor pattern solution converged in 21 iterations, suggesting a moderately stable factor pattern solution (the smaller the number of iterations, the more reliable is the solution). With the SPSSX δ (delta) shift parameter set at zero, the ±0.10 hyperplane count for the principal axis solution was 48.89% (cf. Cattell, 1978; Gorsuch, 1983), indicating only moderate approximation to simple structure of the final rotated solution (Table 2).

8

9 Factor I (27.4% of the unrotated principal components variance) loaded predominantly on both the E and C dimensions, as a single entity. Factor 2 (8.3% of variance) exhibited sizeable loadings on three of the I-variables, as well as on two of the E variables and one S variable. Factor 3 (7.2% of variance) loaded on three of the R and two of the I-variables. Factor 4 (6.6% of variance) loaded appreciably only on two of the R variables. Factor 5 (5.1% of variance) exhibited loadings on I, Sand E themes, while Factor 6 (4.0% of variance) loaded clearly on all five of the A variables. Hence, only the A theme was reproduced cleanly in accord with Holland's (1977) RIASEC typological theory. The intercorrelations of the six extracted and rotated factors are shown in Table 3. As can be seen, many of these correlation coefficients are of moderate magnitude, suggesting a lack of independence between the six RIASEC dimensions. Given the uncertainty surrounding the EFA, a LISREL CFA was also carried out. Use of the CFA enabled statistical model testing, unlike the traditional data-driven, theory-conflating EFA approaches (cf. Anderson, 1987; Bentler, 1985; Long, 1983). The 30 x 30 intercorrelation matrix Table 3. Factor pattern intercorrelations Factor No. 2 4 5 6 1 2 0.40 3 0.23-0.08 4 0.08 0.26 0.07 5 0.28 0.27 0.09 0.25 6 0.21 0.27 0.15 0.04 0.23 Correlations are shown to two decimal places only.

10 Table 5. Covariances between exogenous latent traits (<I> matrix) RIASEC theme R A s E c Realistic Investigative 0.19 Artistic 0.17 0.42 Social 0.14 0.29 0.32 Enterprising 0.13 0.32 0.31 0.33 Conventional 0.05 0.21 0.15 0.19 0.28 Covariances are shown to two decimal places only. served as the starting point for the subsequent LISREL maximum-likelihood parameter estimation, as well as the resulting goodness of fit indices (Chi-Square or Χ 2, Goodness of Fit Index or GFI, Adjusted Goodness of Fit Index or AGFI, and Root Mean Square Residual or RMR)* (cf. Jöreskog & Sörbom, 1986; Olsson, 1979; Poon & Lee, 1987). The measurement model is expressed algebraically as: X= Ax+ b

11 such that the observed variables and 8SQ items are represented by the xs, and the latent variables are represented by the es, respectively. The vector of measurement errors in the x variables is represented by 8 (cf. Cuttance & Ecob, 1987). The corresponding equation for the covariance matrices reported below is given as: L=AlPA'+Bo where A represents the matrix of loadings for the latent traits (RIASEC themes), cji represents the matrix of covariances between the latent traits, and 8 0 represents the matrix of error variances and covariances among the x variables (five variables measuring each RIASEC theme). For the CFA of the best 18 SDS variables (as determined from the two-stage least squares standardized regression equations), the GFI was found to be 0.826, the AGFI was 0.752, while the RMR was now 0.080 (Table 4). Cuttance (1987, p. 260) indicated that in general, acceptable models have an AGFI index of 0.8 or higher. Accordingly, the present goodness of fit results suggested some inadequacy of the six-factor RIASEC model. The corresponding phi matrix of covariances for the RIASEC variables is presented in Table 5. Clearly, the six RIASEC dimensions *GFI/AGFI statistics do not depend on sample size, in contrast to the Χ 2, wherein most models are rejected when the sample size is large. The AGFI and RMR are the preferred indicators of goodness of fit.

12 Table 6. Congeneric factor models for RIASEC themes RIASEC themes (x variables) Standardized (A,) LISREL estimates (ML) Parameter value 0.71 0.60 0.54 0.50 Standard error Significance of 1-value Realistic Ql 0.41 0.00 <0.01 0.17 Q2 0.64 0.25 <0.01 0.40 Q3 0.39 0.19 <0.01 0.15 Q4 0.62 0.24 <0.01 0.38 Q5 0.63 0.24 <0.01 0.39 Coefficient of determination for x variables = 0.631. Goodness of fit statistics: GFI = 0.926; AGFI = 0.778; RMR = 0.091 Investigative Q6 0.64 0.00 <0.01 0.41 Q7 0.65 0.11 <0.01 0.42 Q& 0.57 0.10 <0.01 0.32 Q9 0.63 0.11 <0.01 0.40 QIO 0.45 0.10 <0.01 0.20 Coefficient of determination for x variables = 0.709 Goodness of fit statistics: GFI = 0.957; AGFJ = 0.872; RMR = 0.057 Artistic Qll 0.76 0.00 <0.01 0.58 Ql2 0.71 0.07 <0.01 0.50 Q13 0.75 0.07 <0.01 0.56 QJ4 0.54 O.Q7 <0.01 0.30 QIS 0.61 O.Q7 <0.01 0.37 Coefficient of determination for x variables = 0.797 Goodness of fit statistics: GFI = 0.950; AGFI = 0.849; RMR = 0.049 Social Coefficient of determination for x variables= 0.718 Goodness of fit statistics: GFI = 0.987; AGFJ = 0.962; RMR = 0.032 Enterprising Q21 0.58 0.00 <0.01 0.34 Q22 0.55 0.12 <0.01 0.31 Q23 0.61 0.12 <0.01 0.38 Q24 0.65 0.12 <0.01 0.42 Q25 0.71 0.13 <0.01 0.50 Coefficient of determination for x variables = 0.746 Goodness of fit statistics: GFI = 0.965; AGFI = 0.896; RMR = 0.048 Conventional 0.56 0.31 0.48 0.82 0.89 0.79 Coefficient of determination for x variables= 0.859 Goodness of fit statistics: GFI = 0.967; AGFI = 0.900; RMR = 0.050 The statistical significance of a t-value involves the ratio of unstandardized parameter estimate to its standard error. GFI =Goodness of Fit Index: AGFI = Adjusted Goodness of Fit Index: RMR = Root Mean Square Residual. Factor loadings are shown to two decimal places only. R z were intercorrelated, suggesting significant redundancy (cf. Boyle, 1985). Some multicollinearity therefore existed (cf. Pedhazur, 1982), pointing to the need for modification of the existing instrument, at least within the Australian context.

13 Congeneric factor analyses A similar procedure was employed using a combination of PRELIS and LISREL in undertaking separate congeneric factor analyses of the RIASEC variables. The congeneric factor results are shown in Table 6. The various goodness of fit indices provided partial support for the validity of five of the six RIASEC dimensions. Only the data for the R theme failed to exhibit an adequate fit to the congeneric model. The SDS instrument should be revised by eliminating those items which account for the least amount of shared variance associated with the respective RIASEC themes. Conclusions The present findings provide only partial support for the validity of Holland's RIASEC model of personality types, as implemented in Holland's (1977) SDS instrument. Results from the EFA clearly supported the validity of the A theme, while both the E and C dimensions combined to form a single factor. Results from the LISREL CFA suggest that the fit of the data to the six-factor RIASEC model was less than satisfactory, since the AGFI was <0.8 even for the best 18 SDS variables (cf. Cuttance, 1987). Separate congeneric factor analyses were conducted to test the goodness of fit of each of the six RIASEC themes. The LISREL results indicate that the R theme was not supported, but that more acceptable AGFI and RMR indices were obtained for the remaining five RIASEC dimensions. Taken overall, the present findings suggest that the RIASEC model and the associated SDS instrument needs some refinement in order to improve the existing levels of validity (cf. Kline, 1986).

14 References Anderson, J. G. (1987). Structural equation models in the social and behavioural sciences: Model building. Child Development, 58, 49-64. Bentler, P. M. (1985). Theory and implementation of EQS: A structural equation program. Los Angeles: BMDP Inc. Boyle, G. J. (1985). Self-report measures of depression: Some psychometric considerations. British Journal of Clinical Psychology, 24, 45-59. Boyle, G. J. (1988). Elucidation of motivation structure by dynamic calculus. In Nesselroade, J. R. & Cattell, R. B. (Eds), Handbook of multivariate experimental psychology (Revised 2nd Edn). New York: Plenum. Boyle, G. J., Stanley, G. V. & Start, K. B. (1985). Canonical/redundancy analyses of the Sixteen Personality Factor Questionnaire, the Motivation Analysis Test, and the Eight State Questionnaire. Multivariate Experimental Clinical Research, 7, 113-132. Campbell, N. J. (1985). The Self-Directed Search. Test Critiques, 2, 697-706. Cattell, R. B. (1978). The scientific use of factor analysis in behavioral and life sciences. New York: Plenum. Cuttance, P. (1987). Issues and problems in the application of structural equation models. In Cuttance, P. & Ecob, R. (Eds), Structural modeling by example: Applications in educational, sociological, and behavioral research. New York: Cambridge. Cuttance, P. & Ecob, R. (1987). (Eds) Structural modeling by example: Applications in educational, sociological and behavioral research. New York: Cambridge.

15 Dorans, N. J. (1977). Factorial validity by tautology: Methodological comments on Stewart and Stewart. Multivariate Experimental Clinical Research, 3, 69-73. Gilliland, K. (1980).The interactive effect ofintroversion-extraversion with caffeine induced arousal on verbal performance. Journal of Research in Personality, 14, 482-492. Gorsuch, R. L. (1983). Factor analysis (2nd Edn). Hillsdale, NJ: Erlbaum. Hakstian A. R., Rogers, W. T. & Cattell, R. B. (1982). The behavior of number-of factors rules with simulated data. Multivariate Behavioral Research, 17, 193-219. Hendrickson, L. & Jones, B. (1987). A study of longitudinal causal models comparing gain score analysis with structural equation approaches. In Cuttance, P. & Ecob, R. (Eds), Structural modeling by example: Applications in educational, sociological and behavioral research. New York: Cambridge. Holland, J. L. (1977). Self-Directed Search: A guide to educational and vocational planning. Palo Alto, CA: Consulting Psychologists Press. Holland, J. L. (1978). The occupations finder: For use with the Self-Directed Search. Palo Alto, CA: Consulting Psychologists Press. Holland, J. L. (1979). The Self-Directed Search professional manual. Palo Alto, CA: Consulting Psychologists Press. Holland, J. L. (1985). Making vocational choices: A theory of vocational personalities and work environments. Englewood Cliffs, NJ: Prentice-Hall.

16 Jöreskog, K. G. & Sorböm, D. (1986). LISREL VI: Analysis of linear structural relationships by maximum likelihood, instrumental variables, and least squares methods. Mooresville, IN: Scientific Software. Jöreskog, K. G. & Sorböm, D. (1987). SIMPL/S: Estimating linear structural relationships using two-stage least squares. Mooresville, IN: Scientific Software. Jöreskog, K. G. & Sorböm, D. (1988). PRELIS: A program for multivariate data screening and data summarization: A preprocessor for L/SREL. Mooresville, IN: Scientific Software. Jöreskog, K. G. & Sörbom, D. (1989). LISREL 7: A guide to the program and applications. Chicago: SPSS. Keeling, B. & Tuck, B. F. (1982). The Self-Directed Search (New Zealand revision): Counsellor's manual. Wellington: New Zealand Council for Educational Research. Kline, P. (1986). A handbook of test construction: Introduction to psychometric design. New York: Methuen. Kline, P. (1987). Factor analysis and personality theory. European Journal of Personality, I, 21-36. Lokan, J. (1988). The Self-Directed Search professional manual (Australian supplement). Hawthorn: Australian Council for Educational Research. Long, J. S. (1983). Covariance structure models: An introduction to L/SREL. London: Sage. Norušis, M. J. (1985). SPSSX: Advanced statistics guide. New York: McGraw Hill.

17 Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika, 44, 443-460. Pedhazur, E. J. (1982). Multiple regression in behavioral research: Explanation and prediction (pp. 232-247). New York: Holt, Rinehart & Winston. Poon, W. Y. & Lee, S. Y. (1987). Maximum likelihood estimation of multivariate polyserial and polychoric correlation coefficients. Psychometrika, 52, 409-430. Rachman, D., Amernic, J. & Aranya, N. (1981). A factor analytic study of the construct validity of Holland's Self-directed search test. Educational and Psychological Measurement, 41, 425-437. SPSS-X user's guide, 3rd edition. Chicago: Statistical Package for the Social Sciences (SPSS).