IS THE POTENTIAL BEING FULLY EXPLOITED? ANALYSIS OF THE USE OF EXPLORATORY FACTOR ANALYSIS IN MANAGEMENT RESEARCH: YEAR 2000 TO YEAR 2006 PERSPECTIVE.

Transcription

1 IS THE POTENTIAL BEING FULLY EXPLOITED? ANALYSIS OF THE USE OF EXPLORATORY FACTOR ANALYSIS IN MANAGEMENT RESEARCH: YEAR 2000 TO YEAR 2006 PERSPECTIVE. Anant R. Deshpande University of Texas Pan American Edinburg, Texas, USA Jesús Tanguma University of Texas Pan American Edinburg, Texas, USA ABSTRACT Exploratory factor analysis is a widely used multivariate, interdependence technique which deals with summarization of variables. The use of exploratory factor analysis presents a complex challenge because the application of the technique requires several methodological considerations. This paper identifies critical issues involved in the use and reporting of exploratory factor analysis in management studies. Identification and analysis of 24 articles, which used exploratory factor analysis and published in Academy of Management Journal from year 2000 to 2006, was carried out. General findings of the study revealed flaws in reporting of exploratory factor analysis. Recommendations are offered to the management researcher for better implementation of exploratory factor analysis. INTRODUCTION Factor analysis is a technique which has generated a great deal of interest in the literature (Baird, 1987; Gorsuch, 1974; Holzinger & Harman, 1939; Rummel, 1967; Stewart 1981). Factor analysis is commonly applied by researchers from fields such as psychology, social and behavioral sciences, marketing, finance, and education, to name a few (Ford, MacCallum & Tait, 1986). However, the use of factor analysis has been limited in management literature. Researchers in management have been accused of not being able to blend together the concepts of theory testing and construction (Hughes, Price & Marrs, 1986). Factor analysis is an interdependence technique which analyzes relationships, or more specifically, correlations between different sets of variables in question The contribution of the study to the existing literature on exploratory factor analytic studies in management is threefold. First, the study identifies the key issues of exploratory factor analysis (EFA) which are important from a statistical viewpoint. Second, this study draws attention to the fact that this technique has not been fully utilized by the management researchers. This objective 805

2 is achieved by analyzing all the issues of a leading management journal between years 2000 to Third, the study offers guidelines to improve the usage of exploratory factor analysis and avoid some of the common mistakes in the application of exploratory factor analysis in management research. An Overview of Factor Analysis The purpose of this section is to provide a foundation on factor analysis. Factor analysis is a valuable theory building tool (Hair, Anderson, Tatham & Black, 1998). Factor analysis helps in the identification of the structure of variables of common interest (Srinivasan, Abeele & Butaye, 1989). That is, it helps to reduce the number of observed variables into new variables or factors (Velicer & Jackson, 1990). Factor analysis, depending on the goals of the researcher, can be classified into exploratory and/or confirmatory. Exploratory factor analysis is appropriate when the purpose is to identify the unknown underlying dimensions of a data set (Acito & Anderson, 1980; Baird, 1987; Rummel, 1967). However, if the purpose is hypothesis testing, then confirmatory factor analysis is appropriate (Hughes et al., 1986). Considerable debate exists in literature about adoption of the EFA or the CFA (Hurley, Scandura, Schriesheim, Brannick, M., et al., 1997). The researcher should clearly delineate the purpose of the study at the outset. There have been studies to support the fact that when the purpose of the researcher is to build theory or to analyze which variables fall under a construct, then the use of exploratory factor analysis is recommended (Hair et al., 1998; Hurley et al., 1997; Stewart, 1981). Stated differently, exploratory factor analysis is a means by which latent variables or factors are discovered and tentative relationships between/among them are explored (Loehlin, 1998). Since, the focus of the paper is on exploratory factor analysis, the discussion will be limited to exploratory factor analysis. The importance of confirmatory factor analysis is widely documented in the literature (Hair et al., 1993, Hughes et al., 1986; Shah & Goldstein, 2005) and a discussion of it is beyond the scope of this paper. The field of management involves complex constructs such as Total Quality Management, plant or firm performance, effectiveness of organization, and environmental measures, to name a few. EFA can give the researcher a concise picture of the variables which load under a particular construct. Thus, EFA can be a crucial step for management researchers as it gives an insight into possible complex relationships which might not have been understood before (Hair et al., 1998). Management literature is a diverse field consisting of areas such as organizational behavior, organizational theory, human resource development, and strategic management. Use of different multivariate techniques such as Hierarchical regression, cluster analysis, discriminant analysis, and factor analysis is common in management research ( Janssen & Yperen, 2004; Combs & Skill, 2003; Skaggs & Huffman, 2003). However, EFA is different from other multivariate techniques. The main difference between factor analysis, which is an interdependence technique, and other dependence techniques such as multiple regression, cluster analysis, and canonical correlation, is the presence of dependent or independent variables in the dependence techniques (Hair et al., 1998). For example, in multiple regression there is one dependent or criterion variable and a set of independent or predictor variables. However, in the case of factor analysis 806

3 there is no dependent or independent variable. In EFA, there is a set of original variables provided by the research question (Hair et al., 1998; Pedhazur & Schmelkin, 1991). The role of EFA is that of identification of factors, which might exist for different components. In addition, EFA also helps in data summarization and data reduction of the original variables with minimum loss of information (Hair et al., 1998). One of the misconceptions about factor analysis is that it analyzes the data for the researcher. However, this is not true. It is fair to say that factor analysis is as good as the data. According to the adage garbage in, garbage out, factor analysis simply helps to identify those variables that indicate a particular construct (Churchill, 1979; Hair et al., 1998). From a management perspective, the researcher needs to be very specific about the selected variables. For instance, consider the example from management research about the construct of spirituality in organization. Research might identify variables such as perceptions about God, trust in other employees, and commitment resulting due to enhanced measures for spirituality for organization. Even though some of the variables might represent the construct of spirituality, this might not actually be the case. For example, variable trust might also fall under another construct, honesty. Thus, it becomes the duty of the researcher to verify all the variables and use factor analysis only as a means for identification purposes and not use it for prediction purposes. Because EFA does not give researchers any insight into the identification of dependent or independent variables, the technique has been widely neglected (Stewart, 1981;Ehrenberg & Goodhart, 1978). Furthermore, the availability of computer programs such as SPSS has significantly increased the lack of application of methodology or theory of factor analysis (Stewart, 1981). Researchers have erroneously started to expect computers to figure out the factor solution. Researchers should clearly understand that decisions made at each stage of the EFA should be given thoughtful consideration. The use of factor analysis is common for developing different scales (Hair et al., 1998; Gorsuch, 1997). It provides useful insight for other dependence techniques to be used later in the study by the researcher. The use of factor analysis in published research has often been controversial; some researchers wrongly use factor analysis for the purpose of clustering (Stewart, 1981). According to Stewart (1981), confusion exists between the terms factors and clusters. If the research problem consists of data summarization and identifying structures or data reduction, then the appropriate technique is factor analysis. However, if the research problem consists of grouping persons or objects which are required to be grouped together in particular clusters, then the appropriate technique is cluster analysis (Hair et al., 1998). In other words, persons or objects under one particular cluster would be more similar to one another than persons or objects from another cluster (Hair et al., 1998). Another misconception about the exploratory mode of factor analysis is that it tests the hypothesis of the researcher (Hughes et al., 1986). The exploratory mode of factor analysis is limited in the sense that it helps researchers identify theoretical constructs rather than test their hypotheses (Baird, 1987; Hughes et al., 1986). 807

4 Critical Issues in the Use of Design of Exploratory Factor Analysis There are many important issues to consider when using EFA, as documented in the literature. Hair et al. (1998) mention that some of the basic decisions to be considered by a researcher in designing exploratory factor analysis include: clearly mentioning the objectives for carrying out exploratory factor analysis, fulfilling the sample size requirement, effective consideration of the type of variable, the number of variables used in the study, type of rotation used in factor analysis, factor scores interpretation, and validity and reliability issues (Hair et al., 1998). A study by Ford, MacCallum and Tait (1986) identifies different issues such as type of factor model to be used, the rotation method to be used, the factor loadings, and the number of factors to be identified as important (Ford et al., 1986). Stewart (1981) mentions that issues involving interpretation of factors obtained, modes of factor analysis, type of rotation used, extraction of factors, and analysis of correlation matrices are some of the key issues in factor analysis. Cooper (1982) identifies the type of rotation used and methods such as component analysis and common factor analysis, for extracting the factors as some of the key issues in reporting exploratory factor analysis. Merrifield and Cliff (1963) mention issues such as development and interpretation of hypotheses, use of different modes of factor analysis, method of rotation of factor analysis as important. Glass and Taylor (1966) mention that issues such as number of factors to be extracted, and the type of rotation method are some of the key issues in exploratory factor analysis. Cattell and Sullivan (1962) mention that some of the important issues of factor analysis are: 1) selection of communalities, 2) methods for factor extraction, and 3) type of rotation used. Chatterjee, Jamieson and Wiseman (1991), mention that one of the important issues which deserves the attention of researchers is the presence of sensitive data points and their influence on eigenvalues, loadings and factor scores. As mentioned by Stewart (1981), a range of techniques are available in determining the suitability of factor analysis. One such method is analysis of the correlation matrix. For instance, the study by Stimpert and Duhaime (1997) provides the correlation matrix table. Such matrix indicates the dimension of relatedness and 25 underlying variables. Looking at the correlation matrix, significant correlations are observed at the significance level; hence, factor analysis can be applied. In addition, two other measures exist to indicate the correlation between variables: Bartlett s test of sphericity and mean sampling adequacy (MSA) sample (Hair et al., 1998). Bartlett s test of sphericity indicates the probability of the presence of at least a few significant intercorrelations between variables. The Mean sampling adequacy test gives researchers information about the degree of correlations present between the variables (Hair et al., 1998). To summarize the above mentioned literature, researchers from different fields such as psychology, education, and business have reached a general consensus that some of the key issues in exploratory factor analysis which deserve a great deal of attention include: 1) mode of the factor analysis, 2) method used in the extraction of the factors, and 3) interpretation of the factors. Critical issues and recommendations for management researchers are presented in Table 1. These critical issues will further serve as a basis for evaluation against which the studies reported in a prominent academic journal, Academy of Management Journal, during the period 2000 to 2006 will be compared. 808

5 As mentioned above, some of the issues such as mode of factor analysis, method of factor extraction and number of factors to be retained become very important. In addition, the issues of sample size and validity should be given due consideration by the researcher. Consequently, the researcher is faced with numerous challenges when using exploratory factor analysis. The following section presents an in depth analysis of exploratory factor analysis studies published in the Academy of Management Journal (AMJ). The reporting of exploratory factor analysis in this prominent journal will provide an insight on how researchers have pursued exploratory factor analysis in the field of management from the year 2000 to the year This journal was selected because it covers a broad range of areas in management such as strategy, operations management, organizational behavior, organizational theory, human resource development and so forth. A wide array of multivariate techniques, such as canonical correlation analysis, structural equation modeling and multiple regression techniques, are commonly seen in the articles published in the journal. Because of the popularity of AMJ, the analysis of articles published in the journal which use EFA, should provide a good perspective of the popularity of exploratory factor analysis. METHODOLOGY The Academy of management Journal typically publishes six issues every year. A manual check of each issue in the Academy of Management Journal (AMJ) was carried out to check if the particular article used exploratory factor analysis rather than the use of a specific keyword in selection of articles adopting exploratory factor analysis (EFA) techniques. This approach is similar to one suggested by Shah et al. (2006). The most recent year was 2006 and the most recent issue was number four. For a period of six years, a total of 24 journal articles which EFA were published. A maximum number (eight) of journal articles using exploratory factor analysis were found in the year None of the published articles during 2006 have used EFA. The critical issues mentioned in the previous sections were used to perform the review. Exploratory factor analysis, if used appropriately, can give the researcher an exploratory view of the variables and insight into theory building. Exploratory factor analysis can be used as a guiding force to determine the variables underlying the construct. Researchers in management are commonly using the exploratory factor analysis technique. Given the number of published articles which use EFA and the lack of consensus as to what to report on studies which used EFA, the purpose of this paper is twofold: to focus on applications and misapplications of factor analysis in management research and to analyze the reporting of exploratory factor analysis in the management literature. RESULTS The implementation of exploratory factor analysis becomes very important in order to get correct results. One of the problems hampering the research is the contradictory views presented in the literature about the use of different areas of EFA. Lack of proper guidelines in existing literature about the critical issues and the ambiguity of selection of exploratory factor analysis in certain circumstances have contributed to a downfall of this multivariate technique. Researchers are faced with many challenges while employing exploratory factor analysis. Consequently, the researchers should pay careful attention to the critical factors identified in the study. Some of the 809

6 other critical issues also include the use of principal component analysis or common factor analysis as a suitable method for factor extraction, sample size considerations and so forth as discussed in the earlier sections. In the Year 2000, two studies were identified to have reported using exploratory factor analysis. If performed adequately, the rotation of factors gives the researcher theoretically meaningful factors. Both studies rotated the factors. However, the question whether to use the orthogonal or oblique rotation still remained. For instance, the study by Mitchell, Smith, Seawright, and Morse (2000) was exploratory in nature and support was sought for the cognitive model which has roots in theories of social cognition, information processing, and expertise. The authors used the Varimax rotation (VARIMAX) which is one type of orthogonal rotation in which the factors are not correlated. Since the nature of the study was exploratory, oblique rotation should have been used (Hair et al., 1998; Spicer, 2005), which allows the factors to be correlated. The study performed by Aulak, Kotabe and Teegen (2000) is similar. This study tries to explore the performance of the firm and develop a framework for export strategies. The Varimax rotation is used instead of using an oblique rotation. For instance, the factors, such as marketing standardization and cost leadership, might have some correlation as the market standardization involves product positioning which has some impact on implementing lower cost than competitors. Hence, the management researcher should consider using the oblique rotation. Thus, allowing the factors to be correlated. The issue of reporting of factor loadings, which represents the amount of correlation shared by an item and the factor, still remains a reporting problem. Researchers seem not to mention what factor loadings they consider significant. For instance, Lovelace, Shapiro and Weingart (2001) do mention the importance of high loadings but do not mention what loadings they considered significant to retain in the solution. The case with Chen, Choi and Chi (2002) is similar. However, studies such as Combs and Skill (2003) and Skaggs and Huffman (2003) did mention that the items with loading greater than 0.45 and 0.40, respectively, were retained for analysis. In 2004, two studies by Richard, Barnett, Dwyer and Chadwick (2004) and Janssen and Yperen (2004) reported that factor loadings above.40 were considered significant. However, none of the studies mentioned the importance of selection of factor loadings with regards to sample size. Hair et al. (1998) mention that for specific sample size there is a corresponding significant factor loading. Thus, the reporting of factor loading using sample size considerations seems widely neglected by the management researcher, as can be seen by review of the above mentioned management articles (Richard et al.,2004; Janssen & Yperen, 2004; Combs & Skill, 2003; Skaggs & Huffman, 2003). There is another serious issue, which is the issue of practical significance associated with interpretation of the factors (Hair et al., 1988). Consequently, the evaluation of factor loadings should be done at stricter levels. Also the issues of statistical significance should be given considerations. As the sample size gets very large, the probability of any effect becoming significant increases (Hair et al., 1998). The number of variables should also play an important role in selecting the factor loading to be considered significant. This is important because as the researcher moves from one factor to other factors, the unique and error variances begin to start appearing. This indicates that an adjustment in level of significance is needed (Hair et al., 1998). 810

7 The reporting of validity and reliability issues is another significant problem and considered for discussion here as it helps a researcher to gauge the generalizability of the factor solution obtained. A solution may be reliable but not valid (Hair et al., 1988; Pedhazur & Schemelkin, 1991). None of the studies reviewed addressed the issue of validity. This is a serious concern as overfitting of studies might result. One recommendation to researchers would be to split the sample and perform validation tests on the sample. However, researchers seem content to report the Cronbach s alpha value as a measure of internal consistency. For instance, a study by Raja, Johns and Ntalianis (2004) report that the reliability for two factors extracted is 0.79 and 0.72, respectively. In addition, Gibson and Birkinshaw (2004) mentioned that the measure of internal consistency is Thus, researchers in general seem to meet the criteria suggested by Spector (1992), for the values of Cronbach s alpha to be above Another issue, which seems to be widely neglected by researchers, is the reporting of eigenvalues. In addition, reporting of the criterion used for factor extraction need also to be addressed by the researchers. Only one study by Lewis, Welsh, Dehler and Green (2002) partially mentioned the use of Scree plot as one of the criterion for extraction of factors. Some of the other reporting issues found in general in all the articles were the lack of reporting of the Bartlett s test of sphericity and the Mean sampling adequacy measure. DISCUSSION Some of the common patterns which seem to have emerged from the analysis include: (1) no rationale was used in reporting the factor loadings to be considered significant; (2) lack of adequate sample size considerations, (3) inadequate consideration of the modes of factor analysis. Management researchers also seem to neglect other important issues such as lack of rationale in selecting the type of factor analysis technique used, criterion for selection of rotation of factors, and the reporting of validity and reliability studies. Reliability measures such as the measure of internal consistency or Cronbach s alpha values seem to be popular with researchers and are adequately reported. Most of the studies reported the value of Cronbach s alpha greater than This measure provides the researcher with information about homogeneity of items. Furthermore, one recommendation for the management researcher would be to also report other measures such as item to total correlations in their analyses. The majority of the researchers include correlation matrix, which is a good practice. Based on the analysis of correlation between variables, the implementation of the factor analysis technique can be justified (Hair et al., 1998). A common observation among the above mentioned studies was a problem with the selection of oblique or orthogonal rotation procedures. Researchers continue to favor orthogonal rotation procedures. The decision of the researchers to use orthogonal or oblique solution should be solely based on literature. Researchers should carefully review the literature on the variables under consideration and then make a decision whether factors ought to be correlated. As mentioned above, another glaring discovery was no mention of the validity issues. A recommendation to researchers would be to perform validity analysis. This can be achieved by either splitting the sample or selecting the cases randomly and then performing the analysis. This further helps in ensuring the generalizability of the study. 811

8 CONCLUSIONS AND FUTURE RESEARCH The review of the studies reported in the leading academic management journal indicated the popularity of exploratory factor analysis. However, implementation of the technique left much to be desired. If utilized adequately, exploratory factor analysis can be a very good tool for researchers. Reporting issues should be given adequate considerations. The current study has its limitations. Only one leading academic journal was evaluated. However, considering the fact that this is a top-rated journal, this study was worthwhile. Twenty-four articles were found to be either using the exploratory factor analysis as a sole technique or in combination with other multivariate techniques such as multiple regression, discriminant analysis and so forth. Future studies should include an analysis of other leading management journals for the same time frame. This will ensure generalizability of the results. It is firmly believed that the analysis of the critical issues and the recommendations presented in the study will improve the collective understanding of exploratory factor analysis and its implementation. This study can also provide guidance to researchers to analyze their decisions in implementing exploratory factor analysis. REFERENCES Provided upon request from Anant R. Deshpande: Hardeshpande@utpa.eduH 812

9 Table 1 Critical Issues and Recommendations for Management Researcher Issues Types Description/Recommendations References Missing data Sample size Strongly affects the interpretation of results. The recommended sample size to perform an EFA is 100 or larger. Use imputation methods such as listwise and imputation methods and choose based on stability of results. Analysis should not perform an EFA if the sample size is less than 50 observations. Hair et al., 1998; Martens, 2001; Spicer, Hair et al., 1998; Spicer, Mode of factor analysis R type and Q type factor analysis R type factor analysis: Used for identification purposes of latent variables. Reliability is an issue with Q type factor analysis studies The use of Q type factor analysis is also advocated when the number of observations is less than the number of variables. Jobson, 1991; Johnson, 1970; Ketchen & Shook, 1996; Martens, 2001; Stewart, Q type factor analysis: Used when analysis involves people in groups, as compared to variables. One of the drawbacks of Q type factor analysis is that it is limited to small sample sizes and becomes hard to manage with large sample sizes. Method of factor extraction Common Factor Analysis, Principal Component Analysis, Maximum Likelihood Estimate, Image Analysis, Minimum Residual analysis and Canonical Factor Analysis Most common types used: Common Factor Analysis; Principal Component Analysis. Use of common factor analysis over principal component is recommended if importance is provided to knowledge perspective in research. Results indicate that the number of factors and the method of rotation is of much more importance Acito & Anderson, 1980; Hair et al.,1988; Jolicoeur, 1984; Stewart, 1981; Velicer & Jackson, Issues Types Recommendations References Type of Orthogonal rotation and When the factors axes are at an Brown, 2001; Darton, 813

10 rotation oblique rotation represent the two types of rotation available. The nature of the research should determine the type of the rotation to be used. The solution obtained by oblique rotation are more parsimonious than those obtained by orthogonal rotation. angle of 90 degrees or the factors are uncorrelated, then the factors are said to be orthogonally rotated. The orthogonal rotation is appropriate for data summarization and, on the other hand, the oblique rotation is recommended for exploratory purposes. 1980; Martens, 2001; Merrifield, Philip & Cliff, 1963; Spicer,