NICK HASLAM* AND SIMON M. LAHAM University of Melbourne, Australia

Transcription

1 European Journal of Social Psychology Eur. J. Soc. Psychol. 40, (2010) Published online 8 December 2009 in Wiley InterScience ( Fast track report Quality, quantity, and impact in academic publication NICK HASLAM* AND SIMON M. LAHAM University of Melbourne, Australia Abstract Publication records of 85 social-personality psychologists were tracked from the time of their doctoral studies until 10 years post-phd. Associations between publication quantity (number of articles), quality (mean journal impact factor and article influence score), and impact (citations, h-index, g-index, webpage visits) were examined. Publication quantity and quality were only modestly related, and there was evidence of a quality-quantity trade-off. Impact was more strongly associated with quantity than quality. Authors whose records weighed quality over quantity tended to be associated with more prestigious institutions, but had lesser impact. Quantity- and quality-favoring publication strategies may have important implications for the shape and success of scientific careers. Copyright # 2009 John Wiley & Sons, Ltd. To have a large impact on their field, researchers must publish often and well. Although scientific impact is a major concern for researchers, academic institutions, and funding agencies, basic questions about how it is related to publication quantity and quality remain unanswered. Is quantity or quality the more important contributor to impact? The most influential researchers may be those who publish the most or those who publish in the best journals. Are quality and quantity positively or negatively related? They may both reflect scientific talent and creativity, or pursuing one may mean compromising the other. Do researchers adopt publication strategies that favor quality or quantity? Some researchers may publish prolifically with little regard for journal quality, whereas others may be less productive but more discriminating. Relatively little research has addressed these questions. It is widely recognized that influential researchers publish extensively in prestigious journals, but no studies have investigated whether productivity or publication quality is the more important ingredient of their success. Knowing whether impact, usually assessed using citation-based indices such as h (Hirsch, 2005) or g (Egghe, 2006), is best achieved by emphasizing quality or quantity is vitally important to researchers, and especially to those early in their careers. The relationship between publication quality and quantity is equally uncertain. Scientists may simply vary in accomplishment, the more accomplished publishing a large amount of high-quality work, so that quantity and quality are positively related (Simonton, 1991). On the other hand, they may face a choice between pursuing quality or quantity that implies a negative relationship. Quantity can be maximized by parceling one s research effort into small units, addressing marginal or piecemeal topics, and publishing in unselective journals. Quality can be maximized by producing substantial and programmatic work that addresses major questions and targets top-tier journals. Striving for quantity may therefore reduce quality, and vice versa. If quality and quantity can be traded-off in this fashion, researchers may adopt publication strategies that favor one over the other. Such strategies resemble those described by ecologists (Macarthur & Wilson, 1967), who distinguish organisms that are r- and K-strategists. r-strategists have higher reproductive rates and invest less care in each offspring, whereas K- strategists have fewer offspring and invest more in them. r-strategies favor quantity and are adapted to unstable *Correspondence to: Nick Haslam, Department of Psychology, University of Melbourne, Parkville VIC 3010, Australia. nhaslam@unimelb.edu.au Copyright # 2009 John Wiley & Sons, Ltd. Received 18 June 2009 Accepted 28 October 2009

2 Scientific quality and quantity 217 environments, and K-strategies favor quality and are adapted to stable environments. Farias (2003) likened the publication patterns of several economists to these strategies, and argued that prestigious academic departments promote K-strategies, although these may not always be most impactful. The present study systematically explored relationships among publication quality, quantity, and impact. We examined the relative contribution of quality and quantity to impact, the relationship between quality and quantity (including the possibility of a trade-off), the existence of publication styles akin to r- and K-strategies, and the link between these styles and institutional prestige, using bibliometric indices. Although such indices (e.g., journal impact factors) do not capture the multidimensional complexities of article quality (e.g., Sternberg & Gordeeva, 1996), they are widely used proxies that do not suffer the unreliability that plagues more subjective quality assessments (Cicchetti, 1991). An international sample of social/personality psychologists from a particular cohort (1996 or 1997 PhDs) was studied over a standard period (10 years) to rule out confounding variables related to academic field, historical changes in publication practices, and career stage. METHOD In mid-2008, the 1452 profiles of social/personality psychologists in the Social Psychology Network (SPN) were searched for those listing a 1996 or 1997 PhD and occupying full-time faculty positions. The search yielded 86 psychologists. The Web of Science (WoS) database was then searched for publication and citation data on each psychologist from 3 years before their PhD to 10 years after. Publication types other than articles and review articles (e.g., book reviews) were excluded. When authors shared a surname and first initial with others in the database, articles were attributed to them only after disambiguation in WoS by later initials, first names, or affiliations, and/or by information in their SPN profile or institutional homepage. A final sample of 85 authors (52 men, 33 women; 57 USA-based, 19 Europe-based, 4 Canadabased, 5 other) and 1409 articles from 237 journals was obtained after excluding one psychologist with no articles. Data were recorded on each article and each author. Article Characteristics Every article published by an author in the period of interest was coded for the journal, the journal s impact factor (IF: a mean value across the years ), article influence score (AIS: a weighted citation-based index of journal quality and relative importance for 2007), year of publication, and number of citations. Author Characteristics Information about each author s gender and PhD-granting and current institution was extracted from SPN profiles. Prestige for these institutions was assessed by position in the 2008 Shanghai Jiao Tong (SJT) world university ranking, which gives exact ranks for the top 100 institutions and broad rank bands from An index of the number of visits per month received by the author s SPN profile page was computed. Several author characteristics were derived from the publication data: number of articles (publication quantity index), mean IF and AIS of journals in which the author s articles appeared (publication quality indices), and number of citations, h-index (i.e., the highest number of articles h that have been each cited h times), g-index (i.e., the highest number of articles g that have collectively been cited g 2 times), and profile visits (indices of bibliometric and reputational impact). RESULTS Descriptive statistics are presented in Table 1. Distributions of publications, citations, and profile visits had significant positive skew so all were subjected to square-root transformations prior to all analyses. There were no sex differences on publication quantity, quality, or impact.

3 218 Nick Haslam and Simon M. Laham Table 1. Descriptive statistics for study variables Mean SD Range Quantity (number of articles) Quality Mean IF Mean AIS Impact Citations h-index g-index Profile visits/month Contribution of Quality and Quantity to Impact Table 2 presents correlations among the indices. The bibliometric impact indices (total citations, h-index g-index) intercorrelated very strongly and correlated moderately with reputational impact (profile visits). All four impact indices were more strongly correlated with publication quantity than quality. Multiple regression analyses reported in Table 3 show that although quality (assessed as IF and AIS) contributed significantly to three of these indices, quantity always made a greater contribution. Association Between Quality and Quantity Publication quality and quantity were moderately correlated (r ¼.33 [IF] &.36 [AIS], p <.001). This association was curvilinear (inverted U-shaped), consistent with the existence of a quality-quantity trade-off. A significant quadratic effect Table 2. Correlations among study indices (decimal omitted) (1) Articles 100 (2) IF (3) AIS (4) Citations (5) h-index (6) g-index (7) Profile visits p <.05; p <.001. Table 3. Summary of regression analyses for prediction of impact Dependent variable Quality index Quantity b Quality b R 2 Citations IF AIS h-index IF AIS g-index IF AIS Profile visits IF AIS p <.05; p <.01; p <.001.

4 Scientific quality and quantity 219 Table 4. omitted) Correlations between prestige of Phd-granting and current institution and indices of quality, quantity, and impact (decimal PhD institution Current institution Quantity No. of articles Quality IF AIS Impact Citations h-index g-index Profile visits p <.05; p <.01. emerged when quality was regressed on quantity (linear b ¼.41 [IF] &.44 [AIS], p <.001; quadratic b ¼.26 [IF] &.25 [AIS], p <.05). Expected quality was maximized at about 30 publications (88th percentile) and declined at higher quantities. Among psychologists with above-median productivity, quantity was unrelated to quality (r ¼.03 [IF & AIS], p >.05). When the analysis was reversed, and quantity regressed on quality, the same curvilinear effects emerged, with expected quantity peaking at about the 82nd percentile on quality and declining at higher values. By implication, quantity begins to conflict with quality at relatively high levels of both. Publication Strategy To assess associations between publication quality, quantity and impact and institutional prestige, the indices were correlated with the location of the psychologists PhD-granting and current institution on the SJT ranking (scored ¼ 6, ¼ 5, ¼ 4, ¼ 3, ¼ 2, unranked ¼ 1). Correlations presented in Table 4 show that, as predicted, institutional prestige was associated with publication quality but not quantity. Only prestige of the current institution was associated with impact. K-strategists were defined as those psychologists (n ¼ 14) with an above-median quality (IF) score and a below-median quantity score, and r-strategists as those with the opposite pattern (n ¼ 13). As predicted, K-strategists received their PhDs from more prestigious institutions than r-strategists (t(25) ¼ 2.89, p <.01) and currently worked at more prestigious institutions (t(25) ¼ 2.14, p <.05). However, r-strategists had greater bibliometric impact: higher h-index (t(25) ¼ 2.84, p <.01), higher g-index (t(25) ¼ 4.20, p <.001), and marginally more citations (t(25) ¼ 2.01, p ¼.056). DISCUSSION The present study investigated several basic questions about the relationships among publication quality, quantity, and impact in a cohort of psychology researchers. It indicates that publication quantity is a stronger predictor of impact than publication quality. Greater scientific impact, measured bibliometrically (citations, h-index, g-index) and reputationally (profile visits), was strongly associated with publishing more, and only weakly with publishing better. In part, this difference in predictive weight may reflect the greater variability of publication quantity. The range of publication totals was approximately 5 times the sample median, compared to 2.5 for the quality indices. If researchers differ more in their productivity than in the IFs of the journals in which they publish, then differences in their scientific impact will primarily reflect publication quantity. The relationship between impact and quality may also be limited because a journal s IF is only modestly related to the impact of individual articles that it publishes (Haslam et al., 2008). Article citation rates are highly skewed, most articles in high-if journals receive relatively few citations, and high-impact

5 220 Nick Haslam and Simon M. Laham articles often appear in low-if journals. Thus researchers who tend to publish in high-quality journals are not guaranteed a high level of impact. Publication quality and quantity may be differentially related to scientific impact, but they are positively related to one another. This finding implies that quality and quantity both reflect greater scientific achievement: more accomplished researchers tend to publish more articles and to do so in more prestigious journals. Publication quality and quantity are clearly not antithetical where differences among researchers are concerned. Nevertheless, there can be a trade-off between them. Publication quantity and quality were negatively associated at high levels of both, as if one was purchased at the expense of the other. In short, successful researchers may face a choice between publishing more frequently in less prestigious journals or vice versa. Such a choice or trade-off implies the possibility of systematic publication strategies, as Farias (2003) noted. A quantity-favoring publication style is analogous to an r-strategy, and a quality-favoring style to a K-strategy. Consistent with our prediction, graduates and employees of more prestigious institutions were more likely to follow a K-strategy, tending to publish in higher quality journals than their peers but not publishing more. The ecological argument that K- strategies are adapted to relatively stable environments may help to explain his tendency. Psychologists associated with more prestigious universities may enjoy greater academic resources, job prospects, and other forms of cultural capital, whereas those in less prestigious institutions face a less certain and well-supported academic environment. As a result, they may be more likely to adopt a style of publication that emphasizes quantity over quality. Interestingly, our findings indicate that this style may be effective: r-strategists were more impactful than K-strategists. Our findings may have implications for research mentoring and career development. Junior researchers are often advised to publish in the most selective and prestigious journals, but this advice may be misguided if its aim is to maximize their eventual scientific impact. If productivity outweighs quality as a predictor of career impact, then researchers should be advised that it is at least as important to publish often as it is to publish well. It may be unwise to strive to publish only in top-tier journals, as this may lead to a lower likelihood of publication and foregone opportunities to publish in less selective journals that may contribute significantly to later scientific impact. Researchers cannot always predict the eventual reception of their work by their peers, and if they restrict their publication repertoires by choosing not to publish in less prestigious outlets they may be costing themselves later impact. Nevertheless, our findings do not imply that researchers should aim for quantity above all else: articles that appear in better journals are, on average, more impactful. It is important to recognize several limitations of our work. First, its findings may not generalize to other scholarly fields, which often have different publication norms, or to other cohorts of researchers. In addition, despite the wide reach of SPN some members of the relevant cohort may have been omitted. Second, we acknowledge that research quality and impact are not reducible to bibliometric indices, although these indices are increasingly used for that purpose. Article quality, in particular, is imperfectly gauged by journal-level citation indices, and the implications of our findings about quality should not be extended beyond the bibliometric sense of the term. We do not argue that bibliometric indices represent pure or sufficient measures of a researcher s aptitude. Finally, our study did not include books and book chapters, which are significant proportions of some researchers publication output. Nevertheless, the study reveals some new complexities and tensions in academic publication within social/personality psychology. REFERENCES Cicchetti, D. V. (1991). The reliability of peer-review for manuscripts and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences, 14, Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69, Farias, J. R. (2003). What type of economist are you: r-strategist or K-strategist? Journal of Economic Studies, 30, Haslam, N., Ban, L., Kaufmann, L., Loughnan, S., Peters, K., Whelan, J., et al. (2008). What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, 76, Hirsch, J. E. (2005). An index to quantify an individual s scientific research output. Proceedings of the National Academy of Sciences, USA, 102, Macarthur, R. H., & Wilson, E. O. (1967). The theory of island biogeography. Princeton, NJ: Princeton University Press. Simonton, D. K. (1991). Career landmarks in science: Individual differences and interdisciplinary contrasts. Developmental Psychology, 27, Sternberg, R. J., & Gordeeva, T. (1996). The anatomy of impact: What makes an article influential? Psychological Science, 7,