Accumulating Knowledge through Journals. Barbara Schneider and Ann Owens University of Chicago

Transcription

1 Accumulating Knowledge through Journals Barbara Schneider and Ann Owens University of Chicago 2003

2 How do you determine what constitutes the most important and influential research reported in journals?

3 The Institute for Scientific Information (ISI) claims that it provides a comprehensive database of the world s most important and influential research (Testa, 2002) Important caveat: Comprehensive does not necessarily mean all-inclusive (Garfield, 1990)

4 The Need to Be Selective Recent citation analyses have shown that as few as 150 journals account for half of what is cited and one quarter of what is published. It has also been shown that a core of approximately 2,000 journals now accounts for about 85% of published articles and 95% of cited articles. (Testa 2002).

5 What are the Education Journals listed in the Social Science Citation Index (SSCI)*? The SSCI categorizes 152 journals into one of its three education categories (education and educational research, educational psychology, and special education) *A product of ISI

6 Journals Considered Education by SSCI (AERA journals highlighted) ACADEME-BULLETIN OF THE AAUP ACADEMIC PSYCHIATRY ADULT EDUCATION QUARTERLY ADVANCES IN HEALTH SCIENCES EDUCATION AIDS EDUCATION AND PREVENTION AMERICAN ANNALS OF THE DEAF AMERICAN EDUCATIONAL RESEARCH JOURNAL AMERICAN JOURNAL OF EDUCATION AMERICAN JOURNAL ON MENTAL RETARDATION ANNALS OF DYSLEXIA ANTHROPOLOGY AND EDUCATION QUARTERLY APPLIED MEASUREMENT IN EDUCATION BEHAVIORAL DISORDERS COGNITION AND INSTRUCTION COMPARATIVE EDUCATION COMPARATIVE EDUCATION REVIEW COMPUTERS & EDUCATION CONTEMPORARY EDUCATIONAL PSYCHOLOGY CREATIVITY RESEARCH JOURNAL CURRICULUM INQUIRY DISCOURSE PROCESSES DYSLEXIA EARLY CHILDHOOD RESEARCH QUARTERLY ECONOMICS OF EDUCATION REVIEW EDUCATION AND TRAINING IN MENTAL RETARDATION AND DEVELOPMENTAL DISABILITIES BRITISH EDUCATIONAL RESEARCH JOURNAL BRITISH JOURNAL OF DEVELOPMENTAL DISABILITIES BRITISH JOURNAL OF EDUCATIONAL PSYCHOLOGY BRITISH JOURNAL OF EDUCATIONAL STUDIES BRITISH JOURNAL OF EDUCATIONAL TECHNOLOGY BRITISH JOURNAL OF SOCIOLOGY OF EDUCATION CHILD DEVELOPMENT CHINESE EDUCATION AND SOCIETY EDUCATION AND URBAN SOCIETY EDUCATIONAL ADMINISTRATION QUARTERLY EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT EDUCATIONAL EVALUATION AND POLICY ANALYSIS EDUCATIONAL GERONTOLOGY EDUCATIONAL LEADERSHIP EDUCATIONAL POLICY EDUCATIONAL PSYCHOLOGIST

7 EDUCATIONAL PSYCHOLOGY REVIEW EDUCATIONAL RESEARCH EDUCATIONAL REVIEW EDUCATIONAL STUDIES ELEMENTARY SCHOOL JOURNAL ETR&D-EDUCATIONAL TECHNOLOGY RESEARCH AND DEVELOPMENT EUROPEAN JOURNAL OF PSYCHOLOGY OF EDUCATION EXCEPTIONAL CHILDREN FOCUS ON EXCEPTIONAL CHILDREN FOREIGN LANGUAGE ANNALS GENDER AND EDUCATION GIFTED CHILD QUARTERLY HARVARD EDUCATIONAL REVIEW HEALTH EDUCATION RESEARCH HIGH ABILITY STUDIES HIGHER EDUCATION INFANTS AND YOUNG CHILDREN INNOVATIONS IN EDUCATION AND TEACHING INTERNATIONAL INSTRUCTIONAL SCIENCE INTERNATIONAL JOURNAL OF EDUCATIONAL DEVELOPMENT INTERNATIONAL JOURNAL OF SCIENCE EDUCATION INTERVENTION IN SCHOOL AND CLINIC JAPANESE JOURNAL OF EDUCATIONAL PSYCHOLOGY JOURNAL FOR RESEARCH IN MATHEMATICS EDUCATION JOURNAL FOR THE EDUCATION OF THE GIFTED JOURNAL OF ADOLESCENT & ADULT LITERACY JOURNAL OF AMERICAN COLLEGE HEALTH JOURNAL OF APPLIED RESEARCH IN INTELLECTUAL DISABILITIES JOURNAL OF ART AND DESIGN EDUCATION JOURNAL OF COLLEGE STUDENT DEVELOPMENT JOURNAL OF COMPUTER ASSISTED LEARNING JOURNAL OF COUNSELING PSYCHOLOGY JOURNAL OF CREATIVE BEHAVIOR JOURNAL OF CURRICULUM STUDIES JOURNAL OF EARLY INTERVENTION JOURNAL OF ECONOMIC EDUCATION JOURNAL OF EDUCATION POLICY JOURNAL OF EDUCATIONAL AND BEHAVIORAL STATISTICS JOURNAL OF EDUCATIONAL AND PSYCHOLOGICAL CONSULTATION JOURNAL OF EDUCATIONAL MEASUREMENT

8 JOURNAL OF EDUCATIONAL PSYCHOLOGY JOURNAL OF EDUCATIONAL RESEARCH JOURNAL OF EXPERIMENTAL EDUCATION JOURNAL OF FLUENCY DISORDERS JOURNAL OF GEOGRAPHY IN HIGHER EDUCATION JOURNAL OF HIGHER EDUCATION JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY JOURNAL OF INTELLECTUAL DISABILITY RESEARCH JOURNAL OF LEARNING DISABILITIES JOURNAL OF LEGAL EDUCATION JOURNAL OF LITERACY RESEARCH JOURNAL OF MORAL EDUCATION JOURNAL OF PHILOSOPHY OF EDUCATION JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT JOURNAL OF RESEARCH IN READING JOURNAL OF RESEARCH IN SCIENCE TEACHING JOURNAL OF SCHOOL HEALTH JOURNAL OF SCHOOL PSYCHOLOGY JOURNAL OF SOCIAL WORK EDUCATION JOURNAL OF SPECIAL EDUCATION JOURNAL OF TEACHER EDUCATION JOURNAL OF TEACHING IN PHYSICAL EDUCATION JOURNAL OF THE LEARNING SCIENCES LANGUAGE LEARNING LEARNING AND INDIVIDUAL DIFFERENCES LEARNING AND INSTRUCTION LEARNING DISABILITY QUARTERLY MEASUREMENT AND EVALUATION IN COUNSELING AND DEVELOPMENT MENTAL RETARDATION MINERVA NEW ZEALAND JOURNAL OF EDUCATIONAL STUDIES OXFORD REVIEW OF EDUCATION PERSPECTIVES IN EDUCATION PHI DELTA KAPPAN PSYCHOLOGIE IN ERZIEHUNG UND UNTERRICHT PSYCHOLOGY IN THE SCHOOLS QUEST READING RESEARCH AND INSTRUCTION READING RESEARCH QUARTERLY READING TEACHER

9 REMEDIAL AND SPECIAL EDUCATION RESEARCH IN HIGHER EDUCATION RESEARCH IN SCIENCE EDUCATION RESEARCH IN THE TEACHING OF ENGLISH REVIEW OF EDUCATIONAL RESEARCH REVIEW OF HIGHER EDUCATION REVIEW OF RESEARCH IN EDUCATION RUSSIAN EDUCATION AND SOCIETY SCHOOL EFFECTIVENESS AND SCHOOL IMPROVEMENT SCHOOL PSYCHOLOGY INTERNATIONAL SCHOOL PSYCHOLOGY QUARTERLY SCHOOL PSYCHOLOGY REVIEW SCIENCE EDUCATION SCIENTIFIC STUDIES OF READING SECOND LANGUAGE RESEARCH SOCIOLOGY OF EDUCATION STUDIES IN HIGHER EDUCATION TEACHERS COLLEGE RECORD TEACHING AND TEACHER EDUCATION TEACHING OF PSYCHOLOGY TEACHING SOCIOLOGY TESOL QUARTERLY (TEACHERS OF ENGLISH TO SPEAKERS OF OTHER LANGUAGES) THEORY INTO PRACTICE TOPICS IN EARLY CHILDHOOD SPECIAL EDUCATION URBAN EDUCATION VOLTA REVIEW VOPROSY PSIKHOLOGII YOUNG CHILDREN ZEITSCHRIFT FUR ENTWICKLUNGS PSYCHOLOGIE UND PADAGOGISCHE PSYCHOLOGIE ZEITSCHRIFT FUR PADAGOGIK ZEITSCHRIFT FUR PADAGOGISCHE PSYCHOLOGIE

10 Educational Resources Information Center (ERIC) lists 1,016 education journals. Following are the journals considered education by ISI but not included in ERIC. A third of these journals excluded by ERIC are international journals and nearly twothirds are psychology and developmental journals.

11 Journals considered education by SSCI, but not in ERIC ACADEMIC PSYCHIATRY ADVANCES IN HEALTH SCIENCES EDUCATION AIDS EDUCATION AND PREVENTION BRITISH JOURNAL OF DEVELOPMENTAL DISABILITIES BRITISH JOURNAL OF EDUCATIONAL PSYCHOLOGY BRITISH JOURNAL OF EDUCATIONAL STUDIES CREATIVITY RESEARCH JOURNAL CURRICULUM INQUIRY DYSLEXIA EDUCATIONAL PSYCHOLOGIST EDUCATIONAL PSYCHOLOGY REVIEW ETR&D-EDUCATIONAL TECHNOLOGY RESEARCH AND DEVELOPMENT HEALTH EDUCATION RESEARCH HIGH ABILITY STUDIES INNOVATIONS IN EDUCATION AND TEACHING INTERNATIONAL INTERNATIONAL JOURNAL OF EDUCATIONAL DEVELOPMENT JAPANESE JOURNAL OF EDUCATIONAL PSYCHOLOGY JOURNAL OF APPLIED RESEARCH IN INTELLECTUAL DISABILITIES JOURNAL OF EDUCATION POLICY JOURNAL OF EDUCATIONAL AND PSYCHOLOGICAL CONSULTATION JOURNAL OF FLUENCY DISORDERS JOURNAL OF INTELLECTUAL & DEVELOPMENTAL DISABILITY JOURNAL OF INTELLECTUAL DISABILITY RESEARCH JOURNAL OF PHILOSOPHY OF EDUCATION JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT JOURNAL OF THE LEARNING SCIENCES LEARNING AND INDIVIDUAL DIFFERENCES MINERVA NEW ZEALAND JOURNAL OF EDUCATIONAL STUDIES PERSPECTIVES IN EDUCATION PSYCHOLOGIE IN ERZIEHUNG UND UNTERRICHT REVIEW OF RESEARCH IN EDUCATION SCHOOL PSYCHOLOGY INTERNATIONAL SCIENTIFIC STUDIES OF READING VOPROSY PSIKHOLOGII ZEITSCHRIFT FUR ENTWICKLUNGS PSYCHOLOGIE UND PADAGOGISCHE PSYCHOLOGIE ZEITSCHRIFT FUR PADAGOGISCHE PSYCHOLOGIE

12 There are 153 journals in both the general SSCI journal list and ERIC that are not considered education by SSCI. These, for the most part, are disciplinary based journals, such as the American Historical Review, the American Sociological Review, and Cognitive Psychology.

13 Journals in both SSCI and ERIC, but not considered education by SSCI ADMINISTRATIVE SCIENCE QUARTERLY ADOLESCENCE ALTERNATIVES AMERICAN BEHAVIORAL SCIENTIST AMERICAN HISTORICAL REVIEW AMERICAN INDIAN AND ALASKA NATIVE MENTAL HEALTH RESEARCH AMERICAN JOURNAL OF FAMILY THERAPY AMERICAN JOURNAL OF HEALTH BEHAVIOR AMERICAN JOURNAL OF HEALTH PROMOTION AMERICAN JOURNAL OF PUBLIC HEALTH AMERICAN JOURNAL OF SOCIOLOGY AMERICAN POLITICAL SCIENCE REVIEW AMERICAN PSYCHOLOGIST AMERICAN SOCIOLOGICAL REVIEW ANNUAL REVIEW OF INFORMATION SCIENCE AND TECHNOLOGY (ARIST) CHILD & FAMILY BEHAVIOR THERAPY CHILD ABUSE & NEGLECT: THE INTERNATIONAL JOURNAL CHILD WELFARE CHILDREN AND YOUTH SERVICES REVIEW COGNITION COGNITIVE DEVELOPMENT COGNITIVE PSYCHOLOGY COLLEGE & RESEARCH LIBRARIES COMMUNICATION MONOGRAPHS COMMUNICATION RESEARCH COMPUTERS IN HUMAN BEHAVIOR COUNSELING PSYCHOLOGIST CRIME AND DELINQUENCY DEATH STUDIES DEVELOPMENTAL PSYCHOLOGY DEVELOPMENTAL REVIEW APPLIED LINGUISTICS APPLIED PSYCHOLINGUISTICS APPLIED PSYCHOLOGICAL MEASUREMENT ASSESSMENT BEHAVIOR MODIFICATION CAREER DEVELOPMENT QUARTERLY ELECTRONIC LIBRARY ENGLISH FOR SPECIFIC PURPOSES EVALUATION AND PROGRAM PLANNING EVALUATION REVIEW FAMILY RELATIONS FUTURE OF CHILDREN

14 GERONTOLOGIST GOVERNMENT INFORMATION QUARTERLY HARVARD BUSINESS REVIEW HEALTH & SOCIAL WORK HISPANIC JOURNAL OF BEHAVIORAL SCIENCES HUMAN COMMUNICATION RESEARCH HUMAN DEVELOPMENT INFORMATION PROCESSING & MANAGEMENT INFORMATION TECHNOLOGY AND LIBRARIES INTELLIGENCE INTERNATIONAL JOURNAL OF AGING & HUMAN DEVELOPMENT INTERNATIONAL JOURNAL OF BEHAVIORAL DEVELOPMENT INTERNATIONAL JOURNAL OF REHABILITATION RESEARCH INTERNATIONAL LABOUR REVIEW INTERNATIONAL MIGRATION REVIEW ISSUES IN SCIENCE AND TECHNOLOGY JOURNAL OF ACADEMIC LIBRARIANSHIP JOURNAL OF ADOLESCENCE JOURNAL OF ADOLESCENT RESEARCH JOURNAL OF AMERICAN HISTORY JOURNAL OF APPLIED BEHAVIOR ANALYSIS JOURNAL OF APPLIED COMMUNICATION RESEARCH JOURNAL OF AUTISM AND DEVELOPMENTAL DISORDERS JOURNAL OF BLACK STUDIES JOURNAL OF BUSINESS AND TECHNICAL COMMUNICATION JOURNAL OF CAREER ASSESSMENT JOURNAL OF CAREER DEVELOPMENT JOURNAL OF CHILD & ADOLESCENT SUBSTANCE ABUSE JOURNAL OF CHILD LANGUAGE JOURNAL OF COMMUNICATION JOURNAL OF COMMUNICATION DISORDERS JOURNAL OF CONFLICT RESOLUTION JOURNAL OF CONSULTING AND CLINICAL PSYCHOLOGY JOURNAL OF CONSUMER AFFAIRS JOURNAL OF COUNSELING AND DEVELOPMENT JOURNAL OF CROSS-CULTURAL PSYCHOLOGY JOURNAL OF DOCUMENTATION JOURNAL OF DRUG EDUCATION JOURNAL OF EARLY ADOLESCENCE JOURNAL OF EMOTIONAL AND BEHAVIORAL DISORDERS

15 JOURNAL OF EMPLOYMENT COUNSELING JOURNAL OF EXPERIMENTAL CHILD PSYCHOLOGY JOURNAL OF FAMILY ISSUES JOURNAL OF GEOGRAPHY JOURNAL OF GOVERNMENT INFORMATION JOURNAL OF HUMAN RESOURCES JOURNAL OF INTERPERSONAL VIOLENCE JOURNAL OF LEISURE RESEARCH JOURNAL OF MARITAL AND FAMILY THERAPY JOURNAL OF MARRIAGE AND THE FAMILY JOURNAL OF MULTICULTURAL COUNSELING AND DEVELOPMENT JOURNAL OF PROFESSIONAL NURSING JOURNAL OF RESEARCH IN CRIME AND DELINQUENCY JOURNAL OF RESEARCH ON ADOLESCENCE JOURNAL OF RURAL STUDIES JOURNAL OF SECOND LANGUAGE WRITING JOURNAL OF SOCIAL ISSUES JOURNAL OF SOCIAL PSYCHOLOGY JOURNAL OF THE ASSOCIATION FOR PERSONS WITH SEVERE HANDICAPS JOURNAL OF URBAN TECHNOLOGY JOURNAL OF VOCATIONAL BEHAVIOR JOURNAL OF YOUTH AND ADOLESCENCE LANGUAGE LANGUAGE & COMMUNICATION LANGUAGE AND COGNITIVE PROCESSES LANGUAGE IN SOCIETY LIBRARY & INFORMATION SCIENCE RESEARCH LIBRARY JOURNAL LIBRARY QUARTERLY LIBRARY RESOURCES & TECHNICAL SERVICES LIBRARY TRENDS MODERN LANGUAGE JOURNAL MONOGRAPHS OF THE SOCIETY FOR RESEARCH IN CHILD DEVELOPMENT MONTHLY LABOR REVIEW MULTIVARIATE BEHAVIORAL RESEARCH NURSE EDUCATION TODAY NURSING OUTLOOK ONLINE POLICY REVIEW POPULATION RESEARCH AND POLICY REVIEW

16 PSYCHOLOGICAL ASSESSMENT PSYCHOLOGICAL REVIEW PSYCHOLOGY OF WOMEN QUARTERLY PSYCHOMETRIKA PUBLIC ADMINISTRATION REVIEW PUBLIC INTEREST PUBLIC PERSONNEL MANAGEMENT PUBLIC RELATIONS REVIEW QUARTERLY JOURNAL OF SPEECH REHABILITATION COUNSELING BULLETIN RESEARCH IN DEVELOPMENTAL DISABILITIES RESEARCH ON LANGUAGE AND SOCIAL INTERACTION RESEARCH ON SOCIAL WORK PRACTICE RESEARCH QUARTERLY FOR EXERCISE AND SPORT RURAL SOCIOLOGY SCIENCE COMMUNICATION SCIENCE, TECHNOLOGY AND HUMAN VALUES SIGNS: JOURNAL OF WOMEN IN CULTURE AND SOCIETY SOCIAL FORCES SOCIAL WORK SOCIAL WORK RESEARCH SOCIETY SOCIOLOGY OF SPORT JOURNAL STRUCTURAL EQUATION MODELING SUICIDE AND LIFE-THREATENING BEHAVIOR TECHNICAL COMMUNICATION: JOURNAL OF THE SOCIETY FOR TECHNICAL COMMUNICATION TOPICS IN LANGUAGE DISORDERS WESTERN JOURNAL OF COMMUNICATION WRITTEN COMMUNICATION YOUTH & SOCIETY

17 What is the Evaluation Process? Publishing Standards Citation Analysis Expert Review

18 Publishing Standards Timeliness of publication: The journal must be able to adhere to its stated frequency of publication. Peer-review: Peer review is indicative of a higher quality of research presented in a journal. International editorial conventions: Articles in journals must provide retrievable sources, full bibliographic information, address information for authors, and informative titles for both journals and articles.

19 Citation Analysis Overall citation rate: Number of times the journal has been cited Impact factor: Number of times a journal has been cited in the past year Total number of articles published by the journal in the past two years Immediacy Impact: Average number of times in the year of its publication an article is cited

20 Citation Rate Overall Citation Rate: Number of times in one year a journal is cited. Average citation count for the 153 journals SSI considers education Average citation count for the AERA Journals Educational Researcher (ER) (Not included in the SSCI) American Educational Research Journal (AERJ) Educational Evaluation and Policy Analysis (EEPA) Journal of Educational and Behavioral Statistics (JEBS) Review of Educational Research (RER) Review of Research in Education (RRE) Average citation count for the top 5 education journals Child Development Psychological Review American Psychologist Developmental Psychology American Sociological Review

21 Impact Factor Number of times a journal is cited in one year Total number of articles in the journal in the past 2 years The impact factor controls for the journal s size as journals who publish more articles may have a better chance of being cited. Average impact factor for the 153 journals SSI considers education Average impact factor for the AERA journals Average impact factor for the 5 education journals with the highest impact factors Psychological Review Monographs of the Society for Research in Child Development American Psychologist Cognitive Psychology Child Development

22 Immediacy Index Immediacy Index: Average number of times in the year of its publication an article is cited; indicates the speed with which a journal is cited. Average immediacy index for the 153 journals SSI considers education Average immediacy index for the AERA journals.096 Average immediacy index for the 5 education journals with the highest immediacy index American Psychologist Counseling Psychologist Educational Psychologist Psychological Review School Psychology Review

23 Expert Review The ISI editorial board, made up of professionals from various fields, reviews the journals submitted for inclusion in their databases. The editorial board also asks subscribers to the journals and people working in the field, i.e. teachers, for feedback about the quality and usefulness of the journals.

24 Research in Other Fields Medical journals also use ISI indicators. One selective database for medical journals is the Brandon/Hill Library which produces 3 reference lists, each containing between journals and books. Medical journal databases also look at circulation rate and acceptance rate.

25 Journal Name Educational Researcher American Educational Research Journal AERA Journals Acceptance Rate and Circulation Rate Circulation Rate (based on 2003 Summer/Fall issues) 22,628 19,932 Acceptance Rate (based on Jan.-Jun. 2003) Features--19%; Research News & Comment--9% 7% Education Evaluation and Policy Analysis 6, % Journal of Educational and Behavioral Statistics 3,541 27% Review of Educational Research 18,342 9% Review of Research in Education 6,233* *projected Not applicable

26 Top Five Journals by Citation Count* Acceptance Rate and Circulation Rate *Journal of Consulting and Clinical Psychology and the American Journal of Pubic Health excluded because of their irrelevancy to education Journal Name Child Development Psychological Review Circulation Rate 10,000* 5,370 Acceptance Rate 6% 27% American Psychologist Developmental Psychology 114,969 5,171 8% 35% American Sociological Review 9, % *estimated

27 Top Five Journals by Impact Factor* Acceptance Rate and Circulation Rate *Journal of Consulting and Clinical Psychology excluded because of its irrelevancy to education Journal Name Circulation Rate Acceptance Rate Psychological Review Monographs of the Society for Research in Child Development American Psychologist Cognitive Psychology 5,370 9,000* 114,969 Unavailable 27% 22% 8% Unavailable Child Development 10,000* 6% *estimated

28 Top Five Journals by Immediacy Index* Acceptance Rate and Circulation Rate *Journal of Consulting and Clinical Psychology and the American Journal of Pubic Health excluded because of their irrelevancy to education Journal Name Circulation Rate Acceptance Rate American Psychologist Counseling Psychologist Educational Psychologist Psychological Review 114,969 3,460 2,371 5,370 8% 15% 27% 27% School Psychology Review 22,000-23,000* 15%* *estimated

29 What this Workshop is Going to Explore 1. How policies and practices promote or inhibit the development of high quality journals 2. The promotion of coherent standards among journals 3. Quality issues facing on-line publishing