Bibliometric study on Dutch academic medical centers /2014

NFU-kenmerk: 5.783 Registratiedatum: 3-7-205 Bibliometric study on Dutch academic medical centers 998-203/204 Research report to the Deans of Medical Sciences (DMW) in the Netherlands June 205 Center for Science & Technology Studies (CWTS) Leeuwen@CWTS.NL

Contents. Introduction... 3 2. Results... 5 2. General results of bibliometric analysis... 5 2.2 Research profiles... 26 2.3 Landscape of Dutch academic medical centers... 45 2.4 Analysis of top-research... 57 2.5 Analysis of the publication strategy: research profiles with MNJS values... 7 2.6 The Brute force indicator for Dutch UMCS (p * MNCS values)... 80 3. Final remarks... 8 Literature... 83 Appendix A: Explanation of the bibliometric indicators of CWTS... 84 Appendix B: Changes in the bibliometric indicators of CWTS... 93 2

. Introduction In this report, a bibliometric study is conducted on the research performed within the Dutch academic medical centers. The study consists of two separate parts, of which the first part is presented in this report. The current part of the study is based on a comparison on the output in journals covered by Thomson Scientific in their Web of Science (WoS), on the overall level of academic medical centers in the Netherlands. The second part of the study consists of a lower-level analysis of the organizational structure within each medical center. The findings from that part of the study will be reported confidentially to each medical center separately. The present report relates to the publication output of the academic medical centers at Dutch universities. The medical centers supplied publication lists to CWTS, which were matched with the CWTS in-house bibliometric data-system. The bibliometric analysis is covering the period 998-203/204 for all eight academic medical centers. This study is an update of the study conducted in 204/205, for the update CWTS was supplied with the year 203 as an additional year to the data collected for the previous studies. In this year s analysis, the option was suggested to backwardly add publications to the existing datasets for de UMC benchmark analysis. For several years, the dataset is only altered with publications from the most recent year, now three UMCS have taken the opportunity to add additional publications to the dataset used for the UMC benchmark study. The three UMCS that had used this option are UMCG, UvA AMC, and LU MC. The reason for the other UMCS not to supply with additional publication data was the time aspect in the data collection against the light of the deadline for delivering publication data for the benchmark study. We considered only papers classified in the WoS as normal articles, letters and reviews, published in source serials processed for the WoS database. Please note that in the indicator set of CWTS (see below), letters are weighted with 0.25. Other document types, such as meeting abstracts, editorials, editorial material, corrections, comments, and book reviews were not included. Also, papers in non-wos source journals are not counted. A few journals are only partially processed for the WoS. Here, only papers processed for the WoS were included. 3

Overview of bibliometric indicators P TCS MCS MNCS MNJS MNCS/MNJS PP(uncited) %Self Cit PP(top 0%) Int Cov Number of papers (normal articles, letters, and reviews) published in journals processed for the Web of Science (WoS). Number of citations recorded in WoS journals to all papers involved. Self-citations are excluded. Average number of citations per publication, or citation per publication ratio. Self-citations are excluded. The impact of a research unit s articles, compared to the world citation average in the subfields in which the research unit is active. The impact of the journals in which a research unit has published (the research unit s journal selection), compared to the world citation average in the subfields covered by these journals. The impact of a research unit s articles, compared to the average citation rate of the research unit s journals (not printed in the datatables). Percentage of articles not cited during the time period considered. Percentage of self-citations. A self-citation is defined as a citation in which the citing and the cited paper have at least one author in common (first author or co-author). The share of the number of papers that are among the 0% most frequently cited of all similar papers in the period 998-203/204. This indicates the degree to which a unit refers themselves to the literature covered in the WoS. The reference behavior of a unit thus indicates whether the journal literature is important for the scholarly communication in a field, and as such can be interpreted as indicator of the applicability of bibliometrics in an assessment context. 4

2. Results 2. General results of bibliometric analysis In Table a and b, the overall bibliometric scores for the eight academic medical centers in the Netherlands are shown for the periods 998-203/204 and 2009-203/204, respectively. Table 2 contains the trend data for every single medical center in the period 998-203/204. Table 3 contains likewise results, limited to papers with first authorships of the respective medical center. Table 4 contains overall and trend data for the eight academic medical centers combined, as a cluster of Dutch academic (bio)medical research. In the standard tables, we have introduced a change in citation impact measurement in comparison with last year s analysis. We have moved back to the method in which citation impact is measured for five year maximum in block of publication years of four year maximum. This works as follows: for the first year in a four year block the impact is measured for five years, for the next year in the block we apply a four year citation window, for the third year in the block we apply a three year citation window, and for the last year in the block we apply a two year citation window. As an example, for the publication years 998-200, we apply a citation window that stretches the period 998-2002, with a five year citation window 998-2002 for the papers of 998, a four year citation window of 999-2002 for the 999 publications, a three year citation window (2000-2002) for the year 2000, and finally a two year citation window (200-2002) for the 200 publications. This moves through time like roof tiles, in which the next period overlaps the previous. This approach has several advantages, namely in the first place the full usage of all publication years in the analysis in a similar fashion, which creates a consistent approach, and secondly, the aspect that publications contribute to each block in a different way, maturing in time, and overall creating a more smooth development of research impact measurements. With respect to this latter aspect, we have to stress that due to smaller output numbers, on lower levels of aggregation (such as projects or small teams), these outcomes tend to fluctuate more as compared with output numbers related to aggregates on higher levels (universities, UMCS, or divisions within UMCS). This approach is also used for the longer periods analyzed in the study (998-203/204 and 2009-203/204). In the case of the former period, we measure the full impact of publications from 998 up and until 204, that is, covering 7 years, and covering six years for the 2009 publications. In the case of the latter period, the first 5

year measured is 2009, with a six year citation window, and the last year is 203, with a two year citation window (203-204). 6

Table a: Bibliometric statistics of Dutch academic medical centers, 998-203/204 P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov Erasmus MC 29620,50 97253,25 30,97,64,4 8% 5% 6% 89% LU MC 277,50 63370,00 29,00,5,37 7% 5% 7% 92% Radboud UMC 22603,25 563265,25 24,92,44,32 6% 5% 7% 90% UMC Maastricht 20586,75 574080,00 27,89,54,28 6% 5% 6% 87% UMCG 9569,75 453472,50 23,7,42,35 6% 7% 7% 90% UU UMC 22574,75 6646,25 29,42,58,42 8% 5% 5% 9% UvA AMC 28469,25 74825,50 26,28,50,35 6% 6% 6% 90% VUmc 20040,50 589048,25 29,39,66,35 9% 6% 6% 89% 7

Table b: Bibliometric statistics of Dutch academic medical centers, 2009-203/204 P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov Erasmus MC 963,25 8077,50 5,,76,53 20% 9% 2% 90% LU MC 878,75 28467,75 4,73,63,49 9% 9% 22% 92% Radboud UMC 9460,00 25328,00 3,25,63,46 8% 0% 2% 90% UMC Maastricht 822,25 06637,00 2,97,6,38 8% 0% 2% 88% UMCG 9237,25 7949,25 2,77,57,47 8% % 2% 90% UU UMC 9578,00 34574,50 4,05,68,53 9% 9% 20% 92% UvA AMC 2607,50 6894,25 2,84,59,44 8% 0% 20% 90% VUmc 963,50 2827,25 4,00,74,44 20% 0% 2% 89% 8

Table 2: Bibliometric statistics of Dutch academic medical centers, trend analysis, 998-203/204 Erasmus MC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov Erasmus MC 998-200 590,75 47626,25 9,8,55,33 6% 8% 2% 88% Erasmus MC 999-2002 5326,75 48287,25 9,07,49,32 6% 7% 2% 89% Erasmus MC 2000-2003 5503,00 53906,50 9,80,59,35 7% 6% 20% 89% Erasmus MC 200-2004 5866,25 58973,75 0,05,57,34 7% 6% 20% 90% Erasmus MC 2002-2005 6455,00 68945,75 0,68,57,33 7% 4% 20% 89% Erasmus MC 2003-2006 704,75 7737,25,02,56,35 7% 4% 20% 90% Erasmus MC 2004-2007 755,00 82369,00 0,96,54,35 7% 2% 20% 90% Erasmus MC 2005-2008 7880,00 87332,25,08,55,38 7% 2% 20% 90% Erasmus MC 2006-2009 8202,00 92696,25,30,63,43 8% 2% 20% 90% Erasmus MC 2007-200 8456,50 98934,50,70,7,48 20% 0% 2% 90% Erasmus MC 2008-20 8750,25 0944,75 2,50,72,50 20% 0% 2% 90% Erasmus MC 2009-202 9325,00 9244,50 2,79,79,55 20% 0% 2% 90% Erasmus MC 200-203 9772,75 20435,50 2,32,76,54 20% 0% 22% 90% 9

LU MC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov LU MC 998-200 426,00 38530,00 9,04,45,29 6% 7% 2% 9% LU MC 999-2002 420,25 37680,25 8,95,4,29 5% 6% 20% 9% LU MC 2000-2003 422,25 37977,75 9,02,33,29 4% 5% 20% 92% LU MC 200-2004 4333,25 39768,25 9,8,36,29 5% 4% 2% 92% LU MC 2002-2005 4532,50 4659,00 9,9,34,29 5% 3% 2% 92% LU MC 2003-2006 486,00 45986,25 9,46,36,29 5% 3% 2% 92% LU MC 2004-2007 525,75 54532,50 0,46,44,32 6% 2% 20% 92% LU MC 2005-2008 5538,00 58753,75 0,6,50,35 7% % 2% 92% LU MC 2006-2009 5787,75 64574,50,6,52,4 7% % 2% 92% LU MC 2007-200 675,00 7323,25,84,65,47 8% 0% 2% 93% LU MC 2008-20 6485,00 7840,00 2,09,65,49 9% 0% 22% 92% LU MC 2009-202 6856,50 84674,75 2,35,64,50 9% 0% 23% 92% LU MC 200-203 790,25 92027,00 2,80,66,50 9% 0% 23% 92% 0

Radboud UMC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov Radboud UMC 998-200 389,00 27573,00 7,09,3,9 4% 20% 22% 87% Radboud UMC 999-2002 3945,50 28466,50 7,2,30,2 4% 9% 23% 88% Radboud UMC 2000-2003 4246,00 360,25 7,44,28,20 4% 8% 22% 89% Radboud UMC 200-2004 4466,75 3457,50 7,73,26,20 3% 8% 22% 89% Radboud UMC 2002-2005 4900,50 39305,00 8,02,24,2 3% 5% 2% 89% Radboud UMC 2003-2006 5338,50 42900,00 8,04,26,2 3% 5% 2% 89% Radboud UMC 2004-2007 5464,25 47206,25 8,64,32,25 4% 3% 2% 89% Radboud UMC 2005-2008 5744,25 53637,00 9,34,40,30 5% 3% 20% 90% Radboud UMC 2006-2009 5942,75 59445,00 0,00,50,35 6% 2% 20% 90% Radboud UMC 2007-200 65,00 66298,25 0,78,6,42 7% 2% 20% 90% Radboud UMC 2008-20 672,25 74688,25,,59,45 8% 2% 2% 90% Radboud UMC 2009-202 7224,50 850,75,23,62,47 9% % 2% 9% Radboud UMC 200-203 7869,00 87895,50,7,64,47 9% % 22% 90%

UMC Maastricht Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov UMC Maastricht 998-200 3634,50 26542,75 7,30,48,7 4% 23% 2% 84% UMC Maastricht 999-2002 395,25 3089,75 7,82,43,8 4% 22% 2% 85% UMC Maastricht 2000-2003 4255,00 3675,75 8,63,46,22 5% 9% 20% 85% UMC Maastricht 200-2004 454,75 34823,50 7,7,35,20 5% 8% 2% 86% UMC Maastricht 2002-2005 4775,50 3898,75 8,00,33,8 4% 6% 2% 86% UMC Maastricht 2003-2006 5056,50 43580,50 8,62,38,2 5% 5% 2% 87% UMC Maastricht 2004-2007 5230,50 46833,75 8,95,4,22 5% 3% 2% 87% UMC Maastricht 2005-2008 5248,25 4955,00 9,37,42,24 5% 3% 2% 87% UMC Maastricht 2006-2009 5385,50 585,25 9,63,48,3 6% 2% 2% 88% UMC Maastricht 2007-200 5503,25 53578,50 9,74,54,35 7% % 2% 88% UMC Maastricht 2008-20 5895,25 57388,50 9,73,55,36 8% 3% 22% 88% UMC Maastricht 2009-202 642,00 6907,25 0,77,63,39 8% 2% 22% 88% UMC Maastricht 200-203 679,25 72920,00 0,74,60,38 8% 2% 22% 88% 2

UMCG Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov UMCG 998-200 2729,75 797,25 6,56,27,22 3% 22% 23% 88% UMCG 999-2002 2853,25 20632,50 7,23,28,22 3% 20% 2% 89% UMCG 2000-2003 304,50 22222,75 7,3,28,2 3% 20% 20% 89% UMCG 200-2004 3293,00 26069,75 7,92,32,22 4% 9% 9% 89% UMCG 2002-2005 3695,00 2858,00 7,74,28,20 3% 9% 20% 89% UMCG 2003-2006 499,50 3324,00 7,89,24,2 3% 7% 20% 89% UMCG 2004-2007 4578,00 38227,25 8,35,26,22 3% 6% 20% 89% UMCG 2005-2008 504,00 4309,00 8,24,25,26 3% 5% 20% 90% UMCG 2006-2009 5487,00 46222,75 8,42,33,3 4% 4% 2% 90% UMCG 2007-200 5939,50 53955,25 9,08,46,39 6% 2% 2% 90% UMCG 2008-20 6460,75 64785,75 0,03,53,44 7% 2% 2% 90% UMCG 2009-202 7086,25 76509,00 0,80,58,48 8% 2% 2% 90% UMCG 200-203 7658,00 8402,50 0,97,58,49 8% 2% 22% 90% 3

UU UMC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov UU UMC 998-200 3982,00 36380,50 9,4,52,34 7% 8% 9% 89% UU UMC 999-2002 4203,00 3873,75 9,2,53,35 6% 7% 9% 90% UU UMC 2000-2003 4362,25 4748,00 9,57,5,34 6% 6% 9% 90% UU UMC 200-2004 4495,25 45948,25 0,22,53,33 6% 5% 8% 90% UU UMC 2002-2005 4750,75 49864,00 0,50,49,32 6% 4% 7% 9% UU UMC 2003-2006 5,25 49087,50 9,60,4,30 5% 4% 8% 9% UU UMC 2004-2007 535,75 52693,25 9,9,43,33 6% 2% 9% 9% UU UMC 2005-2008 5578,00 58290,75 0,45,5,38 7% % 9% 9% UU UMC 2006-2009 5897,50 6323,50 0,72,56,44 7% % 9% 9% UU UMC 2007-200 659,25 6907,75,2,65,50 8% 0% 9% 9% UU UMC 2008-20 6674,75 77928,75,68,68,53 9% 9% 20% 92% UU UMC 2009-202 7392,25 87079,25,78,70,54 20% 9% 2% 92% UU UMC 200-203 7944,50 96327,50 2,3,70,54 20% 0% 2% 92% 4

UvA AMC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov UvA AMC 998-200 4726,50 38937,75 8,24,43,27 5% 8% 2% 89% UvA AMC 999-2002 484,00 462,00 8,60,43,26 5% 7% 20% 89% UvA AMC 2000-2003 5074,50 42592,50 8,39,40,25 5% 7% 2% 90% UvA AMC 200-2004 5337,25 4784,25 8,96,42,26 6% 6% 20% 90% UvA AMC 2002-2005 5665,75 53436,00 9,43,40,25 5% 5% 20% 90% UvA AMC 2003-2006 665,75 5937,00 9,63,38,26 5% 5% 20% 90% UvA AMC 2004-2007 6568,25 6274,25 9,55,4,30 6% 4% 20% 90% UvA AMC 2005-2008 702,00 6837,50 9,75,46,33 5% 4% 20% 90% UvA AMC 2006-2009 7572,50 7596,50 0,03,48,36 6% 3% 20% 90% UvA AMC 2007-200 803,25 8524,25 0,50,57,4 7% 3% 20% 90% UvA AMC 2008-20 8854,00 93799,50 0,59,56,42 7% 3% 2% 90% UvA AMC 2009-202 9756,25 0437,00 0,67,58,43 8% 2% 2% 90% UvA AMC 200-203 0504,50 8668,25,30,63,46 8% 2% 2% 90% 5

VUmc Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov VUmc 998-200 3299,00 2769,75 8,39,5,25 6% 7% 22% 88% VUmc 999-2002 345,50 29863,75 8,74,5,26 7% 6% 2% 89% VUmc 2000-2003 3553,50 30468,75 8,57,47,25 7% 6% 2% 89% VUmc 200-2004 372,50 3428,00 9,7,48,25 6% 5% 2% 90% VUmc 2002-2005 4096,75 40220,25 9,82,47,24 6% 4% 20% 90% VUmc 2003-2006 4353,00 44665,50 0,26,5,26 7% 3% 9% 90% VUmc 2004-2007 4535,25 48728,75 0,74,52,30 7% % 9% 89% VUmc 2005-2008 4679,75 50872,75 0,87,53,33 7% % 9% 89% VUmc 2006-2009 4738,50 52437,50,07,60,39 9% % 20% 90% VUmc 2007-200 5208,25 57286,25,00,73,44 20% % 20% 89% VUmc 2008-20 6075,25 6732,75,08,72,45 20% 2% 2% 89% VUmc 2009-202 6926,50 8652,75,79,74,45 20% % 2% 89% VUmc 200-203 7906,25 97093,00 2,28,76,44 20% % 22% 89% 6

Table 3: Bibliometric statistics of Dutch academic medical centers, first authorships only, 998-203/204 Erasmus MC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 7098,50 44787,50 26,5,4,32 6% 5% 6% 90% 2009-203 6604,75 79854,00 2,09,49,39 7% 9% 20% 90% 998-200 326,25 23878,50 7,42,32,28 4% 9% 2% 88% 999-2002 3299,50 25534,50 7,74,3,27 4% 8% 2% 89% 2000-2003 3338,50 2744,25 8,3,36,29 4% 6% 20% 89% 200-2004 354,75 29964,75 8,53,38,27 5% 7% 20% 90% 2002-2005 3805,00 33878,50 8,90,40,26 5% 4% 20% 90% 2003-2006 4069,75 38783,75 9,53,42,28 5% 4% 9% 90% 2004-2007 4353,00 4763,00 9,59,39,28 5% 3% 9% 90% 2005-2008 4539,25 42844,75 9,44,38,3 5% 3% 20% 90% 2006-2009 477,50 45068,00 9,55,45,35 6% 3% 20% 90% 2007-200 482,00 47624,75 9,90,49,37 7% % 20% 90% 2008-20 4963,75 5436,00 0,36,48,39 7% % 20% 90% 2009-202 547,00 53376,75 0,37,5,40 7% % 20% 90% 200-203 5359,75 5964,25 9,70,48,39 7% % 2% 90% 7

LU MC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 372,00 284553,00 25,02,28,26 4% 5% 6% 93% 2009-203 4343,50 50684,75,67,3,33 5% 9% 9% 93% 998-200 2360,75 7772,00 7,53,22,23 3% 9% 20% 9% 999-2002 2273,25 766,50 7,55,22,22 3% 7% 9% 92% 2000-2003 2236,75 7559,00 7,85,9,23 3% 6% 20% 92% 200-2004 2276,00 8765,00 8,24,22,23 4% 5% 20% 93% 2002-2005 2384,75 935,00 8,,23,24 4% 4% 20% 93% 2003-2006 2529,00 2457,75 8,48,25,23 3% 4% 20% 93% 2004-2007 2780,25 25752,25 9,26,32,25 4% 3% 20% 93% 2005-2008 2969,00 282,25 9,47,33,26 4% 2% 20% 93% 2006-2009 3058,25 2933,50 9,59,32,28 4% 2% 9% 93% 2007-200 3280,00 32958,75 0,05,37,32 5% % 9% 93% 2008-20 3350,00 32395,75 9,67,36,34 5% % 20% 93% 2009-202 3450,50 33977,75 9,85,33,34 5% % 20% 93% 200-203 3568,25 34920,50 9,79,3,34 5% 0% 20% 93% 8

Radboud UMC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 2258,00 25662,75 20,90,23,24 4% 6% 6% 90% 2009-203 4830,50 5305,75 0,98,38,36 6% 0% 8% 9% 998-200 2369,50 347,75 5,66,08, % 2% 23% 88% 999-2002 2302,50 3428,00 5,83,09,2 % 20% 23% 89% 2000-2003 2427,50 50,00 6,8,3,2 2% 20% 22% 89% 200-2004 247,50 5228,50 6,6,,2 2% 9% 22% 90% 2002-2005 2635,75 7743,00 6,73,3,5 2% 6% 2% 90% 2003-2006 2848,50 9724,75 6,92,3,6 2% 7% 2% 90% 2004-2007 2938,25 260,50 7,35,8,9 2% 5% 20% 90% 2005-2008 325,25 24257,75 7,76,2,23 3% 4% 20% 90% 2006-2009 3300,25 26329,75 7,98,29,26 4% 2% 9% 90% 2007-200 3405,50 29270,00 8,59,36,29 5% 2% 9% 9% 2008-20 363,25 32667,25 9,04,37,35 5% 2% 20% 9% 2009-202 378,25 34800,25 9,20,38,37 6% 2% 9% 9% 200-203 3952,50 36042,25 9,2,38,36 6% 2% 9% 9% 9

UMC Maastricht Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 0559,00 23452,50 22,8,27,2 4% 5% 6% 87% 2009-203 3934,75 45,25 0,55,38,29 6% 0% 20% 89% 998-200 2054,25 0324,25 5,03,08, % 24% 24% 84% 999-2002 228,25 00,00 5,7,07,2 % 24% 25% 84% 2000-2003 2234,25 2657,00 5,66,,2 % 2% 24% 85% 200-2004 2323,50 3056,25 5,62,08,2 % 20% 23% 85% 2002-2005 2443,00 5250,25 6,24,4,2 2% 6% 22% 86% 2003-2006 2546,50 736,00 6,73,6,5 2% 6% 22% 86% 2004-2007 2682,75 8973,25 7,07,22,7 3% 4% 2% 87% 2005-2008 2772,50 2295,50 7,68,24,8 3% 4% 2% 87% 2006-2009 288,00 2882,25 7,60,25,23 3% 4% 2% 88% 2007-200 2926,75 23449,00 8,0,30,25 3% 3% 2% 88% 2008-20 2978,25 24835,25 8,34,33,28 5% 3% 2% 88% 2009-202 3094,50 2744,75 8,77,38,29 6% 2% 2% 89% 200-203 380,75 2709,25 8,49,38,29 6% 2% 20% 89% 20

UMCG Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 0803,75 202794,00 8,77,3,22 2% 7% 6% 90% 2009-203 457,50 45065,75 9,86,23,3 4% 2% 8% 9% 998-200 767,25 8979,25 5,08 0,99,3 0% 25% 24% 88% 999-2002 825,25 0033,00 5,50,0,3 9% 22% 2% 89% 2000-2003 860,50 047,75 5,60,02,2 9% 2% 2% 89% 200-2004 964,50 309,00 5,76,07,4 0% 2% 20% 90% 2002-2005 285,50 3056,75 5,97,04,2 0% 20% 20% 90% 2003-2006 2444,00 468,25 5,98,03,4 0% 8% 20% 90% 2004-2007 2698,75 77,75 6,56,05,5 0% 7% 20% 90% 2005-2008 2938,25 9008,50 6,47,04,7 0% 6% 20% 90% 2006-2009 355,50 2782,75 6,90,2,2 2% 5% 20% 90% 2007-200 335,50 24000,25 7,24,9,25 2% 3% 20% 9% 2008-20 3479,25 26840,75 7,7,2,27 3% 3% 20% 9% 2009-202 3597,75 30053,75 8,35,23,30 4% 3% 9% 9% 200-203 3695,50 29745,25 8,05,22,32 3% 4% 9% 9% 2

UU UMC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 2542,75 295556,25 23,56,32,3 5% 5% 4% 92% 2009-203 505,50 56068,75 0,98,4,37 6% 9% 9% 92% 998-200 233,00 6638,25 7,4,27,24 5% 20% 9% 90% 999-2002 2445,00 7976,75 7,35,26,26 4% 9% 9% 90% 2000-2003 2528,25 908,00 7,52,22,24 3% 7% 9% 9% 200-2004 2582,25 9967,75 7,73,24,24 3% 6% 9% 9% 2002-2005 267,00 2746,50 8,4,22,26 3% 5% 8% 92% 2003-2006 2843,25 2209,25 7,78,8,26 2% 5% 8% 9% 2004-2007 2948,00 2425,25 8,23,23,29 3% 3% 8% 9% 2005-2008 342,00 2634,75 8,38,28,33 4% 2% 8% 9% 2006-2009 3307,75 28448,00 8,60,32,35 5% 2% 8% 92% 2007-200 348,00 30943,25 9,05,37,37 6% % 8% 92% 2008-20 3634,25 33784,00 9,30,37,37 6% % 9% 93% 2009-202 390,50 36205,25 9,26,42,38 6% 0% 9% 93% 200-203 4233,00 38532,00 9,0,4,37 6% % 9% 92% 22

UvA AMC Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 2594,75 292828,25 23,25,30,26 4% 7% 5% 9% 2009-203 5225,75 56528,50 0,82,37,33 5% % 9% 9% 998-200 242,75 6524,75 6,82,24,9 3% 9% 20% 89% 999-2002 2379,00 7833,25 7,50,26,9 4% 9% 9% 90% 2000-2003 2368,50 8063,50 7,63,26,20 4% 8% 9% 90% 200-2004 2432,00 9337,75 7,95,28,2 5% 8% 9% 90% 2002-2005 259,50 20668,25 8,20,24,20 4% 8% 9% 90% 2003-2006 279,00 22968,25 8,45,22,2 3% 6% 9% 9% 2004-2007 288,00 24399,25 8,47,25,25 4% 4% 20% 9% 2005-2008 305,50 254,50 8,8,28,28 3% 5% 20% 9% 2006-2009 3333,25 28020,00 8,4,27,29 3% 4% 9% 9% 2007-200 3565,00 30948,75 8,68,33,33 5% 4% 9% 9% 2008-20 3855,75 33653,50 8,73,33,3 5% 4% 20% 9% 2009-202 4098,75 36958,75 9,02,38,33 6% 3% 20% 9% 200-203 4320,25 40724,75 9,43,39,34 5% 3% 9% 9% 23

VUmc Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 9930,00 26079,25 26,26,43,23 6% 6% 5% 89% 2009-203 4000,00 45033,75,26,44,29 7% 2% 20% 90% 998-200 85,50 377,50 7,4,39,7 5% 9% 20% 88% 999-2002 927,25 4903,50 7,73,37,8 5% 7% 20% 89% 2000-2003 98,75 469,00 7,38,34,8 5% 6% 2% 89% 200-2004 2060,75 588,50 7,68,30,7 4% 6% 2% 90% 2002-2005 2244,50 8296,75 8,5,30,6 4% 5% 2% 90% 2003-2006 232,75 20398,25 8,79,34,7 5% 4% 9% 90% 2004-2007 2390,75 2932,75 9,7,36,9 5% 2% 9% 90% 2005-2008 247,75 23366,75 9,45,38,22 5% 3% 9% 90% 2006-2009 2479,75 23364,50 9,42,43,26 6% 2% 9% 90% 2007-200 2645,25 24798,75 9,37,47,29 7% % 9% 90% 2008-20 2888,00 2678,25 9,25,45,29 7% 3% 20% 90% 2009-202 3086,75 29366,50 9,5,47,30 8% 3% 20% 90% 200-203 3354,25 32068,00 9,56,43,29 7% 3% 2% 89% 24

Table 4: Bibliometric statistics of all Dutch academic medical centers combined, 998-203/204 Period P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov 998-203 52574,00 44709,50 27,8,50,33 6% 6% 6% 90% 2009-203 6376,25 8424,75 3,22,6,42 8% 0% 20% 90% 998-200 27003,75 23099,50 7,89,40,24 5% 9% 2% 88% 999-2002 27674,25 226853,50 8,20,40,24 5% 8% 2% 89% 2000-2003 28725,25 245284,00 8,54,4,25 5% 7% 20% 89% 200-2004 29966,75 26504,25 8,73,39,24 5% 7% 20% 89% 2002-2005 3.30,25 292842,25 9,,39,24 5% 5% 20% 90% 2003-2006 34675,50 322287,50 9,29,39,25 5% 5% 20% 90% 2004-2007 36689,25 35576,00 9,58,4,27 5% 3% 20% 90% 2005-2008 38739,75 38529,00 9,85,43,3 6% 3% 20% 90% 2006-2009 40746,75 40843,50 0,02,48,34 6% 2% 20% 90% 2007-200 42778,25 445072,00 0,40,56,39 7% 2% 20% 90% 2008-20 45745,50 49269,25 0,74,58,4 8% 2% 2% 90% 2009-202 49353,75 547873,00,0,6,43 8% % 2% 90% 200-203 52693,25 59925,50,23,62,43 8% % 2% 90% 25

2.2 Research profiles In the research profiles for the eight medical centers, displayed in Figures to 8 (with a) and b) figures describing the period 998-203/204 and 2009-203/204 respectively), the output per academic medical center is displayed to the % level of the output (fields with smaller shares than % are not displayed). Per medical center, the total share displayed in the profile with respect to the total output is indicated. We have produced the profiles for the full period covered (now 998 to 203/204), and the last five year period 2009-203/204. This five year period is already shown in the tables, and is now also introduced in the profiles, as it shows the most recent situation of the performance per UMC. While reading and using the profiles, it is important to keep in mind that these profiles display information of the output and impact per UMC across fields, particularly focused on the comparison between UMCS. These profiles do not provide information on the level of organizational units within every single UMC, as there exists in this analysis no organic link between the units producing papers, and the fields these papers belong to. For that particular information, every UMC has its own internal analysis. So the output and impact in for example Immunology can be compared between UMCS, but does not allow any kind of conclusion of the level of units involved in immunological research within every UMC. In general, around 80% of the output of the medical centers is displayed in the research profiles. In Figure 9, the research profile covering the period 2009-203/204 for the combined output of academic medical centers is displayed. 26

Figure a: Output and normalized impact per field (998-203/204) Erasmus MC Field (MNCS) ONCOLOGY (,55) CARDIAC & CARDIOVASCULAR SYSTEMS (,76) ENDOCRINOLOGY & METABOLISM (,28) SURGERY (,44) GENETICS & HEREDITY (,77) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,62) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,45) CLINICAL NEUROLOGY (,8) HEMATOLOGY (,54) MEDICINE, GENERAL & INTERNAL (5,) IMMUNOLOGY (,32) GASTROENTEROLOGY & HEPATOLOGY (,65) PERIPHERAL VASCULAR DISEASE (,63) UROLOGY & NEPHROLOGY (,57) PHARMACOLOGY & PHARMACY (,) NEUROSCIENCES (,3) PEDIATRICS (,33) BIOCHEMISTRY & MOLECULAR BIOLOGY (,27) CELL BIOLOGY (,39) PSYCHIATRY (,8) OBSTETRICS & GYNECOLOGY (,38) MICROBIOLOGY (,49) MULTIDISCIPLINARY SCIENCES (4,52) RESPIRATORY SYSTEM (,6) HEALTH CARE SCIENCES & SERVICES (,38) INFECTIOUS DISEASES (,47) MEDICINE, RESEARCH & EXPERIMENTAL (,45) RHEUMATOLOGY (,92) OPHTHALMOLOGY (,57) PATHOLOGY (,42) VIROLOGY (,34) CRITICAL CARE MEDICINE (,26) 4,3 4, 3,6 3,5 3,4 3, 3, 3 2,9 2,8 2,6 2,6 2,5 2,5 2,4 2,3,8,6,6,5,4,4,3,3,2 7,2 8,3 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 9 Share of the output (%) 27

Figure b: Output and normalized impact per field (2009-203/204) Erasmus MC Field (MNCS) ONCOLOGY (,85) CARDIAC & CARDIOVASCULAR SYSTEMS (,64) SURGERY (,43) GENETICS & HEREDITY (2,27) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,85) ENDOCRINOLOGY & METABOLISM (,22) GASTROENTEROLOGY & HEPATOLOGY (,75) HEMATOLOGY (,87) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,38) CLINICAL NEUROLOGY (,76) PEDIATRICS (,28) PHARMACOLOGY & PHARMACY (,9) IMMUNOLOGY (,54) MEDICINE, GENERAL & INTERNAL (6,5) PERIPHERAL VASCULAR DISEASE (,36) MULTIDISCIPLINARY SCIENCES (3,53) NEUROSCIENCES (,54) UROLOGY & NEPHROLOGY (2,9) BIOCHEMISTRY & MOLECULAR BIOLOGY (,3) PSYCHIATRY (,34) CELL BIOLOGY (,54) HEALTH CARE SCIENCES & SERVICES (,36) OBSTETRICS & GYNECOLOGY (,45) INFECTIOUS DISEASES (,64) RESPIRATORY SYSTEM (,78) RHEUMATOLOGY (,98) DERMATOLOGY (2,6) MEDICINE, RESEARCH & EXPERIMENTAL (,64) MICROBIOLOGY (,78) CRITICAL CARE MEDICINE (,44) VIROLOGY (,5) 4,3 4,2 3,8 3,5 3,3 2,8 2,8 2,7 2,7 2,4 2,4 2,4 2,3 2,3 2,3 2,2,9,8,8,5,5,4,4,3,2,,,, 7,5 7,2 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 28

Figure 2a: Output and normalized impact per field (998-203/204) LU MC Field (MNCS) ONCOLOGY (,3) CARDIAC & CARDIOVASCULAR SYSTEMS (,5) IMMUNOLOGY (,9) HEMATOLOGY (,29) GENETICS & HEREDITY (,75) RHEUMATOLOGY (2) ENDOCRINOLOGY & METABOLISM (,2) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,25) BIOCHEMISTRY & MOLECULAR BIOLOGY (,7) MEDICINE, GENERAL & INTERNAL (3,7) CLINICAL NEUROLOGY (,5) SURGERY (,5) NEUROSCIENCES (,24) PERIPHERAL VASCULAR DISEASE (,37) MULTIDISCIPLINARY SCIENCES (2,92) PHARMACOLOGY & PHARMACY (,08) CELL BIOLOGY (,4) UROLOGY & NEPHROLOGY (,75) OBSTETRICS & GYNECOLOGY (,8) PSYCHIATRY (,45) PEDIATRICS (,24) GASTROENTEROLOGY & HEPATOLOGY (,52) PATHOLOGY (,3) MEDICINE, RESEARCH & EXPERIMENTAL (,69) MICROBIOLOGY (,72) RESPIRATORY SYSTEM (,85) VIROLOGY (,29) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,44) DERMATOLOGY (,86) INFECTIOUS DISEASES (,32) TRANSPLANTATION (,24) 5 4,6 4,5 4,5 4,3 3,8 3,6 3,2 3, 2,7 2,7 2,6 2,4 2,2 2, 2,9,8,7,7,4,4,4,2,,, 5,7 6,9 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 29

Figure 2b: Output and normalized impact per field (2009-203/204) LU MC Field (MNCS) ONCOLOGY (,4) CARDIAC & CARDIOVASCULAR SYSTEMS (,45) RHEUMATOLOGY (2,7) GENETICS & HEREDITY (2,3) ENDOCRINOLOGY & METABOLISM (,26) MULTIDISCIPLINARY SCIENCES (2,49) HEMATOLOGY (,3) IMMUNOLOGY (,8) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,43) CLINICAL NEUROLOGY (,69) MEDICINE, GENERAL & INTERNAL (3,63) BIOCHEMISTRY & MOLECULAR BIOLOGY (,33) SURGERY (,6) NEUROSCIENCES (,38) PSYCHIATRY (,66) PERIPHERAL VASCULAR DISEASE (,2) CELL BIOLOGY (,53) OBSTETRICS & GYNECOLOGY (,4) UROLOGY & NEPHROLOGY (,7) PHARMACOLOGY & PHARMACY (,23) PEDIATRICS (,2) GASTROENTEROLOGY & HEPATOLOGY (2,02) MEDICINE, RESEARCH & EXPERIMENTAL (,8) MICROBIOLOGY (2,5) BIOCHEMICAL RESEARCH METHODS (,52) PATHOLOGY (,35) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,6) INFECTIOUS DISEASES (,47) VIROLOGY (,4) GERIATRICS & GERONTOLOGY (,6) RESPIRATORY SYSTEM (,47) TRANSPLANTATION (,42) 4,8 4,4 4,3 3,8 3,7 3,4 3,3 3 2,9 2,8 2,6 2,3 2,3 2,2,9,9,8,6,5,4,3,2,2,, 5,7 7 6,5 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 30

Figure 3a: Output and normalized impact per field (998-203/204) Radboud UMC Field (MNCS) ONCOLOGY (,36) GENETICS & HEREDITY (,67) NEUROSCIENCES (,2) CLINICAL NEUROLOGY (,29) UROLOGY & NEPHROLOGY (,77) MEDICINE, GENERAL & INTERNAL (3,04) BIOCHEMISTRY & MOLECULAR BIOLOGY (,24) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,76) SURGERY (,37) PHARMACOLOGY & PHARMACY (,3) DENTISTRY/ORAL SURGERY & MEDICINE (,9) RHEUMATOLOGY (,75) IMMUNOLOGY (,48) HEMATOLOGY (,52) ENDOCRINOLOGY & METABOLISM (,) PEDIATRICS (,07) MULTIDISCIPLINARY SCIENCES (2,44) OBSTETRICS & GYNECOLOGY (,3) CELL BIOLOGY (,36) MICROBIOLOGY (,5) DERMATOLOGY (,49) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,42) PSYCHIATRY (,26) INFECTIOUS DISEASES (,48) CARDIAC & CARDIOVASCULAR SYSTEMS (,44) PERIPHERAL VASCULAR DISEASE (,32) OTORHINOLARYNGOLOGY (,5) MEDICINE, RESEARCH & EXPERIMENTAL (,34) HEALTH CARE SCIENCES & SERVICES (,22) ENGINEERING, BIOMEDICAL (,9) GASTROENTEROLOGY & HEPATOLOGY (,35) PATHOLOGY (,4) PHYSIOLOGY (,05) 4,2 4, 3,9 3,4 3,4 3,2 3, 2,8 2,8 2,5 2,4 2, 2, 2,9,8,8,8,7,7,7,5,4,4,4,4,3,3,2, 5,3 7, Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 3

Figure 3b: Output and normalized impact per field (2009-203/204) Radboud UMC Field (MNCS) ONCOLOGY (,57) GENETICS & HEREDITY (2,03) CLINICAL NEUROLOGY (,47) NEUROSCIENCES (,5) MULTIDISCIPLINARY SCIENCES (2,03) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (2,09) UROLOGY & NEPHROLOGY (2,6) SURGERY (,45) MEDICINE, GENERAL & INTERNAL (3,9) DENTISTRY/ORAL SURGERY & MEDICINE (,3) BIOCHEMISTRY & MOLECULAR BIOLOGY (,43) RHEUMATOLOGY (,8) IMMUNOLOGY (,55) PHARMACOLOGY & PHARMACY (,3) HEMATOLOGY (,67) MICROBIOLOGY (,53) ENDOCRINOLOGY & METABOLISM (,2) PSYCHIATRY (,53) OBSTETRICS & GYNECOLOGY (,2) CELL BIOLOGY (,88) INFECTIOUS DISEASES (,7) HEALTH CARE SCIENCES & SERVICES (,) PEDIATRICS (,24) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,86) GASTROENTEROLOGY & HEPATOLOGY (,65) CARDIAC & CARDIOVASCULAR SYSTEMS (,57) DERMATOLOGY (,82) MEDICINE, RESEARCH & EXPERIMENTAL (,52) OTORHINOLARYNGOLOGY (,43) ENGINEERING, BIOMEDICAL (,07) PERIPHERAL VASCULAR DISEASE (,4) REHABILITATION (,46) PATHOLOGY (,42) RESPIRATORY SYSTEM (,5) 3,6 3,3 2,9 2,9 2,9 2,8 2,8 2,5 2,4 2,4 2, 2, 2 2,9,7,7,7,6,5,5,5,4,3,3,2, 4,3 4,2 5,3 6,5 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 Share of the output (%) 32

Figure 4a: Output and normalized impact per field (998-203/204) UMC Maastricht Field (MNCS) CARDIAC & CARDIOVASCULAR SYSTEMS (,83) ONCOLOGY (,37) PERIPHERAL VASCULAR DISEASE (,39) NUTRITION & DIETETICS (,35) PSYCHIATRY (,44) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,3) ENDOCRINOLOGY & METABOLISM (,27) CLINICAL NEUROLOGY (,66) NEUROSCIENCES (,5) MEDICINE, GENERAL & INTERNAL (4,22) SURGERY (,48) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,47) RHEUMATOLOGY (,84) BIOCHEMISTRY & MOLECULAR BIOLOGY (,24) HEALTH CARE SCIENCES & SERVICES (,49) HEMATOLOGY (,24) PHARMACOLOGY & PHARMACY (,2) GENETICS & HEREDITY (,27) UROLOGY & NEPHROLOGY (,82) RESPIRATORY SYSTEM (2,4) PSYCHOLOGY, MULTIDISCIPLINARY (,49) GASTROENTEROLOGY & HEPATOLOGY (,4) OBSTETRICS & GYNECOLOGY (,43) MULTIDISCIPLINARY SCIENCES (2) PSYCHOLOGY, CLINICAL (,22) IMMUNOLOGY (,08) CELL BIOLOGY (,47) PEDIATRICS (,25) PHYSIOLOGY (,28) OPHTHALMOLOGY (,35) EDUCATION, SCIENTIFIC DISCIPLINES (,73) SPORT SCIENCES (,6) REHABILITATION (,3) MEDICINE, RESEARCH & EXPERIMENTAL (,56) TOXICOLOGY (,3) NURSING (,6) Low (< 0,8) Average High (>,2) 4,8 4,5 4,2 3,8 3,8 3,6 3,3 3, 3, 2,9 2,8 2,6 2,6 2,5 2,3 2,3 2,2 2 2,7,7,6,6,5,5,4,4,4,3,,, 0 2 3 4 5 6 Share of the output (%) 33

Figure 4b: Output and normalized impact per field (2009-203/204) UMC Maastricht Field (MNCS) CARDIAC & CARDIOVASCULAR SYSTEMS (,64) ONCOLOGY (,5) PERIPHERAL VASCULAR DISEASE (,4) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,7) NUTRITION & DIETETICS (,39) ENDOCRINOLOGY & METABOLISM (,4) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,44) CLINICAL NEUROLOGY (,66) MULTIDISCIPLINARY SCIENCES (,65) PSYCHIATRY (,74) SURGERY (,6) NEUROSCIENCES (,32) MEDICINE, GENERAL & INTERNAL (4,88) HEALTH CARE SCIENCES & SERVICES (,43) GENETICS & HEREDITY (,34) RHEUMATOLOGY (2,06) BIOCHEMISTRY & MOLECULAR BIOLOGY (,37) HEMATOLOGY (,5) UROLOGY & NEPHROLOGY (,63) OBSTETRICS & GYNECOLOGY (,67) PHARMACOLOGY & PHARMACY (,27) RESPIRATORY SYSTEM (,73) GASTROENTEROLOGY & HEPATOLOGY (,3) PSYCHOLOGY, MULTIDISCIPLINARY (,75) REHABILITATION (,25) PEDIATRICS (,37) OPHTHALMOLOGY (,53) EDUCATION, SCIENTIFIC DISCIPLINES (,68) CELL BIOLOGY (2) TOXICOLOGY (,44) GERIATRICS & GERONTOLOGY (2,8) MEDICINE, RESEARCH & EXPERIMENTAL (,79) IMMUNOLOGY (,23) NURSING (2,5) PHYSIOLOGY (,7) 3,7 3,6 3,6 3,3 3,2 3, 3 2,9 2,9 2,8 2,5 2,4 2,3 2,2 2,9,9,8,7,6,5,4,3,3,3,3,3,,,, 5,2 4,7 4,4 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 34

Figure 5a: Output and normalized impact per field (998-203/204) UMCG Field (MNCS) ONCOLOGY (,24) CARDIAC & CARDIOVASCULAR SYSTEMS (,54) PSYCHIATRY (,72) NEUROSCIENCES (,26) SURGERY (,37) GENETICS & HEREDITY (2,06) MEDICINE, GENERAL & INTERNAL (3,8) PHARMACOLOGY & PHARMACY (,05) CLINICAL NEUROLOGY (,26) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,36) HEMATOLOGY (,23) BIOCHEMISTRY & MOLECULAR BIOLOGY (,5) DENTISTRY/ORAL SURGERY & MEDICINE (,24) GASTROENTEROLOGY & HEPATOLOGY (,45) PEDIATRICS (,5) RESPIRATORY SYSTEM (,54) UROLOGY & NEPHROLOGY (,78) IMMUNOLOGY (,5) ENDOCRINOLOGY & METABOLISM (,06) PERIPHERAL VASCULAR DISEASE (,23) MULTIDISCIPLINARY SCIENCES (,82) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,2) REHABILITATION (,9) RHEUMATOLOGY (,7) PSYCHOLOGY, MULTIDISCIPLINARY (,35) OBSTETRICS & GYNECOLOGY (,6) CELL BIOLOGY (,2) MEDICINE, RESEARCH & EXPERIMENTAL (0,96) MICROBIOLOGY (,35) TRANSPLANTATION (,) ENGINEERING, BIOMEDICAL (,03) SPORT SCIENCES (,26) 2,5 2,5 2,5 2,4 2,4 2,4 2,3 2,2 2,2 2,2 2, 2,8,7,6,4,3,3,2,, 3,7 3,4 3,4 3,3 3,2 3, 3 3 5,7 6,3 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 Share of the output (%) 35

Figure 5b: Output and normalized impact per field (2009-203/204) UMCG Field (MNCS) CARDIAC & CARDIOVASCULAR SYSTEMS (,62) ONCOLOGY (,49) PSYCHIATRY (,98) MULTIDISCIPLINARY SCIENCES (,75) GENETICS & HEREDITY (2,58) MEDICINE, GENERAL & INTERNAL (4,38) SURGERY (,4) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,4) CLINICAL NEUROLOGY (,3) NEUROSCIENCES (,69) PEDIATRICS (,02) PHARMACOLOGY & PHARMACY (,4) ENDOCRINOLOGY & METABOLISM (,2) DENTISTRY/ORAL SURGERY & MEDICINE (,32) BIOCHEMISTRY & MOLECULAR BIOLOGY (,25) HEMATOLOGY (,39) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,) UROLOGY & NEPHROLOGY (,8) PERIPHERAL VASCULAR DISEASE (,2) REHABILITATION (,25) RESPIRATORY SYSTEM (,56) IMMUNOLOGY (,44) GASTROENTEROLOGY & HEPATOLOGY (,58) RHEUMATOLOGY (,94) OBSTETRICS & GYNECOLOGY (,2) PSYCHOLOGY, MULTIDISCIPLINARY (,43) SPORT SCIENCES (,32) MEDICINE, RESEARCH & EXPERIMENTAL (0,99) CELL BIOLOGY (,6) HEALTH CARE SCIENCES & SERVICES (,07) ORTHOPEDICS (0,98) MICROBIOLOGY (,48) TRANSPLANTATION (,24) DERMATOLOGY (,34) 3,2 3,2 3, 3 2,9 2,6 2,4 2,3 2,3 2,3 2,3 2,3 2,2 2,2 2,2 2, 2,9,8,8,7,4,2,2, 4, 4 3,8 5,5 5,3 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 36

Figure 6a: Output and normalized impact per field (998-203/204) UU UMC Field (MNCS) CLINICAL NEUROLOGY (,47) ONCOLOGY (,35) CARDIAC & CARDIOVASCULAR SYSTEMS (,63) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,37) SURGERY (,35) NEUROSCIENCES (,27) HEMATOLOGY (,3) PERIPHERAL VASCULAR DISEASE (,63) IMMUNOLOGY (,6) MEDICINE, GENERAL & INTERNAL (3,85) BIOCHEMISTRY & MOLECULAR BIOLOGY (,75) CELL BIOLOGY (2,0) ENDOCRINOLOGY & METABOLISM (,22) PEDIATRICS (,36) GENETICS & HEREDITY (,94) PHARMACOLOGY & PHARMACY (,5) PSYCHIATRY (,75) GASTROENTEROLOGY & HEPATOLOGY (,2) RHEUMATOLOGY (,63) MULTIDISCIPLINARY SCIENCES (3,2) OBSTETRICS & GYNECOLOGY (,66) UROLOGY & NEPHROLOGY (,32) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,67) MICROBIOLOGY (,57) INFECTIOUS DISEASES (,43) RESPIRATORY SYSTEM (,35) MEDICINE, RESEARCH & EXPERIMENTAL (,26) NUTRITION & DIETETICS (,46) 4,8 4,5 4,4 4 3,9 3,9 3,7 3,6 3,5 3, 3, 2,9 2,9 2,8 2,7 2,7 2,7 2,6 2,5 2,4 2 2,9,6,4,3,2 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 37

Figure 6b: Output and normalized impact per field (2009-203/204) UU UMC Field (MNCS) ONCOLOGY (,3) CARDIAC & CARDIOVASCULAR SYSTEMS (,62) MULTIDISCIPLINARY SCIENCES (2,35) CLINICAL NEUROLOGY (,6) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,43) SURGERY (,52) HEMATOLOGY (,34) NEUROSCIENCES (,57) PERIPHERAL VASCULAR DISEASE (,74) GENETICS & HEREDITY (2,43) MEDICINE, GENERAL & INTERNAL (4,4) IMMUNOLOGY (,2) CELL BIOLOGY (2,4) PHARMACOLOGY & PHARMACY (,) PEDIATRICS (,53) BIOCHEMISTRY & MOLECULAR BIOLOGY (2,05) RHEUMATOLOGY (,52) GASTROENTEROLOGY & HEPATOLOGY (,49) PSYCHIATRY (,79) ENDOCRINOLOGY & METABOLISM (,9) OBSTETRICS & GYNECOLOGY (,94) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,89) UROLOGY & NEPHROLOGY (,28) INFECTIOUS DISEASES (,66) RESPIRATORY SYSTEM (,44) NUTRITION & DIETETICS (,5) MICROBIOLOGY (,69) MEDICINE, RESEARCH & EXPERIMENTAL (,25) REHABILITATION (,38) CRITICAL CARE MEDICINE (,38) 3,7 3,4 3,3 3,2 3, 2,9 2,9 2,8 2,7 2,6 2,6 2,5 2,5 2,4 2,3 2 2,7,5,4,3,2,2, 5, 4,8 4,6 4,5 4,5 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 38

Figure 7a: Output and normalized impact per field (998-203/204) UvA AMC Field (MNCS) CARDIAC & CARDIOVASCULAR SYSTEMS (,57) ONCOLOGY (,34) SURGERY (,43) IMMUNOLOGY (,47) GASTROENTEROLOGY & HEPATOLOGY (,67) MEDICINE, GENERAL & INTERNAL (3,66) HEMATOLOGY (,25) GENETICS & HEREDITY (,42) BIOCHEMISTRY & MOLECULAR BIOLOGY (,24) PERIPHERAL VASCULAR DISEASE (,54) UROLOGY & NEPHROLOGY (,55) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,22) CLINICAL NEUROLOGY (,48) ENDOCRINOLOGY & METABOLISM (,04) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,2) INFECTIOUS DISEASES (,2) PEDIATRICS (,6) MULTIDISCIPLINARY SCIENCES (2,3) NEUROSCIENCES (,26) PHARMACOLOGY & PHARMACY (,6) OBSTETRICS & GYNECOLOGY (,4) PSYCHIATRY (,39) CELL BIOLOGY (,45) RHEUMATOLOGY (,85) VIROLOGY (,23) CRITICAL CARE MEDICINE (,53) MICROBIOLOGY (,33) MEDICINE, RESEARCH & EXPERIMENTAL (,39) OPHTHALMOLOGY (,95) HEALTH CARE SCIENCES & SERVICES (,36) DERMATOLOGY (2,) RESPIRATORY SYSTEM (,29) 4, 4, 4, 4 3,8 3,8 3,3 3,2 3 3 3 2,8 2,8 2,6 2,5 2,3 2,2 2,2 2,2 2,,9,6,6,6,6,5,4,2,,, 5,3 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 39

Figure 7b: Output and normalized impact per field (2009-203/204) UvA AMC Field (MNCS) CARDIAC & CARDIOVASCULAR SYSTEMS (,63) MULTIDISCIPLINARY SCIENCES (,97) ONCOLOGY (,33) MEDICINE, GENERAL & INTERNAL (3,77) GASTROENTEROLOGY & HEPATOLOGY (,73) SURGERY (,56) HEMATOLOGY (,33) GENETICS & HEREDITY (,79) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,4) UROLOGY & NEPHROLOGY (,57) IMMUNOLOGY (,83) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,22) PERIPHERAL VASCULAR DISEASE (,4) CLINICAL NEUROLOGY (,7) OBSTETRICS & GYNECOLOGY (,5) INFECTIOUS DISEASES (,25) ENDOCRINOLOGY & METABOLISM (,2) PEDIATRICS (,7) BIOCHEMISTRY & MOLECULAR BIOLOGY (,37) PSYCHIATRY (,46) PHARMACOLOGY & PHARMACY (,6) NEUROSCIENCES (,5) RHEUMATOLOGY (2,23) CRITICAL CARE MEDICINE (,68) MEDICINE, RESEARCH & EXPERIMENTAL (,43) CELL BIOLOGY (,9) VIROLOGY (,4) HEALTH CARE SCIENCES & SERVICES (,2) RESPIRATORY SYSTEM (,38) ORTHOPEDICS (,03) 4,2 4,2 4 4 3,9 3,4 3,3 3,2 3, 3, 2,8 2,8 2,7 2,6 2,5 2,5 2,5 2,4 2,4,9,8,8,8,4,4,3,3,2, 6, Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 Share of the output (%) 40

Figure 8a: Output and normalized impact per field (998-203/204) VUmc Field (MNCS) ONCOLOGY (,54) CLINICAL NEUROLOGY (2,) ENDOCRINOLOGY & METABOLISM (,67) NEUROSCIENCES (,8) PSYCHIATRY (,57) RHEUMATOLOGY (,56) IMMUNOLOGY (,57) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,54) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,59) MEDICINE, GENERAL & INTERNAL (3,65) GENETICS & HEREDITY (2,) CARDIAC & CARDIOVASCULAR SYSTEMS (,75) SURGERY (,56) HEMATOLOGY (,72) PEDIATRICS (,22) BIOCHEMISTRY & MOLECULAR BIOLOGY (,25) OBSTETRICS & GYNECOLOGY (,37) PHARMACOLOGY & PHARMACY (,32) GASTROENTEROLOGY & HEPATOLOGY (,35) MULTIDISCIPLINARY SCIENCES (2,43) PERIPHERAL VASCULAR DISEASE (,54) NUTRITION & DIETETICS (,32) PATHOLOGY (,89) HEALTH CARE SCIENCES & SERVICES (,64) GERIATRICS & GERONTOLOGY (,58) UROLOGY & NEPHROLOGY (,5) MEDICINE, RESEARCH & EXPERIMENTAL (,59) CELL BIOLOGY (,43) MICROBIOLOGY (,3) REHABILITATION (,59) ORTHOPEDICS (,63) CRITICAL CARE MEDICINE (,26) PSYCHOLOGY, MULTIDISCIPLINARY (,33) SPORT SCIENCES (,68) RESPIRATORY SYSTEM (,45) 3,4 3,2 3,2 3,2 3, 3 2,5 2,5 2,5 2,3 2,3 2,3 2,,9,9,8,7,6,5,5,5,4,4,3,2,,, 4,7 4,3 6, 7,9 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 9 Share of the output (%) 4

Figure 8b: Output and normalized impact per field (2009-203/204) VUmc Field (MNCS) ONCOLOGY (,54) CLINICAL NEUROLOGY (2,42) ENDOCRINOLOGY & METABOLISM (,9) PSYCHIATRY (,67) NEUROSCIENCES (2,5) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,36) MULTIDISCIPLINARY SCIENCES (2,05) RHEUMATOLOGY (,67) GENETICS & HEREDITY (2,69) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,7) MEDICINE, GENERAL & INTERNAL (3,93) CARDIAC & CARDIOVASCULAR SYSTEMS (,92) PEDIATRICS (,23) IMMUNOLOGY (,99) SURGERY (,77) BIOCHEMISTRY & MOLECULAR BIOLOGY (,43) OBSTETRICS & GYNECOLOGY (,3) NUTRITION & DIETETICS (,4) GASTROENTEROLOGY & HEPATOLOGY (,46) HEALTH CARE SCIENCES & SERVICES (,49) HEMATOLOGY (,94) PHARMACOLOGY & PHARMACY (,47) GERIATRICS & GERONTOLOGY (,49) PERIPHERAL VASCULAR DISEASE (,4) CELL BIOLOGY (,79) UROLOGY & NEPHROLOGY (,56) MEDICINE, RESEARCH & EXPERIMENTAL (,67) PSYCHOLOGY, MULTIDISCIPLINARY (,36) REHABILITATION (,28) ORTHOPEDICS (,43) PATHOLOGY (,9) SPORT SCIENCES (,62) RESPIRATORY SYSTEM (,62) INFECTIOUS DISEASES (,45) CRITICAL CARE MEDICINE (,5) 4,2 4 3,9 3,8 3,3 3,2 3, 2,9 2,8 2,7 2,6 2,4 2,3 2, 2,,9,9,9,9,6,5,5,4,3,3,2,, 6,2 7 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 42

Figure 9a: Output and normalized impact per field (998-203/204) All UMCs together Field (MNCS) ONCOLOGY (,38) CARDIAC & CARDIOVASCULAR SYSTEMS (,63) CLINICAL NEUROLOGY (,56) SURGERY (,37) NEUROSCIENCES (,29) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,4) ENDOCRINOLOGY & METABOLISM (,27) IMMUNOLOGY (,3) MEDICINE, GENERAL & INTERNAL (3,6) GENETICS & HEREDITY (,56) BIOCHEMISTRY & MOLECULAR BIOLOGY (,3) HEMATOLOGY (,33) PHARMACOLOGY & PHARMACY (,4) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,43) PERIPHERAL VASCULAR DISEASE (,47) UROLOGY & NEPHROLOGY (,63) GASTROENTEROLOGY & HEPATOLOGY (,44) PSYCHIATRY (,47) RHEUMATOLOGY (,69) PEDIATRICS (,23) MULTIDISCIPLINARY SCIENCES (2,69) CELL BIOLOGY (,47) OBSTETRICS & GYNECOLOGY (,32) MICROBIOLOGY (,46) MEDICINE, RESEARCH & EXPERIMENTAL (,45) RESPIRATORY SYSTEM (,57) INFECTIOUS DISEASES (,33) NUTRITION & DIETETICS (,35) HEALTH CARE SCIENCES & SERVICES (,38) PATHOLOGY (,49) DERMATOLOGY (,66) DENTISTRY/ORAL SURGERY & MEDICINE (,2) 3,6 3,5 3,4 3,3 3,2 3,2 3, 3 3 2,9 2,6 2,6 2,5 2,4 2,3 2,3 2,2 2, 2,9,7,4,3,3,3,2, 4,6 5,9 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 Share of the output (%) 43

Figure 9b: Output and normalized impact per field (2009-203/204) All UMCs together Field (MNCS) ONCOLOGY (,53) CARDIAC & CARDIOVASCULAR SYSTEMS (,6) MULTIDISCIPLINARY SCIENCES (2,3) CLINICAL NEUROLOGY (,69) SURGERY (,47) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,48) GENETICS & HEREDITY (,88) NEUROSCIENCES (,56) ENDOCRINOLOGY & METABOLISM (,39) MEDICINE, GENERAL & INTERNAL (3,97) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,47) IMMUNOLOGY (,47) HEMATOLOGY (,45) BIOCHEMISTRY & MOLECULAR BIOLOGY (,45) PSYCHIATRY (,65) RHEUMATOLOGY (,76) PERIPHERAL VASCULAR DISEASE (,42) PHARMACOLOGY & PHARMACY (,22) GASTROENTEROLOGY & HEPATOLOGY (,59) UROLOGY & NEPHROLOGY (,92) PEDIATRICS (,27) CELL BIOLOGY (,8) OBSTETRICS & GYNECOLOGY (,42) HEALTH CARE SCIENCES & SERVICES (,27) INFECTIOUS DISEASES (,45) MEDICINE, RESEARCH & EXPERIMENTAL (,56) RESPIRATORY SYSTEM (,55) NUTRITION & DIETETICS (,35) MICROBIOLOGY (,63) REHABILITATION (,3) ORTHOPEDICS (,25) DENTISTRY/ORAL SURGERY & MEDICINE (,28) CRITICAL CARE MEDICINE (,58) 3,7 3,6 3,4 3,4 3,2 3, 3 2,9 2,8 2,6 2,5 2,5 2,4 2,4 2,4 2,3 2,2 2,2 2,2,9,7,4,4,3,3,3,2 5 5,5 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 44

2.3 Landscape of Dutch academic medical centers In this section, the focus is on various aspects of scientific activity, like first authorships, international cooperation, single address output. As this section contains descriptions on various aspects of scientific research and publishing, its contents summarizes the situation among the Dutch academic medical centers. Figure 0a and Figure 0b contain the combined insight of output numbers in the period 998-203/204 and 2009-203/204 respectively, and the field normalized impact (MNCS) of this output. Figure a contains the combined insight of output numbers in the period 998-203/204 and the journal-to-field normalized impact (MNJS) of this output, while Figure b contains similar information for the period 2009-203/204. Figure 2a contains the combined insight of output numbers in the period 998-203/204 and the journal impact (MNCS/MNJS) of this output, whereas Figure 2b contains similar data for the period 2009-203/204. Figures 3a to 3c display the trend scores for the three impact indicators displayed in the three previous sets of graphs, MNCS, MNJS, and MNCS/MNJS. This is added to show the development behind the two snap shots presented by the previous graphs. Figure 4a and Figure 4b contain the output resulting from first authorships as a share of the total output of the eight Dutch academic medical centers, combined with the MNCS values of that part of their output, in the periods 998-203/204 and 2009-203/204. Figure 5 contains the trend impact data behind the development of first authored publications by each of the Dutch UMCS. Figure 6a and Figure 6b focus on the part of the output of the Dutch academic medical centers that results from international collaboration, while Figure 8a and Figure 8b concentrate on the share of the output that is the results from every individual academic medical center, in which no other institute is involved. Figures 7 and 9 contain trend impact data (MNCS) for both the characteristics of scientific activity, international co-publishing and single address output, to show the development behind the two sets of graphs presented previously. 45

Figure 0a: Output compared to average field impact, 998-203/204,80,70,60,50 VUmc UU UMC UMC Maastricht LU MC UvA AMC Erasmus MC,40 UMCG Radboud UMC mncs,30,20,0,00 0,90 0,80 0,00 5000,00 0000,00 5000,00 20000,00 25000,00 30000,00 35000,00 Total publications Figure 0b: Output compared to average field impact, 2009-203/204 2,00,80,60 VUmc UU UMC LU MC Radboud UMC UMC Maastricht UMCG Erasmus MC UvA AMC mncs,40,20,00 0,80 0,00 2000,00 4000,00 6000,00 8000,00 0000,00 2000,00 4000,00 total publications 46

Figure a: Output compared to average journal-to-field impact, 998-203/204,50,40,30 UU UMC VUmc LU MC UMCG Radboud UMC UMC Maastricht UvA AMC Erasmus MC,20 mnjs,0,00 0,90 0,80 0,00 5000,00 0000,00 5000,00 20000,00 25000,00 30000,00 35000,00 total publications Figure b: Output compared to average journal-to-field impact, 2009-203/204,60,50,40 UU UMC LU MC UMCG Radboud UMC VUmc UMC Maastricht Erasmus MC UvA AMC,30 mnjs,20,0,00 0,90 0,80 0,00 2000,00 4000,00 6000,00 8000,00 0000,00 2000,00 4000,00 total publications 47

Figure 2a: Output compared to average journal impact, 998-203/204,25 VUmc,20 UMC Maastricht,5 Erasmus MC,0 mncs/ mnjs,05 UMCG UU UMC LU MC Radboud UMC UvA AMC,00 0,95 0,90 0,85 0,80 0,00 5000,00 0000,00 5000,00 20000,00 25000,00 30000,00 35000,00 total publications Figure 2b: Output compared to average journal impact, 2009-203/204,25,20 VUmc,5 UMC Maastricht Erasmus MC,0 mncs/ mnjs,05 Radboud UMC LU MC UU UMC UMCG UvA AMC,00 0,95 0,90 0,85 0,80 0,00 2000,00 4000,00 6000,00 8000,00 0000,00 2000,00 4000,00 total publications 48

Figure 3a: Trend analysis of the field-normalized impact (MNCS) for all Dutch UMCS, 998-203/204. 2,00,80,60 mncs,40 Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG,20 UU UMC UvA AMC VUmc,00 0,80 998-200 999-2002 2000-2003 200-2004 2002-2005 2003-2006 2004-2007 2005-2008 2006-2009 2007-200 2008-20 2009-202 200-203 total publications Figure 3b: Trend analysis of the journal-to-field normalized impact (MNJS) for all Dutch UMCS, 998-203/204.,60,50,40 mnjs,30,20 Erasmus MC LU MC Radboud UMC UMC Maastricht,0 UMCG UU UMC UvA AMC,00 VUmc 0,90 0,80 998-200 999-2002 2000-2003 200-2004 2002-2005 2003-2006 2004-2007 2005-2008 2006-2009 2007-200 2008-20 2009-202 200-203 total publications 49

Figure 3c: Trend analysis of the average journal impact, (MNCS/MNJS) for all Dutch UMCS, 998-203/204.,30,25,20,5,0 mncs/ mnjs,05,00 0,95 Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG UU UMC UvA AMC VUmc 0,90 0,85 0,80 998-200 999-2002 2000-2003 200-2004 2002-2005 2003-2006 2004-2007 2005-2008 total publications 2006-2009 2007-200 2008-20 2009-202 200-203 50

Figure 4a: Output compared to average field impact (MNCS), first authorships only, 998-203/204,50,40 VUmc Erasmus MC mncs,30,20 UU UMC UvA AMC LU MC UMC Maastricht Radboud UMC,0 UMCG,00 0,90 0,80 0 2000 4000 6000 8000 0000 2000 4000 6000 8000 total publications Figure 4b: Output compared to average field impact (MNCS), first authorships only, 2009-203/204,60,40 VUmc UMC Maastricht LU MC UU UMC Radboud UMC UvA AMC Erasmus MC,20 UMCG,00 mncs 0,80 0,60 0,40 0,20 0,00 0 000 2000 3000 4000 5000 6000 7000 total publications 5

Figure 5: Trend analysis of the field-normalized impact (MNCS) for all Dutch UMCS (first authorship output), 998-203/204.,60,50,40,30 mncs,20 Erasmus MC LU MC Radboud UMC UMC Maastricht,0 UMCG UU UMC UvA AMC,00 VUmc 0,90 0,80 998-200 999-2002 2000-2003 200-2004 2002-2005 2003-2006 2004-2007 2005-2008 2006-2009 2007-200 2008-20 2009-202 200-203 total publications 52

Figure 6a: Output compared to average field impact (MNCS), international cooperation only, 998-203/204 2,40 2,20 VUmc UU UMC Erasmus MC 2,00,80 UMCG LU MC UMC Maastricht Radboud UMC UvA AMC mncs,60,40,20,00 0,80 0 2000 4000 6000 8000 0000 2000 4000 total publications Figure 6b: Output compared to average field impact (MNCS), international cooperation only, 2009-203/204 2,40 Erasmus MC 2,20 2,00 UMCG VUmc UU UMC LU MC Radboud UMC UvA AMC UMC Maastricht mncs,80,60,40,20,00 0,80 0 000 2000 3000 4000 5000 6000 7000 total publications 53

Figure 7: Trend analysis of the field-normalized impact (MNCS) for all Dutch UMCS (international cooperation output), 998-203/204 2,60 2,40 2,20 2,00,80 mncs,60,40,20 Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG UU UMC UvA AMC VUmc,00 0,80 998-200 999-2002 2000-2003 200-2004 2002-2005 2003-2006 2004-2007 2005-2008 2006-2009 2007-200 2008-20 2009-202 200-203 total publications 54

Figure 8a: Output compared to average field impact (MNCS), single institute only, 998-202/2023,40 UMC Maastricht,30 VUmc Erasmus MC UU UMC mncs,20,0 UMCG LU MC UvA AMC Radboud UMC,00 0,90 0,80 0 000 2000 3000 4000 5000 6000 7000 8000 total publications Figure 8b: Output compared to average field impact (MNCS), single institute only, 2009-203/204,50,40 UMC Maastricht,30 VUmc UU UMC Erasmus MC,20 LU MC Radboud UMC UvA AMC mncs,0 UMCG,00 0,90 0,80 0 500 000 500 2000 2500 3000 total publications 55

Figure 9: Trend analysis of the field-normalized impact (MNCS) for all Dutch UMCS (single institute output), 998-203/204.,60,50,40,30 mncs,20 Erasmus MC LU MC Radboud UMC UMC Maastricht,0 UMCG UU UMC UvA AMC,00 VUmc 0,90 0,80 998-200 999-2002 2000-2003 200-2004 2002-2005 2003-2006 2004-2007 2005-2008 2006-2009 2007-200 2008-20 2009-202 200-203 total publications 56

2.4 Analysis of top-research Per medical center, a break-down into different classes of journal impact is made. While the data in Table show the overall situation, Table 5 contains the results of an analysis when the publications from each medical center are analyzed according to the journalto-field impact classes that the publications can be counted in. We distinguish five classes, the first class is the set of publications where the MNJS of the journal is ranging between 0.00 and 0.40 (Class A), the second class is ranging from 0.4 to 0.80 (Class B), the third class is ranging from 0.8 to.20 (Class C), the fourth class ranges from.2 to.60 (Class D), while finally the fifth class contains all publications in journals with a higher MNJS value as.6 (Class E). 57

Table 5a: Distribution of output of Dutch academic medical centers over journal impact classes, 998-203/204 P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov Erasmus MC A 442,75 83,25 7,75 0,33 0,28 % 7% 9% 88% Erasmus MC B 630,00 90360,00 4,34 0,73 0,63 5% 7% 8% 89% Erasmus MC C 889,75 76088,75 9,97,09 0,99 % 5% 8% 89% Erasmus MC D 355,00 85720,25 27,7,55,29 7% 4% 6% 89% Erasmus MC E 9902,00 55390,00 55,94 2,92 2,49 36% 2% 5% 90% LU MC A 83,25 8908,00 7,53 0,3 0,27 % 9% 9% 89% LU MC B 4863,50 75270,25 5,48 0,74 0,63 5% 7% 8% 9% LU MC C 657,50 38906,75 2,3,4 0,99 % 5% 8% 92% LU MC D 2228,50 55694,75 24,99,32,29 5% 3% 7% 92% LU MC E 6978,75 352590,25 50,52 2,65 2,45 34% 2% 6% 93% Radboud UMC A 22,25 050,00 8,6 0,34 0,28 % 4% 7% 88% Radboud UMC B 5305,00 70752,50 3,34 0,69 0,63 4% 8% 7% 89% Radboud UMC C 6779,00 24884,25 8,42,06 0,98 0% 5% 8% 89% Radboud UMC D 2397,75 52290,25 2,8,35,30 6% 4% 6% 89% Radboud UMC E 6900,25 304828,25 44,8 2,62 2,38 33% 2% 6% 9% UMC Maastricht A 28,00 922,75 8,09 0,33 0,27 % 8% 20% 85% UMC Maastricht B 4630,75 66866,75 4,44 0,73 0,63 5% 8% 7% 86% UMC Maastricht C 6379,75 27382,25 9,97,5 0,99 % 5% 7% 87% UMC Maastricht D 234,00 6467,75 26,26,49,29 7% 3% 6% 87% UMC Maastricht E 607,25 309240,50 50,63 2,79 2,27 33% 2% 4% 88% 58

P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov UMCG A 098,75 7454,50 6,78 0,3 0,27 % 9% 8% 87% UMCG B 422,00 52450,75 2,43 0,68 0,63 4% 0% 7% 89% UMCG C 5955,50 0648,25 7,07,03 0,99 0% 6% 7% 89% UMCG D 280,25 4890,25 9,2,33,29 5% 6% 8% 89% UMCG E 64,25 250028,75 40,89 2,54 2,4 32% 3% 6% 90% UU UMC A 858,75 668,00 7,7 0,34 0,28 % 6% 6% 89% UU UMC B 4873,00 6964,50 4,29 0,7 0,63 5% 8% 6% 9% UU UMC C 6658,75 33752,50 20,09,0 0,99 0% 5% 6% 9% UU UMC D 2509,00 6834,75 24,65,39,29 6% 4% 6% 9% UU UMC E 7675,25 392296,50 5, 2,76 2,46 35% 2% 4% 92% UvA AMC A 68,50 2668,00 7,53 0,32 0,27 % 20% 9% 88% UvA AMC B 6698,75 9590,25 3,67 0,7 0,62 5% 9% 7% 89% UvA AMC C 843,50 57823,25 9,38,09 0,99 0% 6% 7% 90% UvA AMC D 3227,25 79653,75 24,68,42,29 7% 4% 7% 89% UvA AMC E 878,25 406390,25 46,6 2,74 2,47 34% 3% 5% 9% VUmc A 92,00 799,25 7,89 0,34 0,28 % 20% 9% 88% VUmc B 4276,75 68,00 4,29 0,75 0,63 5% 8% 8% 88% VUmc C 6265,50 25979,50 20,,8 0,99 2% 5% 7% 89% VUmc D 2200,25 59370,75 26,98,57,29 20% 5% 6% 89% VUmc E 6386,00 335380,75 52,52 2,96 2,37 37% 3% 5% 90% 59

Table 5b: Distribution of output of Dutch academic medical centers over journal impact classes, 2009-203/204 P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov Erasmus MC A 367,75 090,00 2,96 0,33 0,28 0% 30% 22% 88% Erasmus MC B 288,75 3306,75 6,08 0,76 0,63 6% 5% 22% 89% Erasmus MC C 367,50 3238,50 8,82,0,00 % 9% 22% 89% Erasmus MC D 383,25 895,50 3,5,53,29 7% 7% 8% 89% Erasmus MC E 4352,00 5797,75 26,6 3,0 2,6 37% 4% 2% 9% LU MC A 304,75,50 3,65 0,37 0,29 2% 35% 23% 90% LU MC B 653,50 849,75 7,7 0,77 0,64 6% 3% 22% 92% LU MC C 28,50 2862,25 0,02,5,00 2% 8% 2% 92% LU MC D 900,25 9576,75 0,64,34,29 6% 7% 2% 92% LU MC E 3048,75 77767,50 25,5 2,76 2,59 35% 4% 22% 93% Radboud UMC A 29,25 068,00 3,67 0,4 0,30 % 29% 22% 89% Radboud UMC B 943,50 0,25 5,72 0,69 0,64 4% 6% 22% 89% Radboud UMC C 2852,50 24780,25 8,69,08 0,99 % 0% 2% 90% Radboud UMC D 096,25 033,25 9,24,32,30 5% 8% 22% 89% Radboud UMC E 3276,50 78236,25 23,88 2,86 2,5 37% 4% 20% 9% UMC Maastricht A 284,00 800,00 2,82 0,32 0,29 0% 37% 3% 84% UMC Maastricht B 625,00 9580,25 5,90 0,73 0,63 5% 7% 2% 87% UMC Maastricht C 2727,75 24034,00 8,8,3 0,99 % 0% 22% 88% UMC Maastricht D 008,00 0965,75 0,88,44,30 7% 6% 23% 89% UMC Maastricht E 2576,50 6257,00 23,78 2,89 2,4 36% 5% 20% 89% 60

P TCS MCS MNCS MNJS PP(top 0%) PP(uncited) %Self Cit Int Cov UMCG A 327,50 93,00 2,79 0,3 0,27 % 34% 25% 86% UMCG B 728,25 0540,50 6,0 0,7 0,63 5% 7% 9% 89% UMCG C 288,00 2294,75 7,96,02 0,99 9% % 2% 90% UMCG D 38,75 597,25 0,8,40,30 6% 0% 22% 89% UMCG E 36,75 7956,75 22,76 2,72 2,56 35% 5% 2% 9% UU UMC A 258,50 93,25 3,60 0,38 0,29 % 30% 2% 90% UU UMC B 769,75 0729,00 6,06 0,72 0,64 5% 5% 2% 9% UU UMC C 2948,25 26080,25 8,85,06,00 0% 9% 22% 92% UU UMC D 065,50 08,00 0,40,38,29 6% 7% 20% 9% UU UMC E 3536,00 85753,00 24,25 2,87 2,59 37% 4% 20% 93% UvA AMC A 545,50 44,25 2,59 0,3 0,27 % 36% 26% 88% UvA AMC B 2696,50 6279,00 6,04 0,72 0,63 5% 5% 22% 89% UvA AMC C 3797,00 386,75 8,39,04,00 0% 0% 2% 90% UvA AMC D 479,75 5427,75 0,43,39,30 7% 7% 22% 89% UvA AMC E 4088,75 969,50 23,70 2,93 2,6 36% 5% 9% 9% VUmc A 325,50 902,00 2,77 0,33 0,28 % 38% 26% 88% VUmc B 785,00 0335,50 5,79 0,76 0,63 6% 7% 23% 87% VUmc C 2966,75 26929,25 9,08,8 0,99 2% 9% 22% 89% VUmc D 992,50 0705,25 0,79,52,30 9% 9% 20% 88% VUmc E 3093,75 79399,25 25,66 3,06 2,50 38% 4% 2% 9% 6

Another method of showing the distribution over the five classes is shown in Figures 20a and 20b. While Figure 20a contains the alphabetical representation of the medical centers, the data in Figure 20b are sorted by descending share in the two highest classes (thus containing all publications in journals with a MNJS value higher than.20). Additional graph 20c displays the level of the impact (MNCS scores) for each of the academic medical centers, to provide an impression of the average effect of the publishing across various classes of journal impact. Similar analyses are conducted for the more recent period 2009-203/204, resulting in consecutive graphs Figure 2a (displaying the alphabetical; order of the UMC s and the distribution of output over classes of MNJS values), Figure 2b, which displays the ordered UMC s, by grouping the two highest MNJS-value classes, and finally Figure 2c, which displays the actual impact scores related to the distribution of output over MNJSclasses. Figure 20a: Distribution of output over journal-to-field impact classes (MNJS), 998-203/204 0% 0% 20% 30% 40% 50% 60% 70% 80% 90% 00% Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG 0 >MNJS <=0.40 0.40 > MNJS <= 0.80 0.80 > MNJS <=.20.20 > MNJS <=.60 MNJS >.60 UU UMC UvA AMC VUmc 62

Figure 20b: Distribution of output over journal-to-field impact classes (by descending highest share in the two classes MNJS >.20), 998-203/204 0% 0% 20% 30% 40% 50% 60% 70% 80% 90% 00% UU UMC Erasmus MC VUmc UMCG LU MC 0 >MNJS <=0.40 0.40 > MNJS <= 0.80 0.80 > MNJS <=.20.20 > MNJS <=.60 MNJS >.60 UvA AMC Radboud UMC UMC Maastricht Figure 20c: Impact level (MNCS) of output over journal-to-field impact classes, 998-203/204 3,50 3,00 2,50 mncs 2,00,50 0 >MNJS <=0.40 0.40 > MNJS <= 0.80 0.80 > MNJS <=.20.20 > MNJS <=.60 MNJS >.60,00 0,50 0,00 Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG UU UMC UvA AMC VUmc 63

Figure 2a: Distribution of output over journal-to-field impact classes (MNJS), 2009-203/204 0% 0% 20% 30% 40% 50% 60% 70% 80% 90% 00% Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG 0 >MNJS <=0.40 0.40 > MNJS <= 0.80 0.80 > MNJS <=.20.20 > MNJS <=.60 MNJS >.60 UU UMC UvA AMC VUmc Figure 2b: Distribution of output over journal-to-field impact classes (by descending highest share in the two classes MNJS >.20), 2009-203/204 0% 0% 20% 30% 40% 50% 60% 70% 80% 90% 00% UU UMC Erasmus MC UMCG Radboud UMC LU MC 0 >MNJS <=0.40 0.40 > MNJS <= 0.80 0.80 > MNJS <=.20.20 > MNJS <=.60 MNJS >.60 VUmc UvA AMC UMC Maastricht 64

Figure 2c: Impact level (MNCS) of output over journal-to-field impact classes, 2009-203/204 3,50 3,00 2,50 mncs 2,00,50 0 >MNJS <=0.40 0.40 > MNJS <= 0.80 0.80 > MNJS <=.20.20 > MNJS <=.60 MNJS >.60,00 0,50 0,00 Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG UU UMC UvA AMC VUmc Yet another aspect of publishing in top journals is shown in Figures 22 and 23. In Figure 22, one finds the number of publications in the field Medicine, General and Internal. This field contains general medicine journals with a multidisciplinary medical content, like The Lancet, JAMA, the New England Journal of Medicine, etc. The field also contains some local/national general medicine journals as well as journals in small specialties, like family practice, diabetes research, etc. In this profile, the benchmark indicator is not MNCS, but we have chosen MNCS/MNJS, as this is a more accurate indicator in this particular case, describing the performance in top-journals. Figure 23 contains the output numbers in international multidisciplinary journals like Science, Nature, and the Proceedings of the National Academy of Sciences of the USA. Again, as above, in this profile, the benchmark indicator is not MNCS, but we have chosen MNCS/MNJS, as this is a more accurate indicator in this particular case, describing the performance in top-journals. As in the previous research profiles, the color coding in the graph indicates the impact level, dark colored bars indicate a high impact, while the lighter colored bars indicate an average impact level. 65

Figure 22: Output numbers compared to average journal impact (MNCS/MNJS) in Medicine, general & internal, 2009-203/204. 0,00 00,00 200,00 300,00 400,00 500,00 600,00 Erasmus MC (,54) LU MC (,09) Radboud UMC (,28) UMC Maastricht (,47) UMCG (,6) UU UMC (,3) UvA AMC (,23) VUmc (,44) Figure 23: Output numbers compared to average journal impact (MNCS/MNJS) in Multidisciplinary journals, 2009-202/2023. 66

0,00 00,00 200,00 300,00 400,00 500,00 600,00 Erasmus MC (,34) LU MC (,44) Radboud UMC (,8) UMC Maastricht (,4) UMCG (,05) UU UMC (,9) UvA AMC (,09) VUmc (,2) 67

A third bibliometric approach of scientific top research or excellence is shown in Table 6 and Figure 24. As indicated before, the top paper analysis can be extended to other parts of the total publications set, and accordingly, other parts of the distribution of impact over publications. Here we present the presence of Dutch academic medical centers among respectively the top-50%, top-20%, the top-0%, the top-5%, the top-2% and the top-% most highly cited publications in the fields these centers are active in. In Table 6, we present the output that relates to the actual numbers of publications per UMC within the various sub-sets of the total distribution most highly cited publications in the fields these centers are active in, while Figure 24 displays these relative outcomes for the eight Dutch academic medical centers graphically. 68

Table 6: Overview of the presence of Dutch academic medical centers in the top-x most highly cited publications, 998-203/204 p 98-202 P in Top % PP(top %) P in Top 2% PP(top 2%) P in Top 5% PP(top 5%) P in Top 0% PP(top 0%) P in Top 20% PP(top 20%) P in Top 50% PP(top 50%) Erasmus MC 29620,5 702,3 2% 298,9 4% 294,8 0% 5404,0 8% 967,7 32% 9747,8 67% LU MC 277,5 433,0 2% 833,5 4% 936,6 9% 3669,4 7% 6748,6 3% 4270,0 66% Radboud UMC 22603,25 43,2 2% 79,6 4% 864,8 8% 3590,6 6% 6696,6 30% 4557,2 64% UMC Maastricht 20586,75 453, 2% 793, 4% 788,6 9% 3362,2 6% 6264, 30% 350,4 66% UMCG 9569,75 324,9 2% 668,9 3% 556,7 8% 3044,4 6% 576,7 29% 2633,6 65% UU UMC 22574,75 46, 2% 895,9 4% 229,9 9% 400,9 8% 732,5 32% 549,7 67% UvA AMC 28469,25 590,7 2% 070,0 4% 2478,3 9% 4692,7 6% 8672,2 30% 8386,6 65% VUmc 20040,5 468,3 2% 93, 5% 2073,8 0% 3778,8 9% 6764,4 34% 3598,0 68% 69

Figure 24: Presence of the Dutch academic medical centers in Top-50% / Top- 20% / Top-0% / Top-5% / Top-2% and Top-% most highly cited publications worldwide, 998-203/204 80% 70% 60% 50% 40% 30% PP(top_%) PP(top_2%) PP(top_5%) PP(top_0%) PP(top_20%) PP(top_50%) 20% 0% 0% Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG UU UMC UvA AMC VUmc 70

2.5 Analysis of the publication strategy: research profiles with MNJS values In this section, we focus on the journals in which the output was published across the fields in which the medical centers were active in. These profiles are similar to the research profiles, with this difference that the MNJS is now used to indicate the impact in the field, not as an indicator of the field impact, but rather as an indicator of the journal impact position in the field. The period analyzed is the period 2009-203/204. Figures 25 through 32 contain such profiles for the eight academic medical centers. 7

Figure 25: Output and journal impact per field (2009-203/204) Erasmus MC Field (MNJS) ONCOLOGY (,58) CARDIAC & CARDIOVASCULAR SYSTEMS (,54) SURGERY (,29) GENETICS & HEREDITY (2,04) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,46) ENDOCRINOLOGY & METABOLISM (,2) GASTROENTEROLOGY & HEPATOLOGY (,5) HEMATOLOGY (,53) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,3) CLINICAL NEUROLOGY (,63) PEDIATRICS (,29) PHARMACOLOGY & PHARMACY (,25) IMMUNOLOGY (,32) MEDICINE, GENERAL & INTERNAL (4,22) PERIPHERAL VASCULAR DISEASE (,38) MULTIDISCIPLINARY SCIENCES (2,64) NEUROSCIENCES (,56) UROLOGY & NEPHROLOGY (,78) BIOCHEMISTRY & MOLECULAR BIOLOGY (,52) PSYCHIATRY (,38) CELL BIOLOGY (,58) HEALTH CARE SCIENCES & SERVICES (,2) OBSTETRICS & GYNECOLOGY (,4) INFECTIOUS DISEASES (,7) RESPIRATORY SYSTEM (,44) RHEUMATOLOGY (,68) DERMATOLOGY (,55) MEDICINE, RESEARCH & EXPERIMENTAL (,59) MICROBIOLOGY (,26) CRITICAL CARE MEDICINE (,33) VIROLOGY (,08) 4,3 4,2 3,8 3,5 3,3 2,8 2,8 2,7 2,7 2,4 2,4 2,4 2,3 2,3 2,3 2,2,9,8,8,5,5,4,4,3,2,,,, 7,5 7,2 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 72

Figure 26: Output and journal impact per field (2009-203/204) LU MC Field (MNJS) ONCOLOGY (,46) CARDIAC & CARDIOVASCULAR SYSTEMS (,58) RHEUMATOLOGY (,69) GENETICS & HEREDITY (,99) ENDOCRINOLOGY & METABOLISM (,24) MULTIDISCIPLINARY SCIENCES (,73) HEMATOLOGY (,23) IMMUNOLOGY (,7) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,28) CLINICAL NEUROLOGY (,63) MEDICINE, GENERAL & INTERNAL (3,34) BIOCHEMISTRY & MOLECULAR BIOLOGY (,32) SURGERY (,38) NEUROSCIENCES (,25) PSYCHIATRY (,34) PERIPHERAL VASCULAR DISEASE (,4) CELL BIOLOGY (,38) OBSTETRICS & GYNECOLOGY (,24) UROLOGY & NEPHROLOGY (,43) PHARMACOLOGY & PHARMACY (,24) PEDIATRICS (,7) GASTROENTEROLOGY & HEPATOLOGY (,6) MEDICINE, RESEARCH & EXPERIMENTAL (,5) MICROBIOLOGY (,37) BIOCHEMICAL RESEARCH METHODS (,39) PATHOLOGY (,33) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,5) INFECTIOUS DISEASES (,3) VIROLOGY (,2) GERIATRICS & GERONTOLOGY (,27) RESPIRATORY SYSTEM (,29) TRANSPLANTATION (,24) 4,8 4,4 4,3 3,8 3,7 3,4 3,3 3 2,9 2,8 2,6 2,3 2,3 2,2,9,9,8,6,5,4,3,2,2,, 5,7 7 6,5 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 73

Figure 27: Output and journal impact per field (2009-203/204) Radboud UMC Field (MNJS) ONCOLOGY (,45) GENETICS & HEREDITY (,99) CLINICAL NEUROLOGY (,49) NEUROSCIENCES (,35) MULTIDISCIPLINARY SCIENCES (,72) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,45) UROLOGY & NEPHROLOGY (,8) SURGERY (,36) MEDICINE, GENERAL & INTERNAL (3,04) DENTISTRY/ORAL SURGERY & MEDICINE (,3) BIOCHEMISTRY & MOLECULAR BIOLOGY (,23) RHEUMATOLOGY (,57) IMMUNOLOGY (,4) PHARMACOLOGY & PHARMACY (,42) HEMATOLOGY (,26) MICROBIOLOGY (,37) ENDOCRINOLOGY & METABOLISM (,2) PSYCHIATRY (,48) OBSTETRICS & GYNECOLOGY (,37) CELL BIOLOGY (,35) INFECTIOUS DISEASES (,44) HEALTH CARE SCIENCES & SERVICES (,9) PEDIATRICS (,9) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,43) GASTROENTEROLOGY & HEPATOLOGY (,45) CARDIAC & CARDIOVASCULAR SYSTEMS (,35) DERMATOLOGY (,49) MEDICINE, RESEARCH & EXPERIMENTAL (,37) OTORHINOLARYNGOLOGY (,3) ENGINEERING, BIOMEDICAL (,22) PERIPHERAL VASCULAR DISEASE (,23) REHABILITATION (,29) PATHOLOGY (,28) RESPIRATORY SYSTEM (,25) 3,6 3,3 2,9 2,9 2,9 2,8 2,8 2,5 2,4 2,4 2, 2, 2 2,9,7,7,7,6,5,5,5,4,3,3,2, 4,3 4,2 5,3 6,5 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 Share of the output (%) 74

Figure 28: Output and journal impact per field (2009-203/204) UMC Maastricht Field (MNJS) CARDIAC & CARDIOVASCULAR SYSTEMS (,45) ONCOLOGY (,44) PERIPHERAL VASCULAR DISEASE (,2) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,3) NUTRITION & DIETETICS (,35) ENDOCRINOLOGY & METABOLISM (,33) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,28) CLINICAL NEUROLOGY (,44) MULTIDISCIPLINARY SCIENCES (,45) PSYCHIATRY (,52) SURGERY (,33) NEUROSCIENCES (,2) MEDICINE, GENERAL & INTERNAL (3,33) HEALTH CARE SCIENCES & SERVICES (,22) GENETICS & HEREDITY (,27) RHEUMATOLOGY (,46) BIOCHEMISTRY & MOLECULAR BIOLOGY (,5) HEMATOLOGY (,29) UROLOGY & NEPHROLOGY (,28) OBSTETRICS & GYNECOLOGY (,44) PHARMACOLOGY & PHARMACY (,9) RESPIRATORY SYSTEM (,28) GASTROENTEROLOGY & HEPATOLOGY (,25) PSYCHOLOGY, MULTIDISCIPLINARY (,4) REHABILITATION (,2) PEDIATRICS (,7) OPHTHALMOLOGY (,28) EDUCATION, SCIENTIFIC DISCIPLINES (,32) CELL BIOLOGY (,25) TOXICOLOGY (,38) GERIATRICS & GERONTOLOGY (,29) MEDICINE, RESEARCH & EXPERIMENTAL (,44) IMMUNOLOGY (,04) NURSING (,55) PHYSIOLOGY (,4) 3,7 3,6 3,6 3,3 3,2 3, 3 2,9 2,9 2,8 2,5 2,4 2,3 2,2 2,9,9,8,7,6,5,4,3,3,3,3,3,,,, 5,2 4,7 4,4 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 75

Figure 29: Output and journal impact per field (2009-203/204) UMCG Field (MNJS) CARDIAC & CARDIOVASCULAR SYSTEMS (,52) ONCOLOGY (,47) PSYCHIATRY (,53) MULTIDISCIPLINARY SCIENCES (,66) GENETICS & HEREDITY (2) MEDICINE, GENERAL & INTERNAL (3,78) SURGERY (,26) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,3) CLINICAL NEUROLOGY (,32) NEUROSCIENCES (,4) PEDIATRICS (,36) PHARMACOLOGY & PHARMACY (,28) ENDOCRINOLOGY & METABOLISM (,3) DENTISTRY/ORAL SURGERY & MEDICINE (,23) BIOCHEMISTRY & MOLECULAR BIOLOGY (,9) HEMATOLOGY (,29) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,2) UROLOGY & NEPHROLOGY (,6) PERIPHERAL VASCULAR DISEASE (,26) REHABILITATION (,8) RESPIRATORY SYSTEM (,38) IMMUNOLOGY (,24) GASTROENTEROLOGY & HEPATOLOGY (,7) RHEUMATOLOGY (,46) OBSTETRICS & GYNECOLOGY (,35) PSYCHOLOGY, MULTIDISCIPLINARY (,49) SPORT SCIENCES (,3) MEDICINE, RESEARCH & EXPERIMENTAL (,9) CELL BIOLOGY (,26) HEALTH CARE SCIENCES & SERVICES (,4) ORTHOPEDICS (,06) MICROBIOLOGY (,37) TRANSPLANTATION (,29) DERMATOLOGY (,53) 3,2 3,2 3, 3 2,9 2,6 2,4 2,3 2,3 2,3 2,3 2,3 2,2 2,2 2,2 2, 2,9,8,8,7,4,2,2, 4, 4 3,8 5,5 5,3 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 76

Figure 30: Output and journal impact per field (2009-203/204) UU UMC Field (MNJS) ONCOLOGY (,42) CARDIAC & CARDIOVASCULAR SYSTEMS (,4) MULTIDISCIPLINARY SCIENCES (,98) CLINICAL NEUROLOGY (,67) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,25) SURGERY (,39) HEMATOLOGY (,25) NEUROSCIENCES (,47) PERIPHERAL VASCULAR DISEASE (,25) GENETICS & HEREDITY (2,06) MEDICINE, GENERAL & INTERNAL (3,68) IMMUNOLOGY (,22) CELL BIOLOGY (,73) PHARMACOLOGY & PHARMACY (,27) PEDIATRICS (,37) BIOCHEMISTRY & MOLECULAR BIOLOGY (,7) RHEUMATOLOGY (,47) GASTROENTEROLOGY & HEPATOLOGY (,49) PSYCHIATRY (,65) ENDOCRINOLOGY & METABOLISM (,27) OBSTETRICS & GYNECOLOGY (,48) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,66) UROLOGY & NEPHROLOGY (,37) INFECTIOUS DISEASES (,52) RESPIRATORY SYSTEM (,42) NUTRITION & DIETETICS (,4) MICROBIOLOGY (,44) MEDICINE, RESEARCH & EXPERIMENTAL (,43) REHABILITATION (,2) CRITICAL CARE MEDICINE (,4) 3,7 3,4 3,3 3,2 3, 2,9 2,9 2,8 2,7 2,6 2,6 2,5 2,5 2,4 2,3 2 2,7,5,4,3,2,2, 5, 4,8 4,6 4,5 4,5 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 Share of the output (%) 77

Figure 3: Output and journal impact per field (2009-203/204) UvA AMC Field (MNJS) CARDIAC & CARDIOVASCULAR SYSTEMS (,59) MULTIDISCIPLINARY SCIENCES (,8) ONCOLOGY (,4) MEDICINE, GENERAL & INTERNAL (3,08) GASTROENTEROLOGY & HEPATOLOGY (,54) SURGERY (,4) HEMATOLOGY (,22) GENETICS & HEREDITY (,55) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,22) UROLOGY & NEPHROLOGY (,9) IMMUNOLOGY (,48) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,24) PERIPHERAL VASCULAR DISEASE (,34) CLINICAL NEUROLOGY (,53) OBSTETRICS & GYNECOLOGY (,42) INFECTIOUS DISEASES (,27) ENDOCRINOLOGY & METABOLISM (,2) PEDIATRICS (,22) BIOCHEMISTRY & MOLECULAR BIOLOGY (,22) PSYCHIATRY (,3) PHARMACOLOGY & PHARMACY (,29) NEUROSCIENCES (,3) RHEUMATOLOGY (,69) CRITICAL CARE MEDICINE (,3) MEDICINE, RESEARCH & EXPERIMENTAL (,26) CELL BIOLOGY (,5) VIROLOGY (,06) HEALTH CARE SCIENCES & SERVICES (,7) RESPIRATORY SYSTEM (,32) ORTHOPEDICS (,07) 4,2 4,2 4 4 3,9 3,4 3,3 3,2 3, 3, 2,8 2,8 2,7 2,6 2,5 2,5 2,5 2,4 2,4,9,8,8,8,4,4,3,3,2, 6, Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 Share of the output (%) 78

Figure 32: Output and journal impact per field (2009-203/204) VUmc Field (MNJS) ONCOLOGY (,49) CLINICAL NEUROLOGY (,78) ENDOCRINOLOGY & METABOLISM (,27) PSYCHIATRY (,38) NEUROSCIENCES (,52) PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH (,28) MULTIDISCIPLINARY SCIENCES (,82) RHEUMATOLOGY (,47) GENETICS & HEREDITY (2,04) RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING (,39) MEDICINE, GENERAL & INTERNAL (2,72) CARDIAC & CARDIOVASCULAR SYSTEMS (,55) PEDIATRICS (,7) IMMUNOLOGY (,43) SURGERY (,36) BIOCHEMISTRY & MOLECULAR BIOLOGY (,9) OBSTETRICS & GYNECOLOGY (,26) NUTRITION & DIETETICS (,7) GASTROENTEROLOGY & HEPATOLOGY (,45) HEALTH CARE SCIENCES & SERVICES (,5) HEMATOLOGY (,36) PHARMACOLOGY & PHARMACY (,35) GERIATRICS & GERONTOLOGY (,29) PERIPHERAL VASCULAR DISEASE (,3) CELL BIOLOGY (,35) UROLOGY & NEPHROLOGY (,2) MEDICINE, RESEARCH & EXPERIMENTAL (,52) PSYCHOLOGY, MULTIDISCIPLINARY (,47) REHABILITATION (,26) ORTHOPEDICS (,05) PATHOLOGY (,35) SPORT SCIENCES (,47) RESPIRATORY SYSTEM (,44) INFECTIOUS DISEASES (,23) CRITICAL CARE MEDICINE (,37) 4,2 4 3,9 3,8 3,3 3,2 3, 2,9 2,8 2,7 2,6 2,4 2,3 2, 2,,9,9,9,9,6,5,5,4,3,3,2,, 6,2 7 Low (< 0,8) Average High (>,2) 0 2 3 4 5 6 7 8 Share of the output (%) 79

2.6 The Brute force indicator for Dutch UMCS (P * MNCS values) In this section, the focus will be on what is called the brute force of a research organization, analog to the ranking of universities (see http://www.leidenranking.com the rankings of universities). Brute force is the product of the total number of publications in a period (P, multiplied by the field-normalized impact score MNCS. The advantage of this indicator is found in the combination of two indicators that normally describe two different aspects or dimensions of scientific performance. Figure 34 contains the trend in Brute force scores for the eight Dutch academic medical centers Figure 33: Trend in the Brute Force of Dutch UMCS (P*MNCS), 998-203/204. 20000 8000 6000 4000 2000 0000 8000 6000 Erasmus MC LU MC Radboud UMC UMC Maastricht UMCG UU UMC UvA AMC VUmc 4000 2000 0 998-200 999-2002 2000-2003 200-2004 2002-2005 2003-2006 2004-2007 2005-2008 2006-2009 2007-200 2008-20 2009-202 200-203 80

3. Final remarks We start this final section with a few general comments on the use of bibliometric indicators for the assessment of research performance. It is our experience in previous studies on research performance in the natural and life sciences, medicine, the humanities, and in the social and behavioral sciences, that bibliometric indicators provide useful information to a peer review committee evaluating research performance. These studies revealed a fair correspondence between the results of bibliometric analyses on the one hand, and judgments on scientific quality by peers on the other hand. In our view, a quality judgment on a research unit, department or institute can only be given by peers, based on a detailed insight into content and nature of the research conducted by the group or institute in question. The citation-based indicators applied in this study, measure the impact at the short or middle-long term of research activities at the international research front, as reflected in publication and citation patterns. Impact and scientific quality are not necessarily identical concepts. Bibliometric indicators cannot be interpreted properly without background knowledge on both the research units that are evaluated, and the subfields in which the research units are active. In fact, in previous studies we have encountered a few cases in which a bibliometric indicator pointed in one direction (e.g., a low impact), while statements by peers or even other indicators pointed in another direction (e.g., a high quality). Analyzing such discrepancies from a bibliometric point of view, specific limitations related to the bibliometric methodology applied in the study in question may be identified. While in most cases such limitations do hardly affect the results or have no effect at all, in exceptional cases the bibliometric outcomes may provide an incomplete or even distorted picture. For instance, the classification of journals into subfields ( journal categories ) may be less appropriate for some research units, particularly when they are active in topics of a multidisciplinary nature. Then, in the calculation of the impact compared to the world subfield citation average, this world average may not be representative for the subfield in which such a research group or institute is active. If there are strong indications that the definition of the (sub)field in terms of WOS journal categories is inadequate, then the journal-based world average (JCSm) is more appropriate. In particular, this latter case pertains to developing new interdisciplinary fields. 8

A second limitation concerns the coverage of the Citation Indices (CI). In specific subfields, particularly in applied or technical sciences, the WOS coverage may be less adequate. Consequently, for research units who are active in such technical/applied subfields, the bibliometric results may provide an incomplete picture. A second point concerns non-wos publications (e.g., articles in journals that are not or no longer covered by CI). For a number of research units, valuable additional information may be obtained by retrieving impact data for non-wos publications. Another example of a limitation of bibliometric analysis relates to time delays. It may take several years for a collection of papers to generate a high impact. We have analyzed research units that had generated only a moderate impact at the time. Confronted with the bibliometric results, several peers stated that these research units had recently made important contributions to the field. When we updated the results after a few years, several research units indeed showed a sharply rising impact curve. We do not wish to imply that all discrepancies between bibliometric indicators and peer judgments are necessarily due to problems or limitations of the bibliometric methods applied. Equally, it would not be appropriate to attribute such discrepancies only to peers expressing incorrect or biased views on the scientific quality of a research unit. Still reasoning from the point of view of the bibliometrician, discrepancies between bibliometric indicators and peer judgments often constitute a research problem in itself and often, a considerable effort is required to examine a discrepancy in sufficient detail. Nevertheless, also peer review has its disadvantages (van Raan 996). Therefore, the appropriate combination of peer-based qualitative assessment and quantitative, particularly bibliometric indicators appears to be the most successful approach in order to reinforce objectivity, transparency, comparability and reproducibility in the assessment of research performance. 82

Literature Glänzel, W. (992). Publication Dynamics and Citation Impact: A Multi-Dimensional Approach to Scientometric Research Evaluation. In: P. Weingart, R. Sehringer, M. Winterhager (Eds.), Representations of Science and Technology. DSWO Press, Leiden 992, 209-224. Proceedings of the International Conference on Science and Technology Indicators, Bielefeld (Germany), 0-2 June, 990. Gonzale-Albo, B & M. Bordons (20). Articles vs. proceedings papers: Do they differ in research relevance and impact? A case study in the Library and Information Science field. Journal of Informetrics, 5, 369-38. Leeuwen, T.N. van, E.J. Rinia and A.F.J. van Raan (996). Bibliometric Profiles of Academic Physics Research in the Netherlands. Research Report to the Netherlands Organisation for Scientific Research (NWO), Physics Division (FOM), Utrecht. Leiden: Center for Science and Technology Studies, Report 96-09. Moed, H.F., R.E. de Bruin and T.N. van Leeuwen (995). New Bibliometric Tools for the Assessment of National Research Performance: Database Description Overview of Indicators and First Applications. Scientometrics, 33, 38-425. Nederhof, A.J., (988), The validity and reliability of evaluation of scholarly performance. In: A.F.J. van Raan (ed) handbook of quantitative studies of science and technology. Amsterdam: North Holland/Elsevier Science Publishers, pp. 93-228 Price, D.S. (963), Little Science, Big Science. New York: Columbia University Press van Raan, A.F.J. (996), Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises, Scientometrics, 36, 397-420. van Raan, A.F.J. (2000). The Interdisciplinary Nature of Science. Theoretical Framework and Bibliometric-Empirical Approach. In: P. Weingart and N. Stehr (Eds.). Practicing Interdisciplinarity. Toronto: University of Toronto Press. Waltman, L., N.J.van Eck, T.N. van Leeuwen, M.S. Visser, and A.F.J. van Raan, Towards a new crown indicator: Some theoretical considerations, Journal of Informetrics, 20, 5 (), 37-47 Waltman, L., N.J.van Eck, T.N. van Leeuwen, M.S. Visser, and A.F.J. van Raan, Towards a new crown indicator: An empirical analysis. Scientometrics, 20, 87 (3), page 467-48 83

Appendix A: Explanation of the bibliometric indicators of CWTS 84

. Data collection. Introduction Bibliometrics is the quantitative study of written products of research. It is assumed that scientific subjects develop at an international research front (Price, 963). Research results are communicated in publications that are submitted to evaluation by professional colleagues. In the references of their papers, scientists acknowledge relevant publications by others, as they build on previous work. Therefore, the number of times a publication is referred to gives a partial indication of the impact of a publication, its reception and use by scientists at the research front. In nearly all scientific fields, the scientific journal is by far the most important medium of communication. The Web of Science database (from here on WoS), which consists of the citation indexes known under acronyms such as SCI, SSCI and A&HCI claims to cover the most important leading international journals and serials (such as Annual Reviews) with a well-functioning referee system. In addition, the overall citation rate of journals is considered, as well as their timeliness of publication, and adherence to international editorial conventions. Regularly, a limited number of new journals are added, while other journals are no longer covered. More peripheral journals, often national in scope, are usually not covered by the CI. The WoS counts about,000 journals during the last decade. Both statistical requirements and imperfections in the citation process (for a discussion see Nederhof, 988) make it desirable to aggregate across individuals, publications, and citations. As scientific (sub)fields differ in publication and citation patterns (as visible in differences in for example length of reference lists, or age of cited literature), it is usually not meaningful to compare directly the raw impact of publications from one (sub)field with those of a different (sub)field. Therefore, in our studies raw impact scores are compared to the impact of similar publications within the same journal, or within the same (sub)field..2 Specifics on data collection The present study relates to the publication output of the academic medical centers at Dutch universities. The medical centers supplied publication lists to CWTS, which were matched with the CWTS in-house bibliometric data-system. The bibliometric analysis is covering the period 998-203/204 for all eight academic medical centers. This study 85

is an update of the study conducted in 204/205, for the update CWTS was supplied with the year 203 as an additional year to the data collected for the previous studies. Actually, the publication data collected in this study are publication years: papers are included for the year in which they were published in the journal, which does not necessarily coincide with the moment the publications are processed for the WoS. In this year s analysis, the option was suggested to backwardly add publications to the existing datasets for de UMC benchmark analysis. For several years, the dataset is only altered with publication form the last year, now three UMCS have taken the opportunity to add additional publications to the dataset used for the UMC benchmark study. The three UMCS that had used this option are UMCG, UvA AMC, and LU MC. The reason for the other UMCS not to supply with additional publication data was the time aspect in the data collection against the light of the deadline for delivering publication data for the benchmark study. We considered only papers classified in the WoS as normal articles, letters and reviews, published in source serials processed for the WoS database. Please note that in the indicator set of CWTS (see below), letters are weighted with 0.25. Other document types, such as meeting abstracts, editorials, editorial material, corrections, comments, and book reviews were not included. Also, papers in non-wos source journals are not counted. A few journals are only partially processed for the WoS. Here, only papers processed for the WoS were included. A statement needs to be made on a specific type of articles in the WoS database. This concerns the articles published in journals, but forthcoming from a scientific conference. These regular journal articles were in 2008 re-classified as proceedings papers, but after a short while, realizing this was an error, this was changed again in to article/proceeding paper. For this study, this does not influence the results, as we still consider these publications as regular journal articles, and are treated as such. In the future, when the CWTS WoS database also contains proceedings publications, a new situation arises, and adequate steps will be taken to handle these new document types correctly (for a report of this issue see Gonzalez-Albo & Bordons, 20). 86

2. Bibliometric indicators 2. Output and impact indicators We calculated the following indicators. The numbering of the indicators corresponds to the position these indicators have in the data tables. A first statistic gives the total number of papers published by the research unit during the entire period (P). We considered only papers classified as normal articles, letters, and reviews. Meeting abstracts, corrections, and editorials are not included. In a few cases, a paper is published in a journal for which no citation data are available, or that is not assigned to a CI journal category. These papers are not considered in the calculation of the indicators presented in the tables below. The next indicator gives the total number of citations received, without self-citations (TCS). In the calculation of all our impact indicators, we disregard author self-citations. We classify a citation as an author self-citation if the citing publication and the cited publication have at least one author name (i.e., last name and initials) in common. In this way, we ensure that our indicators focus on measuring only the contribution and impact of the work of a researcher on the work of other members of the scientific community. Sometimes self-citations can serve as a mechanism for self-promotion rather than as a mechanism for indicating relevant related work. The impact of the work of a researcher on his own work is therefore ignored. As an indication of the self-citation rate we present the percentage of self-citations (% Self Cit), relative to the total number of citations received (Self Cit)). A next indicator is the average number of citations per publication calculated while selfcitations are not included (MCS). Another indicator is the percentage of articles not cited during the time period considered (PP(uncited)), excluding self-citations. International reference values: FCS and JCS Two international reference values are computed. A first value represents the expected mean citation rate of the subfields (journal categories) in which the research unit is active (FMCS, the Field Mean Citation Score). Our definition of subfields is based on a classification of scientific journals into categories developed by Thomson Reuters. Although this classification is certainly not perfect, it is at present the only classification available in the WoS. In calculating FMCS, we apply the following procedure. The FMCS takes into account both the type of paper (e.g., normal article, review, and so on), as well as the specific years in which the research unit's papers were published. For example, the number of citations received during the period 2005-202 by 87

a letter published by a research unit in 2005 in field X is compared to the average number of citations received during the same period (2005-202) by all letters published in the same field (X) in the same year (2005). Self-citations are excluded from the computation of FMCS. In most cases, a research unit is active in more than one subfield (i.e., journal category). In those cases, we apply various field impact scores, as related to the individual publications, the selection of the fields being determined by the journals the research unit has used to publish its research findings. When a journal is classified in multiple subfields, as happens frequently in the WoS, citation scores are computed as follows. Basically, a paper in a journal classified in N subfields is counted as /N paper in each subfield, and so are its FMCS scores, so this creates per individual publication an expected mean field citation score. The second reference value presents the expected mean citation rate of the journals in which the research unit has published (JMCS, the Journal Mean Citation Score). We apply the same exact method for calculating JMCS as we did for FMCS. Main indicators The most important indicators compare the number of citations per individual publication within the output of a research unit (CS) to the two international reference values, namely the corresponding field and journal expected citation scores of individual publications (JMCS and FMCS, respectively), by calculating the ratio for every single publication against both expected citation scores. Self-citations are excluded in the calculation of the ratios CS/FMCS and CS/JMCS, to prevent that citation scores are affected by divergent self-citation behavior. Over all ratios of individual publications, we calculate a mean impact score, for both the fields as well as the journals in which the institute has published. The overall field normalized impact indicator for an institute output is MNCS, the Mean Normalized Citation Score. As this indicator focuses on the broader environment of the group s output, this indicator seems the most suitable indicator of the international position of a research unit. If the MNCS is above (below).0, this means that the output of the research unit is cited more (less) frequently than an 'average' publication in the subfield(s) in which the research unit is active. The FMCS values of the individual publications constitute a world subfield average in a specific (combination of) subfield(s). In this way, one may obtain an indication of the international position of a research unit, in terms of its impact compared to a 'world' average. This 'world' average is calculated for the total population of articles published in WoS journals assigned to a particular subfield or journal category. As a rule, about 70-80 percent of these papers are authored by scientists from the United States, Canada, Western Europe, Australia and Japan. Therefore, this 'world' average is dominated by the Western world. 88

A second important indicator, MNJS, is above (below).0 if the citation score of the journal set in which the research unit has published exceeds the citation score of all papers published in the subfield(s) to which the journals belong. In this case, one can conclude that the research unit publishes in journals with a relatively high (low) impact. The MNCS/MNJS indicator indicates the impact of papers closely to the publication pattern of research units, that is, the journals in which the units have published. If the ratio MNCS/MNJS is above.0, the impact of a research unit's papers exceeds the impact of all articles published in the set of journals in which the particular research unit has published its papers (the research unit's journal set). A limitation of this indicator is that low impact publications published in low impact journals may get a similar score as high impact publications published in high impact journals. Due to this latter disadvantage, we do not print these scores in the data tables, the indicator is present in the landscaping part of the study however, just as in the part of the study focusing on visibility in top journals in general medicine and multidisciplinary sciences. It should be noted that the MNCS, MNJS and the MNCS/MNJS indicators are not independent. The value of each one of these follows directly from the values of the other two indicators. For a more detailed discussion on the differences between the former set of CWTS indicators and the current one we refer to the Appendix B. 2.2 Research profiles: analysis of disciplinary orientation The disciplinary orientation or research profile of a research center is analyzed by classifying its papers according to scientific (sub-)fields. In the WoS, publications are classified by means of the journal in which they appear into (sub-)field categories such as Genetics & heredity, Oncology, Virology, and so on. These WoS journal subject categories are attached to each publication of a research unit. Subsequently, these publications are aggregated for each WoS subfield, and output and impact indicators are computed separately for these aggregates. The purpose of this procedure is to show how frequently a center has published papers in various subfields of science, what and how the impact of the center is in its main subfield(s) of activity. If a paper appears in a journal that is classified in more than one subject category, the paper (and its citations) is proportionally distributed over the subject categories. Thus, a paper with 7 citations published in a journal categorized in three subject categories is counted as 0.33 publication with 2.33 citations in each subject category. 89

As an indication, if the ratio MNCS is lower than 0.8, the impact is said to be low (graphically indicated by a lightest colored bar), if the ratio is higher than.2, the impact is designated as high (graphically indicated by a dark colored bar), while a ratio between 0.8 and.2 is called average (subsequently indicated by a light colored bar). 2.3 Analysis of scientific collaboration The analysis of the various types of scientific cooperation is based upon a typology of papers, which is based on the addresses attached to the publications. In case of the paper carrying only one address, the publications is automatically labeled as a single institute publication. In case of the appearance of at least two different country names on one publication, the publication is automatically considered an international cooperation. The remaining set of publications, carrying two or more addresses within one country, are considered to be the result of national cooperation. Any classification such as this one has some drawbacks. For example, the typology applied has the disadvantage that in the case of international cooperation publications, if a paper also carries two addresses from one country, the international dimension is the dominant factor in labeling the publication. Furthermore, in case of publications labeled as national cooperation, it can happen that these are actually two addresses of one and the same main institution, which makes it an intra-mural cooperation. However, the typology has been designed in order to have mutually exclusive classes thus simplifying the collaboration networks. 2.4 Journal impact profiles: journal to field impact of Dutch academic medical centers In this analysis we have changed the focus to the choice of the journals in which the researchers in the medical centers published their findings. In a similar way of presenting the research profiles, we now indicate the impact per field not by using the field normalized impact indicator MNCS, but apply the MNJS, to indicate the impact level of the journals (in their respective field(s)) in which the researchers have published their output in the period of analysis. 2.5 Top paper analysis In addition to the MNCS indicator, we use another important impact indicator. This is the proportion of publications belonging to the top 0%most highly cited, denoted by PP(top 0%). For each publication of a research group, we determine whether it belongs 90

to the top 0% based on its number of citations of all WoS publications in the same field (i.e., the same WoS subject category) and from the same publication year. The PP(top 0%) indicator of a research entity equals the proportion of its publications belonging to this top 0%. As for MCS and MNCS indicators, letters are given less weight than articles and reviews in the calculation of the PP(top 0%) indicator. If a research group has a PP(top 0%) indicator of 0%, it means that the actual number of top 0% publications of the group equals the expected number. A PP(top 0%) indicator of, for instance, 20% means that a group has twice as many top 0% publications as expected. Of course, the choice to focus on top 0% publications is somewhat arbitrary. Next to the PP(top 0%) indicator, we also calculate PP(top %), PP(top 2%), PP(top 5%), or PP(top 20%) and PP(top 50%) indicators. In the main tables, we use the PP(top 0%) indicator. The other PP(top x%) indicators are presented as a separate analysis in the study. 2.6 Coverage of a bibliometric analysis Finally, the indicator Int Cov shows the degree of the total set of references of a unit referring back to the WoS database, thereby indicating the degree of relevance of that WoS covered literature for the communication processes in the field(s) in which the unit is active. A high score means that many references point directly towards the WoS database, and thus that WoS literature is important for the communication process of the unit. This indicates the relevance of the application of bibliometric techniques in the research assessment of the unit. By inclusion of this indicator in the standard tables, we hope to put the scores in the respective data lines in context (as low scores clearly indicate the extra care one should take in interpreting the results presented in the table). 2.7 Basic elements of bibliometric analysis All above discussed indicators are important in a bibliometric analysis as they relate to different aspects of publication and citation characteristics. Generally, we consider MNCS as an important indicator for analyzing research performance. This indicator relates the measured impact of a research group or institute to a worldwide, fieldspecific reference value. Therefore, it is a powerful internationally standardized impact indicator. This indicator enables us to observe immediately whether the performance of a research institute/group or institute is significantly far below (indicator value < 0.5), below (indicator value 0.5-0.8), about (0.8 -.2), above (.2-2.0), or far above (>2.0) the international (western world dominated) impact standard of the field. The higher the aggregation level, the larger the volume in publications and the more difficult it is to 9

have an average impact significantly above the international level. At the meso-level (e.g., a large institute, or faculty, about 500,000 publications per year), a MNCS value above.2, could be considered that the institute s impact as a whole is significantly above (western-) world average. Therefore, it is important to split up large institutes into smaller groups (e.g. through the specific analysis of the different UMCs). Only this allows a more precise assessment of research performance. Otherwise, excellent work will be hidden within the bulk of a large institute or faculty. We stress that the other indicators provided in the tables, such as PP(uncited) or %Self Cit do contribute to a further understanding of the research performance analysis of units under study. Together these indicators tend to inform the user of bibliometrics on a variety of aspects in research performance analysis. 92

Appendix B: Changes in the bibliometric indicators of CWTS Centre for Science and Technology Studies (CWTS) Leiden University 3 February 20 93