Retractions, Post-Publication Peer Review, and Fraud: Scientific Publishing s Wild West Health Watch USA Lexington, KY November 3, 2017 Ivan Oransky, MD Co-Founder, Retraction Watch Distinguished Writer In Residence, NYU (Journalism) Editor at Large, MedPage Today Clinical Assistant Professor of Medicine, NYU @ivanoransky
Is This Publishing Today?
Is This Publishing Today? Robots No Longer Considered Harmful I.P. Freely, Oliver Clothesoff, Jacques Strap, Hugh Jazz, Amanda Huginkiss
Faking It
Stealing It
Fake It Till You Make It
Retractions By The Numbers Year # of Retractions # of Papers Published % 2000 30 1MM.003 2008 332 1.2MM.028 2010 3917 1.4MM.280* 2014 678 1.6MM.042 2016 1305 1.8MM.073 retractiondatabase.org
Where To Keep Track retractiondatabase.org
Common Reasons for Retractions Duplication ( self-plagiarism ) Plagiarism Image Manipulation Faked Data Fake Peer Reviews Publisher Error Authorship Issues Legal Reasons Not Reproducible
Most Retractions Due to Misconduct PNAS online October 1, 2012
Are We Catching Them All? Allison et al Nature 2016 http://www.nature.com/news/reproducibility-a-tragedy-of-errors-1.19264
Are We Catching Them All? Overall, 3.8% of published papers contained problematic figures, with at least half exhibiting features suggestive of deliberate manipulation. The prevalence of papers with problematic images has risen markedly during the past decade.
Are We Catching Them All? A pooled weighted average of 1.97% (N = 7, 95%CI: 0.86 4.45) of scientists admitted to have fabricated, falsified or modified data or results at least once a serious form of misconduct by any standard and up to 33.7% admitted other questionable research practices. In surveys asking about the behaviour of colleagues, admission rates were 14.12% (N = 12, 95% CI: 9.91 19.72) for falsification, and up to 72% for other questionable research practices. Fanelli, D. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0005738
Who Retracts Most?
Which Journals Retract? -Infection and Immunity 2011
What Happens to Retracted Papers Citations? -Assn of College & Research Libraries 2011
What Happens to Retracted Papers Budd et al, 1999: Citations? Retracted articles received more than 2,000 postretraction citations; less than 8% of citations acknowledged the retraction Preliminary study of the present data shows that continued citation remains a problem Of 391 citations analyzed, only 6% acknowledge the retraction
The Most Highly Cited
Do Journals Get the Word Out?
Do Journals Get the Word Out? Journals often fail to alert the naïve reader; 31.8% of retracted papers were not noted as retracted in any way.
Is This A Useful Retraction Notice? At the request of the authors, the following manuscript has been retracted: [citation] -Journal of Neuroscience
Why The Opacity?
Now This Is Good News The JBC s practice of saying very little in retraction and withdrawal notices has been described by many in the community as opaque and rightfully so. After reviewing the practices of other journals and consulting with our legal counsel and publications committee, we ve reconsidered our approach. JBC retraction and withdrawal notices now will explain, with as much detail as possible, why papers have been withdrawn or retracted. -Journal of Biological Chemistry
What Should Retraction Notices Look Like? www.publicationethics.org
Post-Publication Peer Review On The Rise
Post-Publication Peer Review On The Rise PubPeer was created specifically to bypass the suffocating and restricted channels of correspondence to the editors and journal commenting. The tens of thousands of useful comments that users have posted on PubPeer were previously suppressed by that system, yet facilitated by an open framework encouraging factual discussion. We re not going back. -- Brandon Stell, co-founder, PubPeer
http://nautil.us
False Claims Act
Crime Doesn t Pay Anymore
Doing The Right Thing Does
The Real Problem? Our ranking results seem to suggest that selective reporting, selective citing, and flaws in quality assurance and mentoring are the major evils of modern research. A picture emerges not of concern about wholesale fraud but of profound concerns that many scientists may be cutting corners and engage in sloppy science, possibly with a view to get more positive and more spectacular results that will be easier to publish in high-impact journals and will attract many citations. Bouter et al, Research Integrity and Peer Review, 2016
Contact Info/Acknowledgements oransiø1@nyu.edu http://retractionwatch.com @retractionwatch Thanks: The MacArthur Foundation The Arnold Foundation The Helmsley Trust