Research Misconduct: A look at process and current trends

Research Misconduct: A look at process and current trends K E R I G O D I N A S S O C I A T E D I R E C T O R O F R E S E A R C H C O M P L I A N C E H M S O F F I C E F O R A C A D E M I C A N D R E S E A R C H I N T E G R I T Y

Research Misconduct Defined Fabrication (making up results and recording or reporting them) Falsification (manipulating research materials, data, or processes; or manipulating data such that the research is not accurately represented in the research record) Plagiarism (the appropriation of another person s ideas, results, or words without giving that person appropriate credit) in proposing, performing, or reviewing research, or in reporting research results.

What it is & importantly, what it is not Intentional Knowing Reckless - Not an honest error - Not a difference of scientific opinion - Not a non-compliance related to human subjects or animal subjects - Not misuse of research funds Office of Research Integrity, 42 CFR, Sec. 93.104

William Fals-Stewart, Ph.D. Former University of Buffalo (UB) addictions researcher accused in 2004 of fabricating data NIDA funded September 2007, three witnesses testified via phone on Dr. Fals-Stewart s behalf: Project Director, Project Coordinator & Director of Quality Assurance Faculty panel determined insufficient evidence based on witness testimony Fals-Stewart sued UB for $4 million in US District Court wrongful termination, tarnished reputation Submitted an affidavit including an email from a University of Miami professor re: misconduct allegations coming up during a conference

and the truth emerges Witnesses were actors told they were participating in a mock trial U of Miami professor confirmed he did not send email Dr. Fals-Stewart arrested in 2010 Charged with attempted grand larceny, three counts of perjury, three counts of identity theft, two counts of offering a false instrument and three counts of falsifying a business record Maximum permissible sentence = 15 years in prison

Recent Enforcement Action: Dong-Pyou Han ISU Scientist, admitted to falsifying AIDS vaccine study data supported by multi-million dollar NIH grant Entered plea agreement (Feb. 2015), saying his subterfuge cost the federal government $7 -$20 million. In return for his guilty pleas to two felony charges of making false statements, prosecutors dropped two other charges; could face up to 10 years in prison ISU had to repay $496K; lost $1.4 in grant monies not yet paid

Process and Key Players Complainant - Journals - Anonymous - Proximal to accused (i.e., in lab, collaborator) - Received from federal agency - Retraction Watch Respondent - Anyone performing, proposing, reporting - Can be multiple respondents in a given matter - All levels of appointment can and have been respondents Relevant Agency (NSF, ORI, DoD) - ORI: Jurisdiction over PHS funded work - Informed when review reaches Investigation - Waits for institutional finding before own review

Institutional responsibilities Policy for handling Allegations of Research Misconduct Assessment of all good faith allegations Research Integrity Officer Sequestration of data by Institution & Notification of Accused ALL potentially relevant data Appointment of impartial faculty panel Fair, objective, CONFIDENTIAL process Interviewing witnesses /Forensic analysis of Data Reporting of advancement to investigation, findings of research misconduct (PHS funded)

Costs of Misconduct Direct costs associated with handling research misconduct cases at the institutional level in the US exceed $110 million annually 1 Estimated $525,000 per case 1 Institutional reputation Repayment of sponsored funds Loss of productive faculty Impact on public health & trust in research 1 Michalek et al. (2010)

Not just an Academic Issue Public health is perhaps most at stake when medical research is found to be falsified or fabricated A. Wakefield matter A 2011 article in The Journal of Medical Ethics reviewed ~200 papers that were retracted due to questionable data the published research was tied to 28,000 patients, 6,573 of whom received treatment based on the research presented in the retracted papers (Steen, 2011).

ORI Data: 2012 Annual Report* ORI received 423 allegations in 2012 56% increase over the 240 allegations handled previous year Historical perspective:1992-2007 average of 198 allegations 32/33 ORI Investigations involved allegations of falsification and/or fabrication *Source: http://ori.hhs.gov/images/ddblock/ori_annual_report_2012.pdf

Current trends Fanelli s 2009 PloS One article: Nearly 2% of scientists admitted to falsifying or fabricating data a minimum of one time Nearly 34% reported engaging in other forms of devious research practices Approximately 14% of scientists surveyed witnessed colleagues manipulation or making up of data. Tavare s 2012 British Medical Journal article: 1 in 7 researchers in U.K. witnessed other investigators falsifying/fabricating data to increase publishing potential

Incidence of Misconduct: A look at retractions *Source: Fang et al. (2012), Figure 1A, p. 17029.

Environmental Pressures Growing number of investigators vying for limited funds Recent example: Imperial College London, Prof. Grimm* Decreasing federal research budget Promotions reliant on robust C.V. Incentives to publish Prestige Garner additional funding *Source: https://www.insidehighered.com/news/2014/12/01/imperial-college-londoninvestigates-role-pressure-death-academic

Evolving Academic Research Environment Junior faculty at increased risk

Academic Appointment Level Early and mid-career researchers account for 2/3 of research misconduct findings (Martinson et al., 2005) Pressure to obtain external funding and federal research dollars to support one s salary positively correlated with reports of serious misconduct (Martinson et al., 2009) Professional stressors - being overworked & pressure to complete experiments/produce data with insufficient time allotted - correlated with research misconduct (Davis et al., 2007)

Evolving Academic Research Environment Junior faculty at increased risk Increase in Foreign-born researchers

Foreign-born Investigators Differing cultural norms (Davis, 2003) Lack of English proficiency (Xiguang & Lei, 1996) Fear of asking for help (Davis, Riske-Morris and Diaz, 2007) Lack of Responsible Conduct of Research training/oversight in country of origin (Okonta and Rossouw, 2012)

Evolving Academic Research Environment Junior faculty at increased risk Increase in Foreign-born researchers Mentoring increasingly challenging

Mentoring 62% of mentors had not established procedural standards (Wright et al., 2008) 73% had not reviewed the raw data generated by their trainees (Wright et al., 2008) Improvement in the quality of mentoring in training programs is the path to reducing misconduct at any given institution (Kornfeld, 2012)

Evolving Academic Research Environment Junior faculty at increased risk Increase in Foreign-born researchers Mentoring increasingly challenging Data storage and presentation processes changing

Data Management Schreir and colleagues (2006) survey of 96 university officials charged with institutional oversight of research misconduct cases revealed that inquires and investigations were hampered by inadequate research records. Martinson et al. (2005) similarly reported that 27% of the faculty investigators they interviewed openly confessed to inadequate record-keeping related to research projects. Schreier et al. (2006) lament the shortcoming of the present academic research environment, in which formal record-keeping standards are absent or woefully inadequate, peers rather than superiors teach newcomers record-keeping habits, and foreign investigators often record data in their native languages given insufficient standards dictating the importance of data retention in English.

Potential Consequences Retraction or correction of all pending and published papers and abstracts affected by the misconduct Reprimand, removal from project, rank and salary reduction, dismissal Restitution of funds to the granting agency Ineligibility to apply for Federal grants for years Public awareness & reputational harm federal register/ori website Likely, the end of one s research career

Tools to spot-check data Forensic tools (http://ori.hhs.gov/forensic-tools ) Plagiarism tools etblast: http://etest.vbi.vt.edu/etblast3/ Déjà vu: http://dejavu.vbi.vt.edu/dejavu/ Drop text in Google or Images in Google Image Image analysis Photoshop Advanced Forensic Actions: http://ori.hhs.gov/advanced-forensicactions Earlier versions available for Photoshop CS2 and 3 Forensic Droplets: http://ori.hhs.gov/droplets Other freeware available (More intensive image analysis) Fiji (ImageJ): http://fiji.sc/fiji

Which three lanes are the same? Source: http://ori.hhs.gov/samples

What evidence shows the 67 kda band is the same data as the 32 kda band? Source: http://ori.hhs.gov/samples

What more can administrator s do? Encourage adherence to institutional Policy and sponsorrequired data/record retention Assess lab sizes; mentor-mentee ratios Create a culture and environment where data sharing is expected, and data challenging is encouraged Follow your instincts; repeat offenders of noncompliance may be a red flag of substandard practices If you witness something concerning, don t stay silent 1 http://files.vpr.harvard.edu/files/vprdocuments/files/research_records_and_data_retention_and_maintenance_faq_guidance_7_31_12.pdf

Fake Peer Review Authors review own paper or 3 rd party vendor reviews at a cost Widespread Elsevier, Springer, Taylor & Francis, SAGE and Wiley The fall-out As of last fall, about 110 retractions across several journals. Then Elsevier added 16 more retractions Then, in March 2015, Bio Med Central added 43 170 Retractions thus far Sources: http://www.nature.com/news/publishing-the-peer-review-scam-1.16400 http://retractionwatch.com/2015/03/26/biomed-central-retracting-43-papers-for-fake-peer-review/

Journals and NIH take action: Reproducibility in Science June 2014 NIH/the Nature Publishing Group/Science held a workshop on reproducibility & rigor of research findings Over 30 basic/preclinical science journals represented in which NIH-funded investigators have most often published. The workshop focused on the common opportunities in the scientific publishing arena to enhance rigor and further support research that is reproducible, robust, and transparent. http://www.nih.gov/about/reporting-preclinical-research.htm

References Davis, M.S. (2003). The role of culture in research misconduct. Accountability in Research, 11(3), 189 201. Davis, M.S., Riske-Morris, M., Diaz, S.R. (2007). Causal Factors Implicated in Research Misconduct: Evidence from ORI Case Files. Science and Engineering Ethics, 13, 395 414. Fanelli, D. (2009). How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data. PLoS ONE 4(5): e5738. doi:10.1371/journal.pone.0005738 Fang, F.C., Steen, R.G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028 17033. Kornfeld, D.S. (2012). Perspective: Research Misconduct: the search for a remedy. Academic Medicine, 87(7), 877-882 Martinson, B.C., Anderson, M., & de Vries, R. (2005). Scientists behaving badly. Nature, 435, 737-738. Martinson, B.C., Crain, A.L., Anderson, M.S., & De Vries, R. (2009). Institutions Expectations for Researchers Self- Funding, Federal Grant Holding and Private Industry Involvement: Manifold Drivers of Self-Interest and Researcher Behavior. Academic Medicine, 84(11): 1491 1499. doi:10.1097/acm.0b013e3181bb2ca6 Michalek A.M., Hutson A.D., Wicher, C.P., & Trump, D.L. (2010). The Costs and Underappreciated Consequences of Research Misconduct: A Case Study. PLoS Med, 7(8). doi:10.1371/journal. Okonta, P. & Rossouw, T. (2012). Prevalence of Scientific Misconduct among a group of Researchers in Nigeria. Developing World Bioethics. doi: 10.1111/j.1471-8847.2012.00339.x Schreier, A.A., Wilson, K., Resnik, D. (2006). Academic Research Record-Keeping: Best Practices for Individuals, Group Leaders, and Institutions. Academic Medicine, 81(1), 42-47. Steen, G.R. (2011). Retractions in the medical literature: how many patients are put at risk by flawed research? Journal of Medical Ethics. Retrieved from http://jme.bmj.com/content/early/2011/05/17/jme.2011.043133.abstract?sid=48bd98ec-a822-4286-9c73-2a6787d2e1bd Tavare, A. (2012). Scientific misconduct is worryingly prevalent in the UK, shows BMJ survey. British Medical Journal. Retrieved from http://www.bmj.com/content/344/bmj.e377 Wright, D.E., Titus, S.L., & Cornelison, J.B. (2008). Mentoring and Research Misconduct: An Analysis of Research Mentoring in Closed ORI Cases. Science and Engineering Ethics, 14, 323 336. Xiguang, L. & Lei, X. (1996). Scientific Misconduct: Chinese Researchers debate rash of Plagiarism Cases. Science, 274(5286), 337. DOI: 10.1126/science.274.5286.337