Defending the integrity of medical knowledge: an editor s perspective

Transcription

1 Defending the integrity of medical knowledge: an editor s perspective Fiona Godlee Editor, BMJ Publication Ethics Seminar, Osaka, Japan September

2 Robert Merton The normative structure of science 1942 Communalism Universalism Disinterestedness Organised Skepticism

3 A comment on the state of the medical literature from Drummond Rennie, deputy editor, JAMA There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.

4 McGauran et al. Trials 2010

5

6

7 GI outcomes, VIGOR, NEJM /1000

8 Placebo vs Rofecoxib or Celecoxib: comparison of MI rates JAMA 2001

9 Thrombotic events, Approve, NEJM 2004

10

11

12

13 BMJ 2005

14 Publication bias Selective publication of results, so that positive results are over represented and negative ones under represented

15 Publication bias in medical research Positive studies are more likely to be: Published Stern and Simes, BMJ 1997 Published faster Ioannides, JAMA 1998 Published in higher impact journals Easterbrook et al, Lancet 1991: Tierney and Stewart, 1997 Cited old style narrative review articles

16 Publication bias - Who s to blame? Editors/peer reviewers Mahoney 1977 Epstein 1990 Researchers Dickersin et al, Controlled Clinical Trials 1987 Olson et al, JAMA 2002

17 Publication bias linked to industry funding Rochon et al. Arch Intern Med 1994 Lexchin, Bero et al. BMJ 2003

18 Duplicate publication and salami slicing Trials of odansetron 84 trials that included information on patients In reality only 70 trials and 8645 patients (17% of the studies had been published more than once and the number of patients had been inflated by 28%) Impossible to tell from published studies Four pairs of identical trials were published by completely different authors without any common authorship (Misconduct) (Tramer et al, BMJ 1997: 315: )

19 Effectiveness of odansetron (Tramer et al, BMJ 1997: 315: )

20 Some things that bother editors Bias Data manipulation/suppression Duplicate publication Fabrication Falsification Gift/ghost authorship Plagiarism Self delusion Undeclared conflict of interest Wrong observations/analysis/references

21 Errors in good faith Trimming and cooking Fraud Bias Data manipulation/suppression Duplicate publication Fabrication Falsification Gift/ghost authorship Plagiarism Self delusion Undeclared conflict of interest Wrong observations/analysis/references Stephen Lock, BMJ

22 Errors in good faith Trimming and cooking Fraud Wrong observations Wrong analysis Wrong references Manipulating data Suppressing inconvenient facts Fabrication Falsification Plagiarism Bias Self delusion Gift authorship Duplicate publication Salami publication Undeclared conflicts of interest Stephen Lock, BMJ

23

24

25 Undeclared competing interest/ghosts and guests RCT of Vioxx versus Naproxen Lisse et al. Ann Intern Med 2003; 139: New York Times,24 April 2005 Merck designed the trial, paid for the trial, ran the trial Merck came to me after the study was completed and said, We want your help to work on the paper. The initial paper was written at Merck, and then sent to me for editing.

26 Guests and ghosts are not a thing of the past Surveyed authors of 900 articles published in high impact general medical journals in in 5 respondents admitted to at least one guest author (no change since 1996) Nearly 8% admitted to at least one ghost author (slightly lower than 1996) (Flanagin A, et al. Presented at Peer Review Congress, 2009)

27 Influence of financial links with industry on authors of reviews and letters Stelfox et al, NEJM 1998 Lexchin, Bero et al. BMJ 2003 Barnes DE, Bero LA. JAMA 1998 Yank, Rennie, Bero, BMJ 2007 Wang et al, BMJ 2010

28 Effect of conflict of interest statements on readers perceptions Two RCTs Chaudry et al, BMJ 2002 Schroter et al, BMJ 2004 A research article with a financial conflict of interest was rated less highly by readers than the same paper with no COI or non-financial COI

29 What is it about conflicts of interest that bother us? Having them at all? Failing to declare them? Is disclosure enough?

30 How do we act on disclosed information? Three options Disclosure is enough in itself. After that, anything goes, caveat emptor Disclosure is followed by a judgement about the degree of potential conflict Disclosure of a conflict of interest is a bar to taking part (membership of a panel, authorship of an editorial, involvement in a piece of research)

31

32

33 Science depends on Open critique and debate Correction/retraction Honesty and transparency Identification and minimisation of bias Repeatablity Refutation or confirmation Viewed as a public good Ethics

34 Why does integrity matter? Damage to the integrity of the biomedical literature brings: Pursuit of wrong avenues of scientific enquiry Participants and patients put at risk Waste of time and resources Loss of public trust in science Less willingness to fund research Less willingness among patients and the public to take part in research

35 Sources of pressure on integrity Science is a human enterprise Human beings are fallible Careers and huge sums of money are at Libel laws increasingly being invoked, protecting those with money Science is based on a system of trust Existing checks and balances are only partially effective Training and supervision are inadequate The biomedical literature is fragmented, making it hard to get to the truth With the result that a lot of published research is of poor quality, and some of it (how much?) is fraudulent

36 The community presents a natural work setting for an editor, not the laboratories or surgeries. The editor is a community worker and a teacher. Andrija Stampar, one of the founders of WHO

37 Problems with peer review Slow Expensive Biased Unaccountable Stifles innovation Bad at detecting error Hopeless at detecting fraud And journals have conflicts of interest too

38 How do journals make their money? Ads Subs Reprints BMJ?NEJM

39 Radical transparency Nearly all decision making is carried publicly Not only the ordinary information of interest to the community is made freely available, but all (or nearly all) meta-levels of organizing and decision making are also published Includes draft documents, all arguments for and against a proposal, the decisions about the decision making process itself, and all final decisions Excludes data relating to personal security, passwords etc (from Wikipedia)

40 Moving towards greater transparency Trial registration Protocols Data sharing Mandate data availability Unlimited space online Uniform COI form Open access Open peer review/pre-publication history Article level metrics

41 But will any of this really help? A more radical future Stop using opinion leaders with any financial conflicts of interest? Stop pharmaceutical companies directly evaluating their own products? An end to pre-publication peer review? Publication of entire data sets

42 Why we need raw data Godlee F, Clarke M Why don t we have all the evidence on oseltamivir? BMJ 2009

43 Conclusions The integrity of the biomedical literature matters Loss of integrity distorts the scientific record and threatens public trust in science, funding and participation in research, and patient safety The threats to integrity are many and various Some threats are caused by honest error, but a significant proportion are intentional, for commercial or career gain (misconduct/fraud) Additional threats come from unnecessary and poorly done research

44 Conclusions Current efforts to protect the integrity of medical knowledge are inadequate Some hope lies in better education, greater transparency, and ever more rigorous peer review Tackling the influence of the drug industry remains a major challenge for clinicians, researchers, journals, journalists, and the public More radical alternative forms of research funding and data publication are needed

45 Thank you