Accuracy, Completeness, and Transparency: lessons from Tamiflu experience

Transcription

1 Accuracy, Completeness, and Transparency: lessons from Tamiflu experience Peter Doshi, PhD Postdoctoral fellow in Comparative Effectiveness Research Johns Hopkins University School of Medicine October 12, 2012 ACT Now EQUATOR Conference, Freiburg, Germany T. Jefferson :: R. Hama :: C. Heneghan :: C. Del Mar :: P. Doshi :: M Jones :: M Thompson

2 CHAPTER 1: Conclusions of great benefit

3 The Tamiflu Success Story

4 CDC/ACIP Influenza Rec. (2006) Among influenza virus infected participants in 10 clinical trials, the risk for pneumonia among those participants receiving oseltamivir was approximately 50% lower than among those persons receiving a placebo (339). A similar significant reduction was also found for hospital admissions; a 50% reduction was observed in the small subset of highrisk participants, although this reduction was not statistically significant. Kaiser 2003 Smith NM, Bresee JS, Shay DK, Uyeki TM, Cox NJ, Strikas RA. Prevention and Control of Influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep Jul 28;55(RR-10):

5 HHS Pandemic Influenza Plan (2005) Critical assumptions. Treatment with a neuraminidase inhibitor (oseltamivir [Tamiflu ] or zanamivir [Relenza ]) will be effective in decreasing risk of pneumonia, will decrease hospitalization by about half (as shown for interpandemic influenza), and will also decrease mortality. (p.d-20) U.S. Department of Health and Human Services. HHS Pandemic Influenza Plan [Internet] [cited 2009 Jun 9]. 5

6 HHS 2004 draft pandemic plan The impacts of oseltamivir therapy on lower respiratory tract complications (LRTCs) of influenza and on influenza hospitalizations were calculated in a pooled analysis of 10 randomized placebo-controlled studies that included 3,591 adults and adolescents. Overall, 4.6 percent of oseltamivir treated persons had an LRTC of influenza infection compared with 10.3 percent of persons who received placebo a 55 percent reduction (P<0.001). Kaiser 2003 HHS. DRAFT Pandemic Influenza Preparedness and Response Plan [cited 2010 Jul 5]. 6 emiccore.pdf

7 CHAPTER 1: Conclusions of great benefit CHAPTER 2: Conclusions challenged

8 The Tamiflu Story April July

9 Hayashi s criticism Kaiser 2003 is a Roche authored paper Pooled analysis of 10 Roche funded RCTs from the late 1990s 2/10 published (1397 pts) 8/10 never published (2691 pts) Kaiser et al. Archives of Internal Medicine. 2003; 163: Conclusion: Oseltamivir treatment of influenza illness reduces LRTCs, antibiotic use, and hospitalization in both healthy and at-risk adults. 9

10 CHAPTER 1: Conclusions of great benefit CHAPTER 2: Conclusions challenged CHAPTER 3: Evidence is missing

11 Profs Kaiser and Hayden I suggest to contact Roche directly to get access to the files. from Kaiser August 17, 2009 Slide courtesy Tom Jefferson I have searched but cannot find the original files related to this 2003 publication. Before and again after my 2+ years at WHO in Geneva, I was obliged to move offices at the University several times and downsize. The files appear to have been discarded. My co-author Laurent Kaiser, now professor at the University of Geneva, is copied on this reply, as he may have his own sources. The questions posed by the inquirer are not clear to me, but if original data or unpublished study reports are required, they will likely need to come from Roche, the sponsor of these studies. from Hayden August 14, 2009

12 Professor John Treanor Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, et al. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza: a randomized controlled trial. US Oral Neuraminidase Study Group. JAMA Feb 23;283(8):

13 Professor Karl Nicholson Nicholson KG, Aoki FY, Osterhaus AD, Trottier S, Carewicz O, Mercier CH, et al. Efficacy and safety of oseltamivir in treatment of acute influenza: a randomised controlled trial. Neuraminidase Inhibitor Flu Treatment Investigator Group. Lancet May 27;355(9218):

14 I did not perform an independent analysis of the primary data, which was not required or requested by JAMA at the time of submission, and I do not have access to the primary data, which I also never requested. When asked a similar question, Nicholson said he did not recall seeing the primary data. He said that the statistical analysis had been conducted by Roche and he analysed the summary data. Quotes from Cohen D. Complications: tracking down the data on oseltamivir. BMJ Dec 8;339(dec08_3):b5387.

15 Largest treatment trial abstract only Source: Cohen D. Complications: tracking down the data on oseltamivir. BMJ. 2009;339:b5387.

16 Issues to consider What is the role of trust in the scientific publication and evidence synthesis process? Do named authors have a responsibility to provide data that allows for independent verification of claims? What is one to conclude in the absence of obtaining data for trials that are known to exist?

17 Dec. 2009

18 Cochrane review, Dec 2009 Roche: very happy to have its data reviewed by appropriate authorities or individuals full study reports will also be made available on a password-protected site within the coming days to physicians and scientists undertaking legitimate analyses. Full study report?

19

20 Doshi P, Jones M, Jefferson T. Rethinking credible evidence synthesis. BMJ. 2012;344:d7898.

21

22 New Methods Discussed new methods Identify all trials (trial programme) No more published papers! Identify and retrieve all Clinical Study Reports and regulatory material Table of Contents of the Evidence (TOCE) Weave evidence of trial programmes together Assess if reliable, analyse Group decision Agree Agree Agree Agree Disagree What does reliable mean? Complete? Trustworthy? (Both?) (Original slide courtesy Tom Jefferson)

23 CHAPTER 1: Conclusions of great benefit CHAPTER 2: Conclusions challenged CHAPTER 3: Evidence is missing CHAPTER 4: Some (20,000 pages) evidence retrieved

24 EMA s sea-change in attitude November 30, 2010

25 What we obtained from EMA Trial(s) No of patients Pages obtained JV JV M ML ML MV NP NV NV WP WV WV Trial(s) No of patients Pages obtained WV15673/ WV WV WV WV WV15759/ WV WV15812/ WV WV15876/15819/ WV WV Total

26 Paper needed to print Clinical Study Report for oseltamivir trial WP pages

27 Unexpected findings

28 CERTIFICATE OF ANALYSIS

29 CHAPTER 1: Conclusions of great benefit CHAPTER 2: Conclusions challenged CHAPTER 3: Evidence is missing CHAPTER 4: Some (20,000 pages) evidence retrieved CHAPTER 5: Conclusions of little benefit

30 Latest Cochrane Review (2012) January 2012 Conclusions: ~1 day reduction in time to first alleviation of influenza symptoms No decrease in risk of hospitalization No evidence it can stop the spread of virus Suggestive evidence Tamiflu interferes with natural influenza antibody production

31 Roche: Tamiflu reduces incidence of secondary complications (ie bacterial infections) by 45% FDA: Claim not supported by substantial evidence. Roche promotional material FDA Warning Letter 31

32 Tamiflu label Serious bacterial infections may begin with influenza-like symptoms or may coexist with or occur as complications during the course of influenza. TAMIFLU has not been shown to prevent such complications. Tamiflu Product Labeling (Nov 17, 2000 to present) Before 2009, CHF 7.6 billion worth stockpiled by governments worldwide. Why? 32

33 CHAPTER 1: Conclusions of great benefit CHAPTER 2: Conclusions challenged CHAPTER 3: Evidence is missing CHAPTER 4: Some (20,000 pages) evidence retrieved CHAPTER 5: Conclusions of little benefit