Ian F Tannock MD, PhD, DSc

Transcription

1 Reading the literature on clinical trials with a critical eye: recognizing and learning from the mistakes of others Ian F Tannock MD, PhD, DSc Professor of Medical Oncology and Medical Biophysics Princess Margaret Cancer Centre and University of Toronto th Turkish Oncology Congress,

2 Potential conflicts of interest I have chaired clinical trials evaluating systemic therapy for prostate cancer and participated in multiple trials of systemic therapy for cancer. I have advised companies about the design of trials cancer for which I have received contributions to my research fund. I do not accept personal remuneration from companies th Turkish Oncology Congress,

3 How to Evaluate the Quality (or Internal Validity) of a Trial 1. Does the trial ask an important question? 2. Is the design of the study appropriate? 3. Are the endpoints appropriate and do they reflect benefit to patients? 4. Does the report of the study truly reflect its results? th Turkish Oncology Congress,

4 R A N D O M I Z E Reduction by granulocyte colony-stimulating factor of fever and neutropenia induced by chemotherapy in patients with small-cell lung cancer (NEJM 1991; 325:164-70) Aggressive chemo Aggressive chemo + G-CSF Aggressive treatment is no better than standard chemotherapy associated with lower rates of sepsis ~15% 5th Turkish Oncology Congress, Febrile neutropenia 57% 28%

5 How to Evaluate the Quality (or Internal Validity) of a Trial 1. Does the trial ask an important question? 2. Is the design of the study appropriate? 3. Are the endpoints appropriate and do they reflect benefit to patients? 4. Does the report of the study truly reflect its results? th Turkish Oncology Congress,

6 Levels of Scientific Evidence 1. Large randomized controlled trials (RCTs) or meta-analyses 2. Small RCTs 3. Contemporary (but not randomized) controls 4. Historical controls 5. Case series th Turkish Oncology Congress,

7 Non-randomized clinical trials are important: 1. To demonstrate feasibility of a new treatment (Tolerance- Phase 1) 2. To demonstrate sufficient efficacy and/or insight into mechanisms to warrant further testing (Phase 2) But... They can only generate a hypothesis about the value of a new treatment that is then tested in a randomized trial against standard management (Phase 3) th Turkish Oncology Congress,

8 Factors which limit the value of comparisons between single-arm studies Selection of patients Stage migration Publication (and presentation) bias Therapeutic results with controls tend to have no enthusiasm. Therapeutic results with enthusiasm tend to haveno controls. D. Sackett, th Turkish Oncology Congress,

9 Publication Bias applies even to large randomized trials Krzyzanowska et al JAMA 2003;290:495-50) For 510 RCTs reported in ASCO abstracts, there was delayed publication of negative trials Proportion published ASCO presentation 81% +ve studies 68% -ve studies Log rank p-value = Non-significant, n= 285 Significant, n= Years to publication th Turkish Oncology Congress,

10 Stage Migration: The Will Rogers Phenomenon (Feinstein et al: NEJM 1985;312:1604-8) When the Okies left Oklahoma and moved to California they raised the average intelligence level in both states. Will Rogers (Referring to migration during the 1930 s depression) Stage migration refers to the upstaging of patients with cancer as more sensitive imaging techniques discover smaller deposits of disease - so that there is an improvement in outcome for each stage th Turkish Oncology Congress,

11 Stage migration and high dose chemotherapy for breast cancer (Crump et al JCO 1997) High-dose chemotherapy (HDC) and stem-cell transplantation became widely used for patients with breast cancer. Results were so much better that some considered randomization unethical Patients receiving HDC were more extensively investigated than controls patients with small metastases were detected and excluded Subsequent randomized controlled trials showed no benefit from HDC th Turkish Oncology Congress,

12 Too many resources are wasted on non-randomized trials (Berthold et al: JCO 2009;27:1150) In a review of 100 phase 2 trials showing promising results for a new regimen presented at the major cancer meeting (ASCO) in only 13 were subsequently tested in a phase 3 trial Too often their unstated purpose is: To legitimize a non-standard treatment To improve the CV of the investigator th Turkish Oncology Congress,

13 Always consider the following options 1. To support on ongoing multi-institution phase 3 trial, or 2. To design a new phase 2 study Option #1 has the most likelihood of benefit to patients th Turkish Oncology Congress,

14 It should: If you are reading a report of a phase III randomised trial: compare an experimental treatment with an appropriate standard of care have an appropriate and well-defined primary endpoint (outcome measure) be large enough to detect a meaningful difference in its primary outcome.. but if it is testing palliative treatments for incurable cancer it should not be so large that it detects trivial differences th Turkish Oncology Congress,

15 Of 423 negative RCTS, 67%, and 45% were too small to detect medium (HR<0.67), and large (HR<0.5) effect sizes Failing to find an effect in a small trial does not mean that it does not exist! Trials are described as positive if p<0.05, even if they fail to show a difference in outcome that is less than planned. Finding a statistically significant effect in a large trial does not mean that it is clinically relevant! th Turkish Oncology Congress,

16 How to Evaluate the Quality (or Internal Validity) of a Trial 1. Does the trial ask an important question? 2. Is the design of the study appropriate? 3. Are the endpoints appropriate and do they reflect benefit to patients? 4. Does the report of the study truly reflect its results? th Turkish Oncology Congress,

17 No matter which discipline there are two possible ways that a new treatment might benefit patients Either patients live longer, or they live better th Turkish Oncology Congress,

18 Endpoints (or Outcome Measures) of Clinical Trials Phase 3 Trials should have a clearly-stated primary endpoint that reflects benefit to patients Overall Survival Quality of Life or Symptom Control Endpoints such as tumor response show biological activity but are poorly correlated with survival If a surrogate endpoint (such as DFS or PFS) is used it should be correlated with one of the above primary endpoints th Turkish Oncology Congress,

19 An example of problems from using PFS Chemotherapy +/- bevacizumab for women with metastatic breast cancer th Turkish Oncology Congress,

20 Trials of bevacizumab + CT vs CT alone for Ca breast, prostate, lung, pancreas, stomach, ovary Are reported as negative if primary endpoint is OS Are reported as positive if primary endpoint is PFS In 2 trials the company changed the endpoint from OS to PFS while the trial was in progress In all trials bevacizumab added toxicity th Turkish Oncology Congress,

21 PFS is also subject to biased reporting, especially if there is uneven drop-out of patients in the experimental and control arms Another example: Should we adopt exemestane + everolimus as standard treatment for post-menopausal women with ER+ breast cancer who have progressed on a non-steroidal aromatase inhibitor, based on the BOLERO-2 trial? th Turkish Oncology Congress,

22 PFS Serious Adverse Events (SAEs) due to treatment: 11% vs. 1% Early stopping due to SAE or withdrawn consent: 24% vs 6% No significant difference in OS as yet th Turkish Oncology Congress,

23 Censoring bias: Patients who withdraw from a study (for toxicity or other reasons) are censored if they have not satisfied criteria of progression. A better representation might be difference in time on treatment Bolero-2: time on study (courtesy of Greg Pond) HR= th Turkish Oncology Congress,

24 And use of DFS in adjuvant trials? Note that early and large differences in DFS predict for much smaller (if any) differences in overall survival th Turkish Oncology Congress,

25 Several trials have compared aromatase inhibitors to tamoxifen as adjuvant therapy for postmenopausal women with ER+ breast cancer All of these trials have used DFS as the primary endpoint They have shown a small increase in DFS compared to tamoxifen Does this prove the benefit of AIs as compared to Tamoxifen? th Turkish Oncology Congress,

26 and for switching trials? th Turkish Oncology Congress,

27 The aim of adjuvant hormonal therapy for postmenopausal women with ER+ breast cancer should be to maximize survival and QoL at the most reasonable cost Survival from upfront use of tamoxifen and AIs is identical So the decision as to which to choose should depend on toxicity and cost AIs are more toxic AIs are more expensive th Turkish Oncology Congress,

28 Our conclusion: Tamoxifen should remain standard initial adjuvant therapy for most women with ER+ breast cancer th Turkish Oncology Congress,

29 Good drugs do not need the help of inappropriate endpoints like PFS or DFS Imatinib for CML of GIST Trastuzumab for breast cancer Vemurafenib for melanoma Sunitinib for renal cancer Abiraterone and enzalutamide for prostate cancer All of these drugs improved survival, even if initial trials had PFS or DFS as a primary endpoint th Turkish Oncology Congress,

30 Toxicity Endpoints 1. Are poorly reported 2. Serious but rare toxicity may not be found in a clinical trial - chronic toxicity, associated with targeted agents may occur late. 3. Clinical trials underestimate the toxicity to be expected when a new drug moves to general oncologic practice th Turkish Oncology Congress,

31 Are RCTs sufficient to determine benefit and toxicity of targeted agents? 58% of potentially fatal adverse events are not in the initial FDA drug label, and 39% are not reported in any published randomized trial th Turkish Oncology Congress,

32 Howell A: The 'Arimidex', Tamoxifen, Alone or in Combination (ATAC) Trial: a step forward in the treatment of early breast cancer. Rev Recent Clin Trials 2006;1: Importantly, a decrease in the odds ratio of cardiovascular events was observed with anastrozole compared with tamoxifen FDA (Dec 2008): WARNINGS AND PRECAUTIONS: Ischemic Cardiovascular Events: In women with pre-existing ischemic heart disease, an increased incidence of ischemic cardiovascular events was observed with Arimidex in the ATAC trial (17% of patients on Arimidex and 10% of patients on tamoxifen). Consider risk and benefits of Arimidex therapy in patients with pre-existing ischemic heart disease th Turkish Oncology Congress,

33 th Turkish Oncology Congress,

34 What should we call an agent that increases PFS or DFS, has no effect on survival, and adds toxicity? HARMFUL th Turkish Oncology Congress,

35 How to Evaluate the Quality (or Internal Validity) of a Trial 1. Does the trial ask an important question? 2. Is the design of the study appropriate? 3. Are the endpoints appropriate and do they reflect benefit to patients? 4. Does the report of the study truly reflect its results? th Turkish Oncology Congress,

36 Clinical Trials 101 : In reports of phase 3 randomised controlled trials 1. The primary endpoint should be defined explicitly in the abstract of the paper or presentation 2. That primary endpoint should reflect benefit to patients 3. The concluding sentence of the abstract should relate only to the primary endpoint 4. The abstract should include a statement of major toxicity Unfortunately many reports, both in high level journals and at the ASCO meeting, do NOT meet these simple requirements th Turkish Oncology Congress,

37 We identified 164 trials published of 164 trials showed no significant difference in their primary endpoint In 59% of reports of these 92 trials the concluding statement of the abstract used secondary endpoints to suggest benefit Only one third of reports indicated the frequency of grade 3-4 toxicity in the abstract A positive result for the primary endpoint was associated with under-reporting of toxicity th Turkish Oncology Congress,

38 th Turkish Oncology Congress,

39 External Validity and Relevance of Important to ask: Clinical Trials How do the results of the trial fit with other results and clinical experience? Are results of the trial relevant to patients with similar types and stages of cancer? Many clinical trials are highly selective in recruiting patients those showing small differences may lead to harm when applied in the community th Turkish Oncology Congress,

40 Despite this trivial difference, the p-value was <0.05, and...gemcitabine + Erlotinib was approved by the FDA for treatment of pancreatic cancer 4/2/2014 MOGA, Brisbane

41 Men receiving 3- weekly docetaxel for CRPC Routine practice On-trial patients P-value TAX-327 Number Median survival (95%CI) 13.6 mos ( ) 20.4 mos ( ) mos ( ) PSA RR 45% 54% NS 53% Median number of courses % septic neutropenia 6 8 < % 0% < %

42 Health outcome studies are needed to determine whether results of RCTs apply in every-day practice Very large RCTs (>1000 patients) are being used to detect trivial differences in survival that are statistically significant New agents always adds toxicity Differences in survival may disappear and toxicity will be greater when new treatments are applied to a less selective population with more comorbidity in routine practice The net effect may be harm rather than benefit th Turkish Oncology Congress,

43 th Turkish Oncology Congress, The important test of validity of a clinical trial is not the p-value, it is whether the result is reproducible

44 Recommendations for what to include when reporting a trial in an abstract Rationale for the study Brief description of the intervention Sample size and brief description of participants Planned sample size or power of study Explicit definition of primary endpoint Magnitude of difference for primary endpoint with confidence limits or 2-sided p-value Duration of follow-up Description of major toxicity Source of funding th Turkish Oncology Congress,

45 How to Evaluate a Report of a Phase Ask yourself: III Clinical Trial Is the trial asking a clinically relevant question? Is it comparing an experimental treatment with an acceptable control? Are the endpoints (outcome measures) evaluating true benefit to the patients? Is the trial large enough to detect or rule out an important difference in outcome? Does the trial report its primary outcome? If there is a difference in this outcome is it clinically meaningful? Does the trial describe adequately the toxicity associated with treatment th Turkish Oncology Congress,

46 Eminence Based Medicine Vehemence Based Medicine Eloquence Based Medicine Providence Based Medicine Diffidence Based Medicine Nervousness Based Medicine Confidence Based Medicine White hair helps He who shouts loudest He who speaks well For the religious right Don t know what to do They might sue Only for surgeons 4/2/2014 Clinical Epidemiology Seminar, University of Toronto

47 Analyzing the same data in two ways (Baar & Tannock JCO 1989;7:969-78) We constructed a hypothetical clinical trial in which patients were treated by chemotherapy The data were analyzed: 1. By methods that we consider to be of low quality, but all used in a major cancer journal 2. By methods we consider to be of high quality th Turkish Oncology Congress,

48 EFFECTIVE CHEMOTHERAPY FOR METASTATIC CARCINOMA OF THE GREAT TOE by Colin Apples and Bridget Oranges Solid Tumor Section, Dept of Oncology, University of Transylvania Medical Center, Barstow, TR th Turkish Oncology Congress,

49 th Turkish Oncology Congress,

50 The Drug Regimen (CABOOM) Cyclophosphamide 1gm/m 2 iv day 1 Adriamycin (doxorubicin) 40 mg/m 2 iv day 1 Bleomycin 10 IU days iv days15,22 Oncovin (vincristine) 1gm/m 2 iv day 1 Oral-Methotrexate 15mg/m 2 p.o. days th Turkish Oncology Congress,

51 TNM classification for Cancer of the Great Toe T = Tumor confined to the Toe N = Extension to the Nail M = Extension to the Muscle th Turkish Oncology Congress,

52 Conclusions CABOOM is effective chemotherapy for metastatic carcinoma of the great toe Survival is improved in responders Toxicity is acceptable We suggest a role for CABOOM as adjuvant therapy following amputation of the great toe th Turkish Oncology Congress,

53 LACK OF BENEFIT OF COMBINATION CHEMOTHERAPY FOR METASTATIC CARCINOMA OF THE GREAT TOE by Richard Prose and Bruce Conns Division of Clinical Trials, Dept of Oncology, University of Transylvania Medical Center, Barstow, TR th Turkish Oncology Congress,

54 Differences in the 2 reports Entry criteria and selection of patients A (poor quality) Not stated B (good quality) Stated precisely Protocol violations Not stated Stated precisely Criteria of Response Lax, poorly-defined Stringent, well-defined Analysis of Toxicity Superficial Comprehensive Quality of life assessment No Yes th Turkish Oncology Congress,

55 th Turkish Oncology Congress,

56 th Turkish Oncology Congress,