11 questions to help you make sense of a trial

Transcription

1 CRITICAL APPRAISAL SKILLS PROGRAMME Making sense of evidence about clinical effectiveness 11 questions to help you make sense of a trial These questions consider the following: Are the results of the trial valid? (SECTION A) What are the results? (SECTION B) Will the results help locally? (SECTION C) A number of italicised prompts are given after each question. These are designed to remind you why the question is important. There will not be time in the small groups to answer them all in detail! These materials were developed by the Critical Appraisal Skills Programme (CASP) and we thank them for permission to use the materials. Critical appraisal of additional material on the CELIM RCT submitted by Merck Serono for the Cetuximab STA. (Powerpoint presentation, poster, protocol, letter). October Dr Catherine Meads. xxxxxx. xxxxxxxxxxxxxxxxxxxxxxxxxxx, xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx. xxxxxxxxxxxxxxxxxxxxxxxx. xxxxxxxxxxxxx, xxxxxxxxxxxxxxxxxxxx Page 1 21/01/2009

2 A/ Are the results of the trial valid? Screening Questions 1 Did the trial address a clearly focused issue? An issue can be 'focused' in terms of - the population studied - the intervention given - the comparator given - the outcomes considered X ο ο Population (from protocol) colorectal cancer with nonresectable liver metastases. No extra hepatic disease, Karnofsky performance status >80%, no prior chemotherapy (except adjuvant finishing 6 months beforehand), no concurrent immune/hormone therapy. No information about where patients recruited, 17 centres used. Limited background information (age, sex, malignancy status indicators). Recruited between December 2004 and March Karnofsky status in patients was not given. Intervention cetuximab plus FOLFOX 6. The trial was described as neo-adjuvant treatment. Comparator cetuximab plus FOLFIRI (also neoadjuvant) Outcomes response (defined in protocol as using RECIST criteria), resection rate (mostly by 1 year), progression-free survival, disease-free survival after resection, overall survival, safety. 2 Was the assignment of patients to treatments randomized? X ο ο Stratified randomised trial. Randomisation using centrally held computer-generated list. Stratification according to two factors - technically resectable liver metastases versus technically non-resectable liver metastasis and staging using PET versus staging without PET. Page 2

3 3 Were all of the patients who entered the trial properly accounted for at its conclusion - was follow up complete? - were patients analysed in the groups to which they were randomised? ο ο?x N=111. Two of these patients did not receive therapy. K- ras tested in 99 of whom 71% were wild type. This is an interim analysis (August 2008). ITT specified in protocol. Results given for 105 patients in their randomised groups for response and resection (NB footnote suggested data response confirmation awaited in 14 patients but ) and for safety (described as pre-operative toxicity) (n=109). Also a K-ras wild type combined from both randomised groups for response. Also graph given for time to intervention but rather difficult to see the two separate groups distinct from the all patients line on the graph. No results given for progression-free survival, diseasefree survival after resection, overall survival Detailed Questions 4 Were patients, health workers and study personnel blind to treatment? - were the patients - were the health workers - were the study personnel ο ο X Open randomised trial (unblinded). No mention of blinding of outcome measurement 5 Were the groups similar at the start of the trial? In terms of other factors that might effect the outcome such as age, sex, social class 6 Aside from the experimental intervention, were the groups treated equally? X ο X More rectal cancer, more adjuvant chemotherapy and adjuvant radiotherapy, less prior liver resection, fewer liver metastases in cetuximab and FOLFIRI arm. Slightly younger in cetuximab and FOLFIRI arm (median age 62 vs 65). These differences unlikely to have much impact on results. ο X ο No information available Page 3

4 B/ What are the results? 7 How large was the treatment effect? What outcomes are measured? NB comparisons with CRYSTAL and OPUS not in italics Extra work done on CELIM results in bold A. Response rates (Complete OR Partial response) cetuximab and folfox 6 = 85% (95%CI %) cetuximab and folfiri = 66% (95%CI %) K-ras wild type combined subgroup = 79% (95%CI %) NB Complete or Partial response rates in cetuximab arms of CRYSTAL (cetuximab and folfiri) and OPUS (cetuximab and folfox 4) RCTS liver metastases subgroups were 57.4% (n=122) and 54% (n=50) K-Ras subgroup - were 59.3% (n= 172) and 60.7% (n=61). For the liver metastases subgroup of the K-ras subgroup the CR/PR rates were 77.1% (n=35) and not given (n=20). B. All Resection rates cetuximab and folfox 6 = 40% (95%CI ) cetuximab and folfiri = 43% (95%CI ) NB. From Trial Reports whole trial liver metastases only subgroup all resection rate in CRYSTAL 11.5% (14/122) and not given in OPUS. K-Ras subgroups - the all resection rate in CRYSTAL was 3.5% (6/172) and in OPUS was 13.1% (8/61). And for the liver subgroup of the K-ras subgroup in CRYSTAL was 8.6% (3/35) and not given in OPUS. C. Curative (R0) resection rates cetuximab and folfox 6 = 37% (19/52) cetuximab and folfiri = 34% (18/53). 95%CI not given 95% CI around the 37% and 34% given above were calculated using the traditional method*. Results were 23% to 49% and 21% to 47%. NB. From trial reports - curative metastatic surgery rates in cetuximab arms of CRYSTAL and OPUS RCTS liver metastases only subgroup were 9.8% (12/122) and not given. K-Ras subgroup curative metastatic surgery - were CRYSTAL 1.7% (3/172) and OPUS 9.8% (6/61). Curative metastatic surgery rates were not given for the liver subgroup K-ras subgroups but the numbers would have been very small. Median time to intervention = 5 months Page 4

5 8 How precise was the estimate of the treatment effect? See above What are its confidence limits? * Altman DG et al (eds) Statistics with confidence, 2nd Ed. BMJ Books London 2000 C/ Will the results help locally? 9 Can the results be applied to the local population? Do you think that the patients covered by the trial are similar enough to your population? ο X ο Issues are as follows: 1. The inclusion criteria for the trial state that patients should have non-resectable liver metastases, yet the trial randomisation was stratified by whether the liver metastases were technically resectable or not. Randomisation occurred before any treatment was given, but according to the study schematic, resectability was determined after 8 cycles of treatment (~4 months). 55% patients had technically non-resectable metastases at baseline. Therefore, in this respect, the trial description does not make sense. 2. The patients in the CELIM trial were liver metastases only so comparison is with the CRYSTAL and OPUS liver metastases subgroups. We know very little about the results in these subgroups. We only have the resection rates in CRYSTAL which were ~10% (as opposed to ~35% in the CELIM trial). 3. This is not a randomised comparison of cetuximab vs no cetuximab so one cannot tell whether the higher rates of all resection and curative resection are due to the cetuximab given or due to other factors about how the trial was run and how the patients were cared for. There is no information given about how the patients were actually treated, apart from the intervention given. 4. The CELIM trial has relatively small sample size of ~55 in each arm. 5. The trial was described as neoadjuvant so it might be expected that patients may be a little fitter than in a trial of first line therapy. They also only had liver metastases so the colorectal cancer may be less advanced than in those patients in the CRYSTAL and OPUS trials. In CRYSTAL 122 patients had liver metastases only and all patients had metastases so the proportion with either non-liver or multiple metastases was 89.8% ( /1198) and for OPUS it was 85.1% (336-50/336). Karnofsky performance status was not given for the CELIM trial patients so no comparison can be made with the CRYSTAL and OPUS trials. 6. The two objectives for the CELIM trial (from the protocol) were: Page 5

6 10 Were all clinically important outcomes considered? If not, does this affect the decision? 11 Are the benefits worth the harms and costs? This is unlikely to be addressed by the trial. But what do you think? - to test the feasibility of neoadjuvant treatment with cetuximab / chemotherapy followed by liver resection - to determine the optimal combination (Cetuximab/FOLFOX vs. Cetuximab/FOLFIRI) for further trials that compare chemotherapy ± cetuximab in preoperative chemotherapy. Therefore to find a higher resection rate may have been a chance finding. Alternatively, it could because the patients were at an earlier, more treatable stage than the CRYSTAL and OPUS trials, see point 5 above. Yes ο There is no mention of progression-free survival, disease-free survival after resection or overall survival. These were measured (see protocol and methods slides) but as these are interim analyses they are possibly not available yet. Yes ο The safety results do not help to tell us about the safety of cetuximab as all patients received cetuximab. Costs were not measured as an outcome in this trial. No X No X Page 6