The Managed Introduction of New Medicines How to carry out health technology appraisals and guidance. Learning from the Scottish experience Richard Clark, Principal Pharmaceutical Analyst July 10 th 2009, Glasgow Part 1 (SMC) Remit To provide advice to NHS Boards and their Area Drug and Therapeutics Committees across Scotland about the status of: All newly licensed medicines All new formulations of existing medicines New indications for established products (licensed from January 2002). This advice will be made available as soon as practical after the launch of the product involved. (SMC) Remit Postcode prescribing The objective is timely delivery of effective, cost effective medicines to the population of Scotland Evidence-based medicine Evidence-based Medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients It means integrating individual clinical experience with the best available external clinical evidence from systematic research. Sackett DL BMJ 1996; 312: 71-72
Company submission to SMC New Product assessment Form from sponsor pharmaceutical company On template provided by SMC According to advice published by SMC All supporting references Summary of Product Characteristics SMC process prior to NDC meeting Pharmacy Assessment Team (PAT) Submission of New Medicine Form Scottish Medicines Consortium Assessment Team Assessment Review Economic Assessors Outputs prior to NDC meeting Evaluated submission Incorporating pharmacy assessor s comments/ additions Draft advice document 6-page summary Summary of specialist advice Background information and comment Heath economics checklist Critique of health economic evidence Summary for detailed advice document 1. The evaluated submission Pharmacy assessor to review submission and consider: Which parts of the evidence-base are most relevant to decision making? Are statements in the submission supported by the sources cited? Can information be added to aid understanding, add perspective and achieve balance? What issues emerge during critical appraisal of key evidence? 2. The detailed advice document Summarise the evidence in the evaluated submission in terms of: Comparative efficacy Comparative safety Clinical effectiveness Cost effectiveness (from health economics assessor) Efficacy and Effectiveness Efficacy What the intervention does to the patient blood pressure reduction, cholesterol levels Effectiveness What the intervention does for the patient survival, freedom from symptoms, quality of life Clinical effectiveness Cost effectiveness
Efficacy It does what it says on the tin 3. Expert advice SMC panel of experts within different specialities provide background information: Standard template for generic questions (common to any submission) Follow up with questions specific to the submissions Collate and anonymise expert responses for committees The Advice Accepted for use in NHS Scotland Accepted for restricted use in NHS Scotland Restricted by patient group Restricted by prescriber Not recommended for use in NHS Scotland Breast Cancer Most common cancer in Scottish women Accounts for 27.5% of cancers in women Increasing incidence Decreasing mortality Early detection / improved treatment But substantial morbidity / mortality remains Poor prognosis with metastatic disease Primary therapies Surgery Total mastectomy Conservative surgery Radiotherapy Chemotherapy Combination of approaches Adjuvant/ neo-adjuvant therapies Objective is to improve the outcomes with primary therapy Adjuvant e.g. to remove microscopic residues of malignant cells after surgery Neo-adjuvant e.g. before surgery to shrink/ restrict growth of tumour
Adjuvant neo-adjuvant therapy in breast cancer Adjuvant/ neo-adjuvant regimens of Radiotherapy Chemotherapy e.g. anthracycline- or taxane- based regimens Anti-oestrogen therapy for hormone- receptor positive cases E.g. Tamoxifen, aromatase inhibitors Oestrogen or progestogen-positive patients HER-2 antagonists HER-2 One of a family of transmembrane tyrosine kinases Mediate cell growth, survival and differentiation Over-expression of HER-2 or amplification of HER-2 gene associated with: Aggressive behaviour of breast tumour Early relapse Survival decrease of up to 50% Trastuzumab Humanised monoclonal antibody against HER-2 Alone or in combination with chemotherapy had shown benefit in HER-2 positive breast metastatic cancer Licensed and marketed Not associated with classic chemotherapy toxicity But hypersensitivity (particularly first dose) Cardiac toxicity (particularly congestive heart failure) in about 1.4% of patients New research Adjuvant treatment of surgically resected, HER-2 positive, early breast cancer Study published in New England Journal of Medicine October 2005 Not yet licensed in Europe for early breast cancer
Trastuzumab (Herceptin) SMC submission Submitted in March 2006 Marketing authorisation expected July 2006 Four studies One (HERA) identified as pivotal and used licensed regimen Two others studies described in detail Maintenance dose similar to licensed regimen but given weekly rather than 3-weekly Pivotal Study What makes a study pivotal? Licensed and/or clinically relevant indication and regimen Recent / not superseded Phase III Used to support regulatory approval Adequate design, size, power etc Critical Appraisal Skills Programme (CASP) 10 questions to help you make sense of randomised controlled trials http://www.phru.nhs.uk/pages/phd/casp.ht m CASP Question 1 Did the study ask a clearly focused question? Addresses a specific patient problem Which patients/ population are we interested in? Which intervention should be used? What are the main alternatives to compare with the intervention What outcomes should be considered (and what is really important for the patient)? The stated objective of the HERA study The HERA trial investigated whether the administration of trastuzumab was effective as adjuvant treatment for HER- 2-positive breast cancer if used after completion of the primary treatment e.g. surgery, radiotherapy and chemotherapy given preoperatively [neoadjuvant], postoperatively [adjuvant] or both. All patients also received background adjuvant chemotherapy from a list of approved regimens. Piccart-Gebhart et al. Trastuzumab after adjuvant chemotherapy in HER-2 positive cancer. NEJM 2005;353:1659-72 The population under study Main inclusion criteria Women 18 years or over Histologically confirmed early stage invasive breast cancer with no metastatic spread HER-2 positive Left ventricular ejection fraction (LVEF) at least 55% Primary surgical intervention and radiotherapy Adjuvant chemotherapy from approved list Hormonal therapy for eligible patients
The population under study Main exclusion criteria (from published paper): Distant metastases, previous invasive breast carcinoma, neoplasm not involving the breast Clinical stage T4 tumours Prior mediastinal radiation Cardiac exclusion criteria as specified e.g. congestive heart failure, coronary artery disease The population under study Baseline characteristics Were the groups well balanced in terms of baseline characterises?» Demographic e.g age race» Disease-related e.g. severity of disease, co-morbidity Were there any differences at entry that might have explained any outcome(s) (confounding)?» Prognostic factors including co-morbidity» Factors that might influence response to therapy e.g drug interactions Are there baseline factors that may influence generalisability?» Not representative of the population of interest The intervention Trastuzumab 8mg/kg loading dose then 6mg/kg every three weeks Dose modified for specified adverse events One group assigned to one year s s treatment (reported in submission) n=1694 One group assigned to two years treatment (ongoing at time of submission) n=1694 The Comparator Observation i.e. Standard care without trastuzumab n=1693 No relevant comparator was available The primary outcome and primary analysis Disease-free survival Time to first of either recurrence of breast cancer at any site, development of new breast cancer or non-breast malignant disease or death from any cause. Reported as the hazard ratio for events in the one- year trastuzumab group compared to observation. Secondary outcomes Components of primary composite outcome Overall survival Breast cancer-related Non-breast cancer related Kaplan Meier survival analysis
CASP Question 1 Did the study ask a clearly focused question? Consider if the question is focused in terms of: the population studied the intervention given the outcomes considered CASP Question 2 Was this a randomised controlled trial (RCT)? Yes, no or don t t know? Groups should be the same in all respects except for study intervention From published paper: The Herceptin Adjuvant (HERA) (Breast Cancer International Group [BIG] 01-01) trial is an international, intergroup, open-label, phase 3 randomised trial. CASP intermediate question Is it worth continuing? Be clear about what question(s) you want the paper to answer and how useful the paper will be Do not waste valuable time if it is not useful CASP Question 3 Were subjects appropriately allocated to intervention and control groups Were there robust procedures for randomisation? Randomisation is described in HERA and was stratified Stratification is designed to ensure that relevant sub-groups of patients are represented CASP Question 4 Were participants and study personnel blind to study group allocation? Blinding conceals details of treatment allocation to reduce bias Double blind: patient and investigator Single blind: from one group only Sometimes blinding is not possible, practical or ethical Surgical versus medical intervention
CASP Question 5 This study was open-label It would technically be possible to blind this study No explanation given, but adverse events, such as infusion reactions could compromise blinding Consider if it matters in this study: how much potential for observer bias? Observer bias is more likely with subjective than objective outcomes Look for additional precautions e.g. samples sent for blinded assessment at central laboratory Were all of the participants who entered the trial accounted for at its conclusion? Intention to treat analysis? If so, how does it account for missing data e.g. last observation carried forward? Do the numbers add up? Intention to treat All participants are analysed in the treatment group to which they are randomised, even if they drop out or change treatment Must deal with missing values e.g. for drop-out Last observation carried forward (LOCF) Imputed (estimated) values for missing data Look for sensitivity analysis e.g. different population, different methods of imputation Do the numbers add up? CONSORT diagram CASP Question 6 Were the participants in all groups followed up and data collected in the same way? Reviewed at the same time intervals? Received the same amount of attention from researchers and health workers? Any differences may introduce performance bias. CASP Question 7 Did the study have enough participants to minimise the play of chance? The power of a study is its ability to detect a significant difference in the study sample when it exists in the population. A fundamental determinant of power is the number of patients enrolled The smaller the difference between treatments, the the larger the number of patients needed
CASP Question 8 How are the results presented and what is the main result? How reported? Proportion of responders, mean response, survival curves etc? How large is the result and how clinically meaningful? How would you sum up the bottom line result in one sentence? Trastuzumab results Planned interim analysis performed after pre-specified number of primary events Median follow-up 12 months (range 0 to 36): Significant advantage for trastuzumab in primary outcome Data for one-year arm trastuzumab arm locked and patients in observation arm offered trastuzumab and patients in two-year trastuzumab arm ongoing Trastuzumab results: interim analysis Hazard ratio for primary outcome trastuzumab versus observation: 0.54 (p<0.0001) This is equivalent to a relative reduction in risk of (100-54)=46% From the clinical effectiveness section of the submission: [There is] no doubt that the addition of trastuzumab reduces the risk of relapse after surgery by around 50%. Absolute reduction in risk Percent of patients disease-free trastuzumab versus observation: 85.8% versus 78.2% Percent of patients with event Trastuzumab (100-85.8%) = 14.2% Observation (100-78.2)% = 21.8% Absolute reduction in risk (21.8-14.2)% = 7.6% Consider the observed outcomes A) and hypothetical outcomes B) for the percentage of patients with an event Trastuzumab A) 14.2% B) 0.142% Observation A) 21.8% B) 0.218% The relative reduction in risk does not change from A) to B) Relative reduction in risk takes no account of baseline risk. Again, consider the observed outcomes A) and hypothetical outcomes B) for the percentage of patients with an event Trastuzumab A) 14.2% B) 0.142% Observation A) 21.8% B) 0.218% The absolute reduction in risk is very different for A) and B): A) 7.6% B) 0.076% The absolute reduction in risk gives a measure of the actual benefit we can expect.
Level of significance Level of significance The Null hypothesis is that any difference observed in the study sample has arisen by chance The p-value is the probability that an observed difference could have arisen by chance Below a certain threshold we conclude that this probability is no longer compatible with the null hypothesis We state that the difference is statistically significant at that level of significance A statistically significant result is not necessarily clinically significant Most often the level of significance is set at p<0.05 This is pragmatic: in one in 20 cases we will conclude a significant difference when none exists Scottish CASP Question Medicines Consortium 9 If you torture your data long enough it will confess to something How precise are these results? Is the result is precise enough to make a decision? Has a confidence interval been reported? Would your decision be the same at the upper confidence limit as at the lower confidence limit? Estimation Statistics 95% Confidence Intervals give a range of values around the point estimate such that there is a 95% probability that this will include the true value for the population Hazard ratio for primary outcome: trastuzumab versus observation HR = 0.54 (95% CI: 0.44 to 0.67) The 95% CI for the OR do not pass through 1.0 which represents no difference between treatments
Hazard ratio for overall survival in HERA: trastuzumab versus observation HR = 0.75 (95% CI: 0.47 to 1.21) There would a 95% probability that the true population value lies between 0.47 (advantage for trastuzumab) and 1.21 (advantage for observation). This uncertainty is reflected in the p-value for this outcome (p=0.2379) Overall survival in other studies In a joint analysis of two non-pivotal studies with longer follow-up: Absolute survival at 3 years was 94.3% in the trastuzumab group and 91.7% in the observation group Absolute difference = 2.5% Hazard ratio = 0.67 (95% CI 0.48 to 0.93) P=0.015 CASP Question 10 Were all important outcomes considered so the results can be applied? Could the people included in the trial be different from your population in ways that would produce different results? Does your local setting differ much from that of the trial? Can you provide the same treatment in your setting CASP Question 10 Consider outcomes from the point of view of the: individual policy maker and professionals family/carers wider community
Putting it all together Do you agree with the authors conclusions? Drawing conclusions not supported by the findings Generalisabilty from sample to broader population Individual trials do not exist in isolation and many are unpublished The evidence base may already have been evaluated Other types of evidence and perspective Cost effectiveness Patient perspective Expert opinion Clinical use and practice Key Questions Simplified Is the study useful in answering my clinical question? Why was it done? How was it done What has it found? What are the implications? What else is of interest? How does it apply to my patients Clinical effectiveness issues for trastuzumab Early termination of HERA study Limits availability of data on medium to longterm outcomes including» overall survival» CNS metastases (observed in earlier studies)» cardiotoxicity Crossover of observation patients will confound later results Clinical effectiveness issues Low incidence of congestive heart failure and cardiac death in HERA but: Strict inclusion and exclusion criteria Monitoring of cardiac function throughout Exit criteria for patients with emerging cardiovascular issues Uncertainty about optimal trastuzumab regimen Two-year treatment arm in HERA is ongoing Other studies have used shorter regimens Differences in adjuvant chemotherapy regimens in study» Concomitant or sequential, Taxane or other