Real-world observational data in costeffectiveness analyses: Herceptin as a case study

Real-world observational data in costeffectiveness analyses: Herceptin as a case study DR BONNY PARKINSON, PROFESSOR ROSALIE VINEY, ASSOCIATE PROFESSOR STEPHEN GOODALL AND PROFESSOR MARION HAAS ISPOR AUSTRALIA CONTEMPORARY ECONOMIC MODELLING WORKSHOP - 17 MARCH 2016

Background and motivation 2/15 A key challenge for decision makers is whether: Predicted costeffectiveness when the decision is made (based on clinical trials)? Costeffectiveness in real-world clinical practice CENTRE FOR THE HEALTH ECONOMY 2

Background and motivation 3/15 Randomised controlled clinical trials (RCTs) are the key source of efficacy Considered the gold standard approach But initial RCTs conducted for regulatory purposes: Are often conducted in highly selected and homogenous populations who receive protocol driven care. Often do not collect all evidence required due to ethical, practical or financial reasons Therefore the results of RCTs need to be translated into a form suitable for decision making: E.g. extrapolated beyond the trial or efficacy transformed into patient-relevant health outcomes Requires judgements about: Source of parameters Methods of analysis/synthesis Introduces uncertainty CENTRE FOR THE HEALTH ECONOMY 3

Background and motivation 4/15 interest in Coverage with Evidence Development (CED): Population-level payment or reimbursement linked to prospective data collection. Proposed that observational data collected post-reimbursement can be used to confirm whether: Predicted costeffectiveness when the decision is made (based on clinical trials)? Costeffectiveness in real-world clinical practice CENTRE FOR THE HEALTH ECONOMY 4

Aims 1) To what extent and why might the cost-effectiveness of a treatment differ in clinical practice from that predicted using clinical trial data? 2) What is the value of collecting real-world observational data? 3) When should a Coverage with Evidence Development (CED) arrangement be implemented instead of other risk-sharing arrangements? CENTRE FOR THE HEALTH ECONOMY 5

Case-study: trastuzumab (Herceptin) 6/15 HER2+ breast cancer HER2+ prognosis (15% patients are HER2+) time to progression and overall survival In 2000 and 2001 PBAC rejected 3 submissions trastuzumab for HER2+ metastatic breast cancer (MBC) No evidence of overall survival Expensive: $52,000 for 12 months Unacceptably high ICER Government established the Herceptin Program in December 2001 Rich observational dataset: Captured treatment of almost all patients CENTRE FOR THE HEALTH ECONOMY 6

Methods THREE STAGES 1) An economic model of trastuzumab for the treatment of HER2+ MBC was developed using trial evidence and analysed using value of information methods. 2) Observational data from the Herceptin Program were analysed to inform modifications to model parameters. 3) The model was adjusted to estimate the real-world costeffectiveness of trastuzumab for the treatment of HER2+ MBC CENTRE FOR THE HEALTH ECONOMY 7

Stage 1 AN ECONOMIC MODEL WAS DEVELOPED USING TRIAL EVIDENCE AND ANALYSED USING VALUE OF INFORMATION METHODS

Methods MODEL STRUCTURE 9/15 Reflected first-line treatment of HER2+ MBC with trastuzumab Similar to that which would have been considered by the PBAC in 2001 Built in TreeAge 2013 CENTRE FOR THE HEALTH ECONOMY HER2 testing decision tree Markov model 9

Methods MODEL STRUCTURE TWO SETS OF INPUTS Best available in 2001 and best available in 2013. Efficacy based on trial H0648g Trastuzumab + paclitaxel (n=92) vs paclitaxel (n=96) Only trial available in 2001 Clinicians may not adhere to guidelines (i.e. imperfect implementation of PBS listing) conducted sensitivity analysis of trastuzumab + other chemotherapies using RCTs that became available after 2001 CENTRE FOR THE HEALTH ECONOMY 10

Methods MODEL STRUCTURE TESTING Immunohistochemistry (IHC) and in situ hybridisaiton (ISH) Tests are not perfect Test +ve (including false +ve patients) trastuzumab + paclitaxel Test ve (including false -ve patients) paclitaxel EFFICACY FOLLOWING TESTING Assumed that the health outcomes following treatment with paclitaxel is independent of true HER2 status Post-hoc analysis of trial H0648g Assumed that the efficacy of trastuzumab plus paclitaxel applies regardless of the patient s true HER2 status As trial H0648g analysed on an intent to treat (ITT) basis The health outcome impact of discontinuations, dose reductions, deaths due to AEs and any second-line treatments were not explicitly modelled As trial H0648g analysed on ITT basis CENTRE FOR THE HEALTH ECONOMY 11

Methods INPUTS Best available data in 2001 Best available data in 2013 Test accuracy Efficacy Incidence of grade 3 adverse events (n=40) and discontinuations Published literature depending on availability Trial H0648g, Kaplan-Meier estimates were digitised using Engauge and parametric functions were fitted using STATA. When not available, based on medians and assumed a constant hazard rate of progression Trial H0648g Utilities and disutilities Hauser (2001) + published literature depending on availability Peasgood (2010) + published literature depending on availability Resource use Cost of trastuzumab HER2 testing Chemotherapy and pre-medications, chemotherapy administration Pre-progression: specialist visits, imaging and pathology Post-progression: capecitabine + as per pre-progression Palliative care and adverse events (2013 model only) Assumed = PBS listed price of trastuzumab for the treatment of early breast cancer Unit costs Reported in 2001 A$ Reported in 2012-2013 A$ Time horizon Cycle length Outcomes Perspective Discount rate OFFICE I FACULTY I DEPARTMENT 10 years 1 week Life years gained and Quality Adjusted Life Years (QALYs) Healthcare system 5%pa for costs and outcomes 12

Methods SENSITIVITY ANALYSES UNIVARIATE SENSITIVITY ANALYSIS Upper and lower 95% confidence intervals SCENARIO ANALYSES Imperfect implementation of listing: use of other concomitant chemotherapies (efficacy based on RCTs that became available after 2001) trastuzumab post-progression THRESHOLD ANALYSIS ON PRICE AND RISK-SHARE ARRANGEMENTS ICER = $60,000/QALY gained or $80,000/QALY gained PROBABILISTIC SENSITIVITY ANALYSIS Uninformative priors where evidence is limited CENTRE FOR THE HEALTH ECONOMY 13

Methods THE VALUE OF COLLECTING OBSERVATIONAL DATA It was assumed that the manufacturer suggested that a CED arrangement be implemented with interim funding subject to: collecting observational data linked to PBS and MBS data All analyses based on 2001 model. BEST-CASE SCENARIO ANALYSIS Each parameter value was adjusted to equal the upper or lower 95% CI, depending on the results of the univariate sensitivity analysis. VALUE OF INFORMATION ANALYSIS Expected Value of Perfect Parameter Information (EVPPI)* estimated using Monte Carlo methods (i.e. simulation) It was assumed that: treatment was restricted to concomitant treatment with paclitaxel and not continued post-progression the maximum life-time of trastuzumab was 13 years (2001-2014) 464 patients receive trastuzumab per year. Expected * difference in value the expected of sample net benefit information with perfect information (EVSI) about not a sub-set conducted of parameters because compared observational to the expected net benefit data with captures CENTRE FOR THE HEALTH ECONOMY the 14 treatment current information of almost set (or the all opportunity patients cost of making the wrong decision).

Results BASE CASE 15/47 Treatment arm Cost ($) Incremental cost ($) LY Life years LYG ICER ($/LYG) QALYs QALYs QALYs gained ICER ($/QALY gained) Best-available data in 2001 Paclitaxel only available 16,731 1.94 1.24 Trastuzumab available 44,803 28,072 2.17 0.23 122,859 1.39 0.16 180,910 Best-available data in 2013 Paclitaxel only available 27,319 1.95 0.00 0.90 Trastuzumab available 40,576 13,257 2.05 0.11 124,781 0.96 0.06 221,245 ICER > range considered acceptable BUT the ICER is likely to be over-estimated due to cross-over. Decrease in QALYs gained due to: - different testing parameters - lower utility values - inclusion of additional disutilities associated with AEs. Decrease in incremental costs due to: - availability of the smaller vial - ability to administer trastuzumab three-weekly - a decrease in the PBS cost of paclitaxel due to the loss of patent protection in 2013. CENTRE FOR THE HEALTH ECONOMY 15

Results BASE CASE 16/47 Most sensitive to: Odds ratio and probability of an adverse event Hazard rate ratio (HRR) and duration of time to progression (TTP) CENTRE FOR THE HEALTH ECONOMY OR of AEs (95%CIs) Probability of AEs (95%CIs) Duration of TTP with chemotherapy (95%CIs) HRR of TTP (95%CIs) OR of response with trastuzumab (95%CIs) Probability of response with chemotherapy (95%CIs) HRR of OS (95%CIs) Duration of OS with chemotherapy (95%CIs) Disutility AEs (95%CIs) Utility response (95%CIs) Utility stable disease (95%CIs) Utility progressive disease (95%CIs) OR of discontinuation (95%CIs) Probability of discontinuation (95%CIs) Dose reduction (95%CIs) Frequency of cardiac testing, ECHO/MUGA and ECG (95%CIs) Frequency of CT scans (95% CIs) Probability outpatient (95%CIs) Trastuzumab cost -10% (0% in base case) Probability public given inpatient (95%CIs) Duration of hospitalisation for AEs (95%CIs) Duration of treatment with chemotherapy (upper 95%CI) Cost of palliative care (95%CIs) Cost of AE treatment in hospital (95%CIs) Cost of treatment administration in hospital (95%CIs) Probability trastuzumab weekly (95%CIs) Proportion using CVAD (95%CIs) Frequency of specialist visits(95%cis) Cost of treatment with chemotherapy post progression (95%CIs) Duration of treatment with chemotherapy post progression (95%CIs) Proportion using ECHO vs MUGA (95%CIs) Frequency of pathology tests (95%CIs) Discount rate = 3% and 7% (5% in base case) Time horizon +/-10% (520 in base case) Probability test results in true positive (95%CIs) Probability test results in true negative (95%CIs) HER2+ prevalence (95%CIs) Dominated $0 $50 $100 $150 $200 $250 16 ICER ($/QALY gained) Thousands

Results BASE CASE 17/47 Paclitaxel most cost-effective concomitant chemotherapy Trastuzumab continued post progression ICER Assumes not treatment benefit Base case model, no treatment post-progression Model 2, no treatment post-progression Model 3, no treatment post-progression Model 4, no treatment post-progression Model 5, no treatment post-progression Model 6, no treatment post-progression Base case model, with treatment post-progression Model 2, with treatment post-progression Model 3, with treatment post-progression Model 4, with treatment post-progression Model 5, with treatment post-progression Model 6, with treatment post-progression $0 $100 $200 $300 $400 ICER ($/QALY gained) Thousands CENTRE FOR THE HEALTH ECONOMY 17

Results PROBABILISTIC SENSITIVITY ANALYSIS 18/47 Very low probability of being cost-effective Incremental costs ($) 80,000 70,000 60,000 50,000 40,000 30,000 20,000 10,000 0-0.20 0.00 0.20 0.40 0.60 0.80 1.00 QALYs gained Probability Cost-effective 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 0 50,000 100,000 150,000 200,000 250,000 300,000 Willingness to pay ($/QALY gained) Chemotherapy only available, no price discount or dose cap With 80% price discount With 24 week dose cap With $8,000 per patient fixed price Trastuzumab available, no price discount or dose cap With 80% price discount With 24 week dose cap With $8,000 per patient fixed price CENTRE FOR THE HEALTH ECONOMY 18

Results THRESHOLD ANALYSIS 19/47 For the ICER = $60,000/QALY gained 96% price discount needed For the ICER = $80,000/QALY gained 80% price discount needed Alternatively a dose cap of 24 weeks or a fixed cost of $8,000 per patient would be needed for the ICER =$80,000/QALY gained. Unrealistic in the long-term, but may be reasonable in the short-term while research is conducted. CENTRE FOR THE HEALTH ECONOMY 19

Results THE VALUE OF COLLECTING OBSERVATIONAL DATA 20/47 BEST CASE SCENARIO ANALYSIS Best case without price discount = $113,067/QALY gained Best case with price discount = below $80,000/QALY gained VALUE OF INFORMATION Population EVPPI without price discount = $0 Population EVPPI with price discount > $0 Population EVPPI ($) Millions $7.0 $6.0 $5.0 $4.0 $3.0 $2.0 $1.0 No price discount, observational data + MBS data + PBS data (+/- admitted patient data) 81% price discount, observational data + MBS data + PBS data 81% price discount,observational data + MBS data + PBS data + admitted patient data 24 week dose cap, observational data + MBS data + PBS data CENTRE FOR THE HEALTH ECONOMY $0.0 2 4 6 8 10 12 Time to collect data (years) $8,000 per patient fixed price, observational data + MBS data + PBS data 20

Stage 2 OBSERVATIONAL DATA WERE ANALYSED TO INFORM MODIFICATIONS TO MODEL PARAMETERS

Methods SAMPLE Patients who received treatment between December 2001 and March 2010. Analysis included 3617 patients Linked to PBS and MBS data ANALYSIS Resource use: Trastuzumab: dosage regimen and treatment duration Chemotherapies administered pre- and post-progression: type and treatment duration Medical services: baseline and time between services Health outcomes: Overall survival based on date of death Progression free survival based on a proxy (Joshi et al 2011) Date of progression = date of prescription of a new chemotherapy not previously administered which occurs >42 days after the start of treatment with trastuzumab OR end-date of treatment with trastuzumab if monotherapy. Estimated in STATA using Kaplan-Meier methods to account for censoring CENTRE FOR THE HEALTH ECONOMY 22

Results PROGNOSIS Older patients and more prior chemotherapies in clinical practice vs trials Likely poorer prognosis TESTING ISH testing in clinical practice vs trials Due to PBS restrictions on other chemotherapies Percent 0 10 20 30 Percent 0 20 40 60 80 100 Age of patients at enrolment on Herceptin Program 20 30 40 50 60 70 80 90 100 Years Test used to determine HER2 status 2002 2003 2004 2005 2006 2007 2008 2009 2010 Start year of treatment IHC3+ FISH+ IHC2+ and FISH+ CENTRE FOR THE HEALTH ECONOMY 23

Results TRASTUZUMAB REGIMEN Three weekly dosing regimen Less resources required and more convenient for patients MEDICAL SERVICE USE VS GUIDELINES patients treated in outpatient setting and public inpatients central venous access device (CVAD) use frequent specialist visits and pathology tests frequent: CT scans, BMD scans, ECGs and ECHO/MUGA scans probability that cardiac function tested using ECHO vs MUGA scans Percent 0 20 40 60 80 100 Likely treatment regimen based on weight at enrolment and mean quantity requested 2002 2003 2004 2005 2006 2007 2008 2009 2010 Start year of treatment Weekly Three weekly CENTRE FOR THE HEALTH ECONOMY 24

Results 25/15 INITIAL TREATMENT Broader range of concomitant chemotherapies than in guidelines (taxanes or monotherapy) or where RCT evidence is available. POST PROGRESSION TREATMENT Almost half received trastuzumab Capecitabine and lapatinib CENTRE FOR THE HEALTH ECONOMY Proportion of patients 0.1.2.3.4.5 Selected chemotherapies used after disease progression 2002 2004 2006 2008 2010 Year that treatment ceased Capecitabine Docetaxel Aromatase Gemcitabine Carboplatin Paclitaxel Vinorelbine Lapatinib Doxorubicin Trastuzumab via Herceptin Program 25 25

Results 26/15 TTP/PFS, median weeks (95%CI) Overall survival, median weeks (95%CI) Herceptin Program Clinical trials Herceptin Program data (PFS) (TTP) data Clinical trials All patients 36.3 (34.1, 38.9) - 129.6 (123.6, 136.4) - Any initial concomitant 50.9 (48.3, 53.3) - 151.3 (142.6, 163.4) - chemotherapy Monotherapy 16.0 (14.4, 17.9) - 94.4 (86.4, 102.9) - By initial concomitant chemotherapies (pre-progression) Docetaxel M77001 50.9 (40.0, 58.7) 135.7 (118.7, 182.6) 50.4 (45.4, 54.1) 141.7 (130.1, 154.3) TRAVIOTA** 28.26 (NR) NR HERNATA 53.9 (NR) 155.2 (NR) Paclitaxel H0648g 30.0 (23.0, 43.0) 96.1 (73.5, 133.5) 51.0 (48.0, 57.0) 141.9 (125.9, 164.7) Gasparini 43.00 (NR) NR (2007) Anthracycline 27.9 (14.0, 43.7) 33.9 (30.4, 39.1) 199.7 (137.1, NA) 116.5 (100.0, 143.5) Aromatase inhibitor 68.4 (63.6, 76.0) 20.9 (16.1, 33.5) 213.7 (187.4, 244.1) 123.9 (99.1, 184.4) Vinorelbine TRAVIOTA 39.0 (29.9, 48.1) 37.0 (NR) 101.3 (72.4, 117.1) NR HERNATA 66.5 (NR) 168.7 (NR) Other chemotherapy 38.4 (27.0, 49.1) - 151.6 (129.3, 237.3) - CENTRE FOR THE HEALTH ECONOMY Significantly progression free survival (PFS) if monotherapy. Poorer prognosis patients if administered paclitaxel if administered docetaxel or an anthracycline if administered vinorelbine Results with aromatase inhibitors should be considered with caution. Significantly overall survival if monotherapy. Poorer prognosis patients overall survival with chemotherapies Unless administered vinorelbine Poorer prognosis patients 26

Stage 3 THE MODEL WAS ADJUSTED TO ESTIMATE THE REAL-WORLD COST-EFFECTIVENESS OF TRASTUZUMAB.

Methods 28/15 Typical approach: estimate the average costs and health outcomes for patients receiving a particular treatment or a comparator in an observational dataset. Selection bias issues Often don t have a control group Aimed to adjust health outcomes to reflect real-world clinical practice while leaving relative efficacy unchanged. 28

Methods 29/47 Based on the statistical analysis of the Herceptin Program data: Step 1: Modify the resource use in the economic model to reflect clinical practice. Step 2: Adjust the health outcomes with trastuzumab + chemotherapy to match clinical practice and THEN adjust the health outcomes with chemotherapy alone such that the relative efficacy remains unchanged. If transition probabilities based on median durations applied, then approach similar to calculating and applying a hazard rate ratio (HRR). If digitised Kaplan-Meier curves applied: Iteratively adjust trastuzumab + chemotherapy curve ( or ) until median PFS or overall survival = clinical practice achieved THEN Iteratively adjust chemotherapy alone curve until HRRs = RCTs.

Methods 30/47 Step 3: Estimate a weighted average ICER across age groups, accounting for differences in weight and thus drug costs. Step 4: Estimate a weighted average ICER across concomitant chemotherapies. Weights were based on the proportion of patients in the Herceptin Program receiving each concomitant chemotherapy, or combination thereof, prior to progression

Results 31/15 ICER in real-world clinical practice Treatment postprogression Price discount Taxanes only Incremental cost ($) Life years ICER based on trial evidence = $221,245 QALYs LYG ICER ($/LYG)* QALYs gained ICER ($/QALY)* No 0% No 13,470 0.13 116,043 0.08 193,702 Yes 0% No 24,793 0.13 209,908 0.08 351,847 No 80% No 6,626 0.13 56,680 0.08 94,767 Yes 80% No 8,891 0.13 75,453 0.08 126,396 No 0% Yes 12,630 0.15 84,274 0.09 147,288 No 80% Yes 6,284 0.15 41,655 0.09 73,009 Weighted average net gain/loss from approving trastuzumab Treatment postprogression Price discount Taxanes only Weighted average health outcomes from approving trastuzumab* Weighted average net gain/loss from approving trastuzumab* Life Years gained QALYs gained $60,000/QALY $80,000/QALY No 0% No -40,489,475-33,436,845 Yes 0% No -92,309,978-85,257,348 617 353 No 80% No -9,167,004-2,114,374 Yes 80% No -19,531,104-12,478,474 No 0% Yes -34,109,410-26,211,608 701 395 No 80% Yes -5,066,797 2,831,005 * The ICER and the net loss (gain) is likely to be slightly overestimated (underestimated) due to the inability to control for cross-over. CENTRE FOR THE HEALTH ECONOMY 31

Conclusions

Conclusions 33/47 Trastuzumab = very high cost drug with a very high ICER The ICER was a little lower or much higher, depending whether you considered treatment post-progression. Unenforced treatment guidelines can significantly reduce cost-effectiveness in clinical practice Treatment past-progression and concomitant chemotherapies CENTRE FOR THE HEALTH ECONOMY 33

Conclusions 34/47 In 2001 the available evidence was sufficiently certain to make a decision Sensitivity analysis and best-case scenario ICER remained above the range considered acceptable. Value of collecting observational data was low However no uncertainty around Kaplan-Meier curves or coefficients of the fitted parametric functions, and not able to adjust for cross-over. Should not have established Herceptin Program without a price discount A Coverage with Evidence Development (CED) recommendation was not appropriate on the basis of collecting observational data Another type of risk-share arrangement should have been considered CENTRE FOR THE HEALTH ECONOMY 34

Thank you

Funding and ethics approvals 36 This research forms part of the CHEETAH programme, which is funded by a NHMRC Capacity Building Grant in Health Services Research (NHMRC ID: 571926) The analysis of the Herceptin Program data was approved by the NSW Population & Health Services Research Ethics Committee (protocol HREC/10/CIPHS/11), the Medicare Australia External Request Evaluation Committee (2010/CO07329), and is a part of a research program approved by the University of Technology Sydney Research Ethics Committee (UTS HREC REF NO. 2009-143P). CENTRE FOR THE HEALTH ECONOMY