Appendix A: Value of Information Model Briefs (online only appendix) 1) EGFR mutation testing in maintenance treatment for advanced NSCLC

Transcription

1 Appendix A: Value of Information Model Briefs (online only appendix) 1) EGFR mutation testing in maintenance treatment for advanced NSCLC Figure 1: EGFR testing vs. Standard Care Model Schematic *All patients are followed until death The epidermal growth factor receptor (EGFR) is a cellular protein involved in multiple cancer related cellular processes including apoptosis, angiogenesis, and proliferation. Recent data suggest that patients with EGFR mutations are more responsive to treatment with erlotinib than those without mutations, which may make it possible for doctors and patients to tailor therapy at diagnosis. However, considerable uncertainty remains as to the clinical utility of EGFR testing strategies relative to standard care. To quantitatively assess this uncertainty, and to examine the potential economic value that could be created by additional study of EGFR mutation testing, a value of information model was were created to examine the use of EGFR testing in the maintenance treatment of advanced non-small cell lung cancer (NSCLC). The model uses decision-analytic methods to examine the patient population over a lifetime horizon from a U.S. payer perspective. The decision structure underlying the model is illustrated above (Figure 1). The model includes probabilistic sensitivity analysis functionality, which enables probabilistic calculation of the net monetary benefit of EGFR testing and standard care strategies. Based on these probabilistic outcomes, the per-patient expected value of perfect information (EVPI) was calculated at a willingness-to-pay threshold of $150,000 per qualityadjusted life year (QALY). The 10-year discounted affected population was calculated based on SEER incidence estimates, and utilized to calculate the population EVPI. The model was primarily informed by two trials performed in the maintenance setting, (i.e. after non-progression following 4 cycles of first-line platinum doublet therapy) the SATURN trial evaluating erlotinib and a similar trial evaluating the use of pemetrexed.(16, 17) The SATURN trial demonstrated that in patients treated with erlotinib, those with EGFR mutations had an improved hazard ratio for progression free survival, (HR = 0.10, ) versus patients without mutations, (HR=0.78, ).(16) Health state utility values were derived from a study of NSCLC health states preferences in a community sample of 100 U.K. citizens and utilized visual analogue scale (VAS) and standard 1

2 gamble (SG) utility elicitation methods to examine participant s preferences for 19 NSCLC health states.(23) All drug costs were derived from manufacturer average sales price or average wholesale prices and all procedure costs were derived from Medicare fee schedules. Average monthly NSCLC post-recurrence costs were obtained from previously published research.(24) The model is not intended to capture all expenditures that would typically be incurred in the treatment of advanced NSCLC; rather, the costs included in the model represent major expenditures that are expected to vary across the strategies examined. The structure and calculations of EGFR value of information decision-analytic model relies on several assumptions. First, the models assume that all patients follow their indicated clinical pathway, with EGFR mutation patients receiving erlotinib and mutation negative patients receiving pemetrexed or best supportive care. Second, the value of information calculations assume that EGFR testing would be utilized over a 10-year time horizon, if indicated as the more favorable strategy. Lastly, affected population calculations assume that the proportion of NSCLC patients diagnosed at an advanced stage will remain constant and at 2009 levels over the 10 year lifetime horizon. An inputs table and a cost-effectiveness acceptability curve are provided below. Table 1: EGFR Mutation testing: Inputs Life Expectancy Base Case Low High Distribution Source - Best supportive care Mean OS BSC-overall Normal (16) Mean PFS BSC-overall Normal (16) Mean PFS BSC-EGFR mutation Normal (16) Mean PFS BSC-EGFR wild type Normal (16) -Erlotinib Mean OS ERL-overall Normal (16) Mean PFS ERL-overall Normal (16) Mean PFS ERL-EGFR mutation Normal (16) Proportion EGFR mutant 10% Beta -Pemetrexed Mean OS PEM-overall Normal (17) Mean PFS PEM-overall Normal (17) Mean PFS PEM--EGFR wild type Normal (17) Utility Scores Stable disease on no AE Beta (23) Stable disease on AE Beta (23) Progressive disease Beta (23) Adverse Events Any-ERL 12.0% 9.6% 14.4% Beta (16) Rash-ERL 9% 7.2% 10.8% Beta (16) Any-PEM 16% 12.8% 19.2% Beta (16) Fatigue-PEM 5% 4.0% 6.0% Beta Any-BSC 1.0% 0.8% 1.2% Beta (16) Costs Cost of Erlotinib (150 mg tablets) $4,823 $4,341 $5,305 Gamma (25) Cost of Pemetrexed (500ml vial) $2,529 $2,276 $2,782 Gamma (26) Dexamethazone (4mg) $7 $6 $7 Gamma (26) Folic Acid (5 mg) $1 $1 $1 Gamma (26) Cost of 10 vials of Neupogen (4800 mcg) $3,427 $3,085 $3,770 Gamma (26) Cost per vial of Epogen 40,000 units/ml $3,815 $3,433 $4,196 Gamma (26) Cost of 1 60 g tube of Cleocin-T gel $62 $56 $68 Gamma (27) Cost of 1 bottle loperamide of 30 tablets of 2mg each $10 $9 $11 Gamma (28) Outpatient physician visit (Cpt code: 99215, facility) $107 $97 $118 Gamma (29) IV Administration Cost / hour (CPT code: 96413) $171 $154 $188 Gamma (29) Inpatient consultation (Cpt code: 99255) $207 $187 $228 Gamma (29) Follow-up consultation (Cpt code: 99233) $101 $91 $111 Gamma (29) Cost of SC Administration (Cpt code: 11950) $50 $45 $55 Gamma (29) Cost of RBC Transfusion $1,362 $1,226 $1,499 Gamma (30) 2

3 Neutropenic fever requires hospitalization $5,425 $4,882 $5,967 Gamma (29) Hospitalization for nausea and vomiting $6,160 $5,544 $6,776 Gamma (29) Hospitalization for other AE $6,160 $5,544 $6,776 Gamma (29) Cost per month secondary treatment and terminal care $13,177 $11,859 $14,495 Gamma (24) Chest X-ray (cpt code: 71030) $45 $41 $50 Gamma (29) Chest Ct scan: (Cpt code: 71260) $309 $278 $340 Gamma (29) MRI (cpt code: 71551) $513 $461 $564 Gamma (29) EGFR mutation testing $850 $765 $935 Gamma (31) Figure 2: EGFR mutation testing cost-effectiveness acceptability curve. EGFR testing (mutation)/bsc: Erlotinib for patients with an EGFR mutation and best supportive care for patients without an EGFR mutation; EGFR testing (mutation)/pem: Erlotinib for patients with an EGFR mutation and pemetrexed for patients without an EGFR mutation. 3

4 2) ERCC1 testing in early stage NSCLC (Stage I and Stage II) Figure 3: Stage 1 ERCC1 Model Schematic *All patients are followed until death Figure 4: Stage 2 ERCC1 Model Decision Tree *All patients are followed until death ERCC1 is a DNA-repair protein that has been shown to be associated with response and overall survival in patients with early-stage NSCLC treated with platinum-based regimens.(32) Based on these findings, it is plausible that ERCC1 expression testing could be used to inform adjuvant chemotherapy decisions and improve comparative health outcomes relative to one size fits all approaches based on disease stage. However, considerable uncertainty remains as to the clinical utility of ERCC1-based strategies relative to standard care strategies. To quantitatively assess this uncertainty, and to examine the potential economic value that could be created by additional study of ERCC1 expression testing, value of information models were created to separately examine Stage I and II disease. Each model uses decision-analytic modeling methods to examine the given patient population over a lifetime horizon from a U.S. societal perspective. The decision structure underlying each model is illustrated above (Figures 4 and 5). The models include probabilistic sensitivity analysis functionality, which enables probabilistic calculation of the net monetary benefit of ERCC1 testing and standard care strategies. Based on these probabilistic outcomes, the per-patient expected value of perfect information (EVPI) was calculated at a willingness-to-pay threshold of $150,000 per quality-adjusted life year (QALY). The 10-year discounted affected population was calculated based on SEER incidence estimates, and utilized to calculate the population EVPI. 4

5 The models were primarily informed by a retrospective analysis of patients from the International Adjuvant Lung Cancer Trial.(32) This study demonstrated that in patients treated with platinum-based chemotherapy, those with high levels of ERCC1 expression have less favorable overall survival prognosis relative to patients with low levels of ERCC1 expression.(32) Additionally, this study demonstrated the predictive value of ERCC1 expression level, with nonsignificant differences in survival in treated vs. untreated high ERCC1 patients (HR: 1.14, ), but significant increases in survival in treated vs. untreated low ERCC1 patients (HR: 0.65, ). Because the model compares ERCC1 expression testing outcomes with standard care outcomes, model standard care model inputs were derived from a meta-analysis of early-stage NSCLC treated (with cisplatin-based chemotherapy) and untreated patients that was stratified by disease stage.(33) This study demonstrated that patients with Stage I disease did not derive significant benefit from adjuvant chemotherapy, with a 5-year disease-free survival hazard ratio of 0.95 (95% CI: 0.76 to 1.19).(33) Alternatively, patients with stage II disease were demonstrated to derive significant benefit from adjuvant chemotherapy, with a 5-year diseasefree survival hazard ratio of 0.69 (0.57 to 0.83).(33) Based on these, and similar findings, adjuvant chemotherapy is not typically recommended for Stage I patients, while it is typically recommended for Stage II patients.(34) These findings and treatment recommendations were used to inform the standard care strategies in the models. Health state utility values were derived from a study of NSCLC health states preferences in a community sample of 100 U.K. citizens conducted by Nafees et al. (2006), and from a study NSCLC health state preferences in a sample of 92 Australian lung cancer patients conducted by Manser et al. (2006).(23, 35) The Nafees et al. study utilized visual analogue scale (VAS) and standard gamble (SG) utility elicitation methods to examine participant s preferences for 19 NSCLC health states, while the Manser et al. study derived health state utility values from Australian SF-36 instrument. All drug costs were derived from manufacturer average sales price (ASP) and all procedure costs were derived from Medicare fee schedules. Average monthly NSCLC post-recurrence costs were obtained from previously published research.(36) The model is not intended to capture all expenditures that would typically be incurred in the treatment of early-stage NSCLC; rather, the costs included in the model represent major expenditures that are expected to vary across the strategies examined. The structure and calculations for the ERCC1 value of information decision-analytic models rely on several assumptions. First, the models assume that the ratio of ERCC1 stratum-specific survival to overall survival observed in the study by Olaussen et al. holds when applied to the cohort examined by Douillard et al.(32, 33) This assumption was necessary to create a consistent cohort to be evaluated in the ERCC1 testing and standard care strategies, as Olaussen et al. did not examine outcomes stratified by both ERCC1 expression level and stage. Second, the models assume that all patients follow their indicated clinical pathway, with ERCC1 positive and stage I patients not receiving adjuvant chemotherapy, and ERCC1 negative and 5

6 stage II patients receiving adjuvant chemotherapy. Third, the value of information calculations assume that ERCC1 testing would be utilized over a 10-year time horizon, if indicated as the more favorable strategy. Lastly, affected population calculations assume that the proportion of NSCLC patients diagnosed at an early-stage will remain constant and at 2009 levels over the 10 year life of ERCC1 testing. Input tables and cost-effectiveness acceptability curves are provided below. Table 2: ERCC1 Stage I: Inputs Parameter Name Base Case Low High Distribution Source ERCC1 Positive Patients Mean Overall Survival, No Chemo Normal (32) Time From Recurrence to Death, No Chemo Normal ERCC1 Negative Patients Mean Overall Survival, Chemo Normal (32) Time From Recurrence to Death, Chemo Normal Stage I Patients Mean Overall Survival, Chemo Normal (33) Time From Recurrence to Death, Chemo Normal Mean Overall Survival, No Chemo Normal (33) Time From Recurrence to Death, No Chemo Normal ERCC1 Expression Test Result Proportion Testing ERCC1 Positive 44.6% 40.1% 49.0% Beta (32) Chemotherapy Grade 3/4 Adverse Events Any Grade 3/4 Adverse Event Beta (37) Neutropenia 73.0% 65.7% 80.3% Beta (37) Febrile Neutropenia 7.0% 6.3% 7.7% Beta (37) Fatigue 15.0% 13.5% 16.5% Beta (37) Anorexia 10.0% 9.0% 11.0% Beta (37) Nausea 10.0% 9.0% 11.0% Beta (37) Vomiting 7.0% 6.3% 7.7% Beta (37) Anemia 7.0% 6.3% 7.7% Beta (37) Health State Utility Values No Evidence of Disease (No Chemotherapy) Beta (35) No Evidence of Disease on Chemotherapy Beta (35) No Evidence of Disease and Adverse Event on Chemotherapy Beta (23) Post Recurrence Until Death Beta (23) Costs (USD) Unit costs Cost of Biomarker test $ $ $ Gamma Cisplatin + Vinorelbine cost per cycle $ $ $ Gamma (38) Drug administration per cycle $ $ $ Gamma (38) Outpatient visit $69.25 $62.33 $76.18 Gamma (29) CT scan $ $ $ Gamma (29) Cost per month after recurrence $2, $1, $2, Gamma (36) Cost per month no recurrence at 5 years to death $ $ $ Gamma (39) Erythropoitin (40,000 Units) $ $ $ Gamma (26) Sub-Cutaneous Erythropoitin Injection (Per Injection) $51.43 $46.29 $56.57 Gamma (29) Cefepime (500 mg) $4.44 $3.99 $4.88 Gamma (26) Hospitalization, Febrile Neutropenia (Per Episode) $5, $4, $6, Gamma (29) Clindamycin (30g) $63.01 $56.71 $69.31 Gamma (26) Amoxycillin (500 mg) $0.60 $0.54 $0.66 Gamma (26) Resource use Cycles of chemotherapy Normal (38) Other Proportion of febrile neutropenia cases hospitalized Beta Assumption Discount rate 3% 0% 5% 6

7 Figure 5: ERCC1 testing (stage I) cost-effectiveness acceptability curve 7

8 Table 3: ERCC1 Stage II: Inputs Parameter Name Deterministic Low High Distribution Source ERCC1 Positive Patients Mean Overall Survival, No Chemo Normal (32) Time From Recurrence to Death, No Chemo Normal ERCC1 Negative Patients Mean Overall Survival, Chemo Normal (32) Time From Recurrence to Death, Chemo Normal Stage II Patients Mean Overall Survival, Chemo Normal (33) Time From Recurrence to Death, Chemo Normal Mean Overall Survival, No Chemo Normal (33) Time From Recurrence to Death, No Chemo Normal ERCC1 Expression Level Prevalence Proportion Testing ERCC1 Positive 43.4% 39.1% 47.8% Beta (32) Proportion Testing ERCC1 Negative Chemotherapy Grade 3/4 Adverse Events Any Grade 3/4 Adverse Event (37) Neutropenia 73.0% 65.7% 80.3% Beta (37) Febrile Neutropenia 7.0% 6.3% 7.7% Beta (37) Fatigue 15.0% 13.5% 16.5% Beta (37) Anorexia 10.0% 9.0% 11.0% Beta (37) Nausea 10.0% 9.0% 11.0% Beta (37) Vomiting 7.0% 6.3% 7.7% Beta (37) Anemia 7.0% 6.3% 7.7% Beta (37) Health State Utility Values No Evidence of Disease (No Chemotherapy) Beta (35) No Evidence of Disease on Chemotherapy Beta (35) No Evidence of Disease and Adverse Event on Chemotherapy Beta (23) Post Recurrence Until Death Beta (23) Costs (USD) Unit costs Cost of Biomarker test $300 $270 $330 Gamma Cisplatin + Vinorelbine cost per cycle $292 $263 $321 Gamma (38) Drug administration per cycle $335 $302 $369 Gamma (38) Outpatient visit $69 $62 $76 Gamma (29) CT scan $308 $277 $339 Gamma (29) Cost per month after recurrence $2,146 $1,931 $2,361 Gamma (36) Cost per month no recurrence at 5 years to death $272 $244 $299 Gamma (39) Erythropoitin (40,000 Units) $489 $440 $538 Gamma (26) Sub-Cutaneous Erythropoitin Injection (Per Injection) $51 $46 $57 Gamma (29) Cefepime (500 mg) $4 $4 $5 Gamma (26) Hospitalization, Febrile Neutropenia (Per Episode) $5,530 $4,977 $6,083 Gamma (29) Clindamycin (30g) $63 $57 $69 Gamma (26) Amoxycillin (500 mg) $0.60 $0.54 $0.66 Gamma (26) Resource use Cycles of chemotherapy Normal (38) Other Proportion of febrile neutropenia cases hospitalized Beta Assumption Discount rate 3% 0% 5% Beta 8

9 Figure 6: ERCC1 testing (stage II) cost-effectiveness acceptability curve 9

10 3) CEA, CA 15-3 and CA Testing for Breast Cancer Recurrence Figure 7: Breast Cancer Recurrence Biomarker Model Schematic *All patients are followed until death Cancer Antigen (CA) 15-3, CA and Carcinoembryonic Antigen (CEA) are serum biomarkers that can be elevated in the bloodstream of patients with breast cancer recurrence. Detecting elevated levels of these biomarkers can lead to early identification of those patients with recurrent disease. A prospective clinical trial evaluated the CA assay in stage II and III breast cancer patients and found that it had a sensitivity of 57.7% for detecting recurrent breast cancer, and that test results become elevated an average of 5.3 months before recurrence was clinically established.(40) In theory, early treatment of recurrent disease (i.e. when tumor volume is lowest) can help eliminate disease. However, two randomized, prospective trials conducted failed to show a benefit in terms of improvement in either overall or disease-free survival.(41, 42) Despite the lack of evidence indicating benefit, it is estimated that more than 20% of patients receive at least one serum biomarker test after a diagnosis of breast cancer. Given improvements in medical care including imaging techniques since the 1980s, it is plausible that serial CEA, CA 15-3 and CA testing leading to earlier detection and treatment could produce improved outcomes compared to standard care. Nicolini et al. showed in a retrospective, non-randomized study that early treatment of patients on the basis of an increase in tumor marker levels during post-operative follow-up improved survival length compared to patients treated when metastatic disease was detected during routine followup.(43) There is considerable uncertainty surrounding the clinical utility of using serum biomarkers to detect and treat recurrent breast cancer earlier. To quantitatively, assess this uncertainty, and examine potential economic value that could be created by additional study in this area, a value of information model was created. The model uses decision-analytic modeling techniques to examine patients following termination of primary breast cancer therapy. The decision structure is outlined in Figure 8 with two arms: one involving testing with early treatment based on test results, and the other utilizing standard care. The model uses a ten-year horizon and is based on a five-year serial testing regimen. The model includes the ability to conduct probabilistic sensitivity analysis, enabling the probabilistic calculation of net monetary benefit of testing vs. no testing strategy. The per-patient expected value of perfect information (EVPI) was calculated at a willingness-to- 10

11 pay threshold of $150,000 per quality-adjusted life year (QALY). We assumed that the technology would be in effect for 10 years and used SEER incidence estimates for population disease estimates. We discounted the affected population over the 10-year affected horizon. The model was informed primarily by the prospective trial for the Truquant CA assay.(40) Data from this trial demonstrated that the CA assay had a sensitivity of 57.7% [38%-77%] and specificity of 97.9% [95.4%-100%]. Recurrence data was obtained from the Early Breast Cancer Trialists Collaborative Group meta-analysis (25% recurrence rate over 5 years).(44) Average survival times after recurrence were estimated based on Kaplan-Meier curves from the EBCTCG meta-analysis. Improved survival with early detection and treatment was assumed to be 2.4 years [1.9, 2.9] based on expert feedback from SWOG oncologists. Health state utility values were based on published values in breast cancer trials.(10) However, we assumed that recurrent breast cancer patients undergoing early chemotherapy would have a lower intensity of treatment and hence a higher utility value over the course of treatment (0.52 vs. 0.5). Costs were also estimated from published estimates; for longer surviving early detection patients, the cost was assumed to be proportional to the increase in life expectancy.(45) The structure and calculations for the breast cancer tumor markers value of information decision-analytic models relies on several assumptions. First, data for increased survival times based on earlier detection and therapy are not available in the literature and we utilized expert opinion to provide the necessary estimates. Second, confirmatory testing following elevated breast cancer tumor markers, through imaging (bone scan, CT, and PET) is assumed to be 100% effective in identifying the presence or absence of recurrence. Third, we assumed that all patients followed the indicated clinical pathway. Forth, a patient receiving an elevated tumor marker level but showing up as negative is assigned to a non-recurrence outcome in the 5-year testing horizon. Fifth, breast cancer tumor marker testing is utilized over 5 years following primary breast cancer therapy (following current clinical practice). Finally, the proportion of breast cancer and breast cancer recurrence in the population remains constant and our estimated levels over the 10-year life of tumor marker testing. 11

12 Table 4: Breast Cancer Tumor Markers: Inputs Parameter Name Base Case Low High Distribution Source Life Expectancy Life Expectancy (Recurrence caught early through biomarker detection) Metastatic (Years) Normal SWOG expert opinion Life Expectancy with Recurrence Metastasis (Years) Normal SWOG expert opinion Average time to recurrence (5 year window) (Years) Normal Calculation % Time after recurrence on chemotherapy 80.00% 72.00% 88.00% Beta Assumption Recurrence % Probability of recurrence in 5 years, Standard Surveillance 24.60% 23.60% 25.60% Beta (44) Probability of non-recurrence in 5 years, Standard Surveillance 75.40% (44) Biomarker test performance Sensitivity 57.70% 38.30% 77.10% Beta (40) Specificity 97.90% 95.40% % Beta (40) Health state utility values No Evidence of disease (No chemotherapy) Beta (46) Receiving Chemotherapy (Standard Arm) Beta (46) Receiving Chemotherapy (Early Detection) Beta Assumption Elevated Biomarker Beta Assumption Elevated Biomarker (Non-Recurrence) Beta Assumption Costs Biomarker Test $129 $116 $142 Lognormal (47) Metastatic Disease Monitoring (Annual) $17, $15,361 $18,775 Lognormal (48) Bone Scan $ $211 $258 Lognormal (29) CT Cost $ $298 $364 Lognormal (29) PET Cost $1, $1,350 $1,650 Lognormal (49) Annual Surveillance Costs $421 $379 $463 Lognormal (48) End of Life Care $30,000 $27,000 $33,000 Lognormal (48) Adjuvant Chemotherapy $15,123 $13,611 $16,635 Lognormal (48) Adverse Event Costs (Major) $15,700 $14,130 $17,270 Lognormal (48) Adverse Event Costs (Minor) $2,400 $2,160 $2,640 Lognormal (48) Progressive Metastatic Disease $4,680 $4,212 $5,148 Lognormal (48) Biomarker and Confirmation Parameters % Population with +Biomarker receiving MRI 100% 90% 100% Beta Assumption % Population with +Biomarker receiving CT 66% 60% 73% Beta Assumption % Population with +Biomarker receiving PET 33% 30% 36% Beta Assumption Average number of tests received by non-recurring patients Lognormal SWOG expert opinion Average number of tests in recurring patients, + result Lognormal SWOG expert opinion Average number of tests in recurring patients, - result Lognormal SWOG expert opinion 12

13 Figure 8: Breast Cancer Tumor Markers Cost-effectiveness Acceptability Curve 13

14 Appendix B: Value of Information Analysis Briefing Paper (online only appendix) Summary Value of Information (VOI) methods can estimate the benefit of investing in additional studies to determine whether a particular test or treatment should be brought to clinical practice. VOI methods are particularly useful when the approach to patient care is very uncertain and when the consequences of making the wrong choice are large, both in clinical and economic terms. VOI can focus research prioritization discussions by highlighting critical information and areas of great uncertainty in the clinical management pathway. In this way, VOI is a tool that can help decision makers maximize the impact of research portfolios on medical care and human health. The CANCERGEN team is conducting VOI analyses for some of the top priority cancer genomic tests identified by the External Stakeholder Advisory Group (ESAG). We are interested in exploring whether VOI analyses, in addition to information regarding trial feasibility within the SWOG infrastructure, can help guide selection of specific genomic test topics for high quality comparative effectiveness trials. Value of Information Analysis applies methods from economic theory and decision analysis to evaluate the value of research to society and individuals. Background and Rationale for Value of Research Analysis In 2007, a report summarizing the results of a Phase II trial evaluating bevacizumab for women with advanced breast cancer that had progressed after earlier therapy was published in New England Journal of Medicine. The study found that progression-free survival was extended in bevacizumab-treated patients, but overall survival and quality of life were no different between treated and untreated patients. The FDA approved bevacizumab based on these data. As a clinician or policy maker, should you offer this drug to your patients? Or should you wait for another trial that provides more information about the benefits and risks of bevacizumab for this population? Putting it a different way, how much value would an additional trial provide? This is an issue commonly faced in healthcare. An emerging field in economics Value of Information Analysis (sometimes referred to as the value of research; hereafter referred to as VOI) applies methods from economic theory and decision analysis to evaluate the value of research to society and individuals. Specifically, we want to avoid (1) not adopting a technology that has good health value for expenditure; (2) adopting a test or treatment that increases costs but has no benefit, or harms that exceed 14

15 benefit; (3) adopting a test or treatment with poor value for expenditure; that is, for which greater benefit could be gained elsewhere for the same investment. With proper adaptation, we believe that the VOI method will aid researchers in selecting and designing studies so that the health benefit of the information gained is maximized for the research investment. The Theory Value of perfect information If decision makers had perfect information about the risks, benefits, and cost impact of a particular technology, they would always be able to make correct choices regarding the use of the technology. The right drug or test would be given to the right patient at the right time. The difference between the value of having perfect information and the value of current information (with attendant uncertainty) is known as the value of perfect information (VPI). VPI can be interpreted as the maximum amount we would be willing to spend to learn about the benefits and risks of a particular test or therapy. Although having perfect information is not possible, VPI is the upper level on the value of further research and as such can serve as an initial threshold to support research funding decisions. Value of information from a study Whereas the VPI is an upper level on the returns from all possible further research, the value of study information (VSI) represents the expected value of research before conducting a single trial. The expected benefits of research can be compared with the expected costs of carrying out that research, i.e. the costs of sampling. If the cost of new research is less than the expected value of information from the study (EVSI) the trial is worth the expense. The VSI can be regarded as the societal pay-off to research, and can be calculated for a range of samples, sizes and alternative designs. The Approach VOI utilizes simulation modeling to derive estimates of VPI and VSI. Models use data from existing studies, including measures of effectiveness, safety, cost, and quality of life. Importantly, the life expectancy of the technology and number of persons who are expected to use the technology over that time must be estimated. Also, a variety of willingness to pay thresholds to achieve a given unit of health benefit (e.g., an extra year of life or quality-adjusted life year) can be explored through this approach, which are key parameters for informing decisions to conduct additional research. Example: Value of Information Provided by the National Emphysema Treatment Trial 15

16 Lung volume reduction surgery (LVRS) is a surgical approach to for persons with severe emphysema.(50) In 1995, Cooper et al reported the results of a small, uncontrolled study evaluating a modification of the original volume reduction operation that seemed to preserve the benefits while substantially reducing the mortality and morbidity related to the procedure. Less than 18 months from the initial reports, over 1,200 LVRS procedures had been performed on Medicare beneficiaries nationwide.(51) The clinical and economic implications of adopting LVRS were tremendous. Emphysema affects approximately 3.6 million Americans(52) and reimbursement for LVRS procedures performed on Medicare beneficiaries exceeds $31,000 per procedure.(51, 53) CMS stated that, the current data do not permit a logical and scientifically defensible conclusion regarding the risks and benefits of LVRS. (54) CMS, National Heart, Lung, and Blood Institute (NHLBI), and AHRQ sponsored a randomized, controlled trial of LVRS compared to standard medical therapy for patients with severe emphysema: the National Emphysema Treatment Trial (NETT). The CMS stated that it would only pay for the procedure for patients who agreed to participate in the trial. Results would be used to inform CMS s national coverage decision for LVRS.(55) The trial showed that patients in the LVRS arm had a modest quality of life gain compared to medical therapy patients and concerns about uncommon but severe adverse effects from surgery,(56) has resulted in far fewer emphysema patients choosing surgery than was originally expected.(57) Was it worth investing in the NETT? One could argue that the information gained from the trial helped inform patients such that their choices today are different from what they might have made before the information was available. We applied the VOI method post-hoc in an effort to assess the practical applicability of the method and to determine the value of information gained from the NETT.(58)The cost of the NETT including investigator time, patient time, and direct expenditures for medical care was approximately $59 million, about 20% more than the funding agencies anticipated when they agreed to sponsor the trial. The results of the VOI analysis were as follows: VOI results for LVRS at willingness to pay (WTP) thresholds of $50,000 and $100,000 per QALY Parameter WTP = $50,000/QALY WTP = $100,000/QALY Value of Perfect Information $31 B $607 B Value of Study Information $3.5 B $7.3 B These results suggest that, with a value of information of $3.5-$7.3 billion, the NETT was a very good research investment compared to the cost of covering the surgery, which was$59m. Conclusion 16

17 For each of the genomic test topics selected by the ESAG, the VOI approach will capture what is already known about their respective clinical and economic impacts, while more fully characterizing the uncertainties surrounding these estimates in the form of a decision problem. This type of analytic framework will 1.) help focus the discussion on those parameters which are most critical for understanding how use of the test affects clinical management of the oncology patient, 2.) promote an informed debate of the relative merits of future research investments to study these issues, and 3.) also help decide which endpoints should be included in a proposed comparative effectiveness study. However, professional judgments will inevitably be required and VOI models should be viewed primarily as a tool to support more explicit decisionmaking and to use scarce resources more efficiently. 17