Report. for the Pharmaceutical Oncology Initiative Group (POI)

Transcription

1 Report for the Pharmaceutical Oncology Initiative Group (POI) Assessment and appraisal of oncology medicines: does NICE s approach include all relevant elements? What can be learnt from international HTA experiences? By Martina Garau, Koonal Shah, Adrian Towse, Qing Wang (OHE Consulting); Mike Drummond, Anne Mason (University of York) Submitted in January 2009 Submitted by: OHE Consulting 12 Whitehall London SW1A 2DY For further information please contact; Professor Adrian Towse Director Tel: +44 (0) atowse@ohe.org 1

2 Acknowledgements This report was undertaken as part of a study commissioned by the Pharmaceutical Oncology Initiative Group (POI). We are grateful to Nancy Devlin, Mark Sculpher and two anonymous reviewers for their comments on an earlier draft of this report. The usual disclaimer applies. 2

3 Contents EXECUTIVE SUMMARY INTRODUCTION TERMS OF REFERENCE CONTEXT METHOD STRUCTURE OF THE REPORT NICE S APPROACH INTRODUCTION NICE S CURRENT APPROACH RECENT DEVELOPMENTS OF NICE S APPROACH NICE S PROCESSES AND DECISION OUTCOMES FOR CANCER MEDICINES A COMPARISON WITH OTHER HTA BODIES IN THE UK AND INTERNATIONALLY Which processes are used to make coverage decisions? What decisions on cancer medicines have been reached by HTA bodies under these different processes? SUMMARY ISSUES IN USING THE QALY WHEN ASSESSING CANCER MEDICINES HEALTH GAINS INTRODUCTION SENSITIVITY OF EQ-5D AS A DESCRIPTIVE SYSTEM How can the lack of sensitivity of EQ-5D be addressed? VALUING HEALTH STATES: THE ASSUMPTION OF CPT UNDERLYING TTO How can CPT issue be addressed? PATIENT VERSUS GENERAL PUBLIC PREFERENCES Is it possible to incorporate patient s value in the QALY calculation? SUMMARY ISSUES IN USING THE QALY WHEN APPRAISING CANCER MEDICINES. THE ROLE OF SEVERITY AND OTHER SOCIAL VALUE JUDGEMENTS INTRODUCTION SOCIAL VALUE JUDGMENTS SEVERITY - THEORY SEVERITY EMPIRICAL EVIDENCE Question framing and study design issues Post-treatment severity and realisation of potential Double-counting INCORPORATING SEVERITY AND OTHER SOCIAL VALUE JUDGEMENTS INTO HTA Deliberative processes Multi-criteria decision analysis (MCDA) INTERNATIONAL COMPARISON SUMMARY GENERATING CLINICAL EVIDENCE AND OTHER ISSUES CONCERNING HTA OF ONCOLOGY MEDICINES. RESULTS FROM A SURVEY OF EXPERTS INTRODUCTION GENERATING CLINICAL EVIDENCE COLLECTING PATIENT REPORTED QOL DATA TRANSLATING CLINICAL OUTCOMES INTO BROADER HEALTH OUTCOMES FOR HTA EQ-5D IN CANCER APPRAISING CANCER MEDICINES POTENTIAL IMPROVEMENTS TO NICE S APPROACH

4 5.8 SUMMARY POSSIBLE WAYS FORWARD SUMMARY OF MAIN FINDINGS CHANGES IN THE QALY DESCRIPTIVE SYSTEM CHANGES IN THE WEIGHTING OF QALYS (FROM AN INDIVIDUAL PERSPECTIVE) WEIGHTING OF QALYS (FROM A SOCIETAL PERSPECTIVE). HOW TO INCORPORATE SOCIAL VALUE JUDGEMENTS INTO HTA ATTENDING TO DATA DEFICIENCIES THE RECENT NICE SUPPLEMENTARY ADVICE ON LIFE-EXTENDING, END OF LIFE TREATMENTS...54 REFERENCES

5 Executive Summary Recently issued NICE provisional deliberations on a group of cancer treatments led to protests by patients and the general public and have reinforced perceptions of lower access to cancer medicines in the UK as compared to other countries. Evidence provided in the Cancer Reform Strategy confirms that this the case. It states that usage of new cancer medicines in the UK is about 60% of that in other major European countries. The reasons for this disparity are complex and not fully understood. However, the recent Richards Review speculated that health technology assessment (HTA) methodology, and the fact that NICE has declined to recommend some drugs for use in the NHS which are available elsewhere, may be a contributory factor. Our report sets out to identify whether the current approach by NICE is an influencing factor on access to cancer medicines in this setting and whether there is rationale and scope for bringing about any changes in the HTA approach. The purpose of our study is to: o review the current status of NICE s methods for cancer medicines and compare them with other HTA bodies in the UK and internationally; o identify potential issues concerning the assessment of health gains of oncology medicines using the QALY and factors other than cost per QALY that are deemed important in the context of health care resource allocation (based on a literature review); o discuss the theoretical and practical issues related to the integration of Social Value Judgments (SVJs) in the decision making processes; o provide an overview of the challenges in assessing and appraising cancer therapy (based on practical experience); o develop appropriate options for bringing about any changes within current HTA approaches. As part of the study, the research team conducted detailed reviews of the published literature and consulted with key experts directly involved in the work of UK HTA bodies. The current NICE methods indicate the QALY as the preferred measure of health effects. Other HTA bodies operating in other jurisdictions also have a preference for the QALY but a larger proportion of health care bodies using HTA are less prescriptive than NICE over which type of economic analysis to employ. However, other bodies using HTA have not necessarily approved more cancer drugs than has NICE. Our comparison of coverage decisions of a sample of 10 cancer drugs in five countries using HTA reveals that NICE had the highest approval rate, albeit with only four out of the 10 cancer treatments approved for use in the NHS without restrictions. It is important to note that the analysis includes countries relying on cost effectiveness evidence to make 5

6 coverage decisions and does not therefore include four of the five major European countries. Our literature review points out that the QALY has a number of limitations, mainly due to the way it is structured. In particular, one of the deficiencies of the EQ-5D, which is the generic preference-weighted health-related quality of life measure preferred by NICE, is that it may not be sensitive enough to pick up changes in the health status of cancer patients (e.g. it has no domain to capture changes in vitality or energy). Recommendation: the EQ-5D (EuroQol) group is considering whether the instrument should be modified in this regard. In the meantime, it might be advisable to explore the use of the SF-6D in the evaluation of cancer therapies, in order to assess whether it is more sensitive to change and to try to make better use of cancer specific measures of health related quality of life (HRQoL). On the valuations of health states generated by the EQ-5D questionnaire, which are based on general public preferences, there are issues related to the potential divergence between the values of patients, who directly experience the health state, and of the general public, who have been asked to make choices over hypothetical heath states which they might find difficult to fully understand. Cancer patients preferences may be driven by specific characteristics of the disease. If someone has been told that they only have six months to live, gaining an extra two months might be worth a lot more to them than would a two-month gain if they had five years to live (over and above any discounting arising from the timing of future health effects.) As currently calculated a QALY valuation of health effects would not reflect this. Recommendation: more research is required to test both whether cancer patients (and members of the general public if asked to think about being in this position) attach more value to survival gains when their life expectancy is short and to find the point up to which they are willing to trade quality of life for survival benefits. This would provide a weighting system that better reflect individual utility of health states. It is clear from the literature that there are other aspects of patient experience (such as the extent to which they do or do not adapt to conditions or value other aspects of end of life care) which may not be routinely reflected in valuations of these health states undertaken by the general public. We do not think it appropriate at this stage to depart from using the valuations of the general public. However, more research is required in this area. From a resource allocation perspective, under the current approach all QALYs are deemed to be of equal social value. However, our literature review indicates that there is societal willingness to give priority to the worse-off (people suffering from more severe illness), even if this involves a sacrifice in aggregate health gains. We discuss the ways through which social value judgements such as severity can be integrated in health resource decision making (i.e. a formal QALY weighting system or more flexible approaches such as a deliberative process). Studies commissioned by NICE through NCCHRM have made a significant contribution to the ongoing debate but 6

7 have not identified an exact estimation of the health gain weights which might be used by NICE in the context of its QALY framework. Recommendation: the results of the empirical studies supporting the principle of a societal willingness to give priority to severity should be considered in the appraisal as part of a deliberative process. However, efforts need to be made to ensure the Committee is aware of the assumptions it is making and to achieve a degree of transparency in reporting. One option might be the introduction of multi-criteria decision analysis (MCDA) to assist NICE Appraisal Committees in their deliberative process, which provide a framework for explicitly trading off various priority-setting concerns. Our interview programme points out that there is uncertainty surrounding projections from progression-free survival (PFS) to overall survival (OS). The principal reason for this uncertainty is the difference between the approaches of licensing bodies and of HTA bodies, which reflects their respective roles. Recommendation: NICE and the Medicines and Healthcare products Regulatory Agency (MHRA) could hold discussions with companies to increase joint understanding as to those instances in which PFS is a good predictor of OS and those in which it is not. Advice from NICE on these matters could be reviewed with companies through the consultancy process that NICE has recently established. However, as reliance upon these data is common amongst reimbursement agencies across the developed world, a coordinated and international approach may be the best way to tackle these issues. The recently published NICE advice on Appraising life-extending, end of life treatments makes changes that are in line with the main findings of our study. It indicates that in certain circumstances end of life treatments with a cost per QALY exceeding the standard threshold can be recommended for use. But the lack of detailed rationale for the changes and for the way clinical and cost effectiveness evidence will be appraised by NICE (e.g. mechanisms to ensure transparency) may make it difficult to apply the changes in practice. We note that NICE includes a suggestion for further methodological research to provide more robust basis for the application of the advice. Our report indicates areas where such research could usefully be undertaken. 7

8 1 Introduction 1.1 Terms of reference The Cancer Reform Strategy (Department of Health, 2007) states that usage of new cancer medicines in the UK is about 60% of that in other major European countries. The Pharmaceutical Oncology Initiative group (POI), a group of pharmaceutical companies supplying cancer treatments to the NHS, is concerned about the lower uptake and access to cancer medicines for UK patients as compared to their counterparts in other European countries. In particular it is concerned about the approach currently adopted by the National Institute for Health and Clinical Excellence (NICE) to assess and appraise new oncology medicines. The existing methods used by NICE assume NICE s primary concern is to maximise the health gain as measured by QALYs within the given NHS and public Personal Social Services (PSS) budget. Such an approach on its own arguably takes no account of any equity/efficiency trade-off, rule of rescue or other preferences that may exist within society when allocating scarce health care resources. Although the Appraisal Committee can take other factors into account, there is, as a consequence, a perception on the part of many including the POI that the NICE process in its current form often does not take into account all key factors of importance to society when evaluating the value of new oncology therapies. OHE Consulting has been commissioned by POI to: 1 set out the current approach of NICE to assessing and appraising cancer medicines; 2 evaluate the relevance/appropriateness of the current NICE approach in the case of oncology drugs, identifying relevant issues particularly in relation to methods; 3 compare NICE s approach and performance with other HTA bodies in the UK and internationally; 4 identify the most appropriate next steps towards bringing about any changes, including further research, within NICE s current framework. 1.2 Context Cancer medicines have been subject to significant publicity in the UK in the course of 2008, in particular in relation to patients who have been denied oncology medicines due to NICE decisions (e.g. protests by patients about NICE s provisional rejection of treatments for renal cell carcinoma) and who have, in some cases paid for drugs and other NHS treatment out-of-pocket, and in others not been able to obtain drugs that would help them. This can be seen as a challenge to the methods used by NICE from patients and from the broader community, and has led, among other factors, to the review by Professor Richards (Richards, 2008) and to NICE s consultation on its methods for reviewing end of life treatments (NICE, 2008c). Hence the focus in this 8

9 report as to whether NICE s normal approach is appropriate for assessing oncology drugs and, if not, how possible changes could be researched and introduced. OHE Consulting is also aware that a number of flexible access schemes for particular cancer medicines have been proposed by manufacturers to the DH and some of them implemented within the NHS with the purpose of improving access to cancer medicines in a way that NICE regards as providing value for money within its current approach. These primarily seek to tackle NICE s concerns about the effective cost to the NHS of the drug or about the quality of evidence of patient benefit or both. The new PPRS sets out arrangements for Patient Access Schemes and for Flexible Pricing, both of which have potential application to cancer medicines and which may enable any concerns that NICE may have in future appraisals to be addressed (Department of Health, 2008). However, the objective of our study is not to look at routes that might get around NICE s decisions using its existing approach but to investigate whether NICE s existing approach captures and values appropriately all of the health related benefits that are important to patients and valued by society. Of course issues of the cost of a therapy to the NHS and the quality of evidence will inevitably arise within any framework NICE were to adopt. NICE s role is to consider whether technologies referred to it represent value-for-money for the NHS given other uses that can be made of the NHS budget. Our report does not take issue with NICE s role, but seeks to consider whether the current approach is correctly valuing cancer therapies and if it may not be, then how concerns can be addressed. On this basis, our study focuses on the following main questions: How well does the QALY, and in particular EQ-5D pick up cancer patients health gain from a treatment? How can QALY values for cancer treatments be put in the context of societal preferences for treating cancer patients? How can NICE s cost-per-qaly-based decision making processes better take into account factors including health gains that may not be reflected in the QALY and societal preferences? During the preparation of our final report NICE issued (on 2 nd January 2009) supplementary advice to its Appraisal Committees on Appraising life-extending, end of life treatments (NICE, 2009). We refer to this in Chapter 2 and at the end of the report. We can note however that it does not contain discussion of the rationale for the approach, indeed it suggests that exploration of the methodological issues is needed. To that extent our review complements NICE s revised advice. 1.3 Method Our work was broken down into the following tasks: Searching and reviewing the published literature which examines the following: 9

10 1. the methodological issues concerning the measurement of health outcome of oncology medicines using the QALY; 2. the potential limitations of cost per QALY-driven decision making processes; 3. the theoretical and practical issues related to the integration of Social Value Judgments (SVJs) in the decision making process of NICE and (by comparison) of HTA bodies operating in other countries. Searching and reviewing the published literature on the appraisal methods currently employed for cancer medicines by international HTA bodies; Reviewing the outcomes of past NICE decisions on cancer medicines; Collecting and analysing evidence on reimbursment decisions, available in the public domain, issued by HTA bodies operating in other countries and comparing them with those of NICE; Conducting interviews with key stakeholders including clinicians, patient groups, health economists and decisions makers in the oncology area to identify the main challenges in the evaluation of cancer medicines based on their practical experience. 1.4 Structure of the report This report is set out as follows: Section 2 provides a review of current NICE methods and of its recommendations on the use of oncology drugs within the NHS, and a comparative analysis with other HTA bodies in the UK and internationally; Section 3 discusses the potential limitations of using QALYs in oncology setting out the issues identified in the methodological literature review; Section 4 presents the results of the literature review on social value judgments, with a particular focus on severity, identifying approaches to incorporating them into NICE s Appraisal processes; Section 5 discusses the issues identified in the interview programme, including those concerning the generation of better clinical evidence of treatment effect; Section 6 discusses how manufacturers and other key stakeholders can engage with NICE in order to address the issues identified and analysed in the previous sections in order to ensure that any revised HTA process incorporates all relevant elements of health gain and of societal preference; Appendices A, B and C give details of the literature reviews. In this report we use the terms cancer and oncology medicines interchangeably. 10

11 2 NICE s approach 2.1 Introduction The aim of this chapter is to provide an overview of the current status of the NICE approach to appraise health technologies including oncology medicines and to compare it with the approach adopted in other jurisdictions to make decisions on the provision of new treatments within the national health system. The chapter starts with an analysis of the NICE processes and methods, as indicated in the current Methods Guide, and a brief discussion of the recently issued supplementary advice on end of life treatments. It then presents an updated analysis of the NICE appraisal outcomes for oncology treatments. Section 2.4 provides a comparison between NICE s approach and that of other HTA bodies in the UK and internationally and between coverage decisions for a selected sample of cancer drugs made in five jurisdictions, including England and Wales. The final section summarises the main findings reported in the chapter. 2.2 NICE s current approach The National Institute for Health and Clinical Excellence (NICE) is a Special Health Authority operating within the UK National Health Service (NHS). It was established in April 1999 as a result of increasing concern about unequal access to health care within the country ( post code prescribing ) and growing acknowledgement of the disparity between the demand for health care, driven in part by the availability of new technologies, and the available resources. NICE establishes whether or not individual treatments provide good value for money for the NHS and produces national guidance ( Technology Appraisals, (TAs)) on the appropriate use of selected health technologies within England and Wales. NICE s approach consists of an assessment of the evidence on clinical benefits, financial costs and other relevant consequences of the technology under investigation. This does not include making specific policy recommendations. It reviews costs and effects but does not address the question as to whether the cost is worth paying for given the effect; an appraisal which uses the information on effectiveness and cost effectiveness provided by the assessment to formulate policy decisions. It adds context-specific judgement on the applicability of the evidence, the feasibility and impact of alternative options, relative priorities, and wider social and ethical aspects (Woods, 2002). In other words it considers whether the evidence on costs and effects suggests that use of the technology represents value for money. 11

12 The Institute does not collect directly evidence but relies on assessments submitted by key stakeholders, including manufacturers or sponsors. An independent centre normally led by an academic group reviews and assesses the evidence. The WHO report on NICE s technology appraisals programme (Hill et al., 2003) cited the intensive participation of different stakeholders and the inclusiveness of the approaches taken as one of NICE s achievements which are particularly valuable. NICE has two processes to appraise medicines: the Multiple Technology Appraisal (MTA) and the Single Technology Appraisal (STA). The main differences between the two are the duration of the process, which is shorter in the case of STAs, and the development of the evidence base, which in the case of the STA relies mainly on submissions from manufacturers. The STA was introduced in 2006 to address increasing concerns about the delay between the launch of and the decision of NICE about the use of new drugs within the NHS. A recent review of NICE TAs confirms that STA processes produce guidance more quickly. The average duration of STAs in cancer is 12.8 months as compared to 20.7 months for MTAs (Barham, 2008). However, a key challenge in NICE processes remains that of balancing the quality of the review and the breadth of the evidence base available at the time of launch with the need to provide fast guidance to the NHS. The Institute s recommendations are primarily informed by evidence on clinical and cost effectiveness, although other factors could in principle be considered when reaching decisions. The updated Methods Guide, published in June this year (NICE, 2008a), confirms that the health gains are primarily measured using the qualityadjusted-life-year (QALY) and that the deliberations of NICE s Appraisal Committees are governed by a set of decision making criteria which begins with the cost per QALY ratio but takes other factors into account. Technologies with incremental cost effectiveness ratios (ICERs) below 20,000 are usually accepted but can in principle be rejected as a result of other factors (e.g. if uncertainty is high). As the ratio increases above 20,000 per QALY other criteria may justify the use of a technology. Beyond 30,000 per QALY such criteria would be required to be relevant and to met a sufficient degree for NICE to approve a medicine. The other criteria mentioned in the guidance are: The degree of certainty around the ICER. In particular, the Committee will be more cautious about recommending a technology when they are less certain about the ICERs presented. Whether there are strong reasons to indicate that the assessment of the change in HRQL has been inadequately captured, and may therefore misrepresent the health utility gained. The innovative nature of the technology, specifically if the innovation adds demonstrable and distinctive benefits of a substantial nature which may not have been adequately captured in the QALY measure (NICE, 2008a). NICE s decision rule can be represented as a probability curve, in the sense that if the drug is associated with a cost per QALY below 20,000, the probability of acceptance tends to be high, but as the cost per QALY ratio increases the probability of acceptance decreases (Rawlins and Culyer, 2004). 12

13 The cost per QALY threshold plays a crucial role as it is seen as a way of incorporating the resource allocation problem faced by the NHS (to improve health outcomes given the available limited NHS budget) into NICE s decision making. By comparing the incremental cost per QALY of new interventions against the threshold, the Institute takes into account the opportunity cost of implementing these new interventions, i.e. the value of health benefits displaced when new treatments are recommended by NICE. Concerns have been raised, however, as to how additional factors can be incorporated into the NICE decision-making process, given the broad range of criteria that may be relevant and the issues related to the definition of concepts such as innovation in the context of new health technologies. 2.3 Recent developments of NICE s approach In the recently issued supplementary guidance NICE indicates a list of circumstances under which it may be appropriate for the Institute to approve new treatments with a cost per QALY exceeding 30,000 per QALY (NICE, 2009). The new advice arguably reflects the concerns of patients and the public about the potential denial of certain oncology treatments, and contributes to finding a solution to the issue of copayments arising from the review by Professor Mike Richards (Richards, 2008). Criteria that should be considered alongside a cost per QALY ratio above the cost effectiveness threshold include: The treatment is indicated for patients with a short life expectancy, normally less than 24 months and; There is sufficient evidence to indicate that the treatment offers an extension to life, normally of at least an additional 3 months, compared to current NHS treatment, and; No alternative treatment with comparable benefits is available through the NHS, and; The treatment is licensed or otherwise indicated, for small patient populations. (NICE, 2009) In practice this means that when a NICE Appraisal Committee makes its decisions more weight may be attached to health gains generated by end of life treatments targeting relatively small populations, criteria that are likely to include mainly rarer cancers. The guidance represents a clear intention by NICE to incorporate considerations of severity more formally in the decision making process. 2.4 NICE s processes and decision outcomes for cancer medicines A large number of TAs has been for cancer interventions. This might reflect the increasing number of new medicines developed to treat different types of cancer, including those with no alternative therapy available. In addition, with the purpose of 13

14 improving cancer outcomes and, more specifically, to ensure fast access to high quality treatment for cancer, the Cancer Reform Strategy established that all new cancer drugs and significant new licensed indications would be referred to NICE (Department of Health, 2007). We also note that the STA process was introduced for new medicines, particularly for life-threatening conditions such as cancer treatments (NICE, 2005). We reviewed the TAs issued by NICE since its creation on cancer medicines with a particular focus on the trend in decision outcomes over time. More details on the extracted data are available in Appendix D. We focused on TAs on cancer medicines, both STAs and MTAs, published in the period 2001 to end June 2008, excluding other non-drug technologies such as screening or surgical interventions. Each oncology medicine, identified within each TA, was matched with an overall decision classified as: recommended, when the technology was recommended by NICE as preferred therapy or as an option; restricted, when the technology was recommended only for a sub-group within the licensed indication; or not recommended. It is important to note that the middle item (i.e. restricted) includes only major limitations on the use of a new medicine such as recommendation for a subgroup out of the whole licensed patient population. Minor restrictions, such as use confined to appropriate specialists were classified as positive recommendations, as they reflect clinical practice rather that restrictions on the medicine s use. Following the method used by Mason and Drummond (2007a), we also used another unit of analysis whereby each TA was grouped as: positive, if all included drugs were recommended; negative, if all included drugs were not recommended; and mixed, if there was a mix of advice including recommended, not recommended and restricted. This was done to consider drug recommendations within the same TA as one observation, due to their possible correlation. Up to June 2008, NICE issued 37 sets of TA guidance on cancer medicines out of a total of 150 TAs (i.e. nearly 25 %). Data confirms the focus of NICE STAs on cancer medicines: more than half of STAs completed by June 2008 focused on cancer medicines (10 out of 19). We excluded 8 of the 37 TAs on the ground that they have been reviewed subsequently and so are no longer valid and considered the remaining 29 TAs. We note that in addition to the 29 sets of guidance there were 4 other TAs for cancer which were terminated because no evidence submission was received from the manufacturer or sponsor of the technology (TAs 147, 148, 149, and 150). These were not included in the decision outcome analysis. However, it is important to note that these TAs create a precedent in the NICE process where, because of the lack of evidence, NICE did not feel itself to be in a position to make a decision and so did not recommend whether or not these products can be used by the NHS. We assume that these products will have a position similar to other products which have not been appraised yet or have been turned down by NICE. Often local Primary Care Trusts (PCTs) only allow these to be used on a case-by-case basis (Richards, 2008). 14

15 The 29 appraisals we considered covered 41 technologies in total. Overall, 23 of the 41 (56%) were recommended; 12 were recommended for restricted use (29%); and six medicines were wholly rejected (15%). Figure 1 shows the trend of the decision outcome over time. We note a peak in the number of positive recommendations ( Yes ) in 2006 followed by a decrease and a peak in negative advice ( No ) in Appendix D provides a list of the considered TAs and the corresponding NICE recommendation. Figure 1 Count of NICE decisions by year and by drug No Restricted Yes Decisions per medicine per year * 2008 * it includes one yes with response based rebate scheme This trend was also reported in Mason and Drummond (2007a, 2007b) where the set of NICE TAs on cancer medicines published between 2000 and June 2006 is compared to those published between June 2006 and March The authors suggested that, although no formal statistical testing was possible due to limited data, in the first set of TAs cancer medicines fared quite well whilst in the second set the outcomes were not so favourable (Mason and Drummond (2007b). One of the possible explanations, which should be interpreted with caution, was the introduction of STA process. We compared the decision outcomes of cancer medicines which were appraised via a MTA process versus those which followed a STA process. Figure 2 shows the comparison where we considered the decision outcome drug by drug. 15

16 Figure 2 Count of NICE decisions by appraisal process and by drug Decision per medicine No Restricted Yes MTA STA* * it includes one yes with response based rebate scheme Figure 3 replicates a similar analysis considering NICE outcome decision by TAs rather then by drugs. 16

17 Figure 3 Count of NICE decisions by appraisal process and by TA decision per HTA negative mixed positive MTA STA* * it includes one yes with response based rebate scheme Although the proportion of negative decisions is much higher in the STA set of recommendations as compared to the MTA ones (30% versus 10% in the drug-bydrug analysis and 30% versus 5% in the TA-by-TA analysis), it is still difficult to reach definitive conclusions because of the small number of observations. A further complication is that restricted decisions are much more important in the case of MTAs than STAs and restrictions vary in their impact on the use of the drug given its licensed indication. As at June 2008, there were 24 TAs in development on oncology medicines. Only three of them, two of which are STAs, had reached a stage of the consultation where a provisional deliberation of a NICE Appraisal Committee was available (either as an Appraisal Consultation Document or as a Final Appraisal Determination). In all three cases, the preliminary decision of the Appraisal Committee was negative (i.e. lapatinib for breast cancer; bevacizumab, sorafenib, sunitinib and temsirolimus for renal cell carcinoma; erlotinib for lung cancer). In particular, in the case of renal cell carcinoma the key reason for these provisional refusals was the significantly high cost per QALY ratios associated with the medicines in question (NICE, 2008g). It should be noted that these decisions are not final; however they suggest that, before taking into account the potential impact of NICE s new supplementary guidance, the apparent trend towards more negative decisions may have been expected to continue. 2.5 A comparison with other HTA bodies in the UK and internationally Our review of international HTA bodies involved in coverage decisions focused on how reimbursement decisions are made in 24 countries, including England and Wales 17

18 (see Appendix C). The review found that 19 countries used economic evaluation to inform the reimbursement decision. Of these, most provide clear guidance on the perspective to be used (13/19), methods for handling uncertainty (13/19), the discount rate to be used (11/19), requirements for budgetary impact analysis (14/19) and the role of the QALY in assessing benefits (14/19) Which processes are used to make coverage decisions? A preference for cost-utility analysis (CUA) is specified by the pharmacoeconomic guidelines for England and Wales, New Zealand, Scotland and the Netherlands (NICE, 2008a, PHARMAC, 2007, Scottish Medicines Consortium, 2007, Tsiachristas et al., 2008). Of these four jurisdictions, NICE (England and Wales) expresses the strongest preference for this methodology. The NICE guidelines recommend QALYs as the measure of benefit to be used in the Reference Case, because the QALY is a generic measure that reflects both mortality and HRQL effects (NICE, 2008a, para ). Nonetheless, the NICE guidelines also recognise that there may be cases when the QALY is less appropriate: If the assumptions underlying QALYs (for example, constant proportional trade-off and additive independence between health states) are considered inappropriate in a particular case, then evidence to this effect should be produced and analyses using alternative measures may be presented as an additional non-reference-case analysis. (NICE, 2008a para ). In respect of the other three jurisdictions: the Pharmaceutical Management Agency (PHARMAC) in New Zealand recommends the use of QALYs to measure health-related quality of life to facilitate comparisons across different pharmaceuticals and enable PHARMAC to prioritise investment decisions. However, alternative methods of evaluations are permissible (PHARMAC, 2005, p. 9; 15). the Scottish Medicines Consortium (SMC) has a preference for CUA, but recognises that alternatives may be appropriate and should be justified (Scottish Medicines Consortium, 2004). The Dutch guidelines recommend CUA, although the use of life-years gained is not uncommon (Tsiachristas et al., 2008). More often, however, the use of QALYs to inform reimbursement decisions is optional (Australia, Belgium, Canada, Denmark, Finland, Germany, Ireland, Norway, Portugal and Sweden). For example, the Norwegian Medicines Agency s (SLV) pharmacoeconomic guidelines are not prescriptive over which type of economic analysis is used. However, the guidelines recommend that if cost-utility analysis (CUA) is used, then the analysis should be supplemented with cost-value analyses based on transformed values, where the value of less serious conditions is set at just below one. Tentative transformation functions are used to convert values from different health-status indexes to values that reflect the Norwegian societal emphasis on the degree of severity in priority setting (Nord, 2001a). However, so far, no applications for reimbursement have included such supplementary analyses. 18

19 Just two of the 19 countries who use economic evaluation in their coverage decisions explicitly specify values for the cost-per-qaly threshold. As we noted NICE indicates that the threshold is between 20,000 and 30,000 with a requirement for an increasingly strong case for other factors as the ratio increases (NICE, 2008a). As also discussed in the previous section, NICE has now decided to give greater weights to QALY gains generated by end-of-life treatments in specific circumstances (NICE, 2009). The SMC requires the manufacturer to show under what circumstances the net ICER exceeds 20,000 and 30,000. (Scottish Medicines Consortium, 2007). In six jurisdictions (Australia, Canada, Ireland, New Zealand, Sweden, and the Netherlands), an implicit threshold can be inferred. There has been debate in the Netherlands over whether the Health Care Insurance Board (CVZ) should adopt explicit thresholds for collectively funded care, with proposals that thresholds should vary with disease burden and disease severity. For example, a threshold of up to 80,000 per QALY gained has been suggested for life-threatening conditions (Goossens, 2006). In other countries, the threshold is unclear (Austria, Belgium, Denmark, Finland, Hungary, Norway, Poland, Portugal, Spain, and Switzerland) What decisions on cancer medicines have been reached by HTA bodies under these different processes? Coverage decisions made on the last ten drugs appraised by NICE were sought for 13 countries (see Appendix C). However, in only five of these countries (England and Wales, Scotland, Norway, Finland and Australia) were full data available on all ten drugs. Therefore, this section focuses on findings from five countries. Unsurprisingly, there was considerable heterogeneity in the pattern of decision making between countries. In Australia, all ten drugs were approved but in each case the availability of the drug was restricted. Of the five countries considered, NICE had the highest approval rate without any restrictions (4/10 drugs with positive recommendations). Finland and Norway were most likely to make negative reimbursement decisions (7/10 drugs), whereas only two drugs were completely rejected for use in NHS Scotland. Amongst the five countries, positive recommendations were made for 25% of the drugs, with the drug made available to all patients covered by the licensed indication. The remaining decisions were equally divided between restricted approval and negative recommendations. The nature of restrictions varied both within and between countries. For example, the Australian Pharmaceutical Benefits Advisory Committee (PBAC) listed all 10 drugs on its Schedule as restricted benefits. In one case, the restriction was to a clinical subgroup of the licensed population for limited treatment duration (anastrozole for breast cancer). The remaining nine drugs were restricted by a requirement for prior approval from Medicare Australia or the Department of Veterans' Affairs. The SMC restricted use of 5 of the 10 drugs, 19

20 specifying clinical restrictions over and above those contained in the licensed indication for example, pemetrexed for non-small cell lung cancer was only recommended for use in patients with good performance status who would otherwise be eligible for treatment with docetaxel. For this sample of 10 drugs, restrictions by NICE were related to clinical characteristics of patients, for example recommending that only patients with certain Karnovsky performance-status scores (cetuximab for head and neck cancer) or WHO performance status scores (pemetrexed for mesothelioma; temozolomide for glioma) were eligible to receive the drug on the NHS. The highest rejection rate was found for Finland and Norway, who both decided against any reimbursement for 7 of the 10 drugs. However, inpatient treatments may be funded by hospital budgets, removing the need for reimbursement listing. These results should therefore be interpreted with caution given the differences in the healthcare systems considered and, particularly, in the ways restrictions are classified in different jurisdictions. Special arrangements were evident for some drugs. In Canada, a pricing agreement was negotiated so that pemetrexed could be listed for lung cancer. In England and Wales a risk-sharing agreement for bortezomib (multiple myeloma) was negotiated between the Department of Health and the manufacturer and in Portugal budgetary and pay-back schemes are under negotiation for the same drug (but for first-line use). This limited evidence suggests that a new cancer drug is more likely to be given restricted approval or rejected rather than to be approved for use in all the patients for whom it is licensed. Finland and Norway, which we noted in section 4.6 are the two countries considered where severity considerations explicitly inform decision making had the highest rate of rejection of new cancer drugs. Therefore, the available data suggests that HTA systems incorporating concerns for severity do not necessarily favour cancer treatments, although this may reflect the lack of submissions by companies using the cost value analysis. As the analysis was limited to a small number of countries and drugs, this finding should not be over interpreted. 2.6 Summary Cost per QALY is one of the main drivers of the NICE decision making process, although other factors can be considered on a case by case basis. Our analysis of the outcomes of NICE appraisals on cancer medicines shows that the majority of them received a positive recommendation with no restriction (23 of 41 medicines). According to our review of international HTA processes, in four (England and Wales, New Zealand, Scotland and the Netherland) out of 24 jurisdictions a preference for QALYs is explicitly specified by pharmacoeconomic guidelines. In other countries (10 out of 24), health care bodies have been more flexible when using cost effectiveness evidence to inform decisions and less prescriptive. NICE has not established a process specific to the appraisal of cancer drugs, although a large proportion of TAs has focused on oncology treatments. In spite of this, in a comparison of reimbursement decisions in five countries using HTA for 10 cancer drugs (those appraised in the last 10 NICE TAs) shows that NICE had the highest approval rate without restrictions, albeit at only 40%. These results should be treated 20

21 with caution as they are based on a small sample of drugs. We have also not been able to systematically analyse approval rates with restrictions because of the variable nature of restrictions. There are substantial differences in the health care systems considered (for example in health care spending, drug prices and the role of HTA in coverage decisions). The group of countries considered includes countries relying on cost effectiveness and does not include four of the five major European countries. 21

22 3 Issues in using the QALY when assessing cancer medicines health gains 3.1 Introduction The QALY combines the quality of life (QoL) and the quantity of life (i.e. life years). In order to measure the utility 1 of the QoL, first the health states need to be described, and then the scores attached to these states need to be generated. Currently, NICE seeks use of the EQ-5D. The EuroQoL five dimensional questionnaire is used to describe the health states and their valuations based on a study using the time tradeoff (TTO) technique applied to generate the general public s preferences towards various health states. We set out in Figure 4 a simplified framework showing how the QALY estimates are produced. Figure 4 A simplified QALY framework Patients To obtain descriptive data about the HRQoL of different diseases, the EQ-5D questionnaire can be used. Health states (243 combinations) General public Utilities, Years gained Each health state is converted into a value based on a study involving the preferences of 3000 members of the UK population Duration of the health state in years multiplied by the utility value for the health state Health utilities (from 0 to 1) QALYs gained Measuring health outcomes with a single index such as the QALY allows for consistency and comparability across different technologies, i.e. the QALY acts as a common currency to assess value for money generated by treatments in different diseases areas. However, the QALY, because of the tools used to estimate it, might not be sensitive to some differences in health status that are relevant for particular diseases or interventions. Therefore, we investigate: whether or not the QALY can pick up all the relevant health gains generated by cancer medicines; 1 The term utility is used here to refer to the preferences that individuals or society may have for any particular set of health outcomes (Drummond et al, 2005). 22

23 if not, what are the potential limitations of the QALY; whether the published literature provides any suggestions as to how to address any such limitations. Using the results of a comprehensive literature review, we identified a list of QALY issues. We then focus on a shorter list of key limitations of the QALY measure in the context of oncology treatments. These we review in this chapter. They relate to: the health states description system used to estimate the QALY; the assumptions underlying the method employed to elicit preferences for health states (i.e. the constant proportional trade-off assumption in time tradeoff (TTO)); patient versus general public valuations of health states. A more detailed explanation of our selection process is included in Appendix A. Each section includes an explanation of the issue and a discussion of potential solutions identified in the literature. The final section summarises the material presented in the chapter. The focus of the chapter relates to individual valuation, i.e. assessment of health gains from an individual perspective rather than a societal perspective, which is addressed in Chapter Sensitivity of EQ-5D as a descriptive system In order to assess the impact of health on quality of life different instruments are available. NICE recommends the use of EQ-5D, which is a generic preference-based instrument. The EQ-5D questionnaire contains five dimensions: mobility (MO), selfcare (SC), usual activity (UA), pain/discomfort (PD), and anxiety/depression (AD). Each dimension has three levels: no problems, some problems, and extreme problems. One of the most controversial aspects of using the EQ-5D in the oncology area is related to its sensitivity to the changes of health status in cancer patients. This sensitivity is a function of both the dimensions and the levels in the EQ-5D descriptive system. With regard to the dimensions, EQ-5D has been found to be less sensitive to health state changes in patients where the degree of vitality is an important element when compared to other QoL measurement instruments such as SF-6D. Grieve et al (2008) compared the sensitivity of SF-6D and EQ-5D in detecting the health gains delivered by the antiviral therapy for patients with mild chronic hepatitis C. They found a greater utility gain using SF-6D than using EQ-5D in disease areas where the vitality or social functioning are important aspects. The vitality dimension, which is closely related to fatigue and energy, has significant impact on the HRQoL of cancer patients (Broeckel et al, 1998; Stone et al, 2003; Esbensen et al, 2004; Hofman et al, 2007; Dobrez et al., 2007). However, this dimension is not included in the EQ-5D system as it was considered as having a trivial impact on the health states value (Williams, 1995). Williams (1995) does not give detailed information on the reasons 23

24 for taking this view: whether it is due to the description of health states in general terms in the questionnaire design or because of the elicited preferences of the general public. So far, there is no empirical study looking at the performance of the vitality dimension if added to the original EQ-5D system, although attempts have been made to include other dimensions. For example, a cognitive dimension has been added to the EQ-5D system to assess health status of elderly patients with cognitive impairment (Wolfs et al, 2007). In terms of the number of levels within each dimension, there are issues related to the ceiling effect, which may occur if the EQ-5D measure is not able to capture small changes in health status (Bharmal and Thomas, 2006; Barton et al, 2008). Feeny (2005) shows that this is particularly true in the case of cancer patients whose posttreatment QoL gains tend to be small. Several studies suggested a 5-level EQ-5D system would have less of ceiling effect and increased descriptive power, compared with the 3-level system (Pickard et al, 2007a; Pickard et al, 2007b; Kind, 2007; Janssen et al, 2008a; Janssen et al, 2008b) How can the lack of sensitivity of EQ-5D be addressed? To address concerns about the lack of sensitivity of EQ-5D to the changes in cancer patients health status, two options have been explored. The first is to refine EQ-5D with additional dimensions and levels. There is some evidence that extra levels could increase the discriminative validity. However, there is not much research on the issue of adding an extra vitality dimension into EQ-5D. Also, one can argue that to some extent changes in vitality may be indirectly captured through other dimensions of the EQ-5D, such as usual activities. Therefore, the evidence is not yet strong enough to support the inclusion of this extra dimension. As shown by Grieve et al (2008), other preference-based generic instruments such as SF-6D are more sensitive, although not completely problem-free. Therefore further research on the performance of this tool in the context of cancer treatments is advisable. In terms of the development of a 5-level EQ-5D system, although it is obvious that it would improve the sensitivity of the tools, it would also lead to a significantly larger number of health states. This would further increase the burden of completing and administrating questionnaires as they became more complex. The current three-level EQ-5D system generates 243 different health states (3^5), but a 5-level system would have 3,125 states (5^5). This would increase the difficulty of determining valuation of all possible health states. Currently some health states have been valued directly by the public while the remaining part has been valued using regression analysis (the robustness of which is debatable). Therefore, the increased descriptive power of a refined EQ-5D system has to be balanced out with the difficulties of valuing a significant number of health states. The second option is to transfer non-utility-based (e.g. cancer-specific) QoL scores, which are found to be more sensitive to changes in cancer patients health status (Kind and Macran, 2005; Krahn et al, 2007), into utility-based ones (e.g. EQ-5D) through 24

25 mapping exercises. It appears that the direct mapping of the cancer-specific scores to the EQ-5D is difficult (Chancellor et al, 1997). However, an example of a mapping procedure for QLQ-C30 and EQ-5D is illustrated in Nord et al. (1997) in multiple myeloma. Other studies have involved either developing a set of utility weights that could be used in translating non-preference scores into EQ-5D, by applying values elicited from the general public for health profiles measured by condition-specific measures (Kind and Macran, 2005; Lamers et al, 2007) or to fit a prediction EQ-5D model using patient demographic and disease-specific QoL scores (Wu et al, 2007). 3.3 Valuing health states: the assumption of CPT underlying TTO In generic preference-based measures such as EQ-5D, descriptive data on HRQoL is obtained from patients, then each health state is converted into a value based on a study involving the preferences of members of the general public. One of the methods to elicit values to attach to health states is time trade-off (TTO), which was used to derive the UK population scores for EQ-5D. The TTO method involves presenting participants with two alternatives: 1) state i for time t followed by death; 2) healthy for time x<t followed by death. Time x is varied until the respondent is indifferent between the two alternatives, at which point the required preference score for state i is given h = x/t. (Drummond et al. 2005) One of the assumptions underlying TTO is the constant proportional trade-off (CPT). CPT denotes that a person is willing to trade off a constant proportion of life time to obtain a proportion of improvement in HRQoL, regardless of the number of the life years remaining (Dolan and Stalmeier, 2003). This implies that the proportional amount of time which is traded off (x/t) is independent of the duration t (i.e. the value h should hold whether the state i lasts 5, 10 or 20 years). When developing the weights for EQ-5D health states in 1995, the general public were given a 10-year time trade-off framework (Gudex, 1994), which does not reflect the much shorter typical duration of health states in the context of cancer (particularly in late-stage cancer lasting for less than one year). Dolan and Stalmeier (2003) also show that for more severe states, it might not be appropriate to use values obtained with TTO as these states can be associated with maximum endurable time (MET) preferences, a point beyond which people do not wish to live. It seems that people s trade-off decisions follow a more heuristic method, i.e. trial-and-error, than a constant pattern (Dolan and Stalmeier, 2003). The current inconsistency tests of the CPT have not yet identified a systematic correlation between trade-off preferences and duration of health states. But there is some literature indicating several key turning points. For cancer patients nearing the end of life, there is evidence supporting a non-cpt assumption as the time trade-off may depend on the number of life years remaining (Bleichrodt et al, 2003; Tsuchiya and Dolan, 2005). Miyamoto and Eraker (1988) reported that about 25% of respondents did not trade off any time to improve their current health when the duration was under 1 year, whereas time was traded off when the duration was more than one year. They referred it to as indifference to health quality at short durations. 25

26 3.3.1 How can CPT issue be addressed? Potential solutions for this problem were explored in the early stages of EQ-5D application, but these proposals have only recently been followed up. The EuroQoL group has commissioned a number of research studies which are ongoing. Dolan (1996) proposed that preference weights for health status should be varied according to different time durations. More specifically, compared to direct preference measures such as visual analogue scales (VAS), the choice-based methods such as time trade-off (TTO) were not appropriate techniques for assigning values to health states when the duration is very short, for example, about 1 month or up to 1 year. Therefore, he suggested a two-stage weighting process: first, to generate a mapping function between values derived from the VAS and TTO in the long duration scenario, then transfer short duration VAS values into short duration TTO scores by using the mapping function. Stiggelbout et al (1995) also proposed a two-stage weighting system. At the first stage, they introduced Subjective Expected Life Years (SELYs), which is a life expectancy that is realistic to the patients, as the anchor timeframe. The TTO questionnaires should be designed based on this realistic anchor timeframe, instead of the artificial standardised 10-year duration. However, their study did not explore further how to adjust the TTO questionnaires according to different SELYs. Also, they did not look into individual preferences when there was one year or less time remaining since the shortest life expectancy specified in the study was 3 years. Recent evidence has confirmed that the SELYs did have some impact on the trade-off options, i.e. the shorter the SELYs, the less willingness there was to trade-off time (van Nooten et al, 2008). Nevertheless the practical issues of applying the SELYs in the EQ-5D system have not been addressed sufficiently. We understand that the EuroQol group is commissioning work on this issue. One of the ongoing research projects focuses on some of the suggestions identified in Buckingham and Devlin (2008), including the possibility of either developing a value function describing the relationship between duration and utility or of producing multiple tariffs for different durations. The value of the QoL component part of the QALY would then depend on how long the patient was going to live for. 3.4 Patient versus general public preferences Another important issue is the question of whose views should be used to produce the health state valuation. Should values be obtained from a sample of the general public or from patients experiencing the health state? Currently generic preference-based measures such as EQ-5D base valuation on the preferences of the general public based on the trade-offs that each person considers appropriate for herself. One argument for using the values of the general public is that patients have a vested interest in access to a treatment and so it is likely to be difficult to expect them to give an unbiased view of the value of the health gain it delivers. Furthermore, since the 26

27 eventual goal of the use of economic evaluation in the context of the NHS and health systems elsewhere is to make decisions about health care resources that fulfil the interests of society, it seems reasonable for preference based valuations of health states to be elicited from a representative sample of members of society. For these reasons, the Reference Case developed by the US Panel recommends that public values are used for health state valuation, so long as the judgments are considered to be informed, unbiased and competent (Gold et al., 1996). Similarly, the NICE Methods Guide states that the values used should be based on the preferences of a representative sample of the general population (NICE, 2008a). The argument in favour of the use of patients values is simple. There is often a discrepancy between the values of patients and those of the general population (Ubel et al., 2003). Given their first-hand experience of the condition in question, patients should in principle be better judges of the relative desirability of the relevant health states. However, there are several problems associated with using patient values in addition to the ones we discussed above. For example, it is often difficult for patients to make long-term judgements about future severity, particularly during periods when they are suffering from pain or emotional impairment. Patients are also said to suffer from poor hedonic accounting - they find it difficult to provide a global summary of their quality of life (Mann et al., 2008). There is however an important issue of adaptation. There is evidence of discrepancies between the way in which patients adapt to states of ill-health over time (for example, learning to focus on things in life other than their disabilities) and healthy individuals perception of their ability to adapt (Dolan and Kahneman, 2008). This implies that in many circumstances, members of the general public may overstate the impact of a condition if they are not experiencing it. It is important to note that the majority of evidence indicates that when assessing a particular health state in terms of desirability, patients on average tend to assign higher values than samples of the general population without experience of that health state (Stiggelbout and de Haes, 2001). However, if this observation is being driven by the patient adaptation phenomenon then it may not apply to end-of-life cancer patients who are not afforded the time to adapt to their new, temporary health state Is it possible to incorporate patient s value in the QALY calculation? In addition to the theoretical arguments against the use of patients values summarised above, there are also several practical issues. One is related to the difference between hypothetical (not disease-specific) health states, which are defined in EQ-5D, and the health state directly experienced by patients. Dolan and Kahneman (2008) explore this issue proposing the use of experience utility where people are asked to value their utility at different points of time. However, Brazier (2008) points out that the use of experience value is a developing area where practical methods to incorporate those in the QALY estimation have yet to be defined. 27

28 This is clearly an area where further research is required. We do not at this stage think the evidence suggests that NICE should move away from using the valuations of the general public. In the short term it may not in any case be possible to integrate experience values into the QALY calculation as a methodology to be implemented by HTA bodies has not been developed yet. It may make more sense to concentrate on trying to ensure that members of the public being asked to value health states fully understand what being in a particular health state might involve. Issues of adaptation are also linked to issues of duration in valuation which we discuss above. 3.5 Summary In this chapter we analyse some of the main challenge related to the use of the QALY to measure health benefits. In particular, we discussed the limitations of using a HRQoL measure such as EQ-5D, which is currently the preferred method indicated in the NICE Reference Case. The evidence we reviewed show that EQ-5D may not always be sensitive in the context of cancer medicines. First, because of the omission of the vitality dimension which seem to be particular relevant in cancer; second, because of the limited number of levels associated with each dimension which might lead to a ceiling effect. No empirical study attempting to assess the impact of adding an additional dimension was found. Given the availability of other instruments such as SF-6D, it seems advisable to further explore how they perform in cancer to see if they represent a more valid alternative to EQ-5D. In terms of the number of levels in EQ-5D, a number of studies have showed potential benefits of increasing the number of levels to five. However, the significantly increased number of health states that this would imply has to be considered carefully. Another issue that we explored is the assumption of constant proportional trade-off (CPT) underpinning the EQ-5D elicitation method of time trade-off (TTO). Various solutions to this issue have been identified in the literature, all attempting to adjust the timeframe (10 years in most TTO exercise) to reflect different health states duration. More research is needed to improve the understating of the relationship between length of the health states and corresponding value. We are aware that some studies focusing on this issue have been commissioned by the EuroQoL group. We note that research continues on the differences between patient and general public valuations of health states and the extent to which it is appropriate or feasible to use patient valuations. This work is important but the evidence at the moment does not suggest it is appropriate or feasible to switch from using valuations from the general public, albeit there is a case for ensuring that they are well informed about the health states they are being asked to value. 28

29 4 Issues in using the QALY when appraising cancer medicines. The role of severity and other social value judgements. 4.1 Introduction In this section we move from the concerns about the QALY as a suitable measure of health outcomes from an individual perspective (are we correctly assessing the health effect) to consider questions about resource allocation (i.e. in the appraisal of the evidence). Given the number of QALYs achieved by an intervention, how much priority should this gain be given by society given other potential uses of the resources required to fund the treatment - notably how many QALYs are being given up in other disease areas? Are there factors beyond the QALY that are important in the context of health care decision making? The chapter contains a discussion about factors which, according to some recent empirical literature, are deemed important by society. It then focuses on the factor of severity, given its importance to the public and its relevance to cancer, providing a review of the theoretical and empirical literature on the topic. On this basis, section 4.4 discusses some of the issues related to the elicitation and interpretation of preferences for severity and section 4.5 discusses what are the possible options to incorporate severity, and other social value judgements, into health care decision making. Section 4.6 includes an international comparison looking at the extent of explicit consideration of severity and special process for cancer in other countries HTA processes. 4.2 Social value judgments It is widely assumed by health economists that the principal objective of health care is to maximise population health using available resources (Culyer, 1997) an assumption that is, however, based on a narrow interpretation of the extra-welfarist approach to resource allocation (Brouwer et al., 2008). Since the QALY has been developed to provide a generic measure of health effect, it follows, so the argument goes, that health care resource allocation should seek to maximise the number of QALYs generated (Dolan, 2001). In cost-utility analysis, the overall health benefits of an intervention are calculated by aggregating the QALY gains accruing to individual patients using a simple, unweighted summation. This is commonly referred to as the QALY-maximisation rule. It implies that the social value of the health benefits is determined by a product of four factors: (i) the treatment s effect on health-related quality of life, (ii) the treatment s effect on life expectancy, (iii) the duration of treatment effect, and (iv) the number of individuals affected by the treatment.. It entails distributive neutrality that is, it does not incorporate concerns 29

30 for how the benefits are distributed across individuals, and it does not consider effects outside of the QALY. Culyer refers to this position as QALY egalitarianism all QALYs are of equal social value, regardless of whom they accrue to and the context in which they are enjoyed (Culyer, 1992). In other words, a QALY is a QALY is a QALY under all circumstances. However, maximising health is not the only purpose of health care. From a welfarist perspective, for example, health is not given a special status and the objective of health care instead becomes the maximisation of social welfare. Furthermore, alternative variants of extra-welfarism also allow for objectives beyond the pursuit of aggregate health gain. The NHS also has equity objectives such as seeking to improve the health of the poorest fastest (DH, 2003). This objective suggests that we are interested not only in the total number of QALYs, but also in how these QALYs are distributed amongst individuals. The preference for targeting the poorest involves making a social value judgement that the QALYs gained by some individuals should be given greater weighting than those gained by others. NICE recognises that society may have preferences regarding the use of health care funding that involve some sacrifice of total QALY gains in order to achieve a more desirable distribution of health. As well as making scientific value judgements, the Institute accepts the need to incorporate these social value judgements into its advice: Principle 3: Decisions about whether to recommend interventions should not be based on evidence of their relative costs and benefits alone. NICE must consider other factors when developing its guidance, including the need to distribute health resources in the fairest way within society as a whole. (NICE, 2008a) For a particular value judgement to be incorporated into the HTA process it has been argued that it must demonstrate social legitimacy (Claxton and Culyer, 2008) that is, there must be systematic evidence of societal support for the deviation from the QALY is a QALY is a QALY rule. This makes sense and NICE s position on social value judgments is informed by the findings of empirical research on public preferences and the work conducted by its Citizens Council - a 30-member representative panel of the general population who meet twice annually to discuss questions related to social values within health care. Potential sources of social value can be broadly divided into two categories (Schwappach, 2002): factors concerning specific characteristics of the patient or intervention (e.g. patient s role in society) and factors concerning the nature of the health improvement (e.g. gain in life expectancy). A number of researchers have attempted to capture which factors the public consider to be most important through empirical investigation. A recent example of this can be found in the NICE/NCCRM project led by Donaldson (Donaldson et al., 2008) in which a combination of different techniques was used to identify factors according to which relative weights might be derived. One technique involved asking a group of respondents to rank order a list of ten potential factors and discuss their answers. The average rankings are presented in 30

31 Table 1, with ranks 1 and 10 representing the most important and least important factors respectively. In addition, Donaldson et al. conducted a series of open-ended discussion groups and a Q-methodology exercise (Baker et al., 2006). The findings of these studies were largely consistent with those of the ranking exercise, with respondents showing a tendency to place greater importance on need factors rather than on personal or socioeconomic characteristics. The four highest-ranked factors in Table 1 can all be described as being particularly relevant to cancer. The first and third factors refer to the severity of the patient s condition in the absence of treatment. Whilst cancer and severity are by no means equivalent, cancer patients are typically observed as suffering from more extreme illhealth than most non-cancer patients (Pickard et al., 2007c), although in reality much depends on the cancer type and stage of tumour. Many cancer treatments are in practice aimed at those experiencing severe illness, either in terms of quality of life (in particular, extreme pain/discomfort, anxiety/depression and restrictions on usual activities), or length of life (Mason and Drummond, 2007a) or both. On the whole, there is evidence suggesting that people wish to give greater weight to the QALYs gained by these types of patients (as will be discussed in the following section). The second factor, availability of alternative treatments, is also relevant because of the current dearth of effective treatments for rarer cancers. Table 1 Simple ranking exercise results (adapted from Donaldson et al., 2008) Overall rank Average rank Factor Pre-treatment quality of life Availability of alternative treatments Pre-treatment life expectancy Age Health-related lifestyle Prior health care consumption Dependants Employment status Gender Social class The fourth factor, age, is relevant to the appraisal of cancer treatments because cancer is most common amongst older people, particularly those aged 60 and over (Cancer Research UK, accessed 2008). Although much of the evidence indicates a societal preference for giving priority to the young (Dolan et al., 2005), which would suggest less weighting for cancer treatments, age weighting is a highly controversial subject and most studies report that a significant minority of respondents are heavily opposed 31

32 to using it as a prioritisation rule. NICE applies the principle that patients access to treatment should not depend on age, with a few exceptions (NICE, 2008a). This principle reflects the conclusions of the Citizens Council meeting on age (NICE, 2004). It seems appropriate to focus on the factor of severity for two reasons as well as its relevance to cancer. First, it is highly relevant in terms of current UK health care policy for example, it was the main focus of the Citizens Council meeting in early Second, compared with some of the other candidate factors there is a relative abundance of empirical evidence available on public preferences concerning severity. 4.3 Severity - theory The severity approach to health care resource allocation is based on the maximin theory drawn from John Rawls proposition that when entering a social contract behind the veil of ignorance, individuals will unanimously choose that resources are allocated so that they are to the greatest good of the least advantaged (Rawls, 1972). It is implies that having a special concern for those whose health-related circumstances are poorest (and subsequently redistributing resources so as to target these individuals) is socially valuable. However, it is not always clear what exactly is meant by severity NICE, for example, has no agreed definition (NICE, 2008b). The most popular method in the literature is to define severity in terms of the patient s pretreatment health state or start point. Figure 5 depicts an adapted version of the severity scale used by Nord et al., designed so as to make the intervals between each level appear equally significant in terms of individual utility (Nord et al., 1999). Three left-hand side arrows represent the possible gains from treatment for three hypothetical individuals A, B and C. Figure 5 Eight-level severity scale (adapted from Nord et al., 1999) Problem level Example (in terms of mobility) 1 None (healthy) 2 Slight Can move about anywhere, but has difficulties with walking more than 2km. 3 Moderate Can move about without difficulty at home, but has difficulties in stairs and outdoors. B 4 Considerate Moves about with difficulty at home. Needs assistance in stairs and outdoors. 5 Severe Can sit. Needs help to move about both at home and outdoors. A C 6 Very severe To some degree bedridden. Can sit in a chair part of the day if helped up by others. 7 Completely disabled Permanently bedridden. 8 Dead 32

33 Individuals A and B are both said to have the same capacity to benefit from treatment (three-level improvement in health) and so a decision-maker whose sole objective is to maximise health would have no reason to give priority to one over the other. However, in the absence of treatment A finds herself in a worst health state than B she is more severely ill than B. The severity approach suggests that other things being equal, the health gains accruing to A should be valued more highly than those accruing to B on the grounds that there is intrinsic value in helping the worst-off. Thus, given scarcity of health care resources A should be given priority over B. Those who advocate allocation according to severity might also prefer to give individual C priority over individual B, even though her health gain from treatment is smaller. In this way, the severity approach can involve a sacrifice of aggregate health in order to target the worst-off. In this example there is no mention of the individuals life expectancies, so severity is being measured purely in terms of health-related quality of life. This does not have to be the case some researchers also use pre-treatment life expectancy, or proximity to death, as an indicator of an individual s severity of illness (Dolan and Shaw, 2004). 4.4 Severity empirical evidence Empirical studies of severity-related preferences vary considerably in terms of study design, sample size and subject type. Most of the studies picked up in the literature review entailed asking a sample of the general population to adopt the perspective of a social decision-maker and to indicate their preferred solutions to allocation problems involving various interventions that differed in terms of the size of health gain, pretreatment severity, and other variables. Popular methods for eliciting these preferences include person trade-offs, group discussions and discrete choice methods (see Ubel, Richardson and Menzel (2000), Cookson and Dolan (1999), Ryan (2004) for discussions of these methods). Table 2 provides a summary of these studies. In the final column, Yes indicates that the researcher(s) reported a preference for assigning greater weight to health gains accruing to the severely ill, either by establishing a quantitative estimate of the strength of this preference or by providing details of a persistent and systematic endorsement of severity as a priority-setting criterion. The majority of studies suggest that people are, on the whole, willing to sacrifice aggregate health in order to give priority to the severely ill. However, the results should be treated with caution. 33

34 Table 2 Summary of empirical evidence Study Country # subjects Type of subject Mode Results generally supportive of severity approach? Abelson et al. (1995) CAN gen population, clinicians, deliberative Yes health officials polling Cookson & Dolan (1999) UK general population focus group Yes Dolan (1998) UK 1-49 students questionnaire Yes, up to a certain threshold Dolan & Cookson (2000) UK general population focus group Yes, up to a certain threshold Dolan & Green (1998) UK 1-49 university staff interview No evidence of favouring less severely ill (but see for discussion of methodological issue) Dolan & Tsuchiya (2005) UK gen population questionnaire No see Nord (2006) for subsequent discussion of results Dolan et al. (2008) UK gen population interview Yes, but preferences not linear Donaldson et al. (2008) UK gen population questionnaire Mixed different results for different study designs Gyrd-Hansen (2004) DNK 1,000+ gen population interview Yes, for certain dimensions of health Jacobsson et al. (2005) SWE mostly health care workers questionnaire Yes NICE (2008) UK 1-49 gen population focus group Yes Nord (1993a) NOR 1-49 health policy planners interview Yes Nord (1993b) NOR health policy planners questionnaire Yes Nord (1995) NOR 1-49 gen population questionnaire Yes Nord et al. (1995) AUS gen population questionnaire + Yes interview Richardson (1997) AUS students questionnaire Yes Stolk et al. (2005) NLD students, researchers, policy-makers interview Yes, but other allocation approaches more appropriate Ubel (1999) US gen population questionnaire Yes, but results not stable Ubel et al. (1998) US gen population questionnaire Yes This is for several reasons. First, some of the studies shown in Table 2 used very small samples often because they were conducted in order to trial a particular survey technique or to ascertain a hypothesis that could then be fully tested in a larger study. Second, even in studies that reported strong support for prioritising according to severity there was usually a significant minority of respondents who opposed this approach. Third, there are numerous unresolved issues concerning the best ways to elicit and/or interpret severity preferences some of these are examined below Question framing and study design issues The importance of question-framing is highlighted by Ubel (1999), who replicated an earlier study by Nord (1993a) in the US with the aim of establishing the stability of people s severity preferences. Ubel found that minor changes in the questionnaire (added emphasis in the wording and the removal of the divide resources equally 34

35 option) led to considerably fewer respondents choosing to give priority to the severely ill. This finding suggests that it is important for researchers to consider the cognitive capacities and tendencies of respondents when designing studies. Psychological phenomena such as omission bias (Dolan and Cookson, 2000) and the tendency to view losses as being larger than equal-sized gains (Tversy and Kahnemann, 1991) may distort preferences and responses driven by these factors are unlikely to provide a satisfactory basis for making social policy decisions. The recent project by Donaldson et al. (2008) also demonstrates the importance of choice of study design. In the main part of the study the team used two approaches discrete choice and matching (person trade-offs) to derive potential severity weights. Whilst the discrete choice results suggested that severity has only a very weak impact on respondent choices, the matching approach showed a much stronger effect. A possible explanation for this is that the former approach involved varying equity attribute levels and health gain size in scenarios simultaneously (thereby dampening the impact of severity) whereas the latter involved holding health gains constant (thereby amplifying its impact). The two sub-groups within the research team failed to come to agreement about which of the two approaches was better able to capture actual societal preferences. Both call for further research on the topic, advising that conducting qualitative research and cognitive testing alongside the quantitative surveys might shed light on peculiar response patterns Post-treatment severity and realisation of potential One of the most striking findings of the NICE/NCCRM project led by Dolan (Dolan et al., 2008) was that although the results indicate a preference for treating the severely ill, this preference is non-linear. Based on a 0-100% individual health scale, the team used the survey responses to derive the following weightings: First 25% of health valued as worth 26.8% of full health (in terms of its social value) Next 25% of health valued as worth 34.7% of full health Final 50% of health valued as worth 38.5% of full health This implies that although respondents do place a premium on the health gained by the severely ill, a greater weight is placed on moderate severity than on extreme severity. This seems like an odd result but is echoed elsewhere, notably in Donaldson et al. s matching study. One possible explanation for this is that people s responses may be being driven by a concern about the patient s end-point level of health, or post-treatment severity. The idea here is that respondents treat prioritisation decisions as having to make trade-offs between a concern for pre-treatment severity and a concern about whether or not treatment enables the recipient to achieve a reasonable quality or length of life. Nord (2008) points out that in Dolan et al s study interview participants were asked to values life time scenarios in order to assess not only their preferences for severity but also preferences for other factors, such as inequality between social groups. This led the authors to develop overly complex questions which he argues have not enabled 35

36 the authors to pinpoint the value attached to health gains accruing to severely ill people. In other words the results we set out above may not be valid. Evidence on preferences concerning post-treatment severity is mixed. On one hand there is evidence that many people feel that patients whose health after treatment is still very poor may not be worth saving (Dolan and Green, 1998; Roberts et al., 1999). On the other, there is a sense that it is unfair to discriminate against patients whose future prospects are poor, with some studies reporting a general objection to denying health care to patients on the basis of their prognosis (Abellan-Perpiñan and Pinto Prades, 1999; Nord, 1993). This concern is described by some researchers as the realisation of potential argument (Nord et al., 1999) and can be demonstrated by comparing the health gains of individuals A and C in Figure 5. Both start at the same level of severity and so it might be argued A should be given priority over C on the basis that her capacity to benefit from treatment is greater. However, C s potential for health improvement may still be important and substantial to her and failing to account for this fact might imply that a healthy person s life is more valuable than that of a less healthy person (Harris, 2006). The two positions described here have very different implications for priority-setting. The former implies that people are willing to de-prioritise the treatment of patients whose expected health gain is small, a conclusion that may cause political and ethical discomfort but one that corresponds with the health-maximisation objective that underlies health care resource allocation in the UK. Conversely, the latter implies that (in line with the majority of evidence on pre-treatment severity) health gain is not all that is important. Compared to pre-treatment severity, evidence of preferences concerning posttreatment severity and realisation of potential remains scarce and it is important that future analyses of severity actively consider the ways in which people trade these fairness concerns against each other Double-counting One of the main concerns about formally weighting QALYs to incorporate severity preferences is the fear of double-counting these preferences (NICE, 2008d). From an individual perspective, it is important that on any scale being used to measure health, intervals of equal length on that scale must have an equal interpretation this is the fundamental property of an interval scale. For example, the increase in desirability of a move from 0.2 to 0.4 must be equal to the increase in desirability of a move from 0.6 to 0.8 (Drummond et al., 2005). If this is not the case, then it does become difficult to disentangle preferences for one s own health from preferences about the health of others when trying to establish societal values. In order to illustrate this point we can return to Figure 5. Whilst the scale has been designed such that the movement between each severity level is equally valuable, it is possible that respondents might interpret it differently for example, they might believe that (when thinking about their own health) the move from level 5 to level 4 represents a larger and more significant improvement than the move from level 2 to level 1. If this were the case, then a preference for giving priority to A over B would 36

37 not necessarily represent a desire for society to target the severely ill instead, it might simply indicate that the respondent (wrongly) believes that A s treatment gain is larger than B s, in absolute terms. Describing an earlier pilot study, Nord reported that the respondents had confirmed that they perceived the intervals [as portrayed in Figure 5] as quite equal (Nord, 1993a). However, in another study adopting the same scale, Ubel et al warned against treating their results as undisputed proof of support for the severity approach as they could not be certain that the scale really had captured equal differences between adjacent levels (Ubel et al., 1998). Dolan and Green found evidence of respondents views changing as they proceeded through the interview (Dolan and Green, 1998) having previously declared two health gains to be equally desirable, some then went on at a later stage to describe one as being more valuable than the other. This example highlights the need to validate the properties of the interval scale being used in order to avoid exaggerating the impact of severity on allocation preferences. 4.5 Incorporating severity and other social value judgements into HTA The majority of the empirical evidence on severity preferences points towards an overall societal willingness to target the severely ill (although given the numerous caveats outlined above the evidence certainly cannot be described as conclusive). This suggests that society is willing to pay relatively more for QALYs gained by individuals suffering from extreme ill-health and treatments whose effect is characterised by such gains (such as many cancer medicines) should therefore be appraised differently. There are two main options for incorporating severity preferences into the HTA process. The first is to incorporate observed preferences directly through modifications to the cost-utility calculation. The second is to incorporate them indirectly by feeding results of empirical studies into decision-making as part of a deliberative process. The literature contains several examples of ways of directly incorporating severity concerns. Nord describes how person trade-off questions can be used to elicit the social value attached to each possible movement along a scale such as the one presented in Figure 5 (Nord, 1999). Table 3 provides an example of a complete set of such values, based on an informal synthesis of preference data from a variety of studies. Applying each value to the relevant individual health state gives a measure of societal value rather than a simple sum of individual health gains. Maximising societal value then becomes the objective of cost-utility analysis, or as Nord puts it, cost-value analysis. Another option was indicated by Waugh and Scott (1998) who suggested that the value of extended life expectancy should be trebled for cancer patients if their life expectancy is shorter than 6 months, and be doubled if it is shorter than 12 months. They invited comments on this suggestion but hardly got any feedback. Waugh (2006) raised this issue again but was not able to stimulate further empirical research. 37

38 Table 3 Social value for health improvements (taken from Nord, 1996) Severity with intervention Severity without intervention Dead Alternative methods for deriving and applying relative equity weights were examined as part of the two recent research projects commissioned by NICE/NCCRM (Dolan et al., 2008; Donaldson et al., 2008). Dolan et al. s approach involved specifying a health-related social welfare function (SWF) that includes two parameters: one that reflects societal aversion to inequality of lifetime health outcomes, and one that reflects societal willingness to weight according to non-health attributes such as responsibility. Converting pairwise choice exercise responses into cardinal equity weights provided the team with inputs for the SWF which could then be maximised in order to obtain an efficient outcome that incorporates the relevant equity concerns. As described earlier, the Donaldson et al. work involved the development of two distinct methods for deriving weights discrete choice and matching. Both projects have made considerable contributions to the ongoing QALY-weighting debate. However, neither team has managed to propose a problem-free methodology ready for application to current policy. For example, neither study made a strong attempt to tackle the issues related to realisation of potential type arguments, whilst Donaldson et al. express uncertainty about the extent to which the results of either of their two approaches are consistent with actual societal preferences. Directly incorporating public preferences into cost-utility calculations can be described as being the most scientific approach to managing social value judgements in that it involves the direct application of empirical evidence of societal preferences in a systematic manner. However, the aforementioned projects show that attempts to do this will add a great deal of complexity to the decision-making process. Strictly adhering to a pre-defined weighting formula will also involve a loss of flexibility, and there are fears that Appraisal Committees may not continue to pay special attention to issues related to social value judgements if they believe that they have already been taken into account. Many researchers believe that the long-term search for a universal weighting system or super-algorithm is likely to prove elusive (Oliver, 2004). In the absence of an agreed decision rule, it may instead be preferable to feed the results of research into the appraisal process, mandating appraisal committees to give more weight to social value judgements (and therefore less weight to cost-effectiveness) such as concerns for severity. This was the approach recommended by the Citizens Council (NICE, 38

39 2008d). It is attractive because it offers decision-makers the flexibility to make adjustments in particular contexts whilst enabling them to make allowances for heterogeneous or incompatible preferences. The NICE Methods Guide states that the Appraisal Committee has discretion to consider those factors it believes are most appropriate to each appraisal. (NICE, 2008a) These include the potential benefits of innovation, the degree of clinical need of patients, and advice on social value judgements informed by empirical research and the work of the Citizens Council. Information relating to these factors are integrated with clinical and cost-effectiveness data by means of a deliberative process Deliberative processes A deliberative process is a mechanism by which various forms of evidence are elicited and combined with other inputs (such as theoretical conjectures and contextual information) in order to inform a decision. Such processes typically include extensive consultation with experts and key stakeholders, and are characterised by the considered discussion of the advantages and disadvantages of different options rather than the application of stringent decision rules. Advocates of deliberative processes argue that they account for reasonableness thereby increasing the likelihood of achieving satisfactory decisions (Daniels, 2000). The participatory and consensus-building nature of the deliberations is also likely to have a positive impact on the perceptions of the democratic process, so long as the conditions of publicity and transparency (Daniels and Sabin, 2002) are maintained. Culyer and Lomas (2006) suggest that deliberative processes may be particularly valuable in circumstances where there is uncertainty about technical information, or where there are issues relating to fairness and societal value that need to be taken into account. There are a number of ways in which the methods used by NICE for evaluating health technologies can be described as representing a good model of a deliberative process (Culyer, 2006). The Institute holds extensive consultative exercises through the appraisals process and synthesises opinions and information drawn from a large number of stakeholders ranging from professional scientists/social scientists to lay representatives. Appraisal Committee membership is also set to ensure broad representation. NICE s open and inclusive approach has attracted praise from outsiders for example, the World Health Organisation claims that it is setting an international benchmark through its use of best available evidence in decision-making, transparency, consultation, inclusion of all key stakeholders, and responsiveness to change (Hill et al., 2003). However, some researchers are critical of its failure to formally codify the role of decision criteria other than cost-effectiveness, claiming that its statements on these matters have been vague and uninformative (Schlander, 2008). Whilst the importance of social value judgements and other factors beyond cost-effectiveness is regularly emphasised, examples of interventions with high ICERs being recommended on the basis of such factors are scarce (NICE, 2001; NICE, 2003). Because there is no formal process for assessing and documenting the 39

40 consideration of these factors, it is difficult to understand the extent to which they have contributed to the final recommendation decisions. However, the publicity condition of accountability for reasonableness requires the open dissemination of the rationales behind priority-setting decisions to all stakeholders. Thus, whilst in principle NICE s decision-making fits the description of a sound deliberative process, the lack of explicit reporting of this process means this is not always achieved. There are several ways in which NICE might be able to develop its processes in accordance with the accountability for reasonableness requirements. It would be useful to establish clarity about the Institute s position on social value factors, many of which (notably severity, amongst others) currently have no agreed definition. It is also crucial that efforts are made to eliminate ambiguity surrounding the reasoning behind appraisal decisions, particularly in those cases where the ICER is close to the implied upper threshold value of 30,000 per QALY gained. The introduction of a universal template for documenting the ways in which each relevant factor has been considered by the Appraisal Committee could go a long way towards achieving this goal. At present, it is difficult to establish whether a particular factor has had an influence on the Appraisal Committee s final recommendation unless the Technology Appraisal guidance document explicitly refers to that factor. A template containing a section which prompts the Committee to briefly communicate its overall position on the factor (or to indicate that the factor was deemed not relevant for that evaluation) could help to resolve this problem. It could be used to answer stakeholder concerns, for example that a technology had been undervalued because the Committee had failed to acknowledge the distributional benefits its use would bring Multi-criteria decision analysis (MCDA) A further possibility would be to develop the use of multi-criteria decision analysis (MCDA) methods in health care priority-setting. MCDA is a methodology for appraising options on individual (often conflicting) criteria with the goal of providing a combined appraisal that includes an overall ordering of those options. It provides a framework for explicitly trading off various priority-setting concerns and objectives against each other, and is particularly useful when these objectives do not share a common unit of valuation for example, health care programmes typically involve a mixture of health, monetary, distributional and political objectives. Importantly, several MCDA methods allow for the fact that there may not exist an agreed hierarchy of prioritisation criteria. MCDA has been proposed to support the benefitrisk assessment process conducted for making regulatory decisions (i.e. marketing authorisations or withdrawls for drugs) (Cross and Garrison, 2008). MCDA is used routinely in other disciplines such as environmental and agricultural sciences (Baltussen and Niessen, 2006). A notable example of its use in a controversial area was in the analysis of candidate nuclear waste disposal sites in the UK the MCDA approach was found be superior to the Department of the Environment s decision process, which had failed to include the objectives of all key stakeholders and had not made all value judgements explicit (DLTR, 2000). 40

41 In the context of NICE s appraisal, the Appraisal Committees could identify a list of relevant criteria for decision making (e.g. cost per QALY ratio, availability of an alternative treatment, patients proximity to death). The relative weights for the various criteria could initially be introduced from elsewhere and the Committee asked to review or generated from the deliberations of the Committee. It would generate scores (e.g. the case for any weighting to achieve a tackling inequality objective) relative to the product in question based on the available evidence and judgement. This information could be incorporated into an MDCA model to obtain the total weighted scores. The Committee could review and change the weightings. Sensitivity analyses would help to explore the effects on the overall results of differences in the scores and weights (Phillips and Costa, 2007). The analysis could then be reported in the final Technology guidance or used to report those factors that had been important. A move towards a multi-criteria approach is attractive as it would lead to processes becoming more transparent and systematic. It would capture many of the benefits of directly incorporating additional factors into decision-making whilst avoiding the complexity problems associated with distorting cost-utility calculations. 4.6 International comparison In our literature review covering 24 countries we found six where disease severity is an explicit criterion informing the reimbursement process: In Finland, the Pharmaceuticals Pricing Board (PPB) considers an expert group s assessment of nature of the illness (i.e. severity) (Martikainen and Rajaniemi, 2002, Sorensen et al., 2008) In France, the reimbursement decision is informed by considerations of the seriousness of the disease or condition (Nikolentzos et al., 2008) The German IQWiG evaluates effectiveness evidence in relation to the nature and severity of the disease (Sorensen et al., 2008) Norway s SLV uses reimbursement criteria that are established in law and include disease severity (Nikolentzos et al., 2008). This formally recognised in the Norwegian pharmacoeconomic guidelines which recommend the use of cost-value analysis to ensure that utilities encapsulate concerns for giving priority to the worst off (Nord, 2001b). Spain s pricing and reimbursement decisions are informed by disease severity and duration and by social and therapeutic usefulness (Antonanzas, 2003) Sweden s LFN (now TLV) makes reimbursement decisions assessing manufacturer s evidence including disease severity (Carlsson, 2004) Other countries take broader societal issues into consideration. For example, Ireland s HIQA assesses the quality of life, quality of end of life, social and ethical issues and social benefits (Barry and Tilson, 2007). In Portugal, the principles governing reimbursement decisions are necessity and social justice (Vasco and da Silva, 2001). New Zealand s PHARMAC makes explicit allowance for the consideration of such other criteria as PHARMAC thinks fit, which can include the rule of rescue (Raftery, 2008). Although NICE currently advocates the use of equal 41

42 weights for utilities in CUA, the NICE Citizens Council has recommended that that NICE and its advisory bodies should take the severity of a disease into account when making decisions (NICE, 2008d). However, rather than adopting the Norwegian approach of utility adjustment, the Council advised that severity considerations should be part of the deliberative process, as discussed in the previous section. Eleven of the 24 countries reviewed have some special arrangement for cancer drugs. Of the remaining 13 countries, 10 have no special arrangement and in three countries Austria, Hungary and Poland it was unclear whether such arrangements existed. Special financing arrangements for cancer drugs are a relatively common approach and are found in 11 countries. These include centrally-funded programmes, separate reimbursement processes, and full reimbursement for drugs prescribed in outpatient settings. Centrally-funded programmes may be either specifically for cancer drugs (e.g. Canada (Ontario), Denmark), or for treatments for severe or life-threatening conditions including cancer (Australia, Finland, New Zealand) or for high-cost or high-tech drugs (Ireland). The advantage of this arrangement, which is effectively a form of risk pooling, is that it reduces local variations in the financial impact of treatments upon the health economy. Cancer drugs used for inpatient treatment are sometimes reimbursed separately, with payment additional to the usual remuneration (Belgium, France). For outpatient cancer drugs, full reimbursement is available in several countries (Belgium, Finland, France, Luxembourg, Portugal and Spain). In England, as is already the case in Wales, there are plans exempt cancer patients from prescription charges for all their medicines from April 2009 (Kirby, 2008). Three countries have separate HTA processes for assessing cancer drugs. The most clear example is Canada, which operates the Joint Oncology Drug Review on behalf of all provinces except Quebec. Denmark and New Zealand have specialist subcommittees to review the evidence for cancer drugs. In Denmark, an accelerated assessment process informs the Cancer Steering Committee of the National Board of Health in their decisions on which drugs should be introduced as standard treatment. 4.7 Summary Under the current NICE reference case, all QALYs are deemed to be of equal social value, regardless of whom they accrue to and the context in which they are enjoyed. However, there is a growing body of evidence indicating a societal willingness to give priority to certain individuals, such as those who are suffering from severe illness, even if this involves a sacrifice in aggregate health gains. Many countries already incorporate social value judgements into their decision making processes, with several apparently using severity as an explicit prioritisation criterion. However, although the case for weighting QALYs differently is quite strong, there is little agreement on how best to modify the current NICE methods in order to incorporate these value judgements. An alternative to applying a formal weighting system is to feed the results of empirical research into decision making as part of a deliberative process. In order for this process to account for reasonableness, efforts need to be made to improve the transparency of and ensure explicit reporting of the outcomes of the deliberative process that underpins the appraisal. 42

43 5 Generating clinical evidence and other issues concerning HTA of oncology medicines. Results from a survey of experts. 5.1 Introduction In order to elicit views other than our own as to the nature of the methodological and other challenges that arise in evaluating the benefits generated by oncology treatments using NICE HTA methods and making funding decisions based on this, we conducted a survey involving a number of key experts in the field. These supplemented the literature searches we undertook on the key issues we had identified. With the purpose of examining the topic from different viewpoints, we involved a range of stakeholders. We interviewed 11 people: one patient group representative, one leading oncologist, one professor in psycho-oncology, one senior Department of Health (England) civil servant, four academic health economists who are involved or have been involved in HTA processes, one health economist involved in UK HTA submissions from the industry side, and two (non-health economists) people directly involved in the work of UK HTA bodies. We contacted people involved in HTA processes in Australia and Canada but were not able to arrange interviews. All interviewees were as a result based in the UK. The focus of the discussion was on the debate around methodological issues related to the use of the QALY in the measurement of health benefits of oncology medicines and; the weighting of health gain of oncology treatments (e.g. is society willing to pay more for severe/life-threatening disease?) in the context of NICE s decision making process. Semi-structured interviews were conducted by the phone. The questionnaire sent in advance to interviewees is included as Appendix E. We can group the main themes discussed in the interviews into: assessing health benefits of cancer medicines; appraising cancer medicines and incorporating Social Value Judgements (SVJ) in evidence-based decision making processes; and potential improvements to NICE s approach. Table 4 provides a more detailed list of the issues considered. The following sections discuss the issues listed and the final sections summarises main finding reported. 43

44 Table 4 Key issues identified and discussed in the interview programme Assessing health benefits of cancer medicines: - Generating clinical evidence - Collecting patient reported QoL data - Translating clinical outcomes into broader health outcomes for HTA - EQ 5D in cancer Appraising cancer medicines - Determining society s willingness to pay for severe disease - Methods to incorporate SVJ - Involving stakeholders and expert panels Potential improvements to NICE s approach: - Market access options such as risk sharing schemes and value-based pricing - Better quality of manufacturer submissions - Funding deliberations made at political level - A more expert-driven approach for decision making 5.2 Generating clinical evidence One of the most recurrent issues was related to the generation of clinical trial data, more specifically on the use of surrogate endpoints which are sufficient for licensing purposes but not for economic assessments used by HTA bodies to make coverage decisions. Both FDA and EMEA consider overall survival (OS) the most reliable endpoint as it measures the overall effect of a new treatment in lengthening patient lives (the time from randomisation until death from any cause). However, both have granted marketing authorisation to a number of drugs for the treatments of metastatic cancer based on an improvement in progression-free-survival (PFS), which measures the delay in disease progression (the time from a patient's random assignment to one treatment arm or another until the patient's cancer begins to grow again or the patient dies from their cancer). The use of an endpoint such as PFS to support licensing decisions is due to the consideration that for terminally ill patients (such as those with metastatic cancer), disease progression is often symptomatic and thus it is a benefit in it own right to delay progression (NCI 2008). The use of this endpoint is also related to clinical trial procedures. For ethical reasons, patients in the control arm can cross over if the therapy being tested shows improvements in the experimental arm. Also, when multiple treatment options are available, patients can switch to other therapies if the treatment they are receiving in a clinical trial stops working. At that stage PFS has been observed, as it only measures the effect of the treatment being tested. On the other hand, the effect on OS is more difficult to interpret as the improvement of the 44

45 experimental treatment may be diluted by other treatment received by patients. In other words, it may not be clear the extent to which the improvement is due to the new treatments and how much to other treatments. For these reasons, in many cases licensing authorities have approved therapies for life-threatening diseases based on short clinical trials reporting endpoints other than OS. Although this practice is not specific to cancer, all interviewees who raised this issue agreed that it is less common in other therapy areas where delaying treatments has less severe consequences. For ethical reasons, in oncology trials, it is deemed appropriate to allow cross over once a treatment for an advanced disease such as metastatic cancer has shown benefits in survival. Miksad et al. (2008), for example, shows that in advanced breast cancer, although there is a statistically significant association between both direction and the magnitude of the trial-level treatment effect on PFS and the trial level treatment effect on OS [ ] prediction of OS based on PFS is surrounded by uncertainty (Miksad et al., 2008). This becomes an issue when the new treatment is assessed by HTA/reimbursement bodies to make health care resource allocation decisions, which require long-term, context-specific estimates of clinical and cost effectiveness. Examples of NICE TAs where the issue of cross over occurred include bortezomib for multiple myeloma (NICE STA 129, Oct 2007), lapatinib for advanced or metastatic breast cancer (NICE STA at ACD stage) and bevacizumab, sorafenib, sunitinib and temsirolimus for renal cell carcinoma (NICE MTA at ACD stage). Key questions are therefore: How can we improve the evidence generated in clinical trials, given practical and ethical constraints? Which parties should be involved? How should HTA bodies deal with the high level of uncertainty surrounding the health gains of these medicines? What is the additional evidence that should be considered for reimbursement purposes? With regard to the first set of questions, most of the interviewees agreed that issues related to trial design cannot be solved solely at the UK level. On the second set of questions, a minority of the interviewees thought that the modelling approaches currently used can adequately handle uncertainty. Some emphasised however that, particularly in the case of rarer cancers, a wider range of outcomes and non-rct evidence should be considered. 5.3 Collecting patient reported QoL data In relation to the issue of collecting patient reported outcomes in clinical trials, a variety of views were expressed. Quality of life (QoL) data is collected through standardised instruments (e.g. FACT questionnaire, QLQ-C30) providing an in-depth analysis of harm and benefit of treatments in each individual patient. This data is collected because it shows important aspects related to cancer treatments which might 45

46 not be captured by the standard outcomes such as overall survival. They include disease-free time experienced by patients which can be a relevant outcome in those cancers where the therapeutic aim of the available intervention/s is palliation of symptoms rather than radical cure. One interviewee pointed out that there are practical issues limiting the collection of QoL data in trials, mainly due to difficulties in administering questionnaires to very ill patients and to costs associated with the process. Because of these difficulties the interviewee considered that collecting QoL data is only be useful when two treatments with similar effects in terms of survival but with different side effect profiles are compared (e.g. oral vs intravenous therapy). In the UK, where health related quality of life data (HRQOL) are expressed using generic, preference-based instruments such as EQ-5D, there are concerns related to the paucity of this type of data as discussed in section 3.2 above. The issue here is whether manufacturers should be encouraged to collect HRQOL data using generic instruments which is only requested by a limited number of jurisdictions for reimbursement reasons. 5.4 Translating clinical outcomes into broader health outcomes for HTA All interviewees involved in the HTA process in the UK, mainly in NICE Technology Appraisals (TAs), pointed out that the process of modelling long term outcomes based on the short term outcomes available from clinical trials evidence is increasingly becoming an issue. From a methodological point of view, in each appraisal there is a debate around the model approach and assumptions, which can lead to very different results in the incremental cost effectiveness ratio (ICER). These disagreements are usually discussed and solved on a case by case basis in each appraisal. Two interviewees highlighted that this is not an issue necessarily specific to cancer as it is a recurrent problem in other therapy areas a well. It is simply that there is more awareness in cancer, given the increasing number of medicines appraised for these conditions and the political sensitivity of the disease. From a practical point of view, there are two issues for NICE s appraisal of cancer medicines. One relates to the time spent in discussing modelling issues at a late stage of the appraisal, when more important aspects of the treatments should be considered. This is why one of the interviewees suggested agreeing most model specifications upfront at the scoping stage of the Appraisal. Also it can be very challenging for some of the people around the committee that do not have the right expertise (e.g. Bayesian statistics) to judge what are the most plausible specifications of a model. 46

47 5.5 EQ-5D in cancer Four interviewees thought that EQ-5D was too blunt and limited an instrument to capture small but important changes in quality of life. This is reflected when condition-specific measures (such as QLQ-C30) are translated into EQ-5D. Mapping exercises relying on a regression equation to approximate the relationship have been conducted but more testing is required (one of the observed problems is that the relation works on average but not very well in the extreme cases, i.e. when the patient is well and very unwell). A review of the evidence on mapping from conditionspecific to generic health outcomes measures was conducted by Brazier and colleagues as part of the work of the Office of Health Economics Commission on NHS outcome, performance and productivity (OHE, 2008). The main finding of the review was that the extent to which generic measures correlated with conditionspecific measures was variable and not particularly strong. In nearly all cases the correlation coefficient was less than 0.5, implying that more than half of the variance between individuals generic outcome scores could not be explained by variance between their measured condition-specific outcome scores (OHE, 2008) More research is also needed on the appropriateness of EQ-5D to identify areas where it lacks relevance (e.g. fear, anxiety, concern that might not be captured adequately) due to variability of the description of health states among conditions. 5.6 Appraising cancer medicines Three interviewees argued that there is no consensus as to whether decision makers should give priority to severity. First of all, there are still major issues around the methodology to use to elicit public preferences. Second, empirical evidence is available but it is not mature enough to be translated into either specific QALY weights or guidelines on criteria to be considered by the Appraisal Committee. Finally, it was pointed out that there is a risk of double counting as the QALY already encapsulates a severity element (e.g. if a treatment cures a more severe condition, the QALY gain will be larger than in less severe conditions). Another set of interviewees argued that the current approach for decision making is too health economics-driven. Three indicated that there is a need to involve more people with first-hand experience of the conditions (e.g. patients, clinicians, carers) to help give context and meaning to the clinical and cost effectiveness evidence. They recognised the importance of cost per QALY calculation but they believed that other context-specific elements should be considered alongside it (e.g. side effect profile, unmet clinical need, patients preferences). In other words, they suggested giving more weight to expert and other stakeholders panels with direct experience with the condition in the decision making process. This poses questions about the membership of the NICE Appraisal Committees: how far should resource allocation decisions be informed by expert panels, which might be inclined to push for positive decisions given their perspective. 47

48 5.7 Potential improvements to NICE s approach Some interviewees though that the fundamental problem in the evaluation of cancer drugs is that they produce modest gains in survival compared to gains generated by treatments in other areas, such as those for cardiovascular disease. In their view, the only possible solution is to work on the cost side, i.e. the price of the medicines. The issue here is that there is a need to find a balance between commercial imperative, where manufacturers have to recoup their R&D investment, and the value of the medicine to the patients. Market access options such as risk sharing schemes and value-based pricing were briefly discussed with some of the interviewees. However, they are not further discussed here because the topic was beyond the scope of our study. As an action to improve the process in the short term, two interviewees suggested that better quality of manufacturer submissions would help in reducing appraisal timelines. Some interviewees emphasised that the low level of acceptance of some of the NICE recommendations among the public, the media and the patient community shows that the current systems does not work properly. One thought that a body like NICE should have an advisory role while final deliberations should be made at a political level. However, this would only shift the issues related to the decision-making process rather than solving them. 5.8 Summary In order to collect the view of different parties involved in the process of the assessment and appraisal of cancer medicines, we conducted a survey involving 11 key experts in the field. The main themes discussed with the interviewees were: assessing health benefits of cancer medicines; appraising cancer medicines and incorporating Social Value Judgements (SVJ) in evidence-based decision making processes; and potential solutions. A variety of views were expressed, particularly on the topic of prioritisation of patients with severe diseases, where the majority of the interviewees did not support the option of giving more weight to health gains accrued to these patients in an health care decision making process. Others expressed the view that evidence on cost per QALY is the only visible driver of the current HTA process in England and Wales but a broader perspective should instead be used (e.g. more weight to be given to the views of people with direct experience of the disease in question). A clear consensus emerged among the interviewees on the lack of adequate clinical evidence to inform resource allocation decision at the time of launch, particularly because of the use of surrogate endpoints which are sufficient for marketing authorisation purposes but not for economic assessment informing HTA decisions. 48

49 6 Possible ways forward 6.1 Summary of main findings NICE strongly recommends the use of the QALY as the measure of health outcomes, as set out in its Reference Case. Other HTA bodies also have a preference for cost utility analyses but seem to be less prescriptive over the type of economics analysis. However, when we looked at some practical examples of 10 cancer therapies which have gone through an HTA-led reimbursement process in five different countries, we found that NICE had the highest approval rate without restrictions whilst Finland and Norway had the lowest. These results should be treated with caution as they are based on a small sample of drugs. We have also not been able to systematically analyse approval rates with restrictions because of the variable nature of restrictions. There are substantial differences in the health care systems considered (for example in health care spending, drug prices and the role of HTA in coverage decisions). The group of countries considered includes countries relying on cost effectiveness and therefore does not include four of the five major European countries. Taking the sample of countries using HTA-led reimbursement as whole, it seems that a new cancer drug is more likely to be given either a restricted approval, or rejected for use, than to be approved in the whole licensed population. Although several factors play a role in these decisions, including the high cost of certain cancer therapies, we focus on the assessment and appraisal of health outcomes to see whether current methods are adequate, both from the perspective of the individual health effects that impact on patients and from the perspective of the preferences of society as whole in terms of how it wants to see its health care budget spent. Given the preference for the use of the QALY expressed by NICE, our literature review concentrated on this topic. On the assessment side (where evidence on clinical and cost effectiveness of the treatment/s in question is developed), our literature review shows that the QALY, because of the way it is structured, will have difficulty capturing all of the value of the health gains deemed important by cancer patients. In particular, based on the evidence in the literature, the main issues related to the use of the QALY in cancer are: the sensitivity of the EQ-5D in measuring changes in health status; the validity of the underlying constant proportional trade-off (CPT) assumption used when eliciting health states values with the time trade-off (TTO) method; possible discrepancies between public and patients values of health states. On the appraisal side (where evidence collected through the assessment process is used to inform coverage decisions or use recommendations), there is strong evidence of a societal willingness to place a relatively greater weight on health gains accruing to patients suffering from severe illness, although the magnitude of this willingness still remains to be estimated accurately. Cancer sufferers typically fit this description 49

50 both in terms of severe reductions in quality of life and and in terms of proximity to death, although this may vary depending on the cancer type and the stage of the tumour. Our HTA international comparison demonstrates that many HTA bodies incorporate social value judgments into their decision making process, with some of them using severity as an explicit criterion for prioritisation, although it is not clear what impact this has had in practice. Our interview programme showed that because of the limited clinical evidence available at product launch - primarily data collected in clinical trials often using PFS as an end point and with many patients crossing over from the active to the comparator - estimates of the potential health benefits generated by new oncology medicines are often surrounded by a high degree of uncertainty about how the treatment will in practice add to overall survival and impact on patients quality of life. The reminder of this chapter of the report discusses possible ways forward, drawing on the points made in the earlier sections of the report. This includes further research that might be conducted to impact over the longer term and potential modifications to the NICE s current approach which might be implemented in the short term. The section concludes by briefly discussing how these recommendations fit alongside the recently published supplementary guidance on Appraising life-extending, end of life treatments (NICE, 2009). 6.2 Changes in the QALY descriptive system NICE prefers the EQ-5D as the generic preference-weighted health-related quality of life measure. However, our review suggests that one of the deficiencies of the EQ-5D is that it has no domain to capture changes in vitality or energy. The EQ-5D group is itself considering whether the instrument should be modified in this regard. Our literature review also indicates the need for further research looking into potential modifications of the EQ-5D with additional levels. However, the advantages of having a more sensitive instrument will have to be traded-off with methodological and practical issues related to the increased complexity of the instrument (i.e. it will have a very high number of potential individual health states). In the meantime, it might be advisable to explore the use of the SF-6D in the evaluation of cancer therapies, in order to assess whether this instrument is more sensitive to change as some studies suggest. Our interview programme and review of NICE TAs also shows that data on quality of life available from clinical trials is not usually measured with EQ-5D. More research is needed to validate methods to convert non-utility-based quality of life scores (such as QLQ-C30) into utility-based ones (such as EQ-5D). Where this is not feasible, the NICE Appraisal Committees should consider available data on quality of life (often measured using disease-specific instruments) in addition to any EQ-5D data to inform their decision-making. 50

51 6.3 Changes in the weighting of QALYs (from an individual perspective) Our literature review suggests that because of the approach applied to generate health state scores the QALY may not be capturing differences in health status that are deemed important by individual patients. In particular, EQ-5D defines hypothetical health states that are valued by members of the general public, who are asked to assess these values from their own perspective (i.e. they have to imagine being in those health states). In the context of oncology treatments, it might be difficult to read across from this to reflect the situation faced by terminally-ill patients. For example, individuals might value life years gained from interventions at the end of their lives more highly than those gained at an earlier stage of life. Namely, if someone has been told that they only have six months to live, gaining an extra two months might be worth a lot more to them than would a two-month gain if they had five years to live (over and above any discounting arising from the timing of future health effects.) As currently calculated a QALY valuation of health effects would not reflect this. Therefore, more research is required to test both whether patients with cancer or other conditions attach more value to survival gains when their life expectancy is short and to find the point up to which they are willing to trade quality of life for survival benefits. This could provide the basis for a weighting system that would better reflect the individual utility of health states. One approach might be to undertake discrete choice experiments, in two sets of individuals: one group with a diagnosis of end-stage cancer, the other with normal life expectancy. This might give an estimate of the different value of a QALY gained. Alternatively, the same respondents (i.e. members of the general public) could be asked to imagine the two alternative scenarios. The latter approach would be more consistent with NICE s use of the general public s valuations of a generic instrument to estimate QALYs. However, it would require the respondents to be well informed about the choices they were being asked to make. It is clear from the literature that there are other aspects of patient experience (such as the extent to which they do or do not adapt to conditions or other aspects of end of life care) which may not be reflected in valuations of health states undertaken by the general public. We do not think it appropriate at this stage to depart from using the valuations of the general public. However, more research is required in this area. 6.4 Weighting of QALYs (from a societal perspective). How to incorporate social value judgements into HTA In terms of the resource allocation decisions made in the NICE appraisal process, under the current approach the starting point is that QALYs are equally weighted no matter to whom they accrue. NICE itself recognises this is no more than a default position. It is understandably unsure how to apply different weightings. It has been argued that for a particular value judgement to be incorporated into the HTA process that it must demonstrate social legitimacy (Claxton and Culyer, 2008) i.e. there must 51

52 be systematic evidence of societal support for the deviation from the QALY is a QALY is a QALY rule. This makes sense. So although the case for weighting QALYs differently is generally accepted to be quite strong the main problem is how the weights might be determined in a way that meets the test of social legitimacy The recent report by the NICE Citizen s Council (2008), and the results of the research commissioned by NICE through the NCCHRM (Donaldson et al and Dolan et al, 2008) are far from conclusive, but do suggest that severity of disease and lack of alternative treatments are two factors that the general public feel should be taken into account in allocating healthcare resources. End stage cancer therapy meets both of these criteria. The challenge is to incorporate societal preferences into the HTA decision making process. There are two main options: 1. developing an algorithm which reflects equity weights, based on observed societal preferences. This could be a weighting system to directly modify the cost per QALY calculation or to adjust the cost per QALY threshold; 2. feeding the results of empirical studies into appraisal decision making as a part of a deliberative process. Both types of adjustment are not without problems. Adjusting the QALY or the threshold in relation to a particular disease or group of patients, could appear arbitrary unless well grounded in evidence of societal preferences. It would not necessarily be consistent with what the cost per QALY threshold represents for NICE (i.e. opportunity cost of programmes displaced by new, more costly technology (NICE, 2008a)). In this context, the threshold is not intended to represent society s willingness to pay for health and so logically could not be varied according to disease categories or other factors as if it does. However, we can assume that these adjustments would reflect relative differences in the opportunity costs that society expects to see applied at the margins of NHS expenditure. In terms of QALY-weighting system, a robust enough methodology to be used to modify the current NICE methods has not been indicated yet. Ongoing research exploring the monetary value of a QALY in EU member states may shed light on this issue (EuroVAQ, 2008). Another unresolved issue around formally weighting QALYs is the potential for double counting preferences, from an individual perspective as discussed in the previous section and from a society s perspective. For example, a society may attach a high weight to treating people on the basis of disease severity not realising the extent to which such preferences are already reflected in individual health states valuations. In the absence of an agreed weighting system, these preferences can be fed into decision-making in the appraisal through the deliberative process. NICE states that factors other than the ICER are already considered in its deliberative decision-making process. It can cite examples where departures from the threshold cost per QALY ratio have been made, such as in the case of imitanib for chronic myeloid leukaemia (Rawlins and Culyer, 2004). Nevertheless, our view is that NICE would benefit from 52

53 being more rigorous and explicit in its application of deliberative decision-making. This does not mean replacing the judgement of Appraisal Committee members. It is intended rather to make it more explicit to the members and subsequently to the public, what the Committee thought was important and what it did not. For example, explicit criteria for departure from the standard threshold could be developed. In addition, research could be conducted, either with the general public or NICE s Citizens Council, to establish indicative relative weights for the various criteria which the Appraisal Committee could consider. Alternatively, NICE guidance could routinely include a standard checkbox listing the factors considered this could be common for all guidance, and include severity of disease as one item together with a brief indication of their impact (if any) on the recommendations. Overall, the deliberative decision-making process would be the most flexible way of accommodating the concerns over the appraisal of cancer drugs expressed in this document. However, it needs to be much more rigorous and to be seen to be so. A further possibility would be to develop the use of multi-criteria decision analysis (MCDA) methods to assist NICE Appraisal Committees in their deliberative processes. MCDA is a methodology for appraising options. It involves the trading off of a number of (often conflicting) priority-setting concerns and objectives against each other, and is particularly useful when these objectives do not share a common unit of valuation for example, a mixture of health, monetary, distributional and political objectives (Baltussen and Niessen, 2006). 6.5 Attending to data deficiencies Much of the uncertainty about the benefits offered by new cancer drugs, such as the reliability of modelled projections of benefit, is no different in nature from that existing in other disease areas. However, the current uncertainties surrounding projections from progression-free survival (PFS) to overall survival (OS) need particular attention and are focused on cancer treatments. The principal reasons for these uncertainties are differences between the approaches of licensing bodies and HTA bodies. Many drug licensing agencies, such as the EMEA and the FDA, accept PFS as a legitimate indicator of benefit and manufacturers and clinical researchers are sometimes obliged to terminate trials early on ethical grounds. NICE and the Medicines and Healthcare products Regulatory Agency (MHRA) could hold discussions to increase joint understanding as to those instances in which PFS is a good predictor of OS and those in which it is not. It may also be possible to consider the sorts of quality of life measures that might be applicable, given an understanding of the extent to which they are consistent with generic instruments to derive QALYs and also to specify acceptable modelling approaches for those instances in which only PFS data are available. Advice on these matters could be reviewed with companies through the consultancy process that NICE has recently established. However, as reliance upon these data is common amongst reimbursement agencies across the developed world, a co-ordinated and international approach may be the best way to tackle these issues. It is also clear that if data cannot realistically be collected prelaunch then consideration should be given to collecting it post-launch. 53

54 6.6 The recent NICE supplementary advice on life-extending, end of life treatments As described earlier in this paper, NICE has recently decided to amend the advice it gives to its Appraisal Committees, asking them to take into account other factors alongside cost-effectiveness when apprising end-of-life medicines with cost per QALY ratios exceeding 30,000 (NICE, 2009). Although the consultation does not explain the reasons why it is important to apply special criteria when appraising this set of medicines (e.g. it does not recognise for example that the QALY may not reflect the value of important health benefits in severe diseases), the document goes some way towards improving the clarity surrounding prioritisation criteria. It indicates circumstances in which the Appraisal Committee should apply greater weights to QALYs for interventions in end of life conditions, which are consistent with our findings. However, the lack of a detailed rationale in NICE s paper means there may be difficult issues to resolve as to the magnitude of these weights and the basis on which to apply them. The criteria that a medicine would have to meet in order to justify higher cost per QALY ratios and be recommended for use by the Appraisal Committee appear to be arbitrary in certain aspects. In particular, no evidence is provided to support the cutoff point of 24 months life expectancy as a definition of a terminal illness. Furthermore, the inclusion only of treatments for small populations may reflect the definition of rare diseases, but it is not clear why the two elements (end of life and orphan numbers) have been combined. Drugs for other rare and severe conditions will not be given extra QALY weights as they are excluded from the prioritisation criteria. The expectation that additional data will be collected on the outcomes achieved in practice through use of the treatment helps to meet the concerns identified in our interview programme about the lack of sufficient pre-launch data on outcomes. A programme of evidence development could be designed in order to collect outcomes data (both survival and quality of life) in routine practice. However, it still unclear how the additional evidence, such as estimates of the extension of life, will be reviewed and the new criteria applied by the Appraisal Committee. In other words, is not at all clear how the deliberative process will work. Overall the consultation makes proposals that can pragmatically address a number of the concerns highlighted by the evidence we have reviewed for this report. However, without greater clarity as to the rationale for the approach and about how it will be applied it is difficult to form a definitive view on the advice and it may be difficult for the Appraisal Committees to apply the advice in practice. It is noteworthy that NICE includes a suggestion for further methodological research to provide more robust basis for the application of the advice. Our report indicates areas where such research could usefully be undertaken. 54

55 References Abelson, J., Lomas, J., Eyles, J., Birch, S., and Veenstra, G. (1995) Does the community want devolved authority? Results of deliberative polling in Ontario. Canadian Medical Association Journal, 153, Antonanzas, F. (2003) Challenges to achieving value in drug spending in a decentralized country: The Spanish case. Value in Health, 6, S52-S63. Baker, R., Thompson, C., and Mannion, R. (2006) Q methodology in health economics. Journal of Health Services Research and Policy, 11, Barry, M., and Tilson, L. (2007) Recent developments in pricing and reimbursement of medicines in Ireland. Expert Review of Pharmacoeconomics and Outcomes Research, 7, Baltussen, R., and Niessen, L. (2006) Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effectiveness and Resource Allocation, 2006, 4-14 Barham, L. (2008) Single technology Appraisals by NICE. Are they delivering faster guidance to the NHS? Pharmacoeconomics, 26(12), Barton, G., Sach, T., Doherty, M., Avery, A., Jenkinson, C., and Muir, K. (2008) An assessment of the discriminative ability of the EQ-5D, SF-6D, and EQ VAS, using sociodemographic factors and clinical conditions. European Journal of Health Economics, 2008, 9, Bharmal, M., and Thomas, J. (2006) Comparing the EQ-5D and the SF-6D Descriptive Systems to Assess Their Ceiling Effects in the US General Population. Value in Health, 9(4), Bleichrodt, H., Pinto, J.L., and Abellan-Perpinan J.M. (2003) A consistency test of the time trade-off. Journal of Health Economics, 22, Brazier, J. (2008) Valuing health states for use in cost-effectiveness analysis. Pharmacoeconomics, 26(9), Broeckel, A., Jacobsen, PB., Horton, J., Balducci, L., and Lyman, G.H. (1998) Characteristics and correlates of fatigue after adjuvant chemotherapy for breast cancer. Journal of Clinical Oncology, 16(5), Brouwer, W., Culyer, A.J., Job, N., van Exel, A., and Rutten, F.H. (2008) Welfarism vs. extra-welfarism. Journal of Health Economics, 27(2), Buckingham, K., and Devlin, N. (2008) A note on the nature of utility in time and health and implications for cost utility analysis. Discussion paper series no Available at: 55

56 Cancer Research UK - accessed 21 November 2008 Carlsson, P. (2004) Health technology assessment and priority setting for health policy in Sweden. International Journal of Technology Assessment in Health Care, 20, Cella, D., Eton, D.T., Lai J.S., Peterman, A.H., and Merkel, D.E. (2002) Combining anchor and distribution-based methods to derive minimal clinically important differences on the Functional Assessment of Cancer Therapy (FACT) anemia and fatigue scales. Journal of Pain Symptom Management, 24(6), Chancellor, J.V.M., Coyle, D., and Drummond, M.F. (1997) Constructing health state preference values from descriptive quality of life outcomes: mission impossible? Quality of Life Research, 6, Claxton, K., and Culyer, A.J. (2008) Not a NICE fallacy: a reply to Dr Quigley. Journal of Medical Ethics, 34, Conner-Spady, B.L., Cumming, C., and Nabholtz, J.M. (2005) A longitudinal prospective study of health-related quality of life in breast cancer patients following high-dose chemotherapy with autologous blood stem cell transplantation. Bone Marrow Transplant, 36(3), Cookson, R., and Dolan, P. (1999) Public views on healthcare rationing: a group discussion study. Health Policy, 49, Cross, J., and Garrison, L. (2008) Challenges and opportunities for improving benefitrisk assessment of pharmaceuticals from an economic perspective. OHE briefing available at: e=0 Culyer, A.J. (1992) The morality of efficiency in health care--some uncomfortable implications. Health Economics, 1, 7-18 Daniels, N. (2000) Accountability for reasonableness. British Medical Journal, 323 (7272), Daniels, N., and Sabin, J.E. (2002) Setting limits fairly: Can we learn to share medical resources? Oxford University Press. Department of Health (2003) Tackling health inequalities: a programme for action. London: DoH 56

57 Department of Health (2007) Cancer Reform Strategy. Available at: dguidance/dh_ Department of Health (2008) 2009 Pharmaceutical Price Regulation Scheme. Available at: DLTR (2000) Multi-criteria Analysis Manual. Available at: (accessed 27 November 2008) Dobrez, D., Cella, D., Pickard, S., Lai, J., and Nickolov, A. (2007) Estimation of patient preference based utility weights from the functional assessment of cancer therapy-general. Value in Health, 10,4, Dolan, P. (1998) The measurement of individual utility and social welfare. Journal of Health Economics, 17, Dolan, P. (2001) Utilitarianism and the measurement and aggregation of qualityadjusted life years. Health Care Analysis, 9, Dolan, P., and Cookson, R. (2000) A qualitative study of the extent to which health gain matters when choosing between groups of patients. Health Policy, 51, Dolan, P., Edlin, R., and Tsuchiya, A. (2008) The relative societal value of health gains to different beneficiaries. Available at: ial_value_qaly_final_report_paul_dolan_et_al_2008.pdf. Dolan, P., Gudex, C., Kind, P., and Williams, A. (1995) A social tariff for EuroQoL: results from a UK general population survey. Centre of Health Economics Discussion Paper, 138 Dolan, P. (1996) Modelling valuations for health states: the effect of duration. Health Policy, 38, Dolan, P., and Green, C. (1998) Using the person trade-off approach to examine differences between individual and social values. Health Economics, 7, Dolan, P., and Kahneman, D. (2008) Interpretations of utility and their implications for the valuation of health. The Economic Journal, 118(525), Dolan, P., and Shaw, R. (2004) A note on a discussion group study of public preferences regarding priorities in the allocation of donor kidneys. Health Policy, 68, Dolan, P., Shaw, R., Tsuchiya, A., and Williams, A. (2005) QALY maximisation and people s preferences: a methodological review of the literature. Health Economics, 14,

58 Dolan, P., and Stalmeier, P. (2003) The validity of time trade-off values in calculating QALYs: constant proportional time trade-off versus the proportional heuristic. Journal of Health Economics, 22, Dolan, P., and Tsuchiya, A. (2005) Health priorities and public preferences the relative importance of past health experience and future health prospects. Journal of Health Economics, 24, Donaldson, C. (2008) Weighting and valuing quality adjusted life years: preliminary results from the Social Value of a QALY Project. Available at: html Drummond, M.F., Sculpher, M.J., Torrance, G.W., O Brien, B.J., and Stoddart, G.L. (2005) Methods for the Economic Evaluation of Healthcare Programmes. Oxford University Press. Esbensen, B.A., Osterlind, K., Roer, O., and Hallberg, I.R. (2004) Quality of life of elderly persons with newly diagnosed cancer. European Journal of Cancer Care, 13, EuroVAQ (2008). European Value of a Quality Adjusted Life Year. Feeny, D.H. (2005) The roles for preference-based measures in support of cancer research and policy. In Lipscomb, Gotay, and Snyder (2005) Outcomes assessment in cancer: measures, methods, and applications. Cambridge University Press Fryback, D.G., and Lawrence, W.F. (1997) Dollars may not buy as many QALYs as we think: a problem with defining quality-of-life adjustments. Medical Decision Making, 17, Gold, M.R., Siegel, J.E., Russell, L.B., and Weinstein, M.C. (1996) Cost- Effectiveness in Health and Medicine. Oxford University Press. Goossens, L. (2006) 80,000 per QALY is the limit. imta Newsletter (October), 4, 1-3. Gotay, C.C. (2004) Assessing Cancer-Related Quality of Life Across a Spectrum of Applications. JNCI Monographs, 33, 126 Accessed on 15th November 2008 at: RY=0 Grieve, R., Grishchenko, M., and Cairns, J. (2008) SF-6D versus EQ-5D: reasons for differences in utility and impact on reported cost-utility. European Journal of Health Economics, 2008, Feb 58

59 Gudex, C. (1994) Time trade-off user manual: props and self-completion methods. Centre of Health Economics. Accessed on 15th November 2008 at Gyrd-Hansen, D. (2004) Investigating the social value of health changes. Journal of Health Economics 23: Hill, S., Garattini, S., van Loenhout, J. (2003). Technology appraisal programme of the National Institute for Clinical Excellence. A review by WHO. Copenhagen: WHO Regional Office for Europe, June-July Hofman, M., Ryan, J.L., Figueroa-Moseley, C.D., Jean-Pierre, P., and Morrow, G. (2007) Cancer-related fatigue: the scale of the problem. The Oncologist, 12 (suppl 1), 4-10 Jacobsson, F., Carstensen, J., and Borgquist, L. (2005) Caring externalities in health economic evaluation: how are they related to severity of illness? Health Policy, 73, Jaeschke, R., Singer, J., and Guyatt, G.H. (1989) Measurement of health status. Ascertaining the minimal clinically important difference. Controlled Clinical Trials, 10, Janssen, M.F., Birnie, E., and Bonsel, G.J. (2008a) Quantification of the level descriptors for the standard EQ-5D three-level system and a five-level version according to two methods. Quality of Life Research, 17, Janssen, M.F., Birnie, E., Haagsma, J.A., and Bonsel, G.J. (2008b) Comparing the Standard EQ-5D Three-Level System with a Five-Level Version. Value in Health, 11(2), Kind, P., and Macran, S. (2005) Eliciting social preferences weights for functional assessment of cancer therapy-lung health states. Pharmacoeconomics, 23(11), Kind, P. (2007) Size Matters: EQ-5D in Transition. Medical Care, 45(9), Kirby, J. (2008) Prescription charges to be axed for cancer patients. The Independent (23rd September), London Krabbe, P.F., Peerenboom, L., and Langenhoff, B.S. (2004) Responsiveness of the generic EQ-5D summary measure compared to the disease-specific EORTC QLQ C- 30. Quality of Life Research, 13(7), Krahn, M., Bremner, K.E., Tomlinson, G., Ritvo, P., and Irvine, J. (2007) Responsiveness of disease-specific and genric utility instruments in prostate cancer patients. Quality of life research, 16,

60 Lamers, L.M., Groot, C.A. U-d., and Buijt, I. (2007) The use of disease-specific outcome measures in cost-utility analysis. Pharmacoeconomics, 25 (7), Lipscomb, J., Gotay, C.C., and Snyder, C.F. (2007) Patient-reported outcomes in cancer: a review of recent research and policy initiatives. CA: A Cancer Journal for Clinicians, 57, Mann, R., Brazier, J., and Tsuchiya, A. (2008) A comparison of patient and general population weightings of EQ-5D dimensions. Health Economics, in print. Martikainen, J., and Rajaniemi, S. (2002) Drug reimbursement systems in EU Member States, Iceland and Norway, The Social Insurance Institution, Finland, Helsinki. Mason, A., and Drummond, M. (2007a) European perspective on the costs and costeffectiveness of cancer therapies. Journal of Clinical Oncology, 25(2) Mason, A., and Drummond, M. (2007b) In reply to Many new cancer drugs in the United Kingdom are facing negative NICE rulings by Eric Low Miksad, R., Zietemann, V., Gothe, R., Schwarzer, R., Conrads-Frank, A., Schnell- Inderst, P., Stollenwerk, B., and Siebert, U. (2008) Progression-free survival as a surrogate endpoint in advanced breast cancer. International Journal of Technology Assessment in Health Care, 24(4), Murphy, B.A., Ridner, S., Wells, N., and Dietrich, M. (2007) Quality of life research in head and neck cancer: a review of the current state of the science. Critical Reviews in Oncology/Hematology, 62, Miyamoto, J.M., and Eraker, S.A. (1988) A multiplicative model of the utility of survival duration and health quality. Journal of Experimental Psychology, General, 117, 3-20 NCI (2008) Progression-free Survival: Patient Benefit or Lower Standard? National Cancer Institute Bulletin, 5(10) NICE (2001) Guidance on the use of Riluzole (Rilutek) for the treatment of motor neurone disease. (TA no.20) Available at: NICE (2003) Guidance on the use of imatinib for the treatment chronic myeloid leukaemia. (TA no.70) Available at: NICE (2004) Citizens Council Report on Age. Available at: NICE (2005) Press release. NICE plans faster drugs guidance for the NHS. Available at: 60

61 05/2005_021_nice_plans_faster_drugs_guidance_for_the_nhs.jsp NICE (2008a) Guide to the methods of technology appraisal. Available at: NICE (2008b) Social value judgements: principles for the development of NICE guidance (second edition). NICE: London. Available at: NICE (2008c) Press release 2008/066: NICE announces measures on end of life medicines. Available at NICE (2008d) Citizens Council Report: Quality Adjusted Life Years (QALYs) and the severity of illness. Available at: NICE (2008e) Renal cell carcinoma - bevacizumab, sorafenib, sunitinib and temsirolimus: appraisal consultation document. Available at: NICE (2009) Appraising life-extending, end of life treatments. Available at: omedia=1&mid=88acdae5-19b9-e0b5-d a8ec6d Nikolentzos, A., Nolte, E. and Mays, N. (2008) Paying for (expensive) drugs in the statutory system: an overview of experiences in 13 countries, A report by the London School of Hygiene & Tropical Medicine. Commissioned and funded by the Department of Health, London. Nord, E. (1993a) The trade-off between severity of illness and treatment effect in cost-value analysis of health care. Health Policy 24: Nord, E. (1993b) Health politicians do not wish to maximize health benefits (original in Norwegian). Journal of the Norwegian Medical Association 113: Nord, E. (1995) The person trade-off approach to valuing healthcare programs. Medical Decision Making 15: Nord, E. (1996) A table of values for cost-effectiveness analysis in healthcare (original in Norwegian). Journal of the Norwegian Medical Association 27: Nord, E., Wisloff, F., Hjorth, M., and Westin, J. (1997) Cost-utility analysis of melphalan plus prednisone with or without interferon alpha 2B in newly diagnosed multiple myeloma. Pharmacoeconomics, 12,

62 Nord, E. (1999) Cost-value analysis in healthcare: making sense out of QALYs. Cambridge University Press. Nord, E. (2001a) Severity of illness versus expected benefit in societal evaluation of health care interventions. Expert Review of Pharmacoeconomics and Outcomes Research,1, Nord, E. (2001b) Health state values from multiattribute utility instruments need correction. Annals of Medicine, 33, Nord, E., Pinto Prades, J.L., Richardson, J., Menzel, P., and Ubel, P. (1999) Incorporating societal concerns for fairness in numerical valuations of health programmes. Health Economics, 8, Nord, E., Richardson, J., Street, A., Kuhse, H., and Singer, P. (1995) Who cares about cost? Does economic analysis impose or reflect social values? Health Policy, 34, Nord, E. (2008) The popular value of a QALY: some comments to two reports for NICE. Presentation at The Social Value of a QALY Dissemination Day - 4th Septembe 2008 Welcome Trust, London Unpublished Observatory studies series no. 11, WHO Regional Office for Europe, Copenhagen. Oliver, A. (2004) Prioritizing Health Care: Is "Health" Always an Appropriate Maximand? Medical Decision Making 24: OHE (2008). Report to the Office of Health Economics Commission on NHS outcomes, performance and productivity. Available at: Osoba, D. (2005) The clinical value and meaning of health-related quality-of-life outcomes in oncology. In Lipscomb, Gotay, and Snyder (2005) Outcomes assessment in cancer: measures, methods, and applications. Cambridge University Press Pedrosa Vasco, J. and Alves da Silva, E. (2001) Portugal: Pharmaceutical Pricing and Reimbursement. Available at: PHARMAC (2005) Guidelines for Submissions to PHARMAC and PTAC on New Chemical Entities and Generic Pharmaceuticals. PHARMAC (2007) Prescription for Pharmacoeconomic Analysis (PFPA): methods for cost-utility analysis, PHARMAC, Wellington. Phillips, L.D., and Costa, Cab. (2007) Transparent prioritisation, budgeting and resource allocation with multi-criteria decision analysis and decision conferencing. Annals of Operations Research, 154 (1),

63 Pickard, A.S., De Leon, M.C., Kohlmann, T., Cella, D., and Rosenbloom, S. (2007a) Psychometric Comparison of the Standard EQ-5D to a 5 Level Version in Cancer Patients. Medical Care, 45(3), Pickard, A.S., Kohlmann, T., Janssen, M.F., Bonsel, G., Rosenbloom, S., and Cella, D. (2007b) Evaluating equivalency between response systems: application of the Rasch Model to a 3-level and 5-level EQ-5D. Medical Care, 45(9), Pickard, A.S., Wilke, C.T., Lin, H-W., Lloyd, A. (2007c) Health utilities using the EQ-5D in studies of cancer. Pharmacoeconomics, 25(5), Pickard, A.S., Neary, M.P., and Cella, D. (2007d) Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health and Quality of Life Outcomes, 5(70), Dec 2007 Pliskin, J.S., Shepard, D.S., and Weinstein, M.C. (1980) Utility functions for life years and health status. Operations Research, 28, Raftery, J. P. (2008) Paying for costly pharmaceuticals: regulation of new drugs in Australia, England and New Zealand. Medical Journal of Australia, 188, Rawlins, M.D., and Culyer, A.J. (2004) National Institute for Clinical Excellence and its value judgments. BMJ, 329(7459), Rawls, J. (1972) A Theory of Justice. Oxford University Press. Richards, M. (2008) Improving access to medicines for NHS patients: a report for the Secretary of State for Health by Professor Mike Richards. Department of Health. Available at: dguidance/dh_ Richardson, J. (2007) Critique and some recent contributions to the theory of cost utility analysis. Working Paper 77, Melbourne: Centre for Health Program Evaluation. Roberts, T., Bryan, S., Heginbotham, C., and McCallum, A. (1999) Public involvement in health care priority setting: an economic perspective. Health Expectations, 2, Ryan, M. (2004) Discrete choice experiments in health care. BMJ 328: Sacket DL and Torrance GW (1978) The utility of different health states as perceived by the general public. Journal of Chronic Diseases, 31, Sacket, D.L., and Torrance, G.W. (1978) The utility of different health states as perceived by the general public. Journal of Chronic Disease, 31,

64 Schlander, M. (2008) The use of cost-effectiveness by the National Institute for Health and Clinical Excellence (NICE): no(t yet an) exemplar of a deliberative process. Journal of Medical Ethics, 34, Schwappach, D.L. (2002) Resource allocation, social values and the QALY: a review of the debate and empirical evidence. Health Expectations, 5, Scottish Medicines Consortium (2004) Guidance to manufacturers: Question and answer document on economic submissions to the Scottish Medicines Consortium, SMC, Glasgow. Scottish Medicines Consortium (2007) Guidance to Manufacturers for Completion of New Product Assessment Form (NPAF), SMC, Glasgow. Sorensen, C., Drummond, M., and Kanavos, P. (2008) Ensuring value for money in health care: the role of health technology assessment in the European Union. Stalmeier, P.F.M., Wakker, P.P., and Bezembinder, T.G.G. (1997) Preference reversals: violations of unidimensional procedure invariance. Journal of Experimental Psychology: Human Perception and Performance, 23, Stiggelbout, A.M., and de Haes, J.C.J.M. (2001) Patient Preference for Cancer Therapy: An Overview of Measurement Approaches. Journal of Clinical Oncology, 19, Stiggelbout, A.M., Kiebert, G.M., Kievit, J., Leer, J.W.H., Habbema, J.D.F., and De Haes, J.C.J.M. (1995) The utility of the time trade-off method in cancer patients: feasibility and proportional trade-off. Journal of Clinical Epidemiology, 48(10), Stolk, E.A., Pickee, S.J., Ament, A.H.J.A., and Busschback, J.J.V. (2005) Equity in healthcare prioritisation: an empirical inquiry into social value. Health Policy, 74, Stone, P., Ream, E., Richardson, A., Thomas, H., Andrews, P. Campbell, P., Dawson, T., Edwards, J., Goldie, T., HammIck,M., Kearney, N., Lean, M., Rapley, D., Smith, A.G., Teague, C., and Young, A. (2003) Cancer-related fatigue a different of opinion? European Journal of Cancer Care, 12(1), Sullivan, P.W., and Lawrence, W.F. (2005) A national catalog of preference-based scores for chronic conditions in the United States. Medical Care, 43(7), Tengs, T.O., Wallace, A. (2000) One Thousand Health-Related Quality-of-Life Estimates. Medical Care, 38(6), Tsiachristas, A., Goudriaan, R., and Groot, W. (2008) The welfare effects of innovative pharmaceuticals: A pilot study for the Netherlands, Aarts De Jong Wilms Goudriaan Public Economics bv (APE) and Maastricht University, Den Haag. 64

65 Tsuchiya, A., and Dolan, P. (2005) The QALY model and individual preferences for health states and health profiles over time: a systematic review of the literature Ubel, P.A. (1999) How stable are people s preferences for giving priority to severely ill patients? Social Science & Decision Making, 49, Ubel, P.A., Dekay, M.L., and Asch, D.A. (1998) Public preferences for prevention versus cure: What if an ounce of prevention is worth only an ounce of cure? Medical Decision Making, 18, Ubel, P.A., Loewenstein, G., and Jepson, C. (2003) Whose quality of life? A commentary exploring discrepancies between health state evaluations of patients and the general public. Quality of Life Research, 12, , Ubel, P.A., Richardson, J., and Menzel, P. (2000) Societal value, the person trade-off, and the dilemma of whose values to measure for cost-effectiveness analysis. Health Economics, 9, van Nooten, F. E., Koolman, X., and Brouwer, W. B. F. (2008) The influence of subjective life expectancy on health state valuations using a 10 year TTO. Health Economics, 2008, Aug Waugh, N., and Scott, D (1998) How should different life expectancies be valued? British Journal of Medicine, 316, 1316 Waugh, N. (2006) Health technology assessment in cancer: a personal view from public health. European Journal of Cancer, 42, Wedding, U., Pientka, L., and Hoffken, K. (2004) Quality-of-life in elderly patients with cancer: a short review. European Journal of Cancer, 43, Williams, A. (1995) The role of the EuroQoL Instrument in QALY calculations. Centre of Health Economics Discussion Paper 130 Williams, A. (1997) Intergenerational equity: an exploration of the fair innings argument. Health Economics, 6, Wolfs, C.A.G., Dirksen, C.D., Kessels, A., Willems, D.C.M., Verhey, F.R.J., and Severens, J.L. (2007) Performance of the EQ-5D and the EQ-5D+C in elderly patients with cognitive impairments. Health and Quality of Life Outcomes, 5(33), 1-10 Woods, K. (2002) Health technology assessment for the NHS in England and Wales. International Journal of Technology Assessment in Health Care, 18 (2), Wu, E.Q., Mulani, P., Farrell, M.H., and Sleep, D. (2007) Mapping FACT-P and EORTC QLQ-C30 to patient health status measured by EQ-5D in metastatic hormone-refractory prostate cancer patients. Value in Health, 10(5),