Roflumilast for the management of severe chronic obstructive pulmonary disease

Transcription

1 in collaboration with: Roflumilast for the management of severe chronic obstructive pulmonary disease Produced by Authors Correspondence to Kleijnen Systematic Reviews Ltd. in collaboration with Erasmus University Rotterdam and Maastricht University Rob Riemsma, Reviews Manager, KSR Ltd. Stefan K. Lhachimi, Health Economic Researcher, EUR Nigel Armstrong, Health Economist, KSR Ltd. Thea van Asselt, Health Economist, Maastricht UMC Alex Allen, Information Specialist, KSR Ltd. Nathan Manning, Systematic Reviewer, KSR Ltd. Julie Harker, Systematic Reviewer, KSR Ltd. Doreen Allen Tushabe, Systematic Reviewer, KSR Ltd. Johan L. Severens, Professor of Evaluation in Health Care, EUR. Jos Kleijnen, Director, KSR Ltd., Professor of Systematic Reviews in Health Care, Maastricht University. Rob Riemsma, Kleijnen Systematic Reviews Unit 6, Escrick Business Park Riccall Road, Escrick York, UK YO19 6FD Date completed 20/07/2011 1

2 Source of funding: This report was commissioned by the NIHR HTA Programme as project number 10/06/01 STA. Declared competing interests of the authors None. Acknowledgements Rider on responsibility for report The views expressed in this report are those of the authors and not necessarily those of the NIHR HTA Programme. Any errors are the responsibility of the authors. This report should be referenced as follows: Riemsma R, Lhachimi SK, Armstrong N, Van Asselt, ADI, Allen A, Manning N, Harker J, Tushabe DA, Severens JL, Kleijnen J. Roflumilast for the management of severe chronic obstructive pulmonary disease: a Single Technology Appraisal. York: Kleijnen Systematic Reviews Ltd., Contributions of authors Rob Riemsma acted as project lead and systematic reviewer on this assessment, critiqued the clinical effectiveness methods and evidence and contributed to the writing of the report. Stefan K. Lhachimi acted as health economic researcher on this assessment, critiqued the manufacturer s economic evaluation, conducted new economic analyses and contributed to the writing of the report. Nigel Armstrong acted as health economist on this assessment, critiqued the manufacturer s economic evaluation, critiqued the meta-analysis and conducted new WinBUGS analyses and contributed to the writing of the report. Thea van Asselt acted as health economist on this assessment, critiqued the manufacturer s economic evaluation, and contributed to the writing of the report. Alex Allen critiqued the search methods in the submission and contributed to the writing of the report. Nathan Manning acted as systematic reviewer, critiqued the clinical effectiveness methods and evidence and contributed to the writing of the report. Julie Harker acted as systematic reviewer, critiqued the clinical effectiveness methods and evidence and contributed to the writing of the report. Doreen Allen Tushabe acted as systematic reviewer, critiqued the clinical effectiveness methods and evidence and contributed to the writing of the report. Johan L. Severens critiqued the manufacturer s definition of the decision problem and their description of the underlying health problem and current service provision, contributed to the writing of the report and supervised the health economic sections of the report. Jos Kleijnen critiqued the manufacturer s definition of the decision problem and their description of the underlying health problem and current service provision, contributed to the writing of the report and supervised the project. 2

3 Abbreviations CEAC CI CrI CHMP COPD EQ-5D ERG ERS ICS FEF FEV FVC GOLD HRQoL ICER ISPOR LABA LAMA MS MTC NHS EED NICE NIHR PDE-4 inhibitor PICO PSA QALY RR SABA SAMA SGRQ SOBQ SPC TDI WTP Cost-Effectiveness Acceptability Curves Confidence Interval Credibility Interval Committee for Medicinal Products for Human Use Chronic Obstructive Pulmonary Disease Euroqol Measure of Health Questionnaire Evidence Review Group European Respiratory Society Inhaled Corticosteroid Forced Expiratory Flow Forced Expiratory Volume Forced Vital Capacity Global Initiative for Chronic Obstructive Lung Disease Health Related Quality of Life Incremental Cost-Effectiveness Ratio International Society for Pharmacoeconomics and Outcomes Research Long Acting Beta-Adrenoceptor Agonist Long acting Muscarinic-receptor Antagonist Manufacturer s Submission Mixed Treatment Comparison NHS Economic Evaluation Database National Institute for Health and Clinical Excellence National Institute for Health Research Phosphodiesterase Type 4 Inhibitor Population, Interventions, Comparators and Outcomes Probabilistic Sensitivity Analysis Quality Adjusted Life Year Rate Ratio Short Acting Beta-Adrenoceptor Agonist Short Acting Muscarinic -receptor Antagonist St. Georges Respiratory Questionnaire Shortness of Breath Questionnaire Summary of Product Characteristics Transition Dyspnoea Index Willingness to Pay 3

4 Contents 1 SUMMARY Scope of the manufacturer submission Summary of clinical effectiveness evidence submitted by the manufacturer Summary of cost effectiveness submitted evidence by the manufacturer ERG commentary on the robustness of evidence submitted by the manufacturer Strengths Weaknesses and areas of uncertainty Summary of additional work undertaken by the ERG BACKGROUND Critique of manufacturer s description of underlying health problem Critique of manufacturer s overview of current service provision Critique of manufacturer s definition of decision problem Population Intervention Comparators Outcomes Other relevant factors CLINICAL EFFECTIVENESS Critique of the methods used by the manufacturer to systematically review clinical effectiveness evidence State objective of systematic review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate? What studies were included in the clinical effectiveness review and what were excluded? Provide a table of identified studies. Please identify the most important clinical effectiveness studies Provide details of any relevant studies not discussed in the submission? Why were these studies excluded and how were these studies identified by the ERG? Summary and critique of submitted clinical effectiveness evidence Summary of submitted clinical evidence for each relevant trial Describe and critique the manufacturer s approach to validity assessment for each relevant trial Describe and critique the statistical approach used within each relevant trial Describe and critique the manufacturer s approach to outcome selection within each relevant trial To what extent does each relevant trial include the patient population(s), intervention(s), comparator(s) and outcomes as defined in the final scope? Where appropriate, describe and critique any meta-analysis, indirect comparisons and/ or mixed treatment analysis carried out by the manufacturer Additional clinical work conducted by the ERG Conclusions

5 5 COST EFFECTIVENESS ERG comment on manufacturer s review of cost-effectiveness evidence State objective of cost effectiveness review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate? State the inclusion/exclusion criteria used in the study selection and comment on whether they were appropriate What studies were included in the cost effectiveness review and what were excluded? Where appropriate, provide a table of identified studies. Please identify the most important cost effectiveness studies What does the review conclude from the data available? Does the ERG agree with the conclusions of the cost effectiveness review? If not, provide details Summary and critique of manufacturer s submitted economic evaluation by the ERG NICE reference case checklist Model structure Population Interventions and comparators Perspective, time horizon and discounting Treatment effectiveness Health related quality of life Resources and costs Cost effectiveness results Sensitivity analyses Model validation Additional work undertaken by the ERG Conclusions Impact on the ICER of additional clinical and economic analyses undertaken by the ERG End of life Conclusions Implications for research References Appendix 1a: ERG Search Strategies Appendix 1b: ERG Search Critique Appendix 2: Baseline characteristics Appendix 3: Quality assessment for RCTs included in the ERG analyses Appendix 4: Statistical analyses and data management in included RCTs Appendix 5: Winbugs code for ERG MTC analyses Appendix 6: Phillips et al. Checklist

6 1 SUMMARY 1.1 Scope of the manufacturer submission The NICE scope defines the population as Adults with severe chronic obstructive pulmonary disease (FEV 1 post bronchodilator less than 50% predicted) associated with chronic bronchitis in adult patients with a history of frequent exacerbations. Nevertheless, the MS presents roflumilast studies and a full MTC in patients with moderate to severe (rather than severe) COPD. This ERG report focuses on the population as defined in the scope. A secondary analysis is presented using slightly wider inclusion criteria (FEV 1 65%). The MS presents an additive MTC analysis to provide data for other treatments including roflumilast. However, as the assumptions underlying this analysis were not met, the ERG ignored this analysis and the CEA based on it in this report. The MS presents evidence for comparisons between roflumilast and most comparators mentioned in the scope. However, most evidence was based on trials in moderate to severe COPD or based on the additive analysis. Evidence from trials in severe COPD was presented for only four relevant comparators: LABA, LAMA, ICS and LABA/ICS. The secondary ERG analysis allows a comparison between roflumilast and LABA + LAMA and LABA/ICS + LAMA. The ERG presents results for two sets of data: 1. Data for adults with severe COPD (FEV 1 post bronchodilator less than 50% predicted) from 9 trials, including 7 treatments. 2. Data for adults with moderate to severe COPD (FEV 1 post bronchodilator less than 65% predicted) from 16 trials, including 9 treatments. Regarding theophylline, the MS states that Theophylline is not thought to be an appropriate comparator for roflumilast. The ERG did not have time or recourses to assess the relative (cost-) effectiveness of roflumilast in comparison with theophylline. One of the inclusion criteria for the MTC in the MS was that trials had to report data on the total number of exacerbations and/or the mean annual rate of exacerbations. Therefore, trials reporting other relevant outcomes (as mentioned in the scope) were excluded. It is unclear from the MS which trials were excluded because they did not report exacerbations. Therefore, the ERG analyses and data extraction are based on the same set of trials, but limited to severe COPD only. 1.2 Summary of clinical effectiveness evidence submitted by the manufacturer Twenty-six publications met the inclusion criteria for the MTC in the MS. Combined, these publications reported on 29 clinical trials. However, only 8 out of 29 included trials are in patients with severe COPD (FEV 1 50%). The ERG included one more roflumilast trial that fulfilled the inclusion criteria (M2-111). In addition, Calverley et al (M2-124 and M2-125) is described as roflumilast versus placebo, whilst about 50% of patients in both trials received concomitant LABA. 1 Therefore, this trial should be treated as partly roflumilast vs. placebo; and partly roflumilast + LABA vs LABA, resulting in a network of 7 treatments and ten trials for the MTC. The ERG focussed on these ten trials for their primary analyses. A secondary ERG analysis was performed in a wider population (FEV 1 65%), including 16 trials and 9 treatments. 6

7 Results of the MTC in patients with severe COPD (FEV 1 50%) show that in terms of the mean annual rate of exacerbations roflumilast alone is highly likely (95% Credible Interval does not include rate ratio of 1) to be superior when compared to LABA (RR=1.17 (95% CRI: 1.03, 1.33)); but not when compared to any of the other active treatments. However, LABA/ICS is highly likely to be superior to roflumilast alone (RR=0.83 (95% CRI: 0.72, 0.94)). When compared with other active treatments, roflumilast+laba is also highly likely to be superior to LABA alone (RR=0.78 (95% CRI: 0.69, 0.89)). However, compared with LAMA, ICS and LABA/ICS there are no such differences. Results are similar in the wider population (FEV 1 65%). Looking at the results in comparison with placebo, LABA/ICS+LAMA seems to be the best treatment in terms of reducing the annual rate of exacerbations, LABA/ICS seems to be second best, and roflumilast+laba third best. In comparison with placebo the annual rate ratios are: RR=0.70 (95% CRI: 0.52, 0.90) for LABA/ICS+LAMA, RR=0.73 (95% CRI: 0.67, 0.80) for LABA/ICS, and RR=0.76 (95% CRI: 0.61, 0.94) for roflumilast + LABA. 1.3 Summary of cost effectiveness submitted evidence by the manufacturer Due to lack of published cost-effectiveness studies, as demonstrated by the manufacturers literature review, the manufacturer conducted a de-novo economic analysis. For the economic evaluation, the manufacturer presented results for two populations: ICS-tolerant population and ICS-intolerant population. For each of these a comparison was made for respectively 10 and 6 first line treatment options of which the duration was defined as 12 months. Second line treatment differed for the ICS tolerant and ICS-intolerant populations but was equal for all options as such. The model used by the manufacturer is a parsimonious but adequate Markov model distinguishing health states that reflect the severity of COPD and first or second line treatment. The major model input parameter that caused differences in cost and effectiveness estimates between the treatment options is the probability of experiencing COPD exacerbations, either being community treated or indicating hospital treatment. Parameters that, in the basecase analyses of the submission, were considered to be equal between the treatment options were treatment survival, the balance between community and hospital treated exacerbations, no effect of treatment on lung function, and no effect on progression of COPD. In a deterministic cost-effectiveness analysis all treatment options were compared and based upon these results it was stated that for each population a comparison of only two options, of one including Roflumilast, was subject for decision uncertainty analyses. The costs and resource use of COPD patients are grouped into three categories. First, cost of COPD regimens, that is the medication costs. Second, the cost of maintenance of a COPD patient; these are mostly outpatient consists and additional medication/therapy. Third, the costs of exacerbations; which are further divided into moderate, that is community treated, and severe, that is hospital treated, exacerbations. These cost data are to a large extent consistent with the usual reference costs. The values for the health related quality of life were empirically derived from COPD-patients, whereas the values for the utility decrements were obtained from a preference study in the general population. In this preference study, health state utility scores were calculated as well in the form of regression coefficients. Adverse events are not accounted for in the utility estimates. 7

8 The manufacturer conducted a full incremental deterministic analysis of the treatment options. However, for the PSA only selected options were compared, by that deviating from the requested full incremental analysis. No analysis for BMI and/or smoking subgroups, as requested by the scope, has been conducted. In total six scenario analyses were presented, such as assuming a beneficial effect of treatments on lung function, varying the time on first line therapy before switching to second line regimen, all COPD patients start in the very severe state, or a different set of utility values. 1.4 ERG commentary on the robustness of evidence submitted by the manufacturer Strengths Search methods were clearly presented and reported. The manufacturer searched the required databases, with the exception of the Cochrane Library for indirect and mixed treatment comparisons. The MS provided sufficient detail for the ERG to appraise the searches. Appropriate additional searches of conference abstracts were undertaken in the cost effectiveness chapter. The ERG considers that the Standard MTC model (without the additive model) was methodologically correct and of high quality. The Markov-type model used for the cost-effectiveness analysis is parsimonious but in principle adequate its purpose Weaknesses and areas of uncertainty The main weakness of the submission is that the inclusion criteria for the MTC were considerably wider than the scope. Instead of limiting the search to severe COPD (FEV 1 50%), studies with baseline FEV 1 up to 90% were included. In addition, the analyses relied on an additive model for which the underlying assumptions were not met. In general the searches were not constructed in a systematic fashion. Redundant search terms and misuse of Boolean logic created ambiguity as to what search terms actually made it to the final set. Two of the chapters contained searches with significant structural problems and searches across all chapters failed to make good use of COPD synonyms, truncation and controlled vocabulary. Although the model has been apparently used in prior applications of this nature and the manufacturer claims the model has been validated by experts in the field, no documentation of this is presented in the submission. The implementation of the model, that is the electronic version, is convoluted and lacks the necessary transparency. Furthermore, errors in the implementation, when uncorrected, lead to invalid results when conducting a probabilistic sensitivity analysis. More importantly, the method used to derive the clinical evidence to model the treatment effects, that is the reduction in the number of exacerbations an average patient can expect, of each medication is judged by the ERG is invalid and cannot be used to draw inference about cost-effectiveness of the treatments under assessment. Furthermore, a number of clinical parameters used to model a patient s course of disease, such as average number of exacerbations or proportion of exacerbations requiring hospital treatment, are not sufficiently justified by evidence but based on the subgroup of a single clinical trial. The use of improper clinical evidence concerning the treatment effects and some assumptions concerning the modelling of the patient s disease course, in particular the assumption of equal length of time on treatment, make the cost-effectiveness evidence as presented by the manufacturer invalid and unacceptable. 8

9 1.5 Summary of additional work undertaken by the ERG The ERG conducted a mixed treatment comparison of the clinical evidence. This led to the specification of two different sets of evidence: 1) a MTC based on a narrow definition of the applicable population (FEV1pred 50%) and 2) a MTC based on a wider definition of the applicable population (FEV1pred 65%), both for ICS-tolerant and ICS-intolerant COPD patients. The differentiation between the narrow and wide inclusion criterion is non-trivial as some treatment options exhibit a different treatment effect under the narrow definition than under the wide definition. Furthermore, under the narrow definition less treatment options are admissible than under the wide definition. In fact, some treatment options proposed by the manufacturer could not be included in the further analysis. Those are LAMA+Roflumilast, LABA/ICS+Roflumilast, and LAMA+LABA/ICS+Roflumilast for the ICS-tolerant population and the options LAMA+Roflumilast and LAMA+LABA+Roflumilast for the ICS-intolerant population. Furthermore, the ERG conducted a literature search to find more substantiated evidence for a number of crucial clinical parameters. These were the proportion of severe exacerbations, that is requiring hospital treatment, by COPD status and the ratio of expected annual exacerbations for very severe COPD patients as compared with severe COPD patients. The ERG defined a new basecase scenario based on the narrow inclusion criterion for the MTC for both, the ICS-tolerant and ICS-intolerant population. In addition, an alternative scenario was defined based on the wide inclusion criterion for the MTC as well for both, the ICS-tolerant and the ICSintolerant population. For these in total four different model configurations, a full incremental deterministic analysis and probabilistic sensitivity analysis was undertaken by the ERG. For all but one case, the options including Roflumilast are dominated or dominated by extension. Only in one case of the alternative scenario, that is using the relaxed inclusion criterion, for the ICS-intolerant population LABA+Roflumilast is on the cost-effectiveness frontier with an ICER of approx. 41,000 per QALY gained. This is largely because the mean effectiveness (exacerbation rate ratio) of Roflumilast+LABA increased on moving from the narrow to the wider MTC, largely due to the fact that it is only compared directly to LABA in the MTC and that LABA is more effective in the FEV1pred 65% studies than in the FEV1pred 50% studies. In a scenario analysis assuming a 50% higher number of annual exacerbation and using the treatment evidence based on the wider inclusion criterion, this ICER comes down to approx. 30,000 per QALY gained. Similarly, in a scenario assuming all patients already start as very severe COPD patients, the ICER comes down to approx. 37,000 per QALY gained (Scenario F). 9

10 2 BACKGROUND 2.1 Critique of manufacturer s description of underlying health problem. Does the ERG believe that the manufacturer s description of the underlying health problem is appropriate and relevant to the decision problem under consideration? The MS description of the underlying health problem is in line with NICE guidance 2, and hence appears comprehensive and pertinent to the decision problem. For completeness the following is reproduced from the MS: COPD is a chronic, progressive, inflammatory disease characterised by airflow obstruction which is not fully reversible. COPD is primarily caused by cigarette smoking, although other factors, particularly occupational exposures, may also contribute to its development. The term COPD encompasses patients presenting with chronic bronchitis, emphysema or a combination of both with sufferers typically experiencing a prolonged decline in lung function punctuated by acute exacerbations of symptoms. The subset of patients with chronic bronchitis are readily characterised by the presenting signs of chronic cough and sputum production. There are approximately 900,000 people diagnosed with COPD in the UK and a further 2 million are estimated to have the disease but remain undiagnosed (MS, page 14). The chronic, systemic inflammation which underlies COPD is principally caused by an increase in CD8+ T-lymphocytes, macrophages and neutrophils and is different from the localised inflammation underlying asthma, which is predominately caused by an increased number of CD4+ T-lymphocytes and eosinophils. This difference in fundamental pathology requires that the treatment of COPD and asthma have clearly distinct approaches to clinical management (MS page 15). The prevalence of COPD increases with age, socioeconomic disparities and disease severity. It is most common in people aged 35 years and above. In United Kingdom, the mean age at diagnosis is estimated at 67 years. COPD affects more women than men. Since the mid-1990s, diagnosed COPD prevalence rates have been on the increase in women compared to those in men that have reached a steady level, a factor associated with sex differences in smoking in recent years. COPD is associated mainly with cigarette smoking. Within the cigarette smoking population, nearly half develop some sort of airflow obstruction, and around 10-20% develop COPD over time. 3 Although smoking is a major contribution to development of COPD, other factors do play a role. COPD can develop in nonsmokers too especially those affected with α 1 antitrypsin deficiency and Asthma. Other factors include; passive smoking, air pollution, oxidative stress, nutrition and occupation. The prognosis of COPD is associated with lung inflammation over time, which causes permanent lung damage. The airway walls thicken due to inflammation leading to increased mucus production. This reduces the lung elasticity making it very difficult to breathe causing breathlessness, phlegm and cough. This course of lung damage is irreversible but lifestyle changes can help reduce the extent of the damage. 4 NICE has recently updated the COPD Clinical Guideline (CG101) which stressed the importance of multidimensional assessment of the disease, as opposed to simple categorisation of disease severity through measure of lung function (MS page 15). Spirometry is vital in making a COPD diagnosis and provides a valuable assessment of the severity of the disease. However, relying on it alone can underestimate or overestimate the severity of COPD in 10

11 some patients. It is recommended that Spirometry is offered to smokers and non-smokers aged 35 years and above who present with symptoms. In addition to Spirometry testing at the initial diagnostic stage, chest x-ray and full blood count are also recommended, to provide a more accurate diagnosis. 2 Additionally, the measurements should be carried out routinely at least every three years. Thresholds of severity of airway obstruction 80%, 50% and 30%; defined by mild, moderate and severe respectively, are used as boundaries for diagnosis, prognosis and treatment recommendation. 2 Nonetheless, these boundaries are thought to be used due to their simplicity without any clinical validity. 4 GOLD Strategy Group recommends four stages of COPD severity based on a spirometric classification of Post-Bronchodilator FEV 1, as shown in table 2.1 below: Table 2.1: Spirometric Classification of COPD Severity Based on Post-Bronchodilator FEV 1 Stage I: Mild FEV 1 /FVC < 0.70; FEV 1 80% predicted Symptoms of chronic cough and sputum production may be present, but not always Stage II: Moderate FEV 1 /FVC < 0.70; 50% FEV 1 < 80% predicted With shortness of breath typically developing on exertion and cough and sputum production sometimes also present. This is the stage at which patients typically seek medical attention because of chronic respiratory symptoms or an exacerbation of their disease. Stage III: Severe FEV 1 /FVC < 0.70; 30% FEV 1 < 50% predicted Greater shortness of breath, reduced exercise capacity, fatigue, and repeated exacerbations that almost always have an impact on patients quality of life Stage IV: Very Severe FEV 1 /FVC < 0.70; FEV 1 < 30% predicted or FEV 1 < 50% predicted plus chronic respiratory failure At this stage, quality of life is very appreciably impaired and exacerbations may be life- threatening. Source: GOLD, (2006) FEV 1: forced expiratory volume in one second; FVC: forced vital capacity; Mortality from COPD in United Kingdom has risen in women more than men in the last 30 years. The life expectancy is reduced by 1.1 years in those with mild COPD; 1.7 years for moderate COPD, and 4.1 years for those with severe COPD. 5 COPD is a major health issue in the UK and is the fifth leading cause of death, currently accounting for 26,000 to 30,000 deaths per year. It is the second most common reason for emergency hospital admissions in the UK, with 1 in 8 emergency admissions each year being due to COPD. Whilst COPD is frequently characterised by breathlessness, exacerbations are important events in the natural history of the disease, with both their frequency and severity being associated with a significant worsening of prognosis. In COPD, exacerbations account for the greatest burden on healthcare systems, therefore reducing their severity and frequency is an important therapeutic goal. Exacerbations of COPD are estimated to account for 15.9% of hospital admissions, at a cost to the NHS of more than 253 million per year (MS, Page 14). 2.2 Critique of manufacturer s overview of current service provision Does the ERG believe that the manufacturer s overview of current service provision is appropriate and relevant to the decision problem under consideration? The ERG also broadly agrees with the manufacturer s description of current service provision based on the updated NICE guideline. The following is further reproduced from the MS: Current guidelines for the treatment of patients with COPD recommend the addition of therapies in a stepwise manner according to patients' needs, with the aim of controlling symptoms and preventing exacerbations. NICE has recently updated the COPD Clinical Guideline (CG101) which stressed the importance of multidimensional assessment of the disease, as opposed to simple categorisation of 11

12 disease severity through measure of lung function. The guideline recommends that patients with severe COPD (FEV 1 < 50% predicted) receive a long-acting bronchodilator, either LABA (with ICS) or LAMA. If patients continue to exacerbate or remain breathless, it is recommended that patients progress to triple therapy with LAMA plus LABA / ICS. In patients who cannot tolerate or decline ICS, a combination of the two bronchodilators is recommended (LABA plus LAMA) (MS, page 15). The NICE updated guideline (CG101) recommends that pulmonary rehabilitation should be made available to all appropriate COPD patients including those who have had a recent hospitalisation for an acute exacerbation. The manufacturer did not mention other forms of treatments currently available for severe COPD and in patients who have had an exacerbation. These include; Oxygen Therapy, Non-Invasive Ventilation, Invasive Ventilation and Intensive Care, Surgery, Antibiotics, recommended based on patient diagnosis. 12

13 3 Critique of manufacturer s definition of decision problem 3.1 Population To what extent does the clinical evidence submitted by the manufacturer match the patient population described in the final scope? Where there is a mismatch, provide further details. Does the clinical evidence submitted by the manufacturer reflect the characteristics of the patient population in England and Wales eligible for treatment? If not, provide further comment. The NICE scope defines the population as Adults with severe chronic obstructive pulmonary disease (FEV 1 post bronchodilator less than 50% predicted) associated with chronic bronchitis in adult patients with a history of frequent exacerbations. Nevertheless, the MS presents roflumilast studies and a full MTC in patients with moderate to severe COPD. Regarding roflumilast studies, four of the 10 studies reported in the MS included patients with severe COPD (FEV 1 50% predicted). The other 6 studies included patients with moderate to severe COPD. None of these four trials directly compared roflumilast with the comparators listed in the decision problem. However, around 50% of patients included in trials M2-124 and M2-125 received concomitant LABA. For these patients the comparison is roflumilast + LABA versus LABA + placebo. All other trials compared roflumilast with placebo. None of the roflumilast trials included patients with a history of frequent exacerbations, using a definition of two or more exacerbations in the previous year. Twenty-six publications met the inclusion criteria for the MTC. Combined, these publications reported on 29 clinical trials. Most of these trials were in patients with moderate to severe COPD, inclusion criteria going up to 90% FEV 1 predicted at baseline. Only 8 out of 29 trials included patients with severe COPD. 3.2 Intervention Does the intervention described in the MS match the intervention described in the final scope? What is the technology and what is its relevant or proposed marketing authorisation/ CE mark? The intervention in the NICE scope is defined as Roflumilast in combination with maintenance bronchodilator treatment (LABA, LABA/corticosteroid combination inhaler, LAMA, LAMA plus LABA/corticosteroid combination inhaler or LAMA plus LABA [if ICS not tolerated]). Three treatments including roflumilast have been tested in trials: roflumilast alone, roflumilast + LABA and roflumilast + LAMA. However, the latter (roflumilast + LAMA) has only been tested in one trial (Fabbri et al / M2-128) which included patients with moderate to severe COPD (FEV 1 between 40% and 70%). 6 Therefore, only two treatments (roflumilast alone and roflumilast + LABA) have relevant data in the correct population. The MS presents additive analyses to provide data for other treatments including roflumilast. However, as the assumptions underlying these analyses were not met, the ERG will ignore these analyses in this report. The rationale for not including data for roflumilast + LAMA and for not using the additive analysis will be further explained at the end of this chapter under COMMENT. 13

14 The table below shows the evidence used in the MS for each of the interventions included in the analyses presented in the MS. As can be seen in this table, four out of seven interventions have no underlying evidence. The analyses are purely based on the additive analyses. Two out of seven interventions (Roflumilast+LABA and Roflumilast+LAMA) are based on evidence in the wrong population (not severe COPD, but mainly moderate COPD). The remaining intervention: roflumilast alone, is based on evidence from three trials, one of which is again in the wrong population, and another combines evidence from foflumilast versus placebo with roflumilast+laba versus LABA+placebo. Table 3.1: Evidence for Roflumilast treatments in the MS model. Treatment Comparator Trials Comment Roflumilast alone Placebo M2-124 and M2-125 FEV1 50%, 50% received combined (Calverley et al concomitant LABA, but all 2009) were treated as if Roflumilast vs placebo M2-112 (Calverley et al FEV1 50% 2007) M2-107 (Rabe et al 2005) FEV % Roflumilast+LABA LABA M2-127 (Fabbri et al FEV % 2009) Roflumilast+LAMA LAMA M2-128 (Fabbri et al FEV % 2009) Roflumilast+ICS -- No evidence Used Additive MTC Roflumilast+LABA+LAMA -- No evidence Used Additive MTC Roflumilast+LABA/ICS -- No evidence Used Additive MTC Roflumilast+LABA/ICS+LAMA -- No evidence Used Additive MTC The ERG has used evidence for roflumilast alone versus placebo from three trials (M2-111, M2-112 and M2-124/125 (only patients who did not receive concomitant LABA)) in which all patients had severe COPD. For Roflumilast+LABA versus LABA, the ERG used evidence from one trail: M2-124/125 (only patients who did receive concomitant LABA)) in which all patients had severe COPD. Other roflumilast-based interventions were not included in the ERG analyses because evidence is only presented in the wrong population (roflumilast+lama), or no evidence is presented. 3.3 Comparators Do the comparators described in the MS match the comparators described in the final scope? If not, provide further details. Where evidence is limited or not available for relevant comparators has the manufacturer asked an unbiased clinical panel, or carried out its own survey, and do the views elicited agree with what the clinical advisors to the ERG advocate? The NICE scope defines the comparators as: ICS (in combination with LABA); LABA (in combination with LAMA); LAMA (in combination with LABA/corticosteroid combination inhaler [or with LABA alone if ICS not tolerated]); and theophylline (in combination with maintenance bronchodilator treatment [LAMA plus LABA/corticosteroid combination inhaler, or LAMA plus LABA, if ICS not tolerated]). Regarding theophylline, the MS states that Theophylline is not thought to be an appropriate comparator for roflumilast. (see MS, section 4, page 36). The ERG did not have time or resources to assess the relative (cost-) effectiveness of roflumilast in comparison with theophylline. 14

15 The MS presented evidence for comparisons between roflumilast and most comparators. However, most evidence was based on trials in moderate to severe COPD or based on additive analyses. Evidence from trials in severe COPD was presented for only four relevant comparators: LABA, LAMA, ICS and LABA/ICS. In order to present data for a comparison between roflumilast and LABA + LAMA and LABA/ICS + LAMA, the ERG widened the inclusion to trials including patients with FEV 1 65% (see section of this report). 3.4 Outcomes Do the outcomes in the MS match the outcomes described in the final scope? If not, provide further details. Consider clinical effectiveness, adverse events, quality of life and health economic outcomes and a discussion of appropriate mechanisms for measuring these outcomes. Is the focus of the submission on appropriate outcomes or has it been limited to non-ideal outcomes? The NICE scope mentions the following outcomes: lung function; incidence and severity of acute exacerbations, including hospitalisation; symptom control (e.g. shortness of breath); mortality; adverse effects of treatment; and health-related quality of life. For the roflumilast trials, including those in moderate to severe COPD, most outcomes are reported in the MS (pages 71-89). Additional adverse events are presented for the roflumilast trials in chapter 5.9 (MS, pages ). For the subgroups of patients with and without LABA in M2-124 and M2-125, only data for COPD exacerbations and lung function (Mean Changes in Pre- and Post-bronchodilator FEV 1 ) are presented in the MS (pages 81-84). For non-roflumilast trials included in the MTC, no data were reported. One of the inclusion criteria for the MTC was that trials had to report data on the total number of exacerbations and/or the mean annual rate of exacerbations. Therefore, trials reporting other relevant outcomes (as mentioned in the scope) were excluded. In the response to the clarification letter, the ERG received data for exacerbations (total number of exacerbations and number of person-years) for all trials included in the MTC. 3.5 Other relevant factors For example: Does the MS include a section on equity considerations? Is there an ongoing Patient Access Scheme application? The MS states that there are no equity or equalities issues anticipated for the appraisal of this technology. The MS does not mention any Patient Access Schemes. COMMENT ERG rationale for not including data for Roflumilast + LAMA. The ERG considered adding the comparison with Roflumilast+LAMA vs LAMA from the M2-128 trial to the network, as reported in Fabbri et al This study, as with its sister study, M2-128 (Roflumilast+LABA vs LABA, also reported in Fabbri et al 2009) would not fit the scope in that the FEV 1 inclusion criterion was 40-70% i.e. definitely not very severe and with only a very small overlap 15

16 with severe (40-50%). However, it was the only study that included Roflumilast+LAMA. Therefore, the ERG attempted to estimate an adjustment factor i.e. a rate ratio (RR) for exacerbations for FEV 1 50% vs 40-70% that could be applied to the RR of Roflumilast+LAMA vs LAMA. Such a RR could be estimated by comparing the RR for Roflumilast+LABA vs LABA from M2-124/125 (FEV 1 50%) with that from M2-127 (FEV %). However, when the MS source in Fabbri et al 2009 was examined it appeared to report the rates for any exacerbation i.e. not only moderate and severe (as in all other studies up to 65% FEV 1 ), but also mild. Therefore, the number of moderate and severe exacerbations only (r), were found in the trial report: Roflumilast+LABA (r=55), LABA (r=97). Given the fact that no data were reported on the actual number of years of exposure, the number of person years was estimated by assuming full follow-up (same approach as in the MS), which was 24 weeks for all those randomised (Roflumilast+LABA (n=466), LABA (n=467). This implies total person years (py) of n x 24/52 i.e. about 215 for each arm and therefore a rate of the number of moderate and severe exacerbations / the number of patients years ; i.e. for Roflumilast+ LABA (0.26) and LABA (0.45). The RR for Roflumilast+LABA vs LABA in FEV % is therefore 0.26/0.45=0.57 (95% CI: ). The RR in FEV 1 50% is calculated from (r,n) of Roflumilast+LABA (686,563) vs LABA (971,621) to be: (686/563)/(971/621)=0.78 (95% CI: 0.71,0.86). Given little overlap in the confidence intervals, these results firstly show evidence to support the approach of the ERG not to pool studies with quite different populations. Secondly, an adjustment factor calculated from these results would be: 0.57/0.78=1.37. This could be applied to the RR for Roflumilast+LAMA vs LAMA from trial M2-128, again using the trial report data, which in this case did not report the number of exacerbations, but instead the rates i.e. Roflumilast+LAMA (0.269) and LAMA (0.308). These imply the RR in FEV % of 0.27/0.31=0.87. Applying the adjustment factor would give a RR in FEV 1 50% of 0.87 x 1.37=1.19, which would mean that Roflumilast+LAMA is less effective than LAMA. However, this is not very plausible. Therefore, the ERG chose not to include Roflumilast+LAMA in the network for either FEV 1 50% or 65% on the basis that: 1) The only evidence was from the M2-128 trial, which was in the wrong population of 40-70% 2) Evidence of a comparison of Roflumilast+LABA vs LABA between the FEV 1 50% and 40-70% showed significant heterogeneity. 3) A plausible adjustment factor could not be estimated. ERG rationale for not using the additive analysis. The ERG considers that the additive MTC analysis performed in the MS in order to estimate the effect of the combination treatments (Roflumilast+LABA+LAMA, Roflumilast+LABA/ICS+LAMA, Roflumilast+ICS, Roflumilast+LABA/ICS and LAMA+ICS) for which there was no trial data, was invalid. The reasons for this are: 1. On epistemological grounds, there is no evidence presented, whether from a randomised control trial or from observational data as to the effect of these combinations, which the ERG argues is insufficient grounds for estimation. 2. Underlying assumptions are not met: Independence of the additive effect (on log(rr)) of the addition of one treatment to another on which the analysis relies is an assumption that is both unlikely to be true in all cases and for which the MS provides no evidence for any of the treatments. 16

17 Independence of the rate ratio (RR) implies that if treatment A is added to B then the effect of A+B in terms of rate is the rate of B multiplied by a RR of A vs any other comparator. In other words if A was added to C then the same RR would apply. On a log scale this would imply that the log of the rate of A+B would be found by adding the log of the rate of B to the log of the RR of A vs any other comparator: hence multiplicative independence implies additive independence on the log scale. Another way of expressing this assumption is that the effect of the combination of treatments is simply the sum of the parts i.e. the effect is additive. A break from this assumption would mean that the RR of A+B vs B is different to A+C vs C: there would be an interaction between the treatments. For example, if the RR for A+C vs C was higher than for A+B vs B then one might say that C has a synergistic effect on A. Conversely, B might be said to diminish the effect of A. It seems unlikely that the effect of the treatments Roflumilast, ICS, LABA and LAMA would always be additive. Moreover, if the effect was always additive then there would be no ceiling effect and most people would benefit from receiving as many drugs as were indicated for a condition. 3. There is physiological evidence, at least for the combination of ICS and LABA, that the effect is more than additive. Miller-Larsson and Selroos in their review of ICS and LABA combination therapy conclude: Data from preclinical studies provide evidence of additive, compensatory, complementary and synergistic effects of ICS and LABA in the control of inflammation and airway and lung remodelling. (p.3261) 7 4. There is evidence from the standard MTC analysis (i.e. without the additive assumption) in the MS of the breaking of the additive assumption. As the MS shows (Table B30, page 121), the effect of roflumilast vs placebo is a RR of However, the effect of adding roflumilast to LABA is a RR of 0.80 and 0.84 of adding to LAMA. In other words, the effect of roflumilast is not independent of the drug to which it is added. The effect of the addition of all components, as reported in the MTC (Table B30) is shown below: Table 3.2: Additive effects of different treatment components To Add: Standard MTC Additive MTC placebo Roflumilast LABA LAMA ICS LABA+LAMA Any Roflumilast (Table B31 and Key findings LABA NR LAMA NR ICS NR ICS+LAMA NR 0.82 NR ICS+LABA NR LAMA+LABA NR If the effect of addition of each component was independent of what they were added to then, reading across each row, the values should be the same and clearly they are not: indeed for LABA there is even a change in direction when added to LAMA i.e. adding LABA to LAMA appears to 17

18 make exacerbation more likely. Even if this had occurred by chance, it cannot be ruled out that all differences have occurred by chance. Indeed it is because of this that trials are designed to compare not only single treatments, but combinations. The additive model has been published by Mills et al. in a peer reviewed journal. 8 However, the ERG considers that publication does not imply validity for the analysis in the MS. Indeed, Mills et al acknowledge that the additive assumption might not be correct: This additive assumption may be untenable in situations where particular pairs of components may have either bigger (synergistic) or smaller (antagonistic) effects than would be expected from the sum of their effects alone. (p.121) However, they make no attempt to justify it by reference to any other evidence or to test it. They simply state: The sparsity of our data prevented us from relaxing the additivity assumption. (p.121) In contrast, the ERG considers that there is some evidence (as shown above) that the additive assumption does not hold and that a lack of data should be a reason not to make the assumption rather than not to relax it. Mills et al. do cite a study by Welton et al., which used an additive model. 9 However, the Welton et al study is distinctly different, firstly because it is an entirely different area i.e. psychological therapies in coronary heart disease. Secondly, there appears to be many more comparisons of combined treatments. Most importantly, they do not simply assume that the additive model is correct, but instead they test it against two models that include interactions. The ERG does not argue that a simple additive effect cannot occur, but that in the particular case of this STA it is not appropriate. The manufacturer s response to the clarification letter cites McAlister et al 2003 as support for their use of the additive model, in particular that the proportion of studies reviewed by McAlister et al that showed a statistically significant interaction between treatments was low. 10 However, the company may have misinterpreted the McAlister study since they only reviewed studies with a factorial design i.e. where it was a priori believed that the chances of interaction were low. Moreover, the basis for the review was the concern that there could be interactions and that the best way of dealing with this possibility is to test for it; i.e. not to make an assumption of additive independence. As McAlister et al state: This is all the more important in an era of multiple efficacious therapies for many diseases (such as acute myocardial ischemia) in which questions frequently arise as to whether the benefits of these agents are additive, synergistic, or antagonistic. We believe the factorial trial remains the most valid means by which to evaluate whether combining 2 or more therapies achieves incremental benefits. (p.2551) With regards to the evidence from the standard MTC that the additive assumption does not hold, the company also state: We would comment that many of these values are in fact quite similar, and the confidence intervals around each value are wide accounting for the slight difference in the values. 18

19 However, the ERG would argue that it should be incumbent on the company to present evidence to support the additive assumption. Indeed, as McAlister et al state: However, even when interactions are neither statistically nor clinically significant, they still may distort the evaluation of treatment effects. (p.2551) The company go on to present a table, which they claim shows similarity of estimates between the standard MTC and the multiplicative analysis: Table 3.3: Relative effects (as rate ratios) from the standard and additive MTCs, and from the pair-wise meta-analysis (see Response to Clarification letter, page A9, page 7). Treatment Comparison Standard MTC Analysis RR (95% CrI) Additive MTC Analysis RR (95% CrI) Pair-Wise Meta- Analysis RR (95% CI) LABA vs. Placebo 0.84 (0.75, 0.93) 0.86 (0.80, 0.94) 0.87 (0.79, 0.96) LAMA vs. Placebo 0.74 (0.66, 0.81) 0.74 (0.67, 0.82) 0.74 (0.64, 0.84) LABA + LAMA vs. Placebo 0.80 (0.56, 1.12) 0.64 (0.56, 0.73) - However, whilst it is true that the RRs for LABA or LAMA vs placebo are similar, the ERG considers that those for LABA+LAMA are clearly not (0.80 vs 0.64). Indeed, as the company shows (on the log scale), it is quite clear that the figure of 0.64 is simply estimated by log (0.64)=log(0.86)+log(0.74). This is equivalent to the multiplication of the RRs for LABA and LAMA vs placebo i.e x 0.74=0.64. Moreover, the company attempt to validate the figure of 0.64 by comparing it to a figure of 0.62, calculated from log(0.62)=log(0.84)+log(0.74) i.e. simply adding the effects of LAMA and LABA from the standard MTC. However, it is no surprise that simply adding effects, which are very similar in both models (given that they are informed by actual trial data of LABA vs placebo and LAMA vs placebo), will produce essentially the same result. The validity of the assumption of additive independence cannot be tested by comparison of the result of a model reliant on that assumption (the additive MTC model) with another that also relies on that assumption (adding the effects estimated using the standard MTC). The company finally states: For the combination of LABA + LAMA, the standard MTC analysis relies on only one RCT, and this lack of data may explain the lack of additional efficacy gained by the combination of these two agents derived from the standard MTC analysis. In other words they find the assumption of independence of treatment effect based on no evidence more credible than RCT evidence, albeit from only one trial. 19

20 4 CLINICAL EFFECTIVENESS 4.1 Critique of the methods used by the manufacturer to systematically review clinical effectiveness evidence The MS reports two separate systematic reviews for clinical effectiveness: 1. A systematic literature search of randomised controlled trials, which included roflumilast, in patients with COPD. 2. A mixed treatment comparison (MTC) meta-analysis and a multiplicative analysis using published literature to assess the effects of various combinations of COPD treatments, with regard to their efficacy in reducing COPD exacerbations. The search strategies as well as the inclusion and exclusion criteria used in the selection of evidence for both systematic reviews are different State objective of systematic review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate? List databases and other sources of information including unpublished sources, describe any restrictions. An evidence based checklist for the Peer Review of Electronic Search Strategies (PRESS), developed by McGowan et al. was adapted to serve as a template for this critique. The submission was checked against the Single technology Appraisal (STA) specification for manufacturer/sponsor submission of evidence. 11, 12 The ERG has presented only the major limitations of each search strategy in the main report. Further criticisms of each search strategy can be found in Appendix 1b. To highlight shortcomings in the manufacturer s searching the ERG created a number of comparison search strategies and re-ran some of those created by the manufacturer. The ERG search strategies are presented in Appendix 1a. The ERG was not able to screen search results due to time constraints, and therefore can only show the numerical differences in the numbers of references retrieved between manufacturer and ERG searches without a definitive indication that relevant studies were missed. 1. Review of roflumilast studies Clinical effectiveness Searches were carried out on all databases required by NICE. The search dates and database date spans were not accurately reported for any of the searches in this section. The research question was stated as randomised controlled trials (RCTs) of roflumilast in patients with COPD. The manufacturer translated this research question into appropriate search strategies and the ERG considers these searches to be adequate. Adverse events (roflumilast) The ERG requested clarification as to why no adverse events searching was carried out by the manufacturer. The manufacturer explained in the clarification response (CR, p19) that they had access 20

21 to all clinical trials reports for roflumilast and full adverse events data were provided. The ERG concurs that this is sufficient. 2. Mixed treatment comparison Indirect and mixed treatment comparisons Searches were not carried out on all databases required by NICE, the database omitted was the Cochrane Library. The date of searching was not reported for any of the databases and a correct date span was only reported for Embase. The manufacturer s rationale for not searching the Cochrane Library is that Embase includes [the] Cochrane Library (MS, p314). This is rebutted in chapter of the Cochrane Handbook which states that 170,000 CENTRAL records are from sources other than Medline or Embase. 13 The ERG addressed this in the CL by creating improved systematic searches for indirect and mixed treatment comparisons in Embase, Medline, Medline-In Process and the Cochrane Library. Where possible the manufacturer s original strategies were re-run and the references retrieved were omitted from the ERG s results using the NOT operator. The resulting 2141 references from these searches were sent on to the manufacturer in an Endnote library for screening. The manufacturer screened these results but found no new studies that met the inclusion criteria (CR, p17). Adverse events (comparators) No specific adverse events searching was carried out for roflumilast s comparators, instead adverse events data were taken from the papers found from the indirect and mixed treatment comparator searches. CRD guidance recommends that if searches have been limited by an RCT filter additional searches should be undertaken to ensure that adverse events that are long-term, rare or unanticipated are not missed. 14 It is possible that since the indirect and mixed treatment comparison searches were limited to RCTs then some evidence may be available for the comparator drugs, which was not identified. Cost effectiveness Searches were carried out on all databases required by NICE. The search date was reported for all searches but the date span was not accurately reported for Medline or Medline In-Process. The searches were well reported and reproducible. Additional searches were undertaken for relevant material in conference proceedings of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the European Respiratory Society (ERS). The ERG considers the manufacturer s Medline, Medline In-Process, Embase and NHS EED searches to have significant limitations. Medline & Medline In-Process & Embase searches The PICO was defined as economic evaluations of patients with COPD or severe bronchitis being treated with roflumilast or its comparators. Limitations The PICO was not translated into a strategy with easily recognisable facets. Redundant terms, errors in Boolean logic and line combinations that mix facets together multiple times before the final set made these searches difficult to interpret. The ERG created a flow chart to illustrate the organisation 21

22 of terms and facets in the Medline, Medline In-Process and Embase cost-effectiveness searches in this chapter by using lines #1-86 of the Medline strategy as an example (figure 4.1). The ERG created a new search strategy and flow chart to highlight how systematic searches are normally formulated (figure 4.2). While the manufacturer s linking of lines and terms was unorthodox, three facets did emerge. These were COPD terms (shaded blue), economics terms (shaded orange) and utility terms (shaded green). These were combined at various places during the strategy and again in the final set at the end. A crucial limitation of this strategy was that it did not follow the defined PICO. The manufacturer strategy followed this structure: [COPD] AND [Economic evaluations] AND [Utilities] However the PICO translated correctly would be translated into this structure: [COPD] AND [Economic evaluations] AND [roflumilast/comparators] By not including an intervention facet and having an unnecessary utilities facet in its place, the manufacturer has reduced both the sensitivity and precision of their search. The ERG considers it possible that some cost effectiveness studies were missed due to this misinterpretation of the PICO. Other limitations, such as missing truncation, an ill-defined economics filter, lack of wildcards, missing synonyms, search term redundancy, inconsistent date limits and English language limits are detailed in Appendix 1b. NHS EED Similar to the Medline, Medline In-Process and Embase searches the search facets do not match the stated PICO. Other limitations of this search can be found in Appendix 1b. 22

23 Line 9: Stage III.mp. Line 65: (chronic obstructive pulmonary disease.mp. OR COPD.mp. OR *chronic obstructive lung disease/) AND 9 Figure 4.1: Flow chart of manufacturer's systematic cost-effectiveness Medline strategy (lines #1-86) Line 66: (chronic obstructive pulmonary disease.mp. OR COPD.mp. OR *chronic obstructive lung disease/) AND 10 Line 67: very severe COPD.mp. OR severe chronic obstructive pulmonary disease.mp. OR very severe chronic obstructive pulmonary disease.mp. OR severe chronic obstructive lung disease.mp. OR very severe chronic obstructive lung disease.mp. OR severe COPD.mp. OR 65 OR 66 Line 69: (chronic bronchitis.mp. OR exp chronic bronchitis/) AND 67 Line 68: (chronic bronchitis.mp. OR exp chronic bronchitis/) AND (chronic obstructive pulmonary disease.mp. OR COPD.mp. OR *chronic obstructive lung disease/) Line 10: Stage IV.mp. Line 70: chronic obstructive pulmonary disease.mp.or COPD.mp. or *chronic obstructive lung disease/ OR 67 OR 68 OR 69 Line 83: cost of illness.mp. OR exp "cost of illness"/ OR budgets.mp. OR exp budget/ OR (costs and cost analysis).mp. OR costminimization.mp. OR cost per QALY.mp. OR cost per LY.mp. OR cost per life year.mp. OR time horizon.mp. OR pharmacoeconomics.mp. or exp PHARMACOECONOMICS/ OR pharmacoeconomic.mp. OR pharmaceutical economics.mp. OR resource utilisation.mp. OR resource utilization.mp. OR health care utilisation.mp. OR health care utilisation.mp. OR economic model.mp. OR economic study.mp. OR hospitalisation cost.mp. OR hospitalisation costs.mp. OR hospitalization cost.mp. OR hospitalization costs.mp. OR indirect cost.mp. OR societal cost.mp. OR payer perspective.mp. OR time lost from work.mp. OR productivity.mp. OR cost effectiveness.mp. OR cost data.mp. OR cost effectiveness ratio.mp. OR economic evaluation.mp. OR exp ECONOMICS/ OR *"health care cost"/ OR *"cost benefit analysis"/ OR cost-effectiveness analysis.mp. OR cost-utility.mp. cost minimisation.mp. OR economic efficiency.mp. Line 86: 70 AND 85 AND 83 (final set before limits are applied) Line 74: maintenance.mp. AND 70 Line 85: 75 AND 83 Line 75: Forced expiratory volume.mp. OR exacerbations.mp. OR hospital stay.mp. OR exp hospitalization/ OR hospitalisation.mp. OR exacerbation.mp. OR exp "length of stay"/ OR exp mortality/ OR case fatality.mp. OR mortality rate.mp. OR frequent exacerbations.mp. OR history of frequent exacerbations.mp. OR respiratory physician.mp. OR spirometry.mp. OR influenza vaccination.mp. OR oxygen therapy.mp. OR A&E visit.mp. OR emergency visit.mp. OR *lung function/ OR hospital.mp. OR*outpatient care/ OR outpatient visits.mp. OR 74 23

24 Line 6: 1 OR 2 OR 3 OR 4 OR 5 1 exp Pulmonary Disease, Chronic Obstructive/ 2 exp Pulmonary Emphysema/ 3 (chronic adj3 (obstructi$ or airflow) adj3 (lung$ or pulmonary or airway$ or respiratory or limitation$ or diseas$ or disorder$)).ti,ab,ot,hw. 4 (copd or coad).ti,ab,ot. 5 (chronic bronchitis or emphysema).ti,ab,ot. COPD terms Figure 4.2: Flow chart of ERG's systematic cost-effectiveness Medline strategy Line 14: 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 7 (LABA or LABAs or LAMA or LAMAs or ICS).ti,ab,rn,ot. 8 (Roflumilast or Daxas or Daliresp or ).t i,ab,rn,ot. 9 (Formoterol or eformoterol or Foradil$ or Oxeze or Oxis or Atock or Atimos or Perforomist or ).ti,ab,rn,ot. Roflumilast 10 (Salmeterol or serevent or ).ti,ab,rn,ot. 11 (Tiotropium or Tiova or spririva or or ).ti,ab,rn,ot. and its comparators 12 (Budesonide or Rhinocort or Rhinosol or Budicort or Entocort or pulmicort or ).ti,ab,rn,ot. 13 (Fluticasone or Flovent or Flixotide or Flonase or Flixonase or Advair or Seretide or Nasofan or or ).t i,ab,rn,ot. Line 27: 6 AND 14 AND 26 Line 26: 15 OR 16 OR 17 OR 18 OR 19 OR 20 OR 21 OR 22 OR 23 OR 24 OR economics/ 16 exp "costs and cost analysis"/ 17 economics, dental/ 18 exp "economics, hospital"/ 19 economics, medical/ 20 economics, nursing/ 21 economics, pharmaceutical/ 22 (econom$ or cost or costs or costly or costing or price or prices or pricing or pharmacoeconomic$).ti,ab. 23 (expenditure$ not energy).ti,ab. 24 (value adj1 money).ti,ab. 25 budget$.ti,ab. Cost - effectiveness filter 24

25 Measurement and valuation of health effects Searches were carried out on all databases required by NICE. The search date was reported for all searches but the database date span was not accurately reported for Medline or Medline In-Process. There were significant limitations to the Medline, Medline In-Process and Embase searches in this chapter. These are described below with less significant limitations described in Appendix 1b. Medline & Medline-In Process & Embase The research question was described on page 192 of the MS as studies reporting HRQL/utility values associated with COPD. The search was translated into a search containing three facets: [COPD terms] AND [COPD specific utilities] AND [HRQL terms] Limitations The search would have benefited from the removal of the COPD specific utilities facet (lines #19-26) as this unnecessarily reduced sensitivity. Since FEV1 and exacerbations are the standard measures of COPD severity their inclusion is already implied by the combination of COPD and HRQL. Two of the include papers in this chapter, Oba (2009) and Hajiro et al. (1999), would not be found using the Medline search even though they are indexed in Medline. 15, 16 This is because COPD specific utilities are not included in their title, abstract or subject headings. Instead the manufacturer could have used more focussed quality of life terms to keep references retrieved to a reasonable number for screening. Three lines in the HRQL filter that could be removed from the strategy are: Line #27: quality of life.mp. or exp "quality of life"/ Line #30: life expectancy.mp. or exp life expectancy/ Line #51: exp patient satisfaction/ The latter two lines were not relevant and the former, while relevant, most often refers to a more generalised notion of quality of life and not specifically to the more restrictive concept of HRQL. One other key limitation was the omission of the full title of the condition, chronic obstructive pulmonary disease from the COPD facet. Resource identification, measurement and valuation Searches were carried out on all databases required by NICE. The search date was reported for all databases though the date span was not accurately reported for Medline, Medline In-Process. There was some inconsistency in the use of date limits. Embase and NHS EED searches were limited by date but Medline, Medline In-Process and EconLIT searches were not. The research question could be inferred from the inclusion criteria (MS, p213) as resource use in the UK for the COPD population with a date limit of 1999 to present. The searches in this chapter were an appropriate translation of the research question and were considered adequate by the ERG. One important limitation of the Medline, Medline In-Process, Embase and NHS EED searches was the omission of the full title of the condition, chronic obstructive pulmonary disease, from the strategies. This would have adversely affected recall and may have caused studies to be missed. Summary of searching The clinical effectiveness and resource identification, measurement and valuation searching was carried out to an adequate standard. These searches had a number of limitations but were structurally sound and accurately reflected the research questions they were answering. 25

26 The searches for indirect and mixed treatment comparisons were structurally sound but the Cochrane Library database was omitted. The ERG addressed this by creating improved systematic searches for indirect and mixed treatment comparisons, and the results sent to the manufacturer for screening. The manufacturer screened these results but found no new studies that met the inclusion criteria. No specific searching for adverse events was carried out by the manufacturer. The ERG accepted the manufacturer s reasoning for not undertaking specific adverse events searches for roflumilast, but was unclear why searches were not undertaken for the comparators. No rationale was given for this in the MS or CR. The cost effectiveness and measurement and valuation of health effects searches were structurally unsound. The main cost effectiveness searches did not contain an intervention facet, and instead reduced hits down to manageable levels by introducing unnecessary utility terms, such as exacerbation into the strategy. These terms were also used to reduce the recall in the measurement and valuation of health effects searches. The ERG felt that a more focussed HRQL filter would have been more appropriate State the inclusion/exclusion criteria used in the study selection and comment on whether they were appropriate. 1. Review of roflumilast studies The inclusion and exclusion criteria used in the selection of evidence for the first systematic review were presented in table B1 (MS, page 40) of the MS. The table in the MS was labelled as eligibility criteria used in search strategy but was presented within the description of the study selection process (Section 5.2.1). It was not clear from the MS how many reviewers were involved in the study selection process. Best practice specifies that two reviewers be involved in the application of inclusion and exclusion criteria in order to limit bias in study selection. Details of the inclusion and exclusion criteria applied in the MS are presented in Table 4.1. Table 4.1: Inclusion/exclusion criteria used in roflumilast-study selection (as presented by the manufacturer) Clinical effectiveness Inclusion criteria Exclusion criteria Population patients with COPD Interventions - roflumilast Outcomes exacerbations and FEV 1 Study design randomised controlled trial Language restrictions publication in English Population patients who did not have COPD; healthy subjects Pharmacokinetics or cost effectiveness studies References relating solely to an entry on a clinical trials registry were excluded In addition to this search, the manufacturer was aware of one randomised controlled trial with roflumilast which was not published (M2-111). As this study met the inclusion and exclusion criteria of the systematic search it is included in the summary of RCTs in the MS. 26

27 COMMENT No further description of the population was provided in the MS for the above inclusion and exclusion criteria. In the statement of the decision problem, the population was described as adults with severe COPD (FEV 1 post bronchodilator less than 50% predicted) associated with chronic bronchitis in adult patients with a history of frequent exacerbations (MS, page 36), which is consistent with the final NICE scope. Frequent exacerbations were defined as 2 exacerbations in the previous year (MS, page 84). Comparators were not specified, indicating that all comparators were included. The MS stated that only RCTs were included in the analysis. It was not explicitly stated in the MS whether both phase II and phase III clinical trials were eligible for inclusion. The ERG consider non-rcts to be a valid and important source of evidence for the evaluation of adverse events. Controlled clinical trials may exclude patients at high risk from harms, 17 may be too short in terms of follow-up to detect long-term harms, may not have sufficiently large sample sizes to detect uncommon adverse events, or may not have reported them in a consistent manner. 18, 8, 19, 20 The MS does not provide a justification for not including non-rcts or observational studies in the submission. Most outcomes listed in the decision problem were present in the tabulated results in the MS (MS, section 5.5). The exclusion of studies not available in English was a reasonable decision on the basis of available time. The potential implications of this decision were not discussed. 2. Mixed treatment comparison The inclusion and exclusion criteria used in the selection of evidence for the second systematic review were presented in table B24 (MS, page 92) of the MS. The table in the MS was labelled as eligibility criteria used in search strategy but was presented within the description of the section on Indirect and mixed treatment comparisons (MS, Section 5.7). According to the MS the databases were searched independently and in duplicate. Best practice specifies that two reviewers be involved in the application of inclusion and exclusion criteria in order to limit bias in study selection. Details of the inclusion and exclusion criteria applied in the MS are presented in Table

28 Table 4.2: Inclusion/exclusion criteria used in MTC-study selection (as presented by the manufacturer) Clinical effectiveness Inclusion criteria Exclusion criteria Study design randomised controlled trials Study duration at least 24 weeks (6 months) Population - moderate to severe COPD patients, defined by GOLD/NICE criteria (FEV 1 80% predicted) Interventions LABA (formoterol or salmeterol), LAMA (tiotropium), ICS (fluticasone or budesonide), roflumilast, and any combinations of these interventions Outcome exacerbations of COPD Language restrictions publication in English Study duration less than 24 weeks (6 months) Pharmacokinetic studies and proof of concept studies Language restrictions publication not in English The mixed treatment comparison meta-analysis was based on the rates of exacerbations. Other outcomes listed in the decision problem were not used in the mixed treatment comparison. COMMENT The population for the mixed treatment comparison is described as moderate to severe COPD patients, defined by GOLD/NICE criteria (FEV 1 80% predicted). In the statement of the decision problem, the population was described as adults with severe COPD (FEV 1 post bronchodilator less than 50% predicted) associated with chronic bronchitis in adult patients with a history of frequent exacerbations (MS, page 36), which is consistent with the final NICE scope. Frequent exacerbations were defined as 2 exacerbations in the previous year (MS, page 84). Therefore, the populations for both systematic reviews (roflumilast studies and MTC) are different and neither is consistent with the population as described in the statement of the decision problem or the NICE scope What studies were included in the clinical effectiveness review and what were excluded? Provide a table of identified studies. Please identify the most important clinical effectiveness studies. 1. Review of roflumilast studies The flow diagram in the MS (fig. 4, page 41) shows that 5 publications which report 7 RCTs were included, as well as 5 meeting abstracts which report on 2 RCTs. Details of the studies and their populations as presented in the MS (Table B2, page 43) are presented in Table

29 Table 4.3: Studies included in the systematic review of roflumilast studies. Trial no. (acronym) Intervention Comparator Population Primary study ref. M2-124 & M2-125 Roflumilast 500 µg M2-127 Roflumilast 500 µg o.d. and salmeterol (50 µg b.i.d.) M2-128 Roflumilast 500 µg o.d. and tiotropium (18 µg) M2-111 Roflumilast 500 µg M2-112 Roflumilast 500 µg M2-107 (All further details are provided in Appendix 14, MS) Roflumilast 250 µg and 500 µg FK1-101 Roflumilast 250 µg and 500 µg FK1-103 Roflumilast 500 µg Grootendorst et al Roflumilast 500 µg Placebo Placebo and salmeterol (50 µg b.i.d.) Placebo and tiotropium (18 µg) Placebo Placebo Placebo Placebo Placebo Placebo Patients with severe or very severe COPD (FEV 1 50% predicted) with associated chronic bronchitis and a history of moderate or severe exacerbations (defined as having at least one recorded COPD exacerbation requiring systemic glucocorticosteroids or treatment in hospital, or both, in the previous year). Patients with moderate to severe COPD (post-bronchodilator FEV % predicted). Patients were not required to have chronic bronchitis or a history of exacerbations. Patients with moderate to severe COPD (post-bronchodilator FEV % predicted) with associated chronic bronchitis and frequent use of as-needed SABA (at least 28 puffs per week) during run-in. Patients with severe or very severe COPD (post-bronchodilator FEV 1 50% predicted). Patients were not required to have chronic bronchitis or a history of frequent exacerbations Patients with severe or very severe COPD (post-bronchodilator FEV 1 50% predicted). Patients were not required to have chronic bronchitis or a history of frequent exacerbations. Patients with moderate to severe COPD (post-bronchodilator FEV % predicted). Patients were not required to have chronic bronchitis or a history of frequent exacerbations. Patients with moderate to severe COPD (post-bronchodilator FEV % predicted). Patients were not required to have chronic bronchitis or a history of frequent exacerbations. Patients with moderate to severe COPD (post-bronchodilator FEV % predicted). Patients were not required to have chronic bronchitis or a history of frequent exacerbations. Patients with moderate to severe COPD (post-bronchodilator FEV % predicted). Patients were not required to have chronic bronchitis or a history of frequent exacerbations. Calverley et al, Fabbri et al, Fabbri et al, Clinical Study Report for trial M Calverley et al, Rabe et al, Bredenbroker et al, ; Bredenbroker at al, ; Leichtl et al, ; Leichtl et al, Boszormenyi- Nagy et al, Grootendorst et al,

30 COMMENT In the table above 5 publications, a Clinical Study Report and 5 conference abstracts are listed, reporting on 10 RCTs. Apparently, the flow diagram failed to mention one additional unpublished study (M2-111) which was added by the manufacturer. In 5 studies roflumilast was compared with placebo; the remaining 2 studies compared roflumilast + LABA with placebo + LABA and roflumilast + LAMA with placebo + LAMA, respectively. Four of the 10 studies included patients with severe COPD (FEV 1 50% predicted). The other 6 studies included patients with moderate to severe COPD. Looking at the table above, none of these four included trials in severe COPD directly compare roflumilast with the comparators listed in the decision problem. However, around 50% of patients included in trials M2-124 and M2-125 received concomitant LABA. For these patients the comparison is roflumilast + LABA versus LABA + placebo. None of the roflumilast trials included patients with a history of frequent exacerbations, using a definition of two or more exacerbations in the previous year. 2. Mixed treatment comparison Twenty-six publications met the inclusion criteria for the MTC (the publication by Fabbri et al seems to have been counted twice). Combined, these publications reported on 29 clinical trials. A table of the 26 studies used to conduct the MTC was presented as Table B25 on page 95 of the MS and summarised in the table below. The relevant network diagram is presented in Figure 12 on page 115 of the MS (see Figure 4.3). 742 Fluticasone Proprionate Tiotropium 18 µg od (550) vs 625 Formoterol 12 or 24 µg (425) 1,207 Tiotropium 18 µg od (402) vs 1,022 Budesonide/formoterol 2x160 1,465 Salmeterol 50 µg bid (372) vs Table 4.4: Trials Included in the Primary Analysis (i.e. those which included data on the total number of exacerbations and/or the mean annual rate of exacerbations) Trial Ref # of Patients Treatment Comparisons Duration of Treatment Baseline FEV 1 % Predicted (Inclusion criteria) Paggiaro et al, Fluticasone proprionate 500 µg bid (142) vs Placebo bid (139) 6 months FEV % Burge et al, 3 years FEV 1 < 85% µg bid (372) vs Placebo (370) Casaburi et al., 1 year FEV 1 65% Placebo (371) Rossi et al., 1 year FEV 1 <70% vs Placebo (200) Brusasco et al., 6 months FEV 1 65% Salmeterol 50 µg bid (405) vs Placebo (400) Calverley et al., 1 year FEV 1 50% µg/4.5 µg bid (254) vs Formoterol 2x4.5 µg bid (255) vs Budesonide 2x200 µg bid (257) vs Placebo (256) Calverley et al., 1 year FEV % Fluticasone 500 µg bid (374) vs Salmeterol/fluticasone 50/500 µg bid (358) vs 30

31 1,829 Tiotropium 18 µg od (914) vs 1,010 Tiotropium 18 µg od (500) vs 449 Tiotropium 18 µg od plus Placebo (361) Szafranski et al., Formoterol 4.5 µg bid (201) vs Budesonide/formoterol 160/4.5 µg bid (208) vs Budesonide 200 µg bid (198) vs Placebo (205) Niewoehner et al., Placebo (915) Rabe et al., 1,411 Roflumilast 250 µg or 500 µg 2005 (M2- od (1,131) vs Placebo (280) 107) 17 Dusser et al., Placebo (510) Stockley et al., 634 Salmeterol 50 µg bid (316) vs Placebo (318) Aaron et al., salmeterol 2x25 µg bid (148) vs Tiotropium 18 µg od plus fluticasone/salmeterol 2x250/25 µg bid (145) vs Tiotropium 18 µg od plus Placebo (156) Calverley et al., ,112 Salmeterol 50µg bid (1,521) vs Fluticasone 500µg bid (1,534) vs Salmeterol/fluticasone 50 µg/500 µg bid (1,533) vs Placebo (1,524) Calverley et al., 2007 (M2-112) 21 1,513 Roflumilast 500 µg od (760) vs Placebo (753) 913 Tiotropium 18 µg od (608) vs Chan et al., Placebo (305) Kardos et al., 994 Salmeterol 50 µg bid (487) vs Salmeterol/fluticasone 50/500 µg bid (507) Powrie et al., Tiotropium 18 µg od (69) vs Placebo (73) Ferguson et al., 776 Salmeterol 50 µg bid (385) vs Fluticasone propionate/salmeterol 250/50 µg bid (391) Tashkin et al., ,993 Tiotropium 18µg od (2,987) vs Placebo (3,006) Tonnel et al., 554 Tiotropium 18 µg od (266) vs Placebo (288) Wedzicha et al., 1,323 Salmeterol/fluticasone /500µg bid (658) vs tiotropium 18 µg od (665) Anzueto et al., Salmeterol 50 µg bid (393) vs fluticasone propionate/salmeterol 250/50µg bid (385) Calverley et al., 2009 (M2-124 and M2-125) 1 3,091 Roflumilast 500 µg od (1,537) vs Placebo (1,554) 933 Salmeterol 50 µg bid plus Fabbri et al., 2009 (M2-127) 6 Roflumilast 500 µg od (466) vs Salmeterol 50 µg bid plus Placebo bid (467) 1 year FEV 1 50% 6 months FEV 1 < 60% 6 months FEV % 1 year FEV % 1 year FEV 1 <70% 1 year FEV 1 <65% 3 years FEV 1 <60% 1 year FEV 1 50% 48 weeks FEV 1 <65% 44 weeks FEV 1 <50% 1 year FEV 1 <80% 1 year FEV 1 50% 4 years FEV 1 <70% 9 months FEV % 2 years FEV 1 <50% 1 year FEV 1 50% 1 year FEV 1 50% 6 months FEV % 31

32 743 Tiotropium 18 µg od plus Fabbri et al (M2-128) 6 roflumilast 500 µg od (371) vs Tiotropium 18 µg od plus Placebo (372) 6 months FEV % Figure 4.3: The network of 10 treatments included in the MTC analysis of exacerbations Each node represents a treatment in the network. The links between the nodes represent a direct comparison between pairs of treatments. The numbers shown along the links indicate the number of trials comparing pairs of treatments head to head. COMMENT The inclusion criteria for the MTC are not in accordance with the NICE scope. Therefore, the ERG will focus on trials that fulfil the inclusion criteria from the NICE scope as best as possible. According to the NICE scope the population is defined as: Adults with severe chronic obstructive pulmonary disease (FEV 1 post bronchodilator less than 50% predicted) associated with chronic bronchitis in adult patients with a history of frequent exacerbations. The numbers of patients with a history of frequent exacerbations is unclear in most trials. As seen before, none of the roflumilast trials included patients with a history of frequent exacerbations, using a definition of two or more exacerbations in the previous year. However, the severity of COPD at 32

33 baseline is clearly reported in all trials and only 8 out of 26 included trials fulfil this criterion. In addition one more roflumilast trial was identified that fulfilled this inclusion criterion (M2-111). Therefore, the ERG will focus on these nine trials for their primary analyses (see table below). Table 4.5: Trials Included in the Primary ERG Analysis (i.e. those which included patients with severe COPD, FEV 1 50%) Trial Ref # of Patients Treatment Comparisons Duration of Treatment Baseline FEV 1 % Predicted (Inclusion Calverley et al., ,022 Budesonide/formoterol 2x160 µg/4.5 µg bid (254) vs Formoterol 2x4.5 µg bid (255) vs Budesonide 2x200 µg bid (257) vs Placebo (256) Szafranski et al., 812 Formoterol 4.5 µg bid (201) vs Budesonide/formoterol 160/4.5 µg bid (208) vs Budesonide 200 µg bid (198) Calverley et al., 2007 (M2-112) 21 vs Placebo (205) 1,513 Roflumilast 500 µg od (760) vs Placebo (753) Kardos et al., 994 Salmeterol 50 µg bid (487) vs Salmeterol/fluticasone 50/500 µg bid (507) Ferguson et al., Salmeterol 50 µg bid (385) vs Fluticasone propionate/salmeterol 250/50 µg bid (391) Wedzicha et al., 1,323 Salmeterol/fluticasone /500µg bid (658) vs tiotropium 18 µg od (665) Anzueto et al., Salmeterol 50 µg bid (393) vs fluticasone propionate/salmeterol 250/50µg bid (385) Calverley et al., 2009 (M2-124 and M2-125) 1 3,091 Roflumilast 500 µg od (1,537) vs Placebo (1,554) M ,173 Roflumilast 500 µg od (567) vs Placebo (606) criteria) 1 year FEV 1 50% 1 year FEV 1 50% 1 year FEV 1 50% 44 weeks FEV 1 <50% 1 year FEV 1 50% 2 years FEV 1 <50% 1 year FEV 1 50% 1 year FEV 1 50% 1 year FEV 1 50% According to the description of trials in the MS, this leaves 6 treatments in the MTC. However, Calverley et al (M2-124 and M2-125) is described as roflumilast versus placebo, whilst about 50% of patients in both trials received concomitant LABA. Therefore, this trial should be treated as partly roflumilast vs. placebo; and partly roflumilast + LABA vs LABA. This would result in a network of 7 treatments for the MTC (see Figure 4.4). 33

34 Figure 4.4: The network of 7 treatments included in the MTC analysis of exacerbations 2 ICS 2 LABA 5 2 ICS+ LABA Rofl + LABA Pla 1 LAMA 3 Rofl Potentially there is one more roflumilast combination (roflumilast + LAMA) that could be included in our analyses. However, roflumilast + LAMA has only been tested in one trial (Fabbri et al. 2009, M2-128) which included patients with FEV 1 between 40 and 70%; therefore, these results are not comparable with results from other trials in our network (see also Chapter 3). The MS presents results for 15 treatments for the management of COPD. Seven treatments have been included in the ERG base-case analyses. Two treatments (LABA/ICS+LAMA and LABA+LAMA) can be included in the network if we widen the inclusion criteria to FEV 1 65%. Although this is not the population as described in the scope, the ERG deemed this approach acceptable, since the Appraisal Committee will be particularly interested in the relative effectiveness of roflumilast in comparison with LABA/ICS+LAMA as this was found to be the most effective treatment not involving roflumilast in the MS. Roflumilast + LAMA has only been tested in one trial (Fabbri et al. 2009, M2-128) which included the wrong population. The remaining 5 treatments (LAMA+ICS, roflumilast+ics, roflumilast+laba+lama, roflumilast+laba+ics, and roflumilast+laba +LAMA+ICS) have not been used in severe or moderate to severe COPD. These treatments were assessed based on the additive model, for which the ERG does not agree with the underlying assumptions. Therefore, as no trial evidence is presented in the MS for any of these five treatments, the ERG will ignore these treatments in this report. Widening the inclusion criteria to FEV 1 65% adds six trials to the network (see Table 4.6 and Figure 4.5). One trial (Dusser et al., ) included COPD patients with FEV % and was excluded because very severe patients (FEV 1 30%) were explicitly excluded in the trial. 34

35 1,829 Tiotropium 18 µg od (914) vs 449 Tiotropium 18 µg od plus 6,112 Salmeterol 50µg bid (1,521) 913 Tiotropium 18 µg od (608) vs Table 4.6: Trials Included in the secondary ERG Analysis (i.e. those which included patients with FEV 1 65%) Trial Ref # of Patients Treatment Comparisons Duration of Treatment Baseline FEV 1 % Predicted (Inclusion criteria) Casaburi et al., 921 Tiotropium 18 µg od (550) vs 1 year FEV 1 65% Placebo (371) Brusasco et al., ,207 Tiotropium 18 µg od (402) vs Salmeterol 50 µg bid (405) vs Placebo (400) 6 months FEV 1 65% Niewoehner et 6 months FEV 1 < 60% al., Placebo (915) Aaron et al., 1 year FEV 1 <65% salmeterol 2x25 µg bid (148) vs Tiotropium 18 µg od plus fluticasone/salmeterol 2x250/25 µg bid (145) vs Tiotropium 18 µg od plus Placebo (156) Calverley et al., 3 years FEV 1 <60% vs Fluticasone 500µg bid (1,534) vs Salmeterol/fluticasone 50 µg/500 µg bid (1,533) vs Placebo (1,524) Chan et al., 48 weeks FEV 1 <65% Placebo (305) Figure 4.5: The network of 9 treatments included in the MTC analysis of exacerbations Rofl + 1 LAB A 4 3 ICS 2 Pla ICS+ LAB LAM A LAB A+ Rofl ICS+ LAB 1 35

36 In conclusion, the ERG will present results for two sets of data: 1. Data for adults with severe COPD (FEV 1 post bronchodilator less than 50% predicted) from 10 trials, including 7 treatments. 2. Data for adults with moderate to severe COPD (FEV 1 post bronchodilator less than 65% predicted) from 16 trials, including 9 treatments. A table with baseline characteristics for the trials included in the ERG analyses is presented in Appendix 2. The NICE scope mentions that if the evidence allows, subgroup analysis should be considered for COPD severity, smoking status and Body Mass Index of study participants, at baseline. However, the majority of the studies do not report these characteristics with the exception of three (M2-111, M2-124, M2-125), which report all three. The ERG noted substantial differences in baseline characteristics between the studies used in the analysis (FEV 1 50% and FEV 1 65%). The majority of studies used mainly male populations (8 out of 9 studies from base case and 3 out of 6 from FEV 1 65% had two-thirds or more male participants, with Niewoehner et al. at the extreme of 99% male); notable exceptions were: Aaron et al., Anzueto et al. and Fergusson et al. (with approximately 55% male each). Only four studies (M2-111, M2-124, M2-124 and Wedzicha et al, 2008) reported severity of COPD at baseline. Also, number and/or mean rate of exacerbations at baseline is reported by three studies (Kardos et al, 2007, Wedzicha et al, 2008, Calverley et al, 2007). The Body Mass Index at baseline was reported by 6 studies (Anzueto et al, 2009, M2-112, Fergusson et al, 2008, M2-111, M2-124 and M2-125), and HRQoL at baseline is reported by only 3 studies (Calverely et al, 2003, M2-111 and Wedzicha et al, 2008). Comment Differences in baseline characteristics may present substantial heterogeneity between studies, especially where male or female participants are concerned. There is evidence that suggests that COPD prevalence rates in women have been on the increase since mid-1990s and so has the mortality levels in comparison to the male population, which have been on the decline. 2, 3 Therefore, generalisability of the study results might be questionable Provide details of any relevant studies not discussed in the submission? Why were these studies excluded and how were these studies identified by the ERG? The ERG identified significant limitations to the manufacturer s searches in the cost effectiveness and measurement and valuation of health effects chapters. While the ERG considered it very possible that relevant studies were missed, time constraints did not allow for the ERG to undertake improved searches and screening results. No specific adverse events searches were carried out for the comparator drugs. It was possible that since the indirect and mixed treatment comparison searches were limited to RCTs then some evidence might be available for the comparator drugs which was not identified in the MS. One of the inclusion criteria for the MTC was that trials had to report data on the total number of exacerbations and/or the mean annual rate of exacerbations. Therefore, trials reporting other relevant outcomes (as mentioned in the scope) were excluded. It is unclear from the MS which trials were 36

37 excluded because they did not report exacerbations. The MS does report three additional trials (MS, Table B26, page 112) which reported the binomial outcome of Ever Having an Exacerbation Event for those trials which provided data on the proportion of patients with at least 1 exacerbation during the trial. However, trials reporting other relevant outcomes are not mentioned. As explained in the previous section, the ERG focussed on trials including populations as defined in the NICE scope, i.e. adults with severe COPD (FEV 1 post bronchodilator less than 50% predicted). Therefore trials in patients with moderate to severe COPD were not included in our primary analyses. In a secondary analysis the ERG widened the inclusion criteria to FEV 1 post bronchodilator less than 65% predicted. All other trials included in the MS were excluded in this report. This means that the following trials which were included in the MTC in the MS, were excluded from the ERG analyses: Table 4.7: Trials Excluded in the ERG Analysis Trial Ref # of Patients Treatment Comparisons Paggiaro et al, Fluticasone proprionate 500 µg bid (142) vs Placebo bid (139) Burge et al, 742 Fluticasone Proprionate 500 µg bid (372) vs Placebo (370) Rossi et al., 625 Formoterol 12 or 24 µg (425) vs Placebo (200) Calverley et al., 1,465 Salmeterol 50 µg bid (372) vs Fluticasone 500 µg bid (374) vs Salmeterol/fluticasone 50/500 µg bid (358) vs Placebo (361) Rabe et al., ,411 Roflumilast 250 µg or 500 µg od (M2-107) 17 (1,131) vs Placebo (280) Dusser et al., 1,010 Tiotropium 18 µg od (500) vs Placebo (510) Stockley et al., 634 Salmeterol 50 µg bid (316) vs Placebo (318) Powrie et al., 142 Tiotropium 18 µg od (69) vs Placebo (73) Tashkin et al., 5,993 Tiotropium 18µg od (2,987) vs Placebo (3,006) Tonnel et al., 554 Tiotropium 18 µg od (266) vs Placebo (288) Fabbri et al., Salmeterol 50 µg bid plus (M2-127) 6 Roflumilast 500 µg od (466) vs Salmeterol 50 µg bid plus Placebo bid (467) Fabbri et al Tiotropium 18 µg od plus (M2-128) 6 roflumilast 500 µg od (371) vs Tiotropium 18 µg od plus Placebo (372) Duration of Treatment Baseline FEV 1 % Predicted (Inclusion criteria) 6 months FEV % 3 years FEV 1 < 85% 1 year FEV 1 <70% 1 year FEV % 6 months FEV % 1 year FEV % 1 year FEV 1 <70% 1 year FEV 1 <80% 4 years FEV 1 <70% 9 months FEV % 6 months FEV % 6 months FEV % 37

38 4.2 Summary and critique of submitted clinical effectiveness evidence If there is more than one RCT described in the MS, it may be appropriate to discuss each trial individually using the headings described Summary of submitted clinical evidence for each relevant trial. The MS does not present any data for non-roflumilast trials. Only results from the additive MTC analysis in terms of estimated rate ratios of exacerbations are presented (MS, Table B30, page 121). As this analysis includes both trials in the wrong population and treatments that have never been tested in COPD patients, the ERG will ignore these results. Specific data from trials included in the ERG basecase (FEV 1 50% predicted) and alternative (FEV 1 65% predicted) analyses are presented in tables 4.8 and 4.9. Symptom control was included in the NICE Scope, but has not been included in the results tables below for reasons of clarity. Several studies did not report measures of symptom control (Calverley 2007 (34); Wedzicha 2008; Calverley 2007 (56); Chan 2007; Niewoehner 2005). When symptom control was reported it was measured in a variety of ways. Some studies used symptom scores (shortness of breath, cough, chest tightness and night time awakenings (Calverley 2003; Szafranski 2003). Other measures used included COPD symptom score (M2-111); transition dyspnea index (Calverley 2009 (M2-124 & 125); dyspnea scores and night time awakenings (Anzueto 2009; Ferguson 2008); Medical Research Council dyspnea score (Kardos 2007); Baseline dyspnea index and transition dyspnea index (Brusasco 2003; Casaburi 2002); and transition dyspnea domain of the Chronic Respiratory Disease Questionnaire (Aaron 2007). In addition to the range of measures, data was presented in a variety of ways: change from baseline or comparison with placebo or active treatments. A variety of measures were used for FEV 1 and the post bronchodilator measure used in the roflumilast trials is not reported by many other studies or not reported as change from baseline. Adverse events were reported by the majority of studies, with the exception of Brusasco et al, SGRQ was broadly reported and only two studies do not report this outcome (Calverley M2-124 and M2-125, and Niewoehner et al, 2005). The SGRG scores vary significantly across studies. The majority of the studies do not report SGRG baseline scores, making it very difficult to assess differences between baseline and post-treatment. 38

39 Table 4.8 Results for severe COPD (FEV1 50% predicted) Comparison Trial Arm N Annual of rate exacerbat ions Roflumilast Vs placebo Roflumilast + LABA Vs LABA LABA placebo LABA ICS Vs Vs LABA Vs LABA+ICS ICS placebo Vs ICS Vs ICS+LABA ICS+LABA Vs placebo ICS+LABA Vs LAMA PBD FEV1 (ml) Δ from baseline (95% CI) Outcomes Mortality (%) Serious adverse events (%) SGRQ (mean Δ from baseline) M2-111 Rof NR 11 (1.94) 126a (22.2) Placebo NR 12 (1.98) 132a (21.8) Calverley 2007 (M2-112) Calverley 2009 (M2-124 & M2 125)a Calverley 2009 M2-124 and M2-125b Calverley 2003 Szafranski 2003 Calverley 2003 Szafranski 2003 Rof (1.85) 137a (18) -1.7 Placebo (2.66) 132a (17.5) -2.0 Rof * 50** 42 (2.73)** 301 (19)** NR Placebo * (2.57) 336 (22) NR Rof+ LABA * NR NR NR NR Placebo (LABA) * NR NR NR NR LABA NR 13 (5.1) 85 (33.33) +0.7 Placebo NR 5 (1.95) 66 (25.78) +4.8 LABA NR 6 (2.99) 37 (18.41) -3.6 Placebo NR 9 (4.39) 37 (18.05) ICS NR 6 (2.33) 88 (32.24) +1.0 LABA * NR 13 (5.1) 85 (33.33) +0.7 ICS NR 5 (2.53) 35 (17.68) -1.9 LABA NR 6 (2.99) 37 (18.41) -3.6 Anzueto 2009 LABA+ICS * NR 4 (1.04) 81a (21) LABA * NR 6 (1.53) 71a (18) Calverley 2003 Ferguson 2008 LABA+ICS NR 5 (1.97) 65 (25.6) -2.5 LABA NR 13 (5.1) 85 (33.33) +0.7 LABA+ICS NR 6 (1.53) 86a (22) LABA NR 3 (7.79) 77a (20) Kardos 2007 LABA+ICS * 70 7 (1.38) 76a (15) -2.9 LABA * 50 9 (1.85) 88a (18.1) -0.7 Szafranski 2003 Calverley 2003 Szafranski 2003 Calverley 2003 Szafranski 2003 Calverley 2003 Szafranski 2003 Wedzicha 2008 LABA+ICS NR 6 (2.88) 43 (20.67) -3.9 LABA NR 6 (2.99) 37 (18.41) -3.6 ICS * NR 6 (2.33) 88 (34.24) +1.0 Placebo NR 5 (1.95) 66 (25.78) +4.8 ICS NR 5 (2.53) 35 (17.68) -1.9 Placebo NR 9 (4.39) 37 (18.05) ICS+LABA NR 5 (1.97) 65 (25.6) -2.5 ICS * NR 6 (2.33) 88 (34.24) +1.0 ICS+LABA NR 6 (2.88) 43 (20.67) -3.9 ICS NR 5 (2.53) 35 (17.68) -1.9 ICS+LABA NR 5 (1.97) 65 (25.6) -2.5 placebo NR 5 (1.95) 66 (25.78) +4.8 ICS+LABA NR 6 (2.88) 43 (20.67) -3.9 placebo NR 9 (4.39) 37 (18.05) ICS+LABA (3.19) 197a (30) -1.7 LAMA (5.71) 160a (24) a Absolute numbers have been calculated by ERG when only % reported. * Indicates where the ERG collected data varies from the MS; ** Results reported for total population (with and without LABA); PBD FEV1 = post bronchodilator FEV1; SGRQ=St. George s Respiratory Questionnaire - M2-111 only reports least squares mean change from baseline for post-bronchodilator FEV1. - Calverley 2009 (M2-124 and M2-125) has been divided into two, a and b, as some patients in the roflumilast group also received LABA and some patients in the placebo also received LABA. - Calverley 2009 (M2-124 & M2 125) uses EQ-5D to measure health utility. - Anzueto used mean morning pre-dose FEV1 rather than post bronchodilator FEV1 - Ferguson used pre-dose FEV1. - % are used for severe adverse events when only percentages were reported in study report and absolute numbers have been calculated by the ERG. - M2-124 and 125 used EQ-5D to measure health related quality of life, but change from baseline score was not reported. - Calverley 2007 FEV1 values read from graph. - Szafranski used mean ratio in FEV1 % of baseline. - Wedzicha 2008 reports adjusted mean change for FEV1 (covariates of baseline value, baseline smoking status, disease severity, age, gender, and count 39

40 Table 4.9 Results for COPD where FEV1 65% predicted Comparison Studies Arms N Annual of rate exacerbat ions LABA Vs Brusasco Placebo 2003 LABA Vs ICS LABA Vs ICS + LABA ICS Vs ICS + LABA LAMA Placebo Vs ICS + LABA Vs placebo LABA LAMA Vs Calverley 2007 (ref 56) Calverley 2007 (ref 56) Calverley 2007 (ref 56) Calverley 2007 (ref 56) Brusasco 2003 Casaburi 2002 PBD FEV1 (ml) Δ from baseline (95% CI) Outcomes Mortality (%) Serious adverse events SGRQ (mean Δ from baseline) LABA * NR 6 (1.48) NR -2.8 Placebo * NR 5 (1.25) NR -1.5 LABA 1, * (13.48) 40 (2.63) -0.8 Placebo 1, * (15.12) 41 (2.7) +0.2 ICS 1, * (16.04) 42 (2.74) -1.8 LABA 1, * (13.48) 40 (2.63) -0.8 ICS + 1, * (12.56) 43 (2.8) -3.0 LABA LABA 1, * (13.48) 40 (2.63) -0.8 ICS + 1, * (12.56) 43 (2.8) -3.0 LABA ICS 1, * (16.04) 42 (2.74) -1.8 LAMA * NR 1 (0.25) NR -4.2 Placebo * NR 5 (1.25) NR -1.5 LAMA * (1.27) 99 (18) -3.2 Placebo * (1.89) 78 (21.02) +0.5 Chan 2007 LAMA NR 13 (2.14) 112a (18.4) -5.6 Niewoehner 2005 Calverley 2007 (Ref 56) Brusasco 2003 LAMA Vs LABA+LAMA Aaron 2007 LAMA Vs ICS+LABA+ LAMA ICS+LABA+ LAMA Vs LABA+LAMA Aaron 2007 Aaron 2007 Placebo NR 2 (0.66) 43a (14.1) -2.8 LAMA * NR 22 (2.41) 162 (17.72) NR Placebo * NR 19 (2.08) 156 (17.05) NR ICS + 1, * (12.56) 43 (2.8) -3.0 LABA Placebo 1, * (15.12) 41 (2.7) +0.2 LAMA * NR 1 (0.25) NR -4.2 LABA * NR 6 (1.48) NR -2.8 LAMA * NR 4 (2.56) 10 (6.41) -4.5 LABA+ LAMA * NR 6 (4.05) 9 (6.08) -6.3 ICS+LABA * NR 6 (4.14) 9 (6.21) LAMA LAMA * NR 4 (2.56) 10 (6.41) -4.5 ICS+LABA +LAMA LABA+ LAMA * NR 6 (4.14) 9 (6.21) * NR 6 (4.05) 9 (6.08) -6.3 * Indicates where the ERG collected data varies from the MS; PBD FEV1 = post bronchodilator FEV1; SGRQ=St. George s Respiratory Questionnaire - Brusasco uses mean FEV1 before and after treatment. - Calverley 2007 (ref 56): Total scores on the St. George s Respiratory Questionnaire and post-bronchodilator FEV1 were analyzed as changes from baseline values with the use of repeated measures analysis of covariance (ANCOVA) (p. 777) - Casaburi FEV1 read from graph - Chan used morning pre-dose FEV1. - Niewoehner values for FEV1 only presented in comparison to placebo. - Chan SGRQ baseline score read from graph - Casaburi SGRQ score read from graph - Aaron used prebronchodilator FEV1. In Tables 4.10 and 4.11 the effect sizes of the different comparisons in each trial are reported. Table 4.10 shows effectiveness results for trials in people with severe COPD (FEV 1 50% predicted). Table 4.11 shows results for trials in people with COPD up to 65% FEV 1 predicted. As can be seen in both tables, roflumilast was only compared to placebo and Roflumilast + LABA was compared to LABA. 40

41 Table 4.10 Overview of results for severe COPD (FEV 1 50% predicted) Comparison Trial Arm N Rate Ratio of exacerbations P value, (95% CI) Roflumilast Vs placebo Roflumilast + LABA Vs LABA LABA Vs placebo LABA Vs ICS LABA Vs LABA+ICS ICS Vs placebo ICS Vs ICS+LABA ICS+LABA Vs placebo ICS+LABA Vs LAMA M2-111 Rof Placebo 606 P= (0.713 to 1.051) Calverley Rof Placebo 753 P=0.451 (M2-112) (NR) Calverley Rof (M2-124 Placebo 761 P= & M2 125)a (0.736 to 0.992) Calverley 2009 M2-124 and M2-125b Calverley 2003 Szafranski 2003 Calverley 2003 Szafranski 2003 Anzueto 2009 Calverley 2003 Ferguson 2008 Kardos 2007 Szafranski 2003 Calverley 2003 Szafranski 2003 Calverley 2003 Szafranski 2003 Calverley 2003 Szafranski 2003 Wedzicha 2008 Rof+ 749 LABA Placebo 793 (LABA) LABA 255 Placebo P= (0.690 to 0.911) NR LABA Placebo 205 P=0.895 NR ICS 257 NR LABA 255 ICS 198 LABA 201 NR LABA+ ICS LABA P=<0.001 (0.53 to 0.81) LABA ICS P=0.828 LABA 255 (0.587 to 0.945) LABA ICS P=0.001 LABA 385 (0.58 to 0.83) LABA ICS P=< LABA 487 (0.57 to 0.76) LABA ICS P=0.043 LABA 201 NR ICS 257 Placebo 256 NR ICS Placebo 205 P=0.224 NR ICS+LA BA P=0.308 ICS 257 (0.679 to 1.100) ICS+LA BA P=0.385 ICS 198 NR ICS+LA BA placebo P= NR (0.600 to 0.973) ICS+LA BA P=0.035 placebo 205 NR ICS+LA BA P=0.656 LAMA 665 (0.87 to 1.12) Outcomes PBD FEV1 LS MEANS ±SE P-value (95% CI) 38mL P= NR 39±12 P=0.001 NR Mortality OR (95%CI) 0.98 (0.43, 2.24) 0.59 (0.29,1.21) (0.029, 0.064) (0.029, 0.071) NR P=<0.001 NR 14% increase P=<0.001 ( ) NR NR 84±26 P=0.001 (NR) NR P=0.002 NR 74±27 P=<0.04 NR NR P=0.63 (NR) 1% increase P=0.487 (-2.2, 4.9) NR P=0.145 NR 5% increase P=0.005 (1.5, 9.1) NR P=<0.001 NR 9% increase P=<0.001 (5.4, 13.1) NR P=0.001 NR 15% increase P=<0.001 (11.0, 19.1) P=0.218 (-0.06 to 0.01) (0.65,1.58) 0.37 (0.13,1.06) 0.67 (0.23,1.92) 0.44 (0.17,1.19) 0.84 (0.25, 2.80) 0.68 (0.19,2.42) 0.37 (0.13, 1.06) 1.98 (0.49, 7.99) 0.74 (0.27, 2.01) 0.97 (0.31, 3.04) 1.20 (0.36, 3.98) 0.56 (0.19,1.71) 0.84 (0.25, 2.79) 1.15 (0.34, 3.82) 1.01 (0.29, 3.53) 0.65 (0.23, 1.85) 0.54 (0.32, 1.92) Serious Adverse events OR (95%CI) 0.85 (0.65, 1.12) 1.03 (0.79, 1.35) 0.78 (0.64, 0.96) 1.44 (0.98, 2.11) 1.02 (0.62, 1.70) 1.04 (0.72, 1.50) 0.95 (0.57, 1.59) 1.21 (0.85, 1.72) 0.69 (0.47, 1.01) 1.13 (0.80, 1.59) 0.80 (0.57, 1.12) 1.16 (0.71, 1.89) 1.50 (1.02, 2.19) 0.97 (0.59, 1.62) 0.66 (0.45, 0.97) 1.21 (0.74, 1.99) 0.99 (0.67, 1.47) 1.18 (0.73, 1.93) 1.35 (1.06, 1.72) SGRG Difference between treatments p-value (95%CI) P= NR NR P=0.651 NR ± P= ( ,0.0129) NR P=0.01 NR NR NR NR units P=0.371 NR NR P=0.014 NR units P=0.035 NR -2.3 units P= NR NR NR P=0.05 NR NR NR P=0.001 NR NR NR P=<0.001 NR 3.9 P=0.009 NR 2.1 P=0.038 (-4.0 to -0.1) 41

42 As can be seen in the table above, roflumilast alone reduces the annual rate of exacerbations when compared to placebo across the three studies; however, only one of the studies demonstrated a statistically significant result (M2-124 and M2-125 a, RR=0.854 (95% CI: to 0.992)). Most other treatments also showed mainly non-significant results in terms of exacerbations when compared with placebo. Only ICS/LABA showed significant reductions in the annual rate of exacerbations when compared to placebo across two studies. Roflumilast was better than placebo in terms of post bronchodilator FEV 1 across three studies, all showing significant results. Similarly, all but one of the other treatments were significantly better when compared with placebo in terms of post bronchodilator FEV 1. For mortality, there was only one significant difference, favouring ICS+LABA when compared to LAMA (OR=0.54 (95% CI: 0.32, 0.94). None of the other trials showed significant differences in mortality between treatments. Most results for serious adverse events were not significant; however four studies did report significant results. Significantly more serious adverse events were reported for placebo when compared to roflumilast in one out of three trials, for ICS when compared to placebo in one out of two trials, for ICS when compared to ICS/LABA in one out of two trials, and for ICS/LABA when compared to LAMA in one out of one trial. There were numerous significant results for SGRG. However, only one of the three studies investigating roflumilast was able to show a significant difference compared to placebo (M2-111). The results from the other two roflumilast trials were not significant for SGRG. As with PBD FEV 1, other treatments also showed significant improvements compared with placebo LABA, ICS, and ICS+LABA. Only one other comparison involved roflumilast: roflumilast plus LABA versus LABA, based on data from M2-124 and M2-125 b ; this comparison showed that roflumilast plus LABA significantly reduced the annual rate of exacerbation in comparison to LABA alone (RR=0.793 (95% CI: (0.690 to 0.911)). Also in terms of post bronchodilator FEV 1 roflumilast plus LABA was significantly better than LABA alone. None of the other outcomes were reported separately for roflumilast plus LABA versus LABA in severe COPD. The pooled result for the annual rate of exacerbations for roflumilast versus placebo is reported in Figure 4.6, showing a significant effect in favour of rofumilast. Figure 4.6: Rate ratio of exacerbations for Roflumilast versus Placebo Rate Ratio Rate Ratio Study or Subgroup log[rate Ratio] SE Weight IV, Random, 95% CI IV, Random, 95% CI Calverley % 0.93 [0.84, 1.04] M % 0.87 [0.71, 1.06] M2-124 & M2 125a % 0.85 [0.73, 0.99] Total (95% CI) 100.0% 0.90 [0.83, 0.98] Heterogeneity: Tau² = 0.00; Chi² = 1.13, df = 2 (P = 0.57); I² = 0% Test for overall effect: Z = 2.58 (P = 0.01) Favours roflumilast Favours placebo 42

43 Table 4.11 Overview of results for COPD where FEV 1 65% predicted Comparison Studies Arms N Rate Ratio of exacerbations, P value, (95% CI) LABA Vs Placebo LABA Vs ICS LABA Vs ICS + LABA ICS Vs ICS + LABA LAMA Vs Placebo ICS + LABA Vs placebo LABA Vs LAMA LAMA Vs LABA+ LAMA LAMA Vs ICS+LABA+ LAMA ICS+LABA+ LAMA Vs LABA+ LAMA Brusasco 2003 Calverley 2007 (ref 56) Calverley 2007 (ref 56) Calverley 2007 (ref 56) Calverley 2007 (ref 56) Brusasco 2003 LABA 405 Placebo 400 NR LABA 1, Placebo 1,524 P=<0.001 (0.78,0.93) ICS 1,534 NR LABA 1,521 ICS + 1, LABA P=0.002 LABA 1521 (0.81,0.95) ICS + 1,533 LABA ICS 1,534 LAMA 402 Placebo P=0.02 (0.84,0.99) NR Casaburi LAMA 550 NR 2002 Placebo 371 P=0.045 (NR) Chan 2007 LAMA 608 NR Placebo 305 P=0.599 (NR) Niewoehner LAMA NR Placebo 915 Calverley 2007 (Ref 56) Brusasco 2003 Aaron 2007 Aaron 2007 Aaron 2007 ICS + 1, LABA P=<0.001 Placebo 1,524 (0.69,0.81) LAMA 402 LABA 405 NR LAMA LABA+ 148 LAMA ICS+LA 145 BA+LA MA LAMA 156 ICS+ LABA+ LAMA LABA+ LAMA P=0.51 (0.84, 1.40) 0.85 P=0.24 (0.65, 1.11) NR Outcomes PBD FEV1 LS MEANS±SE P-value (95% CI) NR P=<0.001 (0.025,0.058) NR P=<0.001 (0.034,0.067) P=<0.001 (0.028,0.061) NR NR NR NR P=<0.001 (0.075,0.108) NR NR NR NR Mortality OR (95%CI) 1.19 (0.36, 3.92) 0.87 (0.71, 1.07) 1.23 (1.00, 1.50) 0.92 (0.75, 1.14) 0.75 (0.62, 0.92) 0.20 (0.02, 1.69) 0.67 (0.23, 1.93) 3.31 (0.74, 14.76) 1.16 (0.63, 2.16) 0.81 (0.66, 0.99) 0.17 (0.02, 1.38) 1.04 (0.68, 1.61) 1.64 (0.45, 5.93) 1.02 (0.39,2.65) *Indicates where the ERG collected data varies from the MS; PBD FEV1 = post bronchodilator FEV1 Serious Adverse events OR (95%CI) NR 0.98 (0.63, 1.52) 1.04 (0.67, 1.62) 1.07 (0.69, 1.65) 1.03 (0.67, 1.58) NR 0.82 (0.59, 1.15) 1.38 (0.94, 2.02) 1.05 (0.82, 1.33) NR 0.62 (0.17, 2.25) 1.06 (0.42, 2.68) 0.97 (0.38, 2.45) 1.02 (0.39, 2.65) SGRG Difference between treatments p-value (95%CI) NR -1.0 P=0.06 (-2.0, 0) NR -2.2 P=<0.001 (-3.1, -1.2) -1.2 P=0.02 (-2.1, -0.2) NR NR NR P=0.005 NR NR -3.1 P=<0.001 (-4.1, 2.1) NR P=0.17 NR NR NR NR In contrast to the severe COPD (FEV 1 50%) group, for the FEV 1 65% group LABA is superior in terms of annual rate of exacerbations compared to placebo in one out of two studies. LAMA significantly reduces the annual rate of exacerbations when compared to placebo in one out of four studies. ICS/LABA also significantly reduces the annual rate of exacerbations when compared to placebo in one out of one study. Similarly, ICS/LABA significantly reduces the annual rate of exacerbations when compared to LABA and when compared to ICS in one out of one study each. Lungfunction was not reported in most studies. When it was reported, significant results were reported favouring LABA over placebo and favouring ICS/LABA over LABA, ICS and placebo. Two 43

44 comparisons showed significant differences between treatments in terms of mortality, both in the same study: ICS/LABA resulted in significantly fewer cases of mortality than ICS; while ICS/LABA was also significantly better than placebo. There were no significant differences between treatments for serious adverse events in this population. Significant differences in terms of SGRG were found for four comparisons: ICS/LABA was significantly superior in terms of SGRG when compared to LABA, ICS, or placebo; while LAMA was significantly superior in terms of SGRG when compared to placebo Describe and critique the manufacturer s approach to validity assessment for each relevant trial. The quality assessment of RCTs included in the MTC is reported in Appendix 5 in the MS (pages ). The ERG generally agrees with the assessment in the MS. Differences are explained in the tables in appendix 3 (bold and highlighted). One trial included in the primary ERG analysis (M2-111) was not assessed in the MS Describe and critique the statistical approach used within each relevant trial. The statistical approaches used in each relevant trial are only summarised partially for 6 trials (M2-111, M2-112, M2-124 & M2-125 (combined) M2-127 and M2-128 in the Manufacturer s Submission in Table B8 (MS, pages 60-64). The tables in appendix 4 describe the statistical approaches for all 10 trials used by the ERG in the meta-analysis for severe COPD (Baseline FEV 1 <50%), and the additional meta-analysis of 16 trials to include also trials with inclusion criteria of Baseline FEV 1 < 60-65%. Data taken directly from MS is in blue font Describe and critique the manufacturer s approach to outcome selection within each relevant trial. The NICE scope mentions the following outcomes: lung function; incidence and severity of acute exacerbations, including hospitalisation; symptom control (e.g. shortness of breath); mortality; healthrelated quality of life. Most of these outcomes have been reported in the MS for the roflumilast studies (MS, pages 71-89). However, in patients with severe COPD the roflumilast trials only allow a comparison with placebo. Placebo was not mentioned in the scope as a relevant comparator. The only outcomes reported relevant to the scope were exacerbations and lung function for the subgroups of people receiving concomitant LABA or not (MS, pages 81-84). None of the outcomes mentioned in the scope have been reported for any of the other trials included in the MTC. In the response to the clarification letter, the ERG received data for exacerbations (total number of exacerbations and number of person-years) for all trials included in the MTC. COMMENT The ERG has reported results for most relevant outcomes for studies included in the primary (FEV 1 50% predicted) and secondary (FEV 1 65% predicted) ERG analyses (see section 4.2.1). 44

45 4.2.5 To what extent does each relevant trial include the patient population(s), intervention(s), comparator(s) and outcomes as defined in the final scope? Population Regarding roflumilast studies, four of the 10 studies reported in the MS included patients with severe COPD (FEV 1 50% predicted). The other 6 studies included patients with moderate to severe COPD. None of these four trials directly compared roflumilast with the comparators listed in the decision problem. However, around 50% of patients included in trials M2-124 and M2-125 received concomitant LABA. For these patients the comparison is roflumilast + LABA versus LABA + placebo. All other trials compared roflumilast with placebo. None of the roflumilast trials included patients with a history of frequent exacerbations, using a definition of two or more exacerbations in the previous year. Twenty-six publications met the inclusion criteria for the MTC. Combined, these publications reported on 29 clinical trials. Most of these trials were in patients with moderate to severe COPD, inclusion criteria going up to 90% FEV 1 predicted at baseline. Only 8 out of 29 trials included patients with severe COPD. Intervention The MS presents results for seven treatments that include roflumilast (roflumilast alone, roflumilast+ LABA, roflumilast+lama, roflumilast+ics, roflumilast+laba+lama, roflumilast+laba+ics, and roflumilast+laba+lama+ics). However, only two treatments (roflumilast alone and roflumilast + LABA) have relevant data in the correct population. Comparators Regarding theophylline, the MS states that Theophylline is not thought to be an appropriate comparator for roflumilast. (see MS, section 4, page 36). The ERG did not have time or recourses to assess the relative (cost-) effectiveness of roflumilast in comparison with theophylline. The MS presented evidence for comparisons between roflumilast and most comparators. However, most evidence was based on trials in moderate to severe COPD or based on additive analyses. Evidence from trials in severe COPD was presented for only four relevant comparators: LABA, LAMA, ICS and LABA/ICS. In order to present data for a comparison between roflumilast and LABA + LAMA and LABA/ICS + LAMA, the ERG widened the inclusion to trials including patients with FEV 1 65% (see section of this report). Outcomes For the roflumilast trials most outcomes are reported in the MS (pages 71-89). Additional adverse events are presented for the roflumilast trials in chapter 5.9 (MS, pages ). For the subgroups of patients with and without LABA in M2-124 and M2-125, only data for COPD exacerbations and lung function (Mean Changes in Pre- and Post-bronchodilator FEV 1 ) are presented in the MS (pages 81-84). For non-roflumilast trials included in the MTC, no data were reported. In the response to the clarification letter, the ERG received data for exacerbations (total number of exacerbations and number of person-years) for all trials included in the MTC. Other relevant outcomes as defined in the NICE scope were not reported for non-roflumilast trials. 45

46 4.2.6 Where appropriate, describe and critique any meta-analysis, indirect comparisons and/ or mixed treatment analysis carried out by the manufacturer. This section should include a summary of the manufacturer s methods and results as described in the MS. The ERG should critique the methods used and interpret the results in light of the methods used by the manufacturer and generalisability to patients in England and Wales. The MS describes two types of random effects MTC, standard and additive in order to estimate the treatment effect on exacerbation rate i.e. rate ratio (RR) in comparison to placebo, using a Poisson evidence synthesis model. The same source of 26 studies (MS, Table B28, page 116) was used for both MTC models. In addition the MS also presented an MTC to estimate treatment effect on the relative risk of at least one exacerbation, using a binomial evidence synthesis model. For this, an additional 3 studies were added (MS, Table B29, page 117). All 3 analyses were performed and published by Mills et al. 8 The inclusion criteria for the search to obtain the studies included both moderate and severe COPD and any exacerbation (mild, moderate or severe). All three models were estimated using Winbugs and scripts were provided for the random effects standard Poisson model (referred to in the ERG report as the Standard MTC model) in Appendix 19 and in Appendix 20 (separately provided by manufacturer with MS) for the additive Poisson model (referred to in the ERG report as the Additive model). The results in terms of RRs from the Additive model were used by the manufacturer in the CEA. COMMENT The ERG considers that the Standard MTC model was methodologically correct. Also, the Poisson synthesis model is generally better than the binomial since it allows several exacerbations per person per unit time. For example, it is possible that, a lower proportion of patients on drug A have exacerbations than on drug B each year, which using a binomial model, would imply drug A is more effective with a relative risk of less than 1. However, it is also possible that each individual on drug A who exacerbates does so more frequently per year on drug A than on drug B, which might make A less effective with a rate ratio of greater than 1. The Poisson model does assume that the rate does not change with history e.g. increase with number of exacerbations already had (which is unlikely), but the binomial also requires this assumption when synthesising summary data (number of patients reported at a particular follow-up time). Indeed, a binomial evidence synthesis model also has to rely on an assumption of equal follow-up time within trial. In contrast, the MS calculated number of exacerbations (required for the MTC) from the published rate and number of person years using: rate=number of exacerbations/number of person years exposure. Crucially, this rate was estimated in each study from the individual patient data using a model that adjusted for withdrawal (unequal follow-up time). In most published studies the actual exposure, given early withdrawal of some subjects, was not reported and so the number of person years was estimated as the number of years follow-up, assuming no withdrawal. However, assuming that the rate was correctly estimated in the study (adjusting for withdrawal) then the number of exacerbations estimated this way should be unbiased. However, the inclusion criteria used in the MS were too broad. In particular, the MS included moderate COPD (FEV1%pred>50%), which is outside the scope. 46

47 Table 4.12 (MS, Table B28, page 116): Summary of the trials used in the assessment of total number of exacerbations and/or the mean annual rate of exacerbations No. trials References of trials PBO FLUT BUD SAL FOR TIO ROF FLUT + SAL BUD + FOR TIO + SAL SAL + ROF TIO + ROF FLUT + SAL + TIO 2 Paggiaro et al, 1998;Burge et al, Casaburi et al, 2002; Niewoehner et al, 2005; Dusser et al, 2006; Chan et al, 2007; Powrie et al, 2007; Tashkin et al, 2008; Tonnel et al, Rossi et al, Brusasco et al, 2003; 2 Calverley et al, 2003; Szafranski et al, Calverley et al, 2003; Calverley et al, Rabe et al, 2005; Calverley et al, 2007; Calverley et al, 2009 (M2-124 and M2-125) 1 Stockley et al, Aaron et al, Kardos et al, 2007; Ferguson et al, 2008; Anzueto et al, Wedzicha et al, Fabbri et al, 2009 (M2-127) (Included above) Fabbri et al, 2009 (M2-128) PBO=Placebo; FLUT=Fluticasone proprionate; BUD=Budesonide; SAL=Salmeterol; FOR=Formoterol; TIO=Tiotropium; ROF=Roflumilast 47

48 Table 4.13 (MS, Table B29, page 117): Summary of the additional trials used in the binomial analysis (number of patients with 1 exacerbation) No. trials References of trials PBO FLUT BUD SAL FOR TIO ROF FLUT + SAL BUD + FOR TIO + SAL FOR + TIO SAL + ROF 1 Bourbeau et al, Chapman et al, Vogelmeier et al, 2008 TIO + ROF FLUT + SAL + TIO 48

49 The ERG considers that the additive MTC analysis performed in the MS in order to estimate the effect of the combination treatments (Roflumilast+LABA+LAMA, Roflumilast+LABA/ICS+LAMA, Roflumilast+ICS, Roflumilast+LABA/ICS and LAMA+ICS) for which there was no trial data, was invalid. The reasons for this are: 1) there is no actual evidence presented, whether from a randomised control trial or from observational data as to the effect of these combinations; 2) underlying assumptions are not met; 3) there is evidence, at least for the combination of ICS and LABA, that the effect is more than additive; and 4) there is evidence from the standard MTC analysis (i.e. without the additive assumption) in the MS of the breaking of the additive assumption. This is more extensively described at the end of chapter 3 in this report Additional clinical work conducted by the ERG Provide details of any additional work conducted by the ERG in relation to clinical effectiveness. If the results of any of the additional work affect the size of the ICER, refer the reader to the summary table in Section 6. Since the MS does not present any tables with baseline characteristics or data extraction of studies included in the MTC apart from the roflumilast trials, the ERG produced a table with baseline characteristics (see appendix 2) and results (see section of this report) for the trials included in the ERG base-case analyses (FEV 1 50%) and in the ERG secondary analyses (FEV 1 65%). Furthermore, the ERG produced 2 new MTC analyses, for FEV 1 thresholds at 50% (10 studies) and 65% (16 studies). In order to do this the MS Winbugs code was adapted according to the number of studies and treatments appropriate for each population (see appendix 5). Because the number of treatments was reduced from the 10 (excluding the additive MTC) in the MS, some of the treatments were re-coded in the Winbugs code: 1=placebo, 2=roflumilast, 3=LABA, 4=LAMA, 5=ICS are the same. 6=LABA/ICS (LABA/ICS was 9 and 6 was Roflumilast+LABA) 7=Roflumilast+LABA (Rolfumilast+LABA was 6 and 7 was Roflumilast+LAMA) 8=LABA+LAMA is the same. 9=LABA/ICS+LAMA (LABA/ICS+LAMA was 10 and 9 was LABA/ICS) The ERG corrected the Winbugs code to run the MTC of exacerbations. In all cases, the definition of exacerbation used was moderate or severe i.e. excluding mild. In all studies included in the new MTC analyses the definition of severe exacerbations was consistent as requiring hospitalisation. In all 10 of the FEV 1 50% (see below) studies the definition of moderate exacerbations was requiring oral or parenteral steroids or antibiotics. In 2 of the 16 FEV 1 65% (see below) studies, Brusesco et al 2003 and Casaburi et al 2002, the definition was based only on symptoms. Given that the majority of published studies did not report the number of exacerbations, this number was calculated, as in the MS, from the published rate and number of person years using: rate=number of exacerbations/ number of person years exposure. However, in most studies the actual exposure, given early withdrawal of some subjects, was not reported and so the number of person years was estimated as the number of years follow-up, assuming no withdrawal. Assuming that the rate was correctly estimated in the study (and all studies used a model to account for withdrawal), the number of exacerbations estimated this way should still be unbiased. 49

50 Table 4.14: Corrected data for the new ERG MTC analyses, FEV 1 50% only (first 10 studies) and FEV 1 65% (all 16 studies) placebo Rof LABA LAMA ICS Rof +LABA Rof +LAMA LABA +LAMA LABA/ ICS LABA/ ICS +LAMA r py r py r py r py r py r py r py r py r py r py FEV 1 50% Anzueto et al, 2009 Calverley et al, 2003 (Ref 49) Calverley et al, 2007 (Ref 34) M2-124 and M2-125a M2-124 and M2-125b NA 1 NA NA 1 NA 1 NA 1 NA 1 NA NA NA NA NA 1 NA 1 NA NA NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA NA 1 NA NA 1 NA 1 NA 1 NA 1 Ferguson 2008 NA 1 NA NA 1 NA 1 NA 1 NA 1 NA NA 1 Kardos et al, 2007 NA 1 NA NA 1 NA 1 NA 1 NA 1 NA NA 1 M NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 Szafranski et al, NA NA NA 1 NA 1 NA NA 1 Wedzicha 2008 NA 1 NA 1 NA NA 1 NA 1 NA 1 NA NA 1 FEV 1 65% Aaron et al, 2007 Brusasco et al, 2003; Calverley et al, 2007 (Ref 56) Casaburi et al, 2002 NA 1 NA 1 NA NA 1 NA 1 NA NA NA NA 1 NA 1 NA 1 NA 1 NA 1 NA NA NA NA 1 NA 1 NA NA NA 1 NA NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 Chan et al, NA 1 NA NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 Niewoehner al, 2005 et NA 1 NA NA 1 NA 1 NA 1 NA 1 NA 1 NA 1 50

51 The corrected data for the 16 studies used in the new analyses is shown in Table 4.14, making 16 studies together with data from study M The table presents one row per trial and two columns per comparator, one (r) for number of exacerbations and one (py) for number of person years. NA (for r) or 1 (for py) indicates the absence of a comparison. Winbugs was then run using the same settings as the MS i.e. discarding the first 20,000 and using the next 100,000 simulations. No difference was found by using only one chain and random initial values or the two chains and initial values (adapted for the change in treatment and study number) as used in the MS. The MS did not subdivide M2-124/125 by whether LABA was used concomitantly or not and this information was not published in Calverley et al Therefore, the ERG extracted data on number of exacerbations (moderate or severe) and exposure to treatment (allowing for withdrawal) from the trial report, as shown in Tables 4.15 and The MS also did not include the M2-111 study and so the ERG did the same for these data, which are shown in Table By doing this the ERG were then able to re-do the MTC with all relevant data. Table 4.15: Exacerbations for Roflumilast+LABA vs LABA (M2-124/125) Roflumilast+LABA LABA Roflumilast+LABA LABA n Exacerbations per person Number of persons Number of persons Total exacerbations TOTAL exacerbations for trial Days exposure per person TOTAL years exposure for trial Rate Total exacerbations 51

52 Table 4.16: Exacerbations for Roflumilast vs Placebo (M2-124/125) Roflumilast Placebo Roflumilast Placebo n Exacerbations per person Number of persons Number of persons Total exacerbations TOTAL exacerbations for trial Days per person exposure TOTAL years exposure for trial Rate Total exacerbations Table 4.17: Exacerbations for Roflumilast vs Placebo (M2-111) Roflumilast Placebo Roflumilast Placebo n Exacerbations per person Number of persons Number of persons Total exacerbations TOTAL exacerbations for trial Days per person exposure TOTAL years exposure for trial Rate Total exacerbations The tables show the number of persons who experienced each of a number of exacerbations (from 0 to 9), extracted from the trial reports for M2-124/125 and M They also show the total number of exacerbations (=exac per person x number of persons), which is then summed across all numbers of exacerbations to give the total for the trial. Total years of exposure is calculated as days exposure per person (extracted from the trial report) x n. Total exacerbations and years exposure were inputs in the MTC. For completeness, rate has also been calculated as total exacerbations for trial divided by total years exposure for trial. 52

53 FEV1<50% only (including splitting M2-124/125 into Roflumilast+LABA vs LABA and Roflumilast vs placebo (10 studies)) The ERG ran analysis for a reduced set of studies (with the corrections and M2-111). Because the ERG believes that the populations for many of the studies were not within scope (FEV 1 50%) this reduced the set of eligible studies to 9. However, M2-124/125, which was intended to compare roflumilast with placebo, was contaminated by about 50% taking concomitant LABA. Data from the trial report was found for each of two comparisons, Roflumilast+LABA vs LABA and Roflumilast vs placebo, which in effect increased the set to 10 studies, as shown in Table As can be seen this reduces the number of possible treatments from the original 10 (excluding the additive model) to only 7, excluding Roflumilast+LAMA, LABA+LAMA and LABA/ICS+LAMA. The results of this MTC formed the inputs into the base case CEA model. FEV1<65% (16 studies) The ERG also considered relaxing the cut-off to include any study with FEV 1 65%, the data (with the corrections and M111) which are shown in Table This allowed the addition of two further comparators, LABA+LAMA and LABA/ICS+LAMA and brought the total number of studies to 16. Results of new MTC analyses Primary analysis for patients with severe COPD (FEV 1 50%) Table 4.18: Estimated rate ratios and 95% Credibility Intervals for the relative effects of pairs of treatments, produced by the Mixed Treatment Comparison for severe COPD (FEV 1 50%) 1 Treatment Mean 95% Cred. Interval vs rate ratio % 97.50% Roflumilast Placebo LABA Placebo LAMA Placebo ICS Placebo LABA/ICS Placebo Roflumilast +LABA Placebo LABA Roflumilast LAMA Roflumilast ICS Roflumilast LABA/ICS Roflumilast Roflumilast +LABA Roflumilast LAMA LABA ICS LABA LABA/ICS LABA Roflumilast +LABA LABA ICS LAMA LABA/ICS LAMA Roflumilast +LABA LAMA LABA/ICS ICS Roflumilast +LABA ICS Roflumilast +LABA LABA/ICS

54 1. Note that ALL 10 studies have the same definition of exacerbations i.e. moderate and severe where moderate is defined as needing either antibiotics or steroids and severe as needing hospital admission. 2. RR < 1 is in favour of the treatment in the first column. Table 4.18 shows that roflumilast alone in terms of the mean annual rate of exacerbations is highly likely to be superior to placebo (RR=0.89 (95% CRI: 0.82, 0.97). The results from the MTC are in line with the head-to-head comparison of roflumilast versus placebo as shown in Figure 4.6. When compared with other active treatments, roflumilast alone does not show a high likelihood of superiority when compared to any of the other treatments. However, LABA/ICS is highly likely to be superior to roflumilast alone (RR=0.83 (95% CRI: 0.72, 0.94). Roflumilast+LABA is in terms of the mean annual rate of exacerbations highly likely to be superior to placebo (RR=0.81 (95% CRI: 0.69, 0.95). When compared with other active treatments, roflumilast+laba is highly likely to be superior to LABA alone (RR=0.78 (95% CRI: 0.69, 0.89). However, compared with LAMA, ICS and LABA/ICS there are no such differences. Secondary analysis with wider inclusion criteria (FEV 1 65%) Table 4.19: Estimated rate ratios and 95% Credibility Intervals for the relative effects of pairs of treatments, produced by the Mixed Treatment Comparison for severe COPD (FEV 1 65%) Treatment Mean 95% Cred. Interval vs rate ratio % 97.50% Roflumilast Placebo LABA Placebo LAMA Placebo ICS Placebo LABA/ICS Placebo Roflumilast +LABA Placebo LABA+LAMA Placebo LABA/ICS+LAMA Placebo LABA Roflumilast LAMA Roflumilast ICS Roflumilast LABA/ICS Roflumilast Roflumilast +LABA Roflumilast LABA+LAMA Roflumilast LABA/ICS+LAMA Roflumilast LAMA LABA ICS LABA LABA/ICS LABA Roflumilast +LABA LABA LABA+LAMA LABA LABA/ICS+LAMA LABA ICS LAMA

55 LABA/ICS LAMA Roflumilast +LABA LAMA LABA+LAMA LAMA LABA/ICS+LAMA LAMA LABA/ICS ICS Roflumilast +LABA ICS LABA+LAMA ICS LABA/ICS+LAMA ICS Roflumilast +LABA LABA/ICS LABA+LAMA LABA/ICS LABA/ICS+LAMA LABA/ICS LABA+LAMA Roflumilast +LABA LABA/ICS+LAMA Roflumilast +LABA LABA/ICS+LAMA LABA+LAMA RR < 1 is in favour of the treatment in the first column. The results in Table 4.19 are similar to the results in severe COPD. Although, roflumilast alone is in terms of the mean annual rate of exacerbations only marginally highly likely to be superior to placebo (RR=0.89 (95% CRI: 0.79, 1.00). LABA/ICS is highly likely to be superior to roflumilast alone (RR=0.82 (95% CRI: 0.71, 0.95) and there are no such differences between roflumilast alone and any of the other treatments. Roflumilast+LABA is in terms of the mean annual rate of exacerbations highly likely to be superior to placebo (RR=0.76 (95% CRI: 0.61, 0.94). When compared with other active treatments, roflumilast+laba is highly likely to be superior to LABA alone (RR=0.78 (95% CRI: 0.64, 0.95). However, compared with LAMA, ICS and LABA/ICS, as well as LABA+LAMA and LABA/ICS+ LAMA there are no such differences. Looking at all results in comparison with placebo, LABA/ICS+LAMA seems to be the best treatment in terms of reducing the annual rate of exacerbations, LABA/ICS seems to be second best, and roflumilast+laba third best (see Figure 4.7). An additional observation is that, although the 95%CrIs overlap, the mean RRs for Roflumilast+LABA are lower in the secondary analysis. This can be explained largely by the pooling of additional studies, all of which have lower estimates of the RRs for LABA and the fact that the only comparison in the network for roflumilast+laba, other than roflumilast, is with LABA. This can be seen by applying the improvement in LABA in going from the primary to the secondary analysis (0.97/1.04=0.93) to the RR for roflumilast+laba in the primary analysis to give 0.81*0.93=0.76 i.e. approximately the 0.76 from the secondary analysis. 55

56 Figure 4.7: Rate ratio of exacerbations for Active Treatments versus Placebo Rate Ratio Rate Ratio Study or Subgroup log[rate Ratio] SE IV, Random, 95% CI IV, Random, 95% CI LABA/ICS+LAMA [0.52, 0.93] LABA/ICS [0.67, 0.80] Roflumilast+LABA [0.61, 0.94] LAMA [0.73, 0.89] ICS [0.76, 0.95] LABA+LAMA [0.69, 1.15] Roflumilast [0.79, 1.00] LABA [0.88, 1.06] Favours active treatment Favours placebo 4.3 Conclusions Describe the completeness of the MS with regard to relevant clinical studies and relevant data within those studies. Does the submission contain an unbiased estimate of the technology s (relative and absolute) treatment effects in relation to relevant populations, interventions, comparators and outcomes? Are there any remaining uncertainties about the reliability of the clinical effectiveness evidence? Reference should also be made concerning the extent to which the submitted evidence reflects the decision problem defined in the final scope. The NICE scope defines the population as Adults with severe chronic obstructive pulmonary disease (FEV 1 post bronchodilator less than 50% predicted) associated with chronic bronchitis in adult patients with a history of frequent exacerbations. Nevertheless, the MS presents roflumilast studies and a full MTC in patients with moderate to severe COPD. This ERG report focuses on the population as defined in the scope. A secondary analysis is presented using slightly wider inclusion criteria (FEV 1 65%). The MS presents additive analyses to provide data for other treatments including roflumilast. However, as the assumptions underlying these analyses were not met, the ERG ignored these analyses in this report. The MS presents evidence for comparisons between roflumilast and most comparators mentioned in the scope. However, most evidence was based on trials in moderate to severe COPD or based on additive analyses. Evidence from trials in severe COPD was presented for only four relevant comparators: LABA, LAMA, ICS and LABA/ICS. The secondary ERG analysis allows a comparison between roflumilast and LABA + LAMA and LABA/ICS + LAMA. The ERG presents results for two sets of data: 1. Data for adults with severe COPD (FEV 1 post bronchodilator less than 50% predicted) from 9 trials, including 7 treatments. 2. Data for adults with moderate to severe COPD (FEV 1 post bronchodilator less than 65% predicted) from 16 trials, including 9 treatments. The MS presents results for one outcome only: exacerbations. The ERG presents results for other outcomes as well. 56

57 One of the inclusion criteria for the MTC in the MS was that trials had to report data on the total number of exacerbations and/or the mean annual rate of exacerbations. Therefore, trials reporting other relevant outcomes (as mentioned in the scope) were excluded. It is unclear from the MS which trials were excluded because they did not report exacerbations. Therefore, the ERG analyses and data extraction are based on the same set of trials, but limited to severe COPD only. Twenty-six publications met the inclusion criteria for the MTC in the MS. Combined, these publications reported on 29 clinical trials. However, only 8 out of 29 included trials are in patients with severe COPD (FEV 1 50%). The ERG included one more roflumilast trial that fulfilled the inclusion criteria (M2-111). In addition, Calverley et al (M2-124 and M2-125) is described as roflumilast versus placebo, whilst about 50% of patients in both trials received concomitant LABA. Therefore, this trial should be treated as partly roflumilast vs. placebo; and partly roflumilast + LABA vs LABA, resulting in a network of 7 treatments and ten trials for the MTC. The ERG focussed on these ten trials for their primary analyses. A secondary ERG analysis was performed in a wider population (FEV 1 65%), including 16 trials and 9 treatments. Results of the MTC in patients with severe COPD (FEV 1 50%) show that in terms of the mean annual rate of exacerbations roflumilast alone is not statistically superior when compared to any of the other active treatments. However, LABA/ICS is significantly superior to roflumilast alone (RR=0.84 (95% CRI: 0.72, 0.98). When compared with other active treatments, roflumilast+laba is significantly superior to LABA alone (RR=0.78 (95% CRI: 0.67, 0.91). However, compared with LAMA, ICS and LABA/ICS there are no significant differences. Results are similar in the wider population (FEV 1 65%). Looking at the results in comparison with placebo, LABA/ICS+LAMA seems to be the best treatment in terms of reducing the annual rate of exacerbations, LABA/ICS seems to be second best, and roflumilast+laba third best (see Figure 4.7). 57

58 5 COST EFFECTIVENESS 5.1 ERG comment on manufacturer s review of cost-effectiveness evidence State objective of cost effectiveness review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate? The objective of the search was to identify published cost-effectiveness studies on pharmacological management of COPD. The search strategies for the cost-effectiveness review are discussed in detail in MS-chapter A critique of the search strategy is described in chapter of this report State the inclusion/exclusion criteria used in the study selection and comment on whether they were appropriate. Inclusion criteria: Studies must be an economic evaluation The economic evaluation should relate directly to the management of COPD The economic evaluation should relate to pharmacological therapy with roflumilast or comparators included in the decision being addressed (LABA, LAMA, ICS) Exclusion criteria: Economic evaluations related to non-pharmacological management of COPD e.g. studies of the cost-effectiveness of pulmonary rehabilitation Patients with asthma or COPD mixed with asthma COMMENT The ERG considered the in- and exclusion criteria appropriate for the purpose of the costeffectiveness review What studies were included in the cost effectiveness review and what were excluded? Where appropriate, provide a table of identified studies. Please identify the most important cost effectiveness studies. A total of 37 studies were identified, of which 8 were conference proceedings (abstracts) and 4 were reviews, leaving 25 full papers on original cost-effectiveness studies of COPD-management. From the MS it is not clear what studies were excluded. From the 25 studies identified, most were on treatments such as Salmeterol, Fluticasone Propionate, Tiotropium, and Ipatropium. Only one study investigated the cost-effectiveness of Roflumilast (vs placebo). 58

59 5.1.4 What does the review conclude from the data available? Does the ERG agree with the conclusions of the cost effectiveness review? If not, provide details. Although in appendix 11 of the MS, the manufacturer presents a quality assessment as well as a critical appraisal of all studies identified, there is no general conclusion derived from the data available (presumably because this is not requested in the MS format). 5.2 Summary and critique of manufacturer s submitted economic evaluation by the ERG Summarise and critique the cost effectiveness evidence submitted by the manufacturer (headings to are suggested headings). It is noted that the ERGs may prefer NOT to combine the summary and critique of the submitted economic evidence and instead report summary and critique sections separately. An overall summary of the de novo economic model developed by the manufacturer is given in Table 5.1. Table 5.1 Model States and events Summary of the manufacturer s economic evaluation (with signposts to MS) Approach Source / Justification Signpost (location in MS) Markov model with cohort starting in severe Pg COPD (FEV 1 40%). Cycle duration 1 month. Time horizon was 30 yrs in the ERG-basecase analysis. Health states: -Severe COPD 1 st line regime -Severe COPD 2 nd line regime -Very severe COPD 1 st line regimen -Very severe COPD 2 nd line regimen -Death Transition from Severe to very severe based on constant time dependent disease progression. Transition from 1 st line to 2 nd line regimen is determined from estimated average time on 1 st line regimen, 12 months in base-case. Death can occur either from hospitalization due to exacerbation, or by (state-dependent) standardized mortality rate. Comparators For ICS-tolerant population: LABA LAMA LABA+Rof LAMA+Rof LAMA+LABA LAB/ICS LAMA+LABA+Rof LABA/ICS+Rof 59 Average time in severe COPD is calculated from the rate of decline of lung volume in patients with COPD 49 and in general population 50. Time to transition from 1 st line to 2 nd line regimen is an assumption, as are the standardized mortality rates. Case fatality rate for hospital-treated exacerbations is taken from the National COPD Audit 51 Scope as specified by NICE Pg Pg. 168 Pg. 179 Pg. 176

60 LAMA+LABA/ICS LAMA+LABA/ICS+Rof 2 nd line treatment LAMA+LABA/ICS For ICS-intolerant population: LABA LAMA LABA+Rof LAMA+Rof LAMA+LABA LAMA+LABA+Rof 2 nd line treatment LAMA+LABA In almost all sensitivity analyses, the comparators are limited to: For ICS-tolerant population: LAMA+LABA/ICS+Roflumilast LAMA+LABA/ICS For ICS-intolerant population: LAMA+LABA+Roflumilast LAMA+LABA Comparators of special interest; likely to be most important for use of roflumilast in clinical practice and to provide clarity when presenting sensitivity results Pg. 243 Natural History Treatment effectiveness Adverse events Health related QoL Resource utilisation Disease progression is based on decline in lung function and exclusively a function of time. Mortality is incorporated into the model in two ways: standardized mortality rate (1.5 for severe COPD, 2 for very severe COPD) mortality due to hospital-treated exacerbation (7.7%) Effectiveness of treatments is expressed in terms of exacerbation rates. Exacerbations (when severe) are a possible cause of death, but do not influence disease progression. Adverse events are not taken into account in the model, neither in effects (or QoL), nor on the cost side. QALYs are calculated based on utilities (EQ-5D) for severe and very severe COPD health states. When an exacerbation occurs a decrement is applied for one Markov cycle (1 month). Cost categories are: treatment regimen, COPD maintenance and exacerbations See states and events Pg Pg. 168 Pg. 179 Rate ratios of exacerbations taken from Mills et al. 8 AEs are mild or moderate and transient in nature and therefore do not have an impact. Health state utilities are taken from the LABAsubgroup of the pooled analysis of the M2-124 and M2-125 trials. Utility decrements were from the literature 52 Resource use from literature 53 and Pg , tables B47 and B48 Pg. 189 Pg

61 and costs Discount rates Sub groups Sensitivity analysis assumption that exacerbation implies physician contact. Unit costs from BNF61 (2010), 2009/10 Reference costs or PSSRU (2010). 3.5 % for utilities and costs According to NICE reference case The only subgroup analysis that was done was for the population to start in very severe COPD instead of all of the population starting in severe COPD. The more or less intended subgroup-analyses for smoking status and BMI were not performed. One-way sensitivity analyses are provided for all major model variables in order to identify model drivers. Numerous scenario analyses were run investigating the effect of changing the ERG-basecase analysis assumptions Probabilistic sensitivity analysis was also undertaken. Most sensitivity analyses were limited to the comparators of special interest. Justification (in clarification phase) for not performing analyses for smoking status and BMI was that studies were not powered to do these analyses. Pg. 173, table B44 Pg. 230 (scenario F) Pg The ERG has assessed the manufacturer s economic evaluation using the Philips et al. checklist for quality assessing decision analytic models. 54 This is shown in Appendix 6 and is used to assist the narrative critique in the following MS-sections NICE reference case checklist Elements of the economic evaluation Reference Case 61 Included in submission Comment on whether denovo evaluation meets requirements of NICE reference case No full incremental costeffectiveness analysis provided on all options Comparator(s) Therapies routinely used in the NHS, including technologies regarded as current best practice Partly Type of economic Cost-effectiveness analysis Yes evaluation Perspective on costs NHS and PSS Yes Perspective on All health effects on No Adverse events are not outcomes individuals modelled Time horizon Sufficient to capture Yes Time horizon 30 years differences in costs and outcomes Synthesis of evidence Systematic review Partly A systematic review was

62 on outcomes Measure of health effects Source of data for measurement of HRQL Source of preference data for valuation of changes in HRQL QALYs Reported directly by patients and/or carers Preferences based on general public algorithm Yes Partly Unclear Discount rate Annual rate of 3.5% on costs Yes and health effects Equity weighting No special weighting Yes Sensitivity analysis One way sensitivity analysis and probabilistic sensitivity analysis Yes performed using an additive model. Many important model parameters were based only on M2-124 and M2-125 trials. Absolute values reported directly by patients and deterioration in utility based on literature data Not mentioned in the MS which algorithm was used to calculate utility scores from the EQ-5D Model structure The de novo economic model is a Markov state-transition cohort model with five health states: 1. Severe COPD, first line regimen (S 1 ) corresponds to patients with severe COPD, who are continuing to exacerbate, despite their current treatment regimen (and so are eligible to receive roflumilast). 2. Severe COPD, second line regimen (S 2 ) corresponds to patients with severe COPD who initially receive first line treatment (as described above), but continue to exacerbate, or remain breathless on the model first-line therapy. 3. Very severe COPD, first line regimen (VS 1 ) describes patients who were on a first line treatment regimen and progress to very severe COPD (FEV1 < 30%), while remaining on first line treatment. 4. Very severe COPD, second line regimen (VS 2 ) represents: a) patients who were on a first line treatment regimen in the very severe COPD state and who were then switched to a subsequent second line treatment regimen; b) patients who were on a second line regimen in the severe COPD state and then progressed to very severe COPD, and; c) patients initially in the S 1 health state, who were switched to a second line treatment regimen and progressed to very severe COPD. 5. Dead 62

63 Figure 5.1 Markov Model Structure (Fig 14, MS-page 164) Within each model cycle (one month), in both severe and very severe COPD, patients can experience exacerbations, which are modelled as events occurring within a given health state. These exacerbations may be community treated (moderate), or hospital treated (severe), with an associated risk of in-hospital mortality. Per cycle, a maximum exacerbation rate of one is assumed. Patients can die due to background mortality and COPD, calculated by means of a standardized mortality ratio. In addition, patients can die from a hospital treated exacerbation. The details of all possible transitions are shown in a detailed model scheme in figure 5.2 (Fig 15, MSpage 165): 63

64 Figure 5.2 Detailed Markov Model Structure (Fig 15, MS-page 165) The major structural assumptions of the model are: - The patient cohort starts in the severe COPD state - Patients have 2 exacerbations per year on the common first line comparator (LAMA+LABA/ICS) in the severe COPD state. Exacerbations for the other comparators are all calculated relative to this number by means of a relative rate ratio (RRR) as provided by the additive MTC. - Progression to second line therapy is rapid, with a mean time of 1 year on first line therapy - No lung function benefit due to first (or second) line therapy in the ERG-basecase analysis - Switch to second line therapy is exclusively a function of time on first line therapy, and thus independent of disease progression COMMENT Regarding health states, cycle length, and transition probabilities, the ERG considers the model structure in general adequate. However, some of the assumptions are questionable: 64

65 Patients switch to second line treatment only as a function of time on first line treatment. Variables naturally assumed to have a causal relation with switching treatments, such as adverse events or treatment failure/progression of disease (for instance based on lung function) are not incorporated into the model. The ERG questioned this assumption in the clarification phase, upon which the manufacturer replied that the evidence does not support a significant difference between therapies in the rate of treatment switching, and that sensitivity analyses showed varying the time to switch did not have a substantial effect on the results. Second line therapy is equal for all patients (LAMA+LABA/ICS for ICS-tolerant, LAMA+LABA for ICS-intolerant), and lasts for the rest of the time horizon, i.e. 29 years. So a difference between strategies can only be made in the first year, and whatever difference is established in the first year, will be extrapolated for 29 years, since after that all patients receive the same therapy. The ERG does not consider this a faithful representation of reality. Patients in the common comparator strategy have 2 exacerbations per year on average, exacerbations in all the other comparators are calculated based on this. There was no clear justification for using 2 as the baseline average. In the scope, and also in the MS, the population is described as having frequent exacerbations. In the response to the clarification issues, the manufacturer argued that the submission focussed on a group of patients who experience at least 2 exacerbations per year, which would imply that the average number of exacerbations must be higher than 2. The ERG has accounted for this in their additional analyses. The source (or rationale) for the Standardized Mortality Rates applied in the model (i.e. 1.5 for severe COPD and 2.0 for very severe COPD) is neither mentioned in the main text of the MS (MSpage 168) nor in table B49 (MS-page 186). The SMRs may have been derived from the trials, although that does not seem very likely given the relatively short follow-up Population The modelled population in the ERG-basecase analysis was assumed to have severe COPD, with an average FEV1 of 40% predicted, who continued to exacerbate, despite prior treatment. Average starting age of the cohort was 64 years. The proportion of males was 75%. The population in the scope is defined as: adults with severe chronic obstructive pulmonary disease (FEV1 post bronchodilator less than 50% predicted) associated with chronic bronchitis and a history of frequent exacerbations. COMMENT The modelled population seems in line with the scope, although the terms: continue to exacerbate (modelled population) and history of frequent exacerbations (scope) are not very specific and therefore difficult to compare. 65

66 5.2.4 Interventions and comparators For ICS-tolerant patients in the manufacturers submission 10 treatment options were compared and for ICS-intolerant/declining patients 6 treatment options were defined. These treatment options were relevant for first line treatment only (with a duration of 12 months) for the patients defined. Based on NICE guidance NICE CG101, the second line strategy is equal for all patients: LAMA+LABA/ICS for the ICS tolerant patients, and LAMA+LABA for ICS-intolerant/declining patients for the remaining of the simulations, thus 29 years. So any potential differences between the therapy options are solely a result of the first 12 months of the model runs. An overview of the treatment comparison as used in the submission is given in Tables 5.2 (Table B45, MS-page176) and Table 5.3 (Table B46, MS-page 176). Table 5.2 Therapy options for ICS-tolerant patients (Table B45, MS-page 176) ICS-tolerant patients Therapy 2nd line treatment if patients continue to option no. 1st line treatment exacerbate 1 LABA 2 LAMA 3 LABA+roflumilast 4 LAMA+roflumilast 5 LAMA+LABA LAMA+LABA/ICS 6 LABA/ICS 7 LAMA+LABA+roflumilast 8 LABA/ICS+roflumilast 9 LAMA+LABA/ICS* 10 LAMA+LABA/ICS+roflumilast * Selected common 1 st line comparator (the assigned background exacerbation rate was assumed to be 2 exacerbations per year) Table 5.3 Therapy options for ICS-intolerant/declining patients (Table B46, MS-page 176). ICS-intolerant/declining patients Therapy 2nd line treatment if patients continue to option no. 1st line treatment exacerbate 1 LABA 2 LAMA 3 LABA+roflumilast LAMA+LABA 4 LAMA+roflumilast 5 LAMA+LABA 6 LAMA+LABA+roflumilast The de novo model, in order to estimate exacerbation rate, used a common comparator, which was selected to be LAMA+LABA/ICS. Although not stated explicitly, for the ICS-intolerant population the common comparator was the same. All other strategies were analysed relatively against this common comparator. The model applied rate ratios (RR) for exacerbations (note that these are called relative rate ratios in the submission). Each strategy has an RR, expressing the number of 66

67 exacerbations for that strategy, relative to the number of exacerbations for the common comparator, i.e. LAMA+LABA/ICS. Since in the submission the deterministic analyses show that only LAMA+LABA/ICS+Roflumilast has a value over LAMA+LABA/ICS in the ICS-tolerant population and LAMA+LABA+Roflumilast has a value over LAMA+LABA, in the sensitivity analysis only these options are compared extensively. The other options are left out of the analyses. As stated in the submission: the rationale for this selection is firstly to focus on the comparisons that are likely to be most important for use of roflumilast in clinical practice in each patient group, and secondly, to provide the most clarity when presenting the results of the sensitivity analysis (MS-page 243). Comments: As already mentioned in chapter 4 (effectiveness review), because of the lack of evidence to support the assumption that treatment effect of a drug added to a combination is independent of the nature of the combination, the additive MTC model was found to be invalid. This therefore precludes the use of 5 comparators, which are combinations of roflumilast with: ICS, LABA/ICS, LABA+LAMA, LABA/ICS+LAMA and the combination of LAMA with ICS. Furthermore, studies that lie outside of the scope in terms of FEV1%pred consistent with severe COPD were also considered invalid to be included in the standard model MTC and therefore the MTC of the ERG focussed on trials that fulfil the inclusion criteria from the NICE scope. For this reason the ERG redid the economic evaluation, using the ERG MTC and based on the electronic model as provided by the manufacturer. The economic evaluation done by the ERG will solely focus on the treatment options for which the standard MTC provided evidence for the population that was defined in the scope. Based on the overview of treatment options as provided in the network of treatments included in the MTC analysis of exacerbations in FEV1%pred<50 (Chapter 4, Fig 4.4, p 34) for the ICS-tolerant population the first line treatment options are LABA/ICS, LAMA, LABA, ICS, Roflumilast, and LABA+Roflumilast. As in the systematic review the definition of the population definition from the scope is relaxed to FEV1%pred <65 in an ERG-alternative analysis. This will include LAMA+LABA and LAMA+LABA/ICS, thus increasing the first line treatment options to 8 options (Chapter 4, Fig 4.5, p 35). Based on the same review results the first line treatment options for the ICS-intolerant population are respectively 4 (LAMA, LABA, Roflumilast, and LABA+Roflumilast, ERG-basecase analysis) and 5 options (including LAMA+LABA, ERG-alternative analysis). For the second line treatment the ERG will follow the definition as provided in the submission since this is in line with clinical guidance. An overview of the treatment options as will be used in the cost-effectiveness analyses done by the ERG is provided in Table 5.4. From the comparison of the treatment options in the manufacturers submission and the treatment options in the ERG analyses it can be concluded that in the analyses by the ERG the option of ICS-only is included for the ICS-tolerant population and the treatment options that are left out are: LAMA+Roflumilast, LAMA+LABA+Roflumilast, LAMA+LABA/ICS+Roflumilast, and LABA/ICS+Roflumilast for the ICS-tolerant population; and the options LAMA+Roflumilast and LAMA+LABA+Roflumilast for the ICS-intolerant population. 67

68 Table 5.4 Overview of the definition of the two scenarios, regarding the treatment options included as used in the ERG-cost-effectiveness analyses ERG-basecase analysis FEV1%pred<50 ERG-alternative analysis FEV1%pred<65 Number of studies included in the evidence review: N=10 N=16 ICS-tolerant As in ERG-basecase population analysis and in addition: 1 st line options LABA/ICS LAMA LABA ICS Roflumilast LABA+Roflumilast LAMA+LABA LAMA+LABA/ICS ICS-intolerant population 2 nd line option LAMA+LABA/ICS LAMA+LABA/ICS 1 st line options LAMA As in ERG-basecase LABA analysis and in addition: Roflumilast LABA+Roflumilast LAMA+LABA 2 nd line option LAMA+LABA LAMA+LABA The ERG does not agree with the submission focussing on only two out of ten (for ICS-tolerant population), and two out of six treatment options (for the ICS intolerant population) beyond the deterministic cost-effectiveness analyses. Decision uncertainty concerns all treatment options defined and therefore the submission cannot be considered to be a fully incremental cost-effectiveness analysis. Furthermore, the ERG questions (as was raised in the clarification letter) the evidence base for determining the fixed 12 months duration of first line treatment and whether this duration is independent of treatment options. In the literature it can be found for instance that based on observational data, patients still use their first line drug after 1, 2, and 3 years and that treatment survival differs between LAMA, LABA, and LABA/ICS. 55 It should be acknowledged however, that the data presented in the article were not solely related to patients matching the scope, and therefore the ERG acknowledges that under current circumstances these drug survival data cannot be used. In the manufacturers submission using a sensitivity analysis the time to switching to the second line drug was varied simultaneously Perspective, time horizon and discounting The model used in the manufacturers submission uses a health state transition (Markov) model. The choice of model cycle length of one month is based on the assumption that no more than one exacerbation is likely to occur within one model cycle and that one month is sufficient to capture the disutility of an exacerbation. A cycle length of one month is consistent with previous economic analyses of COPD treatment. As a time horizon 30 years is used which was broken down into a 1 year time horizon for first line treatment and, consequently, the remaining 29 years for second line treatment. In Table 5.5 adherence of the submission to the NICE cost-effectiveness guidance is shown (Table B44, MS-page 173). 68

69 Table 5.5 Key features of cost-effectiveness analysis in the manufacturers submission (Table B44, MS-page 173) Factor Chosen values Time horizon 30 years (COPD patient lifetime) Cycle length 1 month Half-cycle correction Yes Were health effects measured in QALYs? Yes Discount of 3.5% per annum for utilities and costs Yes Perspective (NHS/PSS) NHS NHS, National Health Service; PSS, Personal Social Services; QALYs, quality-adjusted life years COMMENT The ERG concludes that the discount rates used and the perspective of the cost-effectiveness study are in line with the NICE reference case. Although other than what is shown in Table 5.5 ( Table B44, MS-page 173), the submission provides no explanation/justification on the chosen value for the time horizon, 30 years seems long enough to capture lifetime costs and effects in this patient population, given starting age of 64 years Treatment effectiveness The parameters in the model related to the effectiveness of treatment can be divided into those that are treatment independent en those that are treatment dependent. We will first discuss the treatment independent set of parameters and then the treatment dependent set. Treatment independent parameters Baseline patient characteristics The patient population being assessed in the ERG-basecase analysis of the model was assumed to have severe COPD, with an average FEV1 of 40% predicted, who continued to exacerbate, despite prior treatment. Average starting age of the cohort was 64 years. The proportion of males was 75%. The exacerbation rate in the population defined was determined to be 2 per year. Natural decline FEV1 % predicted The cohort that is defined in the MS submission is assumed to start in the severe COPD health state, with a lung function which lies in the middle of the severe COPD FEV1% range, i.e. 40% predicted. The decline in lung function determines the transition to the very severe COPD health states (being in first line or second line treatment). The average time (T) in severe COPD is calculated from the rate of decline of lung volume in patients with COPD (0.052 litres / year; Scanlon et al 2000) 49, and the rate of decline in a general (non-copd) population (Crapo et al 1981) 50, to estimate the time at which lung volume in patients with severe COPD reaches 30% of lung volume in a general adult population. In Figure 5.3 (Fig. 16, MS-page 171), the dotted line represents the threshold over time for a general adult population for transition to 30% predicted FEV1. The solid line represents the declining lung volume in the modelled cohort of patients who start with severe COPD (40% of predicted FEV1). The predicted intersection of these lines determines the average time in the severe COPD state. The transition of patients from the severe COPD state to the very severe COPD state was determined by continuous per-cycle probabilities, defined as 1/T, based upon the estimated average time (T) in the severe COPD states. 69

70 Figure 5.3 Time to disease progression in patients starting in the severe COPD state (illustrative example; no lung function benefit considered) (Fig. 16, MS-page 171) The effects of treatment regarding a lung function benefit are not included in the baseline analysis but this impact is reported via a scenario analysis (scenario B) and is described in chapter Exacerbations in COPD population For the economic model an important parameter is the number of exacerbations per year for a patient as defined in the scope. Besides this number of exacerbations per patient, the proportion of exacerbations requiring hospital treatment (severe exacerbation) is a crucial parameter. It is estimated that patients have a rate of 2 exacerbations per year on the reference treatment (LAMA+LABA/ICS) in the severe COPD state. Exacerbations for the other comparators are all calculated relative to this number by means of a rate ratio (RR)(in the submission named relative rate ratio, RRR) as provided by the additive MTC. For patients treated with the reference treatment LAMA+LABA/ICS, the MS assumes an annual exacerbation rate of 2 for patients with severe COPD. Furthermore, the MS uses an RR of 0.68 of LAMA+LABA/ICS vs. placebo. This implies an annual exacerbation rate of 2.94 for severe COPD patients with placebo treatment. Logically, multiplying 2.94 by the RR of LAMA+LABA/ICS vs. placebo of 0.68, this translates into a rate of exacerbation of 2 in severe COPD patients per year. In the model, rates for exacerbation differ by patient group, i.e. severe COPD and very severe COPD. Patients with very severe COPD have a higher rate of exacerbation than patients with severe COPD. To calculate this higher rate, the MS model uses a multiplier of 1.22, that is, a very severe COPD patient has a 1.22 higher rate of exacerbation than a patient with severe COPD. This higher rate is based on data from the LABA treated subgroup in the M2-124 and M2-125 trials (not mentioned in the submission, but defined as a remark in the electronic model). The exacerbation rates in this subgroup were respectively for severe COPD 1.45 and 1.76 for very severe COPD patients. Hence, 70

71 given the multiplier of 1.22, a patient with very severe COPD treated with LAMA+LABA/ICS has according to the submission an exacerbation rate of 2.44 per year. An exacerbation can be considered moderate, only requiring community based treatment, or severe, requiring hospitalizations. The treatments under assessment are considered to have an effect on the rate of exacerbation but not on their severity, i.e. whether the exacerbation requires hospital or community treatment. So the proportion of exacerbations requiring hospitalization is equal for all treatment options. The MS uses a proportion differentiated by disease state. For patients with severe COPD the submission model assumes a proportion of exacerbations require hospitalization. For patients with very severe COPD the model assumes the proportion of exacerbation requiring hospitalization to be These proportions are based on data from the patient subgroup that was treated with LABA in M2-124 and M Regarding the exacerbations rate and the proportion requiring hospital treatment there was no distinction made between the two populations (ICS-tolerant and ICS-intolerant population). The impact of changing the assumptions of the exacerbations rate was investigated in a scenario analysis (scenario A), however this scenario was not applied for all treatment options but solely on the comparison of LAMA+LABA/ICS+Roflumilast versus LAMA+LABA/ICS+Roflumilast for the ICStolerant population and the comparison LAMA+LABA+Roflumilast versus LAMA+LABA for the ICS-intolerant population. Mortality Mortality as used in the model is calculated by means of a standardized mortality ratio. The all cause mortality rate was estimated from UK life tables from the UK Actuaries Department, based upon the gender distribution from the trial and age specific. The background mortality rate was determined by the mortality rate in the general population, adjusted by the standardised mortality ratio (SMR) which excludes deaths due to hospital-treated exacerbations. The SMR values are assigned according to the degree of severity of COPD: 1.5 for the severe COPD state and 2 for the very severe COPD state. In addition, patients can die from a hospital treated exacerbation (hospital case fatality rate, 7.7%). Treatment dependent parameters Effect of treatment on pulmonary exacerbations In the economic model, treatment effectiveness is accounted for in terms of exacerbations. As mentioned before, the number of exacerbations for the common comparator, i.e. LAMA+LABA/ICS, is assumed to be 2 per year. For all other comparators, a rate ratio is used, expressing the number of exacerbations for that specific comparator, relative to the number in the common comparator. The rate ratios (RR) are taken from the study by Mills et al. 8 The reductions in the rate of exacerbations for different treatments versus LAMA+LABA/ICS were derived from the additive main effects model in the MTC i.e. assuming that the treatment effect (RR) of a drug added to a combination is independent of the nature of the combination. For the purpose of the model the rate ratios and credibility intervals where inverted and these are presented in Table 5.6 and applied to the respective regimens relevant for ICS-tolerant patients and ICS-intolerant/declining patients. In short: rate with treatment = rate with LAMA+LABA/ICS x RR of treatment vs LAMA+LABA/ICS. 71

72 Table 5.6 Rate ratio of exacerbations for treatments vs. the common comparator (LAMA+LABA/ICS), based on Mills et al., 2011 (Table B48, MS-page 178). # Treatment 1 LABA 2 LAMA 3 LABA+roflumilast 4 LAMA+roflumilast 5 LAMA+LABA 6 LABA/ICS 7 LAMA+LABA+roflumilast 8 LABA/ICS+roflumilast 9 LAMA+LABA/ICS Common comparator vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS vs. LAMA+LABA/ICS 10 LAMA+LABA/ICS+roflumilast vs. LAMA+LABA/ICS 1) Assumed to be the average across all S.E. Rate ratio (RR) of 95% CI S.E. exacerbations (1.4567, ) (1.2750, ) (1.1583, ) (1.0052, ) (1.1361, ) (1.2265, ) (0.8842, ) (0.9662, ) (0.70, 1.30) ) (0.7372, ) Patient response In the submission continuation or discontinuation of treatment in a patient is fixed: 12 months for first line treatment and 29 years for second line treatment. In this respect patient response is not part of the analyses. In a scenario analysis (scenario C), the time on first line treatment was varied between 0.5 and 2 years, simultaneously for the two selected comparators (not using a full incremental analysis). In another scenario the switch to second line therapy was based upon disease progression. The results of these scenario analyses are discussed in chapter COMMENT Regarding the baseline frequency of exacerbation that was stated to be 2 per year the ERG concludes that this was not based on published evidence. In a review by Hoogendoorn et al. (2010) it was found that the annual number of exacerbations was dependent on COPD severity, being 1.83 for severe COPD and 2.38 for very severe COPD. 56 Using the prevalence figures of Hurst et al (2010) regarding severe and very severe COPD, the weighted average number of exacerbations is 1.97 per year. 57 It should be noted that these data refer to the general COPD population and that exacerbation data for the population specified in the scope (patients with a history of frequent exacerbations) could not be 72

73 retrieved by the ERG. The ERG considers the number of 2 exacerbations per year to be a conservative estimate since the definition of the population is described as having frequent exacerbations. In the response to the clarification issues, the manufacturer argued that the submission focussed on a group of patients who experience at least 2 exacerbations per year, which would imply that the average number of exacerbations must be higher than 2. In the ERG calculations, based on the number of 2 exacerbations in case of LAMA+LABA/ICS treatment, using the rate ratio of this treatment option compared to placebo of the ERG-review, for untreated severe COPD the number of 2.94 exacerbations annually was determined. However, using the RR derived from the ERG s reanalysis of the treatment effects, this leads to an estimate of 2.86 for patients with severe COPD that are on placebo. The number of exacerbations was varied in a scenario analysis. The source (or rationale) for the standardized mortality rates applied in the model (i.e. 1.5 for severe COPD and 2.0 for very severe COPD) is neither mentioned in the main text (MS-page 168) nor in Table B49 of the submission (MS-page 186). The SMRs may have been derived from the trials, although that does not seem very likely given the relatively short follow-up. Regarding the effect of treatment in the ERG-basecase analysis of the submission it is assumed that the probability of COPD progression does not depend on the line of treatment. An impact on lung function (slowing down lung function deterioration or even lung function improvement) as a possible effect of treatment of COPD would be expected to increase the time spent in a less severe COPD state, with resultant cost savings and QOL benefit. In the model this would be reflected in a lower per cycle probability of progression to very severe COPD. Regarding the effect of treatment in the ERGbasecase analysis of the submission it is assumed that the probability of COPD progression does not depend on the line of treatment. The source of the multiplier (1.22) used to derive the mean rate of exacerbations for patients with very severe COPD is not mentioned in the submission, in fact, the ratio itself is not mentioned at all. The electronic economic model contains a remark indicating that is apparently derived from the pooled analysis of the LABA subgroup of the M2 124/125 trials. The transferability of this ratio to other patient populations has not been motivated. Other data that have been derived from the LABA subgroup from trials M2 124/125 are the proportion of severe exacerbations out of all exacerbations, specifically for severe (0.155) and very severe COPD (0.224). The generalizability of the subgroup to the patients as defined in the scope remains unclear. In the review by Hoogendoorn et al. (2010) these proportions were found to be 0.12 for the severe COPD population and for the very severe COPD population. 56 The ERG analyses were based on the data from Hoogendoorn et al. The ERG considers it not to be realistic to assume that the patients will continue treatment for the rest of their life irrespective of whether there is a benefit or not; it is unlikely that clinicians would prescribe treatment to those patients that get no benefit. Therefore, a continuation rule should have been part of the cost-effectiveness analysis. The ERG agrees that a distinction in exacerbations regarding ICS tolerance cannot be made based on the literature. 73

74 5.2.7 Health related quality of life Health related quality of life was incorporated into the model by assigning a utility score to each disease state, and by subtracting a utility decrement for exacerbations. The utility scores for the disease states (S1 and S2, V1 and V2) were, by means of the EQ-5D, taken from the LABA subgroups of the pooled analysis of studies M2-124 and M The utility in the stable severe disease states (S1 and S2) was assessed at For the very severe disease states (V1 and V2), the utility was sampled by using the distribution of the utility in the severe disease state and the sampled reduction in utility of the very severe state relative to the severe state, resulting in a utility score of This procedure is also used in the PSA, to prevent the very severe health state being associated with a higher utility than the severe health state. Table 5.7 Utility values used in the model, ERG-basecase analysis (last two rows of Table B49, MS-page ) Variable utility in stable severe COPD utility in stable very severe COPD Variable Name Value CI u_sev SE = % CI (0.7378, ) Description LABA subgroup in pooled analysis M2-124 and M u_vsev Not applicable This utility is sampled from the distribution for utility in severe COPD multiplied by the sampled reduction of utility in very severe COPD relative to severe COPD. To avoid the scenario of the very severe health state being associated with a higher utility than the severe health state when sampling from separate distributions, the ratio is also used in PSA. As exacerbations may have occurred between visits, and also because it is not clinically appropriate to administer a health instrument during an exacerbation, the disutility of an exacerbation was not elicited during trials. Instead, utility decrements for exacerbations (community-treated and hospitaltreated) were obtained from a Dutch preference study. 52 Utility valuations in this study were undertaken with adults from the general population using a time-trade-off method and descriptions of exacerbations specifying respiratory and other symptoms, treatment and impact. For non-serious (community-treated) exacerbations the absolute utility decrement was -0.01, and for serious (hospitaltreated) exacerbations it was Both decrements apply to a one year period, and were recalculated into monthly (per cycle) figures by multiplying them by 12, so and per cycle for the non-serious and serious exacerbations, respectively. In doing this, it is assumed the complete burden of the exacerbation is situated in the month where the exacerbation takes place. 74

75 Table 5.8 Utility variables (Table B52, MS-page 203) Utility Variable Value Source Utility for the severe COPD state Studies M2-124 and M2- Utility for the very severe COPD state pooled analysis of the LABA subgroup 58 Absolute utility reduction for a Rutten-van Mölken et al. 52 community-treated exacerbation Absolute utility reduction for a hospitaltreated exacerbation These reductions of utility due to exacerbations are estimated to last for one year, and in application to the economic model, the loss of utility was accounted for within model cycle (see Table B44, MS-page 173). COMMENT The utility scores for disease states were empirically derived from COPD-patients, whereas the values for the utility decrements were obtained from a preference study in the general population (239 Dutch adults). In this preference study, health state utility scores were calculated as well in the form of regression coefficients. Fig 5.4 From the preference study by Rutten-van Molken et al. 52 From this table, the ERG has derived utility values for the severe and very severe states of and 0.522, respectively (severe COPD: = 0.717, very severe COPD: = 0.522). These utility scores are lower than reported in the roflumilast trials, especially for the very severe disease state. Therefore, the ERG thinks that the utility scores may not match the decrements used, since the decrements were obtained in a different population, which clearly showed a tendency towards lower utility values compared to the COPD-patients. On page 202 of the MS, it is stated that the use of different data sources and methods of health state utilities and utility decrements for exacerbations will be explored in the sensitivity analysis. Utility scores reported by Oostenbrink et al of and for Severe and very severe COPD respectively are mentioned in this respect. 53 From the Excel-part of the model, it would seem that indeed these values are applied in a sensitivity analysis, as are the values as found by Rutten-van 75