RE: NICE STA of Roflumilast for the Management of Chronic Obstructive Pulmonary Disease Response to the Appraisal Consultation Document

Transcription

1 MSD Hertford Road Hoddesdon Hertfordshire EN11 9BU UK Telephone +44 (0) Facsimile +44 (0) National Institute for Health and Clinical Excellence MidCity Place 71 High Holborn London WC1V 6NA 30 th September 2011 Dear XXXXX, RE: NICE STA of Roflumilast for the Management of Chronic Obstructive Pulmonary Disease Response to the Appraisal Consultation Document Thank you for providing the opportunity to respond to the Appraisal Consultation Document (ACD) for this STA. We have reviewed the ACD and present a summary of our comments in the first part of this letter, with a more detailed description of the comments in the subsequent sections. 'Only in Research' Recommendation The ACD recommends roflumilast only in the context of research, and specifies that the recommended clinical trial for roflumilast should compare the benefits of roflumilast as add on to current triple therapy (LAMA plus LABA/ICS), or LAMA plus LABA for those intolerant to ICS, and should also compare roflumilast with theophylline added on to triple therapy or LAMA plus LABA in patients who would be suitable for theophylline therapy. The ACD states that the manufacturer could feasibly set up such a study and that setting up the proposed study would be relatively straightforward. We believe that the Committee concluded this based on comments from one or more of the clinical experts on the day of the meeting. Our calculations however demonstrate that such a trial would take at least 4 years to complete (even with only a 12 month duration) and would require a minimum of 2,500 patients to be randomised. Completion of a trial vs. theophylline would take significantly longer, and require randomisation of more than 5,000 patients. We are uncertain whether the Committee has fully taken into account the significant cost to patients (who remain in need of new and effective treatment options) and the NHS, caused by the draft recommendation denying them access to roflumilast for an extended period of time. The number of exacerbations, which could potentially have been prevented with the use of roflumilast, will have significant implications with regards to reduction in patient quality of life, and associated management costs for the NHS. The criteria for the 'only in research' recommendation outlined in the Methods Guide have not been met, as the research would be practically difficult to set up and the balance of the benefits and costs of conducting the research would not be favourable. 1

2 MSD is aware that clinicians experienced in the care of patients with COPD would propose that roflumilast should only be initiated by a specialist (in the patient sub-groups identified), which would modify the benefit/cost balance. Additivity of Treatment Effects & the Additive MTC The ACD highlights the concerns of the Committee around the assumption that the treatment effect of roflumilast is additive as well as the validity of the additive MTC. The additive MTC was conducted to allow the comparisons specified in the scope to be addressed in the cost effectiveness analysis for roflumilast. In vitro, in vivo and clinical studies have demonstrated that roflumilast effectively reduces the neutrophilic inflammatory response associated with COPD, and acts in a different manner to both steroids and bronchodilators. The classes of therapies in the additive MTC (LABA, LAMA, ICS and the PDE4 inhibitor roflumilast) act by different mechanisms, therefore it is appropriate physiologically to assume that the additive model applies in this case. The MS presented clinical evidence supporting the additive effect of roflumilast in reducing exacerbations when added to each of the other main classes of agents used in the treatment of COPD, however this does not appear to be reflected in the ACD. The roflumilast clinical trials studied the efficacy of roflumilast in combination with other COPD therapies and demonstrate that roflumilast reduces the rate of moderate or severe exacerbations by at least 20% when added to LABA, LAMA or ICS individually, compared to that agent alone. This evidence, in addition to current NICE guidelines (CG101) which endorse the additive effects of COPD agents and recommend combination therapy for patients with severe disease, support the assumption that roflumilast has an additive effect when used on top of dual or triple therapy. Internal validation of the additive MTC model was conducted which demonstrated that the model was a good fit to the data 1, and that there was overall consistency between the results of the additive MTC model, the standard MTC model and the direct metaanalysis, supporting the use of the additive MTC. Inclusion of Moderate Patients in the MTC The Committee and ERG also raised concerns over the inclusion of data from patients with moderate COPD in the MTC. The uncertainty of the Committee surrounding this issue appears to have arisen from a comment made by the ERG that there was evidence from clinical trials that roflumilast is more effective in people with moderate disease than in people with severe disease. This statement from the ERG is incorrect, and no trial has found roflumilast to be more effective in patients with moderate disease compared with severe disease. The ERG appears to have misinterpreted data on the reduction of moderate and severe exacerbations as being data on moderate and severe disease severity. To provide further reassurance to the Committee, data from individual trial participants in the four one year roflumilast clinical trials has been assessed for an interaction between disease severity (moderate, severe and very severe COPD) and the efficacy of roflumilast. The interaction test supports that the effect of roflumilast is independent of disease severity, as no significant difference was identified (p=0.6429). Theophylline 2

3 We accept that the Committee is interested in understanding the efficacy of theophylline in relation to roflumilast but disagree that theophylline should be considered as a comparator for roflumilast. There is very little clinical data from RCTs which investigate the efficacy of theophylline in reducing COPD exacerbations, making it challenging to incorporate this agent into the analyses. The poor tolerability, number of contraindications, and need for additional monitoring with theophylline cause it to be positioned very late in the treatment pathway. Current estimates suggest that theophylline is only prescribed in approximately 5% of patients with severe or very severe COPD. Continuation of Roflumilast Treatment for Lifetime in the Economic Model The Committee concluded that a scenario in which roflumilast treatment was continued for the lifetime of the person would be more appropriate than the base case in which people stopped treatment with roflumilast after one year and then remained on triple therapy alone. In response to this, an analysis has been performed in which patients continue on their assigned therapy for the duration of the time horizon, i.e. the model assesses LAMA + LABA/ICS + roflumilast vs. LAMA + LABA/ICS in ICS-tolerant patients and LAMA + LABA + roflumilast vs. LAMA + LABA in patients who are intolerant to ICS, for a 30 year time horizon. The results of this analysis demonstrate that, compared to the base case, continuing treatment with roflumilast for the patient's lifetime has very little impact on the cost effectiveness, changing the ICER by only for ICS-tolerant patients and by for ICS-intolerant patients. Role of the ERG In addition to the mistake regarding the comparative effectiveness of roflumilast in patients with moderate and severe disease, the MTC conducted by the ERG excluded a considerable part of the evidence base for roflumilast, which was relevant to the scope and was considered by the EMA during their decision making. Furthermore, in their alternative analysis, the ERG based their MTC on an arbitrary inclusion criterion of FEV 1 65% predicted, which has no clear clinical basis, and leads to exclusion of the majority of the evidence from all of the drug classes. It is unclear how this can have been conceived as something that would either help critique the MS or provide relevant additional information to help inform the Committee. In summary, we argue that the recommendation for roflumilast to be used only in the context of research is inappropriate in that the criteria for making such a recommendation have not been met. Roflumilast is an additional treatment option for a group of patients who still suffer considerable unmet need, for a disease which places significant financial burden on the NHS. Evidence for the efficacy of roflumilast added to individual COPD therapies has been presented and the additive MTC supports that these benefits would also be present when roflumilast is added to currently recommended dual or triple therapy. 3

4 MSD Response to the ACD for the STA of Roflumilast for the Management of Chronic Obstructive Pulmonary Disease 1) The clinical trial proposed by the Committee would take several years and is not straightforward The ACD describes the criteria outlined in the Methods Guide which should be considered when making an 'only in research' recommendation [ACD, 4.15]. These criteria include the following points: The research can realistically be set up, is already planned, or is already recruiting patients. The broad balance of the benefits and costs of conducting the research are favourable. On consideration of the clinical trial requested by the Committee, we do not believe that either of these criteria are met. A rationale for this is provided below. Criterion: The research can realistically be set up, is already planned, or is already recruiting patients The Committee suggested a clinical trial which would "assess the clinical effectiveness of roflumilast as an add-on to triple therapy or dual therapy and compared, as appropriate, with theophylline" [ACD, 4.18]. The ACD goes on to state that "The Committee also noted that the manufacturer is conducting a study (REACT) to assess the effect of roflumilast on exacerbation rates when added to LABA/ICS". Recruitment for the REACT study began in May 2011, and the design was a condition of the granting of the licence from the EMA. A similar clinical trial was agreed with the FDA, and is a condition of the granting of the licence to Forest Laboratories in the United States. As such, there is no trial which is currently planned, or already recruiting patients, which would meet the request. 4

5 With regards to setting up a trial, "the clinical specialists said it would be relatively straightforward to recruit participants to such a trial and that a 12-month trial should provide useful results, bearing in mind potential problems with participant dropout", and that "the Committee concluded that the manufacturer could feasibly set up a new trial" [ACD, 4.18]. MSD does not believe that these statements can be supported. Sample size calculations were performed to assess the number of patients required in a trial of LAMA + LABA/ICS + roflumilast vs. LAMA + LABA/ICS + placebo, with the rate of moderate or severe COPD exacerbations per patient per year as the primary endpoint. A Poisson regression model was used, with a twosided significance level of 5.0% (90% power). The calculation used an overdispersion factor of 2, and a mean exposure time of 287 days. The correction for overdispersion and the mean exposure time were estimated according to the results of the M2-124 and M2-125 clinical trials. 2 In one analysis the LAMA + LABA/ICS + placebo arm has a background exacerbation rate of 1.25 exacerbations per year, and roflumilast provides a 15% reduction in the exacerbation rate relative to placebo (resulting in a rate of 1.06 exacerbations per patient per year in the roflumilast arm). Such a trial would require 1,762 patients per treatment group, and 3,524 in total (Table 1). Assuming that the roflumilast arm was associated with a 20% reduction in the exacerbation rate (relative to the placebo arm, resulting in a rate of 1.00 exacerbation per patient per year), a sample size of 967 patients per treatment arm would be required, or 1,934 in total. Recruitment for such a trial would take approximately 2 years. Therefore, even with a 12 month treatment duration a trial assessing the effectiveness of roflumilast added to a triple therapy regimen vs. triple therapy + placebo would take at least 4 years from the design phase to completion. 5

6 Table 1: Required sample sizes for a clinical trial of LAMA + LABA/ICS + roflumilast vs. LAMA + LABA/ICS + placebo Background rate of exacerbation /year with LAMA Reduction in exacerbation rate provided by LAMA + LABA/ICS + roflumilast vs LAMA + LABA/ICS + placebo + LABA/ICS + placebo 15% 20% ,524 1,934 Based on the Committee's request for a clinical trial also assessing the effectiveness of roflumilast vs. theophylline (both as add-on agents to LAMA + LABA/ICS), further sample size calculations were performed. A reduction in the exacerbation rate for a theophylline regimen was applied to the background rate of exacerbations in the triple therapy group (assumed to be 1.25 exacerbations per year). However, there is a lack of well-designed, randomised clinical trials investigating the effectiveness of theophylline in reducing exacerbations. Given the lack of data, it was assumed that theophylline, when added to LAMA + LABA/ICS, was associated with a 10% reduction in the rate of exacerbation, implying a background exacerbation rate of per year with a LAMA + LABA/ICS + theophylline regimen. Based on the same assumptions as those outlined above, an analysis assumed that a regimen of LAMA + LABA/ICS + roflumilast would provide an additional 10% reduction over-and-above that of LAMA + LABA/ICS + theophylline. This implies a rate of exacerbations per patient per year in the roflumilast arm. Based on a required two-sided significance level of 5.0% (90% power), such a trial would require 4,518 patients per treatment arm, or 9,036 patients in total (Table 2). Assuming that the roflumilast arm provides an additional 15% reduction in the exacerbation rate (or exacerbations per patient per year), a sample size of 1,958 patients per arm would be required, or 3,916 in total. Table 2: Required sample sizes for a clinical trial of LAMA + LABA/ICS + roflumilast vs. LAMA + LABA/ICS + theophylline Background rate of exacerbation /year with LAMA Reduction in exacerbation rate provided by LAMA + LABA/ICS + roflumilast vs LAMA + LABA/ICS + theophylline + LABA/ICS + theophylline 10% 15% ,036 3,916 6

7 Given the difficulty in the recruitment for a trial assessing an intervention with theophylline (due to the acknowledged contraindications, side effects, and additional monitoring), the trial would take significantly longer than the trial vs. LAMA + LABA/ICS + placebo. It must be noted that none of these estimates account for attrition within the proposed trial programme. The discontinuation rate from the M2-124 and M2-125 clinical trials was approximately 30% 2. Applying this rate to the numbers required in the trial vs. placebo, it is estimated that between 2,500 to 4,500 patients will need to be randomised. The trial vs. theophylline will require significantly higher patient numbers to ensure appropriate power. Therefore, it would not be straightforward to set up the proposed trial, and there is significant difficulty in the practicalities of performing this trial. Criterion: The broad balance of the benefits and costs of conducting the research are favourable. "The Committee noted that in England and Wales there are approximately 1 million people with COPD, and the manufacturer s expectation that almost 200,000 would be eligible for treatment with roflumilast by 2015" [ACD, 4.19]. Based on the underlying prevalence of COPD 3, and recent data on the proportion of patients with severe and very severe disease who continue to experience frequent exacerbations 4, it was estimated that there are approximately 169,000 patients who are eligible to receive roflumilast, according to its licensed indication. This was projected to increase to 196,000 patients by 2015 [MS, Table C1]. However, the submission focussed on two patient sub-groups: those currently receiving LAMA + LABA/ICS, and LAMA + LABA. The clinical specialists and Committee both agreed that it was in these cohorts of patients where roflumilast would most likely be used [ACD, 4.4]. It is this smaller population that should be considered by the when balancing the benefits and costs of research. 7

8 Based on a combination of MS Table C1 and MS Table C2, it is estimated that there is a total population of approximately 76,000 in 2011, rising to 89,000 in 2015 (Table 3). Table 3: Number of patients in England and Wales who are eligible to receive roflumilast, and are currently managed by LAMA + LABA/ICS or LAMA + LABA Severe LAMA + LABA/ICS 47,720 49,500 51,346 53,262 55,248 LAMA + LABA 1,849 1,918 1,989 2,063 2,140 Total 49,569 51,418 53,335 55,325 57,389 Very severe LAMA + LABA/ICS 25,651 26,608 27,600 28,630 29,698 LAMA + LABA 1,125 1,167 1,210 1,255 1,302 Total 26,776 27,775 28,811 29,885 31,000 Total 76,345 79,192 82,146 85,210 88,389 Although this represents the eligible population, it is unlikely that roflumilast would be prescribed to all patients. The uptake of roflumilast in clinical practice was considered MS Table C3. MSD is aware that clinicians experienced in the care of patients with COPD would propose that roflumilast should only be initiated by a specialist (in the patient sub-groups identified). Any decision that incorporate such a proposal would mean that the populations identified above would be further reduced by a significant degree. Alongside this reduction in the patient population, we are uncertain whether the Committee has fully taken into account the significant cost to patients caused by the 'only in research' recommendation denying them access to roflumilast for an extended period of time. Given the documented risk of these patients experiencing a moderate or severe exacerbation (which carries a risk of death) in any 12 month period, such a recommendation implies that highly significant costs will be borne by these patients with regards to impact on quality of life. Clearly, any exacerbation would also lead to significant costs for the NHS in the management of these exacerbations. 8

9 If these factors are taken into account by the Committee, as well as the higherthan-anticipated difficulty of such a trial (and, thus, its cost), the "broad balance of benefits and costs of conducting the research" would no longer be favourable for patients, or the NHS. 2) Validity of the assumption of additivity for the treatment effect of roflumilast and the additive MTC Clinical trial evidence for the additive effect of roflumilast on exacerbations when used in combination with other COPD treatments (LABA, LAMA and ICS) is available and was presented in the MS. However, in the absence of data on some treatment regimens identified in the scope, a MTC was commissioned to examine the possibility of estimating the rate of exacerbation associated with these unstudied regimens. The MTC, which was independently-performed and has been recently published, was based on the latest available clinical data, as described in the MS 1. This analysis allowed all comparisons specified in the scope to be addressed in the cost effectiveness analysis for roflumilast. When discussing the results of the MTC, the ACD states: "The Committee concluded that, although roflumilast s mechanism of action meant it could have an additive effect, without any direct trial evidence for roflumilast in the anticipated position in the treatment pathway, it was not possible to know whether the treatment effect of roflumilast was additive and consistent across all treatment combinations" [ACD, 4.11] The ACD also summarises that: "The ERG considered the additive mixed treatment comparison analysis invalid and that it should not be used to estimate the effect of the combination treatments The ERG believed that there is an interaction between the treatment, such that it is not independent of the drug to which it is added and so the additive approach was therefore inappropriate" [ACD, 3.25]. 9

10 Mechanism of Action The use of an additive MTC is reliant on an underlying assumption that the benefits of the individual treatments are independent of one another. Each class of agents included in the current analysis (LABA, LAMA, ICS and the PDE4 inhibitor roflumilast) acts by a distinct mechanism, making this methodological approach applicable. This was confirmed by the clinical specialists during the Appraisal Committee meeting, as described in the ACD: "The Committee heard from one of the clinical specialists that roflumilast was likely to have an additive effect on top of triple therapy because its mechanism of action was different to the drugs used in triple and dual therapy" [ACD, 4.11]. Roflumilast is a selective PDE4 inhibitor which specifically targets the systemic and pulmonary inflammation associated with COPD. Neutrophils, which are considered to be the key effector cells in COPD 5, are well-acknowledged targets of PDE4 inhibitors, because PDE4 is almost exclusively the PDE enzyme involved in camp degradation in these cells. In vitro studies have demonstrated that roflumilast and its active metabolite roflumilast N-oxide reduce human neutrophil superoxide release, degranulation, formation of LTB 4, chemotaxis and CD11b expression (CD11b is a cell adhesion molecule critical for interaction with endothelial cells thus governing an initial step in neutrophil tissue infiltration). Roflumilast mitigates the airway neutrophilic inflammatory response following exposure of mice and guinea pigs to tobacco smoke in preventive protocols, and guinea pigs in a therapeutic protocol with the PDE4 inhibitor being orally administered over the final 6 days of smoke exposure 6. Roflumilast was also shown to reduce the number of neutrophils in induced sputum of patients with moderate-severe COPD in a randomised, controlled, double blind clinical trial 7. In contrast to roflumilast, in vivo studies have demonstrated that glucocorticoids remain ineffective in reducing the airway neutrophilic inflammatory response to tobacco smoke exposure 6, reflecting steroid-resistant neutrophilic inflammation in COPD. Clinical trials have also shown that inhaled or oral steroids do not affect sputum neutrophils in COPD 8;9. β2-receptor agonists may suppress the immediate neutrophil cellular response in vitro, however, β2 receptors are considered to more rapidly desensitise on inflammatory cells 10

11 compared to contractile airway smooth muscle cells 10, which may account for the observation that LABAs have so far not shown anti-inflammatory effects in patients with COPD. Tiotropium has been shown to reduce airway neutrophil influx following tobacco smoke exposure to mice in a comparable dose range to that which evokes bronchodilation 11;12. To date there are no reports indicating that LABA or LAMA would suppress the airway neutrophilic inflammation in patients with COPD. Roflumilast and roflumilast N-oxide reduce excessive human lung fibroblast activity in vitro 13;14 and such effects are not shared by glucocorticoids. These cellular effects of the PDE4 inhibitors to directly target lung fibroblasts may translate into the potential of roflumilast to mitigate the bleomycin-induced lung fibrotic response in mice and rats in preventive and even therapeutic protocols, while in the latter glucocorticoids were found to be ineffective 15. PDE4 inhibitors, including roflumilast, are usually poor bronchodilators, which has been demonstrated in vitro, in vivo and in clinical trials. Roflumilast does cause an incremental benefit in FEV 1 on top of LABA and LAMA 16, and this increment occurs irrespective of whether FEV 1 is measured pre- or post-bronchodilator. This strongly supports the novel mechanism of roflumilast, which differs from bronchodilation, as bronchodilators induce less incremental benefit of post-bronchodilator FEV 1 compared to pre-bronchodilator FEV 1. Clinical and pre-clinical evidence support the concept that roflumilast has a unique mechanism of action that differentiates the agent from other COPD therapies. Roflumilast may be the first therapy to specifically target the abnormal inflammatory response in COPD. Additive Effect of Roflumilast When Used in Combination with Other Agents Although data are not available from clinical trials to demonstrate the effect of roflumilast when added to dual or triple therapy, there is evidence which shows that the effect of roflumilast is additive when used in combination with LABA, LAMA or ICS individually. The roflumilast clinical trials were not studies of roflumilast vs placebo only, but all were either additive studies (M2-127 and M ) or included sizable populations who 11

12 received concomitant medications (in M2-124 and M ;17 patients could receive concomitant LABA; in M2-111 and M2-112 patients could receive concomitant ICS 18 ). Data from these studies (Table 4) show that roflumilast resulted in at least a 20% decrease in the rate of moderate or severe exacerbations, when used concomitantly with other COPD medication, compared to that agent alone. This demonstrates that the efficacy of roflumilast in reducing the rate of exacerbations is not blunted when used in addition to LABA, LAMA or ICS. Table 4: Mean Rate of Moderate or Severe Exacerbations for Roflumilast in Combination with LABA, LAMA or ICS Roflumilast + LABA vs Placebo + LABA M2-124 and M (approximately 50% patients received concomitant LABA) M2-127 a 19 (all patients received concomitant LABA) Roflumilast + LAMA vs Placebo + LAMA M (all patients received concomitant LAMA) Roflumilast + ICS vs Placebo + ICS M2-111 and M (37% of the total 2686 patients had chronic bronchitis and received concomitant ICS) Mean rate of moderate/severe exacerbations Difference (per patient per year) Roflumilast + LABA Placebo + LABA Rate ratio; % change (two-sided p value) (N = 749) (N = 793) 0.793; -20.7% (p = ) 0.3 (N = 466) 0.5 (N = 467) 0.63; -36.8% (p = ) Roflumilast + LAMA Placebo + LAMA Rate ratio; % change (two-sided p value) (N = 371) (N = 372) 0.768; -23.2% (p = ) Roflumilast + ICS Placebo + ICS Rate ratio; % change (two-sided p value) (N = 492) (N = 493) 0.698; -30.2% (p = 0.001) N = number of patients in the treatment arm who received the specified concomitant therapy. (For M2-111 and M2-112, N = the number of patients in the treatment arm who had chronic bronchitis and who received the specified concomitant therapy.) a In M2-127 the mean rate of moderate/severe exacerbations was calculated in a post-hoc analysis Note: Studies M2-127 and M2-128 were powered for change in lung function and not for exacerbation reduction. The NICE clinical guideline for COPD (CG101) recommends the use of therapies in combination for patients with severe disease, reflecting the same acceptance of additivity of treatment effects as applied in the additive MTC. Randomised controlled trials provide evidence to demonstrate the benefit of adding LABA to ICS compared to LABA alone, both in terms of improving lung function and reducing exacerbations. 12

13 Some evidence is available for triple therapy (LABA + ICS + LAMA) and for the combination of LABA + LAMA, which suggests increased benefit when the agents are used in combination, although for most comparisons data from appropriately designed RCTs are limited. Despite the limitations of the evidence in some cases, the Guideline Development Group (GDG) acknowledged the additive effects of COPD therapies when used in combination, and recommended the combination therapy for patients with severe disease who remain symptomatic on their current treatment. Given that the treatment effects of currently recommended dual and triple therapy are accepted as being additive, as endorsed by CG101, and that the treatment effect of roflumilast is additive when used on top of any one of these agents individually, it is expected that roflumilast would have an additive effect when used as add on to dual or triple therapy. ERG Evidence for the Breaking of the Additive Assumption Table 3.2 in the ERG report (reproduced below) provides the estimated treatment effect (RR) of each class of agents when added to placebo or to the other classes, from the standard MTC. The ERG state: "As the MS shows (Table B30, page 121), the effect of roflumilast vs placebo is a RR of However, the effect of adding roflumilast to LABA is a RR of 0.80 and 0.84 of adding to LAMA. In other words, the effect of roflumilast is not independent of the drug to which it is added" [ERG report, page 17]. However, we would argue that the estimated values of 0.80, 0.84 and 0.85 are similar, particularly if the distributions around the mean estimated values are taken into account. The 95% confidence intervals have been added to the table below to allow the Committee to make a fully informed decision. Once the 95% CIs are considered, it is clear that the estimated RRs for roflumilast + placebo, roflumilast + LABA and roflumilast + LAMA lie within the 95% CI of the other two combinations, demonstrating that the values cannot be treated as being different. Critically, the estimated RRs for all of these combinations lie within the 95% CI for the comparison of roflumilast vs placebo, which is based on direct trial evidence and therefore has the tightest CI. This also applies for all other rows in Table 3.2 of the ERG report, the exception being only where LAMA and LABA are combined. This discrepancy is addressed in the 13

14 manufacturers submission [MS, page 124], and is believed to be due to the very limited data available on the combination of LAMA + LABA (as acknowledged in CG101). Irrespective of this, Table 3.2 in the ERG report demonstrates that the assumption of additivity of treatment effects is valid, a degree of uncertainty for the LABA + LAMA combination where limited data are known to be an issue (Table 5). Table 5: Table 3.2 from ERG Report 'Additive effects of different treatment components' [ERG report, page 17; 95% CI added] Add: To Standard MTC Additive MTC placebo Roflumilast LABA LAMA ICS LABA+LAMA Any Roflumilast 0.85 (0.72, 0.97) 0.80 (0.59, 1.08) 0.84 (0.61, 1.14) 0.84 B31 (Table and Key findings LABA 0.84 (0.75, 0.93) 0.79 (0.54, 1.12) 1.09 (0.77, 1.49) 0.86 (0.76, 0.98) NR LAMA 0.74 (0.66, 0.81) 0.73 (0.50, 1.04) 0.96 (0.66, 1.35) NR ICS NR (0.71, 0.90) (0.74, 0.92) (0.61, 1.18) ICS + LAMA NR 0.82 (0.56, 1.15) NR ICS + LABA 0.69 (0.61, 0.77) 0.92 (0.66, 1.28) NR LAMA + LABA 0.80 (0.56, 1.12) 0.86 (0.58, 1.21) NR Internal Validity of the Additive MTC An assessment of the internal validity of the MTC was conducted by comparing results derived from the direct meta-analysis, the standard MTC and the additive MTC analysis for those treatment comparisons for which direct trial data was available. This demonstrated an overall consistency of the results from the three types of analysis, supporting the use of the additive approach to estimate the treatment effects of the missing combinations. For example, for the comparison of LABA + ICS vs placebo, the direct meta-analysis resulted in a rate ratio of 0.72 (95% CI ) and the standard MTC gave a similar rate ratio of 0.69 ( ) 1. A simple calculation of the additive effect of LABA + ICS vs placebo can be made by multiplying the effect of LABA vs placebo (RR = 0.87) and ICS vs placebo (RR = 0.81) from the direct meta-analysis. This results in an estimation 14

15 of the additive effect of LABA + ICS vs placebo of 0.70 (i.e x 0.81), which is similar to the results obtained from the direct meta-analysis and the standard MTC for this combination vs placebo, demonstrating the consistency of the additive approach with standard meta-analysis and MTC methods. As stated in the published MTC 1, the goodness of fit of each of model, including the additive MTC model, to the data was measured by calculating the residual deviance and comparing it against the number of unconstrained data points. The number of unconstrained data points was obtained by summing up the number of study arms across all studies included in the analyses. Under the null hypothesis that the model provides an adequate fit to the data, the residual deviance is expected to have a mean equal to the number of unconstrained data points. The additive MTC model considered in the analysis provided reasonable fit to the data, as demonstrated by the low residual deviance (69.56) compared to the unconstrained number of data points (62). The authors of the additive MTC, in collaboration with John Ioannidis, are developing a manuscript for publication which supports the additive approach. Professor Mills has given us permission to share an early draft of the manuscript with the Committee. This document is attached with our response and should be treated as Academic in Confidence. 3) Relevance and appropriateness of including evidence from patients with FEV 1 > 50% predicted in the MTC The Committee and ERG raised concerns over the inclusion of data from patients with moderate COPD in the MTC. The MS presented results of a recently published mixedtreatment comparison (MTC), which assessed the current literature on the prevention of COPD exacerbation with pharmacotherapy 1 [MS, 5.6]. The inclusion criteria of the MTC meant that data from people with moderate, severe and very severe COPD were included. Had the inclusion criteria restricted the studies to patients with severe or very severe COPD, this would have resulted in only eight clinical trials being included in the analysis. We did not believe that results from such a MTC would have been informative. 15

16 Furthermore, the scope for this STA defined a range of treatment options to be assessed. Had the inclusion criteria been restricted to trials performed only in patients with FEV 1 < 50% predicted, it would not have been possible to assess several of these treatment regimens. This would have included the loss of both the LAMA + LABA/ICS, and LAMA + LABA treatment regimens, which are considered to be the standards of care for patients with severe and very severe COPD 3. The ACD reports that "The clinical specialists said that from trial evidence they anticipated that the relative effect of roflumilast was likely to be similar in moderate and in severe disease", thereby providing support for widening the inclusion criteria of the MTC [ACD, 4.10]. However, the ACD went on to state "Conversely, the Committee heard from the ERG that there was evidence from COPD trials in which roflumilast had been shown to be more effective in people with moderate disease than people with severe disease" [ACD, 4,10]. This statement is incorrect. This information is not referenced to a clinical trial and it is our understanding that the ERG has mistakenly interpreted data on the reduction of moderate and severe exacerbations as data on moderate and severe COPD severity. No trial has found roflumilast to be more effective in patients with moderate disease, compared with severe disease. The pivotal trials for roflumilast (M2-124 and M2-125) found a statistically significant reduction in the primary endpoint of the rate of moderate and severe exacerbation, between patients in the roflumilast arm vs. the comparator arm (RR 0.83, 95% CI 0.75, 0.92; p = ). When analysed by type of exacerbation, there were numerically similar reductions in the rate of moderate and severe exacerbations (0.83 and 0.82, respectively), although only the reduction in moderate events reached statistical significance. Similar results were noted in other studies in the roflumilast clinical trial program. We believe that it is these results which the ERG has mistakenly taken to refer to greater effectiveness of roflumilast in patients with moderate (vs. severe) COPD. This error by the ERG appears to be the basis for the Committee concluding " that it was uncertain whether treatment response differed in moderate and severe disease. It agreed that the Committee would have benefited from seeing data from individual trial 16

17 participants to show whether or not roflumilast had the same relative effect, independent of disease severity" [MS, 4.10]. To further reassure the Committee of the patient population included within the MTC, data from individual trial participants were assessed for an interaction between disease severity (moderate, severe and very severe COPD) and the effectiveness of roflumilast. Separate analyses were performed using the pooled data from the M2-124 and M2-125 clinical trials, and for the pool of one-year trials (M2-111, M2-112, M2-124 and M2-125; only patients with chronic bronchitis [+/- emphysema] were included from the M2-111 and M2-112 trials). In the pool of the one-year trials the following RRs of moderate or severe COPD exacerbations were seen for the stratified disease severity stages: moderate (p = ); severe (p = ), and; very severe (p = ). The lower number of exacerbations experienced by moderate COPD patients (compared to severe or very severe patients), explains the apparent smaller reduction of exacerbations. When testing for an interaction between disease severity and the effectiveness of roflumilast no significant difference was identified, with results for both analyses showing that the effect of roflumilast is independent of the disease severity. The interaction test based on the pooled data from M2-124 and M2-125 was associated with a p-value of The p-value identified by the analysis of the one-year trials was Given the results of the requested interaction tests and clarification of the ERG's misinterpretation of trial data, we believe that it was both relevant and appropriate to include data from non-severe COPD patients in the MTC. 4) Inappropriateness of a comparison with theophylline As described in the MS, theophylline was not considered to be an appropriate comparator for roflumilast in clinical practice. NICE CG101 positions theophylline very late in the treatment pathway 3. As confirmed by the clinical specialists, this is largely 17

18 due to the side effect profile and the need for additional monitoring with use of theophylline. There is a lack of well-designed randomised clinical trials investigating the effectiveness of theophylline in reducing exacerbations. The dosing of theophylline is limited to a relatively narrow therapeutic window by adverse effects. Non-selective PDE inhibition is likely to account for the most frequent side effects of the agent, including nausea, vomiting and headaches, cardiac arrhythmias and palpitations. As noted in the ACD, "The Committee heard that theophylline is contraindicated in some people because of interactions with other drugs, that it has many side effects, and that it requires additional monitoring" [ACD, 4.2]. This was also confirmed by the evidence submissions of the clinical experts. Current estimates suggest that in clinical practice theophylline is only prescribed in approximately 5% of patients with severe or very severe COPD, including amongst those currently receiving triple therapy with LAMA + LABA / ICS, or LAMA + LABA (in those who do not tolerate or decline ICS). We accept that the Committee is interested in understanding the efficacy of theophylline in relation to roflumilast but based on the factors described above, which include a lack of robust clinical data, we do not agree that theophylline should be considered as a comparator for roflumilast. 5) Revised cost-effectiveness analysis results for patients continuing on baseline treatment For patients able to tolerate ICS, the economic submission assessed a scenario whereby patients receive LAMA + LABA/ICS + roflumilast, or LAMA + LABA/ICS, for one year. For the purposes of the fully incremental cost-effectiveness analysis, after this initial 12 month period, patients receive a common second-line treatment regimen of LAMA + LABA/ICS. That is, patients initially receiving roflumilast are discontinued from the treatment and receive a triple therapy regimen, whereas patients initially receiving triple therapy continue on the regimen for the remainder of the time horizon. 18

19 Scenario analysis E, presented in Table B87 and Table B93 of the MS, also assessed the case whereby LAMA + LABA/ICS + roflumilast is received by both cohorts, after the initial 12 month first-line treatment period. In this scenario, patients initially receiving LAMA + LABA/ICS + roflumilast continued on the regimen throughout the model time horizon. Patients initially receiving LAMA + LABA/ICS were stepped up to receive LAMA + LABA/ICS + roflumilast for the remainder of the time horizon. As noted in Section 4.12 of the ACD, this scenario did not materially affect the ICER. Section 4.12 also states "The Committee heard from the clinical specialists that once a person with COPD starts treatment with a particular drug it is difficult for them to discontinue treatment because of concerns that their condition might deteriorate as a result The Committee concluded that the scenario in which treatment with roflumilast continued for the lifetime of the person with COPD was more likely to reflect clinical practice than the base case in which people stopped treatment with roflumilast after a year and then remained on triple therapy alone." In response, an analysis has been performed which assesses the scenario of patients continuing on their initially assigned therapy for the duration of the time horizon that is, the model assesses the scenario that patients receive LAMA + LABA/ICS + roflumilast vs. LAMA + LABA/ICS for a 30 year time horizon (Table 6 and Table 7). From a model structure perspective, this is equivalent to the model collapsing to a three state Markov model, consisting of health states S 1, VS 1 and death (i.e. there is no second line treatment regimen). A similar analysis was conducted for ICS-intolerant patients, as described below (Table 8 and Table 9). Comparisons: ICS-tolerant patients: LAMA + LABA/ICS + roflumilast vs. LAMA + LABA/ICS ICS-intolerant patients: LAMA + LABA + roflumilast vs. LAMA + LABA 19

20 Table 6: ICS-tolerant patients; Revised scenario analysis LAMA + LABA/ICS + Roflumilast LAMA + LABA/ICS Incremental Cost, 26,602 22,816 3,786 QALY ICER, /QALY 16, LAMA + LABA/ICS + roflumilast minus LAMA + LABA/ICS Table 7: ICS-tolerant patients; Original base case analysis (MS, Table B 67, p. 244) LAMA + LABA/ICS + Roflumilast LAMA + LABA/ICS Incremental Cost, 23,230 22, QALY ICER, /QALY 16, LAMA + LABA/ICS + roflumilast minus LAMA + LABA/ICS The difference in the ICER (revised analysis minus original base case) is 95.35, highlighting that this adjustment only has a small impact on cost-effectiveness results, as also seen in Scenario Analysis E of the submission. Table 8: ICS-intolerant patients: Revised scenario analysis LAMA + LABA + Roflumilast LAMA + LABA Incremental Cost, 25,408 21,814 3,594 QALY ICER, /QALY 13, LAMA + LABA + roflumilast minus LAMA + LABA Table 9: ICS-intolerant patients; Original base case analysis (MS, Table B 68, p. 244) LAMA + LABA + Roflumilast LAMA + LABA Incremental Cost, 22,222 21, QALY ICER, /QALY 13, LAMA + LABA + roflumilast minus LAMA + LABA As per the results for the analysis of ICS-tolerant patients, adjusting the base case analysis to account for lifelong treatment on the initially assigned treatment regimen only has a small effect on the ICER, of (revised analysis minus original base case). 20

21 6) The ERG approach to evidence synthesis has not necessarily always helped inform the Committee As described elsewhere in this response, the approach taken in the independentlyperformed MTC used in the MS was supported by the clinical specialists [ACD, 4.10], and by statistical analyses finding no interaction between COPD disease severity and effectiveness of roflumilast, as requested by the Committee [ACD, 4.10]. Based on a fundamentally different approach to the scope of the appraisal, the Evidence Review Group (ERG) performed two of their own MTCs. The first MTC included only those trials where inclusion criteria were set at FEV 1 < 50% predicted. The second MTC extended this to an arbitrary cut-off point of trials with inclusion criteria of FEV 1 < 65% predicted. We feel there are significant issues with both of these analyses. There was also an error by the ERG in their interpretation of the roflumilast clinical trial data, whereby moderate and severe exacerbations were confused with moderate and severe COPD severity. MTC of trials with FEV 1 < 50% predicted ('ERG base-case analysis') The ACD describes that "the ERG conducted its own mixed treatment comparison, which was more restrictive than the manufacturer s. It sought to provide evidence for the population that was defined in the scope by restricting the evidence base to only those studies that included participants with FEV 1 50% predicted or less. This left nine trials covering seven treatments in the mixed treatment comparison, as opposed to the 29 trials in the manufacturer s submission" [ACD, 3.26]. The MTC presented in the MS was conducted with the intention of providing evidence for all combinations and comparators specified in the final scope. The work conducted by the ERG appears to have extended beyond a critique of the MS. The criteria used by the ERG to select the evidence to present to the Appraisal Committee have the effect of inappropriately limiting the information available to only some of the drug combinations outlined in the scope. The inclusion and evaluation of all 21

22 specified treatment options and comparators will allow the Appraisal Committee to make an informed decision, in line with the scope for the STA. MTC of trials with FEV 1 < 65% predicted ('ERG alternative analysis') Within the ERG report, the following statement was made: "The MS presents results for 15 treatments for the management of COPD. Seven treatments have been included in the ERG base-case analyses. Two treatments (LABA/ICS+LAMA and LABA+LAMA) can be included in the network if we widen the inclusion criteria to FEV 1 65%. Although this is not the population as described in the scope, the ERG deemed this approach acceptable, since the Appraisal Committee will be particularly interested in the relative effectiveness of roflumilast in comparison with LABA/ICS+LAMA as this was found to be the most effective treatment not involving roflumilast in the MS" [ERG report, page 34]. We see this approach as being devoid of any clinical rationale, and entirely arbitrary. If the ERG had chosen a cut-off only 5% different to that chosen for the ERG-alternative analysis (FEV 1 70%, rather than 65% predicted), evidence from a further 7 clinical trials would have been incorporated. This evidence would include the roflumilast clinical trial M , therefore, allowing LAMA + roflumilast to be included in the analysis, and for the Committee to make an informed decision on this treatment option. Issues common to both MTCs conducted by the ERG Both approaches taken by the ERG lead to the exclusion of treatment regimens which were defined in the scope. Based on the comments from the clinical specialists and lack of interaction between disease severity and effect on exacerbations, it is entirely appropriate to include all available data within the MTC, thus allowing all comparators listed in the scope to be assessed. Furthermore, in their conduct of their MTCs, the ERG have also excluded a significant amount of data in patients with FEV 1 < 50%. For example, one-third of patients who received roflumilast in M2-128 had severe COPD [MS, page 53]. These patients should not be excluded from the evidence base. Results for FEV improvement and exacerbation rates in the subgroup of patients with severe COPD from M2-128 were 22

23 reported in the CSR, which was submitted with the MS. The EMA considered the data from M2-128 during their decision making. Based on these data, the combination of LAMA + roflumilast is included in the licence for roflumilast. Excluding data from M2-128 from the evidence base for the STA is inconsistent with the data on which the EMA based their decision, and with the scope for the STA. 23

24 References (1) Mills EJ, Druyts E, Ghement I, Puhan M. Pharmacotherapies for chronic obstructive pulmonary disease (COPD): a multiple treatment comparison meta-analysis. Clinical Epidemiology 2011;3: (2) Calverley PMA, Rabe KF, Goehring UM et al. Roflumilast in symptomatic chronic obstructive pulmonary disease: two randomised clinical trials. Lancet 2009;374: (3) NICE Clinical Guideline 101. Chronic obstructive pulmonary disease: Management of chronic obstructive pulmonary disease in adults in primary and secondary care. Update Guideline Ref Type: Report (4) Hurst JR, Vestbo J, Anzueto A et al. Susceptibility to Exacerbation in Chronic Obstructive Pulmonary Disease. New England Journal of Medicine 2010;363: (5) Sapey E, Stockley RA. Chapter 14 - The Neutrophil and Its Special Role in Chronic Obstructive Pulmonary Disease. Asthma and COPD (Second Edition). Oxford: Academic Press; 2009; (6) Hatzelmann A, Morcillo EJ, Lungarella G et al. The preclinical pharmacology of roflumilast Çô A selective, oral phosphodiesterase 4 inhibitor in development for chronic obstructive pulmonary disease. Pulmonary Pharmacology & Therapeutics 2010;23: (7) Grootendorst DC, Gauw SA, Verhoosel RM et al. Reduction in sputum neutrophil and eosinophil numbers by the PDE4 inhibitor roflumilast in patients with COPD. Thorax 2007;62: (8) Keatings VM, Jatakanon A, Worsdell YM, Barnes PJ. Effects of inhaled and oral glucocorticoids on inflammatory indices in asthma and COPD. Am J Respir Crit Care Med 1997;155: (9) CULPITT SV, MAZIAK WASI, LOUKIDIS STEL, NIGHTINGALE JA, MATTHEWS JL, BARNES PJ. Effect of High Dose Inhaled Steroid on Cells, Cytokines, and Proteases in Induced Sputum in Chronic Obstructive Pulmonary Disease. Am J Respir Crit Care Med 1999;160: (10) Barnes PJ. Effect of beta-agonists on inflammatory cells. Journal of Allergy and Clinical Immunology 1999;104:S10-S17. (11) Wollin L, Pieper MP. Tiotropium bromide exerts anti-inflammatory activity in a cigarette smoke mouse model of COPD. Pulmonary Pharmacology & Therapeutics 2010;23: (12) Wollin L, Gantner F, Pieper M. Tiotropium reduces established pulmonary inflammation in a 12 weeks cigarette smoke mouse model of COPD. Presented at ERS 2011, Abstract Ref Type: Report (13) Sabatini F, Petecchia L, Boero S et al. A phosphodiesterase 4 inhibitor, roflumilast N- oxide, inhibits human lung fibroblast functions in vitro. Pulmonary Pharmacology & Therapeutics 2010;23: