Golimumab for the treatment of ankylosing spondylitis

Transcription

1 Golimumab for the treatment of ankylosing spondylitis Produced by Authors Correspondence to Kleijnen Systematic Reviews Ltd. Rob Riemsma, Reviews Manager, KSR Ltd. Manuela Joore, Health Economist, Maastricht UMC Thea van Asselt, Health Economist, Maastricht UMC Nigel Armstrong, Health Economist, KSR Ltd. Kate Misso, Information Specialist, KSR Ltd. Nathan Manning, Systematic Reviewer, KSR Ltd. Florian Tomini, Health Economist, Maastricht UMC Jos Kleijnen, Director, KSR Ltd. Rob Riemsma, Kleijnen Systematic Reviews Unit 6, Escrick Business Park Riccall Road, Escrick York, UK YO19 6FD Date completed 11/04/2011 Source of funding: This report was commissioned by the NIHR HTA Programme as project number 08/205 STA. Declared competing interests of the authors None. 1

2 Acknowledgements Not applicable Rider on responsibility for report The views expressed in this report are those of the authors and not necessarily those of the NIHR HTA Programme. Any errors are the responsibility of the authors. This report should be referenced as follows: Riemsma R, Joore M, Van Asselt T,, Armstrong N, Misso K, Manning N, Tomini F, Kleijnen J. Golimumab for the treatment of ankylosing spondylitis: a Single Technology Appraisal. York: Kleijnen Systematic Reviews Ltd., Contributions of authors Rob Riemsma acted as project lead and systematic reviewer on this assessment, critiqued the clinical effectiveness methods and evidence and contributed to the writing of the report. Nigel Armstrong acted as health economist on this assessment, critiqued the manufacturer s economic evaluation and contributed to the writing of the report. Kate Misso critiqued the search methods in the submission and contributed to the writing of the report. Nathan Manning acted as systematic reviewer, critiqued the clinical effectiveness methods and evidence and contributed to the writing of the report. Thea van Asselt acted as health economist on this assessment, critiqued the manufacturer s economic evaluation and contributed to the writing of the report. Manuela Joore acted as health economic project lead, critiqued the manufacturer s economic evaluation and contributed to the writing of the report. Florian Tomini acted as health economist on this assessment, critiqued the manufacturer s economic evaluation and contributed to the writing of the report. Jos Kleijnen critiqued the manufacturer s definition of the decision problem and their description of the underlying health problem and current service provision, contributed to the writing of the report and supervised the project. 2

3 Abbreviations AE Adverse Event AiC Academic in Confidence AS Ankylosing Spondylitis ASAS ASsessment in Ankylosing Spondylitis BASDAI Bath Ankylosing Spondylitis Disease Activity Index BASFI Bath Ankylosing Spondylitis Functional Index BASMI Bath Ankylosing Spondylitis Metrology Index BSR British Society for Rheumatology CAS Chemical Abstracts Service CEA Cost effectiveness analysis CiC Commercial in Confidence DMARDS Disease Modifying Anti-Rheumatic Drug DSU Decision Support Unit EQ-5D European Quality of Life 5 Dimensions ERG Evidence Review Group FTC Fixed Treatment Comparison Gol Golimumab HTA Health Technology Assessment HRQL Health Related Quality of Life ICER Incremental Cost Effectiveness Ratio KOL Key Opinion Leaders KSR Kleijnen Systematic Reviews MS Manufacturer Submission MTC Mixed Treatment Comparison MTX Methotrexate NHS National Health Service NICE National Institute for Health and Clinical Excellence NIHR National Institute for Health Research NMB Net monetary benefits NSAIDS Non Steroidal Anti-Inflammatory Drug PAS Patient Access Scheme PSA Probabilistic Sensitivity Analysis PSS Personal and Social Services QALY Quality Adjusted Life Years QOL Quality of Life RCT Randomised Controlled Trial SF-36 Short Form 36 SAE Serious Adverse Events STA Single Technology Assessment TAG Technology Appraisal Guidance TAR Technology Assessment Report TB Tuberculosis TNF Tumour Necrosis Factor WHO World Health Organisation 3

4 Contents 1 SUMMARY Scope of the manufacturer submission Summary of clinical effectiveness evidence submitted by the manufacturer Summary of cost effectiveness submitted evidence by the manufacturer ERG commentary on the robustness of evidence submitted by the manufacturer Strengths Weaknesses Areas of uncertainty Key issues 11 2 BACKGROUND Critique of manufacturer s description of underlying health problem Critique of manufacturer s overview of current service provision 13 3 CRITIQUE OF MANUFACTURER S DEFINITION OF DECISION PROBLEM Population Intervention Comparators Outcomes Other relevant factors 18 4 CLINICAL EFFECTIVENESS Critique of the methods used by the manufacturer to systematically review clinical effectiveness evidence State objective of systematic review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate? Search for clinical evidence Search strategy for cost-effectiveness State the inclusion/exclusion criteria used in the study selection and comment on whether they were appropriate What studies were included in the clinical effectiveness review and what were excluded? Provide a table of identified studies. Please identify the most important clinical effectiveness studies Provide details of any relevant studies not discussed in the submission? Why were these studies excluded and how were these studies identified by the ERG? Summary and critique of submitted clinical effectiveness evidence Summary of submitted clinical evidence for each relevant trial Describe and critique the manufacturer s approach to validity assessment for each relevant trial Describe and critique the statistical approach used within each relevant trial Describe and critique the manufacturer s approach to outcome selection within each relevant trial To what extent does each relevant trial include the patient population(s), intervention(s), comparator(s) and outcomes as defined in the final scope? Where appropriate, describe and critique any meta-analysis, indirect comparisons and/ or mixed treatment analysis carried out by the manufacturer Additional clinical work conducted by the ERG Conclusions 56 4

5 5 COST EFFECTIVENESS ERG comment on manufacturer s review of cost-effectiveness evidence State objective of cost effectiveness review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate State the inclusion/exclusion criteria used in the study selection and comment on whether they were appropriate What studies were included in the cost effectiveness review and what were excluded? Where appropriate, provide a table of identified studies. Please identify the most important cost effectiveness studies What does the review conclude from the data available? Does the ERG agree with the conclusions of the cost effectiveness review? If not, provide details Summary and critique of manufacturer s submitted economic evaluation by the ERG NICE reference case checklist Model structure Population Interventions and comparators Perspective, time horizon and discounting Treatment effectiveness Health related quality of life Resources and costs Cost effectiveness results Sensitivity analyses Model validation Additional work undertaken by the ERG Conclusions IMPACT ON THE ICER OF ADDITIONAL CLINICAL AND ECONOMIC ANALYSES UNDERTAKEN BY THE ERG END OF LIFE CONCLUSIONS Implications for research 109 REFERENCES 110 Appendix 1: Quality Assessment using ScHARR-TAG economic modelling checklist 114 Appendix 2: ERG search strategies 115 Appendix 3: Philips et al. checklist 125 Appendix 4: Cost and QALY outcome for the sensitivity analyses that were performed by the manufacturer, reproduced by ERG 128 5

6 1. SUMMARY 1.1 Scope of the manufacturer submission The NICE scope for this submission was to assess the clinical effectiveness and cost effectiveness of golimumab, administered within its license indication, in the treatment of adults with severe, active ankylosing spondylitis (AS) whose response to conventional therapy has been inadequate. The specified comparators were conventional care, adalimumab and etanercept. 1.2 Summary of clinical effectiveness evidence submitted by the manufacturer The MS included one trial of golimumab versus placebo (the GO-RAISE trial) and additionally seven placebo controlled RCTs of other anti-tnf agents (five with etanercept, and two with adalimumab). The populations in these studies were not completely compatible with the scope. Three out of 8 included trials included patients with disease severity similar to the scope (GO-RAISE, ATLAS and Brandt). None of the studies fulfilled the criterion of 2 or more failed NSAIDs. However, three studies included patients who had failed at least one NSAID (GO-RAISE, ATLAS and Can AS).In GO-RAISE, patients received subcutaneous injection of placebo, or 50mg golimumab or 100mg golimumab at 4 week intervals. According to the marketing authorisation 100mg golimumab is only indicated for patients who exceed 100kg and who have failed to achieve a satisfactory response to the50mg dose regimen after 3 or 4 injections (12 to 14 weeks). Few patients in GO-RAISE exceeded 100kg body mass and the dose regimen of 100mg did not follow failure of response to a 50mg regimen, therefore the clinical effectiveness results observed for the 100mg arm of GO-RAISE are of limited relevance to the decision problem. The primary outcome in GO-RAISE and in most other trials was the proportion of patients in the short term (6 to 16weeks depending on trial) achieving an ASsessment in AS International Working Group criteria 20 (ASAS 20) response. This outcome was reported in all trials. Relative to placebo at 14 weeks the 50mg golimumab regimen in the GO-RAISE trial delivered statistically significant results in favour of golimumab for several outcomes including: ASAS (20, and 40); Bath AS Disease Activity Index 20, 50, 70, and 90 (BASDAI 20, 50, 70, and 90) responses; change from baseline in Bath AS Disease Activity Index (BASDAI) and Bath AS Functional Index (BASFI) scores. Similar results were reported for the placebo controlled trials with other anti-tnf agents. At 16 weeks in the GO-RAISE trial patients were classified as responders or as nonresponders and non-responders could cross over to early escape therapy (to the 50mg regimen for non-responders receiving placebo and to 100mg regimen for non-responders receiving the 50mg regimen). This compromised the analysis of results at the end of the trial (week 24) because 53% of 6

7 patients in the placebo arm had received 50mg doses of golimumab and 18% of patients in the 50mg arm had received 100mg doses of golimumab which, unless the patient happened to exceed a body mass of 100kg, was out of licence. For this reason the ERG excluded 24 week follow-up data from the analyses. There were no head-to-head trials in which one anti-tnf agent was compared to another. In order to estimate the relative effectiveness of the anti-tnf agents relevant to the decision problem the submission conducted Bayesian fixed effects mixed treatment comparisons (MTCs). The outputs from the MTCs also provided inputs for the cost-effectiveness analyses. Generally, the ERG agrees with the MTC analyses. The main difference between the MS data used in the MTC and those used in a new analysis by the ERG, is the inclusion of 12 weeks follow-up data for Brandt et al. in the MS. The study by Brandt et al. reports main outcomes at 6 weeks follow-up: After week 6, patients in the placebo group were switched to etanercept for the next 12 weeks, and patients in the etanercept group continued to receive etanercept for another 6 weeks, to ensure that all patients received etanercept for a total of 3 months. Therefore, data at 12 weeks follow-up are underestimating the effects of etanercept because all patients in the placebo have received etanercept for 6 weeks. The ERG group believes that 6-week data are probably a better reflection of the true effect of etanercept in the study by Brandt et al. Therefore, the ERG used 6-week data from Brandt et al. in the analyses. The MTC results indicated that golimumab, etanercept and adalimumab were more clinically effective than placebo according to Bath Ankylosing Spondylitis disease activity and functional indices (i.e. BASDAI and BASFI measures) and according to ASAS criteria. Relative to placebo the anti-tnf agents were associated with more likely injection site reactions and more discontinuations from treatment. When the anti-tnf agents golimumab, adalimumab and etanercept were compared with each other using the Bayesian MTCs, the 95% credible intervals for most comparisons failed to demonstrate statistically significant differences between golimumab and its comparators; exceptions were a greater risk of discontinuation of treatment for golimumab than for etanercept (OR=5.14 (1.03, 39.21)), superior reduction in the short term in the disease activity index (BASDAI) for golimumab relative to etanercept (MD= (-1.58, -0.14)) and significant improvement in the short term in the Metrology Index (BASMI a measure of changes in spinal movement) for golimumab relative to adalimumab (MD= 0.52 (0.23, 0.80)). Errors and poorly applied limits were identified in several of the search strategies, which may have impacted on the recall of the search process. However, the ERG was unable to determine whether any relevant studies were not identified. 7

8 1.3 Summary of cost effectiveness submitted evidence by the manufacturer The economic review did not identify any published analyses of the cost effectiveness of golimumab in AS. The manufacturer submitted a de novo economic model that compares: - golimumab (Simponi) 50 mg given once a month, on the same date each month against three different treatments: - adalimumab (40 mg adalimumab administered every other week as a single dose), - etanercept (25 mg administered twice weekly, or 50 mg administered once weekly), - conventional therapy (non-biologic DMARDs, NSAIDs, Cox-2 inhibitors, and physiotherapy). In the base-case model, a decision is made to continue or withdraw from TNF-α inhibitors according to probability of response defined as 50% improvement in BASDAI at 12 weeks. After the initial decision tree, patients enter a Markov model with a cycle length of 12 weeks and a time horizon in the base case of 20 years (maximum 60.1 years (up to age 100)). If patients are on TNF-α inhibitors, they either stay on therapy ( On treatment ), or discontinue due to lack of efficacy or adverse events ( Not on TNF-α ). To model the lower disease activity just after discontinuation of TNF-α inhibitor therapy two 12-week tunnel states ( Just discontinued and Discontinued ) were incorporated in the model. Patients in the conventional treatment arm enter the Markov model in the Not on TNF-α state. Patients can die at any point in the model. SAEs and injection site reactions of TNF-α inhibitors treatments are included in the model. The base-case analysis employed response rates from a MTC and sourced a single TNF-α inhibitor discontinuation rate from the literature. AE rates came from the GO-RAISE trial, assuming a single TNF-α inhibitor rate. Disease progression was modelled using BASDAI and BASFI scores, the baseline value coming from the GO-RAISE data and, in the base case assuming BASDAI did not change and BASFI increased at one rate for conventional and 50% of this rate on TNF-α inhibitors. QALY gain from the use of anti-tnf agents was derived using a regression model that related utility to BASFI and BASDAI scores, age and sex. Short term (first 12 weeks) management costs were informed by expert opinion (four key opinion leaders ) on resource use and long term management cost came from a regression model estimated for a previous NICE technology appraisal (TA143), also assuming a single TNF-α value. Unit costs were obtained from standard NHS cost databases. In the base case analysis the costs and QALYs of golimumab were comparable to those of the other TNF-α inhibitors. The ICER of golimumab versus conventional care is 26,597. Adalimumab and etanercept are extended dominated by golimumab. The manufacturer concluded that golimumab is a 8

9 cost-effective treatment option, similar to currently approved biologic therapies, although the uncertainty surrounding the outcomes was substantial. 1.4 ERG commentary on the robustness of evidence submitted by the manufacturer Strengths Search methods were clearly presented and reported. The MS searched the required databases, with the exception of Econlit, which was not searched due to lack of access. Date of searching, date spans and database hosts were, for the most part, well documented. The MS provided sufficient detail for the ERG to reproduce the searches. The evidence for clinical effectiveness of golimumab comes from a well conducted randomised trial and the evidence used for comparison with other anti-tnfs also derives from randomised trials. The MTCs appear to be an appropriate approach to synthesising the available evidence and appear to have been properly undertaken. The GO-RAISE trial was well designed and thus provided a potentially strong evidence base for an economic evaluation of golimumab in the management of AS and the MTC was carried out correctly and provided appropriate data on the relative risks of response to treatment in the short term. The CEA model was constructed with utility and long term cost estimates as used in TA Weaknesses All searches would have benefitted from the inclusion of more comprehensive disease and treatment synonyms. Attempts to limit searches by date or to omit animal studies were poorly applied. Attempts to limit by methodological study design were also poorly executed. The MS did not include infliximab as a comparator treatment; therefore the ERG was unclear why all searches included infliximab terms. The ERG was concerned that the date limits applied to the adverse events searches would have impaired recall of relevant reports of AEs in Golimumab. Although NSAIDS were identified as a comparator treatment, no attempt was made by the MS to identify trials of NSAIDS in AS. Several strategies included consequential errors which the manufacturer was made aware of by the previous ERG report. 1 The ERG concludes that the MS made no attempt to correct these errors whilst preparing this latest submission. Additional mixed treatment comparison searches were included in the manufacturer s response to the ERG clarification letter. 2 These searches were poorly reported, inadequately documented and failed to meet the searching specifications requested by NICE. 3 9

10 Only a single trial provided evidence for the clinical effectiveness of golimumab. Because of the unlicensed dose regimen in one arm of this trial and the early escape to cross over treatments at 16 weeks for 14-week non-responders, the totality of evidence for the comparison of the licensed dose regimen versus placebo depends on measures made at 14 weeks for only 78patients receiving placebo and 137 receiving the 50mg dose regimen. The evidence presented regarding the maintenance of clinical response to the 50mg dose regimen beyond 14 weeks was deemed unreliable because of the large numbers of patients crossing over. No head-to-head evidence is available for anti-tnfs; therefore, only indirect evidence is available. The ERG considered the use of a 20 year time horizon and literature instead of the results of the MTC for the discontinuation rate and the SAE rate arbitrary. Lifetime was specified in the decision problem and the assumption that the TNF-α inhibitor treatments are comparable in terms of discontinuations and AE rate is not in line with the evidence from the trials and MTC. Based on rates for discontinuation and AEs from the MTC and model corrections made, golimumab is less effective than the other two TNF-α inhibitors. Unfortunately, the current model structure does not allow the evaluation of sequential use of TNF-α inhibitors. As a result the impact on the ICER of the use of golimumab as second-line therapy is unknown. The approach to estimating the short term costs of management by relying on the views of four clinical Key Opinion Leaders as to resource quantities did not represent best practice and was not adequately justified. The validation of the model was very limited, and did not involve any technical component: the ERG discovered various errors in the model. Although the ERG did its best to find these errors and all errors were corrected in the ERG s own analyses, this still leaves some question marks with respect to the integrity of the model Areas of uncertainty The MTC point estimates comparing the effectiveness of golimumab with that of etanercept and of adalimumab were associated with considerable uncertainty. The model structure did not allow the assessment of the value of the sequential use of golimumab and there appears to be little evidence of the effectiveness of TNF-α inhibitors used as second-line therapy for AS. Disease progression with TNF-α inhibitors was informed entirely by an assumption of the relative rate in comparison to conventional therapy. 10

11 The validity of the cost inputs for conventional care and long-term costs based on BASFI and a survey Superseded mathematical flaws the ERG has serious concerns about the internal validity of the model. see among four Key Opinion Leaders could be questioned. The ERG discovered several modelling / data incorporation errors in the model. Considering these 1.5 Key issues There were several consequential errors and potential weaknesses in the MS search strategies. These problems may have lead to relevant studies being missed. The clinical evidence updated in the form of the included trials had inclusion criteria that were wider than the scope, in particular not being limited by severity or adequacy of response to conventional therapy. Three out of eight trials included patients with disease severity similar to the scope (GO-RAISE, ATLAS and Brandt). None of the studies fulfilled the criterion of two or more failed NSAIDs. However, three studies included patients who had failed at least page one NSAID (GO-RAISE, ATLAS and Can AS).*The ERG considered the use of a 20 year time horizon and literature instead of the results of the MTC for the discontinuation rate and the SAE rate arbitrary. Lifetime was specified in the decision problem and the assumption that the TNF-α inhibitor treatments are comparable in terms of discontinuations and AE rate is not in line with the evidence. Based on rates for discontinuation and AEs from the MTC and with model corrections made, golimumab is less effective than the other two TNF-α inhibitor treatments and would not be cost effective at any WTP threshold. This does not preclude there being value in the use of golimumab as second-line therapy. The current model structure does however not allow the evaluation of sequential use of golimumab. Additional analysis the ERG undertook did not substantially alter the conclusion of the above described analysis: in none of the additional analyses was golimumab the preferred strategy. In fact, golimumab was mostly found to be extendedly dominated by one or both of the other TNF-α inhibitor treatments. 11

12 2 BACKGROUND 2.1 Critique of manufacturer s description of underlying health problem. Does the ERG believe that the manufacturer s description of the underlying health problem is appropriate and relevant to the decision problem under consideration? The ERG felt that the MS portrays ankylosing spondylitis (AS) in a pessimistic fashion. Below, the ERG outlines an alternative description of the disease which it believes is a more balanced representation and better reflects the heterogeneity of the disease. Ankylosing spondylitis is a chronic inflammatory disease that primarily affects the sacroiliac joints and spine, and less commonly the peripheral joints. AS typically presents during the third decade of life, beginning with lower back pain which progresses to stiffness and may ultimately lead to fusion of the vertebrae. While the hips and knees may also be affected, AS may also present as periodic eye inflammation (uveitis) and is also associated with inflammatory bowel disease (e.g. Crohn s disease). While AS patients may suffer symptoms throughout their life, periods of remission, active inflammation and lessening of symptoms are not uncommon. Diagnosing AS frequently involves long delays of 5-10 years. 4, 5 Dependence on radiographic criteria may be one factor in explaining these delays but this is complicated by the range of conditions which may present with stiffness and pain in the lower back. Confounding these challenges is a lack of awareness of AS among non-rheumatologists. 6, 7 Recent developments in magnetic resonance imaging and ultrasonography can now provide early diagnosis in the absence of radiographic sacroiliitis as well as providing objective information on the degree of inflammation and response to treatment as well as providing therapeutic guidance during disease course. 8 Data on the prognosis of AS is limited. It is an unpredictable disease without a single linear natural history, with periods of relapse and remission possible at any stage. 9 The clinical manifestations of AS range from stiffness through to a totally fused spine, with the potential for peripheral arthritis or bilateral hip involvement. While many outcomes are possible, observational studies indicate that the disease pattern of AS emerges within the first 10 years. 10, 11 In a retrospective study covering a 23 year time period, almost three-quarters of patients with mild stiffness after 10 years did not progress to severe spinal involvement. In contrast 81% of patients presenting with severe restriction were severely restricted within the first 10 years. 10 Similarly, Rudwaleit et al 12 found 37% of early AS patients to have peripheral arthritis, yet this is relatively uncommon across the whole of AS. While severe spinal restriction occurs in around 40% of patients and about 1 in 5 suffer severe deformity, a longitudinal study has found a majority maintain good function and many report no pain

13 Ankylosing spondylitis exposes patients to higher risk of vertebral fracture, but not the fracture of other bones. In contrast to the MS which cited a hazard ratio for the risk of vertebral fracture at 7.6 (95% CI: ), recent data from a large observational study 14 found the risk to be far less odds ratio 3.3 (95% CI: ). The disease tends to affect individuals differently. Women tend to develop a milder experience of the disease compared to men. Over time, AS may affect different organs of the body: eyes, heart, skin, bones, and lungs. At least 3 in 20 AS patients will develop psoriasis and approximately 30 to 40% of AS patients develop iritis (inflammation of cornea and the lens). The figures cited in the MS for other conditions associated with AS (psoriasis, uveitis and inflammatory bowel disease) reflect those reported in other studies. There is an association between AS and cardiovascular risk (CV). However, a higher risk is in individuals with Rheumatoid arthritis than AS. 15 The variation in clinical symptoms should be noted. Some patients may experience persistent symptoms while others may have mild symptoms and long periods of remission between relapses Critique of manufacturer s overview of current service provision Does the ERG believe that the manufacturer s overview of current service provision is appropriate and relevant to the decision problem under consideration? The MS estimates for the potential number of patients eligible for golimumab are reasonable and based upon adequate prevalence data. The MS cites up-to-date guidelines for the assessment and management of AS based on: NICE TA 143; Assessment in Ankylosing Spondylitis (ASAS) International working group and the European League against Rheumatism (EULAR); and The British Society for Rheumatology. Suspected or confirmed cases of AS are typically referred to a rheumatologist. Physiotherapy and exercise can be effective 16 and are often recommended as preliminary therapies to aid patients in remaining mobile. Voluntary organisations, in particular the National Ankylosing Spondylitis Society play an important role in providing support and resources for AS patients. Second line therapies such as sulfasalazine (SSZ) and methotrexate (MTX) have largely been 17, 18 ineffective when applied to AS. As such, conventional therapy relies on non-steroidal antiinflammatory drugs (NSAIDs), analgesics and corticosteroids which, in combination, provide some symptom relief for the majority of patients. With regard to anti-tnfs, adalimumab and etanercept are the most appropriate comparators as, like golimumab, they are administered subcutaneously. 13

14 Population The population eligible for golimumab is appropriately based on current NICE guidance for use of anti-tnfs for AS, which estimated it to be 82,830 in Another way of estimating the population size is to use the number who currently receive one of the comparators, adalimumab or etanercept. A figure of just over 70,000 can be arrived at by using the number of prescriptions per year from the Prescription Cost Analysis 2009 data and assuming 26 per year (every other week) for adalimumab and 52 per year (every week) for etanercept. The industry submission excludes the possibility of sequential Anti-TNFs for AS. Such use of anti-tnfs may occur in clinical practice in the face of drug toxicity or poor clinical response particularly as these drugs, particularly etanercept, have different mechanisms of action. 14

15 3 Critique of manufacturer s definition of decision problem Table 1: Statement of the decision problem (as presented by the manufacturer) Population Final scope issued by NICE Adults with severe active ankylosing spondylitis whose response to conventional therapy has been inadequate. Decision problem addressed in the submission Adults with severe active ankylosing spondylitis whose response to conventional therapy has been inadequate. Intervention Golimumab Golimumab N/A Comparator(s) Adalimumab Etanercept Management without TNF á inhibitors Adalimumab Etanercept Management without TNF á inhibitors Outcomes Economic analysis Other considerations The outcome measures to be considered include: Disease activity Functional capacity Disease progression Adverse effects of treatment Health related quality of life The reference case stipulates that the cost effectiveness of treatments should be expressed in terms of incremental cost per quality adjusted life year. The reference case stipulates that the time horizon for estimating clinical and cost effectiveness should be sufficiently long to reflect any differences in costs or outcomes between the technologies being compared. Costs will be considered from an NHS and Personal Social Services perspective. Guidance will only be issued in accordance with the marketing authorisation. If evidence allows, the appraisal should consider the sequential use of TNF á inhibitors The outcome measures that will be included are: Disease activity Disease progression Adverse effects of treatment Health related quality of life Cost effectiveness of treatments will be expressed in terms of incremental cost per quality adjusted life year. Time horizon considered will be lifetime of the patient. Costs will be considered from an NHS and Personal Social Services perspective. The patient population included in the submission will be in accordance with the marketing authorisation. The evidence for sequential use of TNF á inhibitors in AS is limited and there is no evidence for golimumab in sequential setting. This will therefore not be included in the submission. Rationale if different from the scope N/A No information regarding the functional capacity is available for golimumab and therefore will not be included in the submission. N/A N/A 15

16 3.1 Population To what extent does the clinical evidence submitted by the manufacturer match the patient population described in the final scope? Where there is a mismatch, provide further details. Does the clinical evidence submitted by the manufacturer reflect the characteristics of the patient population in England and Wales eligible for treatment? If not, provide further comment. There was no mismatch between the scope and the decision problem and the population of severe, active ankylosing spondylitis in adults who have responded inadequately to conventional. The included trials however, had inclusion criteria that were broader than this. In particular, they were not limited by severity or adequacy of response to conventional therapy. Only three (GO-RAISE, ATLAS and Brandt) of the eight included trials had inclusion criteria that were of similar severity of disease and only three trials (GO-RAISE, ATLAS and Can AS) specified inadequate response to NSAIDs and in each of these to only one NSAID, rather than the two stipulated in the NICE guidance (NICE TA143) 19. Only two trials (GO-RAISE and ATLAS) had both similar disease severity and inadequate response criteria. However, for the main inputs to the CEA model, mostly the results of only these two trials (GO-RAISE and ATLAS) were actually used; exceptions were: Treatment responder relative risks of BASDAI50, at week 12 were obtained for etanercept vs placebo from two other trials, Brandt 20 and Van der Heijde 21. Discontinuation and mortality rates were not taken from the trials. AE rates and disease progression (BASDAI and BASFI change) were taken only from the GO-RAISE trial in the base case. 3.2 Intervention Does the intervention described in the MS match the intervention described in the final scope? What is the technology and what is its relevant or proposed marketing authorisation/ CE mark? There was no mismatch between scope and decision problem and only data from the trial arm with the licensed dose of 50mg (as opposed to 100mg in the other active treatment arm) from the GO-RAISE trial was used. 16

17 3.3 Comparators Do the comparators described in the MS match the comparators described in the final scope? If not, provide further details. Where evidence is limited or not available for relevant comparators has the manufacturer asked an unbiased clinical panel, or carried out its own survey, and do the views elicited agree with what the clinical advisors to the ERG advocate? There was no mismatch between scope and decision problem. However, one comparator was conventional therapy, which is a variable mixture of NSAIDs and physiotherapy, but the population was defined as patients whose response to conventional therapy was inadequate. This therefore requires both a clear definition of conventional therapy and that the outcomes of that conventional therapy that go into the CEA model should correspond to inadequate response. As stated in section 3.1, only three studies provided data from patients who demonstrated inadequate response to such conventional therapy (one NSAID) whereas the percentage of responders BASDAI50 (18.01%) for conventional therapy in the CEA model was estimated using the MTC meta-analysis from studies that included Brandt and Van der Heijde, which had not demonstrated inadequacy. Indeed, the descriptions of conventional therapy (used by at least some, but not necessarily all patients) varied between the four trials (GO-RAISE, ATLAS, Brandt and Van der Heijde) used to estimate BASDAI50: NSAIDs were stated in all except ATLAS. Corticosteroids were stated in only ATLAS and Van der Heijde, although the response to clarification letter stated in Table 15 that prednisolone was used by some patients in the GO- RAISE trial. The specific DMARDS, Methotrexate, Sulfasalazine and Hydroxychloroquine were stated in only GO-RAISE and ATLAS Brandt simply used the term DMARDS and Van der Heijde used the term antirheumatic drugs. By contrast, Table 14 of the response to clarification letter shows that conventional therapy in the CEA model was defined as a set of percentages of apparently all possible drugs, physiotherapy and patient education. Note that the figures are all less than 1, suggesting that they have been mistakenly presented as proportions: multiplying by 100 converts them to percentages. The manufacturer goes on to say, based on a comparison of Tables 14 and 15 (the GO-RAISE trial data on actual use), that the conventional care arm in the model is compatible with the placebo arm from the GO-RAISE trial. (p.27). However, there are several differences between Table 14 and 15 in percentage use of treatments. For example, the percentage use of NSAIDs in the placebo arm is 92.3, which compares 17

18 to only 56 in the model definition (calculated by summing the percentages for diclofenac, naproxen, indomethacin and ibrupofen). In summary, it is difficult to be sure that conventional therapy as represented by the placebo arms in each trial is either close enough between trials or similar enough to actual patient experience. This does raise doubts as to the validity of the cost and effectiveness estimates of all comparators in the model since they depend on the results of the MTC, which is informed by the trial results. 3.4 Outcomes Do the outcomes in the MS match the outcomes described in the final scope? If not, provide further details. Consider clinical effectiveness, adverse events, quality of life and health economic outcomes and a discussion of appropriate mechanisms for measuring these outcomes. Is the focus of the submission on appropriate outcomes or has it been limited to non-ideal outcomes? There is a mismatch between scope and decision problem in that functional capacity was omitted from the decision problem. The reason given was that no information for golimumab was available. This does not seem to be the case in that one outcome measure reported in trials was BASFI, which is intended to measure function. This outcome was reported in four trials, GO-RAISE, ATLAS, Brandt and Gorman. No relative treatment effect on BASFI was used in the model. The only difference between treatments in effectiveness was estimated using BASDAI50 the proportion of patients experiencing at least a 50% improvement on this measure. There was a sound basis for choosing this measure as it was stated to have been recommended by the Assessments in Ankylosing Spondylitis (ASAS) Working Group. Also, utility in the model was estimated using a regression equation that included terms for BASFI and BASDAI and this model had been used in a previous NICE submission on AS Other relevant factors For example: Does the MS include a section on equity considerations? Is there an ongoing Patient Access Scheme application? The scope specified that sequential treatment that is, treatment switching between the various TNFalphas on failure or discontinuation should be considered, but only if evidence permitted. The company stated that there was insufficient evidence, but in response to the clarification letter they attempted to test for the effect of switching by halving the discontinuation rate. Although this is a poor proxy for switching it would have been too difficult to adapt the model to incorporate switching and the evidence to inform effectiveness after the switch would have probably been lacking. However, assuming some effectiveness after the switch, it is possible that, even if golimumab was not 18

19 cost effective as a first line TNF-alpha, it might have been cost effective as part of a sequence. Unfortunately this is not investigated? The company proposed a Patient Access Scheme (PAS) that would avoid any additional administrative burden or cost over and above that needed to provide a patient with the 50 mg dose. The PAS only applies to those NHS patients who weigh more than 100 kg and whose disease does not show an adequate clinical response after three or four doses of golimumab 50 mg. However, as the additional cost of providing the 100 mg dose for the patients in the GO-RAISE trial that used the early escape option was not included in the model, nor was there an indication of the weight of the early escapers, the ERG concludes that introducing the proposed PAS would not improve the base-case cost effectiveness. A complete evaluation of the consequences of the PAS for cost effectiveness of golimumab is included in section

20 4 CLINICAL EFFECTIVENESS 4.1 Critique of the methods used by the manufacturer to systematically review clinical effectiveness evidence The MS reports a systematic review of the comparative clinical efficacy of adalimumab, etanercept, and golimumab for the treatment of AS State objective of systematic review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate? The MS described searches undertaken to identify relevant clinical effectiveness, indirect and mixed treatment comparisons and adverse events literature for golimumab, plus comparators, for the treatment of ankylosing spondylitis. Full details of strategies were included in the appendices to the MS. Overall the search strategies used for each of the sections of the submission were well-documented. A detailed commentary on the individual searches is provided below. There were several consequential errors and potential weaknesses in the MS strategies. These problems may have led to relevant studies being missed. An amended version of the PRESS evidence-based checklist 24 was used to inform the assessment of the quality of all search strategies presented in the manufacturer s submissions. The submission was checked against the Single technology Appraisal (STA) specification for manufacturer/sponsor submission of evidence Search for clinical evidence Search strategy for section 5.1, clinical evidence Section 8.1 of the MS stated that Medline and Embase were searched. In this section, no mention was made of the other required databases, In-Process and the Cochrane Library. However, section 9.2 Appendix 2: Search strategy for section 5.1, stated that all required databases (Medline, In-Process, Embase& Cochrane Library) were searched and strategies were presented for all these resources. Medline and In-Process were searched using the OvidSP database host; the host for Embase was not stated. According to Section (page 37), the search was intended to retrieve randomised controlled trials (RCTs) of adalimumab, etanercept and golimumab. The search strategies for Medline and Embase presented in Appendix 2, (page ), also included infliximab as a search term. As 20

21 infliximab was not considered a comparator (page 35), inclusion of infliximab in the search strategy was redundant and would retrieve unnecessary references. According to the inclusion criteria (detailed in Table 9. Eligibility criteria used in search strategy), Conventional therapy which included NSAIDs and DMARDs was also included as a comparator. There is no indication that NSAIDs or other DMARDs were included in the search process. No relevant terms for these interventions are given in the search strategies; therefore the ERG concludes relevant reference to these comparator interventions may not have been identified in the MS searches. Appendix 2 (page 209), stated that the clinical effectiveness searches were an update of searches originally undertaken by McLeod et al. 22 However, comparison between the strategies in the MS appendices and the strategies reported by McLeod, show that very different strategies were employed. McLeod s search did not include an RCT search filter. McLeod also employed disease synonyms which were omitted from the MS search e.g. Bekhterev Bechterew mariestrumpell spondylarthiritis [sic] rheumatoid spondylitis Although both the McLeod search and the MS search included etanercept, infliximab, adalimumab and golimumab, the MS included more synonyms for the drug names. The MS search incorporated the Cochrane Highly Sensitive Search Strategy Medline RCT filter (sensitivity maximising). 25 The Medline strategy (page ) contained several consequential errors. Firstly, line 3 of the HSSS RCT filter should have read randomized.ab., but this had been incorrectly entered with the UK spelling variation randomised.ab.. As the Cochrane RCT filters were derived using word-frequency analysis, it was anticipated that changing spelling of included terms could have some impact on sensitivity and specificity. Secondly, the most consequential error of the search strategy was in line 11. Instead of combining the preceding 10 lines to remove animal studies from the RCT reports, the line switched from specified fields and MeSH terms to all searchable fields (.af.). This error rendered lines 1-10 redundant and increased on the number of records retrieved by this facet of the search strategy. The ERG noticed that similar errors in the use of study design filters were also made in Cost-effectiveness searches (9.10.4) The MS strategy should have read (total retrieved = 244): 1 randomized controlled trial.pt. (302107) 2 controlled clinical trial.pt. (81976) 3 randomized.ab. (218231) 4 placebo.ab. (126284) 5 drug therapy.fs. ( ) 21

22 6 randomly.ab. (160980) 7 trial.ab. (225416) 8 groups.ab. ( ) 9 or/1-8 ( ) 10 animals/ not (animals/ and humans/) ( ) 11 9 not 10 ( ) Instead the MS strategy read (total retrieved = 328): 1 randomized controlled trial.pt. (302107) 2 controlled clinical trial.pt. (81976) 3 randomised.ab. (42710) 4 placebo.ab. (126284) 5 drug therapy.fs. ( ) 6 randomly.ab. (160980) 7 trial.ab. (225416) 8 groups.ab. ( ) 9 or/1-8 ( ) 10 animals/ not (animals/ and humans/) ( ) 11 ((randomised controlled trial or controlled clinical trial or randomised or placebo or drug therapy or randomly or trial or groups) not (animals not (humans and animals))).af. ( ) The third consequential error in MS Medline RCT strategy appeared in line 13 (spondil$ adj2 ankylos$).ab,ti. Misspelling spondil$ would fail to retrieve spondylitis variants. The ERG queried this error in Section A1 of the Clarification Letter. The manufacturer responded that this minor error would have ultimately missed 7 golimumab references when combined with the RCT filter. The Manufacturer s response referenced the previous ERG report produced by Connock. 1 However it was clear from comparison of the Manufacturer s response to the KSR ERG Clarification, and the Connock report, that the manufacturer paraphrased Connock s comments on this strategy error. Connock et al. tested the impact of this spelling error on strategy recall in September 2010 (p. 20, 116). 1 The numbers presented in the Clarification response by the manufacturer were given by Connock; therefore they did not represent the numbers of records potentially missed by the manufacturer s latest searches which were undertaken on This spelling error was reproduced in the Embase search strategy, but the correct spelling appears in the CENTRAL translation of the search. The spelling error was also identified in many of the MS strategies, and has been detailed in the following critique. The ERG corrected all 3 errors in the Manufacturer s Medline strategy and re-ran the search. The amended MS search retrieved 244 records, compared to the version with errors (n=328). Although the total retrieved was less than the original MS search, it was expected that more relevant comparator references would have been retrieved. The overall total was less due to removal of the infliximab terms (date of ERG searching: ). 22

23 The ERG identified several additional synonyms for both the AS disease facet and alternative drug names. The ERG re-ran an amended version of the Medline strategy incorporating these additional terms in combination with CAS Drug Registry numbers. The ERG Medline search retrieved 269 records, 25 more than the corrected MS search. Some of these additional references could be relevant for inclusion. Unfortunately the ERG was not able to screen the additional search results due to time constraints. For full ERG search strategies and corrected MS search strategies, please see Appendix 2: ERG search strategies. The date of manufacturer s searching was reported as Although the date of searching and the date limits were given for the Medline search, the specific date span for the Medline search was not given (e.g /03/week 3). The date span is presented for the Embase search ( /week 17). The issue number for the CENTRAL search is not reported. The searches were intended to act as an update to a previous HTA report. 22 Therefore, searches were undertaken for the date span 2005/ /05; with the exception of the CENTRAL search which did not include a date limit. All searches included in the MS were reproduced by the ERG. As the ERG re-ran searches several months after the manufacturer prepared the submission, the numbers retrieved were higher than reported in the MS (date of ERG searching: ). The ERG identified and corrected several errors in the search strategies, which resulted in different numbers of records retrieved. The Embase RCT search strategy reported in the MS appears adequate. The strategy employed an appropriate RCT filter for the database, 26 designed to maximise sensitivity. Although a limit to remove animal studies was applied to the Medline search, no such limit was incorporated into the Embase search strategy. The drug text terms in the Medline strategy searched title, abstract and CAS registry number fields. The drug text terms in the Embase strategy were restricted to title and abstract. The Emtree subject terms for the drugs were restricted to focus. This seemed unnecessarily restrictive. The ERG re-ran the MS search which retrieved 460 references (date of ERG search: ). The ERG then made minor amendments to the fields search and the Emtree restrictions; the revised MS search retrieved 684 references. This identified 224 references which were previously missed. As with the Medline search, the ERG identified several additional synonyms for both the AS disease facet and alternative drug names. An amended ERG version of the Embase strategy incorporating these additional terms in combination with CAS Drug Registry numbers retrieved 686 records, only 2 more than the revised MS search and 226 more than the original MS search. Some of these additional references could be relevant for inclusion (these references were not screened due to ERG time constraints). For full ERG search strategies and corrected MS search strategies, please see Appendix 2: ERG search strategies. 23

24 The flow diagram relating to the clinical effectiveness literature searches (Figure 2, page 39) did not conform exactly to the PRISMA statement flow diagram ( as the following details were missing: i) number of records identified though database searching, ii) number of additional records identified through other sources, iii) number of records after duplicates removed. In addition, the numbers did not add up in the flow-diagram. In section 5.2.2, Figure 2: Study selection flow diagram, the number of records retrieved by the Medline &Embase searches was given as 82 references. From these 82 references, 75 were excluded as irrelevant. The flowchart then stated that the remaining 10 references from the Medline and Embase searches were retrieved for more detailed evaluation. There must be an error in this flowchart, as only 7 references would remain after excluding 75 irrelevant records from an initial set of 82 results. Apparently from the remaining 10 records, all 10 were excluded. The flowchart shows that 8 references remained for analysis. Search strategy for section 5.8, indirect/mixed treatment comparison Although the submission stated that the search strategy for this section was identical to the strategy for the direct comparisons (5.1), the manufacturer s response to the ERG clarification letter 2 presented additional searches undertaken to identify RCTs for the mixed treatment comparison. The additional searches did not meet the search requirements set out in the Specification for manufacturer/sponsor submission of evidence (STA). 3 The summarised strategies presented in Appendix 1 of the response to the clarification letter did not include date searched. Medline and Embase were searched to update previous work undertaken by McLeod 22, the date span for the MS searches was given as 2005/11 to 2008/08. The search strategies were documented in less detail than the other strategies included in the MS. The extra MTC strategies were reported in a summarised tabular format; the whole strategy was not given, several limits were reported as being applied but the exact search syntax was not provided. The disease search terms reproduced the McLeod searches more closely than any other search detailed in the MS, including terms such as bekhterev and mariestrumpell. The database hosts were not reported. Searches were undertaken on Medline and Embase, but it was unclear whether the other required databases, In-Process or the Cochrane Library, were also searched. Several limits were applied to the searches; limits to include only human studies, English language trials, and RCTs, in addition to a date limit. Full search strategies were not presented and no detail was given on the type of RCT study design filter, therefore it was not possible for the ERG to replicate these searches. For this reason, the ERG was unable to verify the number of records retrieved. Application of an English language limit to the MTC search strategies might potentially introduce language bias into the results of the search

25 Search strategy for section 5.9, non-rct evidence The MS stated that no non-rct evidence was used (page 128). On page 228 in the Search Strategy Section for 5.8 (Non-RCT evidence), the MS stated that This information was expected to be available from the search strategy in Appendix 3 (section 9.3). The ERG assumes there was an error in this sentence as Appendix 3 (Section 9.3) relates to the quality assessment of RCTs. Perhaps this should have read Appendix 3 (section 9.2) which referred to the search methods for the clinical effectiveness searches. If this was the case, the clinical effectiveness searches were restricted to RCTs only, and therefore would not have been an effective source of non-rct evidence. Additional searching The MS reports that in addition to literature searching, references in retrieved articles and relevant systematic reviews were also checked. Search strategy for section 5.10, adverse events The MS stated that there are no trials designed to assess the safety outcomes of the interventions discussed (page 129). The manufacturer undertook searches to identify references to adverse effects literature for golimumab and the comparator drugs, adalimumab and etanercept. The strategies and search details were listed in Appendix 8 (pages ). The date of searching was given as and Section described how the searches were conducted as an update to work undertaken by Rodgers in For this reason, the MS stated that date limits were applied to the aspects of the search already covered by the Rodgers searches, but that the golimumab searches were conducted for the entire date period. The date span was clearly reported for Embase, however the Medline/In-Process search did not provide an end date e.g /05/wk 1. Strategies were documented for Medline/In-Process (OvidSP) and Embase. The database host was not reported for Embase, however the ERG assumed the host was also OvidSP due to the syntax and appearance of the search. The amended PRESS checklist was used to appraise the strategies and to identify any areas of concern, and the MS searches were compared to the adverse events (AE) searches documented in the appendices of Rodgers. 28 As with the clinical effectiveness searches, infliximab terms were included in the adverse events searches, even though infliximab was not considered to be a comparator treatment. This would result in irrelevant references being retrieved. The manufacturer failed to carry out a true update of Rodgers searches for several reasons. The Rodgers Medline search employed several synonyms for adalimumab which were not included in the MS search. The MS had an error on line 9; instead of treatment emergent the strategy searched for 25

26 emergency treatment. This would have failed to pick up references to treatment emergent adverse events, and would have potentially picked up less relevant references to emergency treatment. There was a typographical error in line 17 of the MS strategy, which read undesire$ instead of undesir$. This mistake would have failed to retrieve references to undesirable side-effects or adverse events. The Rodgers search strategy included a limit to remove animal studies which was not included in the MS Medline search. There were two further issues concerning the MS Medline AE search. There was a typographical error in line 4. The principal drug under investigation, golimumab, was misspelt as gomimumab. As this drug does not yet have a MeSH indexing term applied, this mistake could have impacted on the recall of relevant reports of adverse events in golimumab. There was a final discrepancy between the intention of the search and reported strategy. In order to act as an update to Rodgers search, the MS search was limited to retrieve studies added to Medline since In line 34 of the MS Medline strategy, the date limit was applied to all results, including golimumab references. This would have potentially failed to retrieve reports of adverse events in golimumab treatment prior to The ERG reproduced the MS search and then re-ran a revised version after correcting the above errors (date of ERG searching: ). The original MS search included infliximab terms and retrieved 26 references. The ERG revised the above errors and re-ran the Medline search. The revised strategy retrieved 22 references. Even though the total retrieved was less than the original MS search, it was expected that more relevant comparator references would be retrieved. The overall total was less due to removal of the infliximab terms (date of ERG searching: ). Similar problems were observed with the MS AE Embase search strategy. Infliximab was included in the strategy (lines 2 and 26), despite being excluded from the comparators list. Treatment emergent was substituted by emergency treatment (line 8). The Rodgers strategy included more synonyms for adalimumab (line 3, 28 ), and additional Emtree terms for the comparator drugs (lines 5-7). The inclusion of these additional Emtree terms made Rodgers search much more sensitive to all the target drugs. This increased sensitivity would potentially result in more references being retrieved. There was a typographical error in line 18 of the MS search; truncation appears within the Emtree term drug surveillance program$/. This did not appear to affect recall or performance of the Emtree thesaurus. There were discrepancies between the Rodgers search and the MS Embase search, such as the MS strategy did not restrict the included Emtree drug terms to focus. This might have slightly increased the number of irrelevant records retrieved. The final two issues related to the way in which the date limit had been applied in lines 40 and 41. As with the MS Medline strategy, all the search results had been limited to 2009 onwards, which meant that prior reports of AEs in golimumab would not be 26

27 retrieved. The MS strategy used the field tag.dp. (date of publication) to limit the search; this would have retrieved reference published from 2009 onwards. A more appropriate way to update an existing search would have been to limit by the date the references were added to Embase, such as entry week (.em.). Using entry week would have been more in-line with the date limit employed in the Medline search (entry date or.ed.); after all, these searches were intend to identify references added to Medline and Embase since Rodgers searches were undertaken. The ERG reproduced the MS search and then re-ran a revised version after correcting the above errors (date of ERG searching: ). The original MS search included infliximab terms and was limited by date of publication; the strategy retrieved 74 references. The ERG revised the above errors so that the search resembled the Rodgers search more closely. The revised strategy used entry week as the date limiter, and retrieved 262 references. The revised search retrieved 188 more references than the original MS strategy, despite the removal of the infliximab terms. The ERG assumed that these extra references may have included potentially relevant reports of adverse events of golimumab and the comparator treatments. For revised MS search strategies, please see Appendix 2: ERG search strategies Search strategy for cost-effectiveness Search strategy for section 6.1, cost-effectiveness The MS presented searches undertaken to identify cost-effectiveness studies of golimumab, adalimumab or etancercept in ankylosing spondylitis. The required databases, Medline, In-Process, Embase and NHS EED were searched. EconLit was not searched due to lack of access to the resource. The database hosts for Medline and Medline In-Process were given as Ovid; NHS EED was searched via the CRD website. The database host is not reported for Embase, however the ERG assumed the host was also OvidSP, due to the syntax and appearance of the search. The date for Embase was clearly reported; the end date for the Medline/In-Process searches was not given. Application of the modified PRESS checklist identified several areas of concern in the Medline and Embase search strategies. The Medline search strategy contained errors in all three of the main strategy facets: the disease facet, the drug facet, the economic study design filter, the date limit applied and the limit to remove animal studies. The disease text term in line 2 of the Medline economics search was spelt incorrectly, in the same way the term appeared in the Medline RCT strategy. Line 2 reads (spondil$ adj2 ankylos$).ab,ti. The ERG assumed this should have read (spondyl$ adj2 ankylos$).ab,ti. Once again, misspelling spondil$ 27

28 would fail to retrieve spondylitis variants. Infliximab terms were again included in the search strategy, even though infliximab was not included as a comparator drug (line 5). It appeared that the MS Medline search incorporated an economics search filter developed by the Centre for Reviews and Dissemination; 29 however instead of combining the economics terms in line 20 as intended, the MS strategy searched for all terms as free-text instead. This would have impacted on the precision of the study design filter, because MeSH terms, terms specified as title/abstract and the publication types were all searched for as all fields (.af.). A similar mistake was also made with the limit to remove animal studies in line 21. Instead of searching for specific MeSH terms, the animal/human terms were searched in the.mp. fields, which included title, abstract, MeSH and unique identifier. The ERG would be very cautious about excluding records where animals appeared anywhere in the title, abstract or subject headings. The conventional limit used cautious reverse logic to exclude ( NOT out) records indexed only as animal/ studies, without human/ MeSH indexing e.g. 20 NOT (animal/ not (animal/ and human/). This incorrect limit might have excluded relevant records. The ERG reproduced the MS search and then re-ran a revised version after correcting the above errors (date of ERG searching: ). The original MS search included infliximab terms and retrieved 22 references. The ERG revised the above errors and re-ran the Medline search. The revised strategy retrieved 17 references. Even though the total retrieved was less than the original MS search, it was expected that more relevant disease references would have been retrieved. The overall total was less due to removal of the infliximab terms (date of ERG searching: ). Section of the MS stated that all searches would be limited from /05/04. The Medline, In-Process and Embase searches were all limited from The NHS EED search had no date limit. The MS economics search of Embase contained similar errors to the Medline search; spondil$ was misspelled, infliximab was included as a drug term and the date limit of was applied. There are Emtree terms for all of the drugs of interest; therefore the search could be made more sensitive by including relevant indexing terms in the strategy. The Embase search also had significant problems in the way an economic study design filter was applied. As before, the MS incorporated the CRD Medline economics filter; 30 this filter includedmesh terms. Embase records are not indexed with MeSH (Medical Subject Headings); Embase indexes using the specially-developed Emtree thesaurus. Running a search with MeSH terms might retrieve records, however, the MeSH terms might not correspond to the appropriate Emtree terms, resulting in records retrieved incorrectly or potentially missing records. CRD also produced an Embase economics study design filter 30, which incorporated more appropriate Emtree terminology. Despite this observation, examination of the MS Embase strategy showed similar problems to the 28

29 Medline strategy. Instead of incorporating the economics filter terms, line 18 incorrectly searched for the filter terms as free text (.af.). An attempt was made to limit the results to exclude animal studies. As before, this was not executed correctly in line 19, as the terms were search as.mp. (title, abstract, subject headings, unique identifier). This was even more problematic in Emtree, as the same approach for Medline was applied in Embase, with no recognition that Emtree terms for human and animal are different. For example, there was no Emtree term for human/; the equivalent Emtree term might be exp human-experiment/. Instead of using the MeSH term animal/, there were several Emtree terms, such as exp animal/, exp animal-experiment/ and nonhuman/. In a similar way, the limit to remove irrelevant publication types (line 15), included historical article. This was not an Embase publication type, but appeared in Medline; therefore this limit would not have worked to exclude any Embase records. The ERG reproduced the MS search and then re-ran a revised version after correcting the above errors (date of ERG searching: ). The original MS search included infliximab terms and retrieved 48 references. The ERG re-ran the exactly same strategy on and retrieved 147 references. Although the ERG would expect a small increase in the numbers retrieved, as the searches were run almost 11 months after the MS date of searching, the ERG cannot explain such a large increase in numbers retrieved by the same strategy. The ERG revised the errors detailed in the above section, and re-ran the Medline search. The revised strategy retrieved 157 references. Even though the total retrieved is similar to the original MS search, it is expected that more relevant disease references would be retrieved (date of ERG searching: ). The MS NHS EED strategy was clearly reported, undertaken on and did not incorporate a date limit. The strategy consisted of a single MeSH term. Additional text terms could have been included, however the ERG considered this search adequate. Search strategy for section 6.4.5, Measurement and valuation of health effects The MS presented searches intended to identify reports of quality of life in ankylosing spondylitis, treated by adalimumab, golimumab and etanercept. The searches were undertaken on for Medline, In-Process and Embase. NHS EED search was searched on EconLit was not available to the manufacturer, and was not searched due to lack of access. The search strategies were reported in section Medline and In-Process were searched via the OvidSP database provider. The database host for Embase was not stated, and the ERG assumes Embase was also accessed via OvidSP. No date limits were applied for any of the searches, therefore all databases were searched from inception. 29

30 The MS Medline search included a spelling error in the disease facet; spondil$ appeared instead of spondyl$. As with many of the other MS search strategies, infliximab was included as a search term, although infliximab was not considered a comparator treatment. The Medline search included two MeSH terms only to identify references relating to measurement and valuation of health effects. Section clearly stated that the utility algorithm used EQ-5d utilities, therefore it would be reasonable to assume terms such as EQ5D, HRQOL and SF36 would be relevant terms to include in this strategy. The ERG reproduced the MS Medline search and retrieved 51 references (date of ERG searching: ). The ERG revised the error above and incorporated additional HRQL terms and re-ran the Medline search. The revised strategy retrieved 43 references. The ERG expected the slight reduction in numbers retrieved was due to the exclusion of infliximab terms. The MS reported the same strategy used on Medline was also used for Embase. Unfortunately the same errors were replicated on Embase, for example the misspelling of Spondil$ and the inclusion of infliximab. The ERG considered the lack of Emtree terms for the included drugs, etanercept, golimumab and adalimumab, potentially decreased the strategy s recall of relevant reports. Only two Emtree terms were used to retrieve HRQL reports, no text words were included. The ERG reproduced the MS Embase search and retrieved 157 references (date of ERG searching: ). The ERG revised the error above and incorporated additional HRQL terms and re-ran the Embase search. The revised strategy retrieved 204 references. Although the ERG was not able to screen the additional 47 references, it was expected that some of these references could be potentially relevant. The same NHS EED search strategy was reported in this section as for section The same comments apply: The MS NHS EED strategy was clearly reported, undertaken on and did not incorporate a date limit. The strategy consisted of a single MeSH term. Additional text terms could have been included, however the ERG considered this search adequate. Search strategy for section 6.5, Resource identification, measurement and valuation The MS reported that the cost-effectiveness searches in section 9.10 were assumed to identify all the relevant literature for this section. Therefore the same ERG comments about errors, synonyms and inappropriate inclusion of infliximab terms applied to the searches for section

31 4.1.2 State the inclusion/exclusion criteria used in the study selection and comment on whether they were appropriate. The inclusion and exclusion criteria used in the selection of evidence for the systematic review were presented in the MS (page 38-39). The table in the MS was labelled as eligibility criteria used in search strategy but was presented within the description of the study selection process (Section 5.2.1). It was not clear from the MS how many reviewers were involved in the study selection process. Best practice specifies that two reviewers be involved in the application of inclusion and exclusion criteria in order to limit bias in study selection. Details of the inclusion and exclusion criteria applied in the MS are presented in Table 2. Table 2: Inclusion/exclusion criteria used in study selection (as presented by the manufacturer) Clinical effectiveness Inclusion criteria Study design: Double blind randomised controlled trials (RCTs) Interventions: Etanercept, Adalimumab, Golimumab, Conventional therapy which included NSAIDs and DMARDs. Participants: Adults with active ankylosing spondylitis Outcomes: ASAS, BASMI, BASDAI, BASFI Safety Outcomes: Discontinuation, Injection site reactions, Severe Adverse Events (number of patients) Language restrictions No Language restrictions were applied. Exclusion criteria Study design Randomised studies were excluded if they: provided only unplanned, interim findings provided data only on a sub group of the enrolled patients were continuing to recruit patients were trials in which patient numbers treated with a specific intervention (i.e. adalimumab, etanercept or infliximab (excluded post hoc)) or disease status (i.e.active AS) could not be determined. The MS stated that only double-blind RCTs were included in the analysis. It was not explicitly stated in the MS whether both phase II and phase III clinical trials were eligible for inclusion. The ERG consider non-rcts to be a valid and important source of evidence for the evaluation of adverse events. Controlled clinical trials may exclude patients at high risk from harms, 31 may be too short in terms of follow-up to detect long-term harms, may not have sufficiently large sample sizes to detect uncommon adverse events, or may not have reported them in a consistent manner The MS stated on page 135 that, as golimumab is a new drug, no non-rcts or observational studies were available at the point of submission. However, it is possible that non-rct evidence may be available for the comparator drugs which was not identified. The interventions and comparators are the same as those mentioned in the decision problem and the NICE scope. The outcomes are those mentioned in the NICE scope except functional capacity, which was mentioned in the decision problem as not being available for golimumab. However, it is possible that functional capacity data may be available for the comparator drugs. Quality of Life was mentioned in the NICE scope and in the manufacturer s decision problem; however, it is not 31

32 mentioned in the inclusion criteria for the systematic review. In fact, QoL data were not extracted from included trials (see also section 4.2.4). The population is wider than the populations described in the decision problem and the NICE scope. In the scope the population is described as Adults with severe active ankylosing spondylitis whose response to conventional therapy has been inadequate. This is repeated for the decision problem in the MS. However, the inclusion criteria for the SR do not mention severe or an inadequate response to conventional therapy. Unfortunately, EMEA does not specify what is meant by severe. Therefore we will use the population described by NICE in the guidance for adalimumab and etanercept for adults with severe active ankylosing spondylitis. According to NICE guidance (NICE TA143) 19, adalimumab or etanercept are recommended as treatment options for adults with severe active ankylosing spondylitis only if all of the following criteria are fulfilled. The patient s disease satisfies the modified New York criteria for diagnosis of ankylosing spondylitis. There is confirmation of sustained active spinal disease, demonstrated by: a score of at least 4 units on the Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) and at least 4 cm on the 0 to 10 cm spinal pain visual analogue scale (VAS). These should both be demonstrated on two occasions at least 12 weeks apart without any change of treatment. Conventional treatment with two or more non-steroidal anti-inflammatory drugs taken sequentially at maximum tolerated or recommended dosage for 4 weeks has failed to control symptoms. Three out of 8 included trials included patients with disease severity similar to the above description (GO-RAISE, ATLAS and Brandt). None of the studies fulfilled the criterion of 2 or more failed NSAIDs. However, 3 studies included patients who had failed at least 1 NSAID (GO-RAISE, ATLAS and Can AS). 32

33 4.1.3 What studies were included in the clinical effectiveness review and what were excluded? Provide a table of identified studies. Please identify the most important clinical effectiveness studies. The flow diagram in the MS (fig. 2, page 39) shows that 8 trials were included. Seven of these trials were included in the previous NICE review (Ankylosing spondylitis - adalimumab, etanercept and infliximab). It should be noted that the numbers in the MS flow diagram (figure 2) do not add up. The diagram states that 85 potentially relevant abstracts were identified from the EMBASE and MEDLINE searches; 72 irrelevant abstracts were excluded; 10 articles were excluded, but 8 trials were included in the SR. Details of the studies and their populations as presented in the MS (Tables 10-18, page 41-82) are reproduced in Table 3. Table 3: Studies included in the systematic review. Trial Intervention/ Duration Population Age (Yrs) Male (%) Dis Dur (Yrs) Allocation Blinding* ATLAS Adalimumab 40mg (n=208) Placebo (n=107) 24 weeks Patients classified as having definite AS based on the modified New York criteria Active disease defined as fulfilment of at least 2 of the following 3 criteria: a Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) score 4, a total back pain score 4 by visual analog scale (VAS; 0 10 cm), or duration of morning stiffness of 1 hour % -- Not clear Not clear Inadequate response or intolerance to 1 or more nonsteroidal anti-inflammatory drugs (NSAIDs) Patients in whom 1 or more disease-modifying anti-rheumatic drugs (DMARDs) had failed were also allowed to participate; patients were allowed to continue any of the following medications if the dosage had remained stable for at least 4 weeks before the baseline visit: sulfasalazine ( 3 gm/day), methotrexate ( 25 mg/week), hydroxychloroquine ( 400 mg/day), or prednisone equivalent ( 10 mg/day), and NSAIDs. Canadian AS Adalimumab 40mg (n = 38) Placebo (n = 44) 24 weeks Adult patients who had AS with an inadequate response to at least one non-steroidal anti-inflammatory drugs (NSAIDs) or disease modifying anti-rheumatic drug (DMARD) (±11.0) 79% 13.3 (±8.8) Not clear Not clear Brandt et al. Etanercept 25mg (n = 14) Placebo (n=16) 6 weeks Patients who fulfilled the modified New York criteria for AS and had active disease as defined by BASDAI index of 4 and spinal pain of >4 on a 0 10 numeric rating scale 35.9 (±8.0) 79% 13.1 (±8.6) Yes Yes Calin et al. Etanercept 25mg (n = 45) Placebo (n = 39) Patients with active AS diagnosed using the modified New York criteria; active disease was diagnosed if the patient had an average score 30 for spinal inflammation 43.0 (±10.5) 79% 12.4 (±8.5) 33

34 Davis et al. Gorman et al. Van der Heijde et al. GO RAISE 12 weeks and a score 30 on at least two of the other three domains. Not clear Yes Etanercept 25mg (n = 138) Placebo (n=139) 24 weeks Etanercept 25mg(n = 20) Placebo (n = 20) 16 weeks Etanercept 50mg (n =155 ) Etanercept 25mg (n =150 ) Placebo (n = 51) 12 weeks Golimumab 50mg (n=138) Golimumab 100mg (n=140) Placebo (n=78) 24 weeks Patients meeting the modified New York criteria for AS. Patients with active AS, defined as 1) a score of 30 mm for morning stiffness (average of 2 scores on a 100 mm visual analog scale [VAS] analyzing duration or intensity of morning stiffness), and 2) scores of 30 mm for 2 of the following parameters: patient s global assessment (on a 100 mm VAS) of disease activity, back pain (average of 2 VAS scores evaluating nocturnal back pain or total back pain), and the BASFI. Adult patients with active ankylosing spondylitis based on the modified New York clinical criteria despite accepted treatments. Active spondylitis was defined as the presence of inflammatory back pain (stiffness and pain that worsened with rest and improved with exercise), morning stiffness for at least 45 minutes, and at least moderate disease activity as assessed by the patient and the physician. Acceptable medications included nonsteroidal antiinflammatory drugs (NSAIDs), oral corticosteroids (< 10 mg per day), gold injections (< 50 mg per month), methotrexate (< 20 mg per week) and sulfasalazine (< 3 g per day). Patients with active ankylosing spondylitis based on the Modified New York Criteria for ankylosing spondylitis. Active ankylosing spondylitis was defined by an average visual analogue scale (VAS) score 30 for duration and intensity of morning stiffness and two or more of the following: patient global assessment of disease activity VAS score 30; mean of nocturnal and total pain VAS scores 30; or Bath Ankylosing Spondylitis Functional Index 30 (all scores on a scale of 0 100). Concomitant oral non steroidal anti inflammatory drugs and oral corticosteroids ( 10 mg/day), if stable for 2 weeks before randomisation, and disease modifying antirheumatic drugs (hydroxychloroquine, sulfasalazine and methotrexate), if stable for 4 weeks before randomisation, were permitted. Adult patients who had AS (as per modified New York Criteria) for 3 months before the first administration of the study agent, a BASDAI score of 4 (0 10 point scale), a spinal pain assessment score of 4 on a visual analog scale (VAS; 0 10 cm scale), and an inadequate response to current or previous NSAIDs or DMARDs. Patients were required to have normal results of a chest radiograph within 3 months before randomisation and to have undergone screening for latent tuberculosis (TB) using a purified protein derivative skin test and the QuantiFERON TB Gold test. Patients in whom latent TB was discovered were required to initiate therapy for TB prior to or simultaneously with the first dose of the study agent. Dis Dur = Disease Duration; Allocation: Was the concealment of treatment allocation adequate?;blinding: Were the care providers, participants and outcome assessors blind to treatment allocation? % 10.2 Yes Yes 38.5 (±10.0) 77.5% 13.5 (±9.5) Yes Yes (Note: 90% male in placebo!) % 9.3 Not clear Not clear 39 (29-50) 71.5% 8.5 (±8.4) Yes Yes 34

35 Table 4 below, reproduces the summary of participant characteristics for the included studies (MS Tables 13-15). Several errors denoted by * or ** - were identified between the MS and original publications. For Brandt et al and the outcomes BASDAI, BASFI and BASMI, results for placebo and etanercept have been swapped. While for Davis and the outcomes Global Assessment mean, BASDAI, BASFI, and BASMI, results are reported as a mean and standard deviation when in the original publications they are Means and standard errors of the mean. Beyond these minor errors the heterogeneity across the included studies is of concern. For Global Assessment, baseline results range from 3 (Gorman) to 7 (GO-RAISE) out of 10. Similarly, for BASFI, baseline results range from 3.2 (Gorman) to 7 (Brandt) out of 10. Together with the differences in inclusion criteria, this heterogeneity across studies highlights marked differences between the study populations. Table 4: Characteristics of participants in the included studies. Trial Intervention/ Duration Global Assessment Mean ± SD BASDAI Mean ± SD BASFI Mean ± SD BASMI Mean ± SD Concomitant Therapy (I/C) % ATLAS Adalimumab 40mg (n=208) Placebo (n=107) A: 6.3 ± 2.2 P: 6.5 ± 2.0 A: 6.3 ± 1.7 P: 6.3 ± 1.7 A: 5.2 ± 2.2 P: 5.6 ± 2.2 A: 3.8 ± 2.2 P: 4.2 ± weeks Canadian AS Adalimumab 40mg (n = 38) Placebo (n = 44) A: 66.1 ± 18.9 P: 67.8 ± 19.1 A: 6.2 ± 1.7 P: 6.5 ± 1.6 A: 5.3 ± 2.0 P: 5.6 ± weeks Brandt et al. Etanercept 25mg (n = 14) Placebo (n=16) 6 weeks -- E: 6.6 ± 1.0** P: 6.5 ± 1.2 E: 6.5 ± 1.2 P: 6.6 ± 1.0 E: 5.3 ± 2.3** P: 6.2 ± 1.8 E: 6.2 ± 1.8 P: 5.3 ± 2.3 E: 3.8 ± 2.1** P: 4.1 ± 1.7 E: 4.1 ± 1.7 P: 3.8 ± Calin et al. Etanercept 25mg (n = 45) Placebo (n = 39) 12 weeks E: 65.6 P: 63.4 E: 61.0 P: 58.6 E: 60.2 P: 57.2 DMARDs 36/41% Sulphasalazine 24/28% MTX 13/13% NSAIDs 89/85% Corticosteroids 16/15% Davis et al. Etanercept 25mg (n = 138) Placebo (n=139) 24 weeks E: 62.9 ± 1.5* P: 62.9 ± 1.6 MEAN ± SEM E: 58.1 ± 1.5* P: 59.6 ± 1.4 MEAN ± SEM E: 51.7 ± 1.8* P: 56.3 ± 1.7 MEAN ± SEM -- DMARDs 32/31% Sulphasalazine 21/22% MTX 11/12% NSAIDs 91/92% Corticosteroids 13/14% Gorman et al. Etanercept 25mg(n = 20) Placebo (n = 20) E: 3.0 ± 0.7 P: 3.0 ± E: 4.5 ± 2.1 P: 3.2 ± DMARDs 40/35% Sulphasalazine -- 35

36 Van der Heijde et al. GO RAISE 16 weeks MTX -- NSAIDs 80/95% Etanercept 50mg (n =155 ) Etanercept 25mg (n =150 ) Placebo (n = 51) 12 weeks Golimumab 50mg (n=138) Golimumab 100mg (n=140) Placebo (n=78) 24 weeks E50: 68.1 ± 18.9 E25: ± 18.1 P: 66.2 ± 16.3 G50: 6.81 ± : 7.03 ± 1.88 P: 7.17 ± 1.69 E50: 62.4 ± 17.0 E25: 59.4 ± 16.7 P: 61.1 ± 13.7 G50: 6.50 ± : 6.89 ± 1.50 P: 6.61 ± 1.52 E50: 60.6 ± 20.3 E25: 57.7 ± 20.1 P: 59.7 ± 19.3 G50: 5.00 ± : 5.17 ± 2.56 P: 5.10 ± G50: 3.29 ± : 3.57 ± 2.18 P: 3.86 ± 2.04 Corticosteroids 25/10% I/C: Intervention/Control; --: Not reported; * Reported in MS as SDs, in original publication as SEM; ** First reported in MS, second as in original trial paper. -- No details of excluded studies were provided. All included studies compared intervention with placebo. No head-to-head trials were available for the analysis of the efficacy of golimumab versus other comparator drugs. Therefore, a mixed treatment comparison (MTC) was undertaken to estimate the effects of golimumab and comparator drugs. A table of the 8 trials used to conduct the MTC was presented as Table 22 on page 96 of the MS. These included 1 trial for golimumab versus placebo (GO-RAISE 36 ), 5 trials of etanercept versus placebo, and 2 trials for adalimumab versus placebo. The relevant network diagram is presented in Figure 1. Figure 1: Network diagram for MTC Golimumab 1 study Placebo 2 studies 5 studies Adalimumab Etanercept 36

37 Two different analyses were performed, a short and long-term analysis: Short-term analysis: data from 8 trials with follow-up between 12 and 16 weeks were used. Long-term analysis: data from 3 trials with follow-up of 24 weeks were used (ATLAS, Davis and GO-RAISE) allowing comparisons between all three active comparators (see table 24, page 105 in the MS). The MS provides the following rational for the inclusion of three trials in the long-term analysis: Although reported, data for adalimumab at 24 weeks was excluded for the Canadian AS trial. Following the rationale of the NICE HTA Report, the main reason for excluding these data was the early escape open label treatment options programmed into this study for patients not achieving a response. This study did not report whether it had data imputation rules for the early escape open label treatment options (unlike ATLAS). (p. 104) Generally, the MS follows the approach of the previous NICE HTA report (TA ), but here it differs. The rational in the previous NICE HTA report (TA ) was as follows: Although reported, data for adalimumab at24 weeks (from the ATLAS and Canadian AS studies) were not included in the meta-analysis. The principal reason for excluding these data is due to the early escape open-label treatment options programmed into these studies. (p. 22) COMMENT The reason for this deviation seems obvious. Following the rational of the previous NICE HTA report (TA ), GO-RAISE would also have been excluded, leaving only data for etanercept at 24 weeks. The ERG group agrees with the NICE HTA report (TA143) that data for adalimumab at24 weeks (from the ATLAS and Canadian AS studies) should not be included in the meta-analysis. This means that data for golimumab at24 weeks (from the GO-RAISE study) should also not be included in the meta-analysis. 37

38 4.1.4 Provide details of any relevant studies not discussed in the submission? Why were these studies excluded and how were these studies identified by the ERG? The ERG believes that it s uncertain whether all relevant studies were included in the MS. Due to poor reporting and mistakes made in the search strategies is it possible that relevant publications were missed. Repeat searches using the manufacturer s search terms were undertaken (although the ERG was not able to sift through the search results due to time constraints). However, the ERG is not aware of the existence of any relevant trials missed. Evidence from completed studies was presented in the clinical effectiveness section. The manufacturer did not specify whether any searches for unpublished evidence or ongoing trials were undertaken in the systematic review. The ERG group searched ClinicalTrials.gov ( and were unable to identify any additional relevant studies of the clinical effectiveness or safety of golimumab in patients with ankylosing spondylitis that had reported findings. Two randomised trials were found that were currently recruiting and 2 observational studies were found that were not yet open for recruitment. However, it is possible that unpublished evidence relating to comparator drugs may have been missed in the identification of evidence by the manufacturer. The MS stated that there were no ongoing studies and there were no details of searches of clinical trials registers. It is reasonable to assume that the manufacturer was aware of all golimumab trials; however, there may have been completed and ongoing trials of the comparator interventions which could have been missed. The ERG undertook brief searches of several relevant clinical trials registers for the comparator treatments, etanercept and adalimumab, for the treatment of ankylosing spondylitis. A search of the NIH Clinical Trials Register ( identified 5 trials for golimumab, 24 for etanercept and 20 for adalimumab. A search of metaregister ( identified 11 trials for golimumab, 36 for etanercept, and 22 for adalimumab. A search of the WHO ICTRP ( identified 5 trials for golimumab, 21 for etanercept, and 15 for adalimumab. Searches of the EU-CTR Clinical Trials Register ( identified 10 trials for golimumab, 21 for etanercept, and 15 for adalimumab. Therefore a total of 182 reports of ongoing and completed trials which were not identified by searching could have included potentially relevant trials reports. The ERG was not able to screen the retrieved results;, therefore it was not possible to determine whether these trials reports met the inclusion criteria. 38

39 No non-rct evidence was included in the systematic review. As stated earlier in this report, the ERG considers non-rcts, including expert opinion, to be a valid and important source of evidence for the evaluation of adverse events. The manufacturer stated that no non-rcts or observational studies of the use of golimumab in patients with ankylosing spondylitis were available at the time of submission. However, limitations in the searches conducted to identify evidence relating to adverse events (as outlined in Section of this report) are such that useful data of the adverse events associated with golimumab and comparator drugs may have been omitted from the MS. 4.2 Summary and critique of submitted clinical effectiveness evidence If there is more than one RCT described in the MS, it may be appropriate to discuss each trial individually using the headings described Summary of submitted clinical evidence for each relevant trial. Full results for all included studies are presented in tables 19, 20 and 21 in the MS (see table 5). However, the reported results do not match with the results used in the WinBugs analyses. For instance, on page 91 (table 21) 6 weeks results for Brandt et al. are reported, but results at 12 weeks are not reported. In the WinBugs code on page 312, 12 week BASDAI50 results for Brandt et al. are used. In the Winbugs code on page 308 for BASFI (12 weeks) results for the Canadian AS study are reported, but these are not included in the corresponding Table on page 293/294 (see Tables 5 and 6 below). 39

40 Table 5: Results reported in tables 19, 20 and 21 of the MS GO-RAISE ATLAS Can. AS Brandt Calin Davis Gorman VdHeijde 14w 24w 12w 24w 12w 24w 6w 12w 12w 24w 16w 12w ASAS20 ASAS40 BASDAI20 BASDAI50 BASDAI70 BASDAI90 ΔBASDAI * * * ΔBASFI BASFI +2p ΔBASMI BASMI +1p ASAS5/6 Partial Rem. ASAS50 ASAS70 BASDAI <40 Morning St. Global Ass. ESR CRP Δ= Change from baseline; +1p= at least 1 point improvement; Rem=Remission; St=Stiffness; Ass=Assessment; *= Reported in WinBugs codes, but not in MTC. Table 6 below presents the values used in the Winbugs code as presented in the MS pp Several minor errors were found between these data and that presented in study publications or clinical study reports: Both the clinical study report and Inman et al report the placebo count for GO-RISE ASAS20 at 14 weeks as 17/78 (21.8%) not 18/78 as in the code; The ASAS20 (12 weeks) values reported by the ATLAS study were 22/107 for placebo, and 121/208 for adalimimab, not 23/107 and 120/208 as in the code; For the ATLAS trial we were unable to tell how the values for change in BASFI at 12 or 24 weeks were arrived at; Based on Davis 2003 we were unable to tell how the values for change in BASDAI or change in BASFI were arrived at; For Davis 2003 the serious adverse event count seems to be 5/139 for placebo and 11/138 for etanercept, not 9/138 as in the code; Based on Gorman 2002 we were unable to tell how the values for change in BASDAI at 16 weeks were arrived at; and It is unclear how the values for serious adverse events in the Van der Heijde 2006 study were arrived at. 40

41 Table 6: Results report in WinBugs codes in the MS GO-RAISE ATLAS Can. AS Brandt Calin Davis Gorman Van der Heijde 14w 24w 12w 24w 12w 24w 6 or **w 12w 12w 24w 16w 12w ASAS20 18/78 82/138 18/78 77/138 23/ /208 19/ /208 12/44 18/38 4/16 11/14 9/39 26/45 39/139 82/138 31/139 78/138 6/20 16/20 19/51 107/150 BASDAI20 ***** ****** BASDAI50 ******* ***** ΔBASDAI MD (se) ΔBASFI MD (se) (0.22) (0.19) 0.03 (0.21) (0.18) Disc. 2/78 9/138 ISR 2/78 12/138 SAEs 5/77 5/138 5/16 12/14 ******* ***** (0.26) (0.21) 0.13 (0.21) (0.19) 17/107 94/ (0.02) -2.6 (0.01) (0.21) (0.15) 5/107 6/208 3/107 21/208 3/107 6/208 16/107 88/ (.02) -2.6 (0.2) (0.20) (0.20) (0.37) (0.31) ********** 2/20 1/ (0.43) (0.44) -0.2 (0.59) -1.9 (0.56) 2/16 2/14 19/139 12/ (0.15) (0.16) (0.17) (0.17) 0.00 (0.18) (0.15) Δ= Change from baseline; Disc= Discontinuations; ISR= Injection Site Reactions; SAEs= Serious Adverse Events; = Results reported for the MTC, but not in the WinBugs codes. Results: placebo first. 0/16 2/14 0/16 0/14 6/39 15/45 13/139 41/138 5/139 9/ (0.62) -2.3 (0.47) 1/20 5/20 0/20 0/20 10/51 87/150 7/51 14/150 6/51 34/150 2/51 6/150 41

42 The most important difference between the data used in the MTC and those used in the WinBugs codes, is the inclusion of 12 weeks follow-up data for Brandt et al. in the WinBugs codes. The study by Brandt et al. reports main outcomes at 6 weeks follow-up: After week 6, patients in the placebo group were switched to etanercept for the next 12 weeks, and patients in the etanercept group continued to receive etanercept for another 6 weeks, to ensure that all patients received etanercept for a total of 3 months. Therefore, data at 12 weeks follow-up are underestimating the effects of etanercept because all patients in the placebo have received etanercept for 6 weeks. Because the RCT outcome results were arranged by trial rather than by outcome it was difficult to get a sense of the magnitude and variation of an outcome across different trials and follow up times. In addition there was no graphical presentation of these results. To aid the committee in this and in order to summarise the data in a convenient form the ERG have therefore formalised the results in forest plots (Figure 2 to Figure 5). Figure 2: Relative risk of ASAS responses in the included RCTs. Anti-TNF Placebo Risk Ratio Risk Ratio Study or Subgroup ASAS20 Events Total Events Total M-H, Random, 95% CI M-H, Random, 95% CI ADA-ATLAS, 12w [1.92, 4.18] ADA-ATLAS, 24w [1.83, 4.33] ADA-Can. AS, 12w [0.97, 3.13] ADA-Can. AS, 24w [0.96, 4.83] ETA-Brandt, 6w [1.29, 7.67] ETA-Calin, 12w [1.40, 4.84] ETA-Davis, 12w [1.57, 2.86] ETA-Davis, 24w [1.80, 3.57] ETA-Gorman, 16w [1.32, 5.39] ETA-Vd Heijde, 12w [1.32, 2.77] GOL-GO-RAISE, 14w [1.75, 4.24] GOL-GO-RAISE, 24w [1.57, 3.72] ASAS40 ADA-ATLAS, 12w [1.82, 5.11] ADA-ATLAS, 24w [1.80, 5.05] ETA-Vd Heijde, 12w [1.43, 4.26] GOL-GO-RAISE, 14w [1.68, 5.07] GOL-GO-RAISE, 24w [1.62, 4.92] ASAS50 ADA-Can. AS, 12w [1.81, 18.49] ADA-Can. AS, 24w [1.08, 7.18] ETA-Brandt, 6w [0.82, 14.33] ETA-Calin, 12w [1.80, 12.64] ETA-Davis, 12w [2.80, 8.04] ETA-Davis, 24w [2.91, 9.68] ASAS70 ADA-Can. AS, 12w [1.21, 70.75] ADA-Can. AS, 24w [1.28, 14.10] ETA-Calin, 12w [0.82, 6.89] ETA-Davis, 12w [2.34, 7.53] ETA-Davis, 24w [3.36, 13.75] Favours Placebo Favours Anti-TNF 42

43 ASAS20 was the primary outcome in most RCTs and the submission reported ASAS20 as an outcome in all of the trials. For almost all ASAS20, 40, 50 and 70 responses at various time points the results indicated a statistically significant superiority of anti-tnf agent over placebo (Figure 2). The ASAS20 RR was quite homogeneous across different drugs and follow up with nearly all in the range 2.0 to 3.0. It should be noted that for time points beyond 14 weeks in the GO-RAISE and ATLAS trials the comparison with placebo is compromised because of the large number of observations carried forward due to early escape to active therapy for large numbers of patients. Figure 3: Relative risk of BASDAI responses in the included RCTs. CONFIDENTIAL INFORMATION REMOVED Study or Subgroup BASDAI20 GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w Anti-TNF Placebo Risk Ratio Risk Ratio Events Total Events Total M-H, Random, 95% CI M-H, Random, 95% CI 2.01 [1.47, 2.75] 2.50 [1.71, 3.66] BASDAI50 ADA-ATLAS, 12w ADA-ATLAS, 24w ETA-Brandt, 12w ETA-Brandt, 6w ETA-Vd Heijde, 24w GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w [1.79, 4.51] 2.83 [1.75, 4.57] 1.27 [0.74, 2.19] 9.14 [1.30, 64.34] 2.96 [1.67, 5.24] 2.98 [1.72, 5.18] 3.46 [1.95, 6.13] BASDAI70 GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w 5.57 [2.07, 15.02] 3.75 [1.67, 8.44] BASDAI90 GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w 7.62 [1.02, 57.17] [1.58, 84.25] Favours Anti-TNF Favours Placebo Figure 3 shows the BASDAI 20, 50,70 and 90 responses reported for the included RCTs. All, except 12-week results for Brandt et al., reached statistical significance in favour of anti-tnf treatments relative to placebo. The BASDAI50 results were used in the MTC so as to generate effectiveness inputs for the economic model. The BASDAI50 results appear quite homogeneous across different drugs and follow uptimes (6 to 24 weeks) with five of the seven RRs falling between 2.8 and 3.5. The important difference is the results for Brandt et al. Results at 6-week show an extreme significant effect in favour of etanercept with very wide confidence intervals; while the 12-week results show a non-significant effect of etanercept when compared with placebo. The 12-week results were used in the economic model in the MS. Continuous outcomes were reported as mean (SD) change from baseline for each group. The results for BASDAI scores are summarised in Figure 4. This indicates that for most trials the mean disease activity index scores decrease from baseline to a statistically significant greater extent in the anti-tnf arms than in placebo arms, indicating health benefit from active intervention. 43

44 Figure 4: Mean difference between anti-tnf and placebo for change from baseline in BASDAI score. Study or Subgroup ADA-ATLAS, 12w ADA-ATLAS, 24w ETA-Brandt, 6w ETA-Davis, 24w ETA-Gorman, 16w GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w Anti TNF Placebo Mean Difference Mean Difference Mean SD Total Mean SD Total IV, Random, 95% CI IV, Random, 95% CI [-2.35, -1.25] [-2.35, -1.25] [-3.41, -0.99] [-2.34, -1.48] [-1.46, -0.54] [-2.60, -1.46] [-2.89, -1.57] Favours Anti TNF Favours placebo Figure 5 shows similar changes for functional index scores (BASFI). Figure 5: Mean difference between anti-tnf and placebo for change from baseline in BASFI score. Study or Subgroup ADA-ATLAS, 12w ADA-ATLAS, 24w ADA-Can. AS, 12w ETA-Brandt, 6w ETA-Davis, 24w ETA-Gorman, 16w GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w Mean Anti TNF Placebo Mean Difference Mean Difference SD Total Mean SD Total IV, Random, 95% CI [-1.92, -0.90] [-2.03, -0.93] [-2.84, -0.96] [-3.30, -0.10] [-1.88, -0.94] [-3.72, -0.68] [-2.21, -1.13] [-2.40, -1.30] IV, Random, 95% CI Favours Anti TNF Favours placebo Figures 6 and 7 show similar changes for quality of life. Figure 6: Mean difference between anti-tnf and placebo for change from baseline in SF-36 Physical Component Summary (PCS) score. CONFIDENTIAL INFORMATION REMOVED Study or Subgroup ADA-ATLAS, 12w ADA-ATLAS, 24w GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w Anti TNF Placebo Mean Difference Mean Difference Mean SD Total Mean SD Total IV, Random, 95% CI IV, Random, 95% CI [3.33, 7.27] [3.38, 7.62] Favours placebo Favours Anti TNF Figure 7: Mean difference between anti-tnf and placebo for change from baseline in SF-36 Mental Component Summary (MCS) score. CONFIDENTIAL INFORMATION REMOVED Study or Subgroup ADA-ATLAS, 12w ADA-ATLAS, 24w GOL-GO-RAISE, 14w GOL-GO-RAISE, 24w Anti TNF Placebo Mean Difference Mean Difference Mean SD Total Mean SD Total IV, Random, 95% CI IV, Random, 95% CI [-2.09, 2.69] 1.20 [-1.19, 3.59] Favours placebo Favours Anti TNF

45 4.2.2 Describe and critique the manufacturer s approach to validity assessment for each relevant trial. A formal appraisal of the validity of the included RCTs was clearly presented in the MS (table 18, page 82 and Appendices 3 and 5). All the criteria listed under Section (page 80) of the MS (as specified in the NICE STA Specification for manufacturer/sponsor submission of evidence) 3 were addressed in the quality assessment findings. The ERG acknowledges that whilst the items listed in the validity assessment tool used in the MS were appropriate, several quality criteria relevant to the critical appraisal of RCTs were not taken into account. Example criteria for the assessment of the risk of bias in RCTs were described by the Centre for Reviews and Dissemination (CRD). 37 The tabulated quality assessment findings for the RCTs (Appendices 3 and 5) referred to CRDs guidance for undertaking quality assessment. 37 However, the following quality criteria recommended by the CRD were not represented in the tabulated quality assessment findings: i) Specification of eligibility criteria; ii) identification of any co-interventions with the potential to impact upon outcomes; iii) assessment of treatment compliance; iv) assessment of success of blinding; v) presence of reported point estimates and measures of variability for the primary outcome measure. Whilst it was described in the MS (page 214) that data abstraction was undertaken by a single reviewer and independently checked by a second reviewer, it was not explicitly stated whether critical appraisal was conducted by a single reviewer or using consensus of multiple reviewers. The ERG checked the quality assessment findings against the original study publications and any additional points are discussed within this Section. The validity of the eight RCTs was assessed according criteria adapted from Centre for Reviews and Dissemination (2009) Systematic reviews. CRD s guidance for undertaking reviews in health care. York: Centre for Reviews and Dissemination 37. The results were summarised in table 18 (page 82) and reproduced below. Responses that the ERG have highlighted and discussed in further detail are marked with an asterisk (*). 45

46 Table 7: Summary of Quality assessment as presented by manufacturer page 82 Trial No (acronym) Was randomisation carried out appropriately? Was the concealment of treatment allocation adequate? Were the groups similar at the outset of the study in terms of prognostic factors, for example, severity of disease? Were the care providers, participants and outcome assessors blind to treatment allocation? If any of these people were not blinded, what might be the likely impact on the risk of bias (for each outcome)? Were there any unexpected imbalances in drop outs between groups? If so, were they explained or adjusted for? Is there any evidence to suggest that the authors measured more outcomes than they reported? Did the analysis include an intention to treat analysis? If so, was this appropriate and were appropriate methods used to account for missing data? GO- ATLAS Canadian Brandt Calin Davis Van der Gorman RAISE AS study Heijde YES YES* YES* YES* YES* YES* YES* YES YES NOT CLEAR NOT CLEAR YES NOT CLEAR YES NOT CLEAR YES* YES YES YES YES* YES YES NO YES NOT CLEAR NOT CLEAR YES YES YES NOT CLEAR NO NO NO NO NO NO NO NO NO NO NO NO NO NO NO NO YES YES YES YES YES YES YES YES YES YES Six of the assessed trials were also assessed by McLeod et al This assessment used a different tool to that in the submission and the overall result was less generous to the quality of the RCTs than in the present submission. COMMENT In contrast to the MS the ERG thought the method of randomisation was not adequately described for six (ATLAS, Canadian AS, Brandt, Calin, Davis, Van der Heijde) out of the eight included trials. There were also notable differences between groups for the GO-RAISE and Calin trials. For GO- RAISE the placebo group, on average, had had AS for 5 years longer than the golimumab groups. For 46

47 Calin, the etanercept group were older and, on average, had had the disease 5 years longer than the placebo group. Given the assessment bias which could occur for example, as a result of injectionsite reaction (more common among treatment groups) it is significant that only Davis and GO- RAISE (only in clinical study report) mention blinding of outcome assessors Describe and critique the statistical approach used within each relevant trial. The statistical approach used within each relevant trial is summarised in the MS in table 17, page 68 (see also table below). Table 8: Summary of statistical analyses in RCTs Trial Statistical analyses Data management. patient withdrawals GO-RAISE ATLAS The proportions of patients responding were compared between treatment groups using the Cochrane Mantel Haenszel test with stratification by the screening CRP level (< 1.5 mg/dl or > 1.5 mg/dl). Changes from baseline in continuous variables were compared between treatment groups using an analysis of variance on the normal van der Waerden test scores. Patients in the placebo or 50 mg groups who met the criteria for early escape at week 16 were considered to be non-responders at week 24. A logistic regression analysis of the ASAS20 response at week 14 was performed based on the following factors: treatment group (placebo or golimumab at either dose), screening CRP level (< 1.5 mg/dl or > 1.5 mg/dl), DMARD use (yes or no), and the continuous variables body weight and duration of AS. ITT analysis: The analyses were performed on an intention to treat basis. All response rates for the adalimumab and placebo treatment groups were compared using Pearson s chi square test. Fisher s exact test was used if at least 25% of cells had expected counts of < 5. Patients who withdrew from the study before week 12 or who had missing values were considered nonresponders. To assess the impact of the missing data, a sensitivity analysis of ASAS20 at week Placebo: Week 16 Of 78 patients administered placebo, 35 patients completed the study, while 41 crossed over to golimumab 50 mg group; 2 patients discontinued the study agent. Week out of 37 patients completed the study. Golimumab 50 mg Group: Week 16 Of 137 patients on golimumab 50 mg, 103 completed the study up to week 16 while 25 crossed over golimumab 100 mg group; 9 patients discontinued the study. Week patients completed the study; among the crossed over group, all 25 patients completed the study. Week 104 Placebo crossover 3 patients discontinued through week 104 golimumab 50 mg only 20 patients discontinued through week 104 Golimumab 100 mg Group Week 16 Among 140 patients on golimumab 100 mg, 134 patients completed the study; 6 discontinued the study agent. Week 24 Among 165 patients who continued on golimumab 100 mg, 159 patients completed the study. Week 104 Golimumab 50 mg increased to 100 mg 8 patients discontinued through week 104. Golimumab 100 mg combined 24 patients discontinued through week 104. Placebo Group: Week 12 Of 107 assigned to placebo, 103 completed the schedule of events through week 12 Week completed the schedule of events through week 24 Adalimumab 40 mg: Week 12 Of 208 patients assigned to

48 Canadian AS study Brandt et al. Calin et al. Davis et al. was performed without including non-responder imputations. Analysis of covariance was used to compare the mean change in each treatment group for the continuous secondary efficacy variables. The analysis of covariance model included a factor for treatment and was adjusted for baseline. A last observation carried forward analysis was used for continuous end points. For laboratory values, mean changes at week 12 were compared between treatment groups using one way analysis of variance. ITT analysis Analyses were performed on the intention to treat population, defined as all randomised patients who received at least 1 dose of the study medication. An intent to treat analysis was used for analyses of the SPARCC scores, and all patients with at least 1 baseline MRI were included. The mean of the 2 readers SPARCC scores was used for all analyses of continuous variables. All statistical tests were 2 sided and comparisons were performed with α = The changes in SPARCC scores from baseline to week 12 were compared between the 2 treatment groups using an analysis of covariance (ANCOVA) model, with the baseline score as a covariate. Changes from week 12 to week 52 were compared between the 2 treatment groups using an ANCOVA model, with the baseline score and the change from baseline to week 12 as covariates. There was no imputation of missing SPARCC scores. Ninety five percent exact Clopper/Pearson confidence bounds were calculated for the response rates. Dropouts and patients who violated the study protocol were treated as non-responders. Means were compared by analysis of covariance with the baseline value as co-variable; values at baseline were compared by Wilcoxon s unpaired rank sum test. The paired t test was used for comparing changes within single BASDAI items. ITT analysis An intent to treat analysis was performed to analyze the response criteria; comparisons were made by Fisher s exact test. Proportions of patients responding were compared using the Cochran Mantel Haenszel mean score test, stratified by baseline DMARD use. Χ2 and Breslow Day tests were used to evaluate direct and interactive effects of DMARDs on ASAS responses. Fisher s exact test was used for safety analyses. Patients who withdrew from the study prematurely were treated as non responders at each assessment interval thereafter for ASAS and other patient reported responses. ITT analysis Analyses were based on the intention to treat population and included all patients who received at least one dose of the blinded test article. Proportions of patients responding were compared using the Cochran Mantel Haenszel mean score test stratified by the presence or absence of concomitant DMARDs at baseline. Patients were considered nonresponders for binary end points at all time points after study drug discontinuation. For continuous 48 adalimumab 40 mg every other week, 204 completed the schedule of events through week 12. Week completed the schedule of events through week 24. Placebo Group: Week 12 Of 44 patients assigned to placebo, all 44 completed the schedule of events through week 12. Week 52 2 patients withdrew resulting in 42 completed subjects. Adalimumab 40mg: Week 12 Of 38 patients assigned to adalimumab 40mg all 38 completed the schedule of events through week 12. Week 52 All 38 patients completed week 52 assessment. Placebo: Week 6 Of 16 patients on placebo, 15 completed week 6; 1 patient was withdrawn due to compliance. Week patients completed week 30 Etanercept: Week 6 All 14 patients completed week 6. Week 24 All 14 patients completed week 24 All 39 patients assigned to placebo completed the study. Of 45 Patients assigned to receive etanercept, 43 completed the study; 2 discontinued for non-safety reasons. Of 139 patients who received placebo, 120 completed 24 weeks of therapy; 19 patients discontinued. Of 138 patients who received etanercept, 126 completed 24 weeks of therapy; 12 patients discontinued.

49 VdHeijde et al. Gorman et al. measures, such as the individual components of the ASAS and the acute phase reactant and spinal mobility parameters, the percent improvement from baseline was calculated, and the van Elteren stratified rank test was used to compare the percent improvement between the 2 treatment groups. For the safety analyses, treatment groups were compared with respect to the proportions of patients with adverse events and infections, using Fisher s exact test. ITT analysis All patients who received at least 1dose of the study drug were included in intent to treat analyses of efficacy and safety. The study was designed to test the non inferiority of etanercept 50 mg once weekly to 25 mg twice weekly at week 12. For the secondary analyses, two sided Fisher s exact tests were conducted. ITT analysis The modified intent to treat population was the primary population for efficacy and safety analyses, and comprised all patients who received at least one dose of the test drugs. The main analysis, based on the intention to treat principle, consisted of a comparison of data from the last visit of each randomised patient with data from the base line visit. For continuous measures, the change from base line in each patient was calculated and compared with the mean changes in the two study groups using the Mann Whitney test for two independent groups. For categorical and ordinal data, the two groups were compared at the last visit with the use of Fisher s exact test (two tailed). Of 51 patients.who received placebo, 44 completed 12 weeks of therapy; 7 patients discontinued. Of 155 patients who received etanercept 50 mg, 141 completed 12 weeks of therapy; 14 patients discontinued. Of 150 patients who received etanercept 25 mg, 136 completed 12 weeks of therapy; 14 patients discontinued. Placebo Group: 4 months Of 20 patients assigned to placebo, 18 completed the schedule of events through 4 months Etanercept 25mg twice weekly: 4 months Of 20 patients assigned to etanercept 25mg 19 completed the schedule of events through 4 months. COMMENT: For the eight included RCTs the statistical analyses employed by the authors of the relevant publications were extracted and reproduced in the submission (table 17, page 68). Some errors were noted by the ERG (e.g. SE confused for SD and small differences in numbers). The main issues of debate are: 1. Whether to include data for adalimumab and golimumab at 24 weeks (from the GO-RAISE, ATLAS and Canadian AS studies) in the meta-analysis. The ERG group agrees with the NICE HTA report (TA143) that these data should not be included in the meta-analysis. 2. Whether to include data for etanercept at 6 weeks (from the study by Brandt et al.) in the meta-analysis. The ERG group thinks that 6-week data are probably a better reflection of the true effect of etanercept in the study by Brandt et al. Results at 12-week follow-up are seriously biased. However, the ERG thinks that 6-week data from Brandt et al. may not be comparable with 12 to 16 week follow-up data from other studies. 49

50 As mentioned before, there is considerable heterogeneity within the studies included in the mixed treatment comparison, making the results unreliable Describe and critique the manufacturer s approach to outcome selection within each relevant trial. Appendix section (pages ) of the submission describes the outcomes selected for data extraction from the included studies as follows: BASDAI: (mean (SD)) BASFI: (mean (SD)) BASMI: (mean (SD)) BASDAI-20: number of responders BASDAI-50: number of responders ASAS-20: number of responders ASAS-40: number of responders ASAS-50: number of responders ASAS-70: number of responders Discontinuations: number of patients Injection site reactions: number of patients Serious adverse reactions: number of patients According to the inclusion criteria (MS, table 9, page 38) the outcomes required were: The ASsessment in AS International Working Group criteria (ASAS) The Bath AS Metrology Index (BASMI) The Bath AS Disease Activity Index (BASDAI) The Bath Ankylosing Spondylitis Functional Index (BASFI) Discontinuation (number of patients) Injection site reactions (number of patients) Severe Adverse Events (number of patients) In addition, the following outcomes were reported in the NICE scope and the decision problem in the MS (page 35): Functional capacity Health related quality of life 50

51 COMMENT The following justification is provided in the MS for not reporting data on Functional capacity: No information regarding the functional capacity is available for golimumab and therefore will not be included in the submission. No justification is provided in the MS for not reporting data on health related quality of life. In Appendix 9.19 SF 36 results from GO RAISE are reported (probably in response to a request from the original ERG group). However, no attempt is made to collect these data from other trials, to compare data across trials or to use these data in the economic model. When we looked for quality of life data, we found that the SF-36 was used in 4 trials (GO-RAISE, ATLAS, Brandt and Gorman), allowing a comparison across all three active interventions. However, no useful data were reported by Brandt and Gorman, leaving only a possible comparison between golimumab and adalimumab, using the GO-RAISE and ATLAS trials (see Figures 6 and 7, above) To what extent does each relevant trial include the patient population(s), intervention(s), comparator(s) and outcomes as defined in the final scope? Population The trials had inclusion criteria that were wider than the population defined in the scope, in particular not being limited by severity or adequacy of response to conventional therapy. Only three (GO- RAISE, ATLAS and Brandt) of the eight included trials had inclusion criteria that were of similar severity of disease and only three trials (GO-RAISE, ATLAS and Can AS) specified inadequate response to NSAIDs and in each of these to only one NSAID. Only two trials (GO-RAISE and ATLAS) had both similar disease severity and inadequate response criteria. Intervention The interventions in the trials were as described in the scope and only data from the trial arm with the licensed dose of 50mg golimumab (as opposed to 100mg in the other active treatment arm) from the GO-RAISE trial was used. Similarly, only data for 25mg etanercept and 40mg adalimumab were used. Comparators All trials used placebo as the comparator. There were no head-to-head comparisons of anti TNFs. Outcomes 51

52 Outcomes have been discussed in section 4.2.4, above. 52

53 4.2.6 Where appropriate, describe and critique any meta-analysis, indirect comparisons and/ or mixed treatment analysis carried out by the manufacturer. This section should include a summary of the manufacturer s methods and results as described in the MS. The ERG should critique the methods used and interpret the results in light of the methods used by the manufacturer and generalisability to patients in England and Wales. METHODS The MS describes a mixed treatment comparison (MTC), starting on page 96 in the MS, with a summary of data used in the analysis in table 23 (pages ) and results in tables (pages ). These analyses are clear and few mistakes were found by the ERG; when mistakes were found they were easy to correct and did not change the overall conclusions. In addition, the MS provides WinBugs codes in appendix 17 (page 297) of the MS. These codes are provided for each individual outcome and are presented with the data for each analysis. These data are not consistent with the data described in the MTC. Sometimes different trials are included and sometimes different data are used (see also section 4.2.1). Generally, the WinBugs codes worked and the analyses were sound. In the MS, the methods are described as follows: Following the same approach as the NICE HTA Report, data were pooled at 2different time points for short term and long term analyses. For all clinical efficacy endpoints, data from studies with outcomes reported at 16 weeks or less were pooled and analyzed as short term data whereas data from studies with outcomes reported at more than 16 weeks were pooled and analyzed together as long term data (Table B16). For clinical efficacy endpoints, data at 6 weeks of follow up for the Brandt et al. study were included with the short term analysis. Similarly, data at 16weeks of follow up for the Gorman et al. study, and data at 14 weeks for the GO RAISE trial were included with analyses at the short term follow up. For all safety endpoints, all studies were pooled together. Given the rarity of events for safety outcomes, this was necessary to derive estimates based on properly converging models. Although reported, data for adalimumab at 24 weeks was excluded for the Canadian AS trial. Following the rationale of the NICE HTA Report, the main reason for excluding these data was the early escape open label treatment options programmed into this study for patients not achieving a 53

54 response. This study did not report whether it had data imputation rules for the early escape open label treatment options (unlike ATLAS). Results section of this report presents the findings of short and long term analyses. Data tables used for these analyses are included in Appendix 16, and the Winbugs code is presented in Appendix 17. COMMENT The ERG considers the long term (24 weeks) results not to be reliable; therefore, only data at 12, 14 or 16 weeks follow-up were used. This is in accordance with the approach in the NICE HTA Report (NICE TA 143) (see section 4.1.3). Table B16 could not be found in the MS, this is probably from the original submission. The ERG assumes this is table 24 in the current submission. The ERG agrees with the statement in the MS that data at 6 weeks of follow up for the Brandt et al. study should be included in the short term analysis. However, the MS used data at 12 weeks of follow up for the Brandt et al. study in the analysis for BASDAI50, see the WinBugs code on page 302 in the MS. This has been corrected in the ERG analysis. RESULTS The results from the MTC are presented in the MS in tables (pages ), separately for each outcome. For convenience the results are summarised below by comparison. Only results for the short term are summarised as the ERG deemed the long term (24 weeks) results not to be reliable. Table 9: Results of MTC for golimumab versus adalimumab and versus etanercept. Golimumab versus Adalimumab (OR or MD (95% CrI)) Outcome Results MS (short term) ASAS20 OR=1.19 (0.55, 2.66) ASAS40 OR=1.04 (0.39, 2.65) ASAS50 -- ASAS70 -- BASDAI20 -- BASDAI50 MS: OR=1.10 (0.45, 2.69) ERG: OR=1.25 (0.44, 2.81) BASDAI (continuous) MD= (-0.81, 0.34) BASMI (continuous) MD= 0.52 (0.23, 0.80) BASFI (continuous) MD= (-1.01, 0.47) Discontinuations OR=5.52 (0.74, 54.51) Injection Site reactions OR=0.96 (0.12, 8.89) Serious Adverse Events OR=0.50 (0.06, 3.44) 54

55 Golimumab versus Etanercept (OR or MD (95% CrI)) Outcome Results MS (short term) ASAS20 OR=1.11 (0.54, 2.42) ASAS40 OR=1.08 (0.37, 3.17) ASAS50 -- ASAS70 -- BASDAI20 OR=0.28 (0.03, 1.90) BASDAI50 MS: OR=0.99 (0.40, 2.63) ERG: OR=0.79 (0.25, 1.94) BASDAI (continuous) MD= (-1.58, -0.14) BASMI (continuous) MD= 1.06 (-0.33, 2.43) BASFI (continuous) MD= 0.30 (-0.93, 1.56) Discontinuations OR=5.14 (1.03, 39.21) Injection Site reactions OR=1.23 (0.28, 8.50) Serious Adverse Events OR=0.34 (0.06, 1.77) OR=Odds Ratio; MD=Mean Difference; CrI=Credibility Interval; --=Not available. OR>1 favours golimumab; MD<0 favours golimumab. Results in bold are significant. The results from the ERG differ from the results in the MS, in that the ERG results included results from Brandt et al. at 6 weeks for BASDAI50, instead of 12 weeks (all participants received etanercept after 6 weeks). The ERG agrees on all other results of the MTC. It is difficult to reproduce the result for BASDAI50. Using the data as reported in the WinBugs code for BASDAI50 (12-week data for Brandt et al.), the MS should have found OR=1.22 (95% CrI: 0.45, 2.69) for golimumab versus adalimumab and OR=1.13 (95% CrI: 0.45, 2.69) for golimumab versus etanercept. Given the fact that the credibility intervals were correct, they seem to have made a typing error. However, if they had used the correct data for Brandt et al. (6-week data) it would have been: OR=1.25 (95% CrI: 0.44, 2.81) for golimumab versus adalimumab and OR=0.79 (95% CrI: 0.25, 1.94) for golimumab versus etanercept Additional clinical work conducted by the ERG Provide details of any additional work conducted by the ERG in relation to clinical effectiveness. If the results of any of the additional work affect the size of the ICER, refer the reader to the summary table in Section 6. No further additional work was conducted by the ERG in relation to clinical effectiveness. 55

56 4.3 Conclusions Describe the completeness of the MS with regard to relevant clinical studies and relevant data within those studies. Does the submission contain an unbiased estimate of the technology s (relative and absolute) treatment effects in relation to relevant populations, interventions, comparators and outcomes? Are there any remaining uncertainties about the reliability of the clinical effectiveness evidence? Reference should also be made concerning the extent to which the submitted evidence reflects the decision problem defined in the final scope. There were several consequential errors and potential weaknesses in the MS search strategies. These problems may have led to relevant studies being missed. The clinical evidence in the form of the included trials had inclusion criteria that were wider than the scope, in particular not being limited by severity or adequacy of response to conventional therapy. Three out of eight trials included patients with disease severity similar to the scope (GO-RAISE, ATLAS and Brandt). None of the studies fulfilled the criterion of two or more failed NSAIDs. However, three studies included patients who had failed at least one NSAID (GO-RAISE, ATLAS and Can AS). Only a single trial provided evidence for the clinical effectiveness of golimumab. Because of the unlicensed dose regimen in one arm of this trial and the early escape to cross over treatments at 16 weeks for 14-week non-responders, the totality of evidence for the comparison of the licensed dose regimen versus placebo depends on measures made at 14 weeks for only 78patients receiving placebo and 137 receiving the 50mg dose regimen. The evidence presented regarding the maintenance of clinical response to the 50mg dose regimen beyond 14 weeks was deemed unreliable because of the large numbers of patients crossing over. Apart from the trial of golimumab versus placebo (the GO-RAISE trial), there are seven placebo controlled RCTs of other anti-tnf agents (five with etanercept, and two with adalimumab). No headto-head evidence is available for anti-tnfs; therefore, only indirect evidence is available. The MTC point estimates comparing the effectiveness of golimumab with that of etanercept and of adalimumab were associated with considerable uncertainty. The MTC results indicated that golimumab, etanercept and adalimumab were more clinically effective than placebo according to Bath Ankylosing Spondylitis disease activity and functional indices (i.e. BASDAI and BASFI measures) and according to ASAS criteria. Relative to placebo the anti-tnf agents were associated with more likely injection site reactions and more discontinuations from treatment. When the anti-tnf agents golimumab, adalimumab and etanercept were compared with each other using the Bayesian MTCs, the 95% credible intervals for most comparisons failed to demonstrate statistically significant differences between golimumab and its comparators; exceptions were a greater risk of discontinuation of 56

57 treatment for golimumab than for etanercept (OR=5.14 (1.03, 39.21)), superior reduction in the short term in the disease activity index (BASDAI) for golimumab relative to that for etanercept (MD= (-1.58, -0.14)) and significant improvement in the short term in the Metrology Index (BASMI a measure of changes in spinal movement) for golimumab relative to that for adalimumab (MD= 0.52 (0.23, 0.80)). 57

58 5 COST EFFECTIVENESS 5.1 ERG comment on manufacturer s review of cost-effectiveness evidence State objective of cost effectiveness review. Provide description of manufacturers search strategy and comment on whether the search strategy was appropriate. If the manufacturer did not perform a systematic review, was this appropriate? The objective of the search was to identify published cost effectiveness studies of therapies used in the treatment of ankylosing spondylitis. The search strategies for the cost effectiveness review are discussed in detail in section State the inclusion/exclusion criteria used in the study selection and comment on whether they were appropriate. The main inclusion criteria for the study selection in the cost effectiveness review clarification were not explicitly mentioned in the initial submission. The ERG has asked for such criteria in the clarification letter. The main inclusion criteria mentioned in the response to the clarification letter can be summarized as: (I) the relevance and informative aspects of CEA studies covering all the comparators of golimumab (infliximab was not included as a comparator in the final version of the submission as it is not recommended by NICE), (II) the different approaches to modelling the effects of TNF inhibitors on AS, and (III) studies that were used alongside the previous NICE MTA 22 to inform expected ranges for ICERs and other factors of model development in early stages of the submission. The manufacturer s initial submission did not include an explicit statement on the exclusion criteria in the study selection for the cost effectiveness review. Comment - The ERG found the inclusion criteria to be appropriate. However, the three selected CEA studies reviewed in the submission compared the effectiveness of TNF-α inhibitors to conventional treatment only What studies were included in the cost effectiveness review and what were excluded? Where appropriate, provide a table of identified studies. Please identify the most important cost effectiveness studies. The manufacturer s submission identified three cost-effectiveness analyses for golimumab: Ara et al. (2007), Botteman et al. (2007) and Kobelt et al. (2004) Details of the analyses conducted were 58

59 summarized in Table 67 (p ). However, results from each study were not included in the Superseded table. The ERG has asked for a summary of such studies and they were provided in the response to clarification (Tables 24-26, pg.47-50). Comment - The manufacturer has not included any further see explanations on the studies that were excluded from the cost effectiveness review What does the review conclude from the data available? Does the ERG agree with the updated studies in the economic review included golimumab as a comparator. page conclusions of the cost effectiveness review? If not, provide details. No specific conclusions from the economic review were provided in the submission. In response on the clarification questions the manufacturer provided the results of the economic review. None of the 5.2 Summary and critique of manufacturer s submitted economic evaluation by the ERG An overall summary of the de novo economic model developed by the manufacturer is given in Table 10. Table 10: Summary of the manufacturer s economic evaluation (with signposts to MS) Model States and events Comparators Approach Source / Justification Signpost (location in MS) Cohort model with population Pg defined according to BASDAI and BASFI. Time horizon was 20 yrs in the base case. Health states: on anti-tnf and Response rates came from the Pg. 155 not on anti-tnf (on MTC of trials. Discontinuation conventional) treatment. Initial rate of 15% per year came from response to treatment was the CEA by McLeod at al 2007 determined according to 50% improvement in BASDAI score. Patients who started in the state of on treatment moved to not on anti-tnf according to a single rate of discontinuation. Golimumab, adalimumab, Scope as specified by NICE etanercept and conventional therapy (including NSAIDs and DMARDs) Natural History Mean BASDAI score was assumed to be constant. BASFI score increased over time to model disease progression. The rate was assumed to increase 0.07 units per year on conventional therapy and on anti-tnf treatment. BASDAI and initial BASFI came from the GO-RAISE trial. Rate of increase in BASFI came from an observational study for conventional and from an assumption for anti-tnf treatment. Pg

60 Treatment effectiveness Response to treatment according to at least 50% decrease in BASDAI. MTC of trials: GO-RAISE, ATLAS, Brandt 2003, van der Heijde 2006 Pg Adverse events Single estimate for all anti-tnfs. Stated to be from the GO-RAISE Pg trial report and included any AE, not only SAEs. Approach Source / Justification Signpost (location in MS) Health related QoL QALYs in the base case are estimated based on a regression model provided by McLeod et al., 2007 accounting for BASDAI, BASFI, gender and age. Resource utilisation and costs Categories were: Drug costs, Short-term (first 12 weeks) AS management costs, long term AS management costs and AE treatment costs. The regression model was used by McLeod et al 2007 and was estimated from a cross-sectional postal survey by Kobelt et al Only one cost study was found to be relevant Kobelt et al. (2004). Drug costs were as in the BNF. Short term management costs were based on resource quantities from a physician survey. Adverse event costs were based on AE rates from the GO-RAISE trial. Long-term management costs are based on a regression equation, exponentially depending on BASFI score, estimated by McLeod et al Pg Pg Discount rates 3.5 % for utilities and costs According to NICE reference case Pg. 147 Sub groups No sub-group analysis was undertaken. No specific justifications were given. Pg Sensitivity analysis One-way sensitivity analyses are provided for all major model variables in order to identify model drivers. Numerous scenario analyses were run investigating the effect of changing the base case assumptions Probabilistic sensitivity analysis was also undertaken. Pg The ERG has assessed the manufacturer s economic evaluation using the Philips et al. 41 checklist for quality assessing decision analytic models. This is shown in Appendix 3 and is used to assist the narrative critique in the following sections. In Table 10 the methods used in the manufacturer s model are also compared to those detailed in the NICE reference case. 60

61 5.2.1 NICE reference case checklist Table 11: Comparison of the MS model with the NICE reference case Elements of the economic evaluation Reference Case Included in submission Comparator(s) Therapies routinely used in the NHS, including Yes technologies regarded as current best practice Type of economic evaluation Cost-effectiveness analysis Yes Perspective on costs NHS and PSS Yes Perspective on outcomes All health effects on individuals Yes Time horizon Sufficient to capture differences in costs and outcomes Yes Comment on whether denovo evaluation meets requirements of NICE reference case At a maximum 60.1 years, at which moment 99% of the population has died Synthesis of evidence on Systematic review Yes /No Not for costs outcomes Measure of health effects QALYs Yes Source of data for measurement of HRQL Reported directly by patients and/or carers No EQ-5D based on an algorithm based on BASDAI Source of preference data for Representative sample of the public Yes valuation of changes in HRQL Discount rate Annual rate of 3.5% on both costs and health Yes effects Equity weighting An additional QALY has the same weight Yes regardless of the other characteristics of the individuals receiving the health benefit Sensitivity analysis Probabilistic sensitivity analysis Yes and BASFI scores Despite its adherence to the reference case methodology, the ERG identified a number of shortcomings with the manufacturer s model. These are discussed in more detail in the sections below Model structure To estimate the cost-effectiveness, a Markov model was developed with an initial decision tree. In the base-case cohort model, a decision is made to continue or withdraw from TNF-α inhibitors according to BASDAI response at 12 weeks. In addition to the 12 week BASDAI response decision rule, the model was constructed with the flexibility to allow a 24 week decision rule. After the initial decision tree, patients enter a Markov model with a cycle length of 12 weeks and a time horizon of at a maximum 60.1 years. If patients are on TNF-α inhibitors, they either stay on therapy ( On treatment ), or discontinue due to lack of efficacy or adverse events ( Not on TNF-α ). To model the lower disease activity just after discontinuation of TNF-α inhibitor therapy, two 12-week tunnel states ( Just discontinued and Discontinued ) were incorporated in the model. Patients in the conventional treatment arm enter the Markov model in the Not on TNF-α state. Patients can die at any point in the model. SAEs and injection site reactions of TNF-α inhibitors treatments are included in the model. They are not considered as a separate health state, but their associated costs and disutilities were taken into account in the calculations. A half-cycle correction was performed. A graphical depiction and explanation of the model were provided. 61

62 From the manufacturer submission: The major structural assumptions are: - Patients stopping TNF-α inhibitor treatment due to inefficacy or SAEs are assumed to revert to conventional care (no switching between TNF-α inhibitor treatments). - Patients stay on conventional treatment comprising of a fixed combination of NSAIDs and DMARDs without consideration of response or switching. - The model defines a response to treatment as BASDAI >=50% improvement from baseline. Comments - The assumption is that patients stay on conventional treatment comprising of a fixed combination of NSAIDs and DMARDs without consideration of response or switching. The justification of the Manufacturer of this assumption is that allowing switching into the conventional care pathway would add considerable complexity to the model without adding much analytical and evaluative value (p. 145 MS). However, as the MS is directed to a minimization of the differences between the TNF-α inhibitor treatments, conventional care is the key comparator. Therefore, a more detailed and realistic modelling of conventional care would have been recommended. - The model structure does not allow switching between TNF treatments. The justifications in the MS for this assumption are: 1) it is not recommended based on TA143, 2) no RCT data is available with regard to switching, and 3) that it will only affect a small proportion of the patients. However, in the BSR guidelines 42 it says: failure to maintain the response on both occasions leads to cessation or change of treatment. Also, considering the previous NICE guidance (TA143), conventional treatment no longer is the treatment of choice for this patient population. Moreover, in a recent study 43 it is concluded that switching to a second TNF-α can be effective in AS and can 62

63 be as useful as in rheumatoid arthritis, although overall effectiveness seems to be somewhat lower than in non-switchers. Therefore, in the clarification letter, the ERG requested that the Manufacturer adapt the model for switching to alternative TNF-α inhibitors / sequential treatment, and incorporate lower treatment efficacy in switchers. The response of the manufacturer (in a telephone conference set up by NICE) was that this was not feasible using the Markov model. Instead the manufacturerstated: From the response on the clarification questions: The ERG questions the use of a single discontinuation rate for all TNF-α inhibitors. Therefore, the ERG does not consider this a valid approximation of the potential effect of sequential use - Patients who are, at any time, in the health state On treatment had a BASDAI >=50% improvement during the first 12 weeks of treatment, and do not discontinue. However, as a result of differences in baseline BASDAI, these patients can be subdivided into two categories: patients who are in remission (BASDAI < 4) and patients who are not in remission (BASDAI > 4). It is unlikely that these patients have comparable quality of life, treatments and costs. Potentially, this causes serious heterogeneity in the On treatment health state. - SAEs and injection site reactions of TNF-α inhibitors treatments are not considered as a separate health state, but their associated costs and disutilities were taken into account in the calculations. 63

64 The ERG considers this a less transparent way of modelling the occurrence and consequences of adverse events Population The manufacturer submission mentions that the economic evaluation focuses on the patients with Active AS that were18 years or older and had no history of TB or latent TB (page 141). The definition used complied with the one defined by NICE 19 and which included three main criteria: (I) The patient s disease satisfies the modified New York criteria for diagnosis of AS; (II) There is confirmation of sustained active spinal disease, demonstrated by: a score of at least 4 units on the BASDAI and at least 4 cm on the 0 to 10 cm spinal pain visual analogue scale (VAS). These should both be demonstrated on two occasions at least 12 weeks apart with no change of treatment; (III) Conventional treatment with two or more non-steroidal anti-inflammatory drugs taken sequentially at maximum tolerated or recommended dosage for 4 weeks has failed to control symptoms. The de novo economic model used data from the GO-RAISE trial of golimumab 36, which satisfied all the above criteria but did not state the number of times that conventional treatment has failed (i.e. 2 NSAIDs/DMARDs failures). Comment: - According to the ERG the population is in-line with the scope Interventions and comparators The de novo model developed for this submission compares golimumab (Simponi) 50 mg given once a month, on the same date each month against three different treatments: 1) adalimumab (40 mg adalimumab administered every other week as a single dose), 2) etanercept (25 mg administered twice weekly, or 50 mg administered once weekly), and 3) conventional therapy (non-biologic DMARDs, NSAIDs, Cox-2 inhibitors, and physiotherapy). The reasons mentioned in the submission for selecting these therapies are twofold: 1) these treatments are currently available and used in clinical practice for the treatment of AS in the UK; and 2) these treatments have been considered in the recent NICE appraisal 19. It is furthered argued that even though TNF-α inhibitors are usually administered together with NSAID in clinical practice, these last ones are excluded due to their negligible costs. 64

65 Comment - The manufacturer s submission excludes the possibility of sequential Anti-TNFs for AS. Such use of anti-tnfs may occur in clinical practice in the face of drug toxicity or poor clinical response particularly as these drugs, particularly etanercept, have different mechanisms of action. - The content of conventional therapy was unclear. In the clarification letter the ERG asked the manufacturer to describe the conventional treatment arm in the model, and to provide further explanation of which combinations of physiotherapy and drugs are included. In addition, it was asked to clarify whether the conventional care in the model is compatible with the placebo arm from the trials and whether the conventional treatment for these patients is the same as the conventional treatment they have had an inadequate response to before randomisation? In the response, the manufacturer provided details with regard to the content of conventional care and health care use in the placebo arm of the GO RAISE trial (table in the clarification document), and stated; From the response on the clarification questions: The ERG considers this a questionable conclusion. First, some drugs are used in GO RAISE placebo, but not in the conventional care (for instance, the DMARD hydroxychloroquine (Plaquenil)). On the other hand, many drugs are used in conventional care, and not reported in GO RAISE placebo (for instance the gastro protectants). Furthermore, MTX, sulfasalazine, oral corticosteroids and NSAIDS are all more frequently used in GO RAISE placebo than in conventional care. As a result, it is unclear whether conventional care in the model reflects the GO RAISE placebo arm Perspective, time horizon and discounting The manufacturer s model applied 20 years time horizon and the model s cycle length is 12 weeks in the base case scenario. This choice was justified based on a previous MTA accepted by NICE (NICE TA 143).In the sensitivity analysis the time horizon could be varied up to lifetime. Lifetime was defined as the period over which 99% of the cohort would have died. The model was developed to use a maximum time horizon of 60.1 years. The discount rate applied was 3.5% for utilities and costs and costs are considered from an NHS and Personal Social Services perspective (pg. 147, Table 68). 65

66 Comment - The discount rates and perspective are in line with the NICE reference case. - In the base case the time horizon is 20 years, however golimumab is a lifetime treatment. Therefore, the ERG conducted an analysis with 60.1 year time horizon as a close approximation of a lifetime time horizon Treatment effectiveness Short term response on treatment The economic evaluation uses BASDAI as the primary short-term clinical outcome and response criteria. A 50% reduction in the BASDAI was used to identify responders at 12 weeks (24 weeks in a sensitivity analysis). Patients responding to treatment were allowed to continue on the TNF-α inhibitor treatments. Non-responders were withdrawn from treatment and switched to conventional treatment. In the base case, the short-term BASDAI50 results were taken from a mixed treatment comparison of adalimumab, etanercept and golimumab for the treatment of AS using conventional treatment with placebo as the common comparator. The studies included in the MTC are: ATLAS trial, Brandt et al, GO-RAISE trial and Van der Heijde et al. From the manufacturer submission: Comments - According to the ERG the analysis that uses the 24 week period to identify responders is not 66

67 valid. The principal reason for excluding 24-week data is due to the early escape open-label treatment options programmed into the golimumab and adalimumab studies. - The ERG questions the use of BASDAI50 as a measure of response. Although the ERG is aware that BASDAI50 is a generally accepted measure of response, ASAS20 is the primary measure of response in all trials, and is the only measure of response that is reported in all trials. The ERG has performed additional analyses with ASAS20 as measure of response (see paragraph 5.3). - The study by Brandt et al. reports main outcomes at 6 weeks follow-up: After week 6, patients in the placebo group were switched to etanercept for the next 12 weeks, and patients in the etanercept group continued to receive etanercept for another 6 weeks, to ensure that all patients received etanercept for a total of 3 months. Therefore, data at 12 weeks follow-up are underestimating the effects of etanercept because all patients in the placebo have received etanercept for 6 weeks. The ERG group believes that 6-week data are probably a better reflection of the true effect of etanercept in the study by Brandt et al. The ERG has performed additional analyses with Brandt et al 6 week data in the MTC (see paragraph 5.3). Long term response on treatment For the long-term TNF-α inhibitor treatment is re-evaluated every 12 weeks and in case of inefficacy or SAEs, patients discontinue the treatment. 42 The proportion of patients discontinuing due to inefficacy and SAEs is expressed in a discontinuation rate. In the base case a combined discontinuation rate for each TNF-α inhibitor treatment from the literature was incorporated (15% annually). As a result, adverse events and discontinuations were assumed the same for all TNF-α inhibitors. The latter was justified in the MS by: - very few data points for AEs and discontinuations in the trial data, - hence, the need for pooling of 12 and 24 week data, - unavailability of data on the long term rate of SAEs requiring treatment cessation and the proportion of patients not responding to the treatment are not available. From the manufacturer submission: 67

68 The relative risks of treatment discontinuation from the MTC were however presented and used in a sensitivity analysis. From the manufacturer submission: Comment - The ERG discovered an error in the discontinuation rate in the model. This value is treated as a rate while it is a probability. Furthermore, the annual probability is used for a cycle (12 weeks). The ERG corrected this, which did not alter the results substantially. - The ERG considered not using MTC results for discontinuations but assuming these are the same for all TNF-α inhibitors (15%) very arbitrary. Glindborg et al found that long term discontinuation rates tend to differ between different TNF-α inhibitor treatments. 44 The ERG requested that in the base case the discontinuations were based on the MTC. In their response to the clarification letter the manufacturer included an analysis with discontinuations based on the MTC results. From the response on the clarification questions 68