Review of methodological issues in costeffectiveness analyses relating to injecting drug users, and case-study illustrations

Transcription

1 J. R. Statist. Soc. A (2014) 177, Part 3, pp Review of methodological issues in costeffectiveness analyses relating to injecting drug users, and case-study illustrations Simon R. White and Sheila M. Bird Medical Research Council Biostatistics Unit, Cambridge, UK and Richard Grieve London School of Hygiene and Tropical Medicine, UK [Received Januuary Revised May 2013] Summary.Guidelines for good practice in cost-effectiveness analyses (CEAs) are available from the UK s National Institute for Health and Clinical Excellence, which address the scope of the CEA study, appropriateness of the data used and how to account for model as well as statistical uncertainty. Within these three broad headings, we identify 10 specific issues that may affect materially the CEA of a public health intervention for which injecting drug users are a major target group. The 10 issues relate to injecting drug users under-representation in randomized controlled trials; their risk of blood-borne viruses such as human immunodeficiency virus and hepatitis C virus which have long-term chronic sequelae, their markedly higher age-specific mortality than in the general population and the relapsing remitting nature of opiate dependence with its associated risk of overdose. We consider how adequately three key UK CEAs accounted for the relevant injecting drug user specific issues that we have highlighted. The three casestudies are antiviral treatment of carriage of hepatitic C virus, opiate substitution therapy and needle and syringe programmes. Keywords: Blood-borne viruses; Cost-effectiveness analysis; Drug-related deaths; Injecting drug users; Model uncertainty; Parameter uncertainty 1. Introduction Cost-effectiveness analysis (CEA) can be used to determine whether the implementation of a particular treatment or public health policy (e.g. provision of additional services) is likely to be cost effective in terms (typically) of cost per quality-adjusted life-year (QALY). We are concerned with injecting drug users (IDUs), who are a group that is often underrepresented within randomized controlled trials (RCTs) for the public health treatments and interventions for which they are a major target group, either because IDUs do not volunteer or are excluded entirely by design to avoid missed appointments and confounding. IDUs have been followed up via record linkage studies and their sex and age-specific morbidity and mortality are known to be significantly higher than in the general population (King et al., 2009; McDonald et al., 2009; Merrall et al., 2012), even after allowance for their relative deprivation. Injection sharing makes IDUs vulnerable to blood-borne viruses which have chronic sequelae, and their injecting of opiates whether with a sterile or contaminated needle puts them at acute risk Address for correspondence: Simon R. White, Medical Research Council Biostatistics Unit, Institute of Public Health, Forvie Site, Robinson Way, Cambridge, CB2 0SR, UK. simon.white@mrc-bsu.cam.ac.uk 2013 Royal Statistical Society /14/177625

2 626 S. R. White, S. M. Bird and R. Grieve of fatal overdose; see, for example, the special issue in 2009 of Statistical Methods in Medical Research on hepatitis C virus (HCV) and injection drug use (Bird, 2009). We identify a series of methodological issues in CEAs relating to IDUs. These methodological issues are then illustrated via three CEA case-studies on which the National Institute for Health and Clinical Excellence (called the National Institute for Health and Care Excellence in 2013) has relied in making recommendations for health technologies and public health interventions that are relevant to IDUs. The three case-studies concern antiviral treatment for chronic HCV infection (Grishchenko et al., 2009), opiate substitution therapy with methadone or buprenorphine (West Midlands Health Technology Assessment Collaboration, 2006) and the public health policy of needle and syringe programmes (Vickerman et al., 2008). For the assessment report in each case-study, the National Institute for Health and Clinical Excellence formally defined the scope of any CEA to be included so that researchers (specially commissioned or otherwise) could refer to the scope when building their cost-effectiveness model: IDUs were highlighted as a subgroup of interest in all three scopes. When performing CEA, there are three broad concerns: the motivating question, typically from policy makers; available evidence; the analysis that the researchers perform and its uncertainty. The 10 key issues for our review relating to IDUs are grouped under these three broad themes and relate to IDU s under-representation in RCTs, their risk of blood-borne viruses, their higher age-specific mortality and the relapsing remitting nature of opiate dependence with its associated overdose risk. We first review Grishchenko et al. (2009) against the methodological issues identified, presenting details of the study design, available data and statistical modelling. We then illustrate the potential effect on the CEA of not fully accounting for IDU-specific model parameters, e.g. IDUs other cause mortality. We highlight how model structure based on simplifying but incorrect assumptions about IDU-specific epidemiology could impact on the applicability of results to IDUs. Our two other case-studies are reviewed more briefly and issues within each are again described under the above three broad themes. Our CEA case-studies highlight failure to account fully for the epidemiology of HCV, the potential long-term benefits of opiate substitution therapy or an ambitious scope which encompassed multiple facets of needle and syringe programmes but led to a cost-effectiveness model with inadequate consideration of model and parameter uncertainty. We conclude with some remarks on all three case-studies, and recommendations for future CEAs relating to IDUs. 2. Methodological issues in cost effectiveness analyses relating to injecting drug users Three broad themes are the scope of the CEA study, the available evidence and accounting for uncertainty. These themes, under varying terminology, are commonly referred to in the CEA and decision modelling literature (Drummond and McGuire, 2001; Philips et al., 2004; Drummond et al., 2005; Briggs et al., 2006; Jackson et al., 2011; Bilcke et al., 2011). Within these themes, we emphasize 10 issues that are relevant to IDUs, some of which are common to many CEAs. The themes give structure to our subsequent critique of the three case-studies Scope of the cost-effectiveness analysis study When analysing the cost-effectiveness of an intervention, the scope of the decision making must be clearly defined so that the researcher can provide the appropriate information to policy makers. A clearly defined problem is necessary to develop a cost-effectiveness model which

3 Methodological Issues in Cost-effectiveness Analyses 627 is capable of delivering an answer, and to provide a framework for critiquing the model and analysis. In cases where policy makers define the scope, and researchers follow their guidelines, then care must be taken to ensure that the appropriate question is being analysed. When the National Institute for Health and Clinical Excellence commissions research, e.g. a health technology assessment including a CEA, a formal scope is produced for researchers to follow, outlining aspects that are important, as well as further considerations. In particular, the scope will define the target population and typically any subgroups of interest, the outcome measures that are important and an appropriate timescale for effectiveness, and any supplementary perspectives beyond the costs to the National Health Service (NHS) and Public Social Service (PSS). Issues related to the scope of a CEA include the following. (a) Mismatched target population: the planned scope and effectiveness of an intervention can only be assessed if the policy s target population is both clearly defined and included within the analysis. There may be poor linkage between the target population and the inclusion criteria for available RCTs. For example, RCTs for antiviral therapy in HCV often exclude IDUs. Thus, any derived CEA may not reflect truly the effect of antiviral therapy for HCV-infected IDUs, despite their being the majority of HCV infectees in developed countries. (b) Mismatched RCT and IDU time horizons: injecting behaviour remits and relapses over a prolonged period (measured in quinquennia, if not decades) before cessation by being finally off injecting or by premature death. Relapses, if precipitate, carry increased risk in the short term of overdose death from reduced tolerance for heroin. Neither the duration of nor the numbers randomized in RCTs of interventions to reduce the harms that IDUs encounter incident blood-borne viruses, overdose death, multiplicity of crimes committed, unemployment are typically sufficient to determine effectiveness at reducing incident blood-borne viruses or overdose deaths, both of which compromise IDUs QALYs quite substantially. Thus short time horizons, e.g. of 1 year only, are likely to exclude important incremental QALY effects of such interventions. (c) Perspective: in England and Wales, the Home Office is the lead department for drug policy, as much on account of the crimes committed by drug users to pay for illegal drugs as because of responsibility for the legislation which has criminalized their possession. However, for the most part, it is public health interventions that have succeeded in reducing IDUs crime in the short term whereas health-related harms are either late sequelae, such as liver cirrhosis or acquired immune deficiency syndrome from blood-borne viruses, or acute but low frequency such as drug-related deaths (DRDs) at a rate of 0.5 1% per annum (Hutchinson et al., 2005). Thus, perspective (i.e. whose costs to take into account) and time horizon affect cost-effectiveness quite dramatically. Technology assessments for the National Institute for Health and Clinical Excellence are typically focused on costs to the NHS and PSS, whereas public health interventions allow a public sector perspective also, e.g. to include the costs to the criminal justice system Appropriate data The National Institute for Health and Clinical Excellence recommends that CEA models use input parameters which are derived from the best available evidence, where best encompasses both high quality and robust data as well as relevance to the target population. Thus, if IDUs comprise the main target population, they should ideally constitute the main group for whom RCTs were designed. The model determines the data required and, where data are meagre or

4 628 S. R. White, S. M. Bird and R. Grieve lacking, can be used to indicate the expected value from further research; see Claxton et al. (2001). Accounting for potential bias in relation to the target group is distinct from the often encountered methodological issue of how to combine data from multiple studies which give evidence on a common set of input parameters, or function of parameters. When the evidence incorporates data from dissimilar studies, perhaps due to non-directly comparable sample populations or study designs, alternative forms of evidence synthesis, either frequentist (Turner et al., 2009) or Bayesian (Spiegelhalter and Best, 2003), define a framework (Welton et al., 2012) in which external as well as internal biases are weighed in the balance. Evidence synthesis of dissimilar studies seeks to identify and resolve conflicts between studies which give inconsistent estimates of input parameters. However, the assumptions that are made to enable evidence synthesis do not replace the need for direct IDU-specific data. The lack of appropriate evidence on IDUs can be seen in issues related to the following. (d) IDU-specific effectiveness: as mentioned previously, IDUs are often excluded from RCTs of therapeutic interventions, even those for which IDUs or former IDUs may be the major target group, on the basis of anticipated higher treatment drop-out rates and higher rates of loss to follow-up. High incarceration and DRD rates of around a third and 1% per annum respectively (Bird and Hutchinson, 2003; Hutchinson et al., 2005) account for trialists wariness. (e) Efficacy and health-related quality of life (HRQOL): evidence on efficacy and HRQOL are typically taken from RCTs (Billingham et al., 1999) which, for the most part, apply to non-idus, yet CEAs applicable to IDUs require us to estimate effectiveness (in QALYs) for IDUs whose off-therapy HRQOL may be substantially lower than for individuals of the same age who have never injected but who acquired their blood-borne virus infection (HCV, for example) by other means, such as a contaminated blood transfusion. (f) IDUs mortality from causes other than DRD: even the risk of mortality from causes other than DRD is raised for IDU and former IDUs, a high proportion of whom are, or became, socially disadvantaged so that their age-specific non-drd mortality rate is several times higher than for their general population counterparts (Merrall et al., 2012) Accounting for uncertainty The use of probability sensitivity analysis (Claxton et al., 2005; Baio and Dawid, 2011) to investigate input parameter uncertainty is recommended practice (Philips et al., 2004; Andronis et al., 2009), and some commentators have advocated a fully Bayesian approach (Spiegelhalter et al., 2000). Whatever approach is taken, it is important not only carefully to report parameter uncertainty, but also to consider model or structural uncertainty which is rarely investigated. Of particular importance in relation to IDUs are blood-borne viruses, and any assumptions that are used to incorporate blood-borne viruses in, or to exclude them from, the model structure should be investigated. If model structure uncertainty is substantial, it may be necessary to use model averaging techniques (Claeskens and Hjort, 2008) to represent fully the uncertainty in the estimates. Although model uncertainty, as a general theme, encompasses many of the issues we emphasize the following IDU-specific issues. (g) Incidence versus prevalence (and progression) of blood-borne viruses: unless there is sustained, high uptake of voluntary testing for blood-borne viruses among susceptible IDUs, blood-borne viruses carriers are unlikely to know when they became infected, and yet progression rates, such as for HCV, depend on age at infection. Progression is generally faster

5 Methodological Issues in Cost-effectiveness Analyses 629 in those who were infected at an older age, and treatment-assisted viral clearance rates may be different for incident or prevalent infections. Systematic reviewing of infectious disease models for blood-borne viruses is a critical starting-point for CEAs. For human immunodeficiency virus (HIV) disease, sexual as well as injection-related transmission matters. Maternal transmission has been reduced dramatically by universal offering of antenatal HIV screening and highly active anti-retroviral therapy for maternal HIV disease disease. For HCV carriers, the risk of sexual transmission is low, but around a 6% HCV transmission rate applies from an HCV carrier mother to her infant (Gibb et al., 2000; Hutchinson et al., 2004). (h) Reinfection rates after clearance of HCV carriage: whether patients have a recent history of injection drug use can matter when it comes to sustained clearance of HCV carriage because those who relapse into injecting may become reinfected (Dalgard et al., 2002). Realistic CEA in respect of HCV clearance for patients with a history of injecting, let alone those who are current IDUs, must take reinfection rates into account a consideration that does not apply to patients who were HCV infected by contaminated blood before HCV screening of the blood supply: in the early 1990s in the UK. (i) Heterogeneity: in common with most CEAs, it is prudent to consider whether there are cost-effective subgroups of IDUs. For example, anti-retroviral treatment for mild HCV disease may be more cost effective for former IDUs, who are no longer at risk of reinfection, or for those who were older at HCV infection because their HCV progression rate is expected to be more rapid. (j) Generality of model structure: when modelling blood-borne viruses, the incidence, reinfection, coinfection and transmission rates, as well as sex- and infection-age-specific progression rates, need to be taken into account, many of which also vary with calendar time because of the evolution of harm reduction and therapeutic interventions. Together, these transitions define the dynamics of the target population. Hence, changes in the dynamics may significantly impact on the cost-effectiveness of an intervention over a given time horizon. The interaction between the model dynamics and the outcome of interest is typically highly non-linear and potentially time dependent. Notably, for example, interventions such as the provision of clean needles aim to reduce the risk of blood-borne viruses but do not impact directly on the risk of overdose death unless, by prolonging injecting careers, they indirectly increase it. Thus, cessation rates and the route out of injecting by remission, recovery or death matter. The extent to which opiate substitution therapy reduces IDUs commission of crime may (or may not) depend on whether illicit heroin was injected or taken by a non-injection route. Likewise, the extent to which former IDUs age-specific non-drd mortality rate is reduced compared with that of their IDU counterparts is poorly evidenced. Implementation of more realistic CEA models may be compromised by a meagre evidence base so sensitivity analyses on both model structure and inputs may need to be quite extensive. 3. Pegylated interferon and ribavirin for patients with chronic hepatitis C virus lacking injecting drug user specific model inputs Grishchenko et al. (2009) considered the cost-effectiveness of providing antiviral treatment with pegylated interferon and ribavirin for patients with chronic HCV. The study of Grishchenko et al. (2009) was based on effectiveness and resource use data from patients in the Trent cohort who received pegylated interferon and ribavirin, in which former IDUs constituted 59% of

6 630 S. R. White, S. M. Bird and R. Grieve the cohort (205/347). However, they did not consider current IDUs as a subgroup; only three patients in the Trent cohort were current injectors. We demonstrate the potential effect of just one specific aspect arising from the exclusion of current injectors, by allowing all-cause mortality to be 2 5 times higher than for the general population, in line with estimates for current IDUs (Hickman et al., 2007; Cohort Studies on Mortality of Opiate-users, 2010; Bird, 2010; Merrall et al., 2012) Overview The CEA model by Grishchenko et al. (2009) extended a previously published cost-effectiveness model on treating mild chronic HCV (Grieve et al., 2006). Both CEAs essentially assessed whether the higher initial costs from antiviral treatment were justified by sufficient improvements in QALYs and reductions in morbidity costs from preventing HCV disease progression. The study of Grishchenko et al. (2009) found that the intervention was cost effective, except for genotype 1 patients with cirrhosis aged over 50 years at presentation for treatment. For this subgroup, the probability of success of treatment was lower, and the QALY gains from antiviral treatment were found to be insufficient compared with the cost of treatment Scope Grischenko et al. (2009) clearly stated that the aim was to assess whether pegylated interferon and ribavirin versus no antiviral treatment was cost effective for patients at different stages of chronic hepatitis C: mild disease, moderate disease or cirrhosis. Rather than basing cost-effectiveness on RCTs which tend to include atypical patients and centres, the study assessed cost-effectiveness in the real world setting of routine clinical practice. The study design therefore extended the earlier CEA of antiviral treatment for mild chronic HCV (Grieve et al., 2006), whose scope was explicitly restricted to populations represented by those patients included in RCTs, and thus excluded both current and former IDUs. Grishchenko et al. (2009) were not explicit about excluding current IDUs from the scope, which was a major exclusion in view of HCV reinfection risk. The National Institute for Health and Clinical Excellence scope for the technology appraisal of peglated interferon and ribavarin listed current IDUs as a subgroup of particular interest that should be considered explicitly if there was sufficient evidence (National Institute for Health and Clinical Excellence, 2006). Patients who are coinfected with HIV were also excluded, which was a lesser consideration because, even in Scotland, they represent only 4.2% of diagnosed HCV infections (McDonald et al., 2009). Grishchenko et al. (2009) considered costs and outcomes over the patients lifetime, which allowed for the possibility of any costs or consequences pertaining to behavioural change in IDUs to be incorporated. For example, this time frame would allow for the long-term consequences of reinfection for those patients who were predicted to have a virological response but who continued to inject. Grishchenko et al. (2009) followed the cost perspective that was recommended in the National Institute for Health and Clinical Excellence s methods guidance for health technology assessment and considered costs to the NHS and PSS. Hence, any broader societal costs or gains following successful treatment, e.g. from increased productivity or reduced costs of incarceration, were not considered. A multistate Markov model was constructed to predict lifetime costs and QALYs by following individuals progression through each stage of HCV disease: Fig. 1. The cycle length was 1 year; individuals at each disease stage faced the probability of remaining in the same state, moving to a connected state or dying. All individuals entered the model in one of three disease states:

7 Methodological Issues in Cost-effectiveness Analyses 631 x2 x5 (a) (b) Fig. 1. Diagram of multistate Markov cost-effectiveness model (a) as presented by Grishchenko et al. (2009) and (b) with two amendments to allow for the IDU-specific issues: higher all-cause mortality rates and reinfection of current IDUs who achieved sustained virological response (, absorbing states in the model;, individuals entering the model;, non-absorbing states transition to all-cause mortality) (SVR, sustained virological response; HCC, hepatocellular carcinoma)

8 632 S. R. White, S. M. Bird and R. Grieve mild disease, moderate disease or cirrhosis; with further disease states for hepatocellular carcinoma, decompensated cirrhosis, liver transplantation and post transplantation. From all states, individuals faced a probability of other cause death, and a probability of liver-related death from the hepatocellular carcinoma, decompensated cirrhosis and post-liver-transplantation health states. As a result of treatment, some patients have a sustained virological response and no longer carry the HCV antigen. The model assumes that these patients no longer face the probability of disease progression and are not HCV reinfected. To assess the cost-effectiveness of pegylated interferon and ribavirin versus no antiviral treatment, two main cohorts were simulated through the model: a control cohort who did not move into the antiviral treatment state, and a treatment cohort, all of whom were treated with pegylated interferon and ribavirin. The model considered separate subcohorts to represent different subgroups according to patients characteristics at treatment presentation: sex, age (30, 40 or 50 years old), genotype (1 or non-1) and disease state (mild disease, moderate disease or cirrhosis). Individuals progressed through the model until death from liver disease or other causes. The model monitored the predicted time in each state, which was weighted by the subgroup-specific costs and HRQOL corresponding to each health state. Finally, the model projected the average lifetime costs and QALYs following antiviral treatment versus control and considered the parameter uncertainty surrounding these estimates Data To address the original study s aims, the model required subgroup-specific data (e.g. separately for current and former IDUs and never IDUs) to evidence the following parameters: sustained virological response rates following pegylated interferon and ribavirin treatment, costs of antiviral treatment, HRQOL and costs for each health state, transition probabilities of moving between each health state and liver-related and other cause mortality. To broaden the scope to estimate cost-effectiveness specifically for current IDUs would ideally have required parameter estimates that were specific to this subgroup, but data for all these parameters were not available. For example, at the time of the study of Grishchenko et al. (2009), HRQOL data specific to IDUs were not available; this data deficiency is now being addressed by Scotland s needle exchange surveillance initiative for IDUs. An important aspect of the multistate model of Grishchenko et al. (2009) is mortality rates, since any QALY gain is affected by the life expectancy, wherein the higher mortality rate for current IDUs may be important. We illustrate the effect that using IDU-specific input parameters can have on cost-effectiveness in Table 1. Here we draw on evidence on the excess mortality for current IDUs versus the age- and gender-matched general population (Hutchinson et al., 2005; Merrall et al., 2012) and re-estimate cost-effectiveness by using other cause mortality rates from each health state that are two or five times that of the general population. Treatment effectiveness can differ when translated from RCTs into routine clinical practice. Grishchenko et al. (2009) reviewed the available evidence and considered effectiveness evidence from systematic reviews of published trials, from which both current and former IDUs were typically excluded. A major concern was that this source of data was not relevant to the scope of the CEA: Grishchenko et al. (2009) anticipated that taking evidence on the effectiveness of antiviral treatment from RCTs would overstate the effectiveness of antiviral treatment in routine practice where a high proportion of those presenting for antiviral treatment for HCV are current or former IDUs. Instead, the Trent HCV cohort, which included former but not current IDUs who received antiviral treatment, was used to derive estimates of effectiveness, which transpired to be similar to those reported in previous systematic reviews of antiviral treatment

9 Methodological Issues in Cost-effectiveness Analyses 633 Table 1. Effect of increased mortality rates for current IDUs on estimated cost-effectiveness of pegylated interferon and ribavirin for HCV: CEA with two and five times higher mortality rates compared with the base case of Grishchenko et al. (2009) Base case rates 2 mortality rates 5 mortality rates Mild Moderate Cirrhosis Mild Moderate Cirrhosis Mild Moderate Cirrhosis Life years Treatment Control QALY Treatment Control Cost Treatment Control Cost per QALY ( ) Point estimates for the cost per QALY are shown, taken from the probability sensitive analysis. effectiveness in RCTs (Shepherd et al., 2007). Similarly, the costs of antiviral treatment were taken from the Trent HCV cohort and the HRQOL associated with being in each health state was from a previous study which had assessed HRQOL at each stage of disease for patients who were former but not current IDUs (Wright et al., 2006). Finally, the transition probabilities between model states were synthesized from multiple sources. The probability of sustained virological response from each disease state and transition rates for early disease states were derived from the Trent HCV cohort, which was the best of three cohorts considered by Sweeting et al. (2006), as IDUs were under-represented even in the register of HCV diagnoses for England and Wales. Event-biased referral (Fu et al., 2007) to the Trent HCV cohort, whereby those whose HCV progression to cirrhosis is faster are more likely to be referred, was not specifically addressed. To assess uncertainty, probability sensitivity analysis was performed on the input parameters, accounting for the uncertainty of the input parameters for effectiveness, costs, HRQOL and transition probabilities. For each parameter, Grishchenko et al. (2009) followed general guidance on the choice of input parameters (Briggs et al., 2006). Specifically, for the effectiveness and transition probability parameters, which are constrained to be on a scale from 0 to 1, the beta distribution was chosen. HRQOL, which is anchored on a scale from 0 to 1, was also resampled from beta distributions and, for non-negative costs, gamma distributions were used. Monte Carlo sensitivity analysis was undertaken by sampling 5000 multivariate inputs from these distributions. The results were summarized by reporting mean measures of incremental cost-effectiveness, such as the incremental costs per QALY that are reported in Table 1, but also by reporting measures of parameter uncertainty summarized as cost-effectiveness acceptability curves. Several model assumptions were also assessed including whether the gain in HRQOL following sustained virological response was absolute or relative, the duration of monitoring following sustained virological response, increased all-cause death rates for patients with cirrhosis and reduced transition probability to hepatocellular carcinoma following treatment without

10 634 S. R. White, S. M. Bird and R. Grieve sustained virological response. In this regard, Grishchenko et al. (2009) are to be commended for assessing both parameter and model uncertainty Issues relating to injecting drug users: model structure The HCV model of Grishchenko et al. (2009) considered cost-effectiveness over the lifetime of an individual, yet it did not consider the injection-related transmissibility of HCV, specifically the additional benefit of retarding the spread of HCV by treating current IDUs who are infected. Nor did the study consider the risk of reinfection, e.g. by relapse into injecting drug use after sustained virological response (see Fig. 1). Although a negligible concern for non- IDUs, reinfection may be a significant risk for those who currently inject. Hence, Grishchenko et al. (2009) may have underestimated the cost-effectiveness due to the additional benefit of controlling the HCV epidemic or overestimated by not adequately accounting for relapse into injecting. Grishchenko et al. (2009) explicitly acknowledged that the choice of parameter estimates was based on the available studies, and they noted the possibility of higher other cause mortality for the excluded groups, including IDUs. However, there was little discussion of how the model would need to be extended for a target population of current IDUs. In particular, they did not make clear how they would consider IDU-specific reinfection rates and adherence to treatment. In the Markov model, successful sustained virological response is associated with no further disease progression, which is consistent with their implicit exclusion of current IDUs from the scope. To estimate carefully the relative cost-effectiveness of antiviral therapy for current IDUs would require structural changes to the model to allow for reinfection among current IDUs, as well as an assessment of the effectiveness of antiviral treatment on repeated infections. For current IDUs, for whom full compliance with regular treatment may be unrealistic, there are issues concerning the effectiveness of partially completed treatments. Furthermore, it would be necessary to estimate HRQOL, cost and disease transition parameters specific to the current IDU population Issues relating to injecting drug users: injecting drug user specific mortality Grishchenko et al. (2009) incorporated evidence that was specific to former IDUs to derive transition probabilities. However, it is unclear whether they apply to current IDUs. Moreover, they failed to consider IDU-specific mortality (see Fig. 1). To give an indication of the effect of an increased mortality rate for IDUs, by a factor of between 2 and 5 compared with the all-cause mortality rates in Grischenko et al. (2009), their CEA was repeated(see Table 1) for genotype 1 patients aged 40 years at presentation for treatment. The assumed range for the excess in all-cause mortality is in line with estimates for drug treatment clients, IDUs, ex-prisoners or the socially deprived(bird and Hutchinson, 2003; Hutchinson et al., 2005; Bloor et al., 2008; Merrall et al., 2012). With increased mortality rates, the expected gain in life-years and QALYs following antiviral treatment are of course reduced and this led to a substantial increase in the incremental cost per QALY. For example, with a fivefold multiplier for mortality rates, the incremental cost per QALY for 40-year-old patients with cirrhosis is above the threshold which tends to provoke a positive recommendation from the National Institute for Health and Clinical Excellence. It can be expected that the intervention would look even less costeffective if lower IDU-specific HRQOLs had been considered, and if HCV reinfection rates were incorporated. Hence, it is important that all input parameters are IDU specific, or the uncertainty from not using target-group-specific parameters is explored.

11 Methodological Issues in Cost-effectiveness Analyses Methadone and buprenorphine for the management of opioid dependence lacking injecting drug user model structure A key document for National Institute for Health and Clinical Excellence (2007) was the West Midlands Health Technology Assessment Collaboration s (2006) cost-effectiveness report, which aimed to determine whether opiate substitution therapy by methadone maintenance therapy (MMT) or buprenorphine maintenance therapy (BMT) was more cost effective than no treatment of opioid dependence, and, of the two, which was the more effective in head-to-head comparison. The health technology assessment report drew on a range of RCTs, observational studies and registries or self-report databases Overview A decision tree model was constructed for the first year of treatment which considered MMT, BMT or no treatment at the start point, and it had five subsequent branches for retention in treatment at 2 weeks, 6 weeks, 13 weeks, 25 weeks and 12 months. Thus, the model required treatment-specific costs, benefits (in terms of QALYs) and retention rates for each stage as well as assumptions for the no-treatment branch. Individuals in the cost-effectiveness model were defined as receiving treatment or not and actively injecting or not, with 44% and 61% respectively of those in treatment or not in treatment estimated to be currently injecting (Gossop et al., 2003) Scope The scope by National Institute for Health and Clinical Excellence (2007) was primarily focused on the comparison of methadone and buprenorphine, particularly the appropriate dose and treatment strategy. The cost-effectiveness modelling was to have a time horizon that was appropriate for the technology, which was stated as the appraisal will consider both the short and longer term, and drug-related mortality was listed as an outcome of interest. The West Midlands Health Technology Assessment Collaboration cost-effectiveness model (West Midlands Health Technology Assessment Collaboration, 2006) considered a 1-year horizon only and modelled retention in treatment. The Collaboration indicated that there were neither data nor time to fit a dynamic infectious disease component. In particular, HIV and HCV incidences were omitted from the model and costings; and no consideration was given to whether opiate substitution therapy s effectiveness waxed or waned beyond the first year of treatment. Similarly, owing to the short time horizon (1 year), which had been dictated by the limited follow-up in, and recruitment to, RCTs, no assessment of mortality was included in the costeffectiveness model, although the Collaboration commented on this limitation and noted that BMT may have lower mortality compared with MMT (Luty et al., 2005). In particular, no scenario analyses were conducted on this aspect. The 1-year horizon, omission of HIV and HCV incidence and morbidity, let alone of overdose mortality, were surprising given that opiate substitution therapy aims to reduce injection of illegal drugs, and thereby blood-borne virus transmission and injector-related risk of DRD. The primary focus of the National Institute for Health and Clinical Excellence s scope was to determine appropriate dose levels for opiate substitution therapy, and to compare MMT and BMT. However, the wider issues relating to opioid dependence may not be easily detached when considering the relative cost-effectiveness of treatments. In terms of costs, the reference case was from the perspective of the NHS or PSS and included, for example, counselling, urine testing for illegal drugs, accident and emergency department visits and overnight hospital stays. An additional non-reference case included criminal justice

12 636 S. R. White, S. M. Bird and R. Grieve costs. The data indicated, apparently counterintuitively, that those on treatment have a higher rate of criminal activity. However, there may be biased recruitment into treatment of the more dependent users. The sensitivity analysis showed that excluding the associated victim costs altered costs significantly but, as in the reference case, the QALY difference between MMT and BMT was minimal Data The 1-year model did not include mortality although, internationally, evidence suggests that the DRD rate may be low in countries which adopted BMT (Janssen, 2009). However, UK evidence is unreliable because, unlike methadone, buprenorphine was not routinely tested for at post mortems for suspected DRDs (Strang et al., 2010). Retention rates were estimated by fitting Kaplan Meier survival curves to trial data on retention in BMT, the less common of the two opiate substitution therapies in the UK. The MMT curve was derived by using a pooled hazard ratio estimated from seven head-to-head trials. These relative retention rates were derived by randomized allocation to treatment. Specifically, there was no accounting for varying retention among subgroups, such as if the individual was injecting still at 13 weeks. Noting the lack of quality-of-life data for drug users, an additional study was undertaken by the Peninsula Technology Assessment Group of injecting and non-injecting drug users (n = 108) to obtain quality-of-life measures. The model s data requirements had led to this additional research Uncertainty The West Midlands Health Technology Assessment Collaboration had hoped to consider individuals entering treatment in several subgroups, namely homeless, comorbidity, by age, pregnant women, injector status, HIV status, sex and setting (healthcare or prison). However, the data were insufficient for many of these subgroups or, as in the case of HIV status and comorbidity (either physical or mental illness), such individuals were generally excluded from RCTs of opiate substitution therapy. Such heterogeneity within the IDU population may have a significant effect on treatment retention, particularly whether the individual had been, or was currently, incarcerated. 5. Needle and syringe programmes: providing people who inject drugs with sterile injecting equipment lacking evidence on transmission and behaviour probabilities Vickerman et al. (2008) wrote a cost-effectiveness report as background for the National Institute for Health and Clinical Excellence s public health policy guidance on needle and syringe programmes (National Institute for Health and Clinical Excellence, 2009). Their report addressed how cost-effective needle and syringe programmes were in terms of costs to the NHS or PSS versus the resulting gains in QALYs. Several needle and syringe programme policies were considered, and the relative merits of each policy were presented. It is interesting to refer back to the report on MMT and BMT that was discussed in Section 4, since part of the additional harm reduction services offered by needle and syringe programmes was encouragement of IDUs to attend for opiate substitution therapy. Unfortunately, the evidence to support such encouragement was insufficient and could not be adequately quantified, so potential effectiveness was explored in several scenario analyses. In the main, the report

13 Methodological Issues in Cost-effectiveness Analyses 637 considered the effect of reducing risky sharing behaviour in IDUs, classified as no sharing, low (1 4 times in the previous 4 weeks) and high (more than four times in the previous 4 weeks), and thus reducing their chance of blood-borne virus infection, notably HIV, HCV and hepatitis B virus Overview Vickerman et al. (2008) combined relatively sophisticated epidemic models for HIV and HCV, which they had used previously, with evidence on the costs and benefits of the possible disease and behaviour states within the models. Within the coupled model were parameters for bloodborne virus incidence rates, cessation of injecting rates and sharing characteristics, for which there was insufficient evidence to estimate directly and so historical blood-borne virus prevalence data were used to calibrate (estimate) these model parameters. Owing to the focus on risky needle sharing behaviour, there was no accounting for the sexual transmission of HIV within the model. HCV has a low risk of sexual transmission but HCV carriage can be transmitted from mother to child (maternal transmission). Several types of individual were considered in the model, by duration of their injecting career, either new injectors (incident) or experienced IDUs (prevalent), and the IDUs needle sharing behaviour as no, low or high, with increased risks for those with high sharing Scope The National Institute for Health and Clinical Excellence s scope included many aspects and specifically stated that any cost-effectiveness modelling must account for the dynamic nature of HIV, HCV and hepatitis B virus within the population and effects on HRQOL. Individuals on opiate substitution therapy were also identified as a subgroup of interest. The cost-effectiveness model attempted to include as many aspects as possible from the scope. The researchers considered three intervention outcomes, the first of which was increasing the number of IDUs who are recruited onto opiate substitution therapy through a needle and syringe programme, the rate of increase being based on a single randomized trial from the USA. Being on opiate substitution therapy is advantageous because it induces a reduction in sharing rate, as estimated from a cross-sectional (not even before after) survey of IDUs in seven UK cities (Hickman et al., 2007). A second objective was increasing the coverage of needle and syringe programmes, classed as below 100% or saturation, with coverage defined as the number of syringes or needles obtained divided by the number of injections per unit time. The third objective was increasing the proportion of HCV-infected IDUs who were diagnosed and treated. The epidemic models considered various behavioural subgroups, such as needle sharing and opiate substitution therapy, but could not include specific subgroups of IDUs, such as those in prison or homeless. The exclusion of prison populations is of note, but of limited consequence, since IDUs regularly move between prisons and the outside community Data The coupled epidemic model had many more states than the MMT BMT model, e.g. the rate of highly active anti-retroviral therapy uptake by HIV-diagnosed individuals, and opiate substitution therapy associated reduction in risky behaviour. Many transitions had little supporting evidence, e.g. the extent to which individuals were recruited onto opiate substitution therapy and/or whether saturated syringe coverage indeed reduced risky behaviour. Similarly, the rate of uptake of antiviral treatment by HCV-diagnosed individuals was poorly evidenced.

14 638 S. R. White, S. M. Bird and R. Grieve Given the long time horizon of 20 years, additional attention was given to mortality rates from the various states, as well as changing behaviour of individuals, especially cessation of injecting. For cessation of injecting, individuals were grouped by the length of injecting career (less than or at least 3 years), with rates of cessation obtained from the calibration step. Owing to the model fitting, whereby parameter sets were selected that were consistent with the epidemiological data, models presented all HIV positive individuals as being coinfected with HCV; this was then taken as an assumption in the modelling (and is reasonable to a first approximation) Uncertainty The coupled epidemic model had many uncertain parameters, both biological and behavioural, for which there was insufficient evidence for direct estimation. In this case, input parameter uncertainty involved a calibration step, which gives rise to more general methodological issues of propagating and reporting uncertainty. To generate a set of consistent input parameters which could be used to simulate forward to a 20-year time horizon for the interventions of interest, Vickerman et al. (2008) utilized historical blood-borne virus prevalence data for calibration. Their calibration step involved using the model with priors on the unknown input parameters, and simulations to assess the fit to the historical data. The simulations (and their input parameters) which generated output close to the observed historical prevalence data were accepted. The accepted sets of prevalence-calibratedparameters were then carried forward to the next stage of analysis. The report does not present sufficient detail on the joint posterior of the calibrated input parameters: only the marginal means and ranges. The full (and joint) posterior distributions are of interest, since the reported prior and posterior ranges are very similar, and readers are otherwise uncertain what was gained from the calibration process. Further, how the accepted parameter sets were carried forward into the CEA is unclear. This is important because resampling from the posterior range independently would ignore the important correlation structure that the previous calibration stage had carefully established. We assume that sampling at the last stage was from the sets of calibrated prior set combinations. Although there are ambiguities in their reporting on parameter uncertainty, the approach of Vickerman et al. (2008) to incorporating parameter uncertainty was important, especially given the complexity of their cost-effectiveness model. However, there are several model assumptions whose effect was not investigated. Death by overdose was included, with a substantially reduced rate for those IDUs who were on opiate substitution therapy, reduced by a factor of 10 (Davoli et al., 2007), but the risk reduction in total mortality may be by a factor of 2 only (i.e. 50%) (Cornish et al., 2010). The model measured benefit in QALYs, but Vickerman et al. (2008) had to impute some of the quality-of-life data. For example, IDUs who were not on opiate substitution therapy had their utility values reduced by the arbitrary factor of 0.9 relative to those on opiate substitution therapy, since it was assumed, plausibly, that IDUs who were not on opiate substitution therapy were likely to have a lower HRQOL. No sensitivity analysis was performed for the reduction in overdose rate, or the reduction in QALYs. The cost-effectiveness model of Vickerman et al. (2008) was constructed with appropriate attention to the epidemiology. Specifically, within the HCV epidemic model, it was possible to become reinfected, which was absent from the HCV model in Section 3. The proportion of individuals who were immune to reinfection was one of the calibrated parameters, with a central estimate of 70%, and was thus considered as part of the model uncertainty. Excluding this factor,

15 Methodological Issues in Cost-effectiveness Analyses 639 as other studies have, could have a significant effect on the cost-effectiveness of various policies, due to the high probability of HCV reinfection in high prevalence areas. 6. Discussion CEAs are built on the available evidence, some of which has been collected by design, and combine data acquired from different studies and study types to answer a well-scoped question which, individually, the source of data may or may not have been capable of answering. It is important not only to understand the uncertainty from each source of data and how its uncertainty propagates through the cost-effectiveness model, but also to consider the model s epidemiological validity, and to test model or structural uncertainty. For example, how many model parameters are determined by a single source of data, and how do model aspects or components interact with those parameters? The illustrated CEA case-studies exhibit data-dependent modelling. For example, Grishchenko et al. (2009) did not directly account for current IDUs HCV reinfection rate nor their sustained virological response due to current IDUs omission from the available RCTs and cohort studies. While attempting to build their cost-effectiveness model, the West Midlands Health Technology Assessment Collaboration (2006) noted a lack of evidence for quality-of-life values for individuals on opiate substitution therapy and commissioned a small investigative study, thus using their model, in effect, to identify required data. Incorporating more complex model components, such as an infectious disease epidemic model, introduces additional parameters that must be estimated from the available evidence. Accounting for the epidemic dynamics of blood-borne viruses is particularly relevant for IDUs, who have an increased risk of infections and may be susceptible to reinfection if they relapse into injecting. When such complex models are used, as attempted for the CEA of needle and syringe programmes by Vickerman et al. (2008), there will be many parameters without any direct evidence. Accurate reporting of the methods that are used to refine or calibrate these parameters, and their associated uncertainty, is important to the reader in interpreting the final analysis, as well as for highlighting further areas of study. Of particular importance is heterogeneity, which is generally addressed by performing CEAs for subgroups rather than by regression approaches. Public health interventions may exhibit considerable variation in effectiveness and/or costliness when targeted at specific subgroups. For example, Grishchenko et al. (2009) showed that antiviral treatment for HCV carriage was not cost effective for genotype 1 patients with cirrhosis aged over 50 years at presentation for treatment. For opiate substitution therapy and needle and syringe programmes, there may be subgroups of IDUs for whom such interventions are more effective (self-referred versus referral by courts, say). Each CEA case-study had the possibility of directly informing a decision by the National Institute for Health and Clinical Excellence about whether or not to recommend that a particular technology is provided in the NHS. For each technology, it issued a scope which outlined the aspects that were deemed important for assessing the intervention. However, there is still substantial latitude for researchers when building their cost-effectiveness model. None of the CEA case-studies was in conflict with their respective scope. Indeed, the study of Grishchenko et al. (2009) was not actually commissioned by the National Institute for Health and Clinical Excellence. A common concern across the three case-studies is that there was a focus on data-driven modelling rather than appropriate epidemiological modelling: excluding the heterogeneity that is represented by IDUs when considering antiviral treatments for HCV and thus excluding the possibility of reinfection, lower average HRQOL and higher relative mortality rate; a 1-year