Methodological requirements for realworld cost-effectiveness assessment Ismo Linnosmaa Department of Health and Social Management, UEF Centre for Health and Social Economics, THL Ismo Linnosmaa
Outline 1. Basic concepts in cost-effectiveness research 2. How to assess cost-effectiveness: data and methods Concepts treatment, intervention or service are used interchangeably 29/08/2018
Basic concepts 29.8.2018
Framework to assess costs and effectiveness Inputs - costs Output - services Effectiveness - change in well-being or health Productivity is assessed using information on outputs and inputs Cost-effectiveness is assessed using information on costs and effectiveness
Effectiveness Well-being/ health Observed well-being/health with the service E = effectiveness Counterfactual effect: Well-being/health without the service Service use begins time
Resource use and costs Steps in measuring costs of interventions or services: Choice of perspective for the evaluation; societal perspective recommended Identification and measurement of resource use Valuing the use of resources: opportunity cost of resource use Role of non-medical and future costs Remark: Costs/Cost reduction can also be outcomes of interest
Incremental cost-effectiveness ratio Consider old (0) and new (1) technologies with costs C 0 and C 1 outcomes E 0 and E 1 Economic analysis often interested in incremental cost-effectiveness ratio C1 C0 ICER0,1, E E 1 0 measuring the cost of one unit of outcome (e.g. health, QALY or wellbeing) gained when the decision-maker decides to replace the current technology with new technology
How to assess cost-effectiveness: data and methods 29/08/2018
Data sources Data from controlled experiments (henceforth experimental data) Data collection or measurements are controlled by experimenters or researchers Sample size often relatively small Focus often on specific populations, like e.g. multimorbid patients older than 75 years waiting for a hip replacement Real-world data Is observational (henceforth observational data): records what goes on in the real world Sample size can be large Examples: data from administrative records, survey data, population surveys
Randomized controlled trials (RCTs) RCTs are often considered to be the best research designs when measuring causal effects of treatments on outcomes, ie. treatment effects In RCTs: Assignment of individuals to treatment and control groups is controlled by researchers Randomization ensures unbiased estimation of treatment effects Random assignment of subjects to treatment and control groups ensures that on average study subjects are similar in treatment and control groups in all other respects than the treatment; hence Any potential differences in outcomes are caused by the treatment Examples: clinical trials, RAND health insurance experiment (Manning et al. 1987)
Drawbacks of RCTs RCTs may be costly to carry out are sometimes considered unethical because of random allocation of subjects Usual practice in health and social care is that services are allocated on the basis of need may not always work as desired Drop-outs after randomization: e.g. more old people die in the treatment group than in the control group (King et al. 2011) Non-compliance to the experimental protocol
Observational data and methods In observational data selection into the treatment may not be purely random but affected by other factors like e.g. need of care Treatment effects may be confounded by factors influencing selection Methodological approaches to control systematic selection (Jones and Rice, 2011) to estimate causal effect I Selection on observables: Regression techniques Matching techniques (e.g. propensity score matching) II Selection on unobservables: Instrumental variables estimation Regression discontinuity Difference-in-differences
Methodological requirements All methodological approaches go with modelling assumptions that need to be satisfied in causal estimation I Selection on observables: Regression technique assumes conditional independence (CI): Conditional on observables x in regression, outcome and treatment are independent (ie. selection is controlled by observables x) Matching assumes CI and weak overlap of observables in treatment and control groups: overlap creates a possibility to match
Methodological requirements II Selection on unobservables: Instrumental variables method assumes existence of strong and valid instruments Regression discontinuity assumes existence of a forcing variable causing a discontinuous change in the assignment to treatment; e.g. eligibility to a disability benefit ends at a certain age Difference-in-differences assumes longitudinal data and parallel trends: without treatment, trends in the treatment and control groups are same Review by Kreif et al. (2013) shows that in the majority of the published cost-effectiveness studies using observational data (N=81), modelling assumptions needed to estimate causal effects were not properly assessed
Conclusions Improved availability of real world data increases possibilities to evaluate treatment effects in cost-effectiveness studies When using real world data, it is important to Evaluate the possibility of selection bias, and Apply proper methods to estimate treatment effects
Literature Jones AM, Rice N (2011) Econometric evaluation of health policies in Glied S, Smith PC (eds.) The Oxford handbook of Health Economics, Oxford Handbooks, Oxford University Press Kreif N, Grieve R, Sadique MZ (2013) Statistical methods for cost-effectiveness analysis that use observational data: A critical appraisal tool and review of current practice, Health Econ 22: 486-500 King G, Nielsen R, Coberley C, Pope JE, Wells A (2011) Avoiding randomization failutre in program emavalution, with application to the Medicare Health Support Program, Population Health Management, 14(1): 11-22 Manning WG, Newhouse JP, Naihua D, Keeler EB, Leibowitz A, Marquis MS (1987) Health Insurance and the Demand for Medical Care: Evidence from a Randomized Experiment, The American Economic Review, 77(3): 251-277 29/08/2018
More information: ismo.linnosmaa@uef.fi 29.8.2018 17