Bisphosphonates for preventing osteoporotic fragility fracture An example of a multi-dimensional modelling problem Sarah Davis, ScHARR Background Project was part of ongoing NICE Multiple Technology Appraisal (ID782) Existing NICE guidance (TA160 / 161) - 2 bisphosphonates (alendronate, risedronate) - for post-menopausal women - treatment criteria expressed using a combination of age, number of risk factors and bone mineral density (BMD) Currently no NICE guidance on bisphosphonates for - men - other bisphosphonates (ibandronate, zoledronate) NICE Clinical Guideline (CG146) recommends risk assessment using one of 2 online risk calculators (FRAX, QFracture) which estimate the 10 year absolute risk of fracture No clear link between fracture risk assessment recommended by the Clinical Guideline and treatment criteria recommended in the existing NICE treatment guidance. 1
Decision problem Patients: All eligible for risk assessment under clinical guideline (CG146) Interventions: - Oral alendronate - Oral risedronate - Oral ibandronate - Intravenous ibandronate - Intravenous zoledronate Comparator: No bisphosphonate treatment Outcome: Incremental cost-effectiveness at varying levels of absolute fracture risk as measured by FRAX and QFracture Single incremental analysis across whole population even though not all treatments are indicated across whole population - Allows single threshold to be estimated rather than individual thresholds in multiple subgroups Problem 1: Patient heterogeneity Age Probability of death following fracture Fracture risk Costs QALYs Some patient characteristics affect cost-effectiveness independently of absolute risk Cost-effectiveness may be a non-linear function of those characteristics Modelling cohort with average characteristics won t accurately estimate average costs and QALYs due to Jensen s inequality 2
26 Patient characteristics FRAX Age Gender BMI (or BMD if known) Prior fracture Steroid use Rheumatoid arthritis Secondary osteoporosis - Malabsorption / malnutrition, endocrine, premature menopause, Type 1 diabetes, osteogenesis impertecta, chronic liver disease Alcohol Smoking Parental history of fracture QFracture (as per FRAX plus) Ethnicity Care home History of falls HRT Antidepressants Epilepsy / anticonvulsants Asthma / COPD Malabsorption Endocine Comorbidities e.g diabetes (type 1 or type 2), SLE, Parkinson s, CKD, chronic liver disease, dementia, cancer, cardiovascular disease NB: I have summarised these to some extent just to give a feel for the number of risk factors For the full list see the FRAX (http://www.shef.ac.uk/frax/) and Qfracture (http://www.qfracture.org/) websites. Simplified conceptual model Didn t have information on co-prevalence of all 27 characteristics Modelled age and gender dependent prevalence for subset of risk factors Age Gender Steroid use Residential status Prior fracture Family history Baseline utility Lifeexpectancy Residential status post fracture Mortality post fracture Site of fracture Costs QALYs Other risk factors Absolute fracture risk 3
Model structure Discrete event simulation - Patient level simulation allowing for heterogeneity - Calculations only occur when events of interest happen - Computationally efficient when modelling a population at low risk - Inherently stochastic, so large numbers of patients must be simulated to get a stable estimate of costs and QALYs - Helpful to minimise variance not due to treatment differences Random number control for 1 st order uncertainty Stored patient characteristics to run same population each time Problem 2: Thresholds expressed in terms of absolute fracture risk 2 risk scoring algorithms (FRAX and QFracture) - Run model once for each - Same patient characteristics for consistency Continuous measure -> infinite number of possible cut-offs to be examined - Simple analysis using risk deciles as subgroups - Regression analysis for INB versus absolute risk 4
Presented INBs for each risk decile *When using QFracture to predict risk Problem 3: Uncertainty NICE requires model uncertainty to be quantified using PSA Stochastic model with heterogeneous pop Few events in low risk patients -> simulate large cohort to accurately estimates costs and QALYs i.e 2 million Double loop too slow even with computationally efficient modelling approach (DES) 5
GAM regression Single loop simultaneously sampling patient characteristics and parameter values Regression using generalised additive model (GAM) for incremental net benefit (INB) vs absolute risk - Smooth estimate of INB when averaging across parameter samples and stochastic variability - Estimate absolute risk cut-offs for optimal treatment GAM regression used to generate CEACs for each risk decile - i.e. regression of INB versus parameter samples used to average over stochastic patient level variability whilst estimating uncertainty in costs and QALYs due to parameter uncertainty GAM regression prediction when using QFracture to estimate absolute risk 6
CEAC for risk decile with average risk of 1.5% (4 th QFracture decile) Summary of dimensions Simplified decision problem to 6 treatment strategies in 1 population rather than looking at multiple subgroups 26 patient characteristics (4 of which were modelled as functions of age and gender) 2 risk scoring algorithms Infinite number of potential risk cut-offs - incremental analysis conducted for 10 risk categories - GAM regression 73 parameters stochastically sampled in PSA 2 million patients simulated to achieve stability of cost and QALY estimates within each risk category 7
Practical challenges Model generated large data files as regression required all parameter samples (73) to be saved along side costs and QALYs for each treatment strategy (6) for each patient simulated (2 million) Patient characteristics, parameter samples and random numbers were stored to ensure comparability between multiple model runs for different interventions to minimise variance and obtain stable estimates of INB from the minimum number of patients Efficient programming of DES in VBA (Excel modules) to minimise run times and handle large data arrays Efficient and meaningful presentation of results for 6 treatments across 10 risk deciles for both QFracture and FRAX risk GAM regression had 73 independent variables - computationally challenging Acknowledgements and disclaimers NICE Assessment report including declarations of conflicts of interest can be found here: https://www.nice.org.uk/guidance/indevelopment/gid-tag462/documents Co-authors: Marrissa Martyn-St James, Jean Sanderson, John Stevens, Edward Goka, Andrew Rawdin, Susi Sadler, Ruth Wong, Fiona Campbell, Matt Stevenson, Mark Strong, Peter Selby, Neil Gittoes. Funding Acknowledgement: This project was funded by the National Institute for Health Research HTA Programme (project number 13/04/001) Department of Health Disclaimer: The views and opinions expressed therein are those of the authors and do not necessarily reflect those of NICE, the HTA Programme, NIHR, NHS or the Department of Health. 8