Regression Discontinuity Designs: An Approach to Causal Inference Using Observational Data

Regression Discontinuity Designs: An Approach to Causal Inference Using Observational Data Aidan O Keeffe Department of Statistical Science University College London 18th September 2014 Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 1 / 22

Introduction Regression discontinuity design project: funded by a Medical Research Council methodology grant - MR/K014838/1 In general, randomised trials are viewed as the best scientific method for the evaluation of treatment efficacy Such trials are not always feasible and may not necessarily reflect how a treatment performs in a more general population Observational data often available from clinical practice Causal inference from observational data not always straightforward Regression discontinuity designs may represent an approach to this problem Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 2 / 22

RD Design: Example Drug for hypertension (the treatment) Designed to reduce blood pressure (the outcome) Prescribed to those patients whose systolic blood pressure (the assignment variable) is greater than 140mmHg (the threshold) Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 3 / 22

RD Design: Basics Assumption: patients whose assignment variables lie just above and just below the threshold belong to the same population Threshold might be seen as a randomising device: assigns treatment to those whose assignment variable lies above the threshold; withholds treatment from those whose assignment variable lies below the threshold A form of quasi-randomisation : of use in an observational setting? Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 4 / 22

RD Design: Types Sharp Design Threshold behaves like a randomising device If the threshold is adhered-to very strictly (sharp design), then we can think of the RD design as removing the confounding due to unobserved factors Fuzzy Design In medicine, sharp threshold is unlikely to be adhered-to (a situation known as a fuzzy design) For example, often GPs override guidelines - generally because, contrary to their recommendations, they feel that patients will benefit from medication Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 5 / 22

RD Designs 0.0 0.2 0.4 0.6 0.8 1.0 10 12 14 16 18 20 Sharp Example, Threshold: x = 0.5 x Outcome Untreated Treated 0.0 0.2 0.4 0.6 0.8 1.0 10 12 14 16 18 20 Fuzzy Example, Threshold: x = 0.5 x Outcome Untreated Treated Sharp RD Design Fuzzy RD Design Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 6 / 22

Application to Statin Prescription in the UK Statins - a class of drugs used to lower LDL cholesterol for prevention of cardiovascular disease (CVD) United Kingdom NICE guideline: statins should be prescribed to those whose risk of developing CVD within 10 years exceeds 20% RDD variables: Assignment variable: 10-year CVD risk score Threshold: A 10-year CVD risk score of 20% Outcome variable: LDL cholesterol level Patient A has a 10-year risk score of 21% and is prescribed statins Patient B has a 10-year risk score of 19% and is not prescribed statins Can we consider these patients as being from different populations? Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 7 / 22

RD Design: Variables X = Assignment variable (e.g. 10-year CVD risk score) Z = Threshold indicator (i.e. Z = 1 if X > x 0 and 0 otherwise) T = Treatment indicator (T = 1 if treated and 0 otherwise) C = O U = set of confounders O fully observed (e.g. gender, age) U fully or partially unobserved (e.g. smoking status) Y = continuous outcome (e.g. LDL cholesterol level) h = bandwidth - consider only those individuals for whom X (x 0 h, x 0 + h) with h R and h > 0 Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 8 / 22

Important Assumptions Association between threshold indicator and treatment Independence between treatment guidelines/rules and confounders Generally plausible: thresholds set by powers-that-be (government agencies etc.) Unconfoundedness: Individuals just above and just below the threshold are similar (exchangeable) This assumption is important for calculation of a causal effect estimate Continuity: E(Y X = x, Z, T, C) is continuous in x (at x 0 ) for T {0, 1} Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 9 / 22

Causal Effect Estimates: ATE Denote X c = X x 0 to be the centered CVD risk score (take x 0 = 0.2 for 20% 10-year CVD risk) Consider a normal linear model: Y = β 0l + β 1l x c + ɛ where l = a = individual above threshold ; l = b = individual below threshold Causal effect estimate (average treatment effect) is given by: ATE = E(Y T = 1) E(Y T = 0) = β 0a β 0b This estimator is appropriate only in the case of a sharp RD design Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 10 / 22

Causal Effect Estimates: LATE In a fuzzy scenario, direct calculation of the ATE is likely to be biased We need to consider the threshold indicator, Z We define the local average treatment effect: LATE = E(Y Z = 1) E(Y Z = 0) P(T = 1 Z = 1) P(T = 1 Z = 0) = β 0a β 0b π a π b In this case, π l should be modelled formally (e.g. generalised linear modelling) Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 11 / 22

Approaches to Inference Linear regression models - frequentist, parametric inference can be performed A Bayesian framework can be employed: Clinical knowledge can be elicited regarding prescription procedures and incorporated into the analysis (e.g. prescription routines at the practice and/or patient levels using mixture models) Estimator for the denominator in the LATE can be controlled using specified beliefs (stops estimate asymptoting to infinity - this might happen where threshold is weakly associated with treatment) Variance estimation is more straightforward Consider nonparametric methods (e.g. splines) in cases where relationship between assignment variable and outcome variable might be non-linear Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 12 / 22

Simulated Data Data were simulated using real clinical practice data (from THIN) based on c. 5000 males aged 55 64 Unobserved confounding: we incorporated HDL cholesterol level as an unobserved confounder and will consider two levels of unobserved confounding here: low and high Threshold: considered an instrumental variable for treatment. In our analyses, we set threshold as either a strong or as a weak instrument for treatment Bandwidth: h determines how close to the threshold observations ought to lie to be included in an RD design. We consider two bandwidths here: h = 0.05 Treatment Effect: effect of statins to reduce LDL cholesterol by 2mmol/L on average, though with some variability and random error incorporated Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 13 / 22

Models Above the threshold, for xi c [0, h), fit: Y i = β 0a + β 1a xi c + ɛ i Below the threshold, for xi c [ h, 0), fit: Y i =β 0b + β 1b xi c + ɛ i Here ɛ i N (0, σ 2 ) Also need to consider π a and π b for LATE estimators in a fuzzy design Set β 0a = β 0b + φ. φ denotes the treatment effect at the threshold. Bayesian analysis: choose prior distributions for (β 0b, φ, β 1a, β 1b ) Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 14 / 22

Bayesian Priors: Details Density 0.0 0.2 0.4 0.6 0.8 1.0 0 1 2 3 4 5 6 LDL Cholesterol level (mmol/l) Prior distribution: β 0b N (3.7, 0.25) Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 15 / 22

Bayesian Priors: Details Weak Prior Strong Prior Density 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Density 0.0 0.1 0.2 0.3 0.4 0.5 0.6 6 4 2 0 2 4 6 Treatment Effect 6 4 2 0 2 4 6 Treatment Effect Weak prior: φ N (0, 2) Strong prior: φ N ( 2, 1) Slope prior: β 1l N (0, 2) for l {a, b} Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 16 / 22

Bayesian Priors: Denominator Recall the definition of the local average treatment effect: LATE = φ π a π b Let T a (T b ) No. of treated patients above (below) the threshold Assume T l Bin(n l, π l ) for l {a, b} Consider two priors for π l Unconstrained: π b and π a free to vary over [0, 1] Constrained: Impose a fixed minimum difference between π a and π b In both cases, use the strong N ( 2, 1) prior for φ Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 17 / 22

Estimators: Summary To summarise, the tables of results show five estimators: Non-Bayesian ATE: ATE std Weak prior ATE: ATE weak Strong prior ATE: ATE strong Unconstrained LATE: LATE uncs Constrained LATE: LATE cons Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 18 / 22

Results: Bandwidth = 0.05 Estimate (95% Confidence/Credible Interval) Instrument Confounding ATE std ATE weak ATE strong LATE uncs LATE cons Strong Low -1.74-1.86-1.87-2.10-2.10 (-1.98, -1.51) (-1.98, -1.74) (-1.99, -1.74) (-2.25, -1.95) (-2.24, -1.95) Strong High -0.74-0.89-0.90-2.20-1.75 (-1.08, -0.41) (-1.02, -0.76) (-1.03, -0.76) (-2.59, -1.83) (-2.03, -1.48) Weak Low -1.01-1.16-1.17-2.19-1.84 (-1.31, -0.72) (-1.29, -1.03) (-1.30, -1.04) (-2.49, -1.91) (-2.07, -1.62) Weak High 0.05-0.08-0.09-45.72-0.51 (-0.16, 0.25) (-0.20, 0.04) (-0.21, 0.03) (-311.52, 207.84) (-1.23, 0.20) Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 19 / 22

Interpretation In general, LATE estimators capture the true treatment effect size (-2), except where unobserved confounding is high and threshold is weakly associated with treatment Some differences between the two LATE estimators ATE weak and ATE strong estimators yield very similar results. ATE std is slightly different. All estimators are unreliable where unobserved confounding is high and threshold is weakly associated with treatment ATE estimators generally more unreliable as unobserved confounding increases and the association between treatment and threshold decreases Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 20 / 22

Summary Large observational data sources in primary care are becoming increasingly rich Making causal inferences from observational data is not straightforward Under given assumptions, an RD design can help to identify treatment effects Much is known about statins - it is of interest to extend these ideas to areas in which there is less knowledge regarding treatment effects Extension to discrete, rather than continuous, outcomes and development of appropriate software Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 21 / 22

Thanks Thanks to: Gianluca Baio, Nick Freemantle, Irwin Nazareth, Irene Petersen (UCL) Sara Geneletti (London School of Economics) Linda Sharples (University of Leeds) Philip Dawid, Sylvia Richardson (University of Cambridge) Aidan O Keeffe (UCL) The Regression Discontinuity Design 18th September 2014 22 / 22