TRIPLL Webinar: Propensity score methods in chronic pain research Felix Thoemmes, PhD Support provided by IES grant Matching Strategies for Observational Studies with Multilevel Data in Educational Research
Increasing use of propensity scores Source: Web of Science 2
Why propensity scores? Is there anything that we can do with propensity scores that we cannot do with multiple regression?
Propensity scores Tool to strengthen Comparison causal conclusions Models relationship between confounders and treatment Specification of functional form can be checked via balance measures Easy assessment of overlap little potential for extrapolation No routine assumptions about linearity and interactions Outcome variable unknown Sample size can be diminished through matching, loss of power Causal effect for treated, untreated, local comparison Regression adjustment Tool to strengthen causal conclusions Models relationship between confounders and outcome Specification of functional form can be checked via examination of residuals Overlap is assessed in multi-dimensional space often extrapolated Classic ANCOVA assumes linearity and absence of interaction, but this can be relaxed Outcome variable part of the model Sample size stays constant, power can increase due to covariates Causal effect extrapolated to population
Propensity scores e(x) = p (z=1 x) A single number summary based on all available covariates that expresses the probability that a given subject is assigned to the treatment condition, based on the values of the set of observed covariates 5 5
Actual assignment Control Treatment Actual assignment Control Treatment Probability of receiving treatment Probability of receiving treatment 6
Selection Estimation Conditioning Model Checks Effect Estimation 7
Selection Estimation Conditioning Model Checks Effect Estimation Selection of covariates is the single most important aspect to ensure unbiasedness of causal effect Debate in literature (see Rubin, Pearl, 2009, Statistics in Medicine) on how to select covariates 8
Selection Estimation Conditioning Model Checks Effect Estimation Include variables that are confounders (based on your theoretical background knowledge) Exclude variables that are affected by the treatment (potential mediators) Exclude variables that are instrumental variables Exclude variables that are collider variables and induce dependencies Correlational evidence as basis for variable selection can mislead 9
Selection Estimation Conditioning Model Checks Effect Estimation Traditionally, estimated using logistic regression Might necessitate iterative model optimization Data mining approaches offer some promise Covariate-balancing propensity score (K. Imai) 10
Selection Estimation Conditioning Model Checks Effect Estimation Matching can be done in MANY different ways 1:1, 1:k nearest neighbor matching 1:1, 1:k optimal matching k:k full matching Kernel matching Synthetic matching 11
Selection Estimation Conditioning Model Checks Effect Estimation Other approaches include Stratification (form subclasses based on estimated propensity score) Weighting (use propensity score to construct weights that balance groups) Regression adjustment (use propensity score as a covariate) 12
Selection Estimation Conditioning Model Checks Effect Estimation Check of covariate balance standardized difference of covariates (and squares, interactions) various diagnostic graphs Region of common support (distributional overlap) graphical assessment (e.g. histograms) 13
Selection Estimation Conditioning Model Checks Effect Estimation Estimate of treatment effect Mean difference Standard error dependent on conditioning scheme 14
PS in chronic pain research Research on chronic pain, especially effects of chronic pain, tends to be observational, i.e., non-randomized This necessitates the need for adjustment
An application in chronic pain research Does chronic pain has a causal effect on the prevalence of depression? Selection Estimation Conditioning Model Checks Effect Estimation
An application in chronic pain research Spent many meetings discussing potential confounders Decided that 43 covariates would be a minimal adjustment set à used causal graphs to think about where potential confounders could lurk in the assumed causal process
Confounders Demographics (age, gender, income, ) Medical (painkiller, activity, BMI, cigs ) Mental health (depression, affect, ) Personality (Big Five) Social (discrimination, marital risk, ) Childhood (welfare, abuse, ) Sufficient? We don t know for sure, but hope so
Complications Missing data on covariates, outcomes, treatment One potential solution to missing data problems is multiple imputation (Unsolved) problem of combining multiply imputed datasets after matching
An application in chronic pain research Estimated the PS using the 43 covariates in each imputed set Matched units on PS using nearest neighbor with defined caliper (maximum allowed distance)
Balance checks Checked balance some imbalances remained Tweaked caliper Reran, checked balance, reran with different random seed (because nearest neighbor is greedy ) Finally, acceptable balance (<.15) on all covariates
Matched sample After matching, analyses proceeded normally Analysis of matched sample with and without covariates (doubly robust model) Comparison with regular regression adjustment
Results In an unadjusted sample there is a sizeable ( prima facie ) effect of chronic pain on depression (d=.3) After PS matching (and after regression adjustment as well), this effect essentially drops to very close to zero (d <.1)
Propensity scores in R and SPSS MatchIt() from Ho et al. performs a wide variety of these tasks PSMATCHING is an SPSS implementation of MatchIt() and several other R packages (e.g., Ritools(), cem, optmatch )
PSM in SPSS Offers most (but not all) of the features of MatchIt In addition Reports Hansen & Bowers overall chi-square test of balance Reports King s multivariate imbalance measure Supports multi-level data (fixed and random effects models)
Download: http://sourceforge.net/ projects/psmspss/ Contact: felix.thoemmes@cornell