Donna L. Coffman Joint Prevention Methodology Seminar

Donna L. Coffman Joint Prevention Methodology Seminar

The purpose of this talk is to illustrate how to obtain propensity scores in multilevel data and use these to strengthen causal inferences about mediation.

Many prevention programs randomly assigned at school level and hypothesized to affect mediators at individual level. Although intervention is randomly assigned, the mediator is not. Need to take into account confounders of the mediator and outcome. e.g., poverty, individual s prior delinquency

One solution is to use propensity scores, which is based on potential outcomes framework. One assumption of this framework is that there is no interference, such that one individual s assignment to treatment level does not influence another individual s outcome.

No-interference is different from a nested data structure. Nested data does not necessarily mean that we need propensity scores at both the school and individual level. Our motivating problem is what is sometimes referred to as 2-1-1 or lower level mediation.

Three-level treatment: control, usual intervention, intervention adapted for rural Cluster-randomized at the school level Outcome- Alcohol use: Binary because highly skewed. 0 = no use and 1 = any use

Wave 2 refusal confidence about drinking Wave 2 peer alcohol use norms Wave 2 positive expectancies about drinking Scores of 1 are good - indicate greater refusal confidence, less positive expectancies, and lower substance use norms.

Demographics: age, gender, race/ethnicity Lifetime and wave 1 measures of recent substance use (including alcohol) Also age at first use Wave 1 measures of mediators Wave 1 measures of delinquency Stealing, destroying property, school expulsion Coping, anxiety, rebelliousness, family relationships & rules, parental knowledge & monitoring

Imputation model included a lot of variables due to all the confounders in the propensity score model. Many of the variables were not normally distributed so joint model (e.g., NORM) would not work. We used chained equations. Even then, this step was computationally intensive and we decided to impute only one data set for illustration purposes.

Propensity score model for treatment Multinomial logistic regression with fixed effects for school Differences are not that large initially but we wanted to reduce variance. Computed stabilized weights Predicted probability from intercept-only model divided by predicted probability from model of treatment regressed on all confounders. Helps to reduce variability of weights and extreme weights.

Propensity score model for each mediator Logistic regression of each mediator on all confounders and fixed effects for school and treatment level Computed weights Numerator of weights was predicted probability from a logistic regression of each mediator on the treatment level

Assess effect of intervention on mediators Logistic regression for each mediator Use only weights for intervention

Assess effect of intervention and mediators on alcohol use. Logistic regression models for each mediator Using product of intervention and mediator weights.

Intervention did not have statistically significant effect on refusal confidence or positive expectancies. Mediators did have statistically significant effect on alcohol use. Intervention did not have a statistically significant direct effect on alcohol use. Intervention did not have a statistically significant total effect on alcohol use.

Refusal Confidence Not statistically significant when either weighted or unweighted Peer Alcohol Use Norms Significant effect: For the usual DRS intervention there is a 19% reduction in lower norms (-.214, odds ratio =.807) when unweighted and a 20% reduction in lower norms (-.228, odds ratio =.796) when weighted. p =.025 and.035 for unweighted and weighted Positive Expectancies Not statistically significant when either weighted or unweighted

Refusal Confidence Statistically significant effect: -.363, p =.038 When refusal confidence is high, alcohol use decreases there is a 30% reduction in alcohol use. Odds Ratio =.696 Peer Alcohol Use Norms Statistically significant effect: -1.10, p <.001 When norms are low, alcohol use decreases there is a 67% reduction in alcohol use. Odds Ratio =.333 Positive Expectancies Statistically significant effect: -1.07, p <.001 When positive expectancies are low, alcohol use decreases there is a 66% reduction in alcohol use. Odd Ratio =.343

Effect of treatment on alcohol use using logistic regression without the mediators in the model. No significant effect either with or without the weights.

We recognize that the mediators may not be independent. We performed an analysis that took this into consideration but results were not different. We did not multiply coefficients together for the effect of the intervention on the mediators and the effect of mediators on alcohol use because with binary mediators and outcomes, this value is not interpretable as an indirect effect although the indirect effect can be tested for significance (Coffman & Zhong, 2012).

It is important to proceed with mediation analysis even in the absence of a total effect. It is important to look at the effects separately. We learned that the mediators do indeed have strong effects on alcohol use, even when controlling for many confounders. But the intervention did not have statistically significant effects on the mediators, perhaps due to implementation issues.

Mediation analyses should consider that individuals and/or groups are not randomly assigned to levels of the mediator(s) and there may be confounders of the mediator and outcome even if the intervention is randomly assigned. This is an active area of research and future work when examine under what conditions propensity score models should include random effects, when propensity scores are needed at each level of nested data, and how to handle multiple mediators simultaneously.

Bethany Bray Wanghuan Chu Michael Hecht and John Graham NIH NIDA P50 DA10075