Centre for Analysis of Youth Transitions AYT STUDY REFERENCE: REP07 Programme name: PREVENTURE: a group of brief, selective, personality-targeted interventions aimed at reducing substance (alcohol and drugs) use and misuse. Contact details/links for further details: Addictions Department, Institute of Psychiatry, King s College London, UK. Université de Montréal, CHU-Hopital Ste-Justine. The author of most studies: Patricia Conrod, Ph.D. Centre de recherche du CHU Ste-Justine Université de Montréal, Bureau 1551, 3175 Chemin de la Côte Sainte-Catherine, Montreal, H3T 1C5. Tel: 514 345 4931 ext. 4051; Institute of Psychiatry, King s College London, Department of Psychological Medicine, 4 Windsor Walk, Denmark Hill, London SE5 8BB. Email: patricia.conrod@kcl.ac.uk, patricia.conrod@umontreal.ca. Programme description, aims and objectives: The intervention initially identifies secondary school students at high risk of substance misuse based on self-reported questionnaires which assess their personality profiles. In most cases a student is deemed to be at high risk of substance misuse if their score in one of the categories: hopelessness, sensation seeking, anxiety, sensitivity or impulsivity, is more than one standard deviation above the mean score in their school. Usually the intervention, involving only high-risk students who voluntarily agree to take part, consists of group sessions targeting the personality factor (hopelessness, sensation seeking, anxiety, sensitivity or impulsivity) for which the student exhibited an elevated score. If the individual scored highly on more than one trait, they participate in the session targeting the trait for which their score showed the highest deviation from the average. There are 2 group sessions spread across 2 weeks, with each session lasting 90 minutes. The sessions include a psycho-educational component, a motivational component and a cognitive behavioural component. They address the target personality variable by educating students about it and its associated behaviours, discussing the motivations behind substance use and its consequences in the context of the target personality traits. The exercises involve discussing emotions, thoughts and behaviours in a personality-specific way as well as identifying and challenging personality-specific distortions which lead to problematic behaviours. Although substance use is discussed, the focus of the sessions is on personality rather than substance use. Depending on the intervention under consideration, the sessions were delivered by external therapists or internal school teachers who had completed a three-day training with satisfactory results. Target population: Secondary-school students (age 13-14 years), assessed to be at high risk of future substance misuse as judged on the basis of their personality factors. The analyses concerned both individuals prior to the onset of substance use (prevention) and individuals who had initiated the use of drugs or alcohol prior to the intervention.
Expected outcomes: Reduction in the probability of alcohol use, the quantity consumed, binge drinking, problem drinking, coping drinking motives, frequency and quantity of illicit drug use, and some herd effects on drinking onset and growth in binge drinking amongst the general population from which high risk youth are selected.
References: Conrod, P.J., Stewart, S.H., Comeau, N., & Maclean, A.M. (2006). Preventative Efficacy of Cognitive Behavioral Strategies Matched to the Motivational Bases of Alcohol Misuse in At-Risk Youth. Journal of Clinical Child and Adolescent Psychology, 35(4), 550-563. Conrod, P.J., Castellanos-Ryan, N., & Mackie, C. (2008). Personality-targeted interventions delay the growth in drinking and binge drinking. Journal of Child Psychology and Psychiatry, 49(2); 181-190. Conrod, P.J., Castellanos-Ryan, N., & Mackie, C. (2011). Long-term effects of personality-targeted interventions to reduce alcohol use in adolescents. Journal of Consulting and Clinical Psychology. Conrod, P.J., Castellanos-Ryan, N., Strang, J. (2010). Brief, personality-targeted coping skills interventions prolong survival as a non-drug user over a two-year period during adolescence. Archives of General Psychiatry, 67(1):85-93 O Leary-Barrett, M., Mackie, C.J., Castellanos-Ryan, N., Al-Khudhairy, N., Conrod, P.J. (2010). Personalitytargeted interventions delay uptake of drinking and decrease risk of alcohol-related problems when delivered by teachers. Journal of the American Academy of Child and Adolescent Psychiatry, 49(9):954-963. Conrod, PJ, O Leary-Barrett, M., Newton, N., Topper, L, Castellanos-Ryan, N., Mackie, C.J., Girard, A. (in press). A cluster randomized trial evaluating a selective, personality-targeted prevention program for adolescent substance misuse: Primary two-year outcomes and secondary herd effects. Archives of General Psychiatry. Related studies: Grant BF, Dawson DA (1998). Age at onset of drug use and its association with DSM-IV drug abuse and dependence: results from the National Longitudinal Alcohol Epidemiologic Survey. Journal of Substance Abuse, 10(2):163-173. Lynskey MT, Heath AC, Bucholz KK, Slutske WS, Madden PAF, Nelson EC, Statham DJ, Martin NG (2003). Escalation of drug use in early-onset cannabis users vs co-twin controls. Journal of the American Medical Association, 289(4):427-433. Study details: All studies focus on the impact of brief, selective, personality-targeted interventions on students self-reported drinking and drug intake outcomes between 4 and 24 months post intervention. The differences between the studies (for example, in terms of how the interventions were delivered) are detailed below.
Study samples: Canadian Trial Conrod et al. (2006) Preventure Trial Wave1 Conrod et al. (2008) Preventure Trial Full cohort Conrod et al. (2010) Preventure Trial Wave2 Conrod et al. (2011) Adventure Trial O Learry-Barrett et al. (2010) Sample size Number of schools 297 9 (Canada) 283 13 (London) 732 24 (London) 347 13 (London) 1129 18 (London) Age of participants 14-19, Mean: 16 years Intervention Group/Control Group Both groups: high-risk students who reported drinking prior to the study 14 years (median) Both groups: high-risk students, whether drinking or not 13-16, Median: 14 years Both groups: high-risk students, most students had not initiated drug use prior to the baseline study 14 years (median) Both groups: high-risk students, whether drinking or not 13.7 (mean) Both groups: high-risk students, whether drinking or not Adventure Trial Conrod et al. (in press) 1210 high risk and 1433 low risk 18 (London) 13.7 (mean) Comparing high-risk students in intervention and control schools; high-risk and low-risk within intervention schools Methodology: In all the studies only high-risk students who were randomly assigned to the intervention group were invited to attend the sessions. Canadian Trial Conrod et al. (2006) Preventure Trial Wave1 Conrod et al. (2008) Preventure Trial full cohort Conrod et al.(2010) Preventure Trial Wave2 Conrod et al. (2011) Adventure Trial O Learry-Barrett et al. (2010) Adventure Trial Conrod et al. (in press) Timing of follow-up sessions Delivery Randomisation Method Outcome of interest 4 months Professional-led sessions 6 and 12 Professional-led months sessions 6,12,18 and Professional-led 24 months sessions 6,12,18 and Professional-led 24 months sessions 6 months Trained School Staff 6,12,18 and 24 months Trained School Staff Student level RCT (Randomised Alcohol Controlled Trial) Student level RCT Alcohol Student level RCT Drugs Student level RCT Alcohol School level Clustered RCT (RCT with randomisation at the school level) Alcohol School level Clustered RCT Alcohol
Results and impact: Cited measures of the size of the impact: NNT Number Needed to Treat shows how many students need to be exposed to the intervention in order to prevent one additional negative outcome relative to the control group. For example, NNT of 10 for Cocaine use means that in order to deter 1 additional student (relative to the control group) from using Cocaine, on average 10 students need to attend the sessions. Cohen s d is a statistical measure used to show the magnitude of the effect from analyses of variance. In general, a value of.80 represents a large effect, a value of.50 a moderate effect, and a value of.20 a small effect (Cohen, 1977, cited in Conrod et al., 2006). The probabilities of outcomes occurring are based on the concept of the Odds Ratio which shows the frequency of an event occurring in the intervention group relative to the control group. For example, an Odds Ratio of 0.6 for the probability of drinking means that for a given likelihood of a control student drinking, an intervention student is only 60% as likely to drink. Note also that, with the exception of the study by Conrod et al. (2006) each substance use outcome is reported for the 6 months prior to the follow-up questionnaire therefore, for example, the effect at 18 months refers to drinking between 12 and 18 months after the intervention. The validity of the criterion O Leary-Barrett et al. (2010) found that at baseline (before the intervention) students classified as at high risk were more likely to drink and binge drink relative to low-risk students. The difference is statistically significant which provides support for the use of the selection criterion. This selection instrument and the cut-offs used to identify high risk youth were shown to be highly sensitive in the prospective prediction of future alcohol and drug related behaviour and problems (Castellanos-Ryan et al., in press). Alcohol Probability of drinking: Several studies found that the intervention reduced the probability of drinking among the high-risk students in the intervention group, relative to those 'high-risk' students who were not invited to the sessions after 4-6 months. Both for the sub-sample of high-risk students who were drinking at baseline (Conrod et al., 2006) and for high-risk students in general (O Leary-Barrett et al., 2010), the Numbers Needed to Treat were similar in magnitude (12.5 and 14.7 respectively) this shows that approximately 13-15 students need to attend the sessions in order to encourage 1 additional (relative to the control group) person to abstain from drinking. Note, however, that Conrod et al. (2008) did not find the effect of the intervention on the probability of drinking at the 6-month follow-up period to be statistically significant. The evidence on sustaining this short-term effect is mixed. Although Conrod et al. (2008) fail to find any significant effect of the intervention on the probability of drinking at a 12-month follow-up period, Conrod et al. (in press) find that overall for the duration of the trial (24 months) the rates of drinking among the high risk students exposed to the intervention are statistically significantly lower than those of their control group counterparts. ORs indicated that the intervention was associated with 29% reduced odds of drinking over the course of the trial in students attending intervention schools relative to students in control schools, OR =.71 (.51-.99) p=0.046.
Some studies find the effect of the intervention to be particularly strong for a certain personality group but the results do not consistently point to the same group. For example, Conrod et al. (2006) find that the intervention is especially successful at encouraging drinking students to abstain for 4 months if they received an anxiety sensitivity intervention (NNT = 4.9) or a hopelessness-focused intervention (NNT = 5.6). On the other hand, the Preventure Trial, which studied a younger cohort of students (Conrod et al., 2008) find that it is the effect on Sensation Seeking students that is particularly noteworthy (with a statistically significant reduction of 61% in the probability of drinking at 12 months relative to the control-group Sensation Seeking students). Sensation seeking youth have been shown to have earlier onset of drinking, and therefore might need earlier intervention to prevent onset of drinking. Frequency: In general the effect of the intervention on the frequency of drinking was not found to be significant. Conrod et al. (2008) found the intervention to have a statistically significant effect on a joint measure of quantity by frequency but the effect is small (Cohen s d = 0.12) and limited to 6 months after the intervention. Quantity: In a third trial which studied a cohort of children who were one year younger than the Preventure Trial, Conrod et al (in press) found that the high-risk students who were subject to the intervention were drinking less than the 'high-risk' students in the control group at the 6 month follow-up period with the difference being statistically significant. Moreover, the growth of the quantity of drinking was also statistically significantly lower in this group, suggesting that the initial gain increased over time. A reduction in the quantity of alcohol consumed was also found for the high-risk students who were drinking at baseline (Conrod et al., 2006) although the magnitude of the effect was small (Cohen s d = 0.26). Binge drinking: For the general sample of high-risk students the intervention was associated with a reduction in the odds of binge drinking at 6 months of approximately 30% - 43% with the effect being statistically significant or just insignificant at 5% confidence level. Over longer periods, the differences between intervention and control groups lose statistical significance (Conrod et al., 2011). However, Conrod et al. (in press) find that the growth of binge drinking rates among high-risk individuals subject to the intervention was significantly lower than for the control group over the 6-24 month follow-up period when interventions are delivered to younger students. There is strong evidence for the effectiveness of the intervention on the sub-sample of high-risk students who were drinking at baseline with several studies finding a statistically significant effect. The Number Needed to Treat was estimated between 4.3 and 6.5 suggesting that exposing 5-7 high-risk students who were drinking at baseline results in preventing 1 person from binge drinking 4-6 months after the intervention. Additionally, Conrod et al. (2008) found exceptionally high effectiveness of the intervention on binge drinking rates among the sub-sample of students who were described as Sensation Seeking and drinking at baseline. Their evidence suggests that the Numbers Needed to Treat are 2.0 for the 6-month and 2.4 for the 12-month period, implying that in order to deter one Sensation Seeking student who was drinking at baseline from binge drinking, only 2-3 students need to attend the sessions. Problem drinking: Problem drinking relates to negative consequences of drinking such as missing school due to drinking. In the Preventure Trial, Conrod et al. (2011) found that for the general high-risk sub-sample the overall effect of
intervention on problem drinking was statistically significant at all time periods (6, 12, 18 and 14 months) although the effect was small-to-moderate (Cohen s d between 0.17 and 0.35 depending on the time frame). In the Adventure Trial, Conrod et al. (in press) found that the high-risk students who participated in the intervention were less likely to report problem drinking symptoms throughout the trial and 42% less likely to report problem drinking symptoms at the end of the trial (24 months). For the high-risk sub-sample who were drinking at baseline, Conrod et al. (2006) argue that approximately 6.5 students need to be treated in order for 1 student to avoid drinking problems 4 months after the intervention. This effect is stronger for students rating high on the Hopelessness and Anxiety Sensitivity scales with NNT s of 3.5 and 4.1 respectively. Drinking motives: Conrod et al. (2011) evaluated the impact of the intervention on the reasons for drinking. The main focus of the analysis was the importance of internal motives ( coping and enhancement ) relative to external motives ( social motive and conformity ). For example, drinking to feel better when one is depressed would signify a coping motive. An example of an enhancement motive is drinking because of liking the feeling. The intervention was found to reduce coping motives among high-risk students relative to the high-risk control group at 12 and 24 months and overall, and this effect was small-to moderate (Cohen s d = 0.20 0.42). This result may be driven by Anxiety Sensitive students - for this sub-sample the intervention's effect on coping motives was significant at 6, 12 and 24 months (but not 18 months). Although the effects of intervention on enhancement motives were not found to be significant overall, there was a statistically significant reduction in enhancement motives at 6 months for students described as Sensation Seeking. Herd effects : Herd effects, also known as spillovers, are the effects of the treatment on those who were not subject to the intervention. Although low-risk students did not attend the sessions, The Adventure Trial (Conrod et al. in press) showed that the intervention was associated with reduced odds of drinking over the course of the trial in low risk students attending intervention schools relative to low risk students in control schools. By the end of the trial, low-risk students in intervention schools were 35% less likely to binge drink than their low-risk counterparts in control schools (the difference is statistically significant). Unlike the intervention effects on problem drinking for high risk youth, there were no statistically significant differences in the likelihood of reporting a drinking problem in low risk youth, who showed much lower rates of problem drinking symptoms than high risk youth.. Looking at both high-risk and low-risk students, attending an intervention school was associated with a statistically significant reduction of 29% in the probability of drinking 6 months after the intervention. It is argued that since the rates of growth of drinking were similar for the intervention and control groups, the initial intervention effect on drinking behaviour was was maintained over the 24 month period. The possible transmission mechanism could be the propagation of a certain model of behaviour due to the high-risk students drinking less. Alternatively, the training provided could have increased the ability of teachers to provide support for students in general. Since the effects on low-risk youth are visible with a delay relative to high-risk youth, Conrod et al. (in press) argue in favour of the first hypothesis. Drugs:
Frequency: Students in the intervention group reduced their drug use over time with the reduction being moderate (Cohen s d = 0.5) 6 months after the intervention and small but statistically significant 24 months after the intervention. By comparison, high-risk students in the control group showed no significant change in the frequency of drug use. Usage: In the intervention group the number of drugs used was reduced at both 6 and 24 months after intervention, with the reduction being of moderate size (Cohen s d = 0.67). Over the same period drug use increased in the control group (the increase was statistically significant). Marijuana: The intervention was associated with a 30% reduction in the probability of taking up Marijuana 24 months after the intervention relative to the control group. This effect is only significant at the 10% level, however, as opposed to the generally accepted 5%. The result suggests that, on average, out of every 18 students participating in the sessions 1 person is prevented from taking up Marijuana 24 months after the intervention (NNT = 18). Cocaine: Over 24 months the intervention is associated with an 80% reduction in the probability of taking up Cocaine and the effect of the intervention is statistically significant for all time frames (from 6 to 24 months). Out of every 10 students attending the sessions, 1 is deterred from using Cocaine 24 months after the intervention (NNT = 10). Other drug use: In order to deter 1 student from using other drugs 24 months after the intervention, 16 need to be treated. The effects of the intervention are statistically significant for the 12 24 month follow-up periods. Type of delivery It is not clear whether the treatment is more effective if delivered by internal teachers rather than external professionals, mostly due to the way in which the experiments were designed: those studies which used randomisation at the school level were also those which relied on teacher-led sessions and targeted younger pupils. Since it has been shown that the intervention has an impact on students who did not receive the treatment, there may be some attenuation of the results in studies which use within-school randomisation. This means that we would expect studies which use between-school randomisation to show higher effects than studies using within school randomisation, all else being equal. If the effect is smaller in schools which use between-school randomisation, therefore, we might attribute the difference to the fact that in those schools the sessions were delivered by internal staff. The evidence is mixed on this point, however. For example, it seems that the effects of the intervention on problem drinking are higher if the treatment is provided by professionals. By contrast, the effects on the QF (Quantity*Frequency) measure are higher for the teacher-led treatment. Overall, the evidence does not provide a clear steer on which model of delivery is preferred. Different samples, statistical approaches and randomisation methods used in the studies do not allow us to assess whether the different methods of delivery are what is driving the different effects found by different studies. Moreover, theory alone does not provide clear suggestions on whether the delivery by teachers or professionals would be more effective. On the one hand, better training of teachers is likely to benefit low-risk students too and could potentially lead to more lasting effects, because teachers are better positioned to revisit intervention material with students showing persistent problems. Furthermore, training teachers might lead to a cultural or attitude shift within the school and promoting better understanding of the risk factors for substance misuse. On the other hand,
interventions delivered by experienced professionals may be more effective. An additional question is whether students are more likely to share their experiences with an outside or an inside session leader. Overall The intervention has been found to have an effect on the probability of consuming alcohol, the quantity of alcohol intake, binge drinking, problem drinking, coping drinking motives, as well as having positive herd effects. The strongest piece of evidence concerns binge drinking, particularly among those who were drinking before the commencement of the study. The intervention seems to be associated with a reduction in the frequency and quantity of drug use with the strongest impact on Cocaine use. Given that the studies evaluate the effects of the intervention only up to 24 months after treatment, it is impossible to tell whether the results represent full prevention or only a delay. Furthermore, for certain outcomes the evidence on the effectiveness of the treatment for periods longer than 6 months is mixed, although Conrod et al., (2011) suggest that this short-lived delay might translate to longer-term protection against alcohol misuse, due to the demonstrated link between age of onset of substance regular substance use and risk for addiction (Grant & Dawson, 1998). Finally, the studies point to the brevity of the treatment (two 90-minute sessions) relative to some other programmes. Reservations The classification of students as 'high-risk' is conducted relative to other students in the same school. Therefore the measure is not an absolute one and we can expect some variation between schools in the absolute levels of the personality traits in question. Moreover, research on this question shows that both approaches are highly sensitive to identifying those at risk for future alcohol and drug use and misuse (Castellanos-Ryan et al., in press). Additionally, the comparison of the studies is not straightforward since those which randomised the intervention at the school level were also those in which the sessions were delivered by 'internal' teachers. Since it has been shown that the intervention has an impact on students who did not receive the treatment, the impact detected by the studies which use within-school randomisation may be understating the true impact. Impact grade: 2 Costs: Figures not available. The costs would include: Initial screening of the students (using freely available self-reported questionnaires to classify them as high-risk or low-risk ). For each eligible high-risk student running two 90-minute group sessions in small groups (for example 2-7 participants). This would involve the costs of extra working hours for staff, small administrative costs (if lessons are rescheduled), potentially the cost of facilities (but as this is conducted in school, during class time, this is a low probability). The costs of manuals and exercise booklets for students and session leaders. 3-day training for the school prospective session leaders, including the evaluation of their successful completion of the training ( 2,500 per group of 4 trainees + 200 per hour of supervised practice for each trainee). Alternatively, the costs of hiring external specialists to run the sessions.
Quality of evaluation evidence: The studies are randomised controlled trials or clustered randomised trials with randomisation at the student and the school level accordingly and hence these are high quality evaluations. Randomisation at the student level ensures that the intervention and control groups are similar in all respects apart from whether they were subject to the treatment. Although randomisation at the school level did not involve matching schools based on their underlying characteristics, the key indicators at baseline such as the probability of drinking or binge drinking were similar for both the control and intervention groups. Quality of evidence grade: 6 Appendix: details of impact grades and quality of evidence grades are set out below Impact grade Description 0 (none) No relationship between the youth service and the outcome in question. 1 (low) Provision of the youth service may be positively related to one but not all outcomes or just for subgroups of the target population. 2 (medium) The youth service has moderate impact on all outcomes and sub-groups or high impact on some outcomes and sub-groups. 3 (high) The youth service has high impact on all outcomes and sub-groups.
Score Type of study More Description Example of a study How to improve the quality of evidence 0 Basic Studies that describe the intervention and collect data on activity associated with it. A study that describes the intervention and states how much it cost or how many hours of services young people received. Collect some before and after data on the outcome of interest for those receiving the intervention. If it is too late for that, collect outcome after data for the group receiving the services and try to compare these outcomes with comparable youth using other sources of data. 1 Descriptive, anecdotal, expert opinion 2 Study where a statistical relationship (correlation) between the outcome and receiving services is established 3 Study which accounts for when the services were delivered by surveying before and after 4 Study where there is both a before and after evaluation strategy and a clear comparison between groups who do and do not receive the youth services 5 As above but in addition includes statistical modelling to produce better comparison groups and of outcomes to allow for other differences across groups 6 Study where youth services are provided on the basis of individuals being randomly assigned to either the treatment or Studies that ask respondents or experts about whether the intervention works. The correlation is observed at a single point in time, outcomes of those who receive the intervention are compared with those who do not get it. This approach compares outcomes before and after an intervention. These studies use comparison groups, also known as control groups. Study with a before and after evaluation strategy, statistically generated control groups and statistical modelling of outcomes. A study that uses focus groups or expert opinion or indeed surveys those who received the intervention after they received it. A study that conducts a survey only after the services have been delivered and concludes that youths who received the services responded more positively than those who did not. A study that conducts a survey before and after the programme. A study that matches two locations where both individuals and areas are comparable and surveys them before and after the programme e.g. pilot studies. A study that uses a statistical method, such as propensity score matching, to ensure that the group receiving the youth services is similar to the comparison group and a statistical model of outcomes (e.g. difference in difference). A study which conducts a Randomised Controlled Trial Collect some before and after data on the outcome of interest for those receiving the services. If it is too late for that, collect outcome after data for the group receiving the services and try to compare these outcomes with comparable youth using other sources of data. This evidence does not allow for the fact that prior to the intervention youths who received the service may have been different from those who did not. Collect some before and after data on the outcome of interest for those receiving the intervention. If it is too late to do that, see if you can compare outcomes for a clearly defined comparison or control group using other before data sources, such as administrative data. If you have before-after data you can measure the change in a particular outcome after the services were delivered. Try to determine whether you can compare this gain in the outcome for those who received the youth services to the gain for a similar group of youth who did not receive the services. You might use administrative data for this. You have most of the data you need. Contact an expert on statistics or econometrics and they will be able to apply various statistical methodologies to improve the robustness of your results e.g. matching methods to define a better control or comparison group. NOTE: this is the minimum level of evaluation quality applied by the Social Research Unit et al (2011), which also stipulates that any such study fulfil various quality criteria. Short of a random control trial, this methodology is the most robust. To improve confidence in the results try to collect additional data, perhaps from administrative sources, on the comparison group to determine any differences between them that may have pre dated the intervention. The gold standard. It is challenging to run a RCT, with cost, ethical and practical issues arising. Even with a RCT you have to think about how generalisable it is to other situations. If the RCT was only males, it cannot tell you about how well the youth service would do for females, for example.