Centre for Analysis of Youth AYT

Transcription

1 Centre for Analysis of Youth Transitions AYT STUDY REFERENCE: REP06 Programme name: European Drug Abuse Prevention (EU-DAP) trial Contact details/links for further details: Contact person: Barbara Zunino Serena Vadrucci Programme description, aims and objectives: EU-DAP is a school-based project designed to prevent adolescents misuse of legal and illegal substances. Unplugged is a program based on the development of life skills, adopting the more effective teaching techniques to involve students in an interactive curriculum designed to improve and develop life skills. The intervention consisted of 12 one-hour lessons, based on a comprehensive social influence model. There were sessions on normative education and information on the effects of smoking and drug use. Interactive teaching techniques were used, with the focus on developing better interpersonal and intrapersonal skills. In a third of the intervention schools, the curriculum was complemented with seminars for parents; in another third, there was additional class-peer involvement. Target population: Adolescents in European countries between the ages of 12 and 14. Expected outcomes: Reduced use of alcohol, tobacco, cannabis and other drugs. Study reference: 1. Fabrizio Faggiano, Maria Rosaria Galanti, Karl Bohrn, Gregor Burkhart, Federica Vigna-Taglianti, Luca Cuomo, Leila Fabiani, Massimiliano Panella, Tatiana Perez, Roberta Siliquini, Peer van der Kreeft, Maro Vassara, Gudrun Wiborg, the EU-Dap Study Group (2008) The effectiveness of a school-based substance abuse prevention program: EU-Dap cluster randomised controlled trial, Preventive Medicine 47 (2008) F Vigna-Taglianti, S Vadrucci, F Faggiano, G Burkhart, R Siliquini, M R Galanti (2009) Is universal prevention against youths substance misuse really universal? Gender-specific effects in the EU-Dap school-based prevention trial, Journal of Epidemiology Community Health 2009; 63; Fabrizio Faggiano, Federica Vigna-Taglianti, Gregor Burkhart, Karl Bohrn, Luca Cuomo, Dario Gregori, Massimiliano Panella, Maria Scatigna, Roberta Siliquini, Laura Varona, Peer van der Kreeft, Maro Vassara, Gudrun Wiborg, Maria Rosaria Galanti, and the EU-Dap Study Group (2010) The effectiveness of a school-based substance abuse prevention program: 18-Month follow-up of the EU-Dap cluster randomized controlled trial, Drug and Alcohol Dependence 108 (2010) Maria Paola Caria, M.Sc, Fabrizio Faggiano, M.D., Ph.D Rino Bellocco, Sc.D, Maria Rosaria Galanti, M.D., Ph.D and the EU-Dap Study Group (2011) Effects of a School-Based Prevention Program on European Adolescents Patterns of Alcohol Use, Journal of Adolescent Health 48 (2011)

2 Related studies: M. Rosaria Galanti, Roberta Siliquini, Luca Cuomo, Juan Carlos Melero, Massimiliano Panella, Fabrizio Faggiano and the EU-Dap Study Group (2007) Testing anonymous link procedures for follow-up of adolescents in a school-based trial: The EU-DAP pilot study, Preventive Medicine 44 (2007) Study details: A number of related studies have evaluated the effectiveness of this programme. All the studies are based on one random control trial. The trial took place in school year , when schools were randomly assigned to either the control group or the treatment of EU-DAP intervention. The studies examined the effectiveness of EU-DAP in preventing the use of tobacco, alcohol and drugs. Information on substance use and personal and social skills was obtained by self-completed questionnaire both before the intervention and 6 months later (3 months after the intervention ended) and 18 months later. Seven outcome variables were investigated, all of them with reference to the 30 days preceding the survey: (i) any cigarette smoking; (ii) frequent cigarette smoking, defined as smoking six or more cigarettes per month; (iii) daily cigarette smoking, defined as smoking 20 or more cigarettes per month; (iv) any episode of drunkenness; (v) frequent drunkenness, defined as three or more episodes; (vi) any cannabis use; (vii) frequent cannabis use, defined as use on three occasions or more. Study sample: 3547 students in 78 intervention schools and 3532 in 65 control schools in 7 European countries, after some schools withdrew participation before the start of the programme. All students aged 12 to 14. The students were instructed to generate individual anonymous code by themselves, which were then used to match individuals before and after the intervention. Mismatches reduced the final sample to 5541 students for the papers that evaluate longer-term impact. Methodology: Schools were randomly selected into control or treatment groups. The idea is to statistically compare the changes in substance use before and after the intervention for the treatment group with the changes observed for the control group. The association between intervention and changes in the outcomes of interest was expressed as prevalence odds ratio (POR), estimated by a 3-level mixed regression model. This accounted for the hierarchical structure of the data with students (level 1) nested within classrooms (level 2), which in turn were nested within centres (level 3). Robustness checks were conducted based on different assumptions on the missing data. Results and impact: There are 4 studies evaluating this programme. The impact of the programme varies by sub-group and in the short and long run. The results from each study are detailed below study This initial study considered the effect of the programme on short run outcomes (3 months after the end of the programme). The study found that the programme had a statistically significant negative effect on

3 students daily use of cigarettes, their likelihood of drinking and their frequency of drinking in the past 30 days. The programme was found to have a marginally statistically significant negative impact on Cannabis use. The results for other outcomes were not significant, but point in the same direction. The magnitude of the effects suggest that this programme is on a par with the most effective of all school-based programs that have undergone peer reviewed evaluation. In addition, the curriculum was found to be particularly successful in preventing baseline non-smokers or sporadic smokers from moving onto daily smoking. But it was not effective in helping baseline daily smokers to reduce or stop smoking study This study assessed the different impacts of the programme on boys and girls, 3 months after the end of the programme. The intervention significantly reduced the risk of substance use for boys. For girls, the programme was associated with a decreased risk of frequent drunkenness in the past 30 days, but the estimate was not statistically significant. There were also gender differences in the impact of the programme on the transitions between different levels of substance use before and after the intervention. Compared to the control group, fewer boys in the treatment group progressed to more advanced stages of smoking and more boys regressed to less frequent use of substance. A similar but less pronounced pattern was observed for girls. There was no significant association between the impact of the intervention and reported self-esteem. Overall, the intervention effectively reduced substance use among boys, but not girls study This study evaluated the longer run impact of the programme. At the 18-month follow-up, the programme was no longer found to decrease cigarette smoking. The programme was however, found to decrease the risk of drunkenness and of frequent cannabis use. Alternative multilevel models, fitted for purpose of sensitivity analysis, confirmed the findings. In line with the findings from the 6 month follow-up, in the 18 month follow up students in the intervention group showed a greater tendency to remain non-users of tobacco or to go from occasional to no use. For daily users the programme showed no effects study This study evaluated the longer run impact of the programme, focusing specifically on alcohol use. The program reduced the risk of a student reporting alcohol-related problems. The estimated relative reduction compared with the control group was about 22% and the absolute risk reduction was 1.9%. This reduction in risk was not statistically significant however, for those who were already drinkers. The frequency of alcohol consumption was not reduced by the program. Non-drinkers and occasional drinkers in the programme progressed toward frequent drinking less often than those who did not go through the programme. Impact grade: 2

4 Costs: estimated 200 per class, including teachers' training and materials. Quality of evaluation evidence: This was a randomised controlled trial. Evidence from a randomised control trial is very high quality and likely to tell us what the causal impact of the programme was. The only caveat is that there might be attrition bias. Some schools were selected for the intervention but didn t participate throughout and were dropped from the sample. If schools dropped out because they perceived the intervention to be ineffective, then comparing the remaining ones with the control group might overstate the effectiveness of the programme. Moreover, the imperfect linkage of individuals through the self-generated anonymous code meant that some individuals are dropped from the sample. If this attrition was related to the effects of the intervention, then the estimates would be biased. Sensitivity analysis did not however, suggest that individuals leaving the trial had introduced important biases. Quality of evidence grade: 6

5 Appendix: details of impact grades and quality of evidence grades are set out below Impact grade Description 0 (none) No relationship between the youth service and the outcome in question. 1 (low) Provision of the youth service may be positively related to one but not all outcomes or just for subgroups of the target population. 2 (medium) The youth service has moderate impact on all outcomes and sub-groups or high impact on some outcomes and sub-groups. 3 (high) The youth service has high impact on all outcomes and sub-groups.

6 Score Type of study More Description Example of a study How to improve the quality of evidence 0 Basic Studies that describe the intervention and collect data on activity associated with it. A study that describes the intervention and states how much it cost or how many hours of services young people received. Collect some before and after data on the outcome of interest for those receiving the intervention. If it is too late for that, collect outcome after data for the group receiving the services and try to compare these outcomes with comparable youth using other sources of data. 1 Descriptive, anecdotal, expert opinion 2 Study where a statistical relationship (correlation) between the outcome and receiving services is established 3 Study which accounts for when the services were delivered by surveying before and after 4 Study where there is both a before and after evaluation strategy and a clear comparison between groups who do and do not receive the youth services 5 As above but in addition includes statistical modelling to produce better comparison groups and of outcomes to allow for other differences across groups 6 Study where youth services are provided on the basis of individuals being randomly assigned to either the treatment or the control group Studies that ask respondents or experts about whether the intervention works. The correlation is observed at a single point in time, outcomes of those who receive the intervention are compared with those who do not get it. This approach compares outcomes before and after an intervention. These studies use comparison groups, also known as control groups. Study with a before and after evaluation strategy, statistically generated control groups and statistical modelling of outcomes. A study that uses focus groups or expert opinion or indeed surveys those who received the intervention after they received it. A study that conducts a survey only after the services have been delivered and concludes that youths who received the services responded more positively than those who did not. A study that conducts a survey before and after the programme. A study that matches two locations where both individuals and areas are comparable and surveys them before and after the programme e.g. pilot studies. A study that uses a statistical method, such as propensity score matching, to ensure that the group receiving the youth services is similar to the comparison group and a statistical model of outcomes (e.g. difference in difference). A study which conducts a Randomised Controlled Trial Collect some before and after data on the outcome of interest for those receiving the services. If it is too late for that, collect outcome after data for the group receiving the services and try to compare these outcomes with comparable youth using other sources of data. This evidence does not allow for the fact that prior to the intervention youths who received the service may have been different from those who did not. Collect some before and after data on the outcome of interest for those receiving the intervention. If it is too late to do that, see if you can compare outcomes for a clearly defined comparison or control group using other before data sources, such as administrative data. If you have before-after data you can measure the change in a particular outcome after the services were delivered. Try to determine whether you can compare this gain in the outcome for those who received the youth services to the gain for a similar group of youth who did not receive the services. You might use administrative data for this. You have most of the data you need. Contact an expert on statistics or econometrics and they will be able to apply various statistical methodologies to improve the robustness of your results e.g. matching methods to define a better control or comparison group. NOTE: this is the minimum level of evaluation quality applied by the Social Research Unit et al (2011), which also stipulates that any such study fulfil various quality criteria. Short of a random control trial, this methodology is the most robust. To improve confidence in the results try to collect additional data, perhaps from administrative sources, on the comparison group to determine any differences between them that may have pre dated the intervention. The gold standard. It is challenging to run a RCT, with cost, ethical and practical issues arising. Even with a RCT you have to think about how generalisable it is to other situations. If the RCT was only males, it cannot tell you about how well the youth service would do for females, for example.

7