NATIONAL EVALUATION Safe Schools/Healthy Students and Substance Use Prevention UPDATE Volume 6, Issue 2 Prevention works. or young people, preventing alcohol and drug use pays particular dividends. It can reduce risk for a constellation of poor outcomes, including school failure, delinquency, and violence. The Safe Schools/Healthy Students (SS/HS) Initiative has been helping to reduce risk factors and promote protective factors among children and youth since 1999. SS/HS supports schools in building local collaborations that address their most pressing problems: preventing substance use and violence, promoting the mental health of students, and creating safe and respectful school climates that enhance academic achievement. SS/HS grantees assess the extent of substance use and other problems among young people in their community. They consider which interventions may be most effective locally, including strategies already in place that merit expansion. They explore the feasibility of implementing their ideas and eventually sustaining them with other funding sources. They then implement a comprehensive plan and measure its impact. SS/HS offers hope. Despite decades of research, drug and alcohol use among young people remains too high. Thirteen percent of 8th graders and 40 percent of 12th graders report drinking alcohol in the past 30 days. Marijuana use has increased among high school students since 2008. Sizeable minorities of youth engage in heavy substance use that puts them at particular risk: 1 in 5 high school seniors report binge drinking, and 1 in 15 report using marijuana daily. 4 SS/HS grantees have shown reductions in alcohol and marijuana use (see below). The potential cost savings associated with these reductions is significant, considering that the average societal cost of underage drinking alone for just 1 month has been estimated at $457 per person. 5 1, 2, 3 SS/HS Grantees Reporting Improved Substance Use Outcomes, 2005 2007 Cohorts Source: Government Performance and Results Act (GPRA) data 6 (continued, p. 2) The Safe Schools/Healthy Students Initiative is a collaboration of the U.S. Departments of Education, Justice, and Health and Human Services.
Substance Use Prevention SS/HS grantees identified a range of substance use issues at the outset of their grants. 7 or example, most were experiencing increased alcohol and drug use among students. Border towns were facing the effects of drug trafficking. Some school districts were seeing a link between youth substance use and increases in gangs and violence. Many communities were struggling with declining budgets and cuts in services for young people and families. or the fourth year, the main back-toschool event will feature a Meth Watch. A representative from the mental health partner agency will talk with parents about the dangers of methamphetamine use, which is a problem locally. As many as 1,500 parents and children are expected to attend. 2007 Grantee SS/HS partners include schools and local mental health, law enforcement, and juvenile justice agencies. Together, they have provided substance use prevention and intervention services, programs, and activities throughout the grant. Evidence-based programs have been central to their plans (see below). Some of the programs have served only students, while others have served parents or entire families. School staff have received training to help them implement the programs and to increase their knowledge and understanding of substance use issues. Each SS/HS partner has provided services relevant to its mission and, in doing so, has contributed to the overall mission of the partnership. or example, most of the requency of Implementation of Selected Evidence-Based Programs Related to Substance Use, 2005 2008 Cohorts ear 1 ear 2 ear 3 Strengthening amilies 8.9% 15.1% 18.5% Project Alert 16.4% 17.1% 15.8% Too Good for Drugs 13.0% 29.5% 30.8% Project Success 7.5% 10.3% 10.3% Reconnecting outh 6.2% 11.6% 12.3% Source: SS/HS Project-Level Survey 8 grantees reported that the juvenile justice partner agencies have monitored substance use among students (e.g., breathalyzers, drug testing); law enforcement agencies have worked to ensure that tobacco and alcohol are not sold to underage youth; and mental health agencies have provided individual and group counseling for students, screenings for family members, and referrals to other community-based services. In some cases, colocating mental health staff in clinics and schools has helped improve collaboration. Grantees have often created substance use committees and task forces with representatives from partner agencies, parents, and community members. These groups have helped select and implement substance use programs and create or modify school polices. SS/HS grantees have complemented substance use programs with other school and community activities to promote awareness of alcohol and drug use. or example, a grantee created an antidrug poster contest for elementary school students, and the posters were displayed in local businesses. At another site, students helped redesign an antialcohol logo that was printed on t-shirts and promotional materials. Other awareness activities have included The Project Towards No Drug Abuse program was recently written into the new health curriculum for the school district. This ensures that the program will continue to be implemented it is not optional. 2008 Grantee Partner presentations and workshops Safe Home Pledges to ensure parents commitment to providing a safe, alcohol-free environment (e.g., being present at their child s parties, not serving alcohol to minors) Public forums and town hall meetings on drug or alcohol abuse Marketing campaigns to increase public awareness of substance abuse among youth using tools such as newsletters and DVDs 2 (continued, p. 3)
Substance Use Prevention SS/HS grantees overcame many challenges to implement their projects successfully: Local funding limitations Lack of public awareness about substance use Inconsistent school or community buy-in to prevention activities Staffing issues (e.g., recruitment and retention, lack of space) The need to build stronger relationships with community partners SS/HS grantees that reported declines in substance use among students attributed their positive outcomes to evidence-based programs, efforts of task forces and committees, increased community involvement, and increased access to student services. Among their most important accomplishments was progress toward sustainability. Many grantees reported that they had succeeded in securing funding to support substance use prevention after the SS/HS grant ended. By investing in safe schools and healthy students now, they can look forward to a future with safer communities and healthier, more productive adults. Students created a public service announcement for local radio stations, citing data on local substance use and encouraging parents to talk with their children about the effects of alcohol and drugs. 2009 Grantee The Substance Use Prevention and Education Resource program (SUPER) is a free, bilingual, early intervention program for parents and their teenage children. Sessions focus on increasing parental involvement, teen self-esteem, and family communication skills and include discussions about the legal, psychological, and physical aspects of substance abuse. 2009 Grantee A new brief from the U. S. Department of Education provides a snapshot of substance use policies from the 100 largest school districts in the United States. Key findings include New rom the U.S. Department of Education A large majority of districts indicate that students may or will be reported to law enforcement for incidents involving the possession or use (86 percent of districts) or sale or distribution (87 percent of districts) of alcohol or drugs. Other responses include principal-determined suspensions (98 percent of districts indicate that students may or will be subject to a principal-determined suspension for possession or use; 84 percent of districts indicate that students may or will be subject to a principal-determined suspension for sale or distribution), recommendation for an expulsion hearing (90 percent for possession or use; 94 percent for sale or distribution), placement in an alternative schooling program (80 percent for possession or use; 71 percent for sale or distribution), and parent conference or notification (85 percent for possession or use; 82 percent for sale or distribution). Nearly one-third of districts (30 percent) report having graduated sanctions for repeat offenses. or example, 15 percent of districts explicitly allow principals to increase the duration of a suspension for possession or use if it is the student s second offense. To view the brief, visit http://ies.ed.gov/ncee/pubs/20124022/ 3
Local Evaluation Tip Triangulating Local Evaluation indings With Other Data Sources Even the most carefully planned and executed evaluations may face challenges when reporting their findings. Stakeholders will want to know whether they can rely on the findings and what the results mean for their community. One way to enhance the validity and relevance of local findings is to triangulate the data with other local, State, and national indicators, which can provide additional context for understanding local results. Using external data sources can help explain, confirm, or counter the local evaluation results. It can also form the basis for hypotheses about why an intervention did or did not work and provide much-needed context. or example, one SS/HS local evaluation incorporated State measures of substance use into its final report. While selfreported substance use among eighth graders had declined statewide, it had increased at the SS/HS site, and the evaluators wanted to understand why. They noted that the SS/HS project had implemented a substance use prevention program late in the grant period and primarily with students younger than those sampled in the State survey. Therefore, the impact of the program would not have been observed during the data collection period, and the population served was not comparable to the State sample. However, recognition of the State trends prompted the site to consult with substance abuse program experts to ensure that the program was being implemented with fidelity. Key questions that can help guide evaluators when triangulating data include Do the local evaluation findings make sense? Are any of them vulnerable to criticism? Should the results spark changes in strategies or implementation? Are the findings a result of the SS/HS programs and activities, or would the outcomes have been similar without those interventions? When citing external data, describe its methodology. Ideally, aspects such as sampling, measurement, and timing should be similar to those of the local evaluation. Also discuss the limitations of the external data, which are usually found in published materials. inally, use caution: direct comparisons across data sources are not advisable. Some national data sources that include indicators of student health and safety are shown below. Other sources include State and local departments of health, education, and justice, as well as private and nonprofit organizations. National Sources of Data on Student Health and Safety Drug Abuse Warning Network A Survey 4 Substance Use Measures Included Safety Risk/Protective actors Education Data Analysis Tool B (multiple data sets) Monitoring the uture C National Assessment of Educational Progress B (The Nation's Report Card) National Survey of Children's Exposure to Violence D National Survey of Children's Health E National Survey on Drug Use and Health A National outh Tobacco Survey outh Risk Behavior Surveillance System A Substance Abuse and Mental Health Services Administration, U. S. Department of Health and Human Services (HHS). B National Center for Education Statistics, U. S. Department of Education. C University of Michigan; Sponsored by National Institute on Drug Abuse, National Institutes of Health, HHS. D University of New Hampshire; Sponsored by Office of Juvenile Justice and Delinquency Prevention, Office of Justice Programs, U. S. Department of Justice, and Centers for Disease Control and Prevention (CDC). E Maternal and Child Health Bureau, Health Resources and Services Administration, HHS, and CDC. CDC, HHS.
Resources or more information on substance use prevention, visit these Web sites. http://www.sshs.samhsa.gov/communications The Communication and Social Marketing Center helps SS/HS grantees develop and deliver effective communication products and social marketing campaigns through training, technical assistance, and resources. Grantees should contact their Communication Specialist for support. http://www.promoteprevent.org The National Center for Mental Health Promotion and outh Violence provides training and technical assistance to SS/HS grantees in areas including strategic planning, partnership development, program assessment, evaluation, and sustainability. Resources include information on evidence-based programs and practices to prevent and address substance use and abuse, and research on substance use prevention and intervention. http://www.nrepp.samhsa.gov SAMHSA s National Registry of Evidence-based Programs and Practices is a searchable online registry of mental health and substance abuse interventions that have been reviewed and rated by independent experts. https://www.stopalcoholabuse.gov Led by SAMHSA, the Interagency Coordinating Committee on the Prevention of Underage Drinking is an interagency committee of 15 ederal agencies. The STOP Alcohol Abuse Web site serves as a portal to comprehensive research and resources on the prevention of underage drinking developed by the participating agencies. http://www.hhs.gov/ash/oah The Office of Adolescent Health (OAH), U.S. Department of Health and Human Services, collects and disseminates information on adolescent health, including resources on substance use. OAH works in partnership with other agencies to support evidence-based approaches to improve the health of adolescents and monitors trends in adolescent health. 1 Komro, K. A., Williams, C. L., orster, J. L., Perry, C. L., arbakhsh, K., & Stigler, M. H. (2000). The relationship between adolescent alcohol use and delinquent and violent behaviors. Journal of Child and Adolescent Substance Abuse, 9, 13 28. 2 Substance Abuse and Mental Health Services Administration. (2010). Results from the 2009 National Survey on Drug Use and Health: Volume I. Summary of National indings. (Office of Applied Studies, NSDUH Series H-38A, HHS Publication No. SMA 10-4586). Rockville, MD: Substance Abuse and Mental Health Services Administration. 3 National Research Council and Institute of Medicine. (2009). Preventing Mental, Emotional, and Behavioral Disorders Among oung People: Progress and Possibilities. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, outh, and oung Adults: Research Advances and Promising Interventions. M.E. O Connell, T. Boat, and K.E. Warner, Eds. Board on Children, outh, and amilies, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. 4 Johnston, L. D., O Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (2012). Monitoring the uture national results on adolescent drug use: Overview of key findings, 2011. Ann Arbor: Institute for Social Research, The University of Michigan. Accessed at http://monitoringthefuture.org/pubs/ monographs/mtf-overview2011.pdf 5 Derived from annual estimate, adjusted for inflation, in Miller, T. R., Levey, D. T., Spicer, R. S., & Taylor, D. M. (2006). Societal cost of underage drinking. Journal of Studies on Alcohol, 67(4), 519 528. 6 There are a total of 86 grantees among the 2005, 2006, and 2007 cohorts. Odds ratio (OR) availability varied by grantee and by GPRA outcome measure. or grantees that reported data for marijuana use (n=63) and alcohol use (n=67), the percentage with positive outcomes is shown. Improvements are defined as positive outcomes that are either statistically significant or statistically nonsignificant GPRA ORs greater than 1.0. Statistical significance is based on review of the 95 percent confidence interval. ear 3 versus pre-grant ORs were prioritized; when unavailable, ear 3 versus ear 1 ORs were substituted, if available. 7 Source: Grantee interviews, 2005 2008 cohorts HSD/GED 8 Grantees from the 2005 cohort in ear 1 responded to a slightly different list of of programs than subsequent cohorts, which may have resulted in lower estimates for ear 1 overall. This newsletter was developed under contract number 280-10-0348 from the Substance Abuse and Mental Health Services Administration (SAMHSA), U.S. Department of Health and Human Services (HHS). The views, policies, and opinions expressed are those of the authors and do not reflect those of SAMHSA or HHS. or information about the SS/HS National Evaluation, visit www.sshs.samhsa.gov or contact: Nainan Thomas, Ph.D., at Nainan.Thomas@samhsa.hhs.gov or Duren Banks at dbanks@manilaconsulting.net Released 4/12