Comprehensive Research Report/ Program Evaluation

Transcription

1 / Prgram Evaluatin July 2012

2 SUMMARY The Arkansas Children s Hspital (ACH), in partnership with the Cnnect with Kids Educatin Netwrk (CWK), is pilting a multimedia prgram in 18 schls, fcusing n health educatin, using the pwer f CWK nline multimedia, reality- and researchbased resurces. The gal f this prgram evaluatin is t determine hw well students learn, and hw their behavir and attitude changes with a mre intense, fcused use f technlgy (nline access and ther media) fr health educatin. CWK vides are designed t harness the pwer f strytelling as real kids share real stries abut issues affecting teen health including bullying preventin, depressin and mental health, besity, nutritin, and physical activity, and alchl and ther drug use preventin. Participating middle and high schls were prvided with nline access t vide and print resurces, as well as curricula binders with DVDs and supprting lessn plans and activities fr the classrm. ACH and CWK wrked with schls t measure prgram efficacy with independent preand pst-testing research t assess impact f the prgram n the students invlved. Participating middle and high schls identified a facilitatr, student participants and a cntrl grup which tk the pre- and pst-tests withut experiencing the curricula. Prgram facilitatrs lgged n t a training webinar, administered a pre-test t a selected class as well as a cntrl grup, and cnducted 10 classrm learning mdules ver a 10-week perid. Tpics addressed include the fllwing health issues: making healthy chices, stress and anxiety, teen depressin, OTC and prescriptin drugs, underage drinking, cyber bullying, managing technlgy, physical fitness and nutritin. Each 30 t 60-minute lessn includes a 7 t 10-minute vide segment and accmpanying activity. At the cnclusin f all lessns, the facilitatr administered pst-tests t the participating students. The pst-test was als administered t the participants in the cntrl grup wh did nt receive the lessns. The survey evaluatin is intended t demnstrate a sustainable change in behavir, the impact f visual media and that difficult-t-measure emtinal qutient in reaching yuth and affecting their decisin-making prcess. The verall initiative was directed at health literacy thrugh technlgy t emply varius media t reach teens with the tpics at hand. Evaluatin results are summarized in this reprt. 2

3 Evaluatin Design The evaluatin was intended t assess the ability f the prgram cmpnents t demnstrate sustainable changes in behavir, as well the impact f visual media t affect the decisin-making f participants, a mix methds design, fcusing n research questins that call fr real-life cntextual understandings and multi-level perspectives. The verall initiative was directed at health literacy thrugh technlgy t emply varius media t reach teens with infrmatin n the tpics at hand. Evaluatin results are summarized in this reprt. The initial evaluatin design targeted sixth- thrugh twelfth- grade health classes in 20 middle and high schls thrughut the state f Arkansas. Due t the cnstraints f varius end-f-the-schl-year activities, tw high schls were unable t cmplete the prgram and were eliminated frm the evaluatin. The prgram was executed using bth an experimental grup that participated in the 10- week prgram and a cntrl grup that did nt. The schls and classes selected were intended t represent the diversity f the student ppulatin. The respnses were reviewed t ensure a representative sampling in bth the cntrl and experimental grups. T match respnses frm the same individual acrss time in the pre- and pst-survey in a paired cmparisn, participants were asked t prvide their mther s first initial and the day f the mnth in which they were brn. These tw values were cmbined int a single unique identifier variable that preserved participants annymity. The pre- and pst-survey bth cnsisted f 28 questins. - Three (3) questins (Questins 1, 2, and 3) were intended t btain student demgraphic infrmatin, including gender, grade, and ethnicity. - Additinal questins prvided utcme measures befre and after the curriculum. These questins cnsist f Likert items designed t capture attitudes and behavirs regarding the prgram cntent areas. Questins are designed t address the varius prgram sessins: Making Healthy Chices (Questins 18, 11, 12, 7, 9, 17) Managing Stress (Questins 6, 13) Depressin and Anxiety (Questins 5, 10) Over-the-Cunter and Prescriptin Drug Abuse (Questin 22) Underage Drinking (Questins 4, 8, 27) Tbacc (Questins 26, 28) Cyberbullying/Bullying Preventin (Questins 15, 16) 3

4 Managing Technlgy (Questins 19, 21, 24) Nutritin/Bdy Image (Questins 20, 23) Physical Fitness (Questin 25) Multimedia Relevance (Questins 14, 30) The pre-survey included tw (2) an additinal qualitative questins t allw participants t prvide input n causes f stress in their lives and whether they preferred t btain infrmatin fr decisin making thrugh nline media r vides, r classrm discussins and lectures. Fur (4) qualitative questins were used in the pst-survey t request input n whether students viewed stress differently after the prgram and their preferred means f btaining decisin making infrmatin thrugh either nline media r vides, r classrm discussins and lectures. Participants were als asked the mst imprtant thing learned frm the prgram, as well as hw the prgram culd be imprved. The technique f affinitizing was used t rganize and summarize the narratives int lgical grupings t better understand the essence f the anecdtal respnses. A survey f facilitatrs was used t btain input n the perceived effectiveness f each prgram sessin, as well as infrmatin n the facilitatr preparatin and administratin f the prgram t assess the fidelity f the prgram administratin. Findings Results frm analysis f respnse data indicated a shift in preference fr the availability f infrmatin thrugh nline media and vides, especially in cmbinatin with the use f classrm discussin grups. Quantitative and qualitative anecdtal respnses indicated that the prgram had a psitive impact n a change in attitude regarding imprtance f making healthy chices within the experimental grup. Amng the ntable findings, results indicated statistically significant imprvements after the curriculum in the fllwing utcmes: Respnses frm student participating in the curse indicated a significant increase in understanding that chices made during the teen years have a cntinued life-lng effect n health. There was a 20 % verall gain in the experimental grup versus a 4 % gain in the respnses f the cntrl grup. Respnses t student surveys indicated increased awareness f cyberbullying and the impact n themselves and thers. There was a 10% increase in the number f students wh realized that they had previusly participated in bullying r making fun f thers nline. Participants indicated a statistically significant increase in cnfidence in their ability t psitively deal with day-t-day stress. There was a 5 % increase in the 4

5 number f students wh indicated that they feel like they have the skills t handle day-t-day stress versus n change in the experimental grup. Students expressed a statistically significant increase in awareness that parents and friends influence decisins and chices that affect healthy decisins. The use f a mixed methds survey, including bth qualitative and quantitative questins, allwed students t pen up and share details f stresses bth in schl and at hme, including bullying by classmates and siblings; parental prblems with alchl, smking, and eating disrders; and feelings f lneliness and discnnectin. Respnses in these areas within the cntrl grup did nt indicate a significant change ver the same 10-week perid. Additinally, several findings based n demgraphic indicatrs included: Respnses did nt differ significantly between ethnic grups. Respnses did differ significantly between gender grups. Respnses did nt differ significantly between schls, althugh cmments indicated that sme f the material was cnsidered by bth students and facilitatrs t be abve the level f 6 th and 7 th graders. Lngitudinal studies typically suffer frm prblems with attritin; hwever, this study had a significantly high (18.75%) attritin rates. Pssible reasns fr this may be lack f sufficient training f facilitatrs in the fundamentals f administering a pre- and pstsurvey within cntrl and experimental grups, which may lead t inability t match preand pst-surveys. Facilitatrs als indicated that the study experienced challenges due t cnflicts with ther end-f-the-schl-year activities. Intrductin PROGRAM OVERVIEW AND EVALUATION DETAILS The Arkansas Children s Hspital, in partnership with Cnnect with Kids Educatin Netwrk, is pilting a multimedia prgram in 18 schls, fcusing n health educatin thrugh the use f nline multimedia, reality- and research-based resurces. The curriculum is interactive, using a reality-based multimedia frmat (real kids talking abut real situatins and utcmes in vides) t engage students in issues affecting teen health including bullying preventin, depressin and mental health, besity, nutritin, and physical activity, and alchl and ther drug use preventin. Participating middle and high schls were prvided with nline access t resurces, as well as curricula binders with DVDs and print materials fr the classrm. Vides and print resurces 5

6 streamed nline at a custm, passwrd-prtected website hsted by Cnnect with Kids: Curriculum Overview Use f a Cllabrative Learning Strategy The curriculum embraces an apprach t enhance the discussins and activities where peers can learn frm each ther under the guidance f an adult facilitatr. This change strategy recgnizes the pwer f ne s peer grup as a vehicle fr slving cmmn prblems. The theretical fundatins emerged frm the field f scial wrk and was develped by William Schwartz and built upn by Lawrence Schulman and Alex Gitterman. It is based n the cncept that there exists a reciprcal relatinship between individuals in a grup (Schwartz, 1961). It recgnizes the strength f a peer grup as a vehicle fr slving cmmn prblems. The peer grup prvides a frum fr trying ut new behavirs; giving and receiving feedback; prviding psitive supprt; and, mst imprtant, learning t help and supprt thers thrugh the aid f a trusted adult. It takes advantage f the psitive rles that peers can play in wrking tgether t find new slutins t ld prblems. Using a multimedia curriculum, the prgram cnsists f ten in-class lessns that are 30 t 60 minutes in length, with vide segments, discussin questins, and in-class activities. Research has indicated that there is a significant difference in learner mtivatin based n attentin and retentin between vide-based instructin and traditinal text-based instructin (Chi & Jhnsn, 2005). Scicultural learning thery supprts the ntin that behaviral change des nt ccur in islatin but rather supprted by scial relatinships and engaging experiences, as well as technical and psychlgical tls (Vygtsky, 1978; Dusenbury, Hansen, & Giles, 2003). Vide-based instructin prvides fr standardizatin f messages, which increases the fidelity f curriculum implementatin. The curriculum incrprates discussins and activities thrugh which students can learn frm their peers under the guidance f an adult facilitatr. Accrding t Shulman, as children and yuth cme t grips with the impact f substance abuse n their wn lives and the lives f friends and significant thers, as they attempt t cpe with the pressures f grwing up that may encurage cnfrmity t unhealthy substance abuse nrms.having the pprtunity t meet with ther yuth experiencing similar struggles and having an adult that desn t judge r lecture them may make all the difference in their successful negtiatin f this vulnerable life stage. (Shulman, 1986). This cllabrative apprach, thrugh its emphasis n invlving peers, takes advantage f peer supprt, a dynamic in adlescence t enfrce healthy decisin making. This apprach is true nt nly fr adlescents but fr adults as well and can be used in bth clinical cunseling sessins and educatinal prgrams. The strength-based perspective in educatinal grups emphasizes that with increased knwledge and practice f skills an individual can mre adequately deal with the challenges f living (Sands & Slmn, 6

7 2003; Letendre, 2007). Many evidenced base educatinal prgrams fr adlescents in schls are lengthy and difficult t implement in a high schl setting. The curriculum is designed t have the greatest impact in the fewest sessins, thereby making it userfriendly fr schls. Successful implementatin f the lessns als requires that the facilitatr have knwledge f the health challenges teens face and pssess an understanding f adlescent develpment. Rle f the Facilitatr At the cre f this apprach is the ntin that the facilitatr must understand the ppulatin t be taught (preparatry empathy). In rder t effectively administer this curriculum and t incrprate this change management apprach, it is essential that the facilitatr prepare fr the adlescent audience by cnsidering: What teens may be thinking and feeling abut the challenges and health decisins regarding drugs, alchl, nline behavir, stress and anxiety, and fitness and nutritin. What it is like t be a teenager in tday s wrld. What reactin will the cunselr have t what is expressed. It is essential that the facilitatr prepare fr each tpic by reviewing the material prir t the sessins. The ability t achieve the maximum ptential benefits frm implementing change depends in part n hw effectively a climate is created and maintained that minimizes resistant behavir and encurages acceptance and supprt fr a given change. Attempts by individuals t impede the intended change is referred t as resistance t change, and has been defined as behavir which is intended t prtect an individual frm the effects f real r imagined change (Sander, 1950) and any cnduct that serves t maintain the status qu in the face f pressure t alter the status qu (Zaltman & Duncan, 1974). The classrm sessins are designed t engage students in a learning prcess where participants are pen t listening t the facilitatr and t each ther. Central t the prcess implementing change are several dynamics that are incrprated thrughut the class lessns. A pwerful frce in the interventin is the realizatin by the student that they are nt alne in his r her thughts, feelings, and emtins. There is smething special in hearing thers articulate similar feelings and experiences. Such mutuality prduces supprt that energizes learning. While it is imprtant fr students t have supprt frm their peer grup and adults, it is nt sufficient t create change in behavir r intentin t change. Research shws behavir and attitude changes ccur when students have sufficient infrmatin t learn new skills (Lewin, 1948; Flger & Skarlicki, 1999). Develping new attitudes and behavirs may require first that ld ideas and behavirs be questined and evaluated. 7

8 The discussin questins and activities ask students t challenge their wn views and thse f their peers thrugh guided rle playing and requests fr feedback. Adlescents are in the best psitin t cnfrnt and challenge each ther s thughts, assumptins and actins n hw and why t make healthy chices. As students respnd t key questins in each lessn, the facilitatr can establish cnnectins by acknwledging the adlescent experience (academic stress, scial anxiety and peer pressure, experimentatin.) Further reinfrced in the reality-based vide segments, students learn ways t make infrmed decisins and healthy chices. Each student has unique life experiences. The discussin questins fllwing each vide segment are designed t assist students with sharing their experiences and beliefs, as well as hw they can develp lifelng strategies that prmte a healthy lifestyle. By encuraging their participatin in the learning prcess, the facilitatr can assist students t listen t each ther while crrecting any misinfrmatin that the students may have abut the health-related tpics at hand. Curriculum Evaluatin Methds Prgrams and curriculum are ften cnducted withut measuring utcmes t ensure bjectives are achieved and the desired change ccurred. Organizatins, bth fr-prfit and nn-prfit, need t cnduct prgram evaluatins t measure the ability t meet bjectives (Kirkpatrick, 1993). This initial prgram assessment fcused n sixth- thrugh twelfth-grade health classes in 18 schls thrughut the state f Arkansas. The prgram was executed using bth an experimental grup that participated in the 10- week prgram and a cntrl grup that did nt. The middle and high schl classes selected were intended t represent the diversity f the student ppulatin. The respnses were reviewed t ensure a representative sampling in bth the cntrl and experimental grups. The primary purpse f this evaluatin research was t apply scial research methds t assess scial interventin prgrams and plicies (Singletn, 2010). The effectiveness f prgrams t implement and sustain change can be measured by a variety f research tls given established evaluatin criteria (Kirkpatrick, 1993). Numerus scial change theries, suggest that peer and grup influences are effective in impacting changes in behavir and attitudes but are frequently difficult t measure (Cperrider, 1999; 2005). Scial Develpment Strategy (Hawkins & Weis, 1985; Hawkins, Catalan, & Wells, 1986) and Scial Learning Thery (Bandura, 1963, 1969, 1977) therizes that behavir change ccurs when individuals are prvided infrmatin regarding risks and benefits f particular behavirs, and use that knwledge t frm attitudes and learn behaviral skills. Behavir is shaped thrugh mdeling behavirs within a scial cntext. Scicultural learning thery supprts the ntin that behaviral change des nt ccur in islatin but rather supprted by scial relatinships and engaging experiences, as well as technical and psychlgical tls (Vygtsky, 1978; Dusenbury, Hansen, & Giles, 2003). 8

9 Justificatin fr Survey Methd This study used a quasi-experimental design and mixed-methds analysis cnsisting f bth qualitative and quantitative data. Qualitative and quantitative methds are ften clsely aligned with survey design techniques and individual case studies as a way t reinfrce and evaluate findings ver a brader scale. The brad area f survey research is an imprtant and frequently used quantitative methd that encmpasses any measurement prcedures invlving the asking questins f respndents. Surveys are frequently used t assess attitudes and characteristics f a wide range f subjects. In a survey, researchers sample a ppulatin. A ppulatin is any set f persns r bjects that pssesses at least ne cmmn characteristic (Basha & Harter, 1980). Since target ppulatins may be extremely large, researchers directly questin nly a sample f the ppulatin. Data are usually cllected thrugh the use f questinnaires, althugh smetimes researchers directly interview subjects. Surveys can use qualitative, such as pen-ended questins, r quantitative, such as frced-chice questins, measures (Babbie, 1990). Data Analysis The researcher checked fr exaggerated and incnsistent respnses t screen every survey prir t use in this study. Exaggerated respnses culd be a very critical prblem, and ne that culd lead t skepticism abut the survey results. Fr a variety f reasns, sme students may be incnsistent in hw they respnd t the survey questins. They may mark their answers randmly, just t get the survey ver with. They may have reading prblems and be unable t understand the questins. They may give false respnses t mess with the survey. Students may, fr example, indicate n ne questin that they d nt make decisins affecting their health (including fd, exercise and substance use/abuse) based upn what their friends are ding, while n anther questin might indicate that friends prvide the mst influence regarding chices affecting health. Checks were made fr incnsistencies and missing infrmatin, and while any student might just mark an answer wrng in a survey this lng, if it happened mre than twice n ne survey, the survey is deleted frm the reprt prcess. Incnsistency, attritin, and inability t match pre- and pst-surveys accunted fr eliminatin f 123 (18.75%) surveys frm the final data analysis. The pre- and pst- student survey was utilized t test the effectiveness f this prgram in develping an awareness and change f attitude and behavir in regards t health and behavir risks, as well as t increase prtective factrs needed t help students t live healthy and prductive lives. In rder t measure the significance f change in attitude based n expsure t the curriculum, the pre- and pst-survey were evaluated in a paired cmparisn (Aczel & Sunderpandian, 2009). By lking at paired-bservatins, the between grup (extraneus variatin) is minimized. T match respnses frm the same individual acrss time, participants were asked t prvide their mther s first initial 9

10 and the day f the mnth in which they were brn. These tw values were cmbined int a single unique identifier variable that preserved participants annymity. The pre- and pst-survey cnsisted f 28 questins designed t capture student demgraphic infrmatin; current attitudes and behavirs and the impact f visual media t affect the teen decisin-making prcess. Three (3) questins are intended t btain student demgraphic infrmatin, including gender, grade, and ethnicity. The pre-survey included tw (2) qualitative questins designed t btain input n causes f stress in the lives f participants and whether they preferred t btain infrmatin critical fr decisin making thrugh nline media r vides, r classrm discussins and lectures. Fur (4) additinal qualitative questins were used in the pst-survey t request input n whether students viewed stress differently after the prgram and their preferred means f btaining decisin making infrmatin thrugh either nline media r vides, r classrm discussins and lectures. Participants were als asked the mst imprtant thing learned frm the prgram, as well as hw the prgram culd be imprved. Respnses were affinitized t rganize and summarize the narratives int lgical grupings t better understand the essence f the anecdtal respnses. Survey Results A ttal f 775 pre-surveys and 737 pst-surveys were received, and f these 652 (334 in the experimental grup and 318 in the cntrl) were cmplete and usable fr later paired cmparisn. Assessments were prvided by the facilitatrs n the effectiveness f each sessin. Pwer analyses indicated that sample size prvided sufficient pwer (pwer =.84) t detect effects f the curriculum. Of the paired surveys analyzed, the cntrlled and experimental grups are cmparable. GENDER EXPERIMENTAL CONTROL TOTAL PERCENT Female % Male % GRADE EXPERIMENTAL CONTROL TOTAL PERCENT 6 th % 7 th % 10

11 8 th % 9 th % 10 th % 11 th % 12 th % ETHNICITY EXPERIMENTAL CONTROL TOTAL PERCENT White % Hispanic % African American % Multi-racial % Other % Asian % American Indian % Pacific Islander % Ntes fr Interpreting Results Reliability is the cnsistency f a measurement, r the degree t which an instrument measures the same each time it is used under the same cnditin with the same subjects (Trchim, p. 80). It is the repeatability f a measurement. A measure is deemed reliable if the result f a test given twice is similar each time. Reliability is estimated, nt measured. Validity refers t the degree t which a study accurately reflects r assesses the specific cncept that the researcher is attempting t measure (Trchim, p. 56). External validity refers t the extent t which the results f a study can be generalized r transferred. Internal validity refers t (1) the rigr with which the study was cnducted, including the study's design, the care taken t cnduct measurements, and decisins cncerning what was and was nt measured; and (2) the extent t which the designers f a study have taken int accunt alternative explanatins fr any causal relatinships they explre (Huitt, 1998). 11

12 Results fr each questin, presented in Appendix II, are shwn separately fr each questin fr respndents. Results frm the pre- and pst-survey fr bth cntrl and experimental grups are shwn side-by-side t examine change in crrect respnses ver time. We expected t find imprvement in measured respnses f the experiment grup in the pst-survey, which immediately fllwed the fur classrm sessins and three hmewrk assignments. Differences between survey times (pre- and pst-survey) were tested fr statistical significance using repeated measures analyses f variance (ANOVA). In additin, gender and ethnicity were als tested fr statistical significance using ANOVA. The Tukey methd f pairwise cmparisns f the ppulatin means was used t cmpare every pssible pair f means using a single level f significance at =0.05 (Aczel & Sunderpandian, 2009). The mean ( ) indicates the average respnse fr each questin, using the Likert scale, while the standard deviatin ( ) indicates the variatin f all respnses t the questin frm the mean. [A lw standard deviatin indicates that the data pints tend t be very clse t the mean, whereas high standard deviatin indicates that the data are spread ut ver a large range f values.] Statistical significance was determined using the F-test f equality f variance with a prbability factr (p) f 0.05% (95% cnfidence). The larger the F value, the greater the significant difference (increase r decrease) between the respnse ppulatins (pre- and pstcurriculum) (Trchim & Dnnelly, 2007). ASSESSMENT OF STUDENT FEEDBACK The narrative respnses t the qualitative questins included in the pre- and pst-survey were affinitized t rganize and summarize the narratives int lgical grupings t prvide fr distillatin f the anecdtal respnses. Fr the pre-survey questin regarding surces f stress and the ability t psitively manage it, female participants tended t be mre frthcming with their feelings; hwever, males als shared their thughts abut tpics including: 1. Schl wrk and plans fr the future. (43%) 2. Feelings f inadequacy r discnnectin frm peers, including bullying in schl and weight issues. (35%) 3. Family issues, such as health f parents/grandparents, divrce, bullying by siblings. (22%) Examples f cmments included: My cause f stress f life is that my English teacher is s mean and my parents are a ttal neglect. I try t get ver the stress, but I just burst int depressin. 12

13 Sme f the causes f stress in my life is [sic] my brther. He is 13 and he punches me and hurts me a lt and he yells at me all the time, s when my friends ask why I have a lt f bruises. I lie because I dn't want t get int truble if he hears abut it. Sme f the causes f stress in my life is that my grades in schl have been getting lwer. I'm nt ding very well in schl, and I have truble turning my wrk in n time. Als, I'm stressed because peple always make fun f me because I have acne n my face. T deal with the stress in a psitive manner, I try t ignre the things peple tell me, but smetimes it's t much fr me and actually hurts my feelings... Keeping up with my grades, wrk, my friends, my lved nes and keeping my rm clean; als paying bills my parents have given me the respnsibility t pay and managing time and mney. I usually just plan everything ut ahead f time and take n each day as its wn thing. Well, my stress is caused by my hme life, abusive sister, all my schl wrk, nt having enugh f my wn space frm peple, and peple always asking the same questins ver and ver. I am nt crazy nly in a fun kidding way, but nt a psych girl. Als sme f the ways I get rid f my stress is reading/writing bks, listening t music, drawing, and singing Watching ver my stepmthers' kids smetimes. I breathe in and ut fr abut 15 secnds. Mst f the stress in my life is frm my family and what my sister say and tell and make fun f me...my lder sister likes t push me arund and push me dwn...sme f the things is abut my mm and what she s dne in the past... and I've been raised t keep my muth shut abut everything... and hw I deal with it is t hide the pain behind a mask. My mm smkes and drinks and wn't stp when I tell her t and it makes me mad because that shws she desn't care. She says that she des but I g t my rm and call up a friend r my dad and get away r I just lie dwn and try t calm dwn. In the pst-survey, thse participates in the experimental grup (52%) indicated an increase in the ability t psitively manage stress, while the cntrl grup either did nt respnd r indicated n change. Yes. I knw nw that I am getting stressed by factrs in my life. But I als knw that it is being relieved in healthy ways. Yes, I talk t my parents abut my stresses and it helps me a lt. 13

14 I deal with a lt f stress because my parents are in a really bad divrce and fighting ver us. Als the ecnmy is super bad s I can't buy everything I want. Als, yesterday my mm mved away fr 3 mnths t help her parents. Yes I used t think that stress was smething wrng with me that I have t just deal with. Nw I knw differently. In respnse t questins regarding a preference fr btaining infrmatin, students in bth the cntrl and experimental grups indicated a slight preference fr vide r nline media ver classrm discussin in the pre-survey (56% vide, 44% classrm discussin). After the 10-week prgram was cmpleted, students in the experimental grup indicated a 61% preference fr vide learning versus 38% classrm; hwever, 65% f thse indicating a preference fr classrm discussin suggested that it shuld be in cmbinatin with vide r nline media presentatin. Students in the cntrl grup did nt indicate a significant shift in learning frmat preference. In respnse t questin regarding the imprtance f the prgram, students indicated: I kind f lked at my bdy wrng. It makes me feel guilty but it des change what I think. It als made me think I'm making the wrng chices. The mst imprtant thing that I learned was nt t be pressured int ding smething that will fllw me fr the rest f my life. The mst imprtant thing that I learned frm this prgram was hw t maintain a healthy lifestyle, hw t manage stress, and hw much drugs influence negative behavir. Even thugh yu are grwing mre mature and wiser, yur health is a big part f life that yu shuld take seriusly. I learned hw dangerus prn is and hw easy it is t get drawn int it. I learned a lt thrugh the nline media and vides but I enjyed the discussins with the grup. I have a greater understanding that things that I say and d might bully thers when I dn t mean t. I never thught abut cyber bullying befre. I kind f lked at my bdy wrng befre. This prgram helped me t change what I thught and t make better chices. 14

15 I dn t knw if there is a way t imprve this prgram. I lve it the way that it is right nw. It has helped me a lt. Stuff I learned is helping me t deal with my parent s divrce. Thank yu! I think differently nw abut the causes f stress in my life and make better chices t deal with prblems. Negative cmments received in respnse t questin regarding the imprtance f the prgram, students indicated: Nthing really except fr the usual "dn't smke" and "stay in schl" kind f things. I knew mst f this already. Nt that much. Just basic stuff. ASSESSMENT OF FACILIATOR FEEDBACK The rle f the facilitatr is t guide students thrugh the prgram. Prviding infrmatin alne has nt prven effective in preventing use, s the curriculum emplys interactive activities and multimedia resurces t engage students within their peer grups. The peer grup is intended t prvide a frum fr develping new behavirs, giving and receiving feedback, and prviding psitive supprt. This help and supprt frm thers is prvided thrugh the aid f the facilitatr as a trusted adult. Fidelity Facilitatrs frm 13 f the participating schls respnded t the request fr feedback. Of the respndents, 53.8% are female and 46.2% are male, with 76.9% f the respndents White and 23.1% Black/African American. Fr each f the ten class sessin, facilitatrs were asked fr feedback n their preparatin prir t each sessin and hw effective they thught each sessin was in supprting healthy decisin making. Overall, 69.2% f the facilitatrs rated the curriculum as Effective, while 30.8% indicated that it was Extremely Effective. The Facilitatr Survey cnsisted f fur (4) questins fr each sessin, and facilitatrs may have selected multiple respnses fr a given questin: 1. Prir t the sessin n making healthy chices, I prepared by reviewing the curriculum by reviewing the materials and cnsidering: What teens may think r feel abut the tpic. Hw teens may react t the material presented. Gals f the lessn. 15

16 Hw t ensure participatin. 2. I presented the vide and led the students in discussin. Students were willing t share their pinins. Students needed encuragement t share their pinins. Students discussed all questins fllwing vide. Students were reluctant t discuss questins fllwing vide. 3. I presented the activity and assisted students in small grup discussins. Students were willing t share in peer grups. Students needed encuragement t share in peer grups. Students cmpleted the activity. 4. Hw effective d yu think this sessin was in equipping students with infrmatin t make healthy chices? Nt Effective At All A Little Effective Effective Extremely Effective Managing Healthy Chices In respnse t Questin 1 regarding preparatin fr sessin, 61.5% f the respndents indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 46.2% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 76.9% f the respndents. Ensuring participatin was a cnsideratin fr 46.2% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 76.9% f the respndents felt that students were willing t share their pinins. Students indicated that 15.4% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 61.5% reprted that students discussed all questins fllwing vide, and 7.7% felt that the students were reluctant t discuss questins fllwing the vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. One hundred (100%) f the facilitatrs felt that students were willing t share their pinins n making healthy chices with their peers, and that 46.2% f the students cmpleted the activity. 16

17 In respnse t Questin 4, this mdule was cnsidered Effective by 84.6% f the facilitatrs, while 15.4% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Managing Stress Regarding preparatin fr sessin (Questin 1), 69.2% f the respndents indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 46.2% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 76.9% f the respndents. Ensuring participatin was a cnsideratin fr 38.5% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 76.9% f the respndents felt that students were willing t share their pinins. Students indicated that 15.4% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 46.2% reprted that students discussed all questins fllwing vide, and 7.7% felt that the students were reluctant t discuss questins fllwing the vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding facilitatrs, 69.2% felt that students were willing t share their pinins n making healthy chices with their peers, while 15.4% needed encuragement. Respndents indicated that 46.2% f the students cmpleted the activity. In respnse t Questin 4, this mdule was cnsidered A Little Effective by 15.4% f the respndents, while 69.2% f the Facilitatrs rated the sessin as Effective and 15.4% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Depressin and Anxiety Regarding preparatin fr sessin (Questin 1), 69.2% f the respndents indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 69.2% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 53.8% f the respndents. Ensuring participatin was a cnsideratin fr 38.5% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 76.9% f the respndents felt that students were willing t share their pinins. Students indicated that 15.4% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 46.2% reprted that students discussed all questins fllwing vide, and 7.7% felt that the students were reluctant t discuss questins fllwing the vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding Facilitatrs, 69.2% felt that students were willing t share their pinins n making healthy chices with their peers. Respndents indicated that 69.2% f the students cmpleted the activity. 17

18 In respnse t Questin 4, this mdule was cnsidered A Little Effective by 15.4% f the respndents, while 76.9% f the Facilitatrs rated the sessin as Effective and 7.7% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Over-the-Cunter and Prescriptin Drug Abuse In respnse t Questin 1, 61.5% f the Facilitatrs indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 69.2% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 76.9% f the respndents. Ensuring participatin was a cnsideratin fr 46.2% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 76.9% f the respndents felt that students were willing t share their pinins. Students indicated that 30.8% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 46.2% reprted that students discussed all questins fllwing vide. Fllwing presentatin f the activity (Questin 3), the Facilitatrs assisted students in small grup discussins. Of the respnding Facilitatrs, 84.6% felt that students were willing t share their pinins n making healthy chices with their peers, while 15.4% needed encuragement. Respndents indicated that 69.2% f the students cmpleted the activity. In respnse t Questin 4, this mdule was cnsidered Effective by 76.9% f the respndents and 23.1% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Underage Drinking In respnse t Questin 1, 76.9% f the facilitatrs indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 61.5% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 69.2% f the respndents. Ensuring participatin was a cnsideratin fr 53.8% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 84.6% f the respndents felt that students were willing t share their pinins. Of the respndents, 53.8% reprted that students discussed all questins fllwing vide, and 7.7% f the students were reluctant t discuss questins fllwing the vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding facilitatrs, 84.6% felt that students were willing t share their pinins n making healthy chices with their peers, while 15.4% needed encuragement. Respndents indicated that 61.5% f the students cmpleted the activity. 18

19 In respnse t Questin 4, this mdule was cnsidered Effective by 76.9% f the respndents and 23.1% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Tbacc In respnse t Questin 1, 53.8% f the facilitatrs indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 61.5% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 69.2% f the respndents. Ensuring participatin was a cnsideratin fr 46.2% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 69.2% f the respndents felt that students were willing t share their pinins. Students indicated that 15.4% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 53.8% reprted that students discussed all questins fllwing vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding facilitatrs, 76.9% felt that students were willing t share their pinins n making healthy chices with their peers, while 15.4% needed encuragement. Respndents indicated that 61.5% f the students cmpleted the activity. In respnse t Questin 4, this mdule was cnsidered Effective by 76.9% f the respndents and 23.1% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Cyberbullying In respnse t Questin 1, 61.5% f the facilitatrs indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 61.5% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 76.9% f the respndents. Ensuring participatin was a cnsideratin fr 38.5% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 69.2% f the respndents felt that students were willing t share their pinins. Students indicated that 23.1% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 61.5% reprted that students discussed all questins fllwing vide, and 7.7% f the students were reluctant t discuss questins fllwing the vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding facilitatrs, 92.3% felt that students were willing t share their pinins n making healthy chices with their peers. Respndents indicated that 53.8% f the students cmpleted the activity. In respnse t Questin 4, 19

20 this mdule was cnsidered Effective by 69.2% f the respndents and 30.8% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Managing Technlgy In respnse t Questin 1, 61.5% f the facilitatrs indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 53.8% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 61.5% f the respndents. Ensuring participatin was a cnsideratin fr 61.5% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 38.5% f the respndents felt that students were willing t share their pinins. Students indicated that 38.5% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 30.8% reprted that students discussed all questins fllwing vide, and 15.4% f the students were reluctant t discuss questins fllwing the vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding facilitatrs, 61.5% felt that students were willing t share their pinins n making healthy chices with their peers, while 23.1% needed encuragement. Respndents indicated that 53.8% f the students cmpleted the activity. In respnse t Questin 4, this mdule was cnsidered Effective by 69.2% f the respndents and 23.1% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. The sessin was cnsidered Nt Effective At All by 7.7%. Nutritin In respnse t Questin 1, 61.5% f the facilitatrs indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 61.5% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 84.6% f the respndents. Ensuring participatin was a cnsideratin fr 46.2% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 84.6% f the respndents felt that students were willing t share their pinins. Students indicated that 15.4% shared that they felt that needed encuragement in rder t share their pinins. Of the respndents, 38.5% reprted that students discussed all questins fllwing vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding facilitatrs, 84.6% felt that students were willing t share their pinins n making healthy chices with their peers, while 7.7% needed encuragement. Respndents indicated that 38.5% f the students cmpleted the activity. 20

21 In respnse t Questin 4, this mdule was cnsidered Effective by 84.6% f the respndents and 15.4% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Physical Fitness In respnse t Questin 1, 69.2% f the facilitatrs indicated that they prepared fr the class by reviewing the materials and cnsidering what students may think r feel abut the tpic. Of the respndents, 46.2% cnsidered hw students might react t the material presented. The lessn gals were a cnsideratin fr 76.9% f the respndents. Ensuring participatin was a cnsideratin fr 53.8% f respndents. After presenting the vide and leading the students in discussin (Questin 2), 84.6% f the respndents felt that students were willing t share their pinins. Of the respndents, 38.5% reprted that students discussed all questins fllwing vide. Fllwing presentatin f the activity (Questin 3), the facilitatrs assisted students in small grup discussins. Of the respnding facilitatrs, 76.9% felt that students were willing t share their pinins n making healthy chices with their peers. Respndents indicated that 69.2% f the students cmpleted the activity. In respnse t Questin 4, this mdule was cnsidered Effective by 76.9% f the respndents and 23.1% rated it as Extremely Effective in prviding the infrmatin t meet learning bjectives. Qualitative cmments received frm the facilitatrs regarding the prgram include: My students began t understand that they had an pinin and that it was healthy fr them t talk abut each lessn withut fear f being judged because f what they knew r had experienced. My students had lts f infrmatin t ffer and mst f them have actually dne research r had been in ther envirnments where these tpics/lessns had been discussed. But the peer discussins were wnderful. I fund the vides t be very effective in reaching my students. One way t imprve the prgram is fr each lessn t have a shrt assessment and key/rubric fr the teacher. I watched students learn that thers in their class have very similar values. Many students were nt friends in this grup prir t starting the lessns. They bnded a bit during lessns and learned frm each ther while frming psitive pinins f each ther and their chices. I think that my students finished the prgram with a greater understanding that all f the decisins that they make are imprtant fr a healthy lifestyle. One imprvement wuld be t have mre time t cmplete sme activities, and t d lessns in the fall t avid end-f-schl-year interruptins. 21

22 The cyberbullying and managing technlgy was awesme. Students are really int the Internet s it was extremely infrmative fr us. Als the OTC drugs [sessin] was great. It's a pilt prgram -- the vides are fresh and relevant and kept students' attentin. T that end, I wuld say t imprve the prgram yu may want t lk at adding cntent t the vide segments fr each lessn. T add mre lessns t what is already there and prvide mre interactive activities. The discussins were WONDERUL! One way t imprve this prgram is t cmplete sme activities, and d lessns in the fall befre end f schl interruptins. 22

23 Recmmendatins and Prgram Enhancements Based upn student and facilitatr input, upcming prgram enhancements will include the fllwing: Develp a mre substantial training prgram than the ne-hur vide training, including guidebk and teaching aids. Update vide cntent peridically t include the latest research and relevant student prfiles. The creatin f an accmpanying wrkbk t facilitate assignment cmpletin and nging discussin, including rubrics and alignment t state framewrk. References Aczel, A. D. & Sunderpandian, J. (2009) Cmplete Business Statistics, (7th ed.) Bstn: Irwin McGraw Hill. Babbie, E. (1990). Survey research methds (2nd ed.). Belmnt, CA: Wadswrth Publishing Cmpany. Baggett, P. (1984). Rle f tempral verlap f visual and auditry material in frming dual media assciatins. Jurnal f Educatinal Psychlgy, 76(3), Chi, H. J. & Jhnsn, S. D. (2005). The effect f cntext-based vide instructin n learning and mtivatin in nline curses. The American Jurnal f Distance Educatin, 19(4), Dusenbury, L. A., Hansen, W. B.; Giles, S. M. (2003). Teacher training in nrm setting appraches t drug educatin: A pilt study cmparing standard and vide-enhanced methds. Jurnal f Drug Educatin, 33(3), Gitterman, A. (1989). Building mutual supprt in grups. Scial Wrk with Grups, 12(2), Gitterman, A. (2004). The mutual aid mdel. In C. Garvin, L. Gutierrez, and M. Galinsky (Eds.), Handbk f scial wrk with grups (pp ). New Yrk and Lndn: The Guilfrd Press. Gitterman, A. & Shulman, L. (Editrs) (2005). Mutual aid grups, vulnerable & resilient ppulatins, and the life cycle (3rd editin). New Yrk: Clumbia University Press. 23

24 LeCry, C. W. & Wtn, L. (2006). Scial skills training in schl settings: Sme practical cnsideratins. In R. Cnstable, C.R. Massat, S. McDnald, and J.P. Flynn (Eds). Schl scial Letendre, J. & Wayne, J. (2008): Integrating Prcess Interventins int a Schl-Based Curriculum Grup, Scial Wrk With Grups, 31(3-4), Letendre, J. (2007). Take Yur Time and Give It Mre: Supprts and Cnstraints t Success in Curricular Schl-Based Grups. Scial Wrk With Grups, 30(3), Sands, R.C. and Slmn, P. (2003). Develping educatinal grups in scial wrk practice. Scial Wrk with Grups, 26 (2), Schwartz, W. (1961). The scial wrker in the grup. In B. Saunders (Ed.), New perspectives n services t grups: Thery, rganizatin, practice (7-29), New Yrk: Natinal Assciatin f Scial Wrkers. Shulman, L. (1986). The dynamics f mutual aid. In A. Gitterman and L. Shulman (Eds.), The legacy f William Schwartz: Grup practice as shared interactin (51 60). New Yrk: Hawrth Press. Trchim, W. M. K. & Dnnelly, J. P. (2007). (3rd ed.). Independence, KY: Cengage Learning. Vygtsky, L. S. (1978). Mind in sciety: The develpment f higher psychlgical prcesses. Cambridge, MA: Harvard University Press. Werch, C.E., Mre, M.J., DiClemente, C.C., Bledse, R., & Jbli, E. (2005). A Multi- Health Behavir Interventin Integrating Physical Activity and Substance Use Preventin fr Adlescents. Preventin Science, 6(3), Zmud, R.W. & Armenakis, A.A. (1978). Understanding the measurement change. Academy f Management Review, July,