Brain Injury Association of Massachusetts Brains At Risk Evaluation Project Final Report

Transcription

1 Brain Injury Association of Massachusetts Brains At Risk Evaluation Project Final Report Survey designed and data compiled and analyzed by Mr. BJ Williams, Manager of Prevention and Court-related Programs Brain Injury Association of Massachusetts July

2 2

3 Acknowledgements This report was a group effort. Though the survey and analysis was completed by BJ Williams, Manager of Prevention and Court-related Programs, the staff at the Brains At Risk sites and the presenters who present the Brains At Risk programs. Sites and presenters include: Acton Safety Facility / Dan Stewart Baystate Medical Center / Christine Walton Braintree Rehabilitation Hospital / Meline Nichols Cambridge Safety Office / Dan Stewart Fall River Police Department / Doreen Huffman Haverhill Police Department / Dan Stewart Head Injury Community Services /April Sheridan Marlboro Police Station / Erika Updegrove Neuro-Rehabilitation At Worcester / Stephanie Goodrum Spaulding Rehabilitation Hospital / Sandra Villante Stoneham Police Station /Erika Updegrove Wakefield Safety Building /Huntley Skinner Warren House / Dan Stewart Westminster Police Station / BJ Williams Wrentham Police Station / Alex Burnham The Brain Injury Association of Massachusetts would like to extend a special thank you to the respondents, who committed their time and effort to completing the program evaluations. 3

4 4

5 Executive Summary Brain Injury Association of Massachusetts Brains At Risk Evaluation Project By: BJ Williams Purpose: This survey was conducted for the purpose of providing the Brain Injury Association of Massachusetts (BIA-MA) with an understanding of Brains At Risk attendee attitudes and opinions regarding the seminar and program effectiveness. This report analyzes data collected as a result of the evaluations. Methodology: The survey was conducted with a sample of individuals that attended the Brains At Risk program in July 2011 through June Respondents completed the evaluation after attending the Brains At Risk seminar at one of the twenty locations throughout the Commonwealth. Respondents had the option of completing the survey at the conclusion of the seminar, however they were not asked to include identifying information. A total of 2,067 useable surveys were returned in time to be included in this report. The evaluation questions were developed by BJ Williams, Manager of Prevention and Court-related Programs, after extensive review of the published research on program evaluation and prior prevention program evaluations conducted by BIA-MA. The final questionnaire included basic demographic questions, in addition to questions designed to learn about satisfaction with the seminar and attendees options regarding the effectiveness of the program curriculum. 5

6 Background Information Purpose of the Survey In January 2008, the Brain Injury Association of Massachusetts conducted the first Brains At Risk program survey throughout the Commonwealth. The purpose of the evaluation is to provide the Brain Injury Association of Massachusetts and the Massachusetts District Courts with an understanding of Brains At Risk attendee satisfaction and opinions regarding program effectiveness. Background Information The Brains At Risk program was established over fifteen years ago and provides education to over 2,000 attendees annually (2,067 FY ). The Brains At Risk program aims to be an effective, prevention and awareness program; the mission of Brains At Risk is to promote responsible driving and raise awareness of traumatic brain injury in order to prevent fatalities and serious injury on Massachusetts roads. The attendees of the Brains At Risk program are overwhelmingly male attendees (75.3 percent male, 24.7 percent female) and 41 percent (846 respondents) fall under the age of 25 years old percent (1,401 respondents) were referred to the program for an impaired driving offense. Brains At Risk, faced with educating an offender population, aims to ensure the connection of choices made behind the wheel and their relationship to brain injury is delivered in a clear and powerful program curriculum. Thus, administration has sought to learn how attendees respond to the current program curriculum. The administration s general questions include: how likely are attendees to apply the information learned in the seminar to their life, and what is the most impactful portion of the program curriculum? 6

7 Methodology Sampling In order for the Brain Injury Association of Massachusetts to gauge attendee satisfaction and opinions, it was determined that a sample including attendees from all program sites over a twelve month period would be the most effective method. Questionnaire Design The questionnaire was designed after reviewing published data on surveys, program evaluation, and research previously conducted on the Brains At Risk program. To be consistent with program evaluation research, but to maintain anonymity of the individuals, the survey only contained general demographic information (age, gender). Understanding that question order effects can greatly change or skew results of the data, the evaluation design placed questions in an effort to minimize such effects. General questionnaire design guidelines governing concerns about how well respondents understand the questions, whether they are willing to provide the information was a consideration. Respondents may purposefully withhold facts because they are inclined to provide the socially desirable answers (e.g. answers that are in line with what the respondent believes to be socially acceptable). Additionally, respondents may not have accurate memories regarding behavior or incidents, which may also influence the validity. This should be noted as a limitation within the research herein. Sample Size and Quality Using non-probability convenience sampling design, 2,067 completed evaluations from Brains At Risk seminars conducted throughout the Commonwealth from July 2011 through June Although the risk of bias for convenience samples is high; this is a concern only when the subjects select themselves for the sample and in which we would expect only people with strong views to respond. All Brains At Risk attendees were asked to complete the program evaluation. 7

8 Sampling error always results from the process of selecting one unit from another, instead of selecting the entire population. Although unavoidable, non-probability sample error s effect on the accuracy of the estimates obtained from the sample evaluation can be measured precisely. Sample size plays a crucial role in determining the accuracy of sample estimates since generally speaking a larger sample size will yield greater accuracy in the data. The sample size accounts for 98 percent of all attendees of the Brains At Risk program in BIA-MA FY

9 Findings Note: Complete answers can be found in Appendix 1: Codebook. Demographics Respondents were asked about gender, age, and offense to determine who the respondents were and what lead to them being referred to the program. The following are the highlights: 36.5 percent of respondents were within the year age range and another 29.2 percent of attendees were within age range. Over 75 percent of respondents were male. Nearly 69 percent of respondents were referred for an impaired driving violation percent (1,362 respondents) said they were very satisfied with the seminar, and another 33.0 (683 respondents) were satisfied with the seminar percent (1,928 respondents) indicated that the program was appropriate for them percent (2,041 respondents) indicated they strongly agree or agree the program is successful in encouraging attendees to wear a seatbelt percent (1,991 respondents) indicated they strongly agree or agree the program is successful in encouraging attendees to drive sober. 9

10 Figure 1 highlights that nearly 69 % of all respondents were referred for an impaired driving violation. It is also demonstrated that 6% are classified in the category of other which can be violations of texting and driving and falling asleep behind the wheel Figure 1: Type of Offense (Question 1) Total Offenders DUI OUI OTE MIN in PO Other MIN in PO 13% Other 6% DUI 33% OTE 12% OUI 36% 10

11 Respondents were asked to list if they were first offenders. If they were not first offenders they were asked to write down how many times they have been classified as an offender. As you can see 88% was the total of first offenders. Figure 2: Number of Offenses (Question 2) First Not first Offenses Not first 12% First 88% 11

12 Applicability of Seminar Curriculum 63.5 percent of respondents indicated their level of knowledge regarding brain injuries was fair, poor, or very poor before attending the seminar Figure 3: Rate Prior Knowledge of Brain Injury (Question 8) Very Poor Poor Fair Good Excellent Knowledge Good 26% Excellent 10% Very Poor 8% Poor 20% Fair 36% 12

13 Applicability of Seminar Curriculum 82.4 percent of respondents indicated their after attending the Brains At Risk seminar they would very frequently, frequently, and occasionally use what they learned at the class to change their decision making. Figure 4: Applicability of Seminar Curriculum (Question 10) Applicability Very Freq. Frequently Occasionally Rarely Never Rarely 10% Never 8% Very Freq. 36% Occasional 20% Frequently 26% 13

14 Appropriateness of Seminar 93.3 percent of respondents thought that the Brains At Risk seminar was appropriate for them. Additionally, the three top factors rated by attendees as being the most influential and powerful are the survivor speakers, survivor stories and photos of crashes. Figure 5: Appropriateness of Seminar (Question 11) Yes No Maybe 0 Appropriate No 5% Maybe 2% Yes 93% 14

15 Presenter Ratings Over 97.9 percent of all respondents rated the Brains At Risk program presenter as excellent or good. Less than 3 percent of respondents rated the presenter as fair or poor. FIGURE 6: Presenter Rating (Question 22) Presenter Quality Excellent Good Fair Poor Fair 2% Good 18% Poor 0% Excellent 80% 15

16 male. Sex of Attendees Over 75 percent of all respondents who attended the Brains At Risk program were FIGURE 7: Sex of Attendees (Question 32) Sex Male Female Female 25% Male 75% 16

17 Age of Attendees 41 percent (846 respondents) fall under the age of 25 years old percent (1,401 respondents) were referred to the program for an impaired driving offense. FIGURE 8: Age of Attendees (Question 33) Age Under Over 65 15% 10% 29% 4% 0% 5% 37% Under Over 65 17

18 Summary Respondents expressed high satisfaction with their Brains At Risk program experience. Respondents were most impacted by the brain injury survivor speakers, survivor stories and photos of crashes and the majority indicate that the program is successful in encouraging sober driving, seatbelt use, following posted speed limits, and for individuals to not drive aggressively or while distracted. The evidence herein supports the continued efforts of the BIA-MA and the Brains At Risk program to educate a high-risk offender population through the judicial system in the Commonwealth. The research can also serve as a foundation for future program evaluation. 18

19 Appendix A: Evaluation 19

20 20

21 This evaluation is completely anonymous. The information you share in this evaluation will not effect your successful program completion. Your answers will help us to measure the effectiveness of our program. Please answer honestly. Today s Date: Location of Seminar: 1) What was the offense that led to you being referred to this program? Please explain. 2) Was this your first offense of this kind? Yes No If no, what number offense is this? Rank how often you engage in the following behaviors: 3) Wear a seatbelt 4) Drive under the influence 5) Speed 6) Drive aggressively 7) Drive while distracted Very Frequently Frequently Occasionally Rarely Never 8) Rate your knowledge level about traumatic brain injuries when you came into the course today: Very Poor Poor Fair Good Excellent 9) What did you learn about preventing brain injuries that you didn t know before? 10) Rate how often what you learned today will affect your behaviors after you leave: Never Rarely Occasionally Frequently Very Frequently 11) Do you think that this program was appropriate for you? Why or why not? 12) How has this program affected you? Please explain 13) What will be your lasting memory of the program? Please explain. 21

22

23 14) What did you find to be the most impact full portion of the program? Please explain. Do you think that the program is successful in encouraging attendees to: 15) Wear a seat belt 16) Not drive under the influence 17) Not speed 18) Not drive aggressively 19) Not drive while distracted Strongly Agree Agree Undecided Disagree Strongly Disagree 20) What do you think influences individuals to driver sober, follow posted speed limits and wear a seat belt? 21) Please RANK THE TOP 3 factors that you believe are the most influential and powerful. (1 = Most important) Photos of Crashes Photos of Ejections Photos of Coma Photos of Medical Photos of Survivors Commercials Treatment Facts on TBI Facts on Alcohol & Drugs Facts on Seatbelts Survivor Stories Survivor Videos Survivor Speakers 22) Overall, how would you rate the presenter? Excellent Good Fair Poor Very Poor Please indicate your opinion about the presenter: 23) Answered questions carefully and completely 24) Used examples to make material easy to understand 25) Knowledgeable on the subject material 26) Stimulated interest in the material 27) Treated attendees fairly and impartially 28) Overall, kept my interest in material Strongly Agree Presenter s Name: Agree Undecided Disagree Strongly Disagree 29) Please rate your satisfaction with the class: Very Satisfied Satisfied Dissatisfied Very Dissatisfied 30) What could be done to improve the presenter s teaching effectiveness? 31) What could be done to improve this program? 32) Please complete the following: Male Female Age: Please turn this sheet in to the presenter. Thank you

24 - 24 -

25 Appendix B: Codebook

26 - 26 -

27 The tables below provide a frequency distribution for each question from the survey. Question 1: 1. What was the offense that led to you being referred to this program? RANGE: 1 to 5 Mean: Freq. % 1) DUI ) OUI ) OTE ) MIN in PO ) Other TOTAL Question 2: 2. Was this you first offense? RANGE: 1 to 2 Mean: Freq. % 1) Yes ) No TOTAL Question 8: 8. Rate your knowledge level about traumatic brain injuries when you arrived at the seminar today. RANGE: 1 to 5 Mean: Freq. % 1) Very Poor ) Poor ) Fair ) Good ) Excellent TOTAL

28 Question 10: 10. Rate how often what you learned today will affect your behaviors after you leave. RANGE: 1 to 5 Mean: Freq. % 1)Very Frequ )Frequently )Occasional )Rarely )Never TOTAL Question 11: 11. Do you think that this program was appropriate for you? RANGE: 1 to 3 Mean: Freq. % 1) Yes ) No ) Maybe TOTAL Question 22: 22. Overall, how would you rate the presenter? RANGE: 1 to 5 Mean: Freq. % 1) Excellent ) Good ) Fair ) Poor 1.04 TOTAL

29 Question 29: 29. Please rate you satisfaction with the class. RANGE: 1 to 4 Mean: Freq. % 1) Very Satis ) Satisfied ) Dissatifie ) Very Dissa 2.1 TOTAL Question 32: 32. Male or Female RANGE 1 to 2 Mean: Freq. % 1) Male ) Female TOTAL Question 33: 33. Age: RANGE: 1 to 7 Mean: Freq. % 1) Under ) ) ) ) ) ) Over TOTAL