Program Evaluation: What s in it for me? Lucia Kaiser, UCCE Specialist Dept of Nutrition, UC Davis

Transcription

1 Program Evaluation: What s in it for me? Lucia Kaiser, UCCE Specialist Dept of Nutrition, UC Davis

2 UC CalFresh Evaluation Taskforce Members Co-chairs: Lucia Kaiser and Connie Schneider, Marisa Neelon, Brenda Roche, Chutima Ganthavorn, Margaret Johns, Susan Algert, Jennifer Benke-Delia, Concepcion Mendoza, Tammy Mc Murdo Jennifer Quigley, Gretchen George, David Ginsburg

3 Create and share knowledge about best practices Advocate for programs Improve outcomes and impact Meet USDA requirements

4

5 How UC CalFresh contributes to outcomes Youth UC Cal Fresh Garden-based nutrition education Food service Family and Adult UC CalFresh Better nutrition and health of individuals Healthy diet & lifestyle and home/school environment Self-esteem, youth development, academic performance Decrease obesity and chronic disease Healthy productive workforce and populace FSNEP Academic Meeting, Feb 10, 2010

6 Youth Evaluation Data Sources TOT & TTT Pre-post tests EAT FIT retro Parent surveys

7 Teacher Observation Tool (TOT) Compared to the beginning of the year, more students now: Bring fruit as a snack Wash hands more often Can identify healthy food choices Are willing to try new foods Choose fruits and/or vegetables in the cafeteria or during classroom parties

8 Percent (%) of teachers responding, n=753 teachers, 18,672 students State data Strongly agree Agree Not sure Disagree 10 0 Snack Wash Identify Willing Choose

9 Meeting USDA reporting requirements Among 18,672 children participating in the UC CalFresh nutrition education and food tasting activities, 17,551 (as reported by teachers) are more able to identify healthy food choices.

10 Is use of different curricula related to better outcomes? Percent of teachers reporting that students are more willing to try new foods Strongly agree Agree Disagree 10 0 One Two Three or more N= 724 teachers, 2011 State data, p < 0.01

11 How does the Retrospective TOT compare to a pre-post TOT? P < , n=61

12 More data needed to see if TOT truly reflects change Student knowledge increased from 71% to 81% correct (p <0.0001) Strongly agree that students can identify healthy choices tended to be associated with greater improvement in test scores

13 Youth report behavior changes (n=1051, Riverside 2011 data) 92% report at least one behavior change 30.9% eat breakfast more often 29.4% eat more fruit and vegetables 35.2% eat less fast food and snack foods 30.8% drink less sweetened beverages

14 UC Cal Fresh Wish list Use TOT with classroom teachers do not lump data across classes Participate in further validation efforts with TOT (w/ good followthrough) Use other evaluation tools (EatFit Retro) for older students

15 Taste Test Tool (TTT) Seen Before: Before today s class, how many students have seen this food before? Tried Before: Before today s class, how many students have tasted this food before? Tried Today: How many students ate (or tasted) the food today? Try Again (School): How many students were willing to eat the food at school again? Try Again (Home): How many students were willing to eat this food at home? Ask for Food: How many students were willing to ask for this food at home?

16 2011 state data Response (%) of youth in UC CalFresh to all foods, n=649 teachers, 16,055 students

17 Meeting USDA reporting requirements Among 16,055 children and youth participating in the UC CalFresh nutrition education and food tasting activities, 12,142 reported being willing to ask for that specific healthy food at home.

18 Percent (%) of students reporting ever having tried this food before 2011 data, P < Grains Fruit Beans Other Vegetable Dairy Nuts & Seeds

19 Using the findings to improve programs Previous Exposure (Ever tried) Low (< 50%) High (> 50%) Willing to Ask Beets, Grapefruit, Red Peppers, Edamame, Purple Cabbage Berries, Cantaloupe, Oranges, Pears, Green Cabbage, Cucumbers Not Willing to Ask Butternut Squash, Garbanzo Beans, Bean Sprouts Green (Bell) Peppers Banana Chips

20 UC CalFresh Wish list Look at TTT data to select foods that are unfamiliar (more veggies, beans) Be clear about the target food Use TTT with elementary school youth Read guidelines on UC CalFresh website

21 Does a child s willingness to ask for a new food at home actually result in this behavior? % of Parents who reported a child request * ** ** Pre Post Cantaloupe Nectarine Plum * p < 0.05 **p < 0.01 Fresno County, 2010, n=55

22 Impact! Significant decrease in waist circumference Wt loss in obese children Fresno County, 2010, n=55

23 Adult Evaluation Data Sources FBC Intent to Change Make Every Dollar Count Adult TTT ESPH

24 Intent to Change: Adult Workshops Cognitive testing in 36 groups (Eng & Spa) Develop visuals and script Data entry portal on UC CalFresh website During the past week, did you eat fruit at least twice a day? Yes No Within the next week, how often will you eat fruit? Same as before More often Less often.

25 Make Every Dollar Count Knowledge BEFORE Program Knowledge AFTER Program Low High Low High Setting Personal Goals Understanding Values Knowing the Difference Between a Need & a Want How to Make Choices Knowing Personal Skills & Resources Knowing Community Resources Using Resources to Make Your Money Go Further Knowing Easy Ways to Save Money on Food Knowing Simple Healthy Meals to Make at Home Understanding Food Ads

26 Make Every Dollar Count: 81% know more about food ads 85% know more about saving money on food 18% reported comparing food prices & 76% intend to do so in the future

27 UC CalFresh Wish List Conduct a controlled study to validate Intent to Change questions Use MEDC retrospective data portal Design and conduct more impact studies with control or comparison groups More UC Delivers stories

28 Happy Holidays and a Wonderful New Year!

29 Childhood Obesity Conference, June 28-30, 2011 San Diego UC-FSNEP Changing The Classroom Environment Ganthavorn C, Neelon M, Roche B, and Kaiser LL Evaluating Youth Preferences for Fruit and Vegetables Mendoza C, Schneider C, George G, Ginsburg D, and Kaiser L Promoting Healthy Living through a Youth Summer Camp Schneider C, Mendoza C, George G, Cogdill A, and Kaiser L

30 Does FSNEP encourage children to try new foods? Children wash hands to prevent flu and other illness UC-FSNEP collaborates with a Healthy Lifestyle Fitness Camp FSNEP helps children learn about food