Program Evaluation: What s in it for me? Lucia Kaiser, UCCE Specialist Dept of Nutrition, UC Davis

Program Evaluation: What s in it for me? Lucia Kaiser, UCCE Specialist Dept of Nutrition, UC Davis

UC CalFresh Evaluation Taskforce Members Co-chairs: Lucia Kaiser and Connie Schneider, Marisa Neelon, Brenda Roche, Chutima Ganthavorn, Margaret Johns, Susan Algert, Jennifer Benke-Delia, Concepcion Mendoza, Tammy Mc Murdo Jennifer Quigley, Gretchen George, David Ginsburg

Create and share knowledge about best practices Advocate for programs Improve outcomes and impact Meet USDA requirements

How UC CalFresh contributes to outcomes Youth UC Cal Fresh Garden-based nutrition education Food service Family and Adult UC CalFresh Better nutrition and health of individuals Healthy diet & lifestyle and home/school environment Self-esteem, youth development, academic performance Decrease obesity and chronic disease Healthy productive workforce and populace FSNEP Academic Meeting, Feb 10, 2010

Youth Evaluation Data Sources TOT & TTT Pre-post tests EAT FIT retro Parent surveys

Teacher Observation Tool (TOT) Compared to the beginning of the year, more students now: Bring fruit as a snack Wash hands more often Can identify healthy food choices Are willing to try new foods Choose fruits and/or vegetables in the cafeteria or during classroom parties

Percent (%) of teachers responding, n=753 teachers, 18,672 students 60 50 2011 State data 40 30 20 Strongly agree Agree Not sure Disagree 10 0 Snack Wash Identify Willing Choose

Meeting USDA reporting requirements Among 18,672 children participating in the UC CalFresh nutrition education and food tasting activities, 17,551 (as reported by teachers) are more able to identify healthy food choices.

Is use of different curricula related to better outcomes? Percent of teachers reporting that students are more willing to try new foods 60 50 40 30 20 Strongly agree Agree Disagree 10 0 One Two Three or more N= 724 teachers, 2011 State data, p < 0.01

How does the Retrospective TOT compare to a pre-post TOT? P < 0.0001, n=61

More data needed to see if TOT truly reflects change Student knowledge increased from 71% to 81% correct (p <0.0001) Strongly agree that students can identify healthy choices tended to be associated with greater improvement in test scores

Youth report behavior changes (n=1051, Riverside 2011 data) 92% report at least one behavior change 30.9% eat breakfast more often 29.4% eat more fruit and vegetables 35.2% eat less fast food and snack foods 30.8% drink less sweetened beverages

UC Cal Fresh Wish list Use TOT with classroom teachers do not lump data across classes Participate in further validation efforts with TOT (w/ good followthrough) Use other evaluation tools (EatFit Retro) for older students

Taste Test Tool (TTT) Seen Before: Before today s class, how many students have seen this food before? Tried Before: Before today s class, how many students have tasted this food before? Tried Today: How many students ate (or tasted) the food today? Try Again (School): How many students were willing to eat the food at school again? Try Again (Home): How many students were willing to eat this food at home? Ask for Food: How many students were willing to ask for this food at home?

2011 state data Response (%) of youth in UC CalFresh to all foods, n=649 teachers, 16,055 students

Meeting USDA reporting requirements Among 16,055 children and youth participating in the UC CalFresh nutrition education and food tasting activities, 12,142 reported being willing to ask for that specific healthy food at home.

80 70 60 50 40 30 20 10 0 Percent (%) of students reporting ever having tried this food before 2011 data, P < 0.0001 Grains Fruit Beans Other Vegetable Dairy Nuts & Seeds

Using the findings to improve programs Previous Exposure (Ever tried) Low (< 50%) High (> 50%) Willing to Ask Beets, Grapefruit, Red Peppers, Edamame, Purple Cabbage Berries, Cantaloupe, Oranges, Pears, Green Cabbage, Cucumbers Not Willing to Ask Butternut Squash, Garbanzo Beans, Bean Sprouts Green (Bell) Peppers Banana Chips

UC CalFresh Wish list Look at TTT data to select foods that are unfamiliar (more veggies, beans) Be clear about the target food Use TTT with elementary school youth Read guidelines on UC CalFresh website

Does a child s willingness to ask for a new food at home actually result in this behavior? % of Parents who reported a child request 70 60 * ** ** 50 40 30 Pre Post 20 10 0 Cantaloupe Nectarine Plum * p < 0.05 **p < 0.01 Fresno County, 2010, n=55

Impact! Significant decrease in waist circumference Wt loss in obese children Fresno County, 2010, n=55

Adult Evaluation Data Sources FBC Intent to Change Make Every Dollar Count Adult TTT ESPH

Intent to Change: Adult Workshops Cognitive testing in 36 groups (Eng & Spa) Develop visuals and script Data entry portal on UC CalFresh website During the past week, did you eat fruit at least twice a day? Yes No Within the next week, how often will you eat fruit? Same as before More often Less often.

Make Every Dollar Count Knowledge BEFORE Program Knowledge AFTER Program Low High Low High 1 2 3 4 5 Setting Personal Goals 1 2 3 4 5 1 2 3 4 5 Understanding Values 1 2 3 4 5 1 2 3 4 5 Knowing the Difference Between a 1 2 3 4 5 Need & a Want 1 2 3 4 5 How to Make Choices 1 2 3 4 5 1 2 3 4 5 Knowing Personal Skills & Resources 1 2 3 4 5 1 2 3 4 5 Knowing Community Resources 1 2 3 4 5 1 2 3 4 5 Using Resources to Make Your 1 2 3 4 5 Money Go Further 1 2 3 4 5 Knowing Easy Ways to Save Money 1 2 3 4 5 on Food 1 2 3 4 5 Knowing Simple Healthy Meals to 1 2 3 4 5 Make at Home 1 2 3 4 5 Understanding Food Ads 1 2 3 4 5

Make Every Dollar Count: 81% know more about food ads 85% know more about saving money on food 18% reported comparing food prices & 76% intend to do so in the future

UC CalFresh Wish List Conduct a controlled study to validate Intent to Change questions Use MEDC retrospective data portal Design and conduct more impact studies with control or comparison groups More UC Delivers stories

Happy Holidays and a Wonderful New Year!

Childhood Obesity Conference, June 28-30, 2011 San Diego UC-FSNEP Changing The Classroom Environment Ganthavorn C, Neelon M, Roche B, and Kaiser LL Evaluating Youth Preferences for Fruit and Vegetables Mendoza C, Schneider C, George G, Ginsburg D, and Kaiser L Promoting Healthy Living through a Youth Summer Camp Schneider C, Mendoza C, George G, Cogdill A, and Kaiser L

Does FSNEP encourage children to try new foods? Children wash hands to prevent flu and other illness UC-FSNEP collaborates with a Healthy Lifestyle Fitness Camp FSNEP helps children learn about food