2016-2017 EHDI WEBINAR BUILDING A LOGIC MODEL AND AN EVALUATION PLAN October 27, 2016
Before We Begin Webinar Housekeeping You have been automatically set to Mute #6 to Unmute *6 to Re-Mute Quick Polls throughout the Webinar If you have a question at any point during the Webinar, type it in Question Box at bottom-right of screen Choice of directing the question to someone in particular, or to entire group 10/27/2016 2
WE WILL COVER TODAY LOGIC MODEL What is it? Uses Parts of a LM Writing good outcomes Other LM examples EVALUATION PLAN What is it? Uses Parts of an EP Examples 10/27/2016 3
LOGIC MODEL 10/27/2016 4
10/27/2016 5
What is a Logic Model? Graphical illustration of how your program works Roadmap or blueprint for how you will reach your outcomes Dynamic, living document 10/27/2016 6
Why do we need a Logic Model? Brings key stakeholders together for discussion, debate, and consensus-building Strengthens program Help you stay focused and keeps you on track Helps to communicate what your program is about 10/27/2016 7
Parts of a Logic Model 1. Inputs 2. Activities 3. Outputs 4. Outcomes (short, intermediate, long) 5. Assumptions 6. External Factors *These occur in a logical sequence *Like baking bread 10/27/2016 8
10/27/2016 9
INPUTS: Resources Examples: funding, staff, partners, curriculum, supplies ACTIVITIES: What you do with your inputs Examples: Recruitment, education, outreach, advocacy, screening OUTPUTS: Products of your activities (volume of work accomplished) Examples: # participants, # classes, # screenings, # brochures 10/27/2016 10
OUTCOMES Timeline is relative Short term: changes that take place immediately after an activity or program ends Knowledge, Awareness, Skills Intermediate term: changes that take place after several months or years Behavior Long term: changes that take place after several years Status, condition, policy/system 10/27/2016 11
Example of Short, Intermediate, and Long Term Outcomes Teen pregnancy prevention program targeting African American girls ages 13-18 in MPS 10/27/2016 12
ASSUMPTIONS & EXTERNAL ASSUMPTIONS If not accounted for, will impact how your program s success (or lack of success) is viewed Provides non-program people with extra information on how you got to where you are now 10/27/2016 13
ASSUMPTIONS Underlying beliefs about how your program will work Example: For your Teen Pregnancy Prevention Program to work well, you are assuming that the following hold true: Program resources are adequate and available (funding, classrooms, transportation, etc.) A culturally appropriate curriculum will be developed and implemented effectively A health educator certified in the curriculum will deliver it The teens parents will give their permission Knowledge change will lead to behavior change 10/27/2016 14
EXTERNAL FACTORS Conditions in the environment in which your program exists over which you have little control Examples: Political climate Social, economic and demographic changes that may affect recruitment and participation Local or national events that may influence public support for your program Changes in city, county, state or federal laws Changes in legislation Changes in organization or MDH (leadership, policies, funding priorities) 10/27/2016 15
10/27/2016 16
10/27/2016 17
Good Outcome Statements 1. Indicate a positive change that benefits the community 2. State the intended beneficiary 3. Have only one change targeted (not multiple changes) 4. Attainable / Measurable 5. Stated in the present tense 10/27/2016 18
Example of Outcome Beneficiary Participants in the A Healthy Heart CVD prevention course increase their level of physical activity. Positive Change 10/27/2016 19
Let s start with #5 5. Stated in the present tense Participants who receive BP screening will understand the results. BETTER: Participants who receive BP screening understand the results. 10/27/2016 17
1. Indicate a positive change People will know about risk factors BETTER: Participants in the A Healthy Heart program increase their awareness of risk factors for developing cardiovascular disease 10/27/2016 18
2. State the intended beneficiary Increased access to mental health services BETTER: African American youth who experience depression increase their access to mental health services 10/27/2016 19
3. Targets only one change Young parents will have clear educational plans, better space future children, and actively pursue future goals BETTER: Young parents in the program have clearly articulated educational goals in their Individual Development Plans Young parents in the program report their intention to better space children in the future Young parents in the program are actively pursuing their future educational goals 10/27/2016 20
4. Are attainable / measurable Youth who participate in the program will grow up to be healthy contributing members of society BETTER: Unintended pregnancies decrease among African American adolescents 10/27/2016 21
Traditional Logic Model vs. Reality 10/27/2016 25
Your logic model must reflect reality! It s important to get input from people on the ground who implement your program. 10/27/2016 26
Reality There are cycles, iterations, interrelationships and interactions among program elements Holistic programs focus on: Individual, family, community Physical, spiritual, mental, emotional, social No two people follow the same path toward healing or recovery Target population has changing needs so need to adapt Cultural values and traditions have to be respected and reflected in approaches and strategies 10/27/2016 26
Traditional Logic Model Based on Eurocentric tradition of logical reasoning that emphasizes and values a cognitive approach (thinking; conceptual) Assumes everything occurs in a linear progression, straight sequence e.g., participants move through program progressively and achieve goals in same order Gives impression that outcomes are hierarchical, and some have higher priority than others 10/27/2016 25
Minneapolis American Indian Center 10/27/2016 27
Indigenous Peoples Task Force 10/27/2016 28
Pacific Islander (Polynesian) 10/27/2016 29
EVALUATION PLAN 10/27/2016 32
10/27/2016 33
What is an Evaluation Plan? Shows how you will monitor and measure progress on specific aspects of your program May also include communication plan Based on logic model 10/27/2016 34
1. Outputs and Outcomes Outputs/Process Volume of work accomplished What you did Outcomes What you are holding yourselves accountable for 10/27/2016 35
2. Guiding evaluation questions Outputs/Process How much did we do? How well did we do it? Outcomes What has changed? How did it change? Who is better off? How are they better off? 10/27/2016 36
3. Indicators Specific information you collect to determine the success of your program (evidence) Answers the question: How will you know it? Must align with outcomes 10/27/2016 37
4. Method Method you will use for gathering evidence Can be quantitative or qualitative Examples: Intake forms, attendance sheets Surveys Interviews Focus groups Document review 10/27/2016 38
5. Sample Individuals from whom you will collect your outcome data Questions to ask: Who is the best source of information for what I want to know? Should I focus on all participants or a sample of participants? How many? If I want to do a follow-up after participation, will I be able to contact them? What s the best way to gather information oneone-one or group; in-person, phone, or mail? 10/27/2016 39
6. Schedule When and how often will you collect the information? Can be: Ongoing (intake forms) Pre & Post, Post only Every year, every 2 years 10/27/2016 40
QUESTION How many participants completed the program? OUTPUT 25 participants complete the program OUTPUT EXAMPLE INDICATOR Number of participants who completed at least 7 out of the 10 sessions METHOD Sign-in sheets SAMPLE All program participants SCHEDULE Every year at the end of the last session 10/27/2016 41
QUESTION Did we implement the A Health Heart curriculum as designed? OUTPUT 10 two-hour sessions held per program year PROCESS EXAMPLE INDICATOR Number of sessions held per program year METHOD Sign-in sheets SAMPLE Program coordinator SCHEDULE Every year at the end of the last session 10/27/2016 42
OUTCOME EXAMPLE QUESTION: To what extent did A Healthy Heart program participants increase their knowledge of CVD risk factors? OUTCOME: Participants in the A Healthy Heart program increase their knowledge of CVD risk factors INDICATOR 1: Ave participant score on the 10-item knowledge section of the Pre-test vs. at Post-test INDICATOR 2: % of participants whose scores on the 10-item knowledge section increased from Pre to Post METHOD: Pre-post test SAMPLE: All program participants SCHEDULE: Pre-test at 1 st session in August and Posttest at last (10 th ) session in January 10/27/2016 43
Evaluation Questions Mia Robillos mia.robillos@state.mn.us 651-201-5406 10/27/2016 44