Program Logic Models March Susan J. Snelling, PhD Health Promotion Field Support Specialist, Evaluation

Transcription

1 Program Logic Models March 2012 Susan J. Snelling, PhD Health Promotion Field Support Specialist, Evaluation 1

2 WEBINAR OBJECTIVES At the end of the session, participants will: 1. Understand and appreciate the purpose and use of program logic models. 2. Be able to identify and describe the components and characteristics of effective program logic models. 2

3 AGENDA Overview of logic models: definition, purposes and examples Theory behind logic models Criteria for effective program logic models Questions 3

4 OVERVIEW OF LOGIC MODELS 4

5 What is a Program Logic Model? A graphic depiction of the relationship between the key elements of a program or initiative (i.e., goals, objectives, target groups, and activities). A diagrammatic representation of a program that shows the relationships or the If, then logic among: WHAT: what does the program/initiative do? WHO: who are the recipients of the program/initiative? WHY: what outcomes are to be achieved? 5

6 Sample Program Logic Model Outline Goal Population(s) of Interest Long-term Outcome Objectives Short-term Outcome Objectives Outputs Activities 6

7 Parts of a Logic Model Goal Strategies Activities Population of Interest Process Objectives/Outputs Resources/Inputs Outcome Objective: Short-term Intermediate Long-term Boxes and arrows What does the program do? With/for whom? Using what? What is the expected result for the client/recipient/ audience/target group? Show the link between items 7

8 Why Use Program Logic Models? To describe a program or initiative at a glance To build common understanding of an initiative by working on a logic model together To describe the logic behind an initiative: the theory of change : what do we think we need to do (activities) with whom (target group/audience) to get the results we want (outcomes) ideally, the theory of change is evidence-based OPHS already have logic models developed it s accepted practice in Ontario public health 8

9 9

10 Super Simple Logic Model Goal: To increase capacity to understand logic models among webinar participants Strategy: Skill building Activity: Webinar on logic models Target group: Public health practitioners Outcome: Increased understanding of purpose and elements of logic models 10

11 Mandate Areas Components Activities Health Promotion Capacity Building Services Logic Model Health promotion program planning Health promotion program evaluation Health communication Policy development Training and Capacity Building Technical Support for Planning Support and promote the Online Health Program Planner Offer guided process and advice giving re Ontario Public Health Standard Offer skill building workshops Offer advice regarding health promotion practice Develop and revise capacitybuilding resources System Planning, Capacity Building, Networking Collaborate with other resource centres to develop and implement capacity building framework Communication, Information and Knowledge Exchange Create and disseminate resources Support knowledge exchange and networking Co-sponsor the Ontario Health promotion E-Bulletin Accountability, Monitoring and Evaluation Develop and implement an annual evaluation plan Contribute to the PHO overall evaluation framework Target Groups Public Health Units Community Health Centres Health-related NGOs Ontario Ministries Other health and social service organizations Short-term Objectives Intermediate Objectives Goal To increase knowledge, skills and confidence level of Ontario health promotion practitioners in the planning, implementation and evaluation of health promotion initiatives More effective mix of health promotion interventions and policies implemented in Ontario communities To improve risk conditions and reduce risk behaviours among those reached with health promotion interventions Ontario 11

12 12

13 OPHS Vaccine Preventable Diseases 13

14 School-Based Resilience Program Goal(s) To empower all school community members to adapt to and/or bounce back from changes and challenges they face to attain and sustain optimal health and developmental potential. (To increase resiliency in our school communities.) Audience(s) The target will be priority population school communities (to be identified through a pending assessment).within the school communities students will be the primary audience. Secondary audience will include: teachers, parents and other school staff, school administration, and school boards.. Long-term outcome objectives Medium-term outcome objectives Short-term (and other) outcome objectives - To increase by 10% knowledge among school community members of resiliency related skills such as adaptive skills, social skills and healthy behaviours, by April 2011 (medium-term) - To increase by 10% the percentage of students (in priority schools) who report confidence or high degrees of confidence in applying resiliency skills such as adaptive skills, social skills and healthy behaviours, by April 2011 (medium-term) - To have 100% of participating priority schools develop a school health plan that is responsive to needs identified through the Roadshow whereby their plan takes a comprehensive school health approach and incorporates the four pillars of The Foundations for a Healthy School (Quality Instruction and Programs, Social Support and Policy, Physical Environment, and Partnerships), by June 2011 (medium-term) - To increase by 10% awareness among priority school community members of positive and negative stressors, as well as risk and protective factors for stress, by January 2011 (short-term). Strategies Health Communication Self-help/mutual support Community mobilization Develop personal skills.. Activities - Website (including podcasts, forum discussions, geographic mapping) - Life planning tools - School Community Needs Assessment - Customized Comprehensive School Health Programming Within Each School - 5 Program Workshops (fuel, play, chill, chat, create) (part of roadshow) 14

15 Process objectives - To produce a website for members of the school community, by fall To produce a set of resiliency resources/tools (both hard copy and online), for students to use to set goals and create life plans, by October To produce a set of questions and a documentation process, for school communities to use to gather input from school community members during the program workshops, by September To produce a list of schools who are clearly identified to be the priority populations, for Health Unit staff to focus their planning/programming time with, by August To produce a guided planning process and set of planning templates, for school communities to use to create a comprehensive school health plan, by September Resources allocated The website will be a simpler version than originally planned/scoped. Resources for website development are limited plus access to internal technical support resources to aid in development. $2500 for tool expansion/development + staff time/human resources. Staff time. Approximately $5000/workshop for development (reduced budget from original vision/concept). 15

16 Comprehensive Tobacco Control Logic Model 16

17 Prevention 17

18 Protection 18

19 Cessation 19

20 Smokefree Environments Smoke-Free Environments (US): Nested Logic Model 20

21 Tobacco Cessation for Pregnant Women RESOURCES ACTIVITIES OUTPUTS OUTCOMES Short-term Intermediate Long-term Objective: To initiate and carry out community-based activities and projects directed towards helping pregnant and post-partum women quit using tobacco Develop home visit counseling/nrt provision program for pregnant and post-partum women and their partners as part of the Healthy Babies Healthy # of partnership with local hospital/health care providers # of referrals generated - Pregnant and post-partum women are motivated and aware of services available to support them in their efforts to quit using tobacco - Increase # of pregnant/post-partum women and their partners who remain clients of the program three-months postregistration - Pregnant /postpartum women and their partners remain tobacco-free Children program # of radio ads Resources: Health Promoter and/or Public Health Nurse public health department Pregnets Toolkit for Health Care Professionals Health department budget for program implementation Hospital staff/health care providers to provide referrals Conduct BabyTalk drop-in clinics in outreach areas Develop a relationship with local hospitals and health care providers to provide referrals to program Advertise program in local media (radio/newspaper/tv ads) Provide counseling, NRT and follow-up care to pregnant and post-partum women and their partners Provide consultation and support to external health care providers Conduct program evaluation through a postparticipation survey # of newspaper ads # of TV ads # of clients on program roster # of clients provided with counseling # of clients provided with NRT # of clients who attending Babytalk drop-in clinics # of surveys completed - Increase # of pregnant and post-partum women who call for information to seek assistance -Increase # of pregnant and post-partum women who register for program -Increase # of referrals received 21

22 These examples show that Logic Models may vary in terms of: How much they cover (scope) and level of abstraction The number of levels included short, medium and/or long-term outcomes? nested logic models for multiple programs/initiatives The naming of component parts The direction of information flow The visual layout 22

23 THEORY BEHIND LOGIC MODELS 23

24 What makes a good logic model? Logic! Passing the reasonable person test: would someone understand your program or initiative based on the logic model? Would they believe that you would be likely to get the desired outcome? Leads to evaluation possibilities: evaluation can be considered a test of the logic behind a program. 24

25 Theory of Change Describe the theory (or set of assumptions) that explain both the steps that lead to the long-term goal of interest and the connections between program activities and outcomes that occur at each step of the way Strengthen ability to claim credit for outcomes that were predicted in the theory Lay out the sequence of outcomes that are expected to occur as the result of an intervention, and plan an evaluation strategy around tracking whether these expected outcomes are actually produced 25

26 Four Types of Theory of Change Assumptions assertions about the connections among long-term, intermediate, and early outcomes on the map justifications supporting the links between program activities and the outcomes they are expected to produce substantiation for the claim that all of the important preconditions for success have been identified identification of contextual or environmental factors that will support or hinder progress toward the realization of outcomes 26

27 27

28 CRITERIA FOR EFFECTIVE LOGIC MODELS 28

29 Completeness Include all appropriate levels and populations of interest? Identify clear and measurable short- and long-term objectives? Process objectives listed under an appropriate strategy? Identify a wide range of resources, including financial, personnel, and in-kind? 29

30 Logic Is this the most logical set of strategies? Are the chosen process objectives likely to result in meeting the short-term objectives? Will the short-term objectives lead to the long-term objectives? Are the resources sufficient to drive strategies and process objectives? 30

31 Presentation Are there too many boxes on the page? Is it easy to follow the arrows and flow of logic? Are the levels in an order that is useful for you and your stakeholders? Is the model user-friendly and easy to follow? 31

32 Resources 32

33 Logic Model Resources Logic Models workbook: Online Health Program Planner: University of Wisconsin Enhancing Program Performance with Logic Models (interactive online course): Kellogg Foundation Logic Model Development Guide : Logic-Model-Development-Guide.aspx Theory of Change: 33

34 Health Promotion Capacity Building Services Free to those working on Ontario-focused projects. Scope varies, depending on need: training sessions; brief, one-time advice; consultations; review your work or product; links to other sources of information and resources. Consultation request form 34

35 QUESTIONS? 35

36 Public Health Ontario wishes to acknowledge and thank THCU staff and many partners who contributed to an earlier version of this document. THCU (originally known as The Health Communication Unit, started in 1993 at the University of Toronto) moved to Public Health Ontario s Health Promotion, Chronic Disease and Injury Prevention Department in 2011.