Program Logic Models June Susan J. Snelling, PhD

Transcription

1 Program Logic Models June 2014 Susan J. Snelling, PhD 1

2 Webinar Features Adobe Connect technology Use landline for audio Chat function to ask questions or if you need help Lecture mode 2

3 Poll #1: Your level of familiarity with logic models How would you rate your level of familiarity with program logic models? 1. I m a beginner 2. I have some idea of what the key concepts are 3. I am very familiar with program logic models istockphoto.com/atnoydur 3

4 WEBINAR OBJECTIVES At the end of the session, participants will be able to: 1. Identify and understand the different components of a program logic model. 2. Understand the benefits of using a logic model. 3. Reflect on how logic models can be developed and applied in the participants contexts. 4

5 Poll #2: Your reason for attending this webinar What is the most important reason you are attending this webinar? 1. I need to create a logic model and I don t know where to start 2. I want to improve my skills in creating logic models 3. I am looking for a refresher on the concepts istockphoto.com/atnoydur 5

6 AGENDA Overview of logic models: definition, purposes and examples Theory behind logic models Criteria for effective program logic models Questions 6

7 OVERVIEW OF LOGIC MODELS 7

8 What is a Program Logic Model? A graphic depiction of the relationship between the key elements of a program or initiative (i.e., goals, objectives, target groups, and activities). A diagrammatic representation of a program that shows the relationships or the If, then logic among: WHAT: what does the program/initiative do? WHO: who are the recipients of the program/initiative? WHY: what outcomes are to be achieved? 8

9 Goal Population(s) of interest Sample program logic model outline Long-term outcome objectives Short-term outcome objectives Activities Outputs/process objectives Resources 9

10 Sample program logic model outline Goal Population(s) of interest Long-term outcome objectives Short-term outcome objectives Summarizes the ultimate direction or desired achievement of a program. What are you trying to accomplish? High level. Specify the target population you intend to serve. May have multiple target groups. What is the expected result for the target group- long-term? What is the expected result for the target group- short-term? Activities What will you be doing or delivering? Outputs/process objectives Resources What are the observable results of your activities? Staff, budget, in-kind contributions? 10

11 Sample program logic model Goal Population(s) of interest Long-term outcome objectives Short-term outcome objectives Activities Outputs/process objectives To increase capacity to use logic models effectively among public health practitioners Public health practitioners Increased ability to develop logic models Increased understanding of purpose and elements of logic models Educational webinar focused on key concepts in logic models One hour webinar Resources Presenter; webinar technology 11

12 Parts of a Logic Model What does the program do? For whom? How? Using What? What is the expected result for the population of interest? Goal Population of Interest/Target Strategies Activities Resources/ Inputs Process Objectives/Outputs Outcomes Shows the link between items Boxes Arrows 12

13 Why Use Program Logic Models? To describe a program or initiative at a glance To build common understanding of an initiative by working on a logic model together To describe the logic behind an initiative: the theory of change : what do we think we need to do (activities) with whom (target group/audience) to get the results we want (outcomes) ideally, the theory of change is evidence-based OPHS already have logic models developed it s accepted practice in Ontario public health 13

14 14

15 SOME EXAMPLES OF LOGIC MODELS 15

16 16

17 OPHS Vaccine Preventable Diseases 17

18 School-Based Resilience Program Goal(s) To empower all school community members to adapt to and/or bounce back from changes and challenges they face to attain and sustain optimal health and developmental potential. (To increase resiliency in our school communities.) Audience(s) The target will be priority population school communities (to be identified through a pending assessment).within the school communities students will be the primary audience. Secondary audience will include: teachers, parents and other school staff, school administration, and school boards.. Long-term outcome objectives Medium-term outcome objectives Short-term (and other) outcome objectives - To increase by 10% knowledge among school community members of resiliency related skills such as adaptive skills, social skills and healthy behaviours, by April 2011 (medium-term) - To increase by 10% the percentage of students (in priority schools) who report confidence or high degrees of confidence in applying resiliency skills such as adaptive skills, social skills and healthy behaviours, by April 2011 (medium-term) - To have 100% of participating priority schools develop a school health plan that is responsive to needs identified through the Roadshow whereby their plan takes a comprehensive school health approach and incorporates the four pillars of The Foundations for a Healthy School (Quality Instruction and Programs, Social Support and Policy, Physical Environment, and Partnerships), by June 2011 (medium-term) - To increase by 10% awareness among priority school community members of positive and negative stressors, as well as risk and protective factors for stress, by January 2011 (short-term). Strategies Health Communication Self-help/mutual support Community mobilization Develop personal skills.. Activities - Website (including podcasts, forum discussions, geographic mapping) - Life planning tools - School Community Needs Assessment - Customized Comprehensive School Health Programming Within Each School - 5 Program Workshops (fuel, play, chill, chat, create) (part of roadshow) 18

19 Process objectives - To produce a website for members of the school community, by fall To produce a set of resiliency resources/tools (both hard copy and online), for students to use to set goals and create life plans, by October To produce a set of questions and a documentation process, for school communities to use to gather input from school community members during the program workshops, by September To produce a list of schools who are clearly identified to be the priority populations, for Health Unit staff to focus their planning/programming time with, by August To produce a guided planning process and set of planning templates, for school communities to use to create a comprehensive school health plan, by September Resources allocated The website will be a simpler version than originally planned/scoped. Resources for website development are limited plus access to internal technical support resources to aid in development. $2500 for tool expansion/development + staff time/human resources. Staff time. Approximately $5000/workshop for development (reduced budget from original vision/concept). 19

20 Comprehensive Tobacco Control Logic Model 20

21 Prevention 21

22 Protection 22

23 Cessation 23

24 RESOURCES ACTIVITIES OUTPUTS OUTCOMES Short-term Intermediate Long-term Objective: To initiate and carry out community-based activities and projects directed towards helping pregnant and post-partum women quit using tobacco Resources: Health Promoter and/or Public Health Nurse public health department Develop home visit counseling/nrt provision program for pregnant and post-partum women and their partners as part of the Healthy Babies Healthy Children program Conduct BabyTalk drop-in clinics in outreach areas # of partnership with local hospital/health care providers # of referrals generated # of radio ads # of newspaper ads # of TV ads - Pregnant and post-partum women are motivated and aware of services available to support them in their efforts to quit using tobacco - Increase # of pregnant and post-partum women who call for information to seek assistance - Increase # of pregnant/postpartum women and their partners who remain clients of the program three-months post-registration - Pregnant /post-partum women and their partners remain tobaccofree Pregnets Toolkit for Health Care Professionals Health department budget for program implementation Hospital staff/health care providers to provide referrals Develop a relationship with local hospitals and health care providers to provide referrals to program Advertise program in local media (radio/newspaper/tv ads) Provide counseling, NRT and follow-up care to pregnant and post-partum women and their partners # of clients on program roster # of clients provided with counseling # of clients provided with NRT # of clients who attending Babytalk drop-in clinics # of surveys completed -Increase # of pregnant and post-partum women who register for program -Increase # of referrals received Provide consultation and support to external health care providers Conduct program evaluation through a post-participation survey 24

25 These examples show that Logic Models may vary in terms of: How much they cover (scope) and level of abstraction The number of levels included short, medium and/or long-term outcomes? More complex logic models for multiple programs/ initiatives The naming of component parts The direction of information flow The visual layout 25

26 VALUE OF LOGIC MODELS IN PLANNING AND EVALUATION 26

27 Logic models support planning by: Building a common understanding of what a program is all about and how the parts fit together Providing an effective communication tool Involving stakeholders in program planning Showing connections between key aspects of a program Noting gaps and inconsistencies within a program Leading to decisions about resources: how much time, money and skill will be needed to deliver the program? 27

28 Logic models support evaluation by: Serving as a resource for evaluability assessments Describing the assumptions underlying the program. Evaluation questions may follow, to test those assumptions. Aiding in the identification of indicators Demonstrating accountability Providing a starting point for engaging stakeholders in participatory evaluations 28

29 THEORY BEHIND LOGIC MODELS 29

30 What makes a good logic model? Logic! Passing the reasonable person test: would someone understand your program or initiative based on the logic model? Would they believe that you would be likely to get the desired outcome? Leads to evaluation possibilities: evaluation can be considered a test of the logic behind a program. 30

31 Theory of Change Describe the theory (or set of assumptions) that explain both the steps that lead to the long-term goal of interest and the connections between program activities and outcomes that occur at each step of the way Strengthen ability to claim credit for outcomes that were predicted in the theory Lay out the sequence of outcomes that are expected to occur as the result of an intervention, and plan an evaluation strategy around tracking whether these expected outcomes are actually produced 31

32 Four Types of Theory of Change Assumptions assertions about the connections among long-term, intermediate, and early outcomes on the map justifications supporting the links between program activities and the outcomes they are expected to produce substantiation for the claim that all of the important preconditions for success have been identified identification of contextual or environmental factors that will support or hinder progress toward the realization of outcomes 32

33 33

34 Poll #3: Effective program logic models What do you think is the most important feature of an effective program logic model? 1. It has to describe the program accurately 2. It has to be show believable associations between activities and outcomes 3. It has to be easy to follow visually istockphoto.com/atnoydur 34

35 CRITERIA FOR EFFECTIVE LOGIC MODELS 35

36 Completeness Include all appropriate levels and populations of interest? Identify clear and measurable short- and long-term objectives? Process objectives listed under an appropriate strategy? Identify a wide range of resources, including financial, personnel, and in-kind? 36

37 Logic Is this the most logical set of strategies? Are the chosen process objectives likely to result in meeting the short-term objectives? Will the short-term objectives lead to the long-term objectives? Are the resources sufficient to drive the strategies and process objectives? 37

38 Presentation Are there too many boxes on the page? Is it easy to follow the arrows and flow of logic? Are the levels in an order that is useful for you and your stakeholders? Is the model user-friendly and easy to follow? 38

39 Resources 39

40 Logic Model Resources Online Health Program Planner: University of Wisconsin Enhancing Program Performance with Logic Models (interactive online course): Kellogg Foundation Logic Model Development Guide : Logic-Model-Development-Guide.aspx Theory of Change: 40

41 Health Promotion Capacity Building Team: Get in Touch, Stay in Touch For service request inquiries, including consultations, tailored webinars, customized face to workshops, or to sign up for our mailing list, please go to: thpromotionservices/pages/service-request-form.aspx To find out more about scheduled events, please visit our calendar at: ent/events/pages/default.aspx

42 QUESTIONS? 42