Design and evaluation of intervention programs

Transcription

1 Design and evaluation of intervention programs Pilvikki Absetz, Adjunct Professor, Senior researcher, PhD 28/03/2011 Absetz / NCD seminar

2 MAJOR PART OF HEALTH PROMOTION IS TO CHANGE BEHAVIOR However, while behavioral and social science theories are applied in interventions......the processes involved in using these theories are not made explicit in the research or practice literature 28/03/2011 Absetz / NCD seminar

3 In this presentation: Some useful frameworks for intervention design and evaluation Practical examples from our own work Kuvitus: Rakel Routarinne 28/03/2011 Absetz / NCD seminar

4 Examples from these complex interventions: DefenceNutri Promoting healthy eating among Finnish conscripts Bingham, C., Jallinoja, P., Lahti-Koski, M., Absetz, P., Paturi, M., Pihlajamäki, H., Sahi, T., Uutela, A. Quality of diet and food choices of Finnish young men a sociodemographic and health behaviour approach. Public Health Nutrition (In press). GOAL Lifestyle Implemtation Trial Absetz, P., Valve, R., Oldenburg, B., Heinonen, H., Nissinen, A., Fogelholm, M., Ilvesmäki, V., Talja, M., Uutela, A. Type 2 diabetes prevention in the real world : One-year results of the GOAL implementation trial. Diabetes Care 30, , GOAL Healthy Ageing Program /03/2011 Absetz / NCD seminar

5 Complex intervention trials Complex interventions include several components, often at different levels e.g. individual, group, setting, system, societallevels. The evaluation of complex interventions is difficult particularly when they are conducted under real world conditions A phased approach to the development and evaluation of complex interventions is important Evaluation of complex interventions requires use of different study designs and approaches to measurement 28/03/2011 Absetz / NCD seminar

6 MRC Framework for development and evaluation of RCTs for Complex Interventions to Improve Health BMJ 2000, BMJ /03/2011 Absetz / NCD seminar

7 Aro, A. R. and Absetz, P.(2009). Guidance for professionals in health promotion: Keeping it simple-but not too simple. [Editorial]. Psychology & Health,24:2, /03/2011 Absetz / NCD seminar

8 28/03/2011 Absetz / NCD seminar

9 An Intervention Mapping Approach IM is a tool for intervention design and evaluation Six steps comprising several specific tasks Completion of the tasks in a step creates a product that is the guide for the subsequent step The intervention map is the result of completion of all the tasks for each of the steps Bartholomew, Parcel, Gottlieb & Kok. PLANNING HEALTH PROMOTION PROGRAMS An Intervention Mapping Approach. Wiley & Sons, /03/2011 Absetz / NCD seminar

10 Steps in Intervention Mapping 1. Needs Assessment 2. Matrices: Define what (behavioral or environmental) changes you want to achieve and what are the underlying factors 3. Select the best theoretical methods and the most feasible practical strategies 4. Plan: How will the program look like? 5. Plan: How will the program be adopted and implemented? 6. Plan: How can the program be evaluated? Bartholomew, Parcel, Gottlieb & Kok. PLANNING HEALTH PROMOTION PROGRAMS An Intervention Mapping Approach. Wiley & Sons, /03/2011 Absetz / NCD seminar

11 Step 1: Needs assessment Defining target groups and their health & behaviour profiles; identifying key individual and environmental influences 28/03/2011 Absetz / NCD seminar

12 PLANNING Health Promotion Health Education Predisposing Factors Personal Determinants Reinforcing Factors Behavior and Lifestyle Health Quality of Life Policy Regulation Environment Organization Enabling External Factors Determinants IMPLEMENTATION EVALUATION Based on PRECEDE / PROCEED-model by Green & Kreuter 28/03/2011 Absetz / NCD seminar

13 Need for promoting healthy diet among conscripts. The DefenceNutri Intervention Concumption of fresh vegetables & salad, days/week 4 3,5 Panssariprikaati Kainuun Prikaati Kaikki 80% of Finnish males attend military service (between ages 18-30) 3 2,5 2 Ennen palvelusta 8 viikon kohdalla 6 kk kohdalla Pre-service At 8 weeks At 6 months Concumption of sweet pastry, days/week P anssariprikaati Kainuun P rikaati Kaikki 2,3 2,1 1,9 1,7 1,5 1,3 Pre-service At 8 weeks At 6 months Ennen palvelusta 8 viikon kohdalla 6 kk kohdalla 28/03/2011 Absetz / NCD seminar

14 All meals are served in the garrison cafeteria as part of service 28/03/2011 Absetz / NCD seminar

15 Leisure-time spend in Soldier s home cafes, with snacks and fast food 28/03/2011 Absetz / NCD seminar

16 Needs asessment for DM2 prevention Primary health care overloaded with patients at risk for DM2 1 Individual counseling (nurse advice) given as default Cost? Effectiveness in producing changes? 2-4 Evidence: DM2 can be prevented by lifestyle change 5 Evidence-based program goals: 1 No more than 30% of energy from fat 2 No more than 10% of energy from saturated fats 3 At least 15g / 1000 kcal fiber 4 At least 30 min / day moderate physical activity 5 At least 5% weight reduction What are the underlying behaviors and their determinants? 1 Peltonen et al. Lihavuuden, diabeteksen ja muiden glukoosiaineenvaihdunnan häiriöiden esiintyvyys suomalaisessa aikuisväestössä. Dehkon 2D-hanke (D2D). [Prevalence of obesity, type 2 diabetes, and other disturbancees in glucose metabolism in Finland - The FIN-D2D survey]. Suomen Laakarilehti 61: , Poskiparta M, Kasila K, Kiuru P. Dietary and physical activity counselling on type 2 diabetes and impaired glucose tolerance by physicians and nurses in primary healthcare in Finland. Scand J Prim Health Care 2006;4: Kiuru P, Poskiparta M, Kettunen T, Saltevo J, Liimatainen L. Advice-giving styles by Finnish nurses in dietary counseling concerning type 2 diabetes care. J Health Commun 2004;4: Kettunen T, Poskiparta M, Kiuru P, Kasila K. Lifestyle counseling in type 2 diabetes prevention: a case study of a nurse's communication activity to produce change talk. Commun Med 2006;3: Tuomilehto et al. (2001). Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. New England Journal of Medicine, 344, /03/2011 Absetz / NCD seminar

17 Step 2: Matrices Defining expected changes and their determinants 28/03/2011 Absetz / NCD seminar

18 Considerations for the choice of a theory for determinants No universal theory exists for behaviour change Theories tend to have overlapping constructs (determinants) Much of the theorising focuses on intention formation Different theories for defining determinants and for defining methods (techniques) to be used for changing the determinants Choices of health behavior models and theory-based methods for intervention studies depend on Evidence Target behaviours Intervention setting Participants Program users (Researchers liking) Prototype Willingness Model Transtheoretical Model of Stages of Change Health Belief Model Social Capital Theory Health Action Process Approach Theory of Planned Behavior Self-Regulation Theories Social Support Theories 28/03/2011 Absetz / NCD seminar

19 Person at risk for type 2 diabetes? 28/03/2011 Absetz / NCD seminar

20 Theories for determinants - the HAPA model Sessions 1-2 Sessions 3-4 Sessions 5-6 Perceived self-efficacy Outcome expectancies Intention Planning Initiative Maintenance Recovery Risk perception Action Barriers and resources Schwarzer, R., Fuchs, R. (1996). Self-efficacy and health behaviors. In M. Conner & P. Norman (eds.): Predicting 28/03/2011 health behaviour: Research and practice with social Absetz cognition / NCD seminar models 2010 (pp ). Buckingham, UK: Open University Press. 20

21 Example of creating a matrix. How to address intention formation? Learning objectives Determinant Theorybased method Practical strategy I m at risk for diabetes Risk perception Selfevaluation Self-monitor diet and PA Life-style changes reduce risk I can make a change Outcome expectation Arguments Information: studies on prevention Self-efficacy Re-attribution Analysis of previous experiences Uutela, Absetz, Valve, Talja, Fogelholm, Nissinen. Health psychological theory in promoting population health in Päijät-Häme, Finland: first steps toward a type 2 diabetes prevention study. Journal of Health Psychology, 2004, 9 (1), GOAL Program for Good Ageing 28/03/2011 Absetz / NCD seminar

22 Example. Learning objectives for a lowfat diet Acknowledge a need for change Become aware of own diet Learn to know about diet-related risks Make a decision Develop a positive attitude towards change Increase self-efficacy in making the change Make an action plan Learn to set goals Learn to plan Have practical skills: Learn to use the Plate model Learn to make a distinction between high vs low fat Learn to identify sources of hidden fat Inform significant others etc. 28/03/2011 Absetz / NCD seminar

23 Performance objectives for environment to support eating a healthy diet Example: Increasing availability of healthy food at the garrisons Management and personnel in garrison catering and soldiers home cafés acknowledge the health benefit of low-fat food and vegetables for conscripts Management and personnel develop healthy food options together: Many options, easy to take & eat, tempting to look, delicious DefenceNutri Intervention to promote healthy eating in the Finnish Defence Forces 28/03/2011 Absetz / NCD seminar

24 Step 3: Methods and strategies 28/03/2011 Absetz / NCD seminar

25 Behaviour Change Technique Checklist 1 Provide general information linking behaviour to health 14 Provide contingent rewards 2 Provide information on consequences 15 Teach to use prompts/ cues 3 Provide information about others approval 16 Agree behavioural contract 4 Prompt intention formation 17 Prompt practice 5 Prompt barrier identification 18 Use follow up prompts 6 Provide general encouragement 19 Provide opportunities for social comparison 7 Set graded tasks 20 Plan social support/ social change 8 Provide Instruction 21 Prompt identification as role model 9 Model/ demonstrate the behaviour 22 Prompt self talk 10 Prompt specific goal setting 23 Relapse prevention 11 Prompt review of behavioural goals 24 Stress management 12 Prompt self-monitoring of behaviour 25 Motivational interviewing 13 Provide feedback on performance 26 Time Management Abraham & Michie: Coding Manual to Identify Behaviour Change Techniques in Behaviour Change Intervention Descriptions. 28/03/2011 Absetz / NCD seminar

26 Methods and strategies in the GOAL Healthy Ageing Program Themes and sub-themes Participatory and creative methods Function in goal oriented approach My life goals and values Eating - Create your own dream using e.g., pictures or storytelling. Present and describe to the group. - Clarification of life goals and values - My daily eating - My normal daily eating: Use pictures, storytelling or other similar methods to describe meal pattern, meal content, amount of food, social context for eating etc. - My ideal daily eating: Use pictures, storytelling or other similar methods to describe your ideal meal pattern, meal content, amount of food, social context for eating etc. - Discuss similarities and discrepancies between normal and ideal daily eating. Is there something I would like to change in my normal eating? What? How can I make the change? - Write down your goal or select a picture or object to take home with you and to remind you of your goal. Try to think as concretely as possible about the goal: What? Where? When? With whom? How can you guarantee you will do it? - Parties and special occasions - Using storytelling to describe social context, eating (what, by whom, with whom etc) - Food recommendations - Create a healthy meal for your partner/friend/father (use pictures, storytelling etc or prepare a real meal). Use your own imagination and food recommendations. What does it contain? How much? Describe and discuss in group. - What components of the healthy meal are already in your own diet? What could you integrate into your own? What would you not choose? Why? - Based on what you know about the recommendation, what would you like to change first in your own meal? Is it something you can easily do yourself or do you need others? Who? How confident are you that you could do it? - Self-monitoring - Could also be used for evaluation, if done repeatedly during the program - Identification of needs - Goal setting - Action planning - Identification of needs, self-monitoring, goal setting - Identification of needs - Goal setting 28/03/2011 Absetz / NCD seminar

27 Step 4: Program components What will the program look like? 28/03/2011 Absetz / NCD seminar

28 Example. DefenceNutri Vegetable War 28/03/2011 Absetz / NCD seminar

29 Photos by Anu Ritsilä, Ikihyvä 28/03/2011 Absetz / NCD seminar

30 My plan for increasing physical activity What s positive about my physical activity habits? Executing my plan How did I succeed in executing my plan? What would I like to do more? Things that helped me: A concrete change I m going to make: Things that made it more difficult: Where will I do it? How will I continue? When will I start? How will I guarantee that I ll follow my plan? GOAL setting and planning sheet adapted from the GOAL LIT 28/03/2011 Absetz / NCD seminar

31 Step 5: Implementation and adoption Who will use the program? Where? When? How? 28/03/2011 Absetz / NCD seminar

32 Example from the DefenceNutri Garrison cafeteria and soldiers home key personnel and management in a planning group together with researchers and key experts from the Finnish Horticultural Products Society and the Finnish Heart Association Commitment to common cause Acceptability of intervention content Feasibility of delivery Implementation supported by organizational management and researchers Further nationwide dissemination and maintenance by the central organizations 28/03/2011 Absetz / NCD seminar

33 Step 6: Evaluation 28/03/2011 Absetz / NCD seminar

34 RE-AIM framework for evaluating complex interventions Reach The absolute number, proportion, and representativeness of individuals who are willing to participate Efficacy/Effectiveness The impact of an intervention on important outcomes, including potential negative effects, quality of life, and economic outcomes. Adoption The absolute number, proportion, and representativeness of settings and intervention agents (people who deliver the program) who are willing to initiate a program. Implementation Setting level: users fidelity to various elements of intervention's protocol, including consistency of delivery as intended and the time and cost of the intervention. Individual level: clients use of the intervention strategies. Maintenance Setting level: the extent to which a program or policy becomes institutionalized or part of the routine organizational practices and policies. Individual level: the long-term effects of a program on outcomes after 6 or more months after the most recent intervention contact. 28/03/2011 Absetz / NCD seminar

35 28/03/2011 Absetz / NCD seminar

36 Attainment of intervention objectives (%) Intervention objectives GOAL* N=352 DPS N=265 Total fat < 30E% Saturated fat < 10E% Fibre > 15g/1000 kcal GOAL participants with 4-5 objectives were more likely to have normal glucose at follow-up (χ2 = 7.120, p < 0.05) Moderate intensity PA > minutes / day Weight reduction > 5% objectives attained * Intention to treat, non-respondents regarded as not reaching the intervention objectives Statistically significant difference between DPS and GOAL (χ2 = 4.614, p < 0.05) Statistically significant difference between DPS and GOAL (χ2 = , p < 0.001) Statistically significant difference between DPS and GOAL (χ2 = , p < 0.001) Statistically significant difference between DPS and GOAL (χ2 = , p < 0.001) Absetz, P., Valve, R., Oldenburg, B., Heinonen, H., Nissinen, A., Fogelholm, M., Ilvesmäki, V., Talja, M., Uutela, A. Type 2 diabetes prevention in the real world : One-year results of the GOAL implementation trial. Diabetes Care 30, , /03/2011 Absetz / NCD seminar

37 Glucose tolerance at year participants had normal glucose tolerance at baseline 10.9% had IGT and 1.6% diabetes at year three. 65 participants had IGT at baseline 12% had diabetes and 43% had returned to normal by year three Conversion rate from IGT to diabetes 12% at year three is reasonable compared to 9% in the intervention and 20% in the control group of the DPS Absetz, P., Oldenburg, B., Hankonen, N., et al. Type 2 diabetes prevention in the real world : Three-year results of the GOAL implementation trial. Diabetes Care, 2009, 32 (8), /03/2011 Absetz / NCD seminar

38 Process evaluation in the GOAL Lifestyle Implementation Trial: Nelli Hankonen Psychosocial Processes of Health Behaviour Change in a Lifestyle Intervention Influences of Gender, Socioeconomic Status and Personality 28/03/2011 Absetz / NCD seminar

39 Implementation and maintenance of lifestyle intervention in health care Quality management in health care Training Standard protocol Self-monitoring and recording of professional practices Feedback Infrastructure that promotes maintenance of the program Management support Co-ordinators Ongoing training programs Electronic patient database to record and evaluate outcomes Absetz & Patja (eds.): Ennaltaehkäisyn kehittäminen Päijät-Hämeessä. Interventiot osana Ikihyvä Päijät-Häme tutkimus- ja kehittämishanketta. [Developing preventive strategies in the province of Päijät-Häme. Interventions within the GOAL Program for Good Aging.] Päijät-Hämeen sosiaali- ja terveysyhtymän julkaisuja [Publications of the Päijät-Häme Social and Health Care District] 68/ /03/2011 Absetz / NCD seminar

40 Translating evidence / adapting interventions to other settings Special Issue on transfer of evidence-based behavioral medicine programs and/or research findings between cultures and countries is scheduled for June, /03/2011 Absetz / NCD seminar

41 From basic research to practice What are the primary risk factors and behaviours causing them? Randomized, controlled trials: Efficacy of lifestyle modification Tuomilehto et al., 2001 Basic research Population-based risk factor studies Applying behavioral sciences to lifestyle change How to change the behaviors? Does it work in routine care? Feasibility of method Implementation trials: Effectiveness and feasibility of lifestyle counselling Absetz ym., 2007 Implementation trial: Guidelines for CVD prevention Kuronen et al., 2006 Adaptability Organizational changes How to identify target group and maximize reach? How to organize follow-up? Quality control How to support maintenance and fidelity of the programs and support professional development? Systemicity Maintenance: Processes and infrastructure for prevention Absetz & Patja (eds.), /03/2011 Absetz / NCD seminar

42 28/03/2011 Absetz / NCD seminar

43 Summary: Designing and evaluating interventions needs to be conducted in a systematic way using both theory and evidence Practical frameworks, models and cook books such as the Intervention Mapping and the RE-AIM exist USE THEM! 28/03/2011 Absetz / NCD seminar