Developing Your MOAPC Strategic Plan: Selecting and Prioritizing Intervening Variables and Strategies May 12, 2015 9:30am to 11:00am Webinar Presentation Scott Formica Social Science Research & Evaluation, Inc.
Presenter and Moderator Moderator: Tracy Desovich, Technical Assistance Provider, MassTAPP Presenter: Scott Formica Social Science Research & Evaluation, Inc. 2
Background for Today s Webinar There are several deadlines approaching a draft of the strategic plan is due to MassTAPP on June 19, 2015 and the final draft is due to BSAS on July 24, 2015. The 13 sites in the first MOAPC cohort found the intervening variable (IV) selection and prioritization process and development of the logic model challenging. Intervening variables????? Where did my risk and protective factors go? Intervening Variables: factors that have been identified through research as being strongly related to and influencing the occurrence and magnitude of substance use and related risk behaviors and their subsequent consequences. These variables are the focus of prevention interventions, changes in which are then expected to affect consumption and consequences. (SAMHSA, 2009) 3
Webinar Objectives Provide an overview (from MOAPC cohort 1 and the literature) of the steps involved in prioritizing intervening variables and selecting strategies as they relate to your final MOAPC strategic plans. Provide an opportunity for peer exchange and a chance to ask questions directed to MassTAPP TA providers, the state evaluators, and/or BSAS. Make the connection between Prioritized IVs Selection of appropriate strategies. Share resources to help assist you with these steps of SAMHSA s Strategic Prevention Framework (SPF) and the development of the strategic plan. 4
IVs MOAPC Strategic Plan Requirements The MOAPC strategic plan template asks you to write a little about your process for collecting data on and prioritizing IVs. Among other things, the template asks you: 1. How did you collect data on IVs as they relate to opioid consumption (1.2a) and consequences (1.2b) in your Cluster? 2. List all IVs that you investigated and the data source for opioid consumption (1.2a) and consequences (1.2b). 3. List the final set of consumption (3.2a) and consequence (3.2b) IVs including how this list was selected (prioritized) from the larger list in Section 1. 5
IVs Identification, Collection, and Prioritization So what can we learn from looking across the 13 cohort 1 MOAPC sites about identifying, collecting, and prioritizing IVs? I came up with four big questions in this area 1. How did sites approach IV data collection? What questions guided these efforts? 2. What consumption and consequence IVs did sites identify and pursue? 3. What prioritization criteria did sites enlist? What questions guided this process? 4. What decision-making processes did sites enlist? How were final decisions made? 6
IVs Identification/Collection Where did cohort 1 sites turn to identify/generate IV data? Key stakeholder interviews (11) Focus groups (11) Surveys youth, community, BRFSS, key stakeholder (7) Death certificates; geographical mapping OD locations by zip code (2) Pharmacy data; PMP data (2) Police and EMS data demos; location; history; context (2) TX provider data demos; history; context (1) MDPH data (1) Prevalence of substance abusing parents (1) Source: MOAPC Cohort 1 Strategic Plans Part I (n=13) 7
IVs Sample Key Stakeholder Questions Boston: Describe the opioid misuse situation in terms of how severe the problem is; how the problem has changed over time; the target groups including age, gender, race/ethnicity who are misusing opioids; the consequences of misuse across the lifespan; when and where misuse occurs; and risk factors that drive opioid misuse. Quincy: Is there something about this area or the South Shore that makes more people likely to use prescription drugs or heroin? Is there a community attitude or characteristic that you think contributes and, (b) Have you noticed patterns in availability of prescription drugs and heroin in the past 6 months? Cambridge: Assess the problem in community; what is currently occurring to prevent opioid misuse and abuse? Revere: What is the nature and extent of problem; who is the target population(s); what factors are driving the problem? Source: MOAPC Cohort 1 Strategic Plans Part I (n=13) 8
IVs Selected Consumption IVs What were some consumption IVs that Cohort 1 sites identified? Perception of harm/risk Peer norms/approval Close friends who use/abuse Parental approval/disapproval Parental involvement Parent knowledge/education/awareness Availability/Ease of Access/Storage Community norms favorable to use Representation across individual, peer, family, and community domains. Source: MOAPC Cohort 1 Strategic Plans Part I (n=13) 9
IVs Selected Consequence IVs What were some consequence-related IVs that Cohort 1 sites identified? Mixing/concomitant use Barriers to calling 911; witnesses leaving scene Loss of tolerance Fluctuations in purity/potency Prior history (overdose; IDU) Treatment involvement Age; gender; race/ethnicity Prescriber willingness to prescribe drugs Lack of employment Attitudes around pain and pain management Good balance between user characteristics, use patterns, contextual/ situational variables, and organizations/systems. Source: MOAPC Cohort 1 Strategic Plans Part I (n=13) 10
IVs Sample Prioritization Criteria What criteria did Cohort 1 sites identify as the primary means by which they prioritized their list of IVs? Importance (6) Changeability (4) Feasibility (4) Capacity (3) Readiness (3) Resource availability (3) Supported by data qualitative and quantitative (2) Cultural Competence (1) Evaluability data available (1) Fit (1) Potential for unintended consequences (1) Sustainability (1) Wisdom of Practice (1) Source: MOAPC Cohort 1 Strategic Plans Part I (n=13) 11
IVs Sample Prioritization Criteria Questions This is an example from the Gloucester cluster on how you might frame a set of questions on multiple criteria: Is the risk/protective factor identified independently by multiple sources? How reliable and valid is the data supporting it? Have local changes in the variable produced changes in use? How actionable is the variable? Is it feasible to address the variable within the time frame of this grant? Are other efforts in place to change this variable? Is there capacity and resources available or that could be developed to address the intervening variable? Source: MOAPC Cohort 1 Strategic Plan Gloucester Cluster 12
IVs Sample Decision-Making Processes Which groups/bodies made the final decisions on IVs in Cohort 1 sites? Small group of cluster/steering committee representatives attempt to reach consensus based on available evidence (7) Regional leadership team given materials from guidance document; summary of regional themes presented to team. Each member ranks top 3 based on prioritization questions and professional/personal experience; Consensus by summation of the rankings and group discussion for further refinement. IV Workgroup makes recommendations to steering committee and they solicit additional input from larger group of key community stakeholders. (2) IV Workgroup makes recommendations to steering committee for final decision. (1) Source: MOAPC Cohort 1 Strategic Plans Part I (n=13) 13
IVs What Do You Need To Do Now? Section 1.2a/b asks you to describe: The process used to collect IV data All IVs investigated data on each variable and the source of evidence Gaps in the IV data and how you plan to address these gaps Additional information on how the assessment of IV data was conducted. Section 3.2a/b asks you to describe: The final set of intervening variable(s) from section 1.2a/b including how this list was selected (prioritized) from among the larger list of variables considered. 14
IVs Ideas, Tips, and Resources Don t feel like you are tied to the literature for identifying IVs. This is still a relatively nascent area of study. There are more holes than bridges in this literature. There is a danger that starting from a list will be self-fulfilling you will find what you are looking for rather than finding the major factors that operate in your community however weakly they may be supported by the current research literature. There is a lack of consistent terminology, even within the literature. 15
IVs Ideas, Tips, and Resources Once you have identified the major IVs in your site then turn to reviews of the literature to determine whether or not it has been linked to consumption/consequences. You probably want to consider around 6 IVs for consumption and 6 for consequences and narrow this down to 3 each. IV s may differ from community to community and gaps are OK. If an IV does not appear in a list of a review, it is worth going into the literature yourself or asking for help from BSAS/MassTAPP. 16
IVs What Lists or Resources Are Out There? Consumption IVs (Use) SAMHSA s CAPT website Prescription Drug Abuse and Misuse Pages. Risk and Protective Factors Associated with Non-Medical Use of Prescription Drugs: A Review of the Literature (2006-2012). A Systematic Review of Risk and Protective Factors (Nargiso, 2015). MOAPC Guidance Document. Consequence IVs (Overdose) MassTAPP website Intervening Variables And Strategies Related to the Prevention of Opioid Overdose MOAPC Guidance Document. 17
IVs Revisiting Prioritization High Importance Low Importance Step 3 in the MOAPC Guidance Document (pages 27-28) discusses prioritizing IVs. Emphasis is on Importance and Changeability the latter includes many of the sub-categories we have discussed (e.g., readiness, resources, suitable interventions, time required for change to occur) Other potential criteria include: Directionality can you interpret the change? Resource/Need gap are resources already going to this? Political will is there political/public will to address this factor? Distance is the IV proximal enough to affect change/measure? 18
IVs Revisiting Prioritization CHANGEABILITY Importance High Low High Low Intervening Variables Feasibility Criteria Data Availability Preventability Low peer disapproval Lack of parental knowledge High Social Availability 19
IVs Additional Things to Consider Efficiency and Synergy Linkages between intervening variables Multiple IVs that can be impacted by a single intervention Is the IV related to other health or behavioral health issues? Appropriateness Has the IV been shown to be linked to consumption or consequences with the group that you are targeting? Age? Gender? Race/Ethnicity? SES? Urban, Suburban, Rural? Etc. Acceptability Did the process and final decision-making process provide adequate opportunity for community input? Are there missing segments? Was the assessment comprehensive and inclusive? 20
Strategies Strategic Plan Requirements Sections 3.2a and 3.2b. of the strategic plan deliverable ask you to describe your strategy selection process and to identify your proposed strategies, including: The final set of IVs from section 1.2a/b including how this list was selected from among the larger list of variables considered. The specific target population(s) for opioid use and consequences The list of opioid use and consequence strategies you propose to implement The rationale for each selected strategy (conceptual & practical fit, research) The cultural competence of the selection process and the selected strategies. The potential sustainability of the selected strategies. Bullets #1 and #4 draw an implicit link between IVs and strategies. Why? Prioritized IVs Strategies 21
Strategies Connection Between IVs/Strategies Intervening Variables: factors that have been identified through research as being strongly related to and influencing the occurrence and magnitude of substance use and related risk behaviors and their subsequent consequences. These variables are the focus of prevention interventions, changes in which are then expected to affect consumption and consequences. If you cannot demonstrate how B in the logic model DIRECTLY addresses A your plan will be returned to you. Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable (A) Strategy (B) Target Group Outputs Short- Term Intermediate Long-Term 22
Let s Take a Logic Model Detour Before Going Into Strategies 23
MOAPC Logic Model Example Problem identified by BSAS: Misuse/abuse of opioids and unintentional deaths/non-fatal hospital events associated with opioid poisoning. Local manifestation of the problem: During calendar year 2014 there were 20 opioid overdose fatalities in our cluster communities: Jamestown (4); Janetown (5); Jilltown (11). Intervening Outcomes Variables Strategy Target Group Outputs Train law Local law enforcement enforcement officers on the officers and 911 Good administrators Samaritan Law Fear of calling 911 due to police involvement Educate community members on the 911 Good Samaritan Law Members of the general community (including users and bystanders) Number of training sessions delivered and law enforcement personnel trained Number and type of community educational efforts including numbers reached Short-Term Intermediate Long-Term Decrease in Decrease in the punitive action number of fatal taken during 911 opioid overdoses calls for an opioid in cluster overdose communities Increase in knowledge and awareness among law enforcement personnel and community members on the 911 Good Samaritan Law Increase in calls to 911 during an opioid overdose event Complete a logic model sheet for each problem identified. Include additional rows for each intervening variable being targeted. 24
Logic Model Part I BSAS Identified Problem Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term The Problem Identified by BSAS This is taken from the RFR for each BSAS initiative. It describes why BSAS has made these grant dollars available. Example Misuse/abuse of opioids and unintentional deaths/non-fatal hospital events associated with opioid poisoning.
Logic Model Part 2 Local Manifestation Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Local Manifestation of the Problem Defines the extent of the problem in the local community (quantitative or qualitative). Example During calendar year 2014 there were 20 opioid overdose fatalities in our cluster communities: Jamestown (4); Janetown (5); Jilltown (11).
Logic Model Part 3 Intervening Variable Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Intervening Variable Biological, social, environmental, and economic factors research has shown to be related to substance use and consequences of use. Subsumes, but not limited to risk and protective factors. Example Fear of calling 911 due to police involvement.
Logic Model Part 4 Strategy Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Strategy (or intervention) Programs, policies, and/or practices to reduce use and/or consequences of use. Expected to affect intervening variable which effects outcomes. It is likely than multiple strategies will be used to address each IV. Example Educate community members on the 911 Good Samaritan Law.
Logic Model Part 5 Target Group Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Target Group The immediate audience of each strategy. Example Members of the general community (including users and bystanders).
Logic Model Part 6 Outputs Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Outputs Measures of the extent to which strategies are being implemented as planned (e.g., head counts of individuals participating in a program, estimated views of a prevention billboard). Examples Number and type of community educational efforts including numbers reached.
Logic Model Part 7 Short-Term Outcomes Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Short-Term Outcomes The immediate effects of a program often focus on the knowledge, attitudes, and skills gained by a target audience. Example Increase in knowledge and awareness among law enforcement personnel and community members on the 911 Good Samaritan Law.
Logic Model Part 8 Intermediate Outcomes Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Intermediate Outcomes Changes in behaviors, norms, and/or policies. Often expressed as changes in the intervening variable. Example Decrease in punitive action taken during 911 calls for an opioid overdose; Increase in calls to 911 during an opioid overdose event.
Logic Model Part 9 Long-Term Outcomes Problem identified by BSAS: Local manifestation of the problem: Outcomes Intervening Variable Strategy Target Group Outputs Short- Term Intermediate Long-Term Long-Term Outcomes The ultimate goals of the program often takes time to achieve. Example Decrease in the number of fatal opioid overdoses in cluster communities.
Strategies Consumption Strategies Consumption Strategies (selected list) SAMHSA s CAPT website Prescription Drug Abuse and Misuse Pages. Strategies to Prevent the Non-Medical Use of Prescription Drugs: Using Prevention Research to Guide Prevention Practice (2013). MOAPC Guidance Document. ONDCP Prescription Drug Abuse Page Trust for America s Health Report List of Strategies from Cohort 1 MOAPC Sites 34
Strategies Consequence Strategies Consequence Strategies (selected list) MassTAPP website Intervening Variables And Strategies Related to the Prevention of Opioid Overdose MOAPC Guidance Document SAMHSA s Opioid Overdose Prevention Toolkit List of Strategies from Cohort 1 MOAPC Sites 35
Strategies Identification You are not limited to the resources listed on the previous slides provided that you can supply evidence of the effectiveness of any strategies that do not appear in any of these resources. Start with the intervening variable not with the strategy. If you can t find an appropriate strategy consider looking at strategies that have been linked to the IV for other behavioral health issues. 36
Strategies Selection Criteria Step 3 in the MOAPC Guidance Document (pages 29-30) discuss strategy selection. Emphasis is on: Evidence of linkages between the targeted intervening variable(s) and the proposed strategy Conceptual fit Practical fit 37
Strategies Fit Conceptual Fit How has the strategy been tested with the identified target population or if it has not how can it be generalized to the target population? How will implementing this strategy in your local community help you achieve your anticipated outcomes? Practical Fit Resources (cost, staffing, access to target population, etc.) Collaborative/Coalition Climate (fit with existing prevention/reduction efforts, willingness to accept new programs, buy-in of key leaders, etc.) Community Climate (community attitude toward the strategy, buy-in of key leaders, etc.) Sustainability of the strategy (community ownership, renewable financial support, community champions, etc.) 38
Strategies A Word of Advice Do a small number of things really well versus trying to do too much and doing it poorly. You probably do not want to propose more than 3 consumption strategies and 3 consequence strategies (at a maximum). Remember that not every community or location in your cluster needs to implement every strategy in your logic model nor do they all have to do the same thing at the same time. You may want to focus on low-hanging fruit at the outset. Strategy-specific workgroups are a good idea. 39
When in Doubt Call Them Not Me 40