Designing CME to Change Competence, Performance and Patient Outcomes

Designing CME to Change Competence, Performance and Patient Outcomes

Learning Objectives After participating in this module, learners should be able to Differentiate between knowledge, competence, and performance in the design of educational activities Develop measures of competence, performance, and patient outcomes for CME activities

New ACCME Criteria C2 The provider incorporates into CME activities the educational needs (knowledge, competence, or performance) that underlie the professional practice gaps of their own learners C3 The provider generates activities/ educational interventions that are designed to change competence, performance, or patient outcomes as described in its mission statement

Definition of a Professional Practice Gap When there is a gap between what the professional is doing or accomplishing compared to what is achievable on the basis of current professional knowledge, there is a professional practice gap.

The Gap is between What could be given the evidence (the ideal) What is

Why Does the Gap Exist? What are the educational needs underlying the gap? Does the gap exist because of a: Lack of knowledge (facts and information) Lack of competence (ability/skill/strategy) Lack of performance (doesn t do it)?

Tips for Analyzing Gaps Ask and answer the following questions: Who is the target audience? How prevalent is the need among the target audience? Specifically what is the issue? The condition is difficult to diagnose, difficult to treat, patients do not comply with the treatment plan, etc. Is a CME activity needed or are there non-cme strategies needed, such as patient education, advocacy, etc.?

Writing the Needs Statement Analyze the answers to these questions and synthesize the analysis into a needs statement which delineates the educational needs underlying the gap(s) and state them in terms of specific needs for knowledge, competence and/or performance.

Planning to Measure CME Effectiveness Goals to measure effectiveness need to be considered in the planning stages The process of defining a gap should be done in the context of the expected outcomes or desired results Backwards planning Learning objectives should be written with the expected outcomes in mind Measurable to determine changes in knowledge, competence, and/or performance

Moore s Levels of Educational Outcomes Participation Level 1 Satisfaction Level 2 Learning Performance Level 5 Patient Health Level 6 Community Health Level 7 Level 3a Knows (Knowledge) Level 3b Knows How (Knowledge) Level 4 Competence (Shows How) Moore DE, et al. JCEHP. 2009;29(1):1-15

Framework for Clinical Assessment Does (Performance) Shows How (Competence) Knows How (Procedural Knowledge) Knows (Declarative Knowledge) The clinician uses the competence they have developed in the care of patients. Clinicians are expected to demonstrate how to do it (ie, show they can do what they learned). The clinician possesses the procedural knowledge, can describe how to do something, but may not be able to do it. Acquisition and interpretation of facts. The clinician knows what to do. Moore DE, et al. JCEHP. 2009;29(1):1-15.

Measuring Competence Include Intent to Change Question on Evaluation (difficult to validate) But will the clinician actually do what they say they can do or will do? Assessment methods for measuring competence Observed Case studies with pre and post assessment (post following knowledge dissemination; utilize ARS or paper-based surveys) Skills checklist in hands-on workshop Standardized patients (expensive and resource intensive) Self-report Post CME activity follow-up on intent to change question 3-6 mos Moore DE, et al. JCEHP. 2009;29(1):1-15.

Measuring Performance Did the clinical performance of the participant improve? Did he/she incorporate what was learned in the CME activity into practice? Assessment methods for measuring performance Observed Chart Audit Data Registry Self-report Physician questionnaire (following CME activity) Patient questionnaire (following CME activity) Performance Improvement (PI) CME activity (see Module 7) Moore DE, et al. JCEHP. 2009;29(1):1-15.

Measuring Patient Outcomes/Health Status Did the patient outcome or health status improve as a results of the provider s participation in the CME activity? Some assessment methods for measuring patient health status: Observed Chart Audit Data Registry Quality data Joint Commission Core Measures NCQA HEDIS Measures Healthy People Objectives 2020 Self-report Physician questionnaire Patient questionnaire

Hypothetical Example: The Skin Condition X Gap Patients with Skin Condition X have higher rates of morbidity and mortality from diabetes and cardiovascular disease due to their condition being associated with metabolic syndrome. Needs assessment and analysis indicate there are knowledge, competence and performance gaps and the educational needs underlying these gaps are delineated in Tables 1 and 2.

Table 1: Hypothetical Example: Skin Condition X Knowledge and Competence Desired Result Content Focus What Are the Learners Needs? Over 20% of providers are unaware of the emerging and replicated findings that Skin Condition X is an independent risk factor for the development of metabolic syndrome (What will change and how will you measure it?) Providers recognize that Skin Condition X is an independent risk factor for the development of metabolic syndrome (knows) Metric: Improved scores on pre and post tests of knowledge Knowledge Providers are familiar with the above emerging findings but are unclear regarding which tests should be utilized to screen their current or new patients with Skin Condition X for metabolic syndrome Providers identify the appropriate tests to screen their patients with Skin Condition X for metabolic syndrome (knows how) Metric: Improved scores on pre and post case-based tests (paper-based or with an electronic audience response system) Competence

Table 2: Hypothetical Example: Skin Condition X Performance & Patient Outcomes Desired Result Content Focus What Are the Learners Needs? Medical group chart audit indicated that less than 20% of providers ordered tests to screen their current or new patients with Skin Condition X for metabolic syndrome Patients with Skin Condition X are not screened for metabolic syndrome and go on to develop diabetes and other components of metabolic syndrome resulting in morbidity & mortality (What will change and how will you measure it?) Medical group chart audit indicates that 94% of providers ordered appropriate tests to screen their current or new patients for metabolic syndrome (does); identified those patients with metabolic syndrome and refers them to specialists in treating metabolic syndrome (does) Metrics: Percent of appropriate screening tests ordered (pre & post) Referral diagnosis data Patients with Skin Condition X are screened for metabolic syndrome, diagnosed with metabolic syndrome and receive appropriate preventive treatment resulting in decreased morbidity and mortality (patient outcomes/health) Metrics: Decreased incidence of morbidity from metabolic syndrome in patient population Decreased incidence of mortality from components of metabolic syndrome in patient population Performance Patient outcome/ health status

Summary Start with the end in mind Backwards Planning (see Module 2) Developing approaches to assessing outcomes is not enough by itself Identifying strategies in the planning stages of the activity for measuring the outcomes or desired results is necessary The metric(s) and/or data sources used to identify the gap can often serve as a template for the outcome measurement The purpose of CME is to improve the quality and delivery of patient care!

Summary Think of designing the CME activity as if you were building a bridge to improvement: Gap(s) Educational Need(s) (KCP) Learning Objectives Improved Outcomes (CP or PO) Plan at the beginning how the outcome(s) will be measured