Combined Milestones in Med-Peds

Transcription

1 Making It Count Twice: Combined Milestones in Med-Peds Emily Machogu, MD Anna Volerman, MD Michael Aylward, MD Benjamin Kinnear, MD

2 Objectives Analyze current use of combined or mapped milestones in Med-Peds residency programs Brainstorm how we would like to see milestones used in Med-Peds programs in future Discuss the development of a validated set of combined Med-Peds milestones

3 Who are you?

4 Tell us about your med-peds program

5 Tell us about milestones at your programs

6 Should med-peds have a combined set of milestones?

7 Competencies and Milestones Patient care Medical knowledge Practice Based Learning and Improvement Systems Based Practice Professionalism Interpersonal Skills and Communication Specialty-specific Developmental points Progression over residency training Inform accreditation

8

9 Milestones for Med-Peds

10 Dreyfuss model Dreyfus & Dreyfus, Carraccio CL et al, Diagram made by ABIM.

11 Framework assessment tools mapping milestones

12 Indiana University Combined Milestones Emily Machogu, MD

13 Our Evaluation Process Evaluations are completed by residents and faculty after each rotation Both IM and Peds evaluation forms use the ACGME milestones directly, as well as one or two rotation specific questions Each rotation evaluates the resident on 5-10 milestones, with all of the reporting milestones being spread out over the various rotations

14 The effect on Med/Peds 21 milestones on pediatrics + 22 milestones on internal medicine = 43 milestones for each resident Rotation schedules and assignments of milestones to specific rotations leaves blanks across the 43 milestones for individual residents in given years Resident [IM] PC [IM] PC [IM] PC [IM] PC-4 n/a [IM] PC [IM] MK-1 4 [IM] MK [IM] SBP-1 n/a [IM] SBP-2 n/a [IM] SBP-3 4 [IM] SBP-4 n/a [IM] PBLI-1 n/a [IM] PBLI-2 n/a [IM] PBLI [IM] PBLI [IM] PROF [IM] PROF [IM] PROF-3 n/a [IM] PROF [IM] ICS [IM] ICS [IM] ICS Resident [PD] PC1 4 [PD] PC2 4.5 [PD] PC3 4.3 [PD] PC4 3.9 [PD] PC5 4 [PD] MK1 3.9 [PD] SBP1 4 [PD] SBP2 3.8 [PD] SBP3 4 [PD] PBLI1 3.5 [PD] PBLI2 3.9 [PD] PBLI3 n/a [PD] PBLI4 n/a [PD] PROF1 n/a [PD] PROF2 4 [PD] PROF3 4.5 [PD] PROF4 3.5 [PD] PROF5 4.5 [PD] PROF6 4.3 [PD] ICS1 4.3 [PD] ICS2 4.5

15 Our solution ACGME milestones from internal medicine and pediatrics were analyzed and compared to seek similarities and overlap A combined set of IUSM Med/Peds specific milestones was developed with 28 total combined milestones Average score on each IM and Peds milestone is extracted from Medhub, along with the number of times the resident received a score for that milestone A weighted average is used in Excel to determine the combined milestone score for that resident IM PC1 (average score) IM PC1 (n) PD PC1 (average score) PD PC1 (n) IM PC1 (n) PD PC1 (n)

16 IM MILESTONE IM/PEDS MILESTONE PEDS MILESTONE PC1: Gathers and synthesizes essential and accurate information to define each patient s clinical problem(s) PC2: Develops and achieves comprehensive management plan for each patient PC3: Manages patients with progressive responsibility and independence PC4: Skill in performing procedures PC5: Requests and provides consultative care PC1: Gathers and synthesizes essential and accurate information about the patient PC2: Develops and achieves comprehensive management plans PC3: Manages patients with progressive responsibility and independence PC4: Skill in performing procedures PC5: Requests and provides consultative care PC6: Organizes and prioritizes responsibilities to provide patient care that is safe, effective, and efficient PC7: Makes informed diagnostic and therapeutic decisions that result in optimal clinical judgment PC1: Gather essential and accurate information about the patient PC5: Develop and carry out management plans PC2: Organize and prioritize responsibilities to provide patient care that is safe, effective, and efficient PC4: Make informed diagnostic and therapeutic decisions that result in optimal clinical judgment

17 Resident PC1 4.2 PC2 4.3 PC3 4.5 PC4 n/a PC5 4.5 PC6 4.5 PC7 3.9 MK1 4.0 MK2 4.5 MK3 3.9 SBP1 4.3 SBP2 3.8 SBP3 4.0 SBP4 4.0 SBP5 4.3 PBL1 3.5 PBL2 4.0 PBL3 4.4 PBL4 n/a PROF1 4.4 PROF2 4.3 PROF3 4.5 PROF4 4.0 PROF5 3.5 PROF6 4.3 ICS1 4.3 ICS2 4.5 ICS3 4.3

18

19

20

21

22

23 ACGME Milestones and Med-Peds Benjamin Kinnear, MD University of Cincinnati College of Medicine

24 Observable Practice Activities (OPAs) What residents DO. Content-Based Observable Practice Activities (COPAs) 1. Initiate basal bolus insulin therapy 2. Manage elevated blood pressure 3. Manage COPD Different for every rotation Process-Based Observable Practice Activities (POPAs) 1. Minimize unfamiliar terms during patient encounters. 2. Use teach-back method Similar for every rotation Entrustment and Mapping of Observable Practice Activities for Resident Assessment Eric J. Warm MD, Bradley R. Mathis MD, Justin D. Held MD, Savita Pai MD, Jonathan Tolentino MD, Lauren Ashbrook MD, Cheryl Lee MD, David Lee MD, Sharice Wood MD, Carl J. Fichtenbaum MD, Ryan Munyon MD, Caroline Mueller MD, JGIM 2014

25 Observable Practice Activities (OPAs) Antibody metaphor Content-Based OPAs (COPAs) Variable portion of antibody Different for every rotation Process-Based OPAs (POPAs) Constant portion of antibody Generally similar for all rotations

26 OPA creation: Iterative Process Each division owned their own OPAs Guideline: 10 or fewer Content OPAs per rotation Residency Educational Coordinators presented these at divisional faculty meetings We attended these meetings Once all OPAs were written, we reviewed them for redundancy and evenness of language

27

28 How many OPAs do we have? >400 always evolving

29 Observable Practice Activities (OPAs) Content-Based (COPA) Process-Based (POPA) Mapping Entrustment (1-5) Reporting Milestones External Reporting ACGME report Longitudinal Assessment (summative/formative) Warm EJ, Mathis BR, Held JD, Pai S, Tolentino J, Ashbrook L, Lee CK, Lee D, Wood S, Fichtenbaum CJ, Schauer D, Munyon R, Mueller C. Entrustment and Mapping of Observable Practice Activities for Resident Assessment. J Gen Intern Med Feb 21. [Epub ahead of print] PubMed PMID:

30 Mapping We mapped with the following in mind: When assessing a trainee on a given OPA, would the evaluator feel as if they were also assessing the mapped milestone? We removed all soft calls Entrustment and Mapping of Observable Practice Activities for Resident Assessment Eric J. Warm MD, Bradley R. Mathis MD, Justin D. Held MD, Savita Pai MD, Jonathan Tolentino MD, Lauren Ashbrook MD, Cheryl Lee MD, David Lee MD, Sharice Wood MD, Carl J. Fichtenbaum MD, Ryan Munyon MD, Caroline Mueller MD, JGIM 2014 accepted for publication

31 COPA: Initiate cost-effective workup for anemia Reporting Milestones PC-1 Gathers and synthesizes essential and accurate information to define each patient s clinical problem(s) MK-1 Clinical knowledge MK-2 Knowledge of diagnostic testing and procedures SBP-3 Identifies forces that impact the cost of health care, and advocates for, and practices cost-effective care Mapping Example

32 Observable Practice Activities Initiate a cost effective work-up for anemia 3 Competencies Patient Care Medical Knowledge Systems Based Practice Reporting Milestones 3 PC-1 - Gathers and synthesizes essential and accurate information to define each patient s clinical problem(s) 3 3 MK-1 - Clinical knowledge MK-2 - Knowledge of diagnostic testing and procedures 3 SBP-3 - Identifies forces that impact the cost of health care, and advocates for, and practices cost-effective care.

33

34 Current process IM OPAs Peds OPAs Mapping Entrustment (1-5) Mapping Entrustment (1-5) IM Reporting Milestones Peds Reporting Milestones

35 What if? IM OPAs Peds OPAs Mapping Entrustment (1-5) Mapping Entrustment (1-5) IM Reporting Milestones Peds Reporting Milestones

36 Should we map direct assessment data to the other side s milestones?

37 Discussion Questions

38 Who could combining the milestones benefit? PDs? Residents?

39 Could you see ways that combined milestones might cloud the picture? For example, what might happen if a resident struggles on one side more than the other?

40 How might combining the milestones help with feedback to the residents? What kind of positive outcomes have you experienced or could you imagine?

41 How does the difference in wording between the anchors in the medicine and pediatric milestones (especially in the lowest level) affect the ability to combine similar milestones?

42 Does it make sense to combine some subcompetencies but not others? In other words, are some more conserved than others across specialties?