Evolution of CPD towards Revalidation: International perspectives

Transcription

1 Evolution of CPD towards Revalidation: International perspectives Professor E A (Liz) Farmer CMES Revalidation Seminar 14 March 2014

2 Overview of presentation Introduction to terms and concepts Four Frames of Assessment International perspectives Conclusions

3 Difference between competence, performance and outcomes Competence = what a doctor is deemed capable of doing Performance = what a doctor is observed to do in actual professional practice Outcomes = indicators of health status of patients

4

5 A caveat in describing practice: Regulators and Colleges use terms differently Lack of common terminology An explosion of TLAs Large variations in term CPD Must drill down to detail Need a common language to advance understanding 5

6 An Organising Framework Four frames of assessment Klass D. Assessing Doctors at Work Progress and Challenges N Engl J Med 356;4 January 25, 2007

7 Pre-entry assess competence and readiness During Practice Measure outcome indicators Klass 4 frames During Practice Inferring competence from CME Professional practice Assess performance

8 Pre-entry Exams Outcomes Revalidation During Practice CME Performance in Practice

9 CME The second frame infers competence in practice from participation in continuing medical education and training programs (Klass op cit) During Practice Inferring competence from CME

10 Assessing performance Canada - history of cutting edge work in this area since the 90s MEPP model Maintenance and Enhancement of Professional Performance in 2000 Professional practice Assessing performance

11 MEPP model

12 Ontario: College Physicians Surgeons Ontario Random Peer Assessment (3900 doctors) In practice over 5 yrs and under 70 Random selection (compulsory to participate) Over 70 compulsory About per year Objective information from medical record review and discussion with doctor (half day) Matched peer reviewer (scope) Assess against standard for regulatory judgment

13 Ontario Peer Assessment Performance- Based Assessment Practice inspection Random peer assessment One Peer Records Discussion Decision Point

14 Value of peer assessment Systematic review Feedback generated from peer assessment has positive effects, when the feedback came from credible peers or authoritative sources The strongest effects were found in studies where performance was evaluated and feedback given over longer periods of time. J Veloski, JR Boex, MJ Grasberger et al. Systematic review of the literature on assessment, feedback and physicians clinical performance BEME Guide no 7. Dundee, Association for Medical Education in Europe (2006).

15 Alberta Physician Achievement Review (PAR) A form of Multi-Source Feedback

16 Council Academic Hospitals Ontario

17 Feasibility high Cost effective (est. $40 Can per yr) Triangulation Self-evaluation and peer-assisted reflection Importance of feedback (Lockyer et al 2003) Patients views Low performers referred to Step 2 MEPP model PAR approaches 17

18 MSF approaches: spreading PAR M-PAR MSF approaches Canada Identifies possible underperformers NS-PAR CAHO Not for underperformance Decision Point

19 Pre-entry Exams Outcomes MEPP Canada During Practice CME CME Peer review Record review Performance in Practice Colleague, Coworker & Patient Feedback (PAR)

20 Assessing performance Shift in entry level and trainee education Introduction of performance assessment Workplace based assessment Mini-CEX Case-based discussion Multi source feedback Professional practice Assessing performance

21 UK: giant leap

22

23 PDP & collegial relationship over time E portfolio Sits in all 3 Klass s frames 23

24 Pre-entry Exams Outcomes Revalidation UK During Practice CME Quality Improvement Activity Sig Events Complaints Compliments Performance in Practice Colleague & Patient Feedback CME

25 USA 24 member Boards abt. 800,000 doctors From 1990 recertification if Board certified Maintenance of certification (MOC) I. Evidence of good professional standing (unrestricted license to practise in a jurisdiction); II. Participation in knowledge self-assessment; III. A secure examination; and IV. A practice audit &improvement exercise (specialty)

26 Secure examination Pre-entry Exams Outcomes US Revalidation During Practice CME Practice audit & improvement exercise Performance in Practice Knowledge self assess

27 Main trends Beyond CME - interest in right touch approaches Significant variation in ways revalidation is carried out Strengthening and broadening existing approaches from different starting points Countries have moved or are considering more uniform national legislation Shared accountability & a programmatic approach Evidence points to the importance of feedback

28 Dual aims of revalidation Improvement and Underperformance