Robyn Tamblyn, BSCN MSc PhD May 2 nd, 2018 Baldwin Seminar Series Accreditation Council for Graduate Medical Education

Robyn Tamblyn, BSCN MSc PhD May 2 nd, 2018 Baldwin Seminar Series Accreditation Council for Graduate Medical Education

Robyn Tamblyn, BScN, MSc, PhD, CM Disclosure None of the above speakers have any conflicts of interest to report 2

Rates for all procedures by health region, Canada, 2010 3 (OECD 2014) Geographic Variations in Health Care: What Do We Know and What Can Be Done to Improve Health System Performance?

% of people with type 1 and type 2 Diabetes who received guidelinerecommended care processes 4 (NHS 2016) The NHS Atlas of Variation in Healthcare

% of diabetic Medicare enrollees receiving appropriate management 5 (2014) The Dartmouth Atlas of Health Care

Rates for all procedures by health region, Canada, 2010 Unexplained by differences in illness Unexplained by patient preference Unexplained by lack of resources % of people with type 1 and type 2 Diabetes who received guideline-recommended care processes A function of physician ability and decision making Identifying variations and reducing them should be a priority % of diabetic Medicare enrollees receiving appropriate management 6

Is there an association between licensing exam scores and primary care practice? Training and Licensure / Certification Practice Performance Patient Outcome: Short Term Patient Outcome: Long Term Undergrad Medical Training Licensing Exam Postgraduate Training Certification Exam Preventive Care Episodic Acute Care Chronic Disease Management Morbidity Mortality Pt Satisfaction Quality of Life Morbidity Mortality Pt Satisfaction Quality of Life Competence to enter postgraduate training Competence to enter practice Coordination of care Resource Use $ $ 7

Study cohort: 984 physicians followed over 4-7 years in Quebec after passing Quebec family medicine certification examination between 1990 and 1993 Change in practice performance outcomes per SD increase in score 190 170 P < 0.0001 Mammography Rate per 1000 patients 150 130 110 90 70 50 P = 0.0003 screening rate per 1000 patients Consultation rate per 1000 patients Contraindicated prescriptions per 1000 elderly 30 P = 0.03 patients 10-2.89 (-3 SD) -2 (-2 SD) -1.11 (-1 SD) -0.22 (mean) 0.67 (+1 SD) 1.56 (+2 SD) 2.45 (+3 SD) Medical Council of Canada Licensing Examination Scores (Standardized) 8 Tamblyn et al. Association between licensure examination scores and practice in primary care. JAMA (2002)

Physician scores on an National Clinical Skills Examination as predictors of complaints to medical regulatory authorities in Ontario and Quebec 4.5 4.26 Complaint rate (per 100 practiceyears) 4 3.5 3 2.5 2 1.5 1 0.5 2.92 2.68 2.51 For every 2 SD decline in the communication score, there is a 38% increase in the rate of complaints 0 1 2 3 4 Communication score, by quartile 9 Tamblyn et al. JAMA (2007)

Conclusions: Licensing examination scores are significant independent predictors of resource use and the quality of care in practice. Training and licensure are one potential target in optimizing evidence-based healthcare. 10 Tamblyn et al. Association between licensure examination scores and practice in primary care. JAMA (2002)

Once physicians enter practice, a multitude of systemic, organizational, and individual factors affect their performance. Years in practice Age Sex Continuing medical education Practice volume Practice type Hours worked Total # of clinical and admin staff # of patient visits per week Healthcare interventions should be developed based on an understanding of the underlying problems and causes. Individual Organizational Payment systems Availability of resources Physician to population ratio Northern practice location Systemic 11 Wenghofer et al. Factors affecting physician performance: Implications for performance improvement and governance. Healthcare Policy (2009)

Multiple prescribing physicians: increase the risk of inappropriate prescriptions >9 Proportion of patients with at least one inappropriate rx 100 80 60 40 20 0 1 2 3-4 5-8 > 9 Number of prescribing physicians 3-4 md: 29% 5-8 MDs 22% MDs 6% 3-4 MDs 29% 1MD 22% 2 MDs 21% MOXXI: 198,051 eligible patients insured by the RAMQ drug insurance from Jan. 2006 to Jan. 2007 12 Tamblyn et al. CMAJ (1993)

Handwritten prescriptions: increase the risk of transcription errors 1.6% of these errors are potentially dangerous 13% Rx with errors 87% Rx without errors 13 Kistner et al. (1994)

MOXXI: Medical Office of the XXIst Century Electronic medical record Computerized drug management system Electronic prescriber Platform for clinical decision support 14

MOXXI: Medical Office of the XXIst Century 1,800 Quebec pharmacies Online payment & adjudication Public Insurer Complete record of all drugs dispensed updated in realtime 15

16 Integrated drug profiler: retrieves a complete drug profile from an integrated drug management system

17 Computerized Prescribing: reduces prescribing potential errors by providing menus for dose selection

18 Computerized Prescribing: handwritten vs. computer-generated prescriptions

Computerized Prescribing: typed prescriptions reduce prescribing and transcription errors 1.8 1.6 1.4 1.2 Risk Ratio 1 0.8 0.6 0.4 0.2 0 19 Ammenworth et al. JAMIA (2008)

Systemic Problems: Care Transitions and Medication Errors 67% of inpatients have at least 1 error in their medication history at admission 12% of medication discrepancies are unintentional, but have potential for harm. Of these, 72% were due to admission errors, while 26% were due to lack of reconciliation at discharge 19% to 23% of patients will have an adverse drug event within 30 days of hospital discharge and 14.3% will be readmitted Adverse drug events are preventable in 58% of the cases Adverse drug events are the 6th leading cause of death at a cost over $5.6 million per hospital per year Pippens JE et al. Classifying and Predicting Errors of Inpatient Medication reconciliation. J Gen Intern Med. 2008. 23(9):1414-22 Forster AJ et al. Adverse events among medical patients after discharge from hospital. CMAJ. 2004;170:345-349. Forster AJ et al. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138:161-167. Coleman EA et al. Posthospital medication discrepancies: prevalence and contributing factors. Arch Intern Med. 2005;165:1842-1847. Leape LL et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274:35-43. Bates DW et al. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA. 1997;277:307-311. 20

21

22 RightRx: Electronic Process for Medication Reconciliation targeting transitions in care

RightRx: Discharge Clear community medication list available for discharge prescription Alignment of medications User forced to act on each medication 23

RightRx: Discharge Legible discharge prescription, including all medications to be stopped Reasons for changes displayed under each medication Number of renewals can be pre-set to user s choice Discharge physician s name and license number printed 24

Time for Medication Reconciliation Form Completion in Medicine Usual care RightRx 3.33 2.43 2.37 Minutes 1.63 CDL DISCHARGE 25 Tamblyn et al. JAMIA (2017)

Medication Reconciliation Completion Rates in the RightRx and Usual Care Units Medication Reconciliation Completion Rates (%) 88.1 46.3 26.2 3.7 4.8 15.1 0.2 6.5 COMPLETE MAJOR INCOMPLETE MINOR INCOMPLETE NOT ATTEMPTED Usual care RightRx 26 Tamblyn et al. JAMIA (2017)

Potential Adverse Drug Event Rates Related to Incomplete Medication Reconciliation in the Intervention and Control Groups 45% 42.3% Rates of Potential Adverse Drug Events (%) 40% 35% 30% 25% 20% 15% 10% 5% 21% 2.3% 9.9% 0% Errors of omission RightRx Group Usual Care Group Therapy duplications 27 Tamblyn et al. JAMIA (2017)

28 Computerized Decision Support: providing automated alerts for prescribing problems

Computerized Decision Support: on-demand vs automated drug alerts % Seen and Revised in 3,449 Patients Characteristic On Demand Alerts Automated Alerts Number of Patients 1,550 1,899 Number of Alerts 4,445 6,506 % Seen 0.9% 10.3% % Revised that were Seen 75.6% 12.1% 29 Tamblyn et al. JAMIA (2008)

Computerized Decision Support: most drug alerts from commercial systems are ignored Physicians override 49% to 96% of alerts for drug-allergy, drug-drug, and drugdisease contraindications Experienced physicians more likely to override alerts than trainees Incorrect data explains most allergy and pregnancy alerts Benefit great than risk reason for most overrides 30 Weingart et al. Archives In Med (2003)

31 Computerized Decision Support: incorporating epidemiological science into risk-benefit assessment

Computerized Decision Support: designing new smart alerts that provide person-specific risk assessment 0.304*class_2 + 0.035*class_3 + 0.253* class_4 + 0.105*class_5 + 0.088*class_6 + 0.088*class_7 hazard _ total = hazard _ nm * EXP + 0.225*class_9 + 0.056*class_11+ 0.038* age_65 0.008*class_13 + 0.047 * + female 0.106*class_15 + 0.123* + gb + hazard _ nm = EXP 0.099*class_16 + 0.047 *class_17 0.287 * ci - 0.0023* lw + 0.147 *class_18 + 0.07 *class_23 ( 1 ( 1 ) ^) ^ hazardnmtotal ) ) baseline risk risk total nm= = 100* 100 100 32

Computerized Decision Support: RCT quantifies risk reduction Risk of injury per 1000 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 Change of - 0.17 (-0.32 to -0.02) in risk of injury per 1000 Greatest change for patients at highest risk 3.1 3 N=2,741 Control N=2,887 TRIPP 33 Tamblyn et al. The effectiveness of a new generation of computerized drug alerts in reducing the risk of injury from drug side effects; a cluster randomized trial. JAMIA (2012)

Nonadherence: to anti-hypertensive treatment reduces the benefits of treatment for stoke and MI prevention and is significantly associated with knowledge and decision making scores on licensing exams 100% 90% 80% 70% % Persistent 60% 50% 40% 30% 20% 10% 0% 0 50 100 150 Days since started therapy 34 Overall Top Quartile Bottom Quartile Tamblyn et al. Archives of Internal Medicine (2010)

Nonadherence: user fees related to drug costs and adverse effects are the most common causes of nonadherence OR 0.56 (0.49-0.64) OR 0.37 (0.32-0.41) 80% OR 3.61 (1.85 7.04) 74.6% Primary nonadherence, % 45% 40% 35% 30% 25% 20% 15% 10% 5% 41.2% 12.6% 12.3% % of intentional nonadherence 70% 60% 50% 40% 30% 20% 10% 25.4% 0% Maximum copayment ($80/mo) Partial copayment ($45/mo) Free medication 0% <5 adverse effects 5 adverse effects 35 Tamblyn et al. Annals of Internal Medicine (2014) Lowry et al. Hypertension (2005)

36 Computerized Decision Support: providing physicians with information on treatment adherence

Computerized Decision Support: Increasing the detection and response to adherence problems with cardiovascular medication in primary care through computerized drug management systems: A randomized controlled trial 30 primary care physicians (2329 patients prescribed cardiovascular drugs with follow-up visit) Control Group Intervention Group 37 Tamblyn et al. Med Decis Making (2010)

Computerized Decision Support: helps identify side-effects as a cause of non-adherence 50% 40% Physicians are more likely to identify side effects as cause for nonadherence % of CV stop/changes 30% 20% 10% 0% Ineffective treatment Intolerance to drug or sideeffect(s) Other Profile Blocked Profile Available 38 Tamblyn et al. Med Decis Making (2009)

Computerized Decision Support: showing comparative patient out-of-pocket costs for antihypertensives 39 Tamblyn et al. Med Decis Making (2009)

Computerized Decision Support: showing comparative patient out-of-pocket costs for antihypertensives 40 Tamblyn et al. Med Decis Making (2009)

Computerized Decision Support: showing comparative patient out-of-pocket costs for antihypertensives % increase in patients newly started on diuretics in out-of-pocket module and usual care groups 38 physicians, 1914 patients in Out-of-Pocket Expenditure Module 56 physicians, 3592 patients 38 physicians, 1678 patients in Usual Care (basic MOXXI system) % Increase in prescription of diuretics 30% 25% 20% 15% 10% 5% 26.6% RR 1.65, 95% CI 1.17-2.33 19.8% 0% Out of Pocket Expenditure Module Usual Care 41 Tamblyn et al. Med Decis Making (2009)

42 Computerized Decision Support: for evidence-based asthma management daily surveillance of ER visits and rescue medication to assess asthma control

43 Computerized Decision Support: for evidence-based asthma management daily surveillance of ER visits and rescue medication to assess asthma control

Computerized Decision Support: for evidence-based asthma management daily surveillance of ER visits and rescue medication to assess asthma control Out-of-Control at First Visit: 704 (15.8%) In-control at First Visit: 3,743 (84.2%) 120 100 80 60 RR: 0.87 95% CI: 0.77, 0.99 120 100 80 60 40 20 40 20 RR: 1.00 95% CI: 0.74, 1.36 0 0 Exacerbation Rate Exacerbation Rate / 100/year Control Intervention Control Intervention 44

Observational studies help identify modifiable determinants of adverse events and health outcomes e.g. training and licensure, multiple prescribers, off-label prescribing, non-adherence Interventions targeting those determinants should be developed based on an understanding of healthcare problems and their causes e.g. MOXXI, RightRx The future of practice-based interventions lies in tailoring interventions to particular physician profiles and preferences 45

Off-label Prescribing Percent of Prescriptions for On-Label and Off-Label Uses Percentage of Antidepressants being Prescribed Off-Label 45.00% 42.40% 40.00% 35.00% 30.00% 29.30% 25.00% 21.80% On-Label Use Off-Label Use with Strong evidence Off-Label Use without Strong evidence 20.00% 15.00% 10.00% 5.00% 0.00% 6.10% 8.14% SSRI SNRI TCA Other All Classes 46 Eguale et al. JAMA Internal Medicine (2016) Wong et al. BMJ (2017)

Off-label Prescribing: associated with adverse drug events in adult population 47 Eguale et al. JAMA Internal Medicine (2016)