VA Residents Improve Access and Financial Value AAMC Fall Meeting Nov. 5, 2011 Office of Academic Affiliations, VHACO T. Michael Kashner, PhD, JD, MPH Barbara K. Chang, MD, MA 1
Outline Current status of literature 2
Outline Current status of literature Data from Department of Veterans Affairs (VA) 3
Outline Current status of literature Data from Department of Veterans Affairs (VA) Metrics workload Resident Education Index Elasticity Mean Resident Contribution 4
Outline Current status of literature Data from Department of Veterans Affairs (VA) Metrics workload satisfaction Resident Education Index Elasticity Mean Resident Contribution Learners' Perceptions Survey Teaching Experience Clinical Experience Working Experience Future Employment 5
Outline Current status of literature Data from Department of Veterans Affairs (VA) Metrics workload satisfaction supervision Resident Education Index Elasticity Mean Resident Contribution Learners' Perceptions Survey Teaching Experience Clinical Experience Working Experience Future Employment Resident Supervision Index Trainee supervision Education outcome Faculty time costs 6
Outline Current status of literature Data from Department of Veterans Affairs (VA) Metrics workload satisfaction supervision outcomes Resident Education Index Elasticity Mean Resident Contribution Learners' Perceptions Survey Teaching Experience Clinical Experience Working Experience Future Employment Resident Supervision Index Trainee supervision Education outcome Faculty time costs Care process Patient health 7
Current Status of Literature Few studies measure clinical workload (of residents) Clinical capacity for residency allocation: # beds / resident # physicians / resident # patient cases / resident Problems failure to consider inpatient & outpatient settings focus on # of patients treated without accounting for care complexity procedures done per clinical encounter service intensity encounters per patient failure to consider resource availability residents & specialty mix professional staff & specialty mix support staff other expenditures medical supplies, equipment, depreciation failure to account for program size scale economies 8
Current Status of Literature Few studies measure clinical workload (of residents) Clinical capacity for residency allocation: # beds / resident # physicians / resident # patient cases / resident Problems Campbell, Gillespie, Romeis, Inquiry 1991 1988 VA data / combined all specialties resident contributions offset supervision costs teaching = non-teaching VAMCs financial performance failure to consider inpatient AND outpatient settings focus on # of patients treated without accounting for care complexity procedures done per clinical encounter service intensity encounters per patient failure to consider resource availability residents & specialty mix professional staff & specialty mix support staff other expenditures medical supplies, equipment, depreciation failure to account for program size scale economies 9
Source of Data: Department of Veterans Affairs ~37,000 medical residents annually 30% US total medical residents 10,300 resident positions 9% US positions 2,200 ACGME-accredited programs 79 medical specialties & subspecialties 10
VA Medical Centers 2000-2010 11
VA Medical Centers 2000-2010 24 VAMCs had no residents 12
VA Medical Centers 2000-2010 includes small and large programs 13
VA Medical Centers 2000-2010 Outpatient Number 1 Inpatient Number 1 Patients /yr. 5.2 Patients / yr. 0.4 Visits 535.5 Inpatient Days 53.0 Procedures 1,653.8 Medicine 27.3 Medicine 515.6 Surgery 14.2 Surgery 37.7 Psychiatry 11.5 Psychiatry 90.4 market value 3 $368b Other medicine 827.3 Dentistry 30.9 Other 151.9 market value 2 $390b 1 measured in millions unless otherwise specified 2 measured in 80 th percentile of paid transaction charges, FY2008. 14
VA Medical Centers 2000-2010 Outpatient Number 1 Inpatient Number 1 Patients /yr. 5.2 Patients / yr. 0.4 Visits 535.5 Inpatient Days 53.0 Procedures 1,653.8 Medicine 27.3 Medicine 515.6 Surgery 14.2 Surgery 37.7 Psychiatry 11.5 Psychiatry 90.4 market value 3 $368b Other medicine 827.3 Dentistry 30.9 Other 151.9 market value 2 $390b $758 billion in services 1 measured in millions unless otherwise specified 2 measured in 80 th percentile of paid transaction charges, FY2008. 15
Resident Education Index (REI): Purpose To view workload: as opportunities to allocate resident positions as contributions to assess financial performance 16
Resident Education Index (REI): Definitions Production Unit: VA medical center including satellite clinics Workload: all clinical procedures and services produced during an academic year by professional resources computed in dollars at market based prices by specialty: medicine, surgery, psychiatry, dental, other Productivity: amount workload produced as output per unit resources as input 17
Resident Education Index (REI): Definitions Resources: Residents / by specialty Attending physicians / by specialty Other resources health professionals (RN, psychologists MSW, etc) expendable supplies and devices building and equipment depreciation Market-based Prices: VA Reasonable Charges : 80 th percentile of total US paid transaction charges for outpatient CPT procedures and inpatient DRG days. 18
Resident Education Index (REI): Definitions Actual workload: amount produced in fact Expected workload: amount produced facility resources but at mean facility productivity 19
Resident Education Index (REI): Computation ratio of actual workload produced ($) per $1 of workload expected, with local facility resources, but at mean productivity REI=0: producing at mean productivity REI>0: producing above mean REI<0: producing below mean 20
Resident Education Index (REI): Computation actual facility workload REI = ln w o w e = lnw lnw o e expected facility workload 21
Resident Education Index (REI): Computation Actual facility workload based on with number of residents r 0 attending staff a 0 other resources i o at productivity p 0 ( 0, a0, i0 p0 ) (, a, i p) w r REI 0 = ln = ln w r0 0 0 0 0 0 0 w r 0 0 0 (, a, i p ) ln w( r, a, i p) Expected facility workload at mean productivity across VA 22
Resident Education Index (REI): VA Medical Centers, 2000-2010 350 300 Number of VA facility reporting years 250 200 150 100 50 0-1.6-1.4-1.2-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 >1.6 med surg psy anc REI score 23
REI vs. Cases per Resident in Medicine by VAMC Reporting Years (2000-2010) r = -.026 p=.430 24
REI vs. Cases per Resident in Surgery by VAMC Reporting Years (2000-2010) r =.149 p<.001 25
REI vs. Cases per Resident in Psychiatry by VAMC Reporting Years (2000-2010) r = -.084 p=.033 26
Workload Elasticity (E): Definition % change in expected workload produced by 1% in #residents #residents increase 1%, then: if E = 0 /workload is unchanged if E = +0.2 /workload 0.2% if E = -0.2: /workload 0.2% 27
Workload Elasticity (E): Computation E = w(r,a,s p ) w(r,a,s p ) 1 0 w(r,a,s p ) 0 r r 1 0 r 0 28
expected facility workload produced with r 1 residents. and mean attending staff (a) other resources (s), productivity (p) E = w(r,a,s p ) w(r,a,s p ) 1 0 w(r,a,s p ) 0 r 1 number of residents Workload Elasticity (E): Computation r r 1 0 r 0 29
expected facility workload produced with r 0 residents and mean attending staff (a) other resources (s), productivity (p) E = Workload Elasticity (E): Computation w(r,a,s p ) w(r,a,s p ) 1 0 w(r,a,s p ) 0 r r 1 0 r 0 r 0 number of residents 30
Workload Elasticity (E): VA Medical Centers, 2000-2010 31
Mean Resident Contribution (MRC): Definition change in expected workload by adding 1 resident workload as %FTE staff physician #residents increase by 1 position, then: if MRC = 0 /workload is unchanged if MRC = +0.1 /workload 0.1 FTE Staff if MRC = -0.1 /workload 0.1 FTE Staff 32
Mean Resident Contribution (MRC): Computation MRC = [ w( r +1,a,s p ) w( r,a,s p )] [ w( r,a +1,s p ) w( r,a,s p )] 33
Mean Resident Contribution (MRC): Computation contribution of mean resident to expected workload at mean facility with mean attending staff, mean other resources, at mean productivity MRC = [ w( r +1,a,s p ) w( r,a,s p )] [ w( r,a +1,s p ) w( r,a,s p )] contribution of mean attending physician to expected workload at mean facility with mean residents mean other resources, at mean productivity 34
Mean Resident Contribution (MRC): VA Medical Centers, 2000-2010 35
Learners' Perceptions Survey (LPS): Purpose & Overview Trainee satisfaction questionnaire OMB approved VA/OAA administered annually since AY2001 All VA trainees 36
Learners' Perceptions Survey (LPS): Satisfaction Domains teaching experience learning environment (15 elements /5-pt Likert scales) clinical faculty & preceptors (13 elements/5-pt Likert scales) clinical experience clinical environment (15 elements/5-pt Likert scales) staff & service availability (13 elements/5-pt Likert scales) staff & service quality (6 elements/5-pt Likert scales) processing handling medical errors (6 elements/5-pt Likert scales) working experience working environment (13 elements/5-pt Likert scales) physical environment (12 elements/5-pt Likert scales) personal experience environment (13 elements/5-pt Likert scales) summary VA versus non-va (9 elements/5-pt Likert scales) choose VA training again (1 element, 4-pt Likert) choose VA future employment (2 elements, mixed) 37
Learners' Perceptions Survey (LPS): Satisfaction Domains teaching experience learning environment (15 elements /5-pt Likert scales) clinical faculty & preceptors (13 elements/5-pt Likert scales) clinical experience clinical environment (15 elements/5-pt Likert scales) staff & service availability (13 elements/5-pt Likert scales) staff & service Consider quality (6 elements/5-pt VA employment? Likert scales) processing handling medical errors (6 elements/5-pt Likert scales) Before rotation 32.8% working experience working environment (13 elements/5-pt Likert scales) physical environment After rotation (12 elements/5-pt 77.2% Likert scales) personal experience - odds environment ratio (13 6.9 elements/5-pt x Likert scales) summary VA versus non-va (9 elements/5-pt Likert scales) choose VA training again (1 element, 4-pt Likert) choose VA future employment (2 elements, mixed) 38
Learners' Perceptions Survey (LPS): Computation Scores adjusted for: gender discipline and specialty level (PGY) US medical school status satisfaction propensity mix of patients seen over 65 years age chronic mental illness chronic medical illness multiple medical illnesses alcohol/ substance use dependence low income socioeconomic status lack social support / family support 39
Learners' Perceptions Survey (LPS): Computation Domain satisfaction rates (LPS d ): positive satisfaction: Expected proportion respondents very satisfied negative satisfaction: Expected proportion respondents very dissatisfied 40
REI versus LPS satisfaction: SURGERY VA Medical Centers, 2000-2010 1 0.9 0.8 0.7 0.6 proportion very satisfied p=.018 p<.003 p=.24 emp lrn cln per 0.5 0.4 0.3-1.6-1.4-1.2-1.0-0.8-0.6-0.4-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 REI score 41
REI versus LPS satisfaction: SURGERY VA Medical Centers, 2000-2010 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 proportion very dissatisfied p<.003-1.6-1.4-1.2-1.0-0.8-0.6-0.4-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 emp lrn cln per REI score 42
REI versus LPS satisfaction: MEDICINE VA Medical Centers, 2000-2010 1 0.9 0.8 0.7 0.6 proportion very satisfied p=.024 p=.70 emp lrn cln per 0.5 0.4 0.3 p=.2-1.6-1.4-1.2-1.0-0.8-0.6-0.4-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 REI score 43
REI versus LPS satisfaction: MEDICINE VA Medical Centers, 2000-2010 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 proportion very dissatisfied p=.31 p<.04 p=.025-1.6-1.4-1.2-1.0-0.8-0.6-0.4-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 emp lrn cln per REI score 44
REI versus LPS satisfaction: PSYCHIATRY VA Medical Centers, 2000-2010 1 0.9 0.8 0.7 0.6 proportion very satisfied p=.57 p=.006 emp lrn cln wrk 0.5 0.4 0.3 p<.001-1.6-1.4-1.2-1.0-0.8-0.6-0.4-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 REI score 45
REI versus LPS satisfaction: PSYCHIATRY VA Medical Centers, 2000-2010 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 proportion very dissatisfied p<.001 p<.004-1.6-1.4-1.2-1.0-0.8-0.6-0.4-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 emp lrn cln wrk REI score 46
Relevance for GME REI is viable assessing clinical workload opportunities for residents Residents make contributions to workload net of supervision varies by specialty and facility Surgery residents: large net contribution Medicine residents: moderate net contribution Ancillary residents: zero net contribution Psychiatry residents: negative net contribution 47
Relevance for GME Workload impacts satisfaction varies by domain: learning, large effect-size clinical, large effect-size personal large effect-size future employment modest effect-size varies by specialty: Surgery residents: very inversely related Medicine residents: moderately inversely related Psychiatry residents: positively related 48
Relevance for GME Workload impacts satisfaction varies by domain: learning, large effect-size clinical, large effect-size personal large effect-size future employment modest effect-size Most intensive contributor to workload varies by specialty: Surgery residents: very inversely related Medicine residents: moderately inversely related Psychiatry residents: positively related Least intensive contributor to workload 49
Questions? Veterans Health Administration Education Mission To educate for VA and for the Nation michael.kashner@va.gov barbara.chang@va.gov http://www.va.gov/oaa 50