USING ACS NSQIP TO PROVIDE SURGEON SPECIFIC OUTCOMES

Transcription

1 USING ACS NSQIP TO PROVIDE SURGEON SPECIFIC OUTCOMES Rocco Ricciardi, MD, MPH Chief Scientific Officer Lahey Hospital & Medical Center Burlington, MA 01805

2 DISCLOSURES None

3 OBJECTIVES Surgeon specific reporting Data to use Has surgeon reporting helped

4 SURGEON SPECIFIC OUTCOMES Quality assessment and improvement Reporting in CR Surgery Biannual reports NSQIP outcomes* Institutional Data* SCIP measures *Similar variables *Which data? Variable 5 yr Mortality 2013 Mortality 1.3 ( ) 1.3% A 2.2% 1.3% B 1.1% 0.6% C 1.2% 1.6% D 1.3% 2.8% E 0.9% 0% F 1.7% 0.8% G 0.6% 1.9% H X 0.7%

5 NSQIP American College of Surgeons National Surgical Quality Improvement Program 1 Clinical data Reports 30-day risk-adjusted morbidity and mortality 1 Identifies risk adjusted outcomes at institutional, service level, and procedural level 2 Ideally suited for quality assessment and improvement Improved surgical outcomes in participating hospitals 3 Institute of Medicine named NSQIP Best In The Nation? Value for surgeon specific outcome reporting 1 Fink et al. Ann Surg 236, ; Cima et al. Surgery 150, , Hall et al. Ann Surg 250, , 2009.

6 INSTITUTIONAL DATA Prospective 100% capture data source Clinical data Reports 30-day morbidity and mortality outcomes Not risk-adjusted NSQIP outcomes Other department initiatives Includes inpatient and outpatient data Valuable in QI projects Time consuming and expensive Do we really need 100% capture data?

7 ASSESSMENT OF SURGEON OUTCOMES -Objective I- Compare NSQIP & Institutional Data Evaluate the comparability of NSQIP and 100% capture institutional quality data to provide individual surgeons with information regarding outcomes

8 METHODS Compared datasets for departmental & physician reporting 7 surgeons active for entire 5 year study period (1/2008 through 12/2012) NSQIP site-specific data Calculated aggregate departmental & surgeon-specific adverse event rates 1 and 5-year Institutional quality data Quality assessment data on all patients undergoing an operative procedure in CRS (100% capture) Calculated aggregate departmental & surgeon-specific adverse event rates 1 and 5-year Dis Colon Rectum 2015;58:247.

9 NSQIP DEFINITIONS NSQIP definitions of 30-day adverse events: Mortality Reoperation Urinary tract infection Deep vein thrombosis Pneumonia Superficial site infection Organ space infection Dis Colon Rectum 2015;58:247.

10 DATA ANALYSIS Aggregate departmental adverse event rates with 95% confidence intervals for each of seven complications for both datasets 1. Assess comparability of datasets Are the departmental/service line rates similar 2. Assess whether the two datasets categorize individual surgeons similarly as outliers for each adverse event Comparing surgeon rates with aggregate point estimates with 95% confidence intervals for each adverse event for each dataset Dis Colon Rectum 2015;58:247.

11 RESULTS: VOLUME/SURGEON CHARACTERISTICS Institutional Data Total aggregate cases over five years 6459 Inpatient 4173 (65%) Emergent 1297 (20%) Average annual case number per surgeon Range of annual case numbers per surgeon NSQIP Data Case volume sampled by NSQIP over five years 1786 Average number of annual cases sampled by NSQIP per surgeon 51 Range of annual case number per surgeon sampled by NSQIP % of case volume sampled by NSQIP 28% Surgeon A B C D E F G Aggregate % inpatient % emergent % of surgeon cases sampled by NSQIP Dis Colon Rectum 2015;58:247.

12 COMPARABILITY OF DATASETS Annual Departmental Results Calculated annual aggregate adverse event rates for entire group of surgeons for both datasets Annual 100% capture institutional adverse event rates were compared to the NSQIP 95% confidence intervals Mortality ROR UTI DVT PNM SSI OSI Key Institutional rate is within NSQIP 95% CI Institutional rate is below NSQIP 95% CI Institutional rate is above NSQIP 95% CI Dis Colon Rectum 2015;58:247.

13 SURGEON OUTLIER STATUS Compared individual surgeon outlier status for each adverse event category within the two datasets Concordance = two datasets classify an adverse event rate similarly Low/low Average/average High/high Discordance = two datasets disagree with one designating average Low/average Average/high Gross discordance Low/high

14 SURGEON RESULTS (ANNUAL) Compared annual outlier status determined by the two datasets for: 7 Surgeons x 7 Adverse event categories 5 Years 245 Comparisons (35 Comparisons per surgeon) Surgeon A B C D E F G Aggregate group % Concordance % Gross Discordance Dis Colon Rectum 2015;58:247.

15 SURGEON RESULTS (5-YEARS) Compared 5-year outlier status determined in each dataset for: x 7 Surgeons 7 Adverse event categories 49 Comparisons 29 agreement (59% concordance) 33% discordance 8% gross discordance Dis Colon Rectum 2015;58:247.

16 SURGEON RESULTS (5-YEARS) Individual surgeons Complication A B C D E F G 95% CI for Aggregate Mortality Institution NSQIP ROR Institution NSQIP UTI Institution NSQIP DVT Institution NSQIP Pneumonia Institution NSQIP SSI Institution NSQIP OSI Institution NSQIP Dis Colon Rectum 2015;58:247. Low outlier Average High outlier Gross discordance

17 SUMMARY Each surgeon could improve Good/bad years Agreement between NSQIP and 100% capture institutional data at the institutional/departmental level Overlap of results Poor concordance of data for individual surgeon outcomes Annual results poorest We needed the 100% capture data for surgeon reporting

18 SURGEON SPECIFIC REPORTING -Objective II- Are We Getting Better Evaluate the effect of surgeon feedback on Process measure adherence Overall surgical outcomes Process measures SCIP measures Removal of foley DVT prophylaxis Antibiotic administration Institutional measures Anastomotic technique Anastomotic leak testing Outcome measures UTI DVT Presented At ASCRS May 2015 Surgical Site Infection Anastomotic leak

19 PATIENT POPULATION p Value n 903(N)-2408(I) 669(N) 2694(I) 518(N)-2873(I) Age NS Male Sex 50% 51% 49% NS Laparoscopy 28.7% 27.7% 29.0% NS Hand-Assist 42% 38% 33% Presented At ASCRS May 2015

20 HAVE WE IMPROVED Variable p Value DC Urinary Catheter 73% 88% 100% 0.01# UTI 5.5% 4.2% 1.9% 0.01 DVT Prophylaxis 99% 100% 100% NS# DVT/PE 1.1% 1.3% 1.2% 0.5 Timely Abx 71% 96% 100% 0.01# SSI 5.3% 6.7% 4.8% 0.3 #SCIP data

21 HAVE WE IMPROVED Variable p Value Hand-Sewn Rights 19% 17% 2% * Leak Testing 88% 95% 95% * Diversion 19.5% 17% 17% NS Leak 5.2% 4.2% 2.9% 0.05* *Institutional data

22 SUMMARY Nearly 100% adherence to SCIP process measures of antibiotic administration did not lead to outcome gains for SSI Improvement in process of care for urinary catheter removal and leak testing were associated with improvements in UTI and overall leak Our strategy of biannual surgeon-specific feedback of outcomes with improved adherence to particular processes of care led to an overall improvement in surgical quality

23 CONCLUSIONS Biannual surgeon-specific feedback was valuable All surgeons were identified as a high outlier in at least one adverse event category Demonstrating the need for continuous quality improvement? Value of Hawthorne effect Providing process measures was helpful Estimates of surgeon-specific outcomes were critical but it is unclear how to best measure them Procedure specific Include larger samples Different outcomes