Introduction to the MBSAQIP Semiannual Report (SAR)

Transcription

1 Introduction to the MBSAQIP Semiannual Report (SAR) Kristopher Huffman, MS Division of Research and Optimal Patient Care American College of Surgeons July 21, 2017

2 Disclosures I have nothing to disclose

3 What s a SAR? SAR = SEMIANNUAL REPORT Delivered 2 times a year (January and July) A report that provides risk and reliability-adjusted assessments of metabolic and bariatric surgical quality Serves as the foundation for the quality improvement in MBSAQIP

4 What s a SAR? General document to be used as a guide Provides an overview of the report Important updates Aggregate demographic Information Definitions for outcomes used to assess surgical quality Aggregate outcome rates and case volumes An overview of the statistical modeling process Glossary of statistical terms and concepts

5 What s a SAR? Contains assessments of site performance

6 What s a SAR? SAR SAR Document/Guide Site Summary Report Bar Plots Mortality Snapshots Case Occurrences Report Model Reports Case Details Report

7 What s a SAR? SAR Site Summary Report Identify areas for QI Case Occurrences Report Identify cases with occurrences Case Details Report Data Drill Down for QI

8 Risk Adjustment Site A 230 sleeves 8 morbidity events Morbidity Rate = 3.5% Site B 213 sleeves 8 morbidity events Morbidity Rate = 3.8% Without knowing anything about the patients at these sites, can we say Site A is performing better than Site B? Mean Age = 35 Mean Age = 55 5% of patients have BMI > 70 20% of patients have BMI > 70 15% of patients have hypertension 75% of patients have hypertension 5% of patients have type 2 diabetes 50% of patients have type 2 diabetes Site B s patients are sicker Level the playing field by giving credit to sites that are doing procedures on sicker patients.

9 All sites 107,041 sleeve cases Risk Adjustment SSI = Yes Every 10 year increase in age doubles risk Site A Age BMI Hypertension Diabetes Statistical Model BMI >70 triples risk compared to BMI Hypertension doubles risk 230 sleeve cases GERD Etc. Expected Rate for Site A SSI = No Performance Metric for Site A

10 Risk Adjustment Site A will have a worse performance evaluation than Site B Site A 230 sleeves 8 morbidity events Morbidity Rate = 3.5% Site B 213 sleeves 8 morbidity events Morbidity Rate = 3.8% Expected Rate = 1.3% Expected Rate = 3.2% Risk Adjusted Benchmarks Without risk-adjustment, we cannot fairly compare sites and would not be able to adequately respond to the claim: But we do riskier procedures on sicker patients.

11 Reliability Adjustment Shrinkage, Smoothing, Pooling, Empirical Bayes Adjustment, etc. Site A 5 cases 0 post-operative SSI Estimated SSI Rate = 0% This hospital looks extremely good Is it reasonable to believe the TRUE SSI Rate at this hospital is 0%? Site B 2 cases 1 post-operative death Estimated Death Rate = 50% This hospital looks extremely bad Is it reasonable to believe the TRUE Death Rate at this hospital is 50%?

12 Reliability Adjustment 5 cases, 0 SSI Estimated SSI rate = 0% Batting Average Based on 1 Game: 5 hits / 5 at bats Grand Average SSI Rate: 2% Do we Grand really Batting think Average: his true batting 250 (0.25) average is 1000? 2 cases, 1 death Estimated Death Rate = 50% Batting Average Based on 1 Game: 0 hits / 2 at bats Grand Average Death Rate: 1% Do we Grand really Batting think his Average: true batting 250 (0.25) average is 0?

13 Reliability Adjustment Hospital Naïve Evaluation Reliability Evaluation Hospital A Excellent Performance Average Performance Hospital B Poor Performance Average Performance The smaller the hospital volume, the more the performance metric will be shrunken towards average

14 Statistical Models Procedure Group Laparoscopic Sleeve Gastrectomy (LSG) Morbidity Outcome LSG Model Morbidity In this SAR 11 Outcomes 2 Procedure Groups 22 Models

15 Fruit Salad 452 centers Median number of cases per center = 4

16 Outcomes Morbidity All Occurrences Morbidity Related Reoperation All Cause Reoperation Related Intervention All Cause Intervention Related Readmission All Cause Readmission Anastomotic/Staple Line Leak Bleeding SSI

17 Two Different Morbidity Outcomes Both outcomes attempt to assess harm that occurred to a patient within 30 days of the procedure All Occurrences Morbidity = Kitchen Sink Morbidity Casts a wider net

18 Two Different Morbidity Outcomes All Occurrences Morbidity Morbidity All Occurrences Morbidity Rate Morbidity Rate

19 Two Different Morbidity Outcomes Definitions for all outcomes are on Pages 4-5