Disclosure. Public Reporting and Transparency of Outcomes Reporting in Pediatric Cardiac Surgery. Definition of Quality. Donabedian s Triad 10/1/2018

Transcription

1 Public Reporting and Transparency of Outcomes Reporting in Pediatric Cardiac Surgery Jeffrey P. Jacobs, MD Professor of Surgery and Pediatrics, Johns Hopkins University Director, Andrews/Daicoff Cardiovascular Program and Surgical Director of Heart Transplantation Johns Hopkins All Children s Hospital Disclosure Chair, STS Workforce on National Databases President-elect, Southern Thoracic Surgical Association Editor-In-Chief, Cardiology in the Young Co-Chair, World Congress of Pediatric Cardiology and Cardiac Surgery 2021 American Society of Extracorporeal Technology (AmSECT) 2018 Pediatric Perfusion Conference October 4-6, 2018 Miami EPIC Hotel Miami, FL Presented October 6, 2018, 10:30 AM 11:10 AM Definition of Quality Donabedian s Triad how good or bad something is a characteristic or feature that someone or something has : something that can be noticed as a part of a person or thing a high level of value or excellence [ Accessed November 10, 2015 Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(Suppl):

2 Michael Porter value defined as the health outcomes achieved per dollar spent Michael E. Porter, Ph.D. Perspective. What Is Value in Health Care? N Engl J Med 2010; 363: Basic Principles 1. Variation in outcomes exist Basic Principles 1. Variation in outcomes exist 2. Patients and their families have the right to know the outcomes of the treatments that they will receive. 2

3 Basic Principles 1. Variation in outcomes exist 2. Patients and their families have the right to know the outcomes of the treatments that they will receive. 3. It is our professional responsibility to share this information with them in a format that they can understand. Basic Principles The solution to risk aversive behavior is proper risk adjustment. The solution to fear of stifling innovation is proper risk adjustment. Our tools for public reporting are not perfect, but they are the BEST available (and these tools will improve).. Adjustment for Case Mix Differences in medical outcomes may result from 1. disease severity, 2. treatment effectiveness, or 3. chance. Adjustment for Case Mix Differences in medical outcomes may result from 1. disease severity (RISK ADJUSTMENT), 2. treatment effectiveness, or 3. chance (CONFIDENCE INTERVALS). Shahian DM, Blackstone EH, Edwards FH, Grover FL, Grunkemeier GL, Naftel DC, Nashef SA, Nugent WC, Peterson ED. STS workforce on evidence-based surgery. Cardiac surgery risk models: a position article. Ann Thorac Surg. 2004;78(5): Shahian DM, Blackstone EH, Edwards FH, Grover FL, Grunkemeier GL, Naftel DC, Nashef SA, Nugent WC, Peterson ED. STS workforce on evidence-based surgery. Cardiac surgery risk models: a position article. Ann Thorac Surg. 2004;78(5):

4 Adjustment for Case Mix Differences in medical outcomes may result from 1. disease severity (RISK ADJUSTMENT), 2. treatment effectiveness, or 3. chance (CONFIDENCE INTERVALS). Because most outcome studies are observational. risk adjustment is necessary to account for case mix Shahian DM, Blackstone EH, Edwards FH, Grover FL, Grunkemeier GL, Naftel DC, Nashef SA, Nugent WC, Peterson ED. STS workforce on evidence-based surgery. Cardiac surgery risk models: a position article. Ann Thorac Surg. 2004;78(5): ST TIME PUBLICLY REPORTED CABG Composite September 2010 AVR Composite January 2013 AVR + CABG Composite August / September 2014 Pediatric and Congenital Heart Surgery Risk Adjusted Mortality January 2015 Lobectomy Composite 2017 MVR Composite 2019 MVR + CABG Composite 2019 Esophagectomy Composite

5 Adult Cardiac Public Reporting Numbers Data Update Participants % Round Harvest 1 Jan Round Harvest 3 Oct / Nov Round Harvest 3 Jan Round Harvest 1 July Round Harvest 3 Jan Round Harvest 1 July Round Harvest 3 Jan Round Harvest 1 July Round Harvest 3 Jan /29/ / % 3/24/ / % 4/23/ / % 9/29/ / 1, % Current Numbers (8/6/2018) 704 / % Congenital Public Reporting Numbers Data Update Participants % 2014 Fall Round 1 January % Harvest 2015 Round 2 Spring Harvest August % 3/24/ / % 4/23/ / % 9/29/ / % Current Numbers (8/6/2018) 87 / % Participation in Public Reporting Participation in Public Reporting % Enrolled Unique STS consents / US & Canada participants (as of Friday, September 29, 2017) Adult Cardiac 59.9% 658 / 1,098 Congenital 66.6% 78 / 117 Thoracic 18.1% 52 / 287 Adult Cardiac Congenital Thoracic % Enrolled Unique STS consents / US & Canada participants (as of August 6, 2018) 65.0% 704 / % 87 / % 82 / 293 5

6 Star Rating THE BEGINNING OF STS PUBLIC REPORTING STS CABG Composite Score 1) On September 7, 2010, Consumers Union (publisher of Consumer Reports) reported the results of coronary artery bypass grafting (CABG) procedures at 221 U.S. cardiac surgery programs. 2) On January 26, 2011, STS reported the results of coronary artery bypass grafting (CABG) procedures at 226 U.S. cardiac surgery programs 99% Bayesian probability that provider differs from STS average STS CABG Composite Score STS CABG Composite Score Star Ratings Over Time Spring 2010 Fall 2010 Spring 2011 Fall 2011 Spring 2012 Fall 2012 Spring 2013 Fall % 10.5% 11% 10.9% 9.6% 9% 9% 9.7% % 76.1% 75.5% 74.8% 76.5% 77% 76% 76.9% 99% Bayesian probability that provider differs from STS average 99% Bayesian probability that provider differs from STS average % 13.4% 13.5% 14.4% 14% 14% 15% 13.4% 6

7 The New England Journal of Medicine The New England Journal of Medicine STS AVR Composite Score STS AVR Composite Score Composite measure with 2 domains: 1. Absence of operative mortality 2. Absence of major morbidity No process measures For AVR there are currently no widely accepted process measures as there are for CABG, i.e., medication measures and IMA use Note: Transcatheter Aortic Valve procedures are not included in the AVR Composite Score 97.5% Bayesian probability that provider differs from STS average No good analogue for IMA use and meds Fewer cases (3 years of data) 7

8 Star Rating Star Rating STS AVR Composite Score STS AVR + CABG Composite Score Fall 2012 Spring 2013 Fall % 3% 2.7% 2 91% 91% 91.4% 3 6% 6% 5.9% 97.5% Bayesian probability that provider differs from STS average Same 2 domains as AVR Composite Score STS AVR + CABG Composite Score Developing Risk Models: Fall % When Risk Models were developed by STS for Isolated CABG: the STS-ACSD included > 50,000 Isolated CAB procedures per year % 3 6.4% In a recent one year period in STS-CHSD: VSD closure: n = % of all Arterial Switch: n = % of all Norwood: n = % of all Same 2 domains as AVR Composite Score More than 150 unique procedure codes 8

9 Congenital Heart Disease Meaningful Multi-institutional Outcomes Analysis Accomplishments Building Blocks Towards Transparency Jeffrey P. Jacobs, M.D., FACS, FACC, FCCP Professor of Surgery and Pediatrics, Johns Hopkins University Director, Andrews/Daicoff Cardiovascular Program Surgical Director of Heart Transplantation Johns Hopkins All Children s Heart Institute Johns Hopkins All Children s Hospital and Florida Hospital for Children 1) Common Language = Nomenclature 2) Mechanism of Data Collection (Database - Registry) 3) Mechanism of Evaluating Case Complexity 4) Mechanism to Verify Data Validity and Accuracy 5) Collaboration Between Subspecialties 6) Longitudinal Follow-Up and Linked Databases 7) Quality Improvement Barach P, Jacobs JP, Lipshultz SE, Laussen P. (Eds.). Pediatric and Congenital Cardiac Care - Volume 1: Outcomes Analysis. Springer-Verlag London. Pages ISBN: (Print) (Online). Published in Congenital Heart Disease Meaningful Multi-institutional Outcomes Analysis Accomplishments 1) Common Language = Nomenclature 2) Mechanism of Data Collection (Database - Registry) 3) Mechanism of Evaluating Case Complexity 4) Mechanism to Verify Data Validity and Accuracy 5) Collaboration Between Subspecialties 6) Longitudinal Follow-Up and Linked Databases 7) Quality Improvement Barach P, Jacobs JP, Lipshultz SE, Laussen P. (Eds.). Pediatric and Congenital Cardiac Care - Volume 1: Outcomes Analysis. Springer-Verlag London. Pages ISBN: (Print) (Online). Published in Congenital Heart Disease Meaningful Multi-institutional Outcomes Analysis Accomplishments 1) Common Language = Nomenclature 2) Mechanism of Data Collection (Database - Registry) 3) Mechanism of Evaluating Case Complexity 4) Mechanism to Verify Data Validity and Accuracy 5) Collaboration Between Subspecialties 6) Longitudinal Follow-Up and Linked Databases 7) Quality Improvement Barach P, Jacobs JP, Lipshultz SE, Laussen P. (Eds.). Pediatric and Congenital Cardiac Care - Volume 1: Outcomes Analysis. Springer-Verlag London. Pages ISBN: (Print) (Online). Published in

10 The Report of the 2015 STS Congenital Heart Surgery Practice Survey undertaken by the Society of Thoracic Surgeons Workforce on Congenital Heart Surgery 125 centers in the United States of America perform pediatric and congenital heart surgery 8 centers in Canada perform pediatric and congenital heart surgery Morales DL, Khan MS, Turek JW, Biniwale R, Tchervenkov CI, Rush M, Jacobs JP, Tweddell JS, Jacobs ML. Report of the 2015 Society of Thoracic Surgeons Congenital Heart Surgery Practice Survey. Ann Thorac Surg Feb;103(2): doi: /j.athoracsur Epub 2016 Aug 20. PMID: Growth in the STS Congenital Heart Surgery Database Participating Centers Per Harvest Participating Centers Jacobs JP, Mayer, Jr. JE, Pasquali SK, Subramanyan RK, MD, PhD. Executive Summary: The Society of Thoracic Surgeons Congenital Heart Surgery Database Twenty-eighth Harvest (January 1, 2014 December 31, 2017). The Society of Thoracic Surgeons (STS) and Duke Clinical Research Institute (DCRI), Duke University Medical Center, Durham, North Carolina, United States, Spring 2018 Harvest. Growth in the STS Congenital Heart Surgery Database Operations per averaged 4 year data collection cycle Growth in the STS Congenital Heart Surgery Database Cumulative operations over time 180, , , , , , , , , , , , , , , , , , , , ,000 80,000 60,000 40,000 20, ,787 16,461 28,351 37,093 45,635 61,014 72,002 91, Operations 12,787 16,461 28,351 37,093 45,635 61,014 72,002 91, , , , , , , , , ,978 Jacobs JP, Mayer, Jr. JE, Pasquali SK, Subramanyan RK, MD, PhD. Executive Summary: The Society of Thoracic Surgeons Congenital Heart Surgery Database Twenty-eighth Harvest (January 1, 2014 December 31, 2017). The Society of Thoracic Surgeons (STS) and Duke Clinical Research Institute (DCRI), Duke University Medical Center, Durham, North Carolina, United States, Spring 2018 Harvest. 103, , , , , , ,000 50, ,237 9,747 16,537 26,404 39,988 58,181 79,399 98, , Cumulative Operations 4,237 9,747 16,537 26,404 39,988 58,181 79,399 98, , , , , , , , , , , ,228 Jacobs JP, Mayer, Jr. JE, Pasquali SK, Subramanyan RK, MD, PhD. Executive Summary: The Society of Thoracic Surgeons Congenital Heart Surgery Database Twenty-eighth Harvest (January 1, 2014 December 31, 2017). The Society of Thoracic Surgeons (STS) and Duke Clinical Research Institute (DCRI), Duke University Medical Center, Durham, North Carolina, United States, Spring 2018 Harvest. 148, , , , ,828 10

11 STS CHSD: Penetrance in USA The STS Congenital Heart Surgery Database (STS-CHSD) is the largest clinical database in the world for congenital and pediatric cardiac surgery. The Report of the 2015 Society of Thoracic Surgeons Congenital Heart Surgery Practice Survey, undertaken by the STS Workforce on Congenital Heart Surgery, estimated that 125 hospitals in the United States of America and 8 hospitals in Canada perform pediatric and congenital heart surgery. The STS-CHSD contains data from 126 hospitals (close to 100% penetrance by hospital) in the United States of America and 3 of the 8 centers in Canada. STS CHSD: Penetrance in USA The STS Congenital Heart Surgery Database (STS-CHSD) is the largest clinical database in the world for congenital and pediatric cardiac surgery. The Report of the 2015 Society of Thoracic Surgeons Congenital Heart Surgery Practice Survey, undertaken by the STS Workforce on Congenital Heart Surgery, estimated that 125 hospitals in the United States of America and 8 hospitals in Canada perform pediatric and congenital heart surgery. The STS-CHSD contains data from 126 hospitals (close to 100% penetrance by hospital) in the United States of America and 3 of the 8 centers in Canada. REPRESENTATIVE Congenital Heart Disease Meaningful Multi-institutional Outcomes Analysis Accomplishments 1) Common Language = Nomenclature 2) Mechanism of Data Collection (Database - Registry) 3) Mechanism of Evaluating Case Complexity 4) Mechanism to Verify Data Validity and Accuracy 5) Collaboration Between Subspecialties 6) Longitudinal Follow-Up and Linked Databases 7) Quality Improvement Barach P, Jacobs JP, Lipshultz SE, Laussen P. (Eds.). Pediatric and Congenital Cardiac Care - Volume 1: Outcomes Analysis. Springer-Verlag London. Pages ISBN: (Print) (Online). Published in Risk stratification Risk stratification is a method of analysis in which the data are divided into relatively homogeneous groups (called strata). 11

12 Risk stratification Two traditional methodologies for Risk Stratification The Aristotle Basic Complexity Levels (ABC Levels) 2002 The Risk Adjustment for Congenital Heart Surgery Categories (RACHS-1) 2006 The STS-EACTS Mortality Categories (STAT Mortality Categories) ) Risk Adjustment in Congenital Heart Surgery-1 (RACHS-1 ) 2) Aristotle Complexity Score Aristotle Basic Complexity Score (ABC Score) Aristotle Comprehensive Complexity Score Jacobs JP, Jacobs ML, Lacour-Gayet FG, Jenkins KJ, Gauvreau K, Bacha EA, Maruszewski B, Clarke DR, Tchervenkov CI, Gaynor JW, Spray, TL, Stellin G, O'Brien SM, Elliott MJ, Mavroudis C. Stratification of Complexity Improves Utility and Accuracy of Outcomes Analysis in a Multi-institutional Congenital Heart Surgery Database Application of the RACHS-1 and Aristotle Systems in the STS Congenital Heart Surgery Database. Pediatric Cardiology, 2009, DOI /s STS 2006 Congenital Heart Surgery Database 45,635 cases STS 2006 Congenital Heart Surgery Database 45,635 cases % Mortality 4 % Mortality & % Mortality RACHS-1 Category % Mortality Aristotle Basic Level Jacobs JP, Jacobs ML, Lacour-Gayet FG, Jenkins KJ, Gauvreau K, Bacha EA, Maruszewski B, Clarke DR, Tchervenkov CI, Gaynor JW, Spray, TL, Stellin G, O'Brien SM, Elliott MJ, Mavroudis C. Stratification of Complexity Improves Utility and Accuracy of Outcomes Analysis in a Multi-institutional Congenital Heart Surgery Database Application of the RACHS-1 and Aristotle Systems in the STS Congenital Heart Surgery Database. Pediatric Cardiology, 2009, DOI /s Jacobs JP, Jacobs ML, Lacour-Gayet FG, Jenkins KJ, Gauvreau K, Bacha EA, Maruszewski B, Clarke DR, Tchervenkov CI, Gaynor JW, Spray, TL, Stellin G, O'Brien SM, Elliott MJ, Mavroudis C. Stratification of Complexity Improves Utility and Accuracy of Outcomes Analysis in a Multi-institutional Congenital Heart Surgery Database Application of the RACHS-1 and Aristotle Systems in the STS Congenital Heart Surgery Database. Pediatric Cardiology, 2009, DOI /s

13 From Subjective Probability to Objective Data STAT Mortality Score The Society of Thoracic Surgeons - European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Score and STAT Mortality Categories The Society of Thoracic Surgeons - European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Categories O'Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour-Gayet FG, Pizarro CP, Welke KF, Maruszewski B, Tobota Z, Miller WJ, Hamilton L, Peterson ED, Mavroudis C, Edwards FH. An empirically based tool for analyzing mortality associated with congenital heart surgery. The Journal of Thoracic and Cardiovascular Surgery, 2009 Nov;138(5), November STAT Mortality Categories STAT Mortality Score and Categories were developed based on analysis of 77,294 operations entered in the STS Congenital Heart Surgery Databases and the EACTS Congenital Heart Surgery Database EACTS = 33,360 operations STS = 43,934 operations O'Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour-Gayet FG, Pizarro CP, Welke KF, Maruszewski B, Tobota Z, Miller WJ, Hamilton L, Peterson ED, Mavroudis C, Edwards FH. An empirically based tool for analyzing mortality associated with congenital heart surgery. The Journal of Thoracic and Cardiovascular Surgery, 2009 Nov;138(5), November STAT Mortality Categories Procedure-specific mortality rate estimates were calculated using a Bayesian model that adjusted for small denominators. STAT Mortality Categories Operations were sorted by increasing risk and grouped into 5 categories that were designed to minimize within-category variation and maximize between-category variation O'Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour-Gayet FG, Pizarro CP, Welke KF, Maruszewski B, Tobota Z, Miller WJ, Hamilton L, Peterson ED, Mavroudis C, Edwards FH. An empirically based tool for analyzing mortality associated with congenital heart surgery. The Journal of Thoracic and Cardiovascular Surgery, 2009 Nov;138(5), November O'Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour-Gayet FG, Pizarro CP, Welke KF, Maruszewski B, Tobota Z, Miller WJ, Hamilton L, Peterson ED, Mavroudis C, Edwards FH. An empirically based tool for analyzing mortality associated with congenital heart surgery. The Journal of Thoracic and Cardiovascular Surgery, 2009 Nov;138(5), November

14 Combined ECHSA/EACTS and STS Congenital Heart Surgery Databases: 111,494 index cardiac operations % Mortality STAT Category % Mortality Jacobs JP, Jacobs ML, Maruszewski B, Lacour-Gayet FG, Tchervenkov CI, Tobota Z, Stellin G, Kurosawa H, Murakami A, Gaynor JW, Pasquali SK, Clarke DR, Austin EH 3rd, Mavroudis C. Initial application in the EACTS and STS Congenital Heart Surgery Databases of an empirically derived methodology of complexity adjustment to evaluate surgical case mix and results. Eur J Cardiothorac Surg Nov;42(5): doi: /ejcts/ezs026. Epub 2012 Jun 14. PMID: Variable Age a Primary procedure b Weight (neonates and infants) Prior cardiothoracic operation Any non-cardiac congenital anatomic abnormality (except Other noncardiac congenital abnormality with code value = 990) Any chromosomal abnormality or syndrome (except Other chromosomal abnormality with code value = 310 and except Other syndromic abnormality with code value = 510) Prematurity (neonates and infants) Preoperative Factors Preoperative/Preprocedural mechanical circulatory support (IABP, VAD, ECMO, or CPS) c Shock, Persistent at time of surgery Mechanical ventilation to treat cardiorespiratory failure Renal failure requiring dialysis and/or Renal dysfunction Preoperative neurological deficit Any other preoperative factor (except Other preoperative factors with code value = 777) d a Modeled as a piecewise linear function with separate intercepts and slopes for each STS-defined age group (neonate, infant, child, adult). b The model adjusts for each combination of primary procedure and age group. Coefficients obtained via shrinkage estimation with The Society of Thoracic Surgeons European Association for Cardio-Thoracic Surgery (STS-EACTS [STAT]) Mortality Category as an auxiliary variable. c CPS = cardiopulmonary support; ECMO =extracorporeal membrane oxygenation; IABP = intraaortic balloon pump; VAD = ventricular assist device d Any other preoperative factor is defined as any of the other specified preoperative factors contained in the list of preoperative factors in the data collection form of the STS Congenital Heart Surgery Database, exclusive of 777 = Other preoperative factors. All index cardiac operations in the STS-CHSD (January 1, 2010 December 31, 2013) were eligible for inclusion. Isolated PDA closures in patients <2.5kg were excluded, as were centers with >10% missing data and patients with missing data for key variables. 52,224 operations from 86 centers were included 14

15 Model Covariates Development Validation Model Covariates Development Validation 1 STAT Levels C = C = STAT Levels + C = C = STAT Levels C = C = STAT Levels + C = C = STAT Levels + + C = C = STAT Levels + + C = C = Primary procedure C = C = Primary procedure C = C = (Final Model) Primary procedure + + C = C = (Final Model) Primary procedure + + C = C = Model Covariates Development Validation Model Covariates Development Validation 1 STAT Levels C = C = STAT Levels + C = C = STAT Levels C = C = STAT Levels + C = C = STAT Levels + + C = C = STAT Levels + + C = C = Primary procedure C = C = Primary procedure C = C = (Final Model) Primary procedure + + C = C = (Final Model) Primary procedure + + C = C =

16 Model Covariates Development Validation Model Covariates Development Validation 1 STAT Levels C = C = STAT Levels + C = C = STAT Levels C = C = STAT Levels + C = C = STAT Levels + + C = C = STAT Levels + + C = C = Primary procedure C = C = Primary procedure C = C = (Final Model) Primary procedure + + C = C = (Final Model) Primary procedure + + C = C = Fig 1. Distribution of hospital-specific observed-to-expected (O/E) ratios for operative mortality with 95% confidence intervals (gray lines). 63 Fig 1. Distribution of hospital-specific observed-to-expected (O/E) ratios for operative mortality with 95% confidence intervals (gray lines)

17 Fig 1. Distribution of hospital-specific observed-to-expected (O/E) ratios for operative mortality with 95% confidence intervals (gray lines). 65 Fig 1. Distribution of hospital-specific observed-to-expected (O/E) ratios for operative mortality with 95% confidence intervals (gray lines). 66 What about Confidence Intervals Total Programs higher-than expected mortality same-as expected mortality lower-than expected mortality 80% Confidence Intervals Number (%) Number (%) Number (%) Number (%) 86 (100%) 19 (22%) 52 (60%) 15 (17%) 90% Confidence Intervals 86 (100%) 13 (15%) 63 (73%) 10 (12%) Fig 1. Distribution of hospital-specific observed-to-expected (O/E) ratios for operative mortality with 95% confidence intervals (gray lines) % Confidence Intervals 86 (100%) 12 (14%) 67 (78%) 7 (8%) 99% Confidence Intervals 86 (100%) 6 (7%) 78 (91%) 2 (2%) 68 17

18 What about Confidence Intervals What about Confidence Intervals Total Programs higher-than expected mortality same-as expected mortality lower-than expected mortality Total Programs higher-than expected mortality same-as expected mortality lower-than expected mortality Number (%) Number (%) Number (%) Number (%) 86 (100%) 19 (22%) 52 (60%) 15 (17%) Number (%) Number (%) Number (%) Number (%) 86 (100%) 19 (22%) 52 (60%) 15 (17%) 80% Confidence Intervals 80% Confidence Intervals 90% Confidence Intervals 86 (100%) 13 (15%) 63 (73%) 10 (12%) 90% Confidence Intervals 86 (100%) 13 (15%) 63 (73%) 10 (12%) 95% Confidence Intervals 86 (100%) 12 (14%) 67 (78%) 7 (8%) 69 95% Confidence Intervals 86 (100%) 12 (14%) 67 (78%) 7 (8%) 70 99% Confidence Intervals 86 (100%) 6 (7%) 78 (91%) 2 (2%) 99% Confidence Intervals 86 (100%) 6 (7%) 78 (91%) 2 (2%) What about Confidence Intervals What about Confidence Intervals Total Programs higher-than expected mortality same-as expected mortality lower-than expected mortality Total Programs higher-than expected mortality same-as expected mortality lower-than expected mortality Number (%) Number (%) Number (%) Number (%) 86 (100%) 19 (22%) 52 (60%) 15 (17%) Number (%) Number (%) Number (%) Number (%) 86 (100%) 19 (22%) 52 (60%) 15 (17%) 80% Confidence Intervals 80% Confidence Intervals 90% Confidence Intervals 86 (100%) 13 (15%) 63 (73%) 10 (12%) 90% Confidence Intervals 86 (100%) 13 (15%) 63 (73%) 10 (12%) 95% Confidence Intervals 86 (100%) 12 (14%) 67 (78%) 7 (8%) 71 95% Confidence Intervals 86 (100%) 12 (14%) 67 (78%) 7 (8%) 72 99% Confidence Intervals 86 (100%) 6 (7%) 78 (91%) 2 (2%) 99% Confidence Intervals 86 (100%) 6 (7%) 78 (91%) 2 (2%) 18

19 What about Confidence Intervals higher-than same-as lower-than expected expected expected mortality mortality mortality Total Programs Number (%) Number (%) Number (%) Number (%) 86 (100%) 19 (22%) 52 (60%) 15 (17%) 80% Confidence Intervals STS Congenital Heart Surgery Database Participants January 1, 2011 to December 31, 2014 One Star Programs = 11 Two Star Programs = 79 Three Star Programs = 6 No Star Rating = 20 90% Confidence Intervals 86 (100%) 13 (15%) 63 (73%) 10 (12%) 95% Confidence Intervals 86 (100%) 12 (14%) 67 (78%) 7 (8%) 73 99% Confidence Intervals 86 (100%) 6 (7%) 78 (91%) 2 (2%) STS Congenital Heart Surgery Database Participants January 1, 2011 to December 31, 2014 One Star Programs = ZERO out of 11 Two Star Programs = 27 out of 79 Three Star Programs = 5 out of 6 No Star Rating = 20 STS Congenital Heart Surgery Database Participants January 1, 2014 to December 31, 2017 One Star Programs = 13 Two Star Programs = 85 Three Star Programs = 12 No Star Rating = 9 19

20 STS Congenital Heart Surgery Database Participants January 1, 2014 to December 31, 2017 One Star Programs = 10 out of 13 Two Star Programs = 63 out of 85 Three Star Programs = 12 out of 12 No Star Rating = 9 Science tells us what we can do; Guidelines what we should do; & Registries what we are actually doing. Lukas Kappenberger MD Heart Rhythm Society Policy Conference Washing ton DC 2005 Outcomes Analysis Quality Improvement Patient Safety 20