Supplementary Appendix This appendix has been provided by the authors to give readers additional information about their work. Supplement to: Sutton M, Nikolova S, Boaden R, Lester H, McDonald R, Roland M. Reduced mortality with hospital pay for performance in England. N Engl J Med 2012;367:1821-8. DOI: 10.1056/NEJMsa1114951
Association between Mortality and Hospital Pay for Performance in England Supplementary Material Table of Contents A.1. List of investigators... 2 A.2. Diagnostic codes used to identify patients with the incentivized conditions... 3 A.3. Risk adjustment method... 3 A.4. Regression methods... 4 A.5. Quarterly changes in mortality in the North West and the rest of England... 6 A.6. Pre-trends tests... 6 A.7. Robustness tests... 7 A.8. Analysis of changes in discharges to care homes... 8 A.9. Analysis of mortality changes by type of hospital... 8 A.10. Participating hospitals achievements on the process quality measures... 9 A.11. Figures... 10 A.12. Tables... 12 1
A.1. List of investigators Matt Sutton, Centre for Health Economics, Institute of Population Health, University of Manchester, UK Silviya Nikolova, Centre for Health Economics, Institute of Population Health, University of Manchester, UK Ruth Boaden, Manchester Business School, University of Manchester, UK Helen Lester, Primary Care Clinical Sciences, University of Birmingham, UK Ruth McDonald, Business School, University of Nottingham, UK Martin Roland, Cambridge Centre for Health Services Research, University of Cambridge, UK 2
A.2. Diagnostic codes used to identify patients with the incentivized conditions We used the rules specified for the P4P scheme to identify patients with the three incentivized conditions. For all conditions, patients were only included if they were aged 18 years or over. For acute myocardial infarction, patients are included if they have a primary diagnosis of I21.0, I21.1, I21.2, I21.3, I21.4, I21.9, I22.0, I22.1, I22.8 or I22.9. For heart failure, patients are included if they have a primary diagnosis of I11.0, I13.0, I13.2, I50.0, I50.1 or I50.9 and did not have a Left Ventricular Assistive Device (LVAD) or Heart Transplant procedure (OPCS codes K01.1, K01.2, K01.8, K01.9, K02.1, K02.2, K02.3, K02.4, K02.5, K02.6, K02.8, K02.9, K54.1, K54.2, K54.8 or K54.9). For pneumonia, patients are included if they have a primary diagnosis of J13, J14, J15, J16.0, J16.8, J18.0, J18.1, J18.2, J18.8 or J18.9, or if they have a primary diagnosis of A40.0, A40.1, A40.2, A40.3, A40.8, A40.9, A41.0, A41.1, A41.2, A41.3, A41.4, A41.5, A41.8, A41.9, J96.0 or J96.2 and a secondary diagnosis included in the primary pneumonia list. A.3. Risk adjustment method We adjust the observed mortality rates using expected mortality rates that are obtained from a patient-level logistic regression model. For each health condition, the binary outcome at patient level is regressed on vectors of binary variables for gender and age-group interaction dummy variables (where age is measured in five-year bands), the first four digits of the full primary ICD-10 diagnosis code, 31 Elixhauser conditions as co-morbidities, 1 the admission method and the admission source. The predicted probabilities from these patient level models are then averaged across strata defined by quarter of admission and admitting hospital. For heart failure, we have not controlled for Elixhauser congestive heart failure codes as these diagnoses are included in the primary diagnoses for one of the incentivized conditions. For acute myocardial infarction, we do not control for arrhythmia codes, as the incentive scheme is specifically designed to prevent these. In principle, we might have wished to control for pre-existing arrythmias, but we cannot distinguish between pre-existing arrythmias and those which occurred while in hospital, so we excluded arrythmias in the main analysis. However, we provide an analysis with arrythmias included as a sensitivity analysis. Inclusion of arrhythmias in the risk adjustment 1 Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, Saunders LD, Beck CA, Feasby TE, Ghali WA. Coding Algorithms for Defining Comorbidities in ICD-9-CM and ICD-10 Administrative Data. Medical Care 2005; 43(11): 1130-9. 3
produces a triple-difference estimate equal to -0.64 [95% CI -1.68-0.40]. Exclusion of arrhythmias produces a triple-difference estimate equal to -0.63 [95% CI -1.67-0.41]. A.4. Regression methods For each outcome and for each health condition group, we present the results of three regression models: (i) a difference-in-differences analysis between the North West and the rest of England on the incentivised conditions; (ii) a difference-in-differences analysis between the incentivised and unincentivised conditions in the North West of England; and (iii) a triple-difference analysis between the North West and the rest of England and between the incentivised and unincentivised conditions. All of the regression analyses are undertaken at hospital level, are weighted for the number of patients and use standard errors corrected for heteroskedasticity. Between-region difference-in-differences analysis between the North West and the rest of England for each condition The first model is a difference-in-differences (DiD) estimation comparing the North West with the rest of England. The outcome, y jt, for Trust j in quarter t is the percentage of patients that died within 30 days minus the percentage of patients that are expected to die based on the patient-level risk equations. This outcome is modelled as a function of Trust fixed effects (u j ) and time fixed effects (v t ) and a random error term with zero mean ( ). We define a variable which takes a value of 1 if the provider Trust is in the North West (NW) of England. We further create a variable which equals 1 if the observation belongs to the period after the start of the AQ program. The interaction of and identifies NW trusts after the introduction of AQ. We estimate the following equation on the sample of patients with each condition:. The main effects of are omitted as they are perfectly collinear with the Trust fixed effects (u j ). The main effects of are omitted as they are perfectly collinear with the time fixed effects (v t ). 4
The time fixed effects are quarters and are captured by eleven dummy variables, with the first quarter (April-June 2007) being the reference category. The coefficients on these dummy variables reveal how the mean value of the outcome changes over time, conditional on the Trust effects. Our particular interest lies in the impact of the programme on mortality for the three incentivized conditions under the AQ program and mortality for the selected group of control conditions. The estimate on the coefficient shows how the mean risk-adjusted outcome differs between the AQ and the non-aq Trusts in the last six quarters, conditional on the Trust effects and the time effects. This difference-in-differences estimate of the effect of the AQ program could be confounded by any other factor that impacts overall mortality in the North West and the rest of England differently over the period of our analysis. Within-region difference-in-differences analysis between the incentivized and unincentivized conditions in the North West of England We define a third dummy variable which takes a value of 1 for the AQ-qualifying conditions and a value of 0 for reference conditions. We then estimate the following equation on data from hospitals in the North West of England:. As before, the main effects of are omitted as they are perfectly collinear with the time fixed effects (v t ). Triple-difference analysis between the North West and the rest of England and between the incentivized and unincentivized conditions Using all of the data, we estimate the following regression model:.... 5
where is the coefficient of interest. It captures the effect of the AQ program on in-hospital mortality for the AQ conditions in the North West having netted out the effects of changes over time in mortality for AQ conditions due to factors other than the initiative itself, changes over time in overall mortality in the North West and differences in mortality between the AQ and reference conditions between the North West and the rest of England. A.5. Quarterly changes in mortality in the North West and the rest of England Figures S1 to S4 show the risk-adjusted mortality rates for the non-incentivized and the three incentivized conditions by quarter for the North West and the rest of England. A.6. Pre-trends tests We tested whether the risk-adjusted mortality rates in the North West of England had a different trend to those in the rest of England prior to the introduction of the program using a pre-trends test. Using data from before the introduction of the program, we estimated the following regression model for each condition:... in which t represents the quarter since the start of the data period, is an estimate of the quarterly trend in the rest of England and is the difference between the quarterly trend in the North West of England and the quarterly trend in the rest of England. We were able to accept the null hypothesis of equal pre-trends for each condition. The estimated values for were as follows: acute myocardial infarction -0.39 (95% CI -1.00-0.22; P=0.21); heart failure 0.28 (95% CI -0.45-1.02; P=0.45); pneumonia -0.13 (95% CI -0.72-0.46; P=0.66); non-incentivized conditions -0.57 (95% CI -1.15-0.02; P=0.06). 6
A.7. Robustness tests In addition to the pre-trends tests, we have undertaken three forms of robustness tests. First, to examine whether our results are confounded by differential changes in the volume of patients treated with the incentivized and non-incentivized conditions we include the volume of patients treated as an additional regressor. This volume measure is the natural logarithm of the number of cases at each hospital in each quarter. The effects on our results are shown in Table S1. We found that volume was only statistically significant (Coefficient = -1.16 (95% CI -1.85--0.47; P<0.01) in the model for acute myocardial infarction mortality and that the impact results reported in the main analysis were unaffected. Second, we have replaced the hospital fixed-effects with measures of average risk-adjusted mortality at each hospital for each condition in the period prior to the introduction of the program. The results are shown in Table S1. For acute myocardial infarction and pneumonia, replacement of the hospital fixed effects with baseline performance results in smaller between-region difference-in-differences estimates but the pneumonia result remains statistically significant. For heart failure, the betweenregion difference-in-differences estimate become positive and remains not statistically significant. For heart failure and pneumonia, replacement of the hospital fixed-effects with baseline performance reduces the estimated magnitude of the triple-difference estimates by 0.2 percentage points but the pneumonia results remains statistically significant. For acute myocardial infarction, the tripledifference estimate increases by 0.2 percentage points and becomes just statistically significant (Coefficient = -0.8 (95% CI -1.5, 0.0; P=0.049)). Third, we have restricted our group of control hospitals to hospitals in northern regions of England, which also have higher than average mortality rates. We excluded hospitals in the following four Strategic Health Authorities: South East Coast; London; South Central and South West. The riskadjusted mortality rate for the whole of the rest of England decreased from 18.5% in the first 18 months to 17.5% in the second 18 months. For the control hospitals in the northern regions of England, the risk-adjusted mortality rate was higher in the first 18 months at 19.2% and decreased by a similar amount, to 18.2%. The regression results using this restricted control group are shown in Table S1. Most of the results are affected by no more than 0.1 percentage points. The between-region difference-in-differences estimate for heart failure reduces by 0.3 percentage points to 0.0 (95% CI - 1.0-0.9). The triple-difference estimate for pneumonia increases in magnitude by 0.4 percentage points to -2.3 (95% CI -3.4--1.1). 7
A.8. Analysis of changes in discharges to care homes Our measure of mortality captures patients dying within 30 days of admission during the initial hospital stay or in a subsequent stay in any other hospital in England. It is possible, therefore, that this measure omits deaths of patients that are discharged to non-hospital settings such as hospices and care homes. We examined whether there were differential changes in rates of discharge to one of the following destinations: NHS run nursing home, residential care home or group home Local authority residential accommodation Non-NHS run hospital Non-NHS (other than local authority) run residential care home Non-NHS (other than Local Authority) run nursing home Non-NHS (other than Local Authority) run hospice Non-NHS institution We calculated risk-adjusted rates of discharges to these care settings using the same method as for mortality and we tested for differential effects using the same difference-in-differences and tripledifferences frameworks. All of the estimates were smaller than 0.3 percentage points and none was statistically significant. We can therefore rule out the possibility that changes in discharge destinations could account for the magnitudes of the changes in mortality that we have estimated. A.9. Analysis of mortality changes by type of hospital We examined whether the mortality reductions were concentrated in particular types of hospitals in the North West by calculating the changes in risk-adjusted mortality between the two 18-month periods before and after the scheme s introduction for each hospital and regressing these changes on indicators for the type of hospital and interactions between each hospital type and the North West region. 8
Our examination of mortality changes by hospital type showed that none of the mortality changes for specific types of hospitals was statistically different in the North West compared to the rest of England for acute myocardial infarction or heart failure, but most were for pneumonia (Table S2). The magnitudes of the mortality changes for pneumonia did not differ by whether the hospital had been granted additional managerial and financial freedoms as a Foundation Trust or by the rating of their financial management by the national regulator. The additional mortality reductions for pneumonia were largest for small hospitals. Hospitals rated as fair or weak for the quality of their services by the national regulator in 2007 did not reduce pneumonia mortality more than similar hospitals in other parts of England, but hospitals rated as excellent or good for the quality of their services in the North West of England had larger mortality reductions than similarly-rated hospitals in other parts of England. A.10. Participating hospitals achievements on the process quality measures Mean achievements across hospitals on each of the quality measures at the start and end of the period are shown in Table S3. 9
A.11. Figures Figure S1: Risk-adjusted mortality for non-incentivized conditions by quarter RoE NW Difference from expected mortality rate 4 3 2 1 0-1 -2-3 Figure S2: Risk-adjusted mortality for acute myocardial infarction by quarter RoE NW Difference from expected mortality rate 3 2.5 2 1.5 1 0.5 0-0.5-1 -1.5 10
Figure S3: Risk-adjusted mortality for heart failure by quarter RoE NW Difference from expected mortality rate 5 4 3 2 1 0-1 -2-3 Figure S4: Risk-adjusted mortality for pneumonia by quarter RoE NW Difference from expected mortality rate 4 3 2 1 0-1 -2 11
A.12. Tables Table S1: Effects of robustness tests on the between-region difference-in-differences and the triple difference estimates Between-region difference-indifferences Triple-difference Non-incentivized conditions Main analysis 0.3 [-0.4, 1.1] Including volume of cases 0.4 [-0.4, 1.1] Including baseline performance 0.6 [0.0, 1.1] Excluding southern regions 0.5 [-0.3, 1.3] Acute myocardial infarction Main analysis -0.3 [-1.0, 0.4] -0.6 [-1.7, 0.4] Including volume of cases -0.4 [-1.1, 0.4] -0.7 [-1.7, 0.3] Including baseline performance -0.2 [-0.7, 0.4] -0.8 [-1.5, 0.0] Excluding southern regions -0.2 [-1.0, 0.5] -0.7 [-1.8, 0.4] Heart failure Main analysis -0.3 [-1.2, 0.6] -0.6 [-1.8, 0.6] Including volume of cases -0.3 [-1.2, 0.6] -0.6 [-1.8, 0.5] Including baseline performance 0.2 [-0.5, 0.9] -0.4 [-1.3, 0.5] Excluding southern regions 0.0 [-1.0, 0.9] -0.5 [-1.8, 0.7] Pneumonia Main analysis -1.6 [-2.4, -0.8] -1.9 [-3.0, -0.9] Including volume of cases -1.6 [-2.4, -0.8] -1.9 [-3.0, -0.9] Including baseline performance -1.2 [-1.8, -0.6] -1.7 [-2.5, -0.9] Excluding southern regions -1.8 [-2.6, -1.0] -2.3 [-3.4, -1.1] 12
Table S2: Estimated additional changes in risk-adjusted mortality by type of hospital in the North West of England Acute myocardial Heart failure Pneumonia Nonincentivized Hospital characteristic infarction conditions Scale and Scope Teaching/Specialist -1.1 (-3.6, 1.3) -1.6 (-4.0, 0.9) -0.6 (-2.6, 1.3) -1.2 (-2.2, -0.2) Large General -0.7 (-2.7, 1.3) 0.2 (-1.2, 1.6) -2.0 (-3.4, -0.6) 0.4 (-0.8, 1.6) Medium General 0.8 (-0.6, 2.3) -0.5 (-1.7, 0.8) -0.8 (-1.7, 0.0) 0.5 (-0.8, 1.9) Small General -1.4 (-3.5, 0.7) 0.8 (-1.0, 2.5) -4.2 (-6.6, -1.8) 1.7 (-0.3, 3.7) Foundation Trust status * Non-Foundation Trust -0.7 (-2.0, 0.5) -0.1 (-1.0, 0.8) -1.7 (-2.7, -0.6) 0.4 (-0.6, 1.5) Foundation Trust 0.4 (-1.4, 2.2) -0.7 (-2.5, 1.1) -1.5 (-2.7, -0.3) 0.4 (-0.6, 1.4) Quality Rating Excellent -0.3 (-1.9, 1.2) -1.0 (-2.6, 0.6) -1.8 (-2.7, -0.8) 1.4 (-0.1, 2.9) Good -0.1 (-1.6, 1.4) 0.0 (-1.0, 1.1) -2.0 (-3.3, -0.7) 0.2 (-0.8, 1.1) Fair/Weak -1.5 (-4.0, 0.9) 0.5 (-1.4, 2.5) 0.2 (-1.2, 1.7) -0.3 (-1.4, 0.7) Financial Rating Excellent -0.1 (-1.8, 1.5) -1.0 (-2.6, 0.6) -1.5 (-2.8, -0.2) 0.3 (-0.7, 1.3) Good -1.5 (-3.7, 0.7) 0.9 (-0.4, 2.2) -1.5 (-3.2, 0.2) 1.4 (-0.3, 3.1) Fair/Weak 0.5 (-0.5, 1.4) -0.1 (-1.1, 0.9) -1.6 (-2.6, -0.7) -0.2 (-1.8, 1.4) * As at 2007. Composite rating of performance in 2007 by the national regulator (the Healthcare Commission) against core standards, existing national targets and new national targets for quality. Composite rating of performance in 2007 by the national regulator (the Healthcare Commission) on financial standing, management and control. 13
Table S3: Mean achievements on the quality measures at the start and end of the period Clinical condition / quality measure Quarter 1 (October December 2008) Quarter 6 (January March 2010) Quarter 6 minus Quarter 1 Acute Myocardial Infarction Aspirin at arrival 96.0% 98.2% 2.2% Aspirin prescribed at discharge 97.8% 99.2% 1.4% ACEI or ARB for LVSD 96.7% 99.2% 2.4% Smoking cessation advice / counselling 80.6% 89.5% 8.9% Beta blocker prescribed at discharge 90.4% 96.3% 5.9% Fibrinolytic therapy within 30 minutes of arrival 81.0% 83.6% 2.6% Heart Failure Evaluation of LVS Function 86.2% 94.2% 8.1% ACEI or ARB for LVSD 89.4% 90.8% 1.4% Discharge instructions 22.4% 37.5% 15.2% Smoking cessation advice/counselling 41.7% 62.9% 21.2% Pneumonia Oxygenation assessment 96.0% 98.9% 2.9% Initial antibiotic selection in immunocompetent patients 81.9% 87.0% 5.0% Blood cultures performed prior to initial antibiotic selection 58.9% 73.7% 14.8% Initial antibiotic received within 6 hours of hospital arrival 67.1% 71.5% 4.4% Smoking cessation advice / counselling 36.5% 56.6% 20.1% 14