Appendix A: Difference-in-Difference Estimation Estimation Strategy We define a simple difference-in-differences (DD) estimator for the treatment effect of Hospital Compare (HC) from the perspective of states without state reporting systems (intentto-treat states): Δ HC = [P Post-HC,Treatment - P Pre HC,Treatment ] [P Post HC,Control - P Pre HC,Control ] Adjusting for characteristics of the admission in question we obtain a preliminary equation (1): P ihjt = a 0 + a 1 HCt + a 2 NR ijt + a 3 HCt*NR ijt + a 4 Z ijt + f h + f t + e ihjt where i indexes the patient admission, h indexes hospital, j indexes state, and t indexes year. P is price for the individual admission, HC is an indicator for the post period, NR is an indicator for admission in intent-to-treat states having no state report card systems prior to HC (these states are listed in Appendix Exhibit A.1), Z is a vector of medical characteristics of the case and insurance type (as described in Appendix Exhibit A.2), f h and f t are binary indicators for hospital and year fixed effects, and e ihjt is a random error term. The effect of HC on the intent to treat group of states is given as follows: 1
Δ HC = [(a 1 + a 2 + a 3 ) - a 2 ] - [a 1-0] = a 3. Thus a 3 provides the average treatment effect in those states. To accommodate various distributional assumptions, we estimate several specifications of equation (1). First, we estimate the linear model with hospital fixed effects using the generalized method of moments(gmm). This method provides heteroskedasticityrobust standard errors; fixed effects summarize hospital characteristics. Models replacing fixed effects with a subset of hospital covariates showed that prices were higher in for profit and rural hospitals, but not at teaching hospitals, holding patient risk and clinical variants constant (risk adjusters include both primary diagnosis and other major comorbidities). We run two sets of models, one for CABG prices and one for PCI. The GMM hospital fixed effects models are reported in Appendix Exhibit A.3. Second, while linearity in P ihjt is required in difference-indifference modeling, we also explore estimation of equation (1) with the dependent variable specified as the natural log (lnp ihjt ). To test for distributional assumptions, we compare gamma and normal distributions in generalized linear modeling (GLM) with a log link function, using maximum-likelihood. Based on the Akaike information criteria, the log-normal distribution produced the better fit, while gamma was the dominant 2
distribution in a modified Park test (see Yang 2005, Carey and Stefos 2011). However, the regression coefficients in the two distributions were virtually identical. Moreover, marginal effects were very similar to models with the identity link, the GLM equivalent of a levels regression, indicating that skewness was not an issue. GLM logged models produced results very similar to the GMM model, so they are not reported. Third, we also create a counterfactual to the model, to test for the validity of the various quasi-experiment designs above. The counterfactual is based on rerunning the models using noncardiac procedures that should be weakly susceptible to the information from Hospital Compare with respect to pricing. Following Ryan, Nallamothu, and Dimick (2012), who employ gastrointestinal diagnoses as the counterfactual for AMI when evaluating the mortality consequences of HC initiation, we employ surgeries for gastrointestinal cancers as the comparison to CABG and PCI. A primary example is surgery for colorectal cancer (colectomy). ICD-9 and CPT codes needed to define colectomy related admissions are found in a previous related study (Dor et al., 2012). We present the GMM linear estimates for colectomy in Appendix Exhibit A.4. As expected, the HC public reporting does not impact colectomy prices. For a description of the colectomy sample, see Dor et al 2013. 3
Finally, the price trends found for PCI and CABG in the narrative should not be confused with the more broadly known general hospital price indices which are composites of multiple hospital services. We checked the trends in prices against national trends in charges (prices prior to discounts) and found that they comport. See Appendix Exhibit A.5. Appendix A. References Carey K, Stefos T. 2011. Measuring the Cost of Hospital Adverse Patient Safety Events. Health Economics 20(12):1470-1430. Dor A, Deb P, Grossman M., et al 2013. "Impact of Mortality Based Performance Measures on Hospital Pricing: the Case of Colon Cancer Surgeries," NBER Working Papers 19447, National Bureau of Economic Research, Inc. Dor A, Koroukian F, Xu F, Stulberg J, Delaney C, Cooper G. 2012. Pricing of surgeries for colon cancer: Patient severity and market factors. Cancer 118(23):5741-748. Ryan AM, Nallamothu BK, Dimick JB 2012. Medicare s public reporting initiative on hospital quality had modest or no impact on mortality from three key conditions. Health Affairs 31(3):585-590. Yang, Y. 2005. "Can the strengths of AIC and BIC be shared?", Biometrika 92: 937 950. 4
Appendix Exhibit A.1 Hospital State Report Card History from 2005 State Report Card Year(s) to Which Report Cards Pertain New York CABG 2003-2005, 2004-2006, 2005-2007, and 2006-2008 PTCA (Angioplasty) 2003-2005, 2004-2006, 2005-2007, and 2006-2008 Pennsylvania CABG Surgery; Valve Surgery 2005, 2005-2006, 2006-2007, 2007-2008 and 2008-2009 New Jersey CABG 2004, 2006, 2007, and 2008 California CABG 2004-2005, 2005-2006, 2007, and 2007-2008 CABG 2005, 2006, 2007, 2008, and 2009 Massachusetts PTCA (Angioplasty) 2005, 2006, 2007, 2008, and 2009 Florida CABG 2005, 2006, 2007, 2008, 2009 PTCA (Angioplasty) 2005, 2006, 2007, 2008, 2009 Washington CABG 2010, voluntary/incomplete reporting Notes: Washington is not included in the report card states for purposes of this study. 5
Appendix Exhibit A.2: Patient Sample Descriptive Statistics CABG PCI $48,659 $24,316 Hospital Price (48,747) (42,251) Hospital Compare.284.303 Reporting (yes, no) (.451) (.459).851.819 No State Report (yes, no) (.356) (.385) Hospital Compare* No.221.221 State Report (.415) (.415) Hospital Cardiac.360.364 Herfindahl Index (.308) (.309).205.211 HMO Market Penetration (.085) (.092).155 -- One vessel bypassed (.302).303 -- Two vessels bypassed (.459).387 -- Three vessels bypassed (.487) Four or more vessels.156 -- bypassed (.362) 0.856 Stent (0) (.351) Age < 54 Age 54-59 Age 60-64 Female Union.189.202 HMO-insured (.392) (.401) AMI Stroke Congestive Heart Failure Arrhythmia Diabetes Catheterization Number of Procedures N 18,532 54,301 Source: MarketScan 2005-2010. Standard deviations in parentheses..273.347.380.229.295.883.118.006.167.206.116 13.375.340.335.324.246.288.944.017.005.101.111.168 11.127 (.382) (.476) (.485) (.420) (.456) (.322) (.323) (.080) (.373) (.404) (.321) (2.810) (.450) (.472) (.468) (.431) (.453) (.231) (.127) (.072) (.301) (.314) (.374) (3.907) 6
Appendix Exhibit A.3: Estimates of the Impact of Hospital Compare Reporting on Private Hospital Prices (GMM Estimation, Hospital Fixed Effects) Hospital Compare Reporting No State Report Hospital Compare* No State Report Hospital Cardiac Herfindahl Index HMO Market Penetration Two vessels bypassed Three vessels bypassed Four or more vessels bypassed CABG PCI 13,258*** 5,250*** (2621) (901) 1,026 555 (2935) (877) -6,474*** -2,739*** (2482) (985) 1,311 198 (3578) (884) -246-367 (7179) (2105) 1,003 (973) -- -267 (1034) -- 3,942 *** (1261) Stent -- -535 (506) Age 54-59 253-144 (911 ) (216) Age 60-64 -914 319 (933) (597) Female 670-893** (954) (388) Union -2,215** -192 (974) (1050) HMO-insured -4,452*** -2,111*** (1399) (528) AMI -15,079*** -6,273*** (3643) (832) Stroke 5,202*** 7,076*** (1289) (1171) Congestive Heart Failure 19,096** 2,241 (9123) (2656) Arrhythmias 8,603*** 3,120*** (1395) (439) Diabetes 210 1,293*** (981) (320) Catheterization 5,540*** -1,613*** (1416) (438) Number of procedures 2,762*** 833*** (240) (40) Hospital Fixed Effect Yes Yes Notes: GMM, with heteroskedasticity-robust standard errors in parentheses. Time fixed effects not shown. * p <.1, ** p <.05, *** p <.01. -- 7
Appendix Exhibit A.4: Counterfactual Estimates of the Impact of Hospital Compare Reporting on Private Hospital Prices for Colectomy (2005-2010) Post-Hospital Compare No State Report Hospital Compare* No State Report Notes: N=5,858 GMM (price) $5,562*** (1,913) -$1,872 (2,390) -$926 (1,736) Hospital fixed effects GMM using the covariates of Appendix Exhibit A.2, with heteroskedasticity-robust standard errors in parentheses. * p <.1, ** p <.05, *** p <.01. Source: HCUP State Inpatient Databases; unadjusted charges, medians 8
Appendix B: Matched States Analysis higher prices in no-report states fall back to general trend after HC initiation Appendix Exhibit B.1: Estimated Impact of Hospital Compare on CABG Prices, 3 to 1 match 9
Exhibit B.2: Estimated Impact of Hospital Compare on PCI Prices, 3 to 1 match States Matched: Report-card states: CA, FL, PA, MA, NJ, NY 3 to 1 Match: CT, IL, SD, ND, VA, HI, MI, DC, VT, OH, NC, WI, ME, RI, TN, WV, MN, IA 10
Matching Algorithm Exhibit B.3 Propensity Score model (Logistic regression) Variable Coef. Std. Err. Log(income) - 6.577 6.343 Unemployment rate 0.247 0.181 Population per sq mile 0.001 0.000 Beds per capita*** 1331.483 412.416 Percent elderly*** 108.439 39.229 Docs per capita 0.001 0.010 Poverty rate* - 77.657 44.096 Percent black 4.512 7.304 Percent low education 0.721 4.171 HMO rate 6.340 4.881 Percent voted for Bush - 0.184 0.112 State politics 0.029 0.022 Log(revenues)*** 5.014 1.407 Constant - 54.679 63.493 Matching Method = nearest neighbor, Metric = pscore * p <.1, ** p <.05, *** p <.01. Using the psmatch2 routine in Stata, for years 2001-2010 we match states for propensity to be in the CABG control using log(state income), unemployment, pop per square mile, hospital beds per pop, percent age >=65, docs per capita, poverty rate, percent black, low education rate, percent female, HMO penetration rate, state institutional politics (0 to 1, 1 liberal), percent voted for Bush in 2004, log(state revenue) (Data Sources: Area Resource File; and Berry WD, Evan J. Ringquist EJ, Fording RC; Russell L. Hanson. Replication data for: Measuring Citizen and Government Ideology in the American States, 1960-93. 2007). 11
Exhibit B.4 Sample Means in Matched and Unmatched States: Mean %reduction in Variable Sample Treated Control %bias bias t p> t Control State Unmatched 1 0... Matched 1 0.... Log(income) Unmatched 10.967 10.809 107.3 6.040 0.000 Matched 10.967 10.943 16.4 84.7 0.590 0.560 Unemployment Unmatched 6.679 6.254 20.0 1.260 0.208 Matched 6.679 6.539 6.6 67.0 0.290 0.773 Pop per sq mile Unmatched 2567.300 642.930 119.8 12.930 0.000 Matched 2567.300 2156.800 25.6 78.7 0.730 0.470 Beds per capita Unmatched 0.003 0.003-4.8-0.230 0.816 Matched 0.003 0.004-19.3-304.2-0.690 0.492 Percent elderly Unmatched 0.070 0.066 52.7 3.180 0.002 Matched 0.070 0.077-87.2-65.6-2.520 0.014 Docs per capita Unmatched 291.060 222.310 111.9 7.410 0.000 Matched 291.060 286.110 8.1 92.8 0.190 0.851 Poverty rate Unmatched 0.108 0.124-66.7-3.510 0.001 Matched 0.108 0.107 7.8 88.4 0.370 0.712 Percent black Unmatched 0.100 0.103-3.5-0.170 0.867 Matched 0.100 0.116-23.5-575.8-1.270 0.208 Percent low education Unmatched 0.083 0.060 22.7 1.550 0.121 Matched 0.083 0.063 19.7 13.4 0.790 0.430 HMO rate Unmatched 0.254 0.190 99.2 7.190 0.000 Matched 0.254 0.221 50.7 48.9 2.040 0.045 Pct voted for Bush Unmatched 50.863 57.703-91.3-5.300 0.000 Matched 50.863 52.362-20.0 78.1-0.880 0.384 State politics Unmatched 54.278 49.535 22.0 1.260 0.207 Matched 54.278 53.607 3.1 85.8 0.110 0.914 Log(revenue) Unmatched 25.139 24.117 164.9 8.840 0.000 Matched 25.139 25.045 15.2 90.8 0.790 0.434 The median absolute value of the bias in the covariates between control states and treatment states was reduced from 66.7 to 19.3 due to the matching (See Rosenbaum and Rubin, 1985). 12
Difference-in-Difference We then reran the difference in difference analyses of Appendix A, Exhibit A.3, to estimate the impact of Hospital Compare on CABG and PCI prices. The following Exhibit B.5 highlights the main coefficients. The results are also graphed in the above Exhibits B.1 and B.2. Exhibit B.5: Hospital Fixed Effects Estimates of the Impact of Hospital Compare Reporting on Private Hospital Prices using the 3 to 1 Matched Samples Hospital Compare Reporting No State Report CABG 8,711*** (2,164) 4,033 (3,142) PCI 4,301*** (499) 956 (684) Hospital Compare *No State Report -3,672* (2,159) -1,584*** (504) N 9,915 30,624 Notes: Heteroskedasticity-robust standard errors in parentheses. Not shown, all covariates are included as in the full regressions of Appendix A Exhibit A.3. * p <.1, ** p <.05, *** p <.01. 13
Appendix B References: Leuven, E. and Sianesi, B. (2003), PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing. http://ideas.repec.org/c/boc/bocode/s432001.html Rosenbaum, P.R. and Rubin D.B. (1985), Constructing a control group using multivariate matched sampling methods that incorporate the propensity score, The American Statistician, Vol. 39 No. 1, pp. 33-38. 14